I'm trying to get Mathematica to run on OpenBSD via Linux emulation, but 
I am getting the following error:

 error while loading shared libraries: libvml.so: cannot enable executable 
stack as shared object requires: Premission denied.

Does anyone have any suggestions about how I might solve this problem?

Note that libvml.so is the Intel Vector Math Library at
 http://www.intel.com/cd/software/products/asmo-na/eng/307757.htm

When I run ktrace on Mathematica, I'm getting the following:

 20188 ktrace   RET   ktrace 0
 20188 ktrace   CALL  execve(0xcfbc4337,0xcfbc41f0,0xcfbc41f8)
 20188 ktrace   NAMI  "/usr/local/emul/fedora/usr/local/Wolfram/Mathematica/6.0/SystemFiles/Kernel/Binaries/Linux/MathKernel"
 20188 ktrace   NAMI  "/emul/svr4/lib/ld-linux.so.2"
 20188 ktrace   NAMI  "/emul/linux/lib/ld-linux.so.2"
 20188 ktrace   NAMI  "/emul/linux"
 20188 MathKernel NAMI  "/emul/linux/lib/ld-linux.so.2"
 20188 MathKernel EMUL  "linux"
 20188 MathKernel RET   oldolduname 0
 20188 MathKernel CALL  getuid()
 20188 MathKernel RET   getuid 1000/0x3e8
 20188 MathKernel CALL  geteuid()
 20188 MathKernel RET   geteuid 1000/0x3e8
 20188 MathKernel CALL  getgid()
 20188 MathKernel RET   getgid 1000/0x3e8
 20188 MathKernel CALL  getegid()
 20188 MathKernel RET   getegid 1000/0x3e8
 20188 MathKernel CALL  brk(0)
 20188 MathKernel RET   brk 154390528/0x933d000
 20188 MathKernel CALL  uname(0xcfbc1a3e)
 20188 MathKernel RET   uname 0
 20188 MathKernel CALL  access(0x56e82564,0x4)
 20188 MathKernel NAMI  "/emul/linux/etc/ld.so.preload"
 20188 MathKernel NAMI  "/etc/ld.so.preload"
 20188 MathKernel RET   access -1 errno 2 No such file or directory
 20188 MathKernel CALL  open(0x56e84603,0,0x1)
 20188 MathKernel NAMI  "/emul/linux/etc/ld.so.cache"
 20188 MathKernel NAMI  "/emul/linux"
 20188 MathKernel NAMI  "/emul/linux/etc/ld.so.cache"
 20188 MathKernel RET   open 3
 20188 MathKernel CALL  fstat64(0x3,0xcfbc1320)
 20188 MathKernel RET   fstat64 0
 20188 MathKernel CALL  mmap(0xcfbc1304)
 20188 MathKernel RET   mmap 1422512128/0x54c9d000
 20188 MathKernel CALL  close(0x3)
 20188 MathKernel RET   close 0
 20188 MathKernel CALL  open(0xcfbc1330,0,0)
 20188 MathKernel NAMI  "/emul/linux/lib/tls/libML32i3.so"
 20188 MathKernel NAMI  "/emul/linux"
 20188 MathKernel NAMI  "/emul/linux/lib/tls/lib...
[ message continues ]

OK, I have some question about the ktrace.  The first notable call is the 

In OpenBSD, the only legal values for the third argument of mprotect are 
the bitwise sums of

#define	PROT_NONE	0x00	/* no permissions */
#define	PROT_READ	0x01	/* pages can be read */
#define	PROT_WRITE	0x02	/* pages can be written */
#define	PROT_EXEC	0x04	/* pages can be executed */

But in Linux, bitwise sums of

#define PROT_SEM	0x8		/* page may be used for atomic ops */
#define PROT_GROWSDOWN	0x01000000	/* mprotect flag: extend change to start of growsdown vma */
#define PROT_GROWSUP	0x02000000	/* mprotect flag: extend change to end of growsup vma */

are also allowed, in addition to those allowed by OpenBSD.  So, at this 
point in the ktrace, a Linux-specific call to mprotect is being made.  
Namely, PROT_READ + PROT_WRITE + PROT_EXEC + PROT_GROWSDOWN.

Is this the source of the error?  If so, could OpenBSD's compat_linux be 
patched?

But then it appears that OpenBSD recovers from the error and strips off 

But this appears to be unsuccessful, too.  Why?

Tens of thousands of unsuccessful calls to mprotect are made at various 

What is the significance of this error?  Is it caused by OpenBSD's 
ProPolice stack protection?

As it turns out, this bug in compat_linux was encountered in NetBSD in 
2005 and then fixed.  The NetBSD patches are

 http://cvsweb.netbsd.org/bsdweb.cgi/src/sys/compat/linux/common/linux_mmap.h.diff?r1=1...
 http://cvsweb.netbsd.org/bsdweb.cgi/src/sys/compat/linux/common/linux_misc.c.diff?r1=1...

the complete NetBSD bug report is

 http://www.netbsd.org/cgi-bin/query-pr-single.pl?number=30008

and a discussion of the bug (and what the PROT_GROWSDOWN flag does) can be 
found at

 http://mail-index.netbsd.org/current-users/2005/10/25/0026.html

Could I convince one of the developers to port these patches to OpenBSD?  
Should I just file a bug report?

I have successfully ported the relevant material from the NetBSD patches

 http://cvsweb.netbsd.org/bsdweb.cgi/src/sys/compat/linux/common/linux_misc.c.diff?r1=1... 
 http://cvsweb.netbsd.org/bsdweb.cgi/src/sys/compat/linux/common/linux_mmap.h.diff?r1=1...
 http://cvsweb.netbsd.org/bsdweb.cgi/src/sys/compat/linux/arch/i386/syscalls.master.dif...

to OpenBSD.  The OpenBSD patch is given at the end of this message.  Be 
sure to run the make file at

 /usr/src/sys/compat/linux/

after applying the patch.  The make file will update some important files.

This patch allows OpenBSD to use the PROT_GROWSUP and PROT_GROWSDOWN flags 
to mprotect under Linux emulation.  I have tested this new functionality 
with the Linux version of the Mathematica 6.0 Kernel, which now runs 
successfully.  Presumably, OpenBSD will also be able to run other Linux 
applications that use these flags, such as Adobe Acrobat Reader (for which 
the NetBSD patches were originally intended).

What steps need to be taken to ensure that this fix is merged into the 
official OpenBSD source code?


*** /usr/src/sys/compat/linux/linux_misc.c.orig	Wed Feb 14 18:07:51 2007
--- /usr/src/sys/compat/linux/linux_misc.c	Mon Jun 25 00:13:42 2007
***************
*** 718,723 ****
--- 718,787 ----
  
  }
  
+ int
+ linux_sys_mprotect(p, v, retval)
+ 	struct proc *p;
+ 	void *v;
+ 	register_t *retval;
+ {
+ 	struct linux_sys_mprotect_args /* {
+ 		syscallarg(const void *) start;
+ 		syscallarg(unsigned long) len;
+ 		syscallarg(int) prot;
+ 	} */ *uap = v;
+ 	struct vm_map_entry *entry;
+ 	struct vm_map *map;
+ 	vaddr_t end, start, len, stacklim;
+ 	int prot, grows;
+ 
+ 	start = (vaddr_t)SCARG(uap, start);
+ 	len = round_page(SCARG(uap, len));
+ 	prot = SCARG(uap, prot);
+ 	grows = prot & (LINUX_PROT_GROWSDOWN | LINUX_PROT_GROWSUP);
+ 	prot &= ~grows;
+ 	end = start + len;
+ 
+ 	if (start & PAGE_MASK)
+ 		return EINVAL;
+ 	if (end < start)
+ 		re...
[ message continues ]

As some of you may recall, there are Linux binaries (such as Mathematica) 
which will not run under OpenBSD's Linux emulation because the 
Linux mprotect system-call uses flags that are not supported by OpenBSD.  
To remedy this problem, I had posted a patch for OpenBSD 4.1

 http://marc.info/?l=openbsd-misc&m=118275040601937

I have included a new version of that patch, updated for OpenBSD 4.2, at 
the end of this message.

After applying the patch to the OpenBSD 4.2 source-code, be sure to run 
make in the directory

 /usr/src/sys/compat/linux/

If you forget to run that special makefile, then the patch will fail.

As before, I should mention that I am not the author of the majority of 
the code in this patch.  Instead, it has been ported from the 
corresponding NetBSD patches

 http://cvsweb.netbsd.org/bsdweb.cgi/src/sys/compat/linux/common/linux_misc.c.diff?r1=1... 
 http://cvsweb.netbsd.org/bsdweb.cgi/src/sys/compat/linux/common/linux_mmap.h.diff?r1=1...
 http://cvsweb.netbsd.org/bsdweb.cgi/src/sys/compat/linux/arch/i386/syscalls.master.dif...

The OpenBSD 4.2 patch follows:


--- /usr/src/sys/compat/linux/linux_misc.c.orig	Wed Feb 14 18:07:51 2007
+++ /usr/src/sys/compat/linux/linux_misc.c	Sun Nov 11 02:40:01 2007
@@ -718,6 +718,70 @@
 
 }
 
+int
+linux_sys_mprotect(p, v, retval)
+	struct proc *p;
+	void *v;
+	register_t *retval;
+{
+	struct linux_sys_mprotect_args /* {
+		syscallarg(const void *) start;
+		syscallarg(unsigned long) len;
+		syscallarg(int) prot;
+	} */ *uap = v;
+	struct vm_map_entry *entry;
+	struct vm_map *map;
+	vaddr_t end, start, len, stacklim;
+	int prot, grows;
+
+	start = (vaddr_t)SCARG(uap, start);
+	len = round_page(SCARG(uap, len));
+	prot = SCARG(uap, prot);
+	grows = prot & (LINUX_PROT_GROWSDOWN | LINUX_PROT_GROWSUP);
+	prot &= ~grows;
+	end = start + len;
+
+	if (start & PAGE_MASK)
+		return EINVAL;
+	if (end < start)
+		return EINVAL;
+	if (end == start)...
[ message continues ]

There have been requests that I resubmit the patch as a unified 
diff, rather than a context diff.  The patch is given below.  Remember to 
run the make file at /usr/src/sys/compat/linux/ after applying the patch.


--- /usr/src/sys/compat/linux/linux_misc.c.orig	Wed Feb 14 18:07:51 2007
+++ /usr/src/sys/compat/linux/linux_misc.c	Mon Jun 25 00:13:42 2007
@@ -718,6 +718,70 @@
 
 }
 
+int
+linux_sys_mprotect(p, v, retval)
+	struct proc *p;
+	void *v;
+	register_t *retval;
+{
+	struct linux_sys_mprotect_args /* {
+		syscallarg(const void *) start;
+		syscallarg(unsigned long) len;
+		syscallarg(int) prot;
+	} */ *uap = v;
+	struct vm_map_entry *entry;
+	struct vm_map *map;
+	vaddr_t end, start, len, stacklim;
+	int prot, grows;
+
+	start = (vaddr_t)SCARG(uap, start);
+	len = round_page(SCARG(uap, len));
+	prot = SCARG(uap, prot);
+	grows = prot & (LINUX_PROT_GROWSDOWN | LINUX_PROT_GROWSUP);
+	prot &= ~grows;
+	end = start + len;
+
+	if (start & PAGE_MASK)
+		return EINVAL;
+	if (end < start)
+		return EINVAL;
+	if (end == start)
+		return 0;
+
+	if (prot & ~(PROT_READ | PROT_WRITE | PROT_EXEC))
+		return EINVAL;
+	if (grows == (LINUX_PROT_GROWSDOWN | LINUX_PROT_GROWSUP))
+		return EINVAL;
+
+	map = &p->p_vmspace->vm_map;
+	vm_map_lock(map);
+	if (!uvm_map_lookup_entry(map, start, &entry) || entry->start > start) {
+		vm_map_unlock(map);
+		return ENOMEM;
+	}
+
+	/*
+	 * Approximate the behaviour of PROT_GROWS{DOWN,UP}.
+	 */
+
+	stacklim = (vaddr_t)p->p_limit->pl_rlimit[RLIMIT_STACK].rlim_cur;
+	if (grows & LINUX_PROT_GROWSDOWN) {
+		if (USRSTACK - stacklim <= start && start < USRSTACK) {
+			start = USRSTACK - stacklim;
+		} else {
+			start = entry->start;
+		}
+	} else if (grows & LINUX_PROT_GROWSUP) {
+		if (USRSTACK <= end && end < USRSTACK + stacklim) {
+			end = USRSTACK + stacklim;
+		} else {
+			end = entry->end;
+		}
+	}
+	vm_map_unlock(map);
+	re...
[ message continues ]

thanks, good work.

It appears that a similar bug was encountered by Debian in 2005.  See

 http://lists.debian.org/debian-glibc/2005/08/msg00289.html
 http://lists.debian.org/debian-glibc/2005/08/msg00311.html
 http://lists.debian.org/debian-glibc/2005/08/msg00483.html

This would suggest that it might be a problem with OpenBSD's compat_linux.

right.  i'm not even sure what the growsdown flag does.  linux compat
is really only maintained to the extent that people find issues and
fix them.  you could try having the mprotect mask out the bad bits and
cross your fingers.

> right.  i'm not even sure what the growsdown flag does.

Indeed, Linux doesn't have the same standard of documentation as OpenBSD.  
The PROT_GROWSDOWN flag for mprotect is not even mentioned in mprotect's 
man page on linux

 http://linux.about.com/library/cmd/blcmdl2_mprotect.htm

what does ktrace say about the problem?

The ktrace output is as follows:

 12262 ktrace   RET   ktrace 0
 12262 ktrace   CALL  execve(0xcfbd0bd3,0xcfbd0a8c,0xcfbd0a94)
 12262 ktrace   NAMI  "/usr/local/emul/fedora/usr/local/Wolfram/Mathematica/6.0/SystemFiles/Kernel/Binaries/Linux/MathKernel"
 12262 ktrace   NAMI  "/emul/svr4/lib/ld-linux.so.2"
 12262 ktrace   NAMI  "/emul/linux/lib/ld-linux.so.2"
 12262 ktrace   NAMI  "/emul/linux"
 12262 MathKernel NAMI  "/emul/linux/lib/ld-linux.so.2"
 12262 MathKernel EMUL  "linux"
 12262 MathKernel RET   oldolduname 0
 12262 MathKernel CALL  getuid()
 12262 MathKernel RET   getuid 1000/0x3e8
 12262 MathKernel CALL  geteuid()
 12262 MathKernel RET   geteuid 1000/0x3e8
 12262 MathKernel CALL  getgid()
 12262 MathKernel RET   getgid 1000/0x3e8
 12262 MathKernel CALL  getegid()
 12262 MathKernel RET   getegid 1000/0x3e8
 12262 MathKernel CALL  brk(0)
 12262 MathKernel RET   brk 154390528/0x933d000
 12262 MathKernel CALL  uname(0xcfbf2b12)
 12262 MathKernel RET   uname 0
 12262 MathKernel CALL  access(0x53994564,0x4)
 12262 MathKernel NAMI  "/emul/linux/etc/ld.so.preload"
 12262 MathKernel NAMI  "/etc/ld.so.preload"
 12262 MathKernel RET   access -1 errno 2 No such file or directory
 12262 MathKernel CALL  open(0x53996603,0,0x1)
 12262 MathKernel NAMI  "/emul/linux/etc/ld.so.cache"
 12262 MathKernel NAMI  "/emul/linux"
 12262 MathKernel NAMI  "/emul/linux/etc/ld.so.cache"
 12262 MathKernel RET   open 3
 12262 MathKernel CALL  fstat64(0x3,0xcfbf23f4)
 12262 MathKernel RET   fstat64 0
 12262 MathKernel CALL  mmap(0xcfbf23d8)
 12262 MathKernel RET   mmap 1284878336/0x4c95b000
 12262 MathKernel CALL  close(0x3)
 12262 MathKernel RET   close 0
 12262 MathKernel CALL  open(0xcfbf2410,0,0)
 12262 MathKernel NAMI  "/emul/linux/lib/tls/libML32i3.so"
 12262 MathKernel NAMI  "/lib/tls/libML32i3.so"
 12262 MathKernel RET   open -1 errno 2 No such file or directory
 12262 MathKernel CALL  open(0xcfbf2410,0,0)
 12262 MathKernel NAMI  "/emul/linux/lib/libML32i3.so"
 12262 MathKernel NAMI ...
[ message continues ]

another problem?