OK, I have some question about the ktrace.  The first notable call is the 
following:

quoted text
>  20188 MathKernel CALL  mprotect(0xcfbc1000,0x1000,0x1000007)
>  20188 MathKernel RET   mprotect -1 errno 22 Invalid argument

In OpenBSD, the only legal values for the third argument of mprotect are 
the bitwise sums of

#define	PROT_NONE	0x00	/* no permissions */
#define	PROT_READ	0x01	/* pages can be read */
#define	PROT_WRITE	0x02	/* pages can be written */
#define	PROT_EXEC	0x04	/* pages can be executed */

But in Linux, bitwise sums of

#define PROT_SEM	0x8		/* page may be used for atomic ops */
#define PROT_GROWSDOWN	0x01000000	/* mprotect flag: extend change to start of growsdown vma */
#define PROT_GROWSUP	0x02000000	/* mprotect flag: extend change to end of growsup vma */

are also allowed, in addition to those allowed by OpenBSD.  So, at this 
point in the ktrace, a Linux-specific call to mprotect is being made.  
Namely, PROT_READ + PROT_WRITE + PROT_EXEC + PROT_GROWSDOWN.

Is this the source of the error?  If so, could OpenBSD's compat_linux be 
patched?

But then it appears that OpenBSD recovers from the error and strips off 
the offending bit that corresponds to PROT_GROWSDOWN:

quoted text
>  20188 MathKernel CALL  mprotect(0xcfbba000,0x8000,0x7)
>  20188 MathKernel RET   mprotect 0

But this appears to be unsuccessful, too.  Why?

Tens of thousands of unsuccessful calls to mprotect are made at various 
memory addresses, until finally an error is returned:

quoted text
>  20188 MathKernel CALL  mprotect(0x4d092000,0x8000,0x7)
>  20188 MathKernel RET   mprotect 0
>  20188 MathKernel CALL  mprotect(0x4d08a000,0x8000,0x7)
>  20188 MathKernel RET   mprotect -1 errno 13 Permission denied

What is the significance of this error?  Is it caused by OpenBSD's 
ProPolice stack protection?