Reading through the archives I have found several people say that

encrypting via an svnd device isn't much slower than writing directly to

a raw unencrypted disk. While I found this to be true for svnd devices

backed by files, svnd devices backed by whole disks and disk partitions

are extremely slow. I have tried tuning many parameters, namely the

fragment and block size and the cylinders per group in the disklabel

associated with the svnd, but nothing has improved the performance.
I am running 4.1 on a single i386 800MHz P3. The encrypting of an

underlying device (file, partition, disk) works perfectly otherwise. I

also double checked my procedure at

https://www.mainframe.cx/~ckuethe/encrypted_disks.html.
Not knowing what to tune to speed things up, I started by using all

combinations of the following in the svnd disklabel (assuming they get

passed to newfs):
fragment size: 2K, 4K

block size: 16K, 32K

cyl. per group: 16, 1568, 1936, 4K, 8K, 16K (sometimes after newfs'ing,

cpg was reset to some other value? That's where the 1568 and 1936 come from)
I have also tried mounting the svnd device using the async and noatime

flags, but that doesn't really matter.
Using vnconfig, I also tried associating the svnd device with the raw

direct access device (i.e. /dev/rwd1[ac]), but then fdisk'ing on the

svnd device complains. I tried this because I thought there may be a

double buffering issue.
I also tried encryption with and without a salt file, but that didn't

make any noticeable difference.
Here are some write performance numbers using dd and cp:

* for dd I used block sizes of 512, 1K, 2K, 4K, 8K, 16K

* for cp I used the command `cd /<enc-dev>; time cp -R /bin /sbin .`

* all dd commands made files > 40MB, which is more than 4 times the

disk's cache
Direct disk (no svnd)

  dd: 49MB/s - 100MB/s

  cp: 2.43s real
svnd backed by disk (wd1c)

  dd: 248K - 500K

  cp: 1m21.44s real
svnd backed by partition (wd1a)

  dd: 1.8MB/s - 2.8MB/s

  cp: ...
[ message continues ]

I experimented with a few more things, but nothing helped. Someone said

run a bonnie++ benchmark to verify the performance. bonnie++ basically

told me what dd did, that svnd backed by a file is slow and svnd backed

by a disk or partition is floppy disk slow.
Nonetheless, the bonnie++ results may provide some insight to the

problem for an experienced guru. What I found interesting is that the

CPU usage is really low for writes and rewrites when svnd is backed by

the whole disk. This is also the slowest configuration.
It seems like there may be some alignment issues between the underlying

storage device and the svnd device. That's why I was trying all

combinations of block and fragment sizes, cylinders per group,

geometries (CHS) for fdisk, etc. Hopefully someone can shed some light

on this problem.
bonnie++ benchmark

------------------
wd0d (slow old disk)

Version  1.03       ------Sequential Output------ --Sequential Input-

--Random-

                     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block--

--Seeks--

Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP

/sec %CP

foo.mokaz.com   40M  4956  13  4934   4  2950   2  8622  30  8754   3

183.1   0
wd1d (fast new disk)

Version  1.03       ------Sequential Output------ --Sequential Input-

--Random-

                     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block--

--Seeks--

Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP

/sec %CP

foo.mokaz.com   40M 45424  97 42832  38 10362   7 26344  91 47501  17

366.7   1
svnd0a (ass. w/ wd1c; fdisk: used disk/bios geometry)

Version  1.03       ------Sequential Output------ --Sequential Input-

--Random-

                     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block--

--Seeks--

Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP

/sec %CP

foo.mokaz.com   40M   230   3   235   2  3609  63  6786  67  8615  57

131.3  13
svnd0a (ass. w/ wd1c; fdisk: used OpenBSD MBR partition geomet...
[ message continues ]

instead of pondering problems with using the whole disk, you could

just use svnd with a file.

Well, I think he just found the itch.  Now the question is whether

he'll scratch it, or will someone else find it interesting enough to

scratch.
--

"This officer's men seem to follow him merely out of idle curiosity."

-- Sandhurst officer cadet evaluation.

http://www.youtube.com/watch?v=tGvHNNOLnCk

"Securing an environment of Windows platforms from abuse - external or

internal - is akin to trying to install sprinklers in a fireworks

factory where smoking on the job is permitted."  -- Gene Spafford

Yeah but doesn't this hint at some horrible inefficiency in the stack somewhere?
-Nick

it hints at using tools the wrong way leading to poor results.

Generally speaking this would mean that the CPU is waiting on I/O

completion instead of doing actual work. Maybe I should run an OpenBSD

in a WMare and see what's wrong... (that is if I weren't so lazy ^^)

This is really really really weird. You'd think that files, having the

filesystem to go through before getting to the disk, would necessarily

be slower. There must be some kind of weirdness with the thing.
-Nick

I concur.
My experience with full volume encryption is that impact on

performance should be below 5%, depending on the implementation, the

algorithms used, etc.
Is there a way to see where the CPU spends most of this time? This

sounds like there are unneccessary requests between the different

actors.
-- 
Die Gestalt

Instead of e.g. /dev/sd0a try /dev/rsd0a. I didn't try with svnd, but

when copying partitions with dd I use this.
--knitti

I tried that, but like I said fdisk complained when the svnd device is

associated with the raw direct access disk device. For example
# vnconfig -k svnd0 /dev/rwd1c
# fdisk -c 19457 -h 255 -s 63 -i svnd0  # disk CHS

fdisk: error initializing MBR: bad address
# fdisk -c 19456 -h 254 -s 63 -i svnd0  # OpenBSD partition CHS

fdisk: error initializing MBR: bad address
# fdisk -i svnd0

Warning CHS values out of bounds only saving LBA values

fdisk: error initializing MBR: bad address

well, the 'c' slice is a bit 'special', perhaps try an 'a' slice filling the

whole disk but the first track? After all, I think its weird not to have

an MBR etc. on the real disk. (Which doesn't mean that I couldn't

imagine that).
--knitti

I understand that the "c" partition is special. But when the entire disk

(i.e. wd0c) is just acting as a storage backend and the svnd device is

the front end, I don't think it matters. It's not like I'm going to be

running newfs or fsck on the actual disk. Anyway, I have tried on the

"a" partition too, but I get the same results.
Like I said, everything works fine if I use the buffered device,

wd0[ac], not rwd0[ac], but it is so slow. dd'ing to the unbuffered

device is much faster than dd'ing to the buffered. That is why I thought

I would try to associate the svnd device with the unbuffered disk device.
Thanks for you input.