Slow Performance on Encrypted svnd

Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]
To: OpenBSD Misc <misc@...>
Date: Wednesday, November 14, 2007 - 6:02 am

Reading through the archives I have found several people say that
encrypting via an svnd device isn't much slower than writing directly to
a raw unencrypted disk. While I found this to be true for svnd devices
backed by files, svnd devices backed by whole disks and disk partitions
are extremely slow. I have tried tuning many parameters, namely the
fragment and block size and the cylinders per group in the disklabel
associated with the svnd, but nothing has improved the performance.

I am running 4.1 on a single i386 800MHz P3. The encrypting of an
underlying device (file, partition, disk) works perfectly otherwise. I
also double checked my procedure at
https://www.mainframe.cx/~ckuethe/encrypted_disks.html.

Not knowing what to tune to speed things up, I started by using all
combinations of the following in the svnd disklabel (assuming they get
passed to newfs):

fragment size: 2K, 4K
block size: 16K, 32K
cyl. per group: 16, 1568, 1936, 4K, 8K, 16K (sometimes after newfs'ing,
cpg was reset to some other value? That's where the 1568 and 1936 come from)

I have also tried mounting the svnd device using the async and noatime
flags, but that doesn't really matter.

Using vnconfig, I also tried associating the svnd device with the raw
direct access device (i.e. /dev/rwd1[ac]), but then fdisk'ing on the
svnd device complains. I tried this because I thought there may be a
double buffering issue.

I also tried encryption with and without a salt file, but that didn't
make any noticeable difference.

Here are some write performance numbers using dd and cp:
* for dd I used block sizes of 512, 1K, 2K, 4K, 8K, 16K
* for cp I used the command `cd /; time cp -R /bin /sbin .`
* all dd commands made files > 40MB, which is more than 4 times the
disk's cache

Direct disk (no svnd)
dd: 49MB/s - 100MB/s
cp: 2.43s real

svnd backed by disk (wd1c)
dd: 248K - 500K
cp: 1m21.44s real

svnd backed by partition (wd1a)
dd: 1.8MB/s - 2.8MB/s
cp: 11.53s real

svnd backed by file
dd: 8.6MB/s - 9.7MB/s
cp: 2.66s real

The system was dedicated to these tests and the CPU was about 80% idle
during the running of the dd and cp commands.

What I really want is to encrypt the whole disk or a single partition
covering the whole disk. If I could get the write performance of the
disk/partition up to "svnd backed by file" speeds, I would be happy.
This is my network backup server where almost 20 machines backup to, so
1MB/s to 2MB/s just isn't going to cut it.

In case somebody asks, I want to encrypt my backup data because I
periodically pull the disk and store it at my girlfriends office.

Any performance enhancing suggestions or alternate methods would be
greatly appreciated. I have thought about encrypting each backup using
openssl, but I would have to script something for that. I am looking for
automation and I feel vnconfig with encryption does it, just not very
quickly.

-pachl

Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]

Messages in current thread:
Slow Performance on Encrypted svnd, Clint Pachl, (Wed Nov 14, 6:02 am)
Re: Slow Performance on Encrypted svnd, Clint Pachl, (Thu Nov 15, 7:36 pm)
Re: Slow Performance on Encrypted svnd, Ted Unangst, (Fri Nov 16, 2:32 pm)
Re: Slow Performance on Encrypted svnd, bofh, (Fri Nov 16, 5:38 pm)
Re: Slow Performance on Encrypted svnd, Nick Guenther, (Fri Nov 16, 4:55 pm)
Re: Slow Performance on Encrypted svnd, Ted Unangst, (Fri Nov 16, 6:34 pm)
Re: Slow Performance on Encrypted svnd, Die Gestalt, (Fri Nov 16, 9:46 am)
Re: Slow Performance on Encrypted svnd, Nick Guenther, (Thu Nov 15, 10:22 am)
Re: Slow Performance on Encrypted svnd, Die Gestalt, (Thu Nov 15, 12:07 pm)
Re: Slow Performance on Encrypted svnd, knitti, (Wed Nov 14, 7:41 am)
Re: Slow Performance on Encrypted svnd, Clint Pachl, (Wed Nov 14, 4:51 pm)
Re: Slow Performance on Encrypted svnd, knitti, (Wed Nov 14, 6:36 pm)
Re: Slow Performance on Encrypted svnd, Clint Pachl, (Wed Nov 14, 6:54 pm)