After Cliff Wright posted a patch for vnd(4) to handle compressed disk
images some time ago[1] and Florian Stoehr wrote a userland tool to
(de)compress disk images (using the cloop2 format also found on other
systems)[2], I sat down to play with the two things tonight.
In short: very successfully!
What I basically did was build a kernel and vnconfig(8) with the patch in
[1], and build Florian's tool (which I received in private from him).
After booting the patched kernel, I did the following steps:
Verify I was running the kernel I just built (this is all in qemu):
qemubsd# uname -a
NetBSD qemubsd 3.99.6 NetBSD 3.99.6 (GENERIC) #1: Mon Jun 13 01:12:47 MEST 2005 feyrer@miyu:/usr/cvs/src-cloop2/sys/arch/i386/compile/obj.i386/GENERIC i386
Create some toy filesystem:
qemubsd# /root/vndz/vnconfig -l
vnd0: not in use
vnd1: not in use
vnd2: not in use
vnd3: not in use
qemubsd# makefs
usage: makefs [-t fs-type] [-o fs-options] [-d debug-mask] [-B endian]
[-S sector-size] [-M minimum-size] [-m maximum-size] [-s image-size]
[-b free-blocks] [-f free-files] [-F mtree-specfile] [-x]
[-N userdb-dir] image-file directory
qemubsd#
qemubsd# makefs inc.fs /usr/include
Calculated size of `inc.fs': 11157504 bytes, 1273 inodes
Extent size set to 8192
inc.fs: 10.6MB (21792 sectors) block size 8192, fragment size 1024
using 1 cylinder groups of 10.64MB, 1362 blks, 1408 inodes.
super-block backups (for fsck -b #) at:
32,
Populating `inc.fs'
Image `inc.fs' complete
Mount the filesystem the usual (uncompressed) way:
qemubsd# /root/vndz/vnconfig vnd0 /tmp/inc.fs
qemubsd# /root/vndz/vnconfig vnd0 -l
vnd0: / (/dev/wd0a) inode 10839
vnd1: not in use
vnd2: not in use
vnd3: not in use
qemubsd#
qemubsd# disklabel vnd0a
disklabel: Invalid signature in mbr record 0
# /dev/rvnd0a:
type: vnd
disk: vnd
label: fictitious
flags:
bytes/sector: 512
sectors/track: 32
tracks/cylinder: 64
sectors/cylinder: 2048
cylinders: 10
total sectors: 21792
rpm: 3600
interleave: 1
trackskew: 0
cylinderskew: 0
headswitch: 0 # microseconds
track-to-track seek: 0 # microseconds
drivedata: 0
4 partitions:
# size offset fstype [fsize bsize cpg/sgs]
a: 21792 0 4.2BSD 0 0 0 # (Cyl. 0 - 10*)
d: 21792 0 unused 0 0 # (Cyl. 0 - 10*)
disklabel: boot block size 0
disklabel: super block size 0
qemubsd#
qemubsd# mount -o ro /dev/vnd0a /mnt/fs
qemubsd# ls -l /mnt/fs | head
total 788
-r--r--r-- 1 root wheel 3178 Jun 4 23:54 a.out.h
drwxr-xr-x 2 root wheel 512 Jun 5 23:58 adosfs
drwxr-xr-x 2 root wheel 512 Jun 5 23:58 altq
-r--r--r-- 1 root wheel 2721 Jun 4 23:54 ar.h
drwxr-xr-x 2 root wheel 512 Jun 5 23:58 arpa
-r--r--r-- 1 root wheel 3993 Jun 4 23:54 assert.h
-r--r--r-- 1 root wheel 4439 Jun 4 23:54 bitstring.h
-r--r--r-- 1 root wheel 2265 Jun 4 23:54 bm.h
-r--r--r-- 1 root wheel 7897 Jun 4 23:55 bzlib.h
Now compress the filesystem image with Florian's clconfig tool:
qemubsd# ls -l
total 10926
-rwxr-xr-x 1 root wheel 11157504 Jun 13 01:34 inc.fs
qemubsd#
qemubsd# /root/vndz/clconfig
usage: clconfig -c disk/fs-image compressed-image [blocksize]
clconfig -d compressed-image disk/fs-image
qemubsd#
qemubsd# /root/vndz/clconfig -c inc.fs inc.fsz
Using blocksize: 65536 (170 complete and 1 zero-padded blocks)
qemubsd#
qemubsd# ls -l
total 13056
-rwxr-xr-x 1 root wheel 11157504 Jun 13 01:34 inc.fs
-rw-r--r-- 1 root wheel 2164224 Jun 13 01:35 inc.fsz
clconfig shrank the image from ~11MB to ~2MB, i.e. down to ~19% of the
original size. Assuming the same compression ration, this would fit about
3.2GB of data on a 650MB CD. 'Nuff for KDE and OpenOffice. :-)
Next, mount the compressed image:
qemubsd# /root/vndz/vnconfig -z vnd1 inc.fsz
qemubsd# /root/vndz/vnconfig -l
vnd0: / (/dev/wd0a) inode 10839
vnd1: / (/dev/wd0a) inode 10840
vnd2: not in use
vnd3: not in use
qemubsd# ls -li inc*
10839 -rwxr-xr-x 1 root wheel 11157504 Jun 13 01:34 inc.fs
10840 -rw-r--r-- 1 root wheel 2164224 Jun 13 01:35 inc.fsz
qemubsd#
qemubsd# disklabel vnd1
disklabel: Invalid signature in mbr record 0
# /dev/rvnd1d:
type: vnd
disk: vnd
label: fictitious
flags:
bytes/sector: 512
sectors/track: 32
tracks/cylinder: 64
sectors/cylinder: 2048
cylinders: 10
total sectors: 21888
rpm: 3600
interleave: 1
trackskew: 0
cylinderskew: 0
headswitch: 0 # microseconds
track-to-track seek: 0 # microseconds
drivedata: 0
4 partitions:
# size offset fstype [fsize bsize cpg/sgs]
a: 21888 0 4.2BSD 0 0 0 # (Cyl. 0 - 10*)
d: 21888 0 unused 0 0 # (Cyl. 0 - 10*)
disklabel: boot block size 0
disklabel: super block size 0
qemubsd#
qemubsd# mount -o ro /dev/vnd1a /mnt/fsz
qemubsd# ls -l /mnt/fsz | head
total 788
-r--r--r-- 1 root wheel 3178 Jun 4 23:54 a.out.h
drwxr-xr-x 2 root wheel 512 Jun 5 23:58 adosfs
drwxr-xr-x 2 root wheel 512 Jun 5 23:58 altq
-r--r--r-- 1 root wheel 2721 Jun 4 23:54 ar.h
drwxr-xr-x 2 root wheel 512 Jun 5 23:58 arpa
-r--r--r-- 1 root wheel 3993 Jun 4 23:54 assert.h
-r--r--r-- 1 root wheel 4439 Jun 4 23:54 bitstring.h
-r--r--r-- 1 root wheel 2265 Jun 4 23:54 bm.h
-r--r--r-- 1 root wheel 7897 Jun 4 23:55 bzlib.h
A very basic test to see if both file systems contain the same data:
qemubsd# diff -bur /mnt/fs /mnt/fsz
qemubsd#
No differences. Success!
Clean up everything:
qemubsd# umount /mnt/fs
qemubsd# umount /mnt/fsz
qemubsd#
qemubsd# /root/vndz/vnconfig -u vnd0
qemubsd# /root/vndz/vnconfig -u vnd1
qemubsd# /root/vndz/vnconfig -l
vnd0: not in use
vnd1: not in use
vnd2: not in use
vnd3: not in use
qemubsd#
Next steps?
* find a better name for the "clconfig" tool, maybe with some input from
Linux. (What's the equivalent tool named there?)
* find someone who'll feel confident to commit the kernel code
* find someone who'll feel confident to commit the userland code
* find someone to write documentation ("how to create a live CD" :-)
Volunteers?
- Hubert
[1] http://mail-index.netbsd.org/tech-kern/2005/03/29/0032.html
[2] http://mail-index.netbsd.org/tech-kern/2005/03/30/0004.html
