Btrfs Online Resizing, Ext3 Conversion, and More

Submitted by Jeremy
on January 17, 2008 - 8:06am

Chris Mason announced version 0.10 of his new Btrfs filesystem, listing the following new features, "explicit back references, online resizing (including shrinking), in place conversion from Ext3 to Btrfs, data=ordered support, mount options to disable data COW and checksumming, and barrier support for sata and IDE drives". He noted that the disk format in v0.10 has changed, and is not compatible with the v0.9 disk format. Regarding back reference support, Chris explained, "the core of this release is explicit back references for all metadata blocks, data extents, and directory items. These are a crucial building block for future features such as online fsck and migration between devices. The back references are verified during deletes, and the extent back references are checked by the existing offline fsck tool." He then detailed the new Ext3 to Btrfs conversion utility:

"The conversion program uses the copy on write nature of Btrfs to preserve the original Ext3 FS, sharing the data blocks between Btrfs and Ext3 metadata. Btrfs metadata is created inside the free space of the Ext3 filesystem, and it is possible to either make the conversion permanent (reclaiming the space used by Ext3) or roll back the conversion to the original Ext3 filesystem."


From: Chris Mason
Subject: [ANNOUNCE] Btrfs v0.10 (online growing/shrinking, ext3 conversion, and more)
Date: Jan 15, 8:52 am 2008

Hello everyone,

Btrfs v0.10 is now available for download from:

http://oss.oracle.com/projects/btrfs/

Btrfs is still in an early alpha state, and the disk format is not finalized.
v0.10 introduces a new disk format, and is not compatible with v0.9.

The core of this release is explicit back references for all metadata blocks,
data extents, and directory items.  These are a crucial building block for
future features such as online fsck and migration between devices.  The back
references are verified during deletes, and the extent back references are
checked by the existing offline fsck tool.

For all of the details of how the back references are maintained, please
see the design document:

http://oss.oracle.com/projects/btrfs/dist/documentation/btrfs-design.html

Other new features (described in detail below):

* Online resizing (including shrinking)
* In place conversion from Ext3 to Btrfs
* data=ordered support
* Mount options to disable data COW and checksumming 
* Barrier support for sata and IDE drives

[ Resizing ]

In order to demonstrate and test the back references, I've added an online
resizer, which can both grow and shrink the filesystem:

mount -t btrfs /dev/xxx /mnt

# add 2GB to the FS
btrfsctl -r +2g /mnt

# shrink the FS by 4GB
btrfsctl -r -4g /mnt

# Explicitly set the FS size
btrfsctl -r 20g /mnt

# Use 'max' to grow the FS to the limit of the device
btrfsctl -r max /mnt

[ Conversion from Ext3 ]

This is an offline, in place, conversion program written by Yan Zheng.  It
has been through basic testing, but should not be trusted with critical data.

To build the conversion program, run 'make convert' in the btrfs-progs
tree. It depends on libe2fs and acl development libraries.

The conversion program uses the copy on write nature of Btrfs to preserve the
original Ext3 FS, sharing the data blocks between Btrfs and Ext3 metadata.
Btrfs metadata is created inside the free space of the Ext3 filesystem, and it
is possible to either make the conversion permanent (reclaiming the space used
by Ext3) or roll back the conversion to the original Ext3 filesystem.

More details and example usage of the conversion program can be found here:

http://oss.oracle.com/projects/btrfs/dist/documentation/btrfs-converter.html

Thanks to Yan Zheng for all of his work on the converter.

[ New mount options ]

mount -o nodatacsum disables checksumming on data extents

mount -o nodatacow disables copy on write of data extents, unless a given
extent is referenced by more than one snapshot.  This is targeted at database
workloads, where copy on write is not optimal for performance.

The explicit back references allow the nodatacow code to make sure copy
on write is done when multiple snapshots reference the same file, maintaining
snapshot consistency.

mount -o alloc_start=num forces allocation hints to start at least num bytes
into the disk.  This was introduced to test the resizer.  Example usage:

mount -o alloc_start=16g /dev/xxxx /mnt
(do something to the FS)
btrfsctl -r 12g /mnt

The btrfsctl command will resize the FS down to 12GB in size.  Because
the FS was mounted with -o alloc_start=16g, any allocations done after
mounting will need to be relocated by the resizer.

It is safe to specify a number past the end of the FS, if the alloc_start is too
large, it is ignored.

mount -o nobarrier disables cache flushes during commit.

-chris
-
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Quickly followed by v0.11...

Jeremy
on
January 17, 2008 - 11:48am

From: Chris mason
Subject: Re: [Btrfs-devel] [ANNOUNCE] Btrfs v0.10 (online growing/shrinking, ext3 conversion, and more)
Date: Jan 17, 11:25 am 2008

On Tuesday 15 January 2008, Chris Mason wrote:
> Hello everyone,
>
> Btrfs v0.10 is now available for download from:
>
> http://oss.oracle.com/projects/btrfs/

Well, it turns out this release had a few small problems:

* data=ordered deadlock on older kernels (including 2.6.23)
* Compile problems when ACLs were not enabled in the kernel

So, I've put v0.11 out there.  It fixes those two problems and will also 
compile on older (2.6.18) enterprise kernels.

v0.11 does not have any disk format changes.

-chris


-
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Isn't there a

xerces8
on
January 18, 2008 - 4:46am

Isn't there a program/script, that can convert between any fs types in place ?

Converting in-place is

anonymous (not verified)
on
January 20, 2008 - 2:13pm

Converting in-place is somewhat tricky and no one may expect to have a generic method to do that. Every method uses some features of file systems to be (was) used. There was an idea belong to mr. Tsucanov how make any2any fs conversion. In few words it looked like the following:

1. make a sparse loop file on source fs;
2. create target fs in loop file;
3. mount loop file (tgt fs);
4. move all data from your src fs to loop file fs;
5. perform loop file fs mapping from loop device to your hdd partition device (there was a script).

That is it.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.