zfs: unexpected resilver of old disk when attaching a new mirror disk

Previous thread: Re: kern/145339: [zfs] deadlock after detaching block device from raidz pool by Alex Bakhtin on Wednesday, April 21, 2010 - 12:50 pm. (1 message)

Next thread: Re: kern/145339: [zfs] deadlock after detaching block device from raidz pool by Steve Polyack on Thursday, April 22, 2010 - 8:20 am. (1 message)
From: Martin Simmons
Date: Thursday, April 22, 2010 - 7:41 am

When I attach a second disk to a single-disk zpool to make a mirrored pool,
the original disk is partially resilvered.  Is that expected?  It makes me
worry that some of the good data is being overwritten.

It happens on 8.0-RELEASE (pool version 13) and the 9.0-CURRENT-201004
snapshot (pool version 14).  It doesn't happen on Open Solaris snv_111b (also
pool version 14).

My setup commands for the test, making a 64MB zpool containing a 16MB file
are:

dd if=/dev/zero of=/tmp/zdisk0 bs=1m count=64
zpool create ztest /tmp/zdisk0
dd if=/dev/zero of=/ztest/a-file bs=1m count=16

The pool looks as expected:

  pool: ztest
 state: ONLINE
 scrub: none requested
config:

	NAME           STATE     READ WRITE CKSUM
	ztest          ONLINE       0     0     0
	  /tmp/zdisk0  ONLINE       0     0     0

errors: No known data errors

I then attach a second device:

dd if=/dev/zero of=/tmp/zdisk1 bs=1m count=64
zpool attach ztest /tmp/zdisk0 /tmp/zdisk1

and the pool looks like this after a while:

  pool: ztest
 state: ONLINE
 scrub: resilver completed after 0h0m with 0 errors on Thu Apr 22 00:29:32 2010
config:

	NAME             STATE     READ WRITE CKSUM
	ztest            ONLINE       0     0     0
	  mirror         ONLINE       0     0     0
	    /tmp/zdisk0  ONLINE       0     0     0  86K resilvered
	    /tmp/zdisk1  ONLINE       0     0     0  16.1M resilvered

errors: No known data errors

As expected, /tmp/zdisk1 was resilvered with 16MB of data for /ztest/a-file,
but why was a small amount of /tmp/zdisk0 also resilvered?

The amount varies varies (I've seen up to 220K) but it is never 0.  With real
disks constaining 5GB, the amount resilvered on the old disk is up to 70MB and
it increases gradually from 0 as the resilvering of the new disk proceeds.

__Martin
_______________________________________________
freebsd-fs@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-fs
To unsubscribe, send any mail to ...
From: Adam Nowacki
Date: Thursday, April 22, 2010 - 8:11 am

My best guess is metadata updates to refer to the 2nd disk. I've seen 
same behavior when replacing a bad disk in 10 disk raidz2 pool. No data 
was affected.


_______________________________________________
freebsd-fs@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-fs
To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org"
From: Martin Simmons
Date: Thursday, April 22, 2010 - 10:58 am

I would expect all new writes to automatically go to all disks though, so they
wouldn't be counted as resilvering.

__Martin
_______________________________________________
freebsd-fs@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-fs
To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org"
From: Chris Ruiz
Date: Thursday, April 22, 2010 - 7:13 pm

I don't know the reason why this happens but I do know that it's
normal and that your data is ok.

-- Chris

-----------------------------------------
http://twitter.com/chrisattack
http://chrisattack.com
_______________________________________________
freebsd-fs@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-fs
To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org"
From: Volodymyr Kostyrko
Date: Wednesday, May 5, 2010 - 12:54 pm

1. ZFS tries to store checksums as far as it can. Another drive preferrably.
2. ZFS tries to store metadata copies in different location as far as it 
can.

-- 
Sphinx of black quartz judge my vow.
_______________________________________________
freebsd-fs@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-fs
To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org"
Previous thread: Re: kern/145339: [zfs] deadlock after detaching block device from raidz pool by Alex Bakhtin on Wednesday, April 21, 2010 - 12:50 pm. (1 message)

Next thread: Re: kern/145339: [zfs] deadlock after detaching block device from raidz pool by Steve Polyack on Thursday, April 22, 2010 - 8:20 am. (1 message)