Re: tree corrupted on disk quota full

!MAILaRCHIVE_VOTE_RePLACE
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]
To: Michael S. Tsirkin <mst@...>
Cc: Junio C Hamano <junkio@...>, Git Mailing List <git@...>, Andy Whitcroft <apw@...>
Date: Thursday, January 11, 2007 - 6:24 pm

On Thu, 11 Jan 2007, Michael S. Tsirkin wrote:

Well, it's a bug, and it's not "expected", but I think it's understood. 

What happens is that you pulled objects that were perfectly fine, and when 
you ran out of diskspace git could create the _filenames_ and inodes for 
them, but when it actually wrote the data itself, it failed.

And it failed in a very unfortunate place: when unpacking the pack-file 
(which you got when doing the "git pull"), it would call 
"write_sha1_file()" to actually write the unpacked object. That one does 
everything correctly and is actually very careful, but then it does

	if (write_buffer(fd, compressed, size) < 0)
		die("unable to write sha1 file");

which _should_ have caught the case that the buffer couldn't be written, 
and died cleanly with an error message, and it would neve rhave moved the 
temporary file that it wrote to into the real database.

In other words, it was doing everything right. A partial file would never 
have actually been allowed to be moved into the database at all, and you'd 
have gotten a nice error message, and the "git pull" would have failed 
gracefully.

However, "write_buffer()" itself was buggy. It did:

	static int write_buffer(int fd, const void *buf, size_t len)
	{
	        ssize_t size;
	
	        size = write_in_full(fd, buf, len);
	        if (!size)
	                return error("file write: disk full");
	        if (size < 0)
	                return error("file write error (%s)", strerror(errno));
	        return 0;
	}

and the problem here is that if the write was _partial_ (not totally 
empty), it would see a positive size, but not a full write, and it would 
incorrectly return zero for "everything is fine". Even though it wasn't.

So my patch to make "write_in_full()" have better semantics (and make it 
simpler too) should have fixed the fundamental problem that caused your 
failure in the first place. The other patch I just sent out should just 
make the error messages be a bit nicer, and isn't important in itself.

Btw, that "write_buffer()" bug was introduced pretty recently. It used to 
be that write_buffer() did the right thing: it used to do:

	while (len) {
		ssize_t size;

		size = write(fd, buf, len);
		if (!size)
			return error("file write: disk full");
		if (size < 0) {
			if (errno == EINTR || errno == EAGAIN)
				continue;
			return error("file write error (%s)", strerror(errno));
		}
		len -= size;
		buf = (char *) buf + size;
	}

which is safe and does the right thing for partial writes and disk full 
errors (of course it is: I wrote that piece of code).

So this bug was _literally_ introduced three days ago, and it was 
introduced by a set of patches that tried to _avoid_ the problem.

Sad. The old code was perfectly good. The new code was unsafe, but thought 
it was better. See commit 93822c2239a336e5cb583549071c59202ef6c5b2 for 
details.

Junio, pls apply my "fix write_in_full" patch asap if you haven't already. 
I find it very irritating when people "clean up" my code, and introduce 
bugs when they do so.

		Linus
-
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]

Messages in current thread:
tree corrupted on disk quota full, Michael S. Tsirkin, (Thu Jan 11, 8:57 am)
Re: tree corrupted on disk quota full, Linus Torvalds, (Thu Jan 11, 2:31 pm)
Re: tree corrupted on disk quota full, Michael S. Tsirkin, (Thu Jan 11, 5:11 pm)
Re: tree corrupted on disk quota full, Linus Torvalds, (Thu Jan 11, 5:50 pm)
Re: tree corrupted on disk quota full, Michael S. Tsirkin, (Thu Jan 11, 5:58 pm)
Re: tree corrupted on disk quota full, Linus Torvalds, (Thu Jan 11, 6:24 pm)
Re: tree corrupted on disk quota full, Michael S. Tsirkin, (Thu Jan 11, 6:39 pm)
Re: tree corrupted on disk quota full, Linus Torvalds, (Thu Jan 11, 8:42 pm)
Re: tree corrupted on disk quota full, Shawn O. Pearce, (Thu Jan 11, 8:51 pm)
Re: tree corrupted on disk quota full, Linus Torvalds, (Thu Jan 11, 3:19 pm)
Re: tree corrupted on disk quota full, Andy Whitcroft, (Thu Jan 11, 5:17 pm)
Re: tree corrupted on disk quota full, Linus Torvalds, (Thu Jan 11, 5:59 pm)
Re: tree corrupted on disk quota full, Andy Whitcroft, (Thu Jan 11, 6:10 pm)
Re: tree corrupted on disk quota full, Linus Torvalds, (Thu Jan 11, 6:32 pm)
Re: tree corrupted on disk quota full, Junio C Hamano, (Thu Jan 11, 5:27 pm)
Better error messages for corrupt databases, Linus Torvalds, (Thu Jan 11, 6:09 pm)
Re: Better error messages for corrupt databases, Shawn O. Pearce, (Thu Jan 11, 8:40 pm)
Re: tree corrupted on disk quota full, Junio C Hamano, (Thu Jan 11, 5:02 pm)
Re: tree corrupted on disk quota full, Linus Torvalds, (Thu Jan 11, 4:03 pm)
Re: tree corrupted on disk quota full, Andreas Ericsson, (Thu Jan 11, 9:40 am)
Re: tree corrupted on disk quota full, Michael S. Tsirkin, (Thu Jan 11, 9:59 am)
Re: tree corrupted on disk quota full, Andreas Ericsson, (Thu Jan 11, 10:28 am)
Re: tree corrupted on disk quota full, Andy Whitcroft, (Thu Jan 11, 9:43 am)