Re: index-pack died on pread

!MAILaRCHIVE_VOTE_RePLACE
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]
To: Linus Torvalds <torvalds@...>
Cc: Junio C Hamano <gitster@...>, Alex Riesen <raa.lkml@...>, Robin Rosenberg <robin.rosenberg.lists@...>, Michal Rokos <michal.rokos@...>, GIT <git@...>
Date: Friday, July 27, 2007 - 9:38 am

On Thu, 26 Jul 2007, Linus Torvalds wrote:


The (only) user in Git does care indeed.  The sequence of events goes 
like this:

 - Receive pack from remote location, parsing objects along and writing
   a copy to the local pack file, as well as computing the SHA1 of non
   delta objects.

 - When all objects are parsed, the received pack SHA1 is verified.  
   If SHA1 doesn't match due to corruption in received data then everything 
   stops here with a "pack is corrupted (SHA1 mismatch)" error.

 - For each received deltas: resolve those deltas to compute their 
   object SHA1.  This is where pread() is involved on the local pack 
   file. If pread() fails to return proper data then the data won't 
   inflate properly and everything stops here with a "serious inflate 
   inconsistency" error.

 - [OPTIONAL] Complete a thin pack with missing base objects for deltas 
   that weren't resolved yet.  This involves pread() again, but it 
   _also_ append data to the same local pack file in the process.  In 
   this case it is important that pread() restores the file position 
   when it returns or the appended objects won't be written where they 
   should.  

   This is optional because a thin pack is not received all the time, 
   not on a clone for example.

 - Write trailing SHA1 to the local pack before moving it to its final 
   location.  This also relies on pread() restoring the file position on 
   the local pack file or the trailing pack SHA1 won't be written where 
   it should.

So if pread() doesn't properly restore the file position then local pack 
corruption will occur and the pack will be unusable.  If pread() doesn't 
properly read the asked data then index-pack will die.


Well, we have usage cases for a real pread() as well as our own 
emulation which work.  And the emulated pread() works in all cases so 
far, so...


Nicolas
-
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]

Messages in current thread:
index-pack died on pread, Michal Rokos, (Mon Jul 23, 8:52 am)
Re: index-pack died on pread, Linus Torvalds, (Mon Jul 23, 1:04 pm)
Re: index-pack died on pread, Robin Rosenberg, (Wed Jul 25, 7:15 pm)
Re: index-pack died on pread, Linus Torvalds, (Wed Jul 25, 7:44 pm)
Re: index-pack died on pread, Alex Riesen, (Thu Jul 26, 8:42 am)
Re: index-pack died on pread, Linus Torvalds, (Thu Jul 26, 12:13 pm)
Re: index-pack died on pread, Junio C Hamano, (Thu Jul 26, 11:43 pm)
Re: index-pack died on pread, Linus Torvalds, (Fri Jul 27, 1:36 am)
Re: index-pack died on pread, Nicolas Pitre, (Fri Jul 27, 9:38 am)
Re: index-pack died on pread, Tomash Brechko, (Fri Jul 27, 5:50 am)
Re: index-pack died on pread, Tomash Brechko, (Fri Jul 27, 6:33 am)
Re: index-pack died on pread, Alex Riesen, (Thu Jul 26, 12:51 pm)
Re: index-pack died on pread, Linus Torvalds, (Thu Jul 26, 2:02 pm)
Re: index-pack died on pread, Nicolas Pitre, (Mon Jul 23, 2:03 pm)
Re: index-pack died on pread, Alex Riesen, (Mon Jul 23, 11:32 am)
Re: index-pack died on pread, Michal Rokos, (Wed Jul 25, 4:07 pm)
Re: index-pack died on pread, Alex Riesen, (Wed Jul 25, 4:48 pm)