login
Header Space

 
 

Re: Git's database structure

Score:
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]
To: Theodore Tso <tytso@...>
Cc: Junio C Hamano <gitster@...>, Git Mailing List <git@...>
Date: Tuesday, September 4, 2007 - 5:54 pm

On 9/4/07, Theodore Tso <tytso@mit.edu> wrote:

It is very easy to get bogged down in performance arguments on
database design when the correct answer is that there are always lots
of different ways to achieve the same goal. I wanted to defer debating
performance until we closely looked at the relationships between the
data at an abstract level.

Since git hasn't stored all of the fields in the object table (the
path is encoded in the index) we are never going to be able to build
an alternative way of indexing the object table. Not being able to
build alternative indexes is likely to cause problems when the
database starts getting really big. Without an index every query that
can't use the path name index is reduced to doing full table scans.

A few things that could benefit from alternative indexing, blame,
full-text search, automating the Maintainers file, etc.

I'm just asking if we really want to make full table scans the only
possible way to implement these types of queries. If the answer is no,
then let's first explore how to fix things at an abstract level before
diving into the performance arguments.

An obvious parallel from the file system world is the locate database
and how it is forced to continuously rescan the file system and store
full path names.




-- 
Jon Smirl
jonsmirl@gmail.com
-
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]

Messages in current thread:
Git's database structure, Jon Smirl, (Tue Sep 4, 11:23 am)
Re: Git's database structure, Julian Phillips, (Tue Sep 4, 1:19 pm)
Re: Git's database structure, Jon Smirl, (Tue Sep 4, 1:30 pm)
Re: Git's database structure, Andreas Ericsson, (Tue Sep 4, 2:51 pm)
Re: Git's database structure, Jon Smirl, (Tue Sep 4, 12:28 pm)
Re: Git's database structure, Junio C Hamano, (Tue Sep 4, 1:25 pm)
Re: Git's database structure, Jon Smirl, (Tue Sep 4, 1:44 pm)
Re: Git's database structure, Theodore Tso, (Tue Sep 4, 5:25 pm)
Re: Git's database structure, Jon Smirl, (Tue Sep 4, 5:54 pm)
Re: Git's database structure, Andreas Ericsson, (Wed Sep 5, 3:18 am)
Re: Git's database structure, Jon Smirl, (Wed Sep 5, 9:41 am)
Re: Git's database structure, Andy Parkins, (Wed Sep 5, 3:52 pm)
Re: Git's database structure, Andreas Ericsson, (Wed Sep 5, 10:51 am)
Re: Git's database structure, Jon Smirl, (Wed Sep 5, 11:37 am)
Re: Git's database structure, Julian Phillips, (Wed Sep 5, 11:54 am)
Re: Git's database structure, Jon Smirl, (Wed Sep 5, 12:12 pm)
Re: Git's database structure, Martin Langhoff, (Thu Sep 6, 8:33 pm)
Re: Git's database structure, Johannes Schindelin, (Thu Sep 6, 8:56 am)
Re: Git's database structure, Steven Grimm, (Thu Sep 6, 2:14 pm)
Re: Git's database structure, Andreas Ericsson, (Thu Sep 6, 4:49 am)
Re: Git's database structure, Junio C Hamano, (Thu Sep 6, 5:09 am)
Re: Git's database structure, Wincent Colaiuta, (Thu Sep 6, 7:03 am)
Re: Git's database structure, Mike Hommey, (Wed Sep 5, 1:39 pm)
Re: Git's database structure, Julian Phillips, (Wed Sep 5, 1:31 pm)
Re: Git's database structure, Kyle Moffett, (Wed Sep 5, 9:27 pm)
Re: Git's database structure, Junio C Hamano, (Tue Sep 4, 2:06 pm)
Re: Git's database structure, Mike Hommey, (Tue Sep 4, 2:04 pm)
Re: Git's database structure, Reece Dunn, (Tue Sep 4, 3:44 pm)
Re: Git's database structure, Andreas Ericsson, (Tue Sep 4, 12:31 pm)
Re: Git's database structure, Jon Smirl, (Tue Sep 4, 12:47 pm)
Re: Git's database structure, Andreas Ericsson, (Tue Sep 4, 12:51 pm)
Re: Git's database structure, Andreas Ericsson, (Tue Sep 4, 11:55 am)
Re: Git's database structure, Junio C Hamano, (Tue Sep 4, 1:21 pm)
Re: Git's database structure, Jon Smirl, (Tue Sep 4, 12:19 pm)
Re: Git's database structure, David Tweed, (Tue Sep 4, 4:17 pm)
Re: Git's database structure, Jeff King, (Tue Sep 4, 1:09 pm)
Re: Git's database structure, Andreas Ericsson, (Tue Sep 4, 12:29 pm)
Re: Git's database structure, Mike Hommey, (Tue Sep 4, 12:07 pm)
Re: Git's database structure, Andreas Ericsson, (Tue Sep 4, 12:10 pm)
speck-geostationary