[PATCH 2/2] Implement a simple delta_base cache

!MAILaRCHIVE_VOTE_RePLACE
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]
To: Junio C Hamano <junkio@...>, Nicolas Pitre <nico@...>
Cc: Git Mailing List <git@...>
Date: Saturday, March 17, 2007 - 3:44 pm

This trivial 256-entry delta_base cache improves performance for some 
loads by a factor of 2.5 or so.

Instead of always re-generating the delta bases (possibly over and over 
and over again), just cache the last few ones. They often can get re-used.

Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
---

This should have some other people doing performance testing too, since 
it's fairly core. But *dang*, it's really simple.

diff --git a/sha1_file.c b/sha1_file.c
index f11ca3f..a7e3a2a 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -1352,16 +1352,57 @@ static void *unpack_compressed_entry(struct packed_git *p,
 	return buffer;
 }
 
+#define MAX_DELTA_CACHE (256)
+
+static struct delta_base_cache_entry {
+	struct packed_git *p;
+	off_t base_offset;
+	unsigned long size;
+	void *data;
+	enum object_type type;
+} delta_base_cache[MAX_DELTA_CACHE];
+
+static unsigned long pack_entry_hash(struct packed_git *p, off_t base_offset)
+{
+	unsigned long hash;
+
+	hash = (unsigned long)p + (unsigned long)base_offset;
+	hash += (hash >> 8) + (hash >> 16);
+	return hash & 0xff;
+}
+
 static void *cache_or_unpack_entry(struct packed_git *p, off_t base_offset,
 	unsigned long *base_size, enum object_type *type)
 {
+	void *ret;
+	unsigned long hash = pack_entry_hash(p, base_offset);
+	struct delta_base_cache_entry *ent = delta_base_cache + hash;
+
+	ret = ent->data;
+	if (ret && ent->p == p && ent->base_offset == base_offset)
+		goto found_cache_entry;
 	return unpack_entry(p, base_offset, type, base_size);
+
+found_cache_entry:
+	ent->data = NULL;
+	*type = ent->type;
+	*base_size = ent->size;
+	return ret;
 }
 
 static void add_delta_base_cache(struct packed_git *p, off_t base_offset,
 	void *base, unsigned long base_size, enum object_type type)
 {
-	free(base);
+	unsigned long hash = pack_entry_hash(p, base_offset);
+	struct delta_base_cache_entry *ent = delta_base_cache + hash;
+
+	if (ent->data)
+		free(ent->data);
+	ent->p = p;
+	ent->base_offset = base_offset;
+	ent->type = type;
+	ent->data = base;
+	ent->size = base_size;
 }
 
 static void *unpack_delta_entry(struct packed_git *p,
-
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]

Messages in current thread:
cleaner/better zlib sources?, Linus Torvalds, (Thu Mar 15, 9:04 pm)
Re: cleaner/better zlib sources?, Davide Libenzi, (Thu Mar 15, 9:33 pm)
Re: cleaner/better zlib sources?, Davide Libenzi, (Thu Mar 15, 10:06 pm)
Re: cleaner/better zlib sources?, Jeff Garzik, (Thu Mar 15, 9:11 pm)
Re: cleaner/better zlib sources?, Linus Torvalds, (Thu Mar 15, 9:46 pm)
Re: cleaner/better zlib sources?, Linus Torvalds, (Thu Mar 15, 9:54 pm)
Re: cleaner/better zlib sources?, Davide Libenzi, (Thu Mar 15, 10:43 pm)
Re: cleaner/better zlib sources?, Linus Torvalds, (Thu Mar 15, 10:56 pm)
Re: cleaner/better zlib sources?, Davide Libenzi, (Thu Mar 15, 11:16 pm)
Re: cleaner/better zlib sources?, Linus Torvalds, (Fri Mar 16, 12:21 pm)
Re: cleaner/better zlib sources?, Nicolas Pitre, (Fri Mar 16, 1:06 pm)
Re: cleaner/better zlib sources?, Linus Torvalds, (Fri Mar 16, 1:51 pm)
Re: cleaner/better zlib sources?, Nicolas Pitre, (Fri Mar 16, 2:09 pm)
Re: cleaner/better zlib sources?, Jeff Garzik, (Fri Mar 16, 12:35 pm)
Re: cleaner/better zlib sources?, Shawn O. Pearce, (Fri Mar 16, 7:22 pm)
Re: cleaner/better zlib sources?, Nicolas Pitre, (Fri Mar 16, 1:12 pm)
Re: cleaner/better zlib sources?, Matt Mackall, (Fri Mar 16, 12:42 pm)
Re: cleaner/better zlib sources?, Linus Torvalds, (Fri Mar 16, 12:51 pm)
Re: cleaner/better zlib sources?, Davide Libenzi, (Fri Mar 16, 12:24 pm)
Re: cleaner/better zlib sources?, Linus Torvalds, (Fri Mar 16, 12:35 pm)
Re: cleaner/better zlib sources?, Davide Libenzi, (Fri Mar 16, 3:21 pm)
Re: cleaner/better zlib sources?, Linus Torvalds, (Fri Mar 16, 8:01 pm)
Re: cleaner/better zlib sources?, Linus Torvalds, (Fri Mar 16, 9:11 pm)
Re: cleaner/better zlib sources?, Nicolas Pitre, (Fri Mar 16, 11:28 pm)
Re: cleaner/better zlib sources?, Linus Torvalds, (Sat Mar 17, 1:55 pm)
Re: cleaner/better zlib sources?, Linus Torvalds, (Sat Mar 17, 3:40 pm)
[PATCH 2/2] Implement a simple delta_base cache, Linus Torvalds, (Sat Mar 17, 3:44 pm)
Re: [PATCH 2/2] Implement a simple delta_base cache, Linus Torvalds, (Sat Mar 17, 6:44 pm)
Re: [PATCH 2/2] Implement a simple delta_base cache, Linus Torvalds, (Sat Mar 17, 5:45 pm)
Re: [PATCH 2/2] Implement a simple delta_base cache, Morten Welinder, (Sat Mar 17, 9:14 pm)
Re: [PATCH 2/2] Implement a simple delta_base cache, Avi Kivity, (Sun Mar 18, 2:28 am)
Re: [PATCH 2/2] Implement a simple delta_base cache, Linus Torvalds, (Sat Mar 17, 9:29 pm)
Re: [PATCH 2/2] Implement a simple delta_base cache, Linus Torvalds, (Sat Mar 17, 9:44 pm)
Re: [PATCH 2/2] Implement a simple delta_base cache, Nicolas Pitre, (Sat Mar 17, 9:38 pm)
Re: [PATCH 2/2] Implement a simple delta_base cache, Linus Torvalds, (Sat Mar 17, 9:55 pm)
Re: [PATCH 2/2] Implement a simple delta_base cache, Nicolas Pitre, (Sat Mar 17, 10:03 pm)
Re: [PATCH 2/2] Implement a simple delta_base cache, Linus Torvalds, (Sat Mar 17, 10:20 pm)
[PATCH 3/2] Avoid unnecessary strlen() calls, Linus Torvalds, (Sat Mar 17, 11:06 pm)
Re: [PATCH 3/2] Avoid unnecessary strlen() calls, Junio C Hamano, (Sun Mar 18, 5:45 am)
Re: [PATCH 3/2] Avoid unnecessary strlen() calls, Linus Torvalds, (Sun Mar 18, 11:54 am)
Re: [PATCH 3/2] Avoid unnecessary strlen() calls, Junio C Hamano, (Mon Mar 19, 11:16 pm)
Re: [PATCH 3/2] Avoid unnecessary strlen() calls, Linus Torvalds, (Tue Mar 20, 12:31 am)
Re: [PATCH 3/2] Avoid unnecessary strlen() calls, Shawn O. Pearce, (Tue Mar 20, 12:39 am)
Re: [PATCH 3/2] Avoid unnecessary strlen() calls, Linus Torvalds, (Tue Mar 20, 12:57 am)
Re: [PATCH 3/2] Avoid unnecessary strlen() calls, Johannes Schindelin, (Mon Mar 19, 11:05 pm)
Re: [PATCH 3/2] Avoid unnecessary strlen() calls, Shawn O. Pearce, (Mon Mar 19, 11:29 pm)
Re: [PATCH 3/2] Avoid unnecessary strlen() calls, Shawn O. Pearce, (Mon Mar 19, 11:40 pm)
Re: [PATCH 3/2] Avoid unnecessary strlen() calls, Linus Torvalds, (Tue Mar 20, 12:11 am)
Re: [PATCH 3/2] Avoid unnecessary strlen() calls, Junio C Hamano, (Tue Mar 20, 1:44 am)
Re: [PATCH 3/2] Avoid unnecessary strlen() calls, Shawn O. Pearce, (Tue Mar 20, 12:18 am)
Re: [PATCH 3/2] Avoid unnecessary strlen() calls, Linus Torvalds, (Tue Mar 20, 12:45 am)
Re: [PATCH 3/2] Avoid unnecessary strlen() calls, Linus Torvalds, (Sun Mar 18, 11:57 am)
Re: [PATCH 3/2] Avoid unnecessary strlen() calls, Shawn O. Pearce, (Sun Mar 18, 5:38 pm)
Re: [PATCH 3/2] Avoid unnecessary strlen() calls, Linus Torvalds, (Sun Mar 18, 5:48 pm)
Re: [PATCH 2/2] Implement a simple delta_base cache, Nicolas Pitre, (Sat Mar 17, 11:00 pm)
Re: [PATCH 2/2] Implement a simple delta_base cache, Linus Torvalds, (Sat Mar 17, 11:31 pm)
Re: [PATCH 2/2] Implement a simple delta_base cache, Robin Rosenberg, (Sun Mar 18, 6:53 am)
Re: [PATCH 2/2] Implement a simple delta_base cache, Linus Torvalds, (Sun Mar 18, 1:34 pm)
Re: [PATCH 2/2] Implement a simple delta_base cache, Robin Rosenberg, (Sun Mar 18, 2:29 pm)
Re: [PATCH 2/2] Implement a simple delta_base cache, David Brodsky, (Mon Mar 19, 9:16 am)
Re: [PATCH 2/2] Implement a simple delta_base cache, Robin Rosenberg, (Tue Mar 20, 2:35 am)
Re: [PATCH 2/2] Implement a simple delta_base cache, David Brodsky, (Tue Mar 20, 5:13 am)
Re: [PATCH 2/2] Implement a simple delta_base cache, Linus Torvalds, (Tue Mar 20, 10:37 pm)
Re: [PATCH 2/2] Implement a simple delta_base cache, Nicolas Pitre, (Tue Mar 20, 10:54 pm)
Re: [PATCH 2/2] Implement a simple delta_base cache, Shawn O. Pearce, (Sun Mar 18, 5:25 pm)
Re: [PATCH 2/2] Implement a simple delta_base cache, Julian Phillips, (Sun Mar 18, 1:30 am)
Re: [PATCH 2/2] Implement a simple delta_base cache, Linus Torvalds, (Sun Mar 18, 1:23 pm)
Re: [PATCH 2/2] Implement a simple delta_base cache, Junio C Hamano, (Sat Mar 17, 7:12 pm)
Re: [PATCH 2/2] Implement a simple delta_base cache, Linus Torvalds, (Sat Mar 17, 7:24 pm)
Re: [PATCH 2/2] Implement a simple delta_base cache, Jon Smirl, (Sat Mar 17, 7:52 pm)
Re: [PATCH 2/2] Implement a simple delta_base cache, Junio C Hamano, (Sat Mar 17, 6:37 pm)
Re: [PATCH 2/2] Implement a simple delta_base cache, Nicolas Pitre, (Sat Mar 17, 9:13 pm)
Re: [PATCH 2/2] Implement a simple delta_base cache, Junio C Hamano, (Sun Mar 18, 3:47 am)
Re: [PATCH 2/2] Implement a simple delta_base cache, Linus Torvalds, (Sat Mar 17, 7:09 pm)
Re: [PATCH 2/2] Implement a simple delta_base cache, Linus Torvalds, (Sat Mar 17, 7:54 pm)
Re: cleaner/better zlib sources?, Shawn O. Pearce, (Sat Mar 17, 1:19 am)
Re: cleaner/better zlib sources?, Matt Mackall, (Thu Mar 15, 9:14 pm)
Re: cleaner/better zlib sources?, Shawn O. Pearce, (Thu Mar 15, 9:10 pm)