Re: absurdly slow git-diff

!MAILaRCHIVE_VOTE_RePLACE
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]
To: Davide Libenzi <davidel@...>
Cc: Linus Torvalds <torvalds@...>, Abhijit Menon-Sen <ams@...>, Pierre Habouzit <madcoder@...>, Git Mailing List <git@...>
Date: Saturday, November 8, 2008 - 1:30 am

Davide Libenzi <davidel@xmailserver.org> writes:


Thanks, will apply like this, but I am not sure if you meant windowN or
just window...

-- >8 --
From: Davide Libenzi <davidel@xmailserver.org>
Date: Fri, 7 Nov 2008 21:24:33 -0800
Subject: [PATCH] xdiff: give up scanning similar lines early

In a corner case of large files whose lines do not match uniquely, the
loop to eliminate a line that matches multiple locations adjacent to a run
of lines that do not uniquely match wasted too much cycles.  Fix this by
giving up early after scanning 100 lines in both direction.
---
 xdiff/xprepare.c |   15 +++++++++++++--
 1 files changed, 13 insertions(+), 2 deletions(-)

diff --git a/xdiff/xprepare.c b/xdiff/xprepare.c
index e87ab57..6a70cdf 100644
--- a/xdiff/xprepare.c
+++ b/xdiff/xprepare.c
@@ -23,10 +23,9 @@
 #include "xinclude.h"
 
 
-
 #define XDL_KPDIS_RUN 4
 #define XDL_MAX_EQLIMIT 1024
-
+#define XDL_SIMSCAN_WINDOWN 100
 
 
 typedef struct s_xdlclass {
@@ -313,6 +312,18 @@ static int xdl_clean_mmatch(char const *dis, long i, long s, long e) {
 	long r, rdis0, rpdis0, rdis1, rpdis1;
 
 	/*
+	 * Limits the window the is examined during the similar-lines
+	 * scan. The loops below stops when dis[i - r] == 1 (line that
+	 * has no match), but there are corner cases where the loop
+	 * proceed all the way to the extremities by causing huge
+	 * performance penalties in case of big files.
+	 */
+	if (i - s > XDL_SIMSCAN_WINDOWN)
+		s = i - XDL_SIMSCAN_WINDOWN;
+	if (e - i > XDL_SIMSCAN_WINDOWN)
+		e = i + XDL_SIMSCAN_WINDOWN;
+
+	/*
 	 * Scans the lines before 'i' to find a run of lines that either
 	 * have no match (dis[j] == 0) or have multiple matches (dis[j] > 1).
 	 * Note that we always call this function with dis[i] > 1, so the
-- 
1.6.0.3.674.gdf99f
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]

Messages in current thread:
absurdly slow git-diff, Abhijit Menon-Sen, (Fri Nov 7, 4:01 pm)
Re: absurdly slow git-diff, Linus Torvalds, (Fri Nov 7, 5:37 pm)
Re: absurdly slow git-diff, Pierre Habouzit, (Fri Nov 7, 8:14 pm)
Re: absurdly slow git-diff, Davide Libenzi, (Fri Nov 7, 7:04 pm)
Re: absurdly slow git-diff, Davide Libenzi, (Fri Nov 7, 7:18 pm)
Re: absurdly slow git-diff, Linus Torvalds, (Fri Nov 7, 7:42 pm)
Re: absurdly slow git-diff, Davide Libenzi, (Fri Nov 7, 7:48 pm)
Re: absurdly slow git-diff, Junio C Hamano, (Sat Nov 8, 1:30 am)
Re: absurdly slow git-diff, Davide Libenzi, (Sat Nov 8, 12:27 pm)
Re: absurdly slow git-diff, Linus Torvalds, (Fri Nov 7, 7:57 pm)
Re: absurdly slow git-diff, Junio C Hamano, (Sat Nov 8, 5:02 pm)
Re: absurdly slow git-diff, Abhijit Menon-Sen, (Sat Nov 8, 12:57 am)
Re: absurdly slow git-diff, Mike Hommey, (Fri Nov 7, 5:28 pm)