Re: kernel.org mirroring (Re: [GIT PULL] MMC update)

Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]
From: Rogan Dawes
Date: Friday, December 8, 2006 - 5:57 am

H. Peter Anvin wrote:

How about extending gitweb to check to see if there already exists a 
cached version of these pages, before recreating them?

e.g. structure the temp dir in such a way that each project has a place 
for cached pages. Then, before performing expensive operations, check to 
see if a file corresponding to the requested page already exists. If it 
does, simply return the contents of the file, otherwise go ahead and 
create the page dynamically, and return it to the user. Do not create 
cached pages in gitweb dynamically.

Then, in a post-update hook, for each of the expensive pages, invoke 
something like:

# delete the cached copy of the file, to force gitweb to recreate it
rm -f $git_temp/$project/rss
# get gitweb to recreate the page appropriately
# use a tmp file to prevent gitweb from getting confused
wget -O $git_temp/$project/rss.tmp \
   http://kernel.org/gitweb.cgi?p=$project;a=rss
# move the tmp file into place
mv $git_temp/$project/rss.tmp $git_temp/$project/rss

This way, we get the exact output returned from the usual gitweb 
invocation, but we can now cache the result, and only update it when 
there is a new commit that would affect the page output.

This would also not affect those who do not wish to use this mechanism. 
If the file does not exist, gitweb.cgi will simply revert to its usual 
behaviour.

Possible complications are the content-type headers, etc, but you could 
use the -s flag to wget, and store the server headers as well in the 
file, and get the necessary headers from the file as you stream it.

i.e. read the headers looking for ones that are "interesting" 
(Content-Type, charset, expires) until you get a blank line, print out 
the interesting headers using $cgi->header(), then just dump the 
remainder of the file to the caller via stdout.

Rogan
-
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]

Messages in current thread:
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), Linus Torvalds, (Thu Dec 7, 12:05 pm)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), H. Peter Anvin, (Thu Dec 7, 12:16 pm)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), Olivier Galibert, (Thu Dec 7, 12:30 pm)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), Linus Torvalds, (Thu Dec 7, 12:30 pm)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), Shawn Pearce, (Thu Dec 7, 12:39 pm)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), H. Peter Anvin, (Thu Dec 7, 12:57 pm)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), Linus Torvalds, (Thu Dec 7, 12:58 pm)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), H. Peter Anvin, (Thu Dec 7, 12:58 pm)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), Junio C Hamano, (Thu Dec 7, 1:05 pm)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), H. Peter Anvin, (Thu Dec 7, 1:09 pm)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), Junio C Hamano, (Thu Dec 7, 3:11 pm)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), Michael K. Edwards, (Thu Dec 7, 4:33 pm)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), Olivier Galibert, (Thu Dec 7, 4:50 pm)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), H. Peter Anvin, (Thu Dec 7, 4:56 pm)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), Jakub Narebski, (Fri Dec 8, 2:43 am)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), Rogan Dawes, (Fri Dec 8, 5:57 am)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), Jakub Narebski, (Fri Dec 8, 6:38 am)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), Rogan Dawes, (Fri Dec 8, 7:31 am)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), Jonas Fonseca, (Fri Dec 8, 8:38 am)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), H. Peter Anvin, (Fri Dec 8, 9:16 am)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), Linus Torvalds, (Fri Dec 8, 9:35 am)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), H. Peter Anvin, (Fri Dec 8, 9:42 am)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), Jeff Garzik, (Fri Dec 8, 9:54 am)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), H. Peter Anvin, (Fri Dec 8, 10:04 am)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), Jeff Garzik, (Fri Dec 8, 10:40 am)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), Lars Hjemli, (Fri Dec 8, 12:49 pm)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), H. Peter Anvin, (Fri Dec 8, 12:51 pm)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), Lars Hjemli, (Fri Dec 8, 12:59 pm)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), H. Peter Anvin, (Fri Dec 8, 1:02 pm)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), Linus Torvalds, (Fri Dec 8, 4:27 pm)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), Michael K. Edwards, (Fri Dec 8, 4:46 pm)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), H. Peter Anvin, (Fri Dec 8, 4:49 pm)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), Michael K. Edwards, (Fri Dec 8, 5:18 pm)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), H. Peter Anvin, (Fri Dec 8, 5:23 pm)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), Linus Torvalds, (Fri Dec 8, 5:49 pm)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), H. Peter Anvin, (Fri Dec 8, 5:51 pm)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), Martin Langhoff, (Fri Dec 8, 6:28 pm)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), Martin Langhoff, (Fri Dec 8, 6:56 pm)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), H. Peter Anvin, (Fri Dec 8, 7:03 pm)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), Martin Langhoff, (Fri Dec 8, 7:52 pm)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), Michael K. Edwards, (Fri Dec 8, 9:36 pm)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), H. Peter Anvin, (Fri Dec 8, 10:09 pm)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), Martin Langhoff, (Fri Dec 8, 10:34 pm)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), Steven Grimm, (Sat Dec 9, 12:56 am)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), Jeff Garzik, (Sat Dec 9, 2:27 am)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), Jakub Narebski, (Sat Dec 9, 4:51 am)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), Jeff Garzik, (Sat Dec 9, 5:42 am)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), Jakub Narebski, (Sat Dec 9, 6:37 am)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), Jeff Garzik, (Sat Dec 9, 7:43 am)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), H. Peter Anvin, (Sat Dec 9, 9:26 am)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), Jakub Narebski, (Sat Dec 9, 10:02 am)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), Jeff Garzik, (Sat Dec 9, 10:27 am)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), Linus Torvalds, (Sat Dec 9, 11:04 am)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), H. Peter Anvin, (Sat Dec 9, 11:30 am)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), Martin Langhoff, (Sat Dec 9, 8:55 pm)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), Martin Langhoff, (Sat Dec 9, 9:07 pm)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), H. Peter Anvin, (Sun Dec 10, 12:05 am)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), Jakub Narebski, (Sun Dec 10, 3:09 am)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), Jeff Garzik, (Sun Dec 10, 5:41 am)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), Jakub Narebski, (Sun Dec 10, 6:02 am)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), Jeff Garzik, (Sun Dec 10, 6:45 am)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), Jakub Narebski, (Sun Dec 10, 12:11 pm)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), Linus Torvalds, (Sun Dec 10, 12:50 pm)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), Jakub Narebski, (Sun Dec 10, 1:27 pm)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), Linus Torvalds, (Sun Dec 10, 1:30 pm)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), H. Peter Anvin, (Sun Dec 10, 2:01 pm)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), Martin Langhoff, (Sun Dec 10, 3:01 pm)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), Jeff Garzik, (Sun Dec 10, 3:05 pm)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), Jeff Garzik, (Sun Dec 10, 3:08 pm)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), Jeff Garzik, (Sun Dec 10, 3:14 pm)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), Jakub Narebski, (Sun Dec 10, 3:59 pm)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), Martin Langhoff, (Sun Dec 10, 7:16 pm)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), Jakub Narebski, (Mon Dec 11, 1:59 am)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), Martin Langhoff, (Mon Dec 11, 3:18 am)
Re: kernel.org mirroring (Re: [GIT PULL] MMC update), Jakub Narebski, (Tue Dec 12, 2:19 pm)