Re: Today's 'master' leaves .idx/.pack in 0400

Previous thread: Re: If not readdir() then what? by Neil Brown on Wednesday, April 11, 2007 - 9:46 pm. (2 messages)

Next thread: none
To: <git@...>
Cc: <linux-kernel@...>
Date: Wednesday, April 11, 2007 - 10:09 pm

The latest maintenance release GIT 1.5.1.1 is available at the
usual places:

http://www.kernel.org/pub/software/scm/git/

git-1.5.1.1.tar.{gz,bz2} (tarball)
git-htmldocs-1.5.1.1.tar.{gz,bz2} (preformatted docs)
git-manpages-1.5.1.1.tar.{gz,bz2} (preformatted docs)
RPMS/$arch/git-*-1.5.1.1-1.$arch.rpm (RPM)

GIT v1.5.1.1 Release Notes
==========================

Fixes since v1.5.1
------------------

* Documentation updates

- The --left-right option of rev-list and friends is documented.

- The documentation for cvsimport has been majorly improved.

- "git-show-ref --exclude-existing" was documented.

* Bugfixes

- The implementation of -p option in "git cvsexportcommit" had
the meaning of -C (context reduction) option wrong, and
loosened the context requirements when it was told to be
strict.

- "git cvsserver" did not behave like the real cvsserver when
client side removed a file from the working tree without
doing anything else on the path. In such a case, it should
restore it from the checked out revision.

- "git fsck" issued an alarming error message on detached
HEAD. It is not an error since at least 1.5.0.

- "git send-email" produced of References header of unbounded length;
fixed this with line-folding.

- "git archive" to download from remote site should not
require you to be in a git repository, but it incorrectly
did.

- "git apply" ignored -p<n> for "diff --git" formatted
patches.

- "git rerere" recorded a conflict that had one side empty
(the other side adds) incorrectly; this made merging in the
other direction fail to use previously recorded resolution.

- t4200 test was broken where "wc -l" pads its output with
spaces.

- "git branch -m old new" to rename branch did not work
without a configuration file in ".git/config".

- The sample hook for notification e-mail was misnamed.

- gitweb did not show type-changing patch correc...

To: <git@...>
Cc: <linux-kernel@...>
Date: Sunday, April 22, 2007 - 2:16 am

The latest maintenance release GIT 1.5.1.2 is available at the
usual places:

http://www.kernel.org/pub/software/scm/git/

git-1.5.1.2.tar.{gz,bz2} (tarball)
git-htmldocs-1.5.1.2.tar.{gz,bz2} (preformatted docs)
git-manpages-1.5.1.2.tar.{gz,bz2} (preformatted docs)
RPMS/$arch/git-*-1.5.1.2-1.$arch.rpm (RPM)

GIT v1.5.1.2 Release Notes
==========================

Fixes since v1.5.1.1
--------------------

* Bugfixes

- "git clone" over http from a repository that has lost the
loose refs by running "git pack-refs" were broken (a code to
deal with this was added to "git fetch" in v1.5.0, but it
was missing from "git clone").

- "git diff a/ b/" incorrectly fell in "diff between two
filesystem objects" codepath, when the user most likely
wanted to limit the extent of output to two tracked
directories.

- git-quiltimport had the same bug as we fixed for
git-applymbox in v1.5.1.1 -- it gave an alarming "did not
have any patch" message (but did not actually fail and was
harmless).

- various git-svn fixes.

- Sample update hook incorrectly always refused requests to
delete branches through push.

- git-blame on a very long working tree path had buffer
overrun problem.

- git-apply did not like to be fed two patches in a row that created
and then modified the same file.

- git-svn was confused when a non-project was stored directly under
trunk/, branches/ and tags/.

- git-svn wants the Error.pm module that was at least as new
as what we ship as part of git; install ours in our private
installation location if the one on the system is older.

- An earlier update to command line integer parameter parser was
botched and made 'update-index --cacheinfo' completely useless.

* Documentation updates

- Various documentation updates from J. Bruce Fields, Frank
Lichtenheld, Alex Riesen and others. Andrew Ruder started a
war on undocumented options.

---------...

To: Junio C Hamano <junkio@...>
Cc: <git@...>, <linux-kernel@...>
Date: Sunday, April 22, 2007 - 1:22 pm

Well, by "available" you probably mean "not available", because it doesn't
actually work.

I get EPERM on pack-e00affefe0f779d0f9b0507aef25a1733f4a9117.idx/pack,
because they are

-r-------- 1 junio junio 1120880
-r-------- 1 junio junio 15709370

respectively.

As a result, nothing really works, ie:

[torvalds@hera git.git]$ git log
error: Could not read 42c4b58059fa9af65e90f2c418bb551e30d1d32f

and doing a "git pull" will just result in lots of

error: refs/heads/maint does not point to a valid object!
error: refs/heads/next does not point to a valid object!
error: refs/heads/todo does not point to a valid object!
...

which is perhaps a bit of a misleading error message (technically true,
but ..)

Oops.

Linus
-

To: Linus Torvalds <torvalds@...>
Cc: <git@...>, <linux-kernel@...>, Randal L. Schwartz <merlyn@...>, Nicolas Pitre <nico@...>
Date: Sunday, April 22, 2007 - 2:47 pm

Ok, to clarify the situation.

* This breakage does not make v1.5.1.2 is a dud release, as what
caused this is not contained in it.

* A topic to update git-pack-objects has been cooking in 'next'
and it graduated to 'master' last night after v1.5.1.2 was
cut from 'maint'. This series had a bug that left permission
bits as set by mkstemp(); newer glibc leaves it as 0600.

* I almost always run 'master' on kernel.org; I ran "git-repack
-a -d" there using 'master' that contained the breakage.

* git-repack lets git-pack-objects to create pack/idx, and then
drops the write permission bits from them. That is how they
got 0400.

* I've fixed the pack/idx 0444 manually at kernel.org; the
repository has already mirrored out to git.kernel.org, so
people should be able to fetch from there now. alt-git.git
at repo.or.cz should be usable as well, although I suspect
the site did not even have this issue.

* I sent a quickfix as a workaround. Because we rely on
git-repack to drop write permissions anyway, and some tests
to plumbing expects the resulting files to be writable by the
owner, tweaking 0444 in the quickfix to 0644 would be a good
workaround to fix the problem -and- pass all the test.

So I'll push out a 0644 quickfix on 'master' shortly.

However.

* To reiterate, people planning to update to v1.5.1.2 should
not be alarmed with this gotcha. That comes from the
maintenance branch and is not affected by this breakage.

* Letting mkstemp() to create the file and then forcing 0644 is
not strictly correct, as it makes the files created by people
with 007 umask readable to the general public. The code
before mkstemp() conversion was aware of this issue and used
open(..., 0666) to let the user's umask applied, I think.

So while the quickfix is an improvement, it not a real fix X-<.

-

To: Linus Torvalds <torvalds@...>
Cc: <git@...>, <linux-kernel@...>
Date: Sunday, April 22, 2007 - 1:58 pm

I've run fsck and then flipped the bits manually on these files
to unblock people, but I have no idea how this happened. These
two were the only files that had the funny bits in the
repository.

-r-------- 1 junio junio 1120880 Apr 22 06:46
-r-------- 1 junio junio 15709370 Apr 22 06:46

I can tell from the output of "last" on that machine that I was
present and I am reasonably sure this was from "git repack -a -d"
I manually did. But my umask is 0002...

Gaah. "git repack -a -d" from 'master' leaves packs with 0400

Oops indeed.

-

To: Linus Torvalds <torvalds@...>
Cc: <git@...>, <linux-kernel@...>
Date: Sunday, April 22, 2007 - 2:06 pm

With recent glibc, mkstemp() creates 0400 file. Updated
pack-objects uses it in pack/idx writing without fixing this,
hence this problem.

Will have a fix hopefully shortly.

-

To: Junio C Hamano <junkio@...>
Cc: Linus Torvalds <torvalds@...>, <git@...>, <linux-kernel@...>
Date: Sunday, April 22, 2007 - 2:25 pm

Oops. I guess I'm guilty for this. I didn't bother looking at the
permission on the pack for git-pack-objects since git-repack seemed to
take care of that. But it only _remove_ write permissions.

Nicolas
-

To: Nicolas Pitre <nico@...>
Cc: Linus Torvalds <torvalds@...>, <git@...>, <linux-kernel@...>
Date: Sunday, April 22, 2007 - 2:27 pm

Ok, then probably we can change the 0444 in my "quickfix" patch
to 0644. That should also let the 5300 test pass.

-

To: Junio C Hamano <junkio@...>
Cc: Linus Torvalds <torvalds@...>, <git@...>, <linux-kernel@...>
Date: Sunday, April 22, 2007 - 2:34 pm

Well, actually there is no point making pack files writable. If they're
modified, they get corrupted.

Here's the fix I wanted to propose:

diff --git a/builtin-pack-objects.c b/builtin-pack-objects.c
index c72e07a..85c6e6e 100644
--- a/builtin-pack-objects.c
+++ b/builtin-pack-objects.c
@@ -1786,11 +1786,13 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)
if (rename(pack_tmp_name, tmpname))
die("unable to rename temporary pack file: %s",
strerror(errno));
+ chmod(tmpname, 0444);
snprintf(tmpname, sizeof(tmpname), "%s-%s.idx",
base_name, sha1_to_hex(object_list_sha1));
if (rename(idx_tmp_name, tmpname))
die("unable to rename temporary index file: %s",
strerror(errno));
+ chmod(tmpname, 0444);
puts(sha1_to_hex(object_list_sha1));
}
if (progress)

Nicolas

-

To: Nicolas Pitre <nico@...>
Cc: Junio C Hamano <junkio@...>, <git@...>, <linux-kernel@...>
Date: Sunday, April 22, 2007 - 2:54 pm

I think this is wrong (as is Junio's). I think we should still honor the
repository permission setting, and default to honoring umask.

So I think that if the user has a umask that says "nobody else can read",
then we should *not* make it world readable (unless the
"shared_repository" thing is set to override it, of course).

Linus
-

To: Linus Torvalds <torvalds@...>
Cc: Nicolas Pitre <nico@...>, <git@...>, <linux-kernel@...>
Date: Sunday, April 22, 2007 - 3:03 pm

I obviously agree, but as a tentative measure, I'll push out
0644 version anyway.

-

To: Linus Torvalds <torvalds@...>
Cc: Nicolas Pitre <nico@...>, <git@...>, <linux-kernel@...>
Date: Sunday, April 22, 2007 - 3:20 pm

How about this as a replacement (hot off the press -- still
running the tests).

-- >8 --
pack-objects: adjust the permission bits of created files.

The updated pack-objects let mkstemp() to create new pack/idx
pair, without fixing the permission bits on them.

Signed-off-by: Junio C Hamano <junkio@cox.net>
---
builtin-pack-objects.c | 18 ++++++++++++++++++
1 files changed, 18 insertions(+), 0 deletions(-)

diff --git a/builtin-pack-objects.c b/builtin-pack-objects.c
index c72e07a..34350bf 100644
--- a/builtin-pack-objects.c
+++ b/builtin-pack-objects.c
@@ -1612,6 +1612,13 @@ static void get_object_list(int ac, const char **av)
traverse_commit_list(&revs, show_commit, show_object);
}

+static int adjust_perm(const char *path, mode_t mode)
+{
+ if (chmod(path, mode))
+ return -1;
+ return adjust_shared_perm(path);
+}
+
int cmd_pack_objects(int argc, const char **argv, const char *prefix)
{
int depth = 10;
@@ -1780,14 +1787,25 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)
last_obj_offset = write_pack_file();
if (!pack_to_stdout) {
unsigned char object_list_sha1[20];
+ mode_t mode = umask(0);
+
+ umask(mode);
+ mode = 0666 & ~mode;
+
write_index_file(last_obj_offset, object_list_sha1);
snprintf(tmpname, sizeof(tmpname), "%s-%s.pack",
base_name, sha1_to_hex(object_list_sha1));
+ if (adjust_perm(pack_tmp_name, mode))
+ die("unable to make temporary pack file readable: %s",
+ strerror(errno));
if (rename(pack_tmp_name, tmpname))
die("unable to rename temporary pack file: %s",
strerror(errno));
snprintf(tmpname, sizeof(tmpname), "%s-%s.idx",
base_name, sha1_to_hex(object_list_sha1));
+ if (adjust_perm(idx_tmp_name, mode))
+ die("unable to make temporary index file readable: %s",
+ strerror(errno));
if (rename(idx_tmp_name, tmpname))
die("unable to rename temporary index file: %s",
strerror(errno));

-

To: Junio C Hamano <junkio@...>
Cc: Nicolas Pitre <nico@...>, <git@...>, <linux-kernel@...>
Date: Sunday, April 22, 2007 - 3:29 pm

I would really suggest just defaulting to

mode = 0444 & ~mode;

since there simply is never any reason to allow a writable pack-file.

The fact that we have some tests that try to corrupt a pack-file is not
really a reason. Just make them do "chmod +w" before corrupting it.

But your patch is an obvious improvement regardless, so I certainly don't
think this is a *big* issue.

Linus
-

To: Junio C Hamano <junkio@...>
Cc: Linus Torvalds <torvalds@...>, <git@...>, <linux-kernel@...>
Date: Sunday, April 22, 2007 - 2:47 pm

OK there are those test cases.

Well... Either we chmod to 0644, or we fix the tests to 'chmod +w' like
it is already done in t5302.

In any case I wouldn't die() but only error() on a failure to chmod().
It is sure inconvenient if the pack isn't world readable, but it is not
a "fatal" problem for the repack.

Nicolas
-

To: Nicolas Pitre <nico@...>
Cc: Linus Torvalds <torvalds@...>, <git@...>, <linux-kernel@...>
Date: Sunday, April 22, 2007 - 2:52 pm

If it DIED, I would not have to have been embarrassed in public
like this!

Grumble.

-

Previous thread: Re: If not readdir() then what? by Neil Brown on Wednesday, April 11, 2007 - 9:46 pm. (2 messages)

Next thread: none