git clone questions relating to cpio

Previous thread: what's a useful definition of full text index on a repository? by David Tweed on Monday, October 1, 2007 - 12:33 pm. (4 messages)

Next thread: Problems setting up bare repository (git 1.5.3.3) by Barry Fishman on Monday, October 1, 2007 - 5:46 pm. (15 messages)
To: Git <git@...>
Date: Monday, October 1, 2007 - 3:28 pm

Hi,

I am running a Linux From Scratch 6.2 system that does not have cpio
installed on it. This means that I can't clone a local repository
unless I install cpio. Is it possible to use a fallback method if cpio
is not present, as there is no NO_CPIO option on make like there is
for OpenSSH, cURL and expat?

Also, I have an external USB hardrive that is mounted onto the virtual
filesystem. Will clones from the USB harddrive (or a USB flash drive
that is mounted) result in a copy being performed, not a hardlink?

Ideally, the hard linking for local clones should be optional. What if
I want to move a repository because, for example, I have imported a
CVS repository and now want to push it to a new bare repository?

- Reece
-

To: Reece Dunn <msclrhd@...>
Cc: Git <git@...>
Date: Monday, October 1, 2007 - 7:23 pm

Hi,

You might be interested in the workaround Hannes did in mingw.git; he made
a wrapper script called 'cpio' using 'tar'.

Ciao,
Dscho

-

To: Johannes Schindelin <Johannes.Schindelin@...>
Cc: Reece Dunn <msclrhd@...>, Git <git@...>
Date: Monday, October 1, 2007 - 7:32 pm

I think that may be good enough as workaround, but I do not
think you would get the space saving from hardlinks that way.
-

To: Junio C Hamano <gitster@...>
Cc: Johannes Schindelin <Johannes.Schindelin@...>, Reece Dunn <msclrhd@...>, Git <git@...>
Date: Tuesday, October 2, 2007 - 2:11 am

FWIW, I'm thinking about changing the cpio -p (passthrough) part to use
'cp -l --parents --target-directory=...' instead of tar; this gives us hard
links, even on NTFS. But it needs GNU's cp, of course.

-- Hannes

-

To: Johannes Sixt <j.sixt@...>
Cc: Johannes Schindelin <Johannes.Schindelin@...>, Reece Dunn <msclrhd@...>, Git <git@...>
Date: Tuesday, October 2, 2007 - 2:23 am

Yeah, that's the reason it is not appropriate than cpio, even if
we forget about Windows.
-

To: <git@...>
Cc: Reece Dunn <msclrhd@...>
Date: Monday, October 1, 2007 - 5:42 pm

Using "file://" when specifying the source repo will force git-clone to use
the git protocol, instead of doing a copy/hardlink.

I.e. change "git clone foo bar" to "git clone file://foo bar" in order to
prevent git-clone from calling cpio.

However, grepping for cpio in the git source tree reveals a couple of uses

Hardlinks are impossible across filesystems. If you're cloning to a

<quote src="git-clone(1)">
--local, -l

When the repository to clone from is on a local machine, this flag
bypasses normal "git aware" transport mechanism and clones the repository
by making a copy of HEAD and everything under objects and refs directories.
The files under .git/objects/ directory are hardlinked to save space when
possible. This is now the default when the source repository is specified
with /path/to/repo syntax, so it essentially is a no-op option. To force
copying instead of hardlinking (which may be desirable if you are trying to
make a back-up of your repository), but still avoid the usual "git aware"
transport mechanism, --no-hardlinks can be used.

--no-hardlinks

Optimize the cloning process from a repository on a local filesystem by
copying files under .git/objects directory.
</quote>

And as I said above, you can use "file://" to force the "git aware"
transport mechanism, which bypasses the whole local copy/hardlink issue

Even if you were to use hardlinks, cloning a repo followed by deleting the
original will be safe (as long as you don't supply '--shared' to
git-clone). That's the beauty of hardlinks.

I also think it's fairly safe to just 'mv' the whole repository to its new
location.

Have fun! :)

...Johan

--
Johan Herland, <johan@herland.net>
www.herland.net
-

To: Johan Herland <johan@...>
Cc: <git@...>
Date: Tuesday, October 2, 2007 - 5:09 pm

Looks like I'll need to install cpio, then (also allowing me to take
advantage of the fast local clones via hardlinks).

Exactly. I was asking this to clarify cpio (and therefore git)

This also works, as long as you are not moving between a bare and

Will do :)

Thanks,
- Reece
-

To: <git@...>
Cc: Reece Dunn <msclrhd@...>, Junio C Hamano <gitster@...>
Date: Tuesday, October 2, 2007 - 7:42 pm

Signed-off-by: Johan Herland <johan@herland.net>
---

Doing the following in the git repo:
$ grep -r cpio *
git-clone.sh: find objects -depth -print | cpio -pumd$l "$GIT_DIR/" || exit 1
git-merge.sh: cpio -0 -o >"$GIT_DIR/MERGE_SAVE"
git-merge.sh: cpio -iuv <"$GIT_DIR/MERGE_SAVE"

reveals that cpio is not mentioned anywhere in the documentation,
nor in the requirements section of the INSTALL file.

Have fun!

...Johan

 INSTALL |    3 +++
 1 files changed, 3 insertions(+), 0 deletions(-)

diff --git a/INSTALL b/INSTALL
index 289b046..f1eb404 100644
--- a/INSTALL
+++ b/INSTALL
@@ -79,6 +79,9 @@ Issues of note:
  - "perl" and POSIX-compliant shells are needed to use most of
   the barebone Porcelainish scripts.
 
+ - "cpio" is used by git-merge for saving and restoring the index,
+  and by git-clone when doing a local (possibly hardlinked) clone.
+
  - Some platform specific issues are dealt with Makefile rules,
    but depending on your specific installation, you may not
    have all the libraries/tools needed, or you may have
--
1.5.3.3.1144.gf10f2
-

To: Johan Herland <johan@...>
Cc: Reece Dunn <msclrhd@...>, <git@...>
Date: Tuesday, October 2, 2007 - 8:14 pm

Thanks.

We use many other tools that are typically found in bog-standard
UNIX environments, like sed, echo, cat, sort, etc. and we do not
list them in the INSTALL file (nor we would want to). cpio used
to be in the "bog standard" category but perhaps Linux distros
do not install it by default, so it is worth listing it there.

Are there other commands we rely on that may not be universally
installed? I myself consider "cut" to be in the category, but
other than that I do not think of anything offhand.

-

To: Junio C Hamano <gitster@...>
Cc: Reece Dunn <msclrhd@...>, Johan Herland <johan@...>, <git@...>
Date: Wednesday, October 3, 2007 - 3:40 am

when using git in a chroot, i obviously had coreutils/sed/grep installed
and the only "extra" package i needed (besides the curl an openssl libs)
was cpio

- VMiklos

To: <git@...>
Cc: Miklos Vajna <vmiklos@...>, Reece Dunn <msclrhd@...>, Junio C Hamano <gitster@...>
Date: Wednesday, October 3, 2007 - 4:27 am

Includes:
- Mention dependency on "core" utilities, including coreutils, sed, cut, grep
- Mention dependency on cpio
- Fix up some whitespace and linebreaking issues

Signed-off-by: Johan Herland <johan@herland.net>
---

Ok, here's a more complete patch.

...Johan

INSTALL | 18 +++++++++++-------
1 files changed, 11 insertions(+), 7 deletions(-)

diff --git a/INSTALL b/INSTALL
index 289b046..244470f 100644
--- a/INSTALL
+++ b/INSTALL
@@ -54,6 +54,8 @@ Issues of note:
- Git is reasonably self-sufficient, but does depend on a few external
programs and libraries:

+ - Common "core" utilities including coreutils, sed, cut, and grep.
+
- "zlib", the compression library. Git won't build without it.

- "openssl". Unless you specify otherwise, you'll get the SHA1
@@ -63,22 +65,24 @@ Issues of note:
that come with git (git includes the one from Mozilla, and has
its own PowerPC and ARM optimized ones too - see the Makefile).

- - "libcurl" and "curl" executable. git-http-fetch and
- git-fetch use them. If you do not use http
- transfer, you are probably OK if you do not have
- them.
+ - "libcurl" and "curl" executable. git-http-fetch and git-fetch
+ use them. If you do not use http transfer, you are probably OK
+ if you do not have them.

- expat library; git-http-push uses it for remote lock
management over DAV. Similar to "curl" above, this is optional.

- - "wish", the Tcl/Tk windowing shell is used in gitk to show the
- history graphically, and in git-gui.
+ - "wish", the Tcl/Tk windowing shell is used in gitk to show the
+ history graphically, and in git-gui.

- - "ssh" is used to push and pull over the net
+ - "ssh" is used to push and pull over the net.

- "perl" and POSIX-compliant shells are needed to use most of
the barebone Porcelainish scripts.

+ - "cpio" is used by git-merge for saving and restoring the index,
+ and by git-clone when doing a local (possibly hardlinked)...

To: Johan Herland <johan@...>
Cc: Miklos Vajna <vmiklos@...>, Reece Dunn <msclrhd@...>, Junio C Hamano <gitster@...>, <git@...>
Date: Wednesday, October 3, 2007 - 11:56 am

Hi,

The commit message could use some of that, too... ;-)

Ciao,
Dscho

-

To: Johannes Schindelin <Johannes.Schindelin@...>
Cc: Miklos Vajna <vmiklos@...>, Reece Dunn <msclrhd@...>, Johan Herland <johan@...>, Junio C Hamano <gitster@...>, <git@...>
Date: Wednesday, October 3, 2007 - 12:05 pm

Don't forget that the computer must be plugged in and have
electricity as well. I see users all too often wonder why everything
they type is black, on a black background...

--
Shawn.
-

To: <git@...>
Date: Wednesday, October 3, 2007 - 2:09 am

I'd think it might be a good idea to list any utilities used that
aren't in, say, the Single Unix Specification, for example.
--
Chris Larson - clarson at kergoth dot com
Software Engineer - MontaVista - clarson at mvista dot com
Core Developer/Architect - TSLib, BitBake, OpenEmbedded, OpenZaurus
-

Previous thread: what's a useful definition of full text index on a repository? by David Tweed on Monday, October 1, 2007 - 12:33 pm. (4 messages)

Next thread: Problems setting up bare repository (git 1.5.3.3) by Barry Fishman on Monday, October 1, 2007 - 5:46 pm. (15 messages)