Re: [PATCH 2/2 v4] Git.pm: add test suite

To: <git@...>

Cc: Lea Wiemann <LeWiemann@...>

Subject: [PATCH 1/2 v2] t/test-lib.sh: add test_external and test_external_without_stderr

Date: Tuesday, June 17, 2008 - 2:59 am

This is for running external test scripts in other programming

languages that provide continuous output about their tests.  Using

test_expect_success (like "test_expect_success 'description' 'perl

test-script.pl'") doesn't suffice here because test_expect_success

eats stdout in non-verbose mode, which is not fixable without major

file descriptor trickery.
Signed-off-by: Lea Wiemann <LeWiemann@gmail.com>

---

Changed since v1

<http://article.gmane.org/gmane.comp.version-control.git/83415>: Fixed

whitespace.
Both functions have been tested with real perl tests and are working

fine.
These two patches apply on master.
 t/test-lib.sh |   58 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++

 1 files changed, 58 insertions(+), 0 deletions(-)
diff --git a/t/test-lib.sh b/t/test-lib.sh

index c861141..b46fe68 100644

--- a/t/test-lib.sh

+++ b/t/test-lib.sh

@@ -302,6 +302,64 @@ test_expect_code () {

 	echo >&3 ""

 }
+# test_external runs external test scripts that provide continuous

+# test output about their progress, and succeeds/fails on

+# zero/non-zero exit code.  It outputs the test output on stdout even

+# in non-verbose mode, and announces the external script with "* run

+# <n>: ..." before running it.  When providing relative paths, keep in

+# mind that all scripts run in "trash directory".

+# Usage: test_external description command arguments...

+# Example: test_external 'Perl API' perl ../path/to/test.pl

+test_external () {

+	test "$#" -eq 3 ||

+	error >&5 "bug in the test script: not 3 parameters to test_external"

+	descr="$1"

+	shift

+	if ! test_skip "$descr" "$@"

+	then

+		# Announce the script to reduce confusion about the

+		# test output that follows.

+		say_color "" " run $(expr "$test_count" + 1): $descr ($*)"

+		# Run command; redirect its stderr to &4 as in

+		# test_run_, but keep its stdout on our stdout even in

+		# non-verbose mode.

+		"$@" 2>&4

+		if [ "$?" = 0 ]

+		then

+			test_ok_ "$descr...
[ message continues ]

Subject: Re: [PATCH 1/2 v2] t/test-lib.sh: add test_external and test_external_without_stderr

From: Olivier Marin <dkr+ml.git@...>

To: Lea Wiemann <lewiemann@...>

Cc: <git@...>

Date: Wednesday, June 18, 2008 - 4:24 pm

Hi,

Just a typo here: s/eror/error/

Olivier.
--

To: <git@...>

Cc: Lea Wiemann <LeWiemann@...>

Subject: [PATCH 1/2 v3] t/test-lib.sh: add test_external and test_external_without_stderr

Date: Thursday, June 19, 2008 - 2:18 pm

This is for running external test scripts in other programming

languages that provide continuous output about their tests.  Using

test_expect_success (like "test_expect_success 'description' 'perl

test-script.pl'") doesn't suffice here because test_expect_success

eats stdout in non-verbose mode, which is not fixable without major

file descriptor trickery.
Signed-off-by: Lea Wiemann <LeWiemann@gmail.com>

---
Thanks for spotting this, and also the missing ampersand in the other

patch!
(This typo is the only change since v2.)
 t/test-lib.sh |   58 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++

 1 files changed, 58 insertions(+), 0 deletions(-)
diff --git a/t/test-lib.sh b/t/test-lib.sh

index 3ac8755..dc2736e 100644

--- a/t/test-lib.sh

+++ b/t/test-lib.sh

@@ -304,6 +304,64 @@ test_expect_code () {

 	echo >&3 ""

 }
+# test_external runs external test scripts that provide continuous

+# test output about their progress, and succeeds/fails on

+# zero/non-zero exit code.  It outputs the test output on stdout even

+# in non-verbose mode, and announces the external script with "* run

+# <n>: ..." before running it.  When providing relative paths, keep in

+# mind that all scripts run in "trash directory".

+# Usage: test_external description command arguments...

+# Example: test_external 'Perl API' perl ../path/to/test.pl

+test_external () {

+	test "$#" -eq 3 ||

+	error >&5 "bug in the test script: not 3 parameters to test_external"

+	descr="$1"

+	shift

+	if ! test_skip "$descr" "$@"

+	then

+		# Announce the script to reduce confusion about the

+		# test output that follows.

+		say_color "" " run $(expr "$test_count" + 1): $descr ($*)"

+		# Run command; redirect its stderr to &4 as in

+		# test_run_, but keep its stdout on our stdout even in

+		# non-verbose mode.

+		"$@" 2>&4

+		if [ "$?" = 0 ]

+		then

+			test_ok_ "$descr"

+		else

+			test_failure_ "$descr" "$@"

+		fi

+	fi

+}

+

+# Like test_external, but in additi...
[ message continues ]

To: <git@...>

Cc: Lea Wiemann <LeWiemann@...>

Subject: [PATCH 2/2 v5] Git.pm: add test suite

Date: Thursday, June 19, 2008 - 2:18 pm

Add a shell script (t/t9700-perl-git.sh) that sets up a git repository

and a perl script (t/t9700/test.pl) that runs the actual tests.
Signed-off-by: Lea Wiemann <LeWiemann@gmail.com>

---

Changes since v4:
- Added missing ampersand (thanks Olivier!).

- Use 5.006002 (lowest possible version for Test::More).

- Use File::Temp instead of the external IO::String.
Tested with Perl 5.6, 5.8, 5.10.  Diff against v4 of this patch:
index 592d79a..b2fb9ec 100755

--- a/t/t9700-perl-git.sh

+++ b/t/t9700-perl-git.sh

@@ -25,7 +25,7 @@ test_expect_success \

      git-config --add color.test.slot1 green &&

      git-config --add test.string value &&

      git-config --add test.dupstring value1 &&

-     git-config --add test.dupstring value2 &

+     git-config --add test.dupstring value2 &&

      git-config --add test.booltrue true &&

      git-config --add test.boolfalse no &&

      git-config --add test.boolother other &&

diff --git a/t/t9700/test.pl b/t/t9700/test.pl

index 8318fec..4d23125 100755

--- a/t/t9700/test.pl

+++ b/t/t9700/test.pl

@@ -1,6 +1,7 @@

 #!/usr/bin/perl

 use lib (split(/:/, $ENV{GITPERLLIB}));
+use 5.006002;  # Test::More was introduced in 5.6.2

 use warnings;

 use strict;
@@ -9,7 +10,6 @@ use Test::More qw(no_plan);

 use Cwd;

 use File::Basename;

 use File::Temp;

-use IO::String;
 BEGIN { use_ok('Git') }
@@ -69,20 +69,21 @@ is($r->ident_person("Name", "email", "123 +0000"), "Name <email>",
 # objects and hashes

 ok(our $file1hash = $r->command_oneline('rev-parse', "HEAD:file1"), "(get file hash)");

-our $iostring = IO::String->new;

-is($r->cat_blob($file1hash, $iostring), 15, "cat_blob: size");

-is(${$iostring->string_ref}, "changed file 1\n", "cat_blob: data");

-our $tmpfile = File::Temp->new();

-print $tmpfile ${$iostring->string_ref};

+our $tmpfile = File::Temp->new;

+is($r->cat_blob($file1hash, $tmpfile), 15, "cat_blob: size");

+our $blob...
[ message continues ]

To: <git@...>

Cc: Lea Wiemann <LeWiemann@...>

Subject: [PATCH 2/2 v5] Git.pm: add test suite

Date: Thursday, June 19, 2008 - 4:32 pm

Add a shell script (t/t9700-perl-git.sh) that sets up a git repository

and a perl script (t/t9700/test.pl) that runs the actual tests.
Signed-off-by: Lea Wiemann <LeWiemann@gmail.com>

---

[Resent with fixed diff to v4 so git-am doesn't get confused. ;-)]
Changes since v4:
- Added missing ampersand (thanks Olivier!).

- Use 5.006002 (lowest possible version for Test::More).

- Use File::Temp instead of the external IO::String.
Tested with Perl 5.6, 5.8, 5.10.  Diff against v4 of this patch:
    index 592d79a..b2fb9ec 100755

    --- a/t/t9700-perl-git.sh

    +++ b/t/t9700-perl-git.sh

    @@ -25,7 +25,7 @@ test_expect_success \

          git-config --add color.test.slot1 green &&

          git-config --add test.string value &&

          git-config --add test.dupstring value1 &&

    -     git-config --add test.dupstring value2 &

    +     git-config --add test.dupstring value2 &&

          git-config --add test.booltrue true &&

          git-config --add test.boolfalse no &&

          git-config --add test.boolother other &&

    diff --git a/t/t9700/test.pl b/t/t9700/test.pl

    index 8318fec..4d23125 100755

    --- a/t/t9700/test.pl

    +++ b/t/t9700/test.pl

    @@ -1,6 +1,7 @@

     #!/usr/bin/perl

     use lib (split(/:/, $ENV{GITPERLLIB}));
    +use 5.006002;  # Test::More was introduced in 5.6.2

     use warnings;

     use strict;
    @@ -9,7 +10,6 @@ use Test::More qw(no_plan);

     use Cwd;

     use File::Basename;

     use File::Temp;

    -use IO::String;
     BEGIN { use_ok('Git') }
    @@ -69,20 +69,21 @@ is($r->ident_person("Name", "email", "123 +0000"), "Name <email>",
     # objects and hashes

     ok(our $file1hash = $r->command_oneline('rev-parse', "HEAD:file1"), "(get file hash)");

    -our $iostring = IO::String->new;

    -is($r->cat_blob($file1hash, $iostring), 15, "cat_blob: size");

    -is(${$iostring->string_ref}, "changed file 1\n", "cat_blob: data");...
[ message continues ]

From: Jakub Narebski <jnareb@...>

To: Lea Wiemann <lewiemann@...>

Cc: Lea Wiemann <LeWiemann@...>, <git@...>

Subject: Re: [PATCH 2/2 v5] Git.pm: add test suite

Date: Thursday, June 19, 2008 - 4:53 pm

Isn't "use Test::More" enough, so this line is not strictly
necessary?

--
Jakub Narebski
Poland
ShadeHawk on #git
--

To: Jakub Narebski <jnareb@...>

Cc: <git@...>

Subject: Re: [PATCH 2/2 v5] Git.pm: add test suite

Date: Thursday, June 19, 2008 - 4:55 pm

I'd prefer to have a 'wrong Perl version' error instead of 'module not
found'. Also it's good to document what we support, to remind me that I
have to run it with perl5.6.

-- Lea
--

To: <git@...>

Cc: Lea Wiemann <LeWiemann@...>

Subject: [PATCH 2/2 v4] Git.pm: add test suite

Date: Tuesday, June 17, 2008 - 2:59 am

Add a shell script (t/t9700-perl-git.sh) that sets up a git repository

and a perl script (t/t9700/test.pl) that runs the actual tests.
Signed-off-by: Lea Wiemann <LeWiemann@gmail.com>

---

Changed since v3

<http://thread.gmane.org/gmane.comp.version-control.git/83425/focus=83480>:

Only test the vanilla Git.pm, as I'm not intending to extend it right

now.  So no Git::get_hash (or Git::rev_parse) method there.
I also simplified the test repository setup code a bit.
 t/t9700-perl-git.sh |   39 ++++++++++++++++++++

 t/t9700/test.pl     |   99 +++++++++++++++++++++++++++++++++++++++++++++++++++

 2 files changed, 138 insertions(+), 0 deletions(-)

 create mode 100755 t/t9700-perl-git.sh

 create mode 100755 t/t9700/test.pl
diff --git a/t/t9700-perl-git.sh b/t/t9700-perl-git.sh

new file mode 100755

index 0000000..592d79a

--- /dev/null

+++ b/t/t9700-perl-git.sh

@@ -0,0 +1,39 @@

+#!/bin/sh

+#

+# Copyright (c) 2008 Lea Wiemann

+#

+

+test_description='perl interface (Git.pm)'

+. ./test-lib.sh

+

+# set up test repository

+

+test_expect_success \

+    'set up test repository' \

+    'echo "test file 1" > file1 &&

+     echo "test file 2" > file2 &&

+     mkdir directory1 &&

+     echo "in directory1" >> directory1/file &&

+     mkdir directory2 &&

+     echo "in directory2" >> directory2/file &&

+     git add . &&

+     git commit -m "first commit" &&

+

+     echo "changed file 1" > file1 &&

+     git commit -a -m "second commit" &&

+

+     git-config --add color.test.slot1 green &&

+     git-config --add test.string value &&

+     git-config --add test.dupstring value1 &&

+     git-config --add test.dupstring value2 &

+     git-config --add test.booltrue true &&

+     git-config --add test.boolfalse no &&

+     git-config --add test.boolother other &&

+     git-config --add test.int 2k

+     '

+

+test_ext...
[ message continues ]

From: Olivier Marin <dkr+ml.git@...>

To: Lea Wiemann <lewiemann@...>

Cc: <git@...>

Date: Wednesday, June 18, 2008 - 4:32 pm

While I was running the whole test suite, your test cases randomly
failed. I did not understand until I see the missing & here. ;-)

Olivier.

From: Junio C Hamano <gitster@...>

To: Lea Wiemann <lewiemann@...>

Cc: <git@...>

Date: Tuesday, June 17, 2008 - 11:16 pm

The system I first tried this did not have IO::String installed. Is this
something we would want to rely on being available?

It also indicates a slight problem in test_external. Can it become a bit
easier to tell such a basic problem apart from real test errors? I needed
to look into /tmp/* to see where it is failing but it would be nice if we
can somehow see the breakage with "sh t9700-*.sh -v".

To: Junio C Hamano <gitster@...>

Cc: <git@...>

Date: Wednesday, June 18, 2008 - 2:04 pm

My bad, I should've checked this before sending it off. I guess we'll
solve this in Michael's patch thread; don't expect a reply from me in

Hm, is your test output somehow different from the following? It states
the error quite clearly at the bottom.

$ ./t9700-perl-git.sh -v
* expecting success: echo "test file 1" > file1 &&
echo "test file 2" > file2 &&
[...]
git-config --add test.int 2k

* ok 1: set up test repository
[...]
* run 2: Perl API (perl ../t9700/test.pl)
* FAIL 2: Perl API
perl ../t9700/test.pl
* expecting no stderr from previous command
* FAIL 3: no stderr: Perl API
perl ../t9700/test.pl
Stderr is:
Can't locate IO/String.pm in @INC (@INC contains:
/home/lea/G/t/../perl/blib/lib /home/lea/G/t/../perl/blib/arch/auto/Git
/home/lea/.perl/lib /etc/perl /usr/local/lib/perl/5.10.0
/usr/local/share/perl/5.10.0 /usr/lib/perl5 /usr/share/perl5
/usr/lib/perl/5.10 /usr/share/perl/5.10 /usr/local/lib/site_perl .) at
../t9700/test.pl line 12.
BEGIN failed--compilation aborted at ../t9700/test.pl line 12.
# Looks like your test died before it could output anything.
* failed 2 among 3 test(s)

-- Lea
--

From: Junio C Hamano <gitster@...>

To: Lea Wiemann <lewiemann@...>

Cc: <git@...>