Hello file system developers,

There are several POSIX file system test suites: closed source, commercial, 
one which needs reading 174 pages installation guide, etc. Because of these 
frustrations when Pawel Jakub Dawidek ported ZFS to FreeBSD, he also wrote 
such a test suite quickly.

Last year the NTFS-3G team ported it to Linux/ext3 and Linux/NTFS-3G to 
validate Jean-Pierre Andre's full file permissions and ownership support 
for NTFS-3G. We sent our patches to Pawel for integration but this doesn't 
seem to happen him (he didn't see problems but is busy).

Since this topic regularly appears on several lists, we are also often 
asked about it and NTFS-3G does need it to be maintained, hence we decided 
to release it and if nobody else would like to maintain it then we will do 
so.

The test suite mostly checks POSIX compliance and works for FreeBSD, 
Solaris, and Linux with UFS, ZFS, ext3, and NTFS-3G file systems. The list 
of system calls tested is: chmod, chown, link, mkdir, mkfifo, open, rename, 
rmdir, symlink, truncate, unlink. There are currently 1950 regression 
tests.

Availability:

	http://ntfs3g.org/sw/qa/pjd-fstest-20080402.tgz

  and in the NTFS-3G CVS as pjd-fstest module:

	http://sourceforge.net/cvs/?group_id=181143

The usage is extremely simple:

  # tar czf pjd-fstest-20080402.tgz
  # cd pjd-fstest-20080402
  # vi tests/conf
  Change 'fs' to file system type you want to test (UFS, ZFS, ext3, ntfs-3g).
  # make
  It will compile fstest utility which is used by regression tests.
  # cd /path/to/file/system/you/want/to/test/
  The test must be run as root user and requires a few basic Perl modules.
  # prove -r /path/to/fstest/

It's also possible to run individual set of tests:

  # /path/to/fstest/tests/chown/00.t

Or make single system call tests:

  # fstest mkdir foo 0750
  0
  # fstest mkdir foo 0750
  mkdir returned -1
  EEXIST

The test suite is easy to understand, modify and extend. For instance doing 
a test cases for the a...
[ message continues ]

Hi,

Thanks for the release. 

I have a general question regarding FUSE. Why do all the tests pass on a
regular ext3 file system while there are quite a lot of errors (~80%
success) when run on top of the fuse example file system as found in the
fuse tarball? I thought the fuse filesystem just passes every access to 
the underlying file system. So what's causing the failures? Or am i doing
something wrong?

Thanks,
Marc

-- 
 Marc Andre Tanner >< http://www.brain-dump.org/ >< GPG key: CF7D56C0
--

The current xfs-dev tree:

Failed Test                    Stat Wstat Total Fail  Failed  List of Failed
-------------------------------------------------------------------------------
/root/posix/tests/chown/00.t                171    2   1.17%  84 88
/root/posix/tests/symlink/02.t                7    2  28.57%  6-7
Failed 2/184 test scripts, 98.91% okay. 4/1950 subtests failed, 99.79% okay.

Cheers,

Dave.
-- 
Dave Chinner
Principal Engineer
SGI Australian Software Group
--

Symlink tests 6 and 7:

expect 0 symlink ${name256} ${n0}
expect 0 unlink ${n0}

Test 6 is failing with ENAMETOOLONG
Test 7 is failing (correctly) with ENOENT because test 6 failed.

So there's only one failure here, and that is that that we're rejecting
${name256} as too long. I think that getname() is doing this. Seems sane
to me to disallow symlinking to pathnames that can't be constructed,
even if POSIX apparently allows it.


Chown tests 84 and 88:

Test 84 appears to be checking the result of test 83:

expect 0 -u 65534 -g 65533,65532 chown ${n0} 65534 65532
case "${os}" in
Linux)
        expect 06555,65534,65532 lstat ${n0} mode,uid,gid
        ;;
*)
        expect 0555,65534,65532 lstat ${n0} mode,uid,gid
        ;;
esac

And running these manually I get:

# /root/posix/fstest -u 65534 -g 65533,65532 chown z 65534 65532
changing groups to 65533,65532
changing uid to 65534
0
# /root/posix/fstest lstat z mode,uid,gid
0555,65534,65532

Which matches the "non-Linux" output. Looks like bits 06000 are
the set-uid and set-gid bits. Ok, Posix says:

"If the chown() function is successfully invoked on a file that is
not a regular file and one or more of the S_IXUSR, S_IXGRP, or
S_IXOTH bits of the file mode are set, the set-user-ID and
set-group-ID bits may be cleared."

So, either result is valid. Hence i suggest that test 84 and test 88
(same failure) are special cased to "ext3" behaviour.

That means XFS is not failing any tests at all.

Cheers,

Dave.
-- 
Dave Chinner
Principal Engineer
SGI Australian Software Group
--

Openssl is replaced with md5sum+cut in the CVS. 




I added the xfs target but left the symlink Test 6 fail because POSIX says

 "The string pointed to by path1 shall be treated only as a character 
  string and shall not be validated as a pathname" 

and 
 
 "the length of the path1 argument is longer than {SYMLINK_MAX}" 

where {SYMLINK_MAX} is typically not defined on Linux or it's {PATH_MAX}.

	Szaka

--
NTFS-3G:  http://ntfs-3g.org

--

i'd rather expect this to be the component validation in xfs_symlink.
It's superflous and not done by any other fiesystem.

--

Ah yes, you are right - PATH_MAX != NAME_MAX...

Cheers,

Dave.
-- 
Dave Chinner
Principal Engineer
SGI Australian Software Group
--

Hmm.. I ran it against ext2, ext3, jfs, btrfs. I don't see all "pass"
on ext3. What am I missing ?

btrfs seems to have little more failures.

Thanks,
Badari

ext2:
====

Failed Test                     Stat Wstat Total Fail  Failed  List of Failed
-------------------------------------------------------------------------------
/root/posix/tests/chmod/00.t                  58    2   3.45%  3 19
/root/posix/tests/chown/00.t                 171    4   2.34%  141 145 149 153
/root/posix/tests/link/00.t                   82    6   7.32%  3 5-6 8-10
/root/posix/tests/open/05.t                   12    2  16.67%  5 9
/root/posix/tests/rename/00.t                 79    9  11.39%  3 6 8-9 11 13 37
                                                               39 42
/root/posix/tests/symlink/00.t                14    2  14.29%  2 5
/root/posix/tests/truncate/05.t               15    5  33.33%  5-6 10-12
/root/posix/tests/truncate/12.t                3    1  33.33%  2
/root/posix/tests/truncate/13.t                4    2  50.00%  2-3
Failed 9/184 test scripts, 95.11% okay. 33/1950 subtests failed, 98.31% okay.


ext3:
====

Failed Test                     Stat Wstat Total Fail  Failed  List of Failed
-------------------------------------------------------------------------------
/root/posix/tests/chmod/00.t                  58    2   3.45%  3 19
/root/posix/tests/link/00.t                   82    6   7.32%  3 5-6 8-10
/root/posix/tests/open/05.t                   12    2  16.67%  5 9
/root/posix/tests/rename/00.t                 79    9  11.39%  3 6 8-9 11 13 37
                                                               39 42
/root/posix/tests/symlink/00.t                14    2  14.29%  2 5
/root/posix/tests/truncate/05.t               15    5  33.33%  5-6 10-12
/root/posix/tests/truncate/12.t                3    1  33.33%  2
/root/posix/tests/truncate/13.t                4    2  50.00%  2-3
Failed 8/184 test scripts, 95.65% okay. 29/1950 subtests failed, 98.51% okay.


jfs:
...
[ message continues ]

Your unique, consistently failing test cases for all file systems suggest 
that you have a buggy private kernel or some other individual issue in your 
test environment.

You could use the -x debug shell option in the test scripts, rerun the 

If you find the reason for the unexpected failures then the btrfs result 
will be quite good. Apparently it has only a few link, truncate, and unlink 
ctimes update problems. I think that's quite impressive in its state of 
development.


--
NTFS-3G:  http://ntfs-3g.org


--

I am running 2.6.25-rc8-mm1. What I noticed is, these failures happen

Yes. Indeed. btrfs has only these tests failing now (x86-64).


Failed Test                     Stat Wstat Total Fail  Failed  List of Failed
-------------------------------------------------------------------------------
/root/posix/tests/link/00.t                   82    2   2.44%  56 63
/root/posix/tests/truncate/00.t               21    1   4.76%  15
/root/posix/tests/unlink/00.t                 55    3   5.45%  17 22 53
Failed 3/184 test scripts, 98.37% okay. 6/1950 subtests failed, 99.69% okay.

Thanks,
Badari

--

Very interesting.  ocfs2, running as 'ext3' mode, gets:
Failed 9/184 test scripts, 95.11% okay. 32/1950 subtests failed, 98.36% okay.

Thanks!
Joel

-- 

"In the beginning, the universe was created. This has made a lot 
 of people very angry, and is generally considered to have been a 
 bad move."
        - Douglas Adams

Joel Becker
Principal Software Developer
Oracle
E-mail: joel.becker@oracle.com
Phone: (650) 506-8127
--

That's not bad as a start and it doesn't necessarily mean that there is 
anything wrong with ocfs2. There are many cases when SuS says that the 
behavior can be implementation specific.

But we did find that following ext3 as close as possible will reduce the 
number of bug reports. We started from "574/1950 subtests failed, 70.56% 
okay."

Regards,
	    Szaka

--
NTFS-3G:  http://ntfs-3g.org
--

Yeah, I haven't looked into the failures.  Some day when I have
free time :-)

Joel

-- 

"When I am working on a problem I never think about beauty. I
 only think about how to solve the problem. But when I have finished, if
 the solution is not beautiful, I know it is wrong."
         - Buckminster Fuller

Joel Becker
Principal Software Developer
Oracle
E-mail: joel.becker@oracle.com
Phone: (650) 506-8127
--