Re: What's slated for inclusion in 2.6.24-rc1 from the NFS client git tree...

Previous thread: Re: + add-documentation-w1w1-masters-00-index.patch added to -mm tree by Randy Dunlap on Wednesday, October 3, 2007 - 5:38 pm. (3 messages)

Next thread: Re: Point of gpl-only modules (flame) by Robert Hancock on Wednesday, October 3, 2007 - 10:08 pm. (1 message)
To: <linux-kernel@...>, Andrew Morton <akpm@...>
Cc: <nfs@...>, <nfsv4@...>
Date: Wednesday, October 3, 2007 - 7:41 pm

Aside from the usual updates from Chuck for NFS-over-IPv6 (still
incomplete) and a number of bugfixes for the text-based mount code, the
main news in the NFS tree is the merging of support for the NFS/RDMA
client code from Tom Talpey and the NetApp New England (NANE) team.

We also have the 64-bit inode support from RedHat/Peter Staubach.

There is also the addition of a nfs_vm_page_mkwrite() method in order to
clean up the mmap() write code.
Finally, I've been working on a number of updates for the attribute
revalidation, having pulled apart most of the dentry and attribute
revalidation into separate variables. A number of fixes that address
existing bugs fell out of that review, which should hopefully result in
more efficient dcache behaviour...

The NFS client git tree can be found at

git://git.linux-nfs.org/pub/linux/nfs-2.6.git

or on gitweb at

http://linux-nfs.org/cgi-bin/gitweb.cgi?p=nfs-2.6.git;a=summary

Finally, a full set of patches may be found on

http://client.linux-nfs.org/Linux-2.6.x/2.6.23-rc9/

Cheers
Trond

-------------------

Adrian Bunk (1):
[2.6 patch] net/sunrpc/rpcb_clnt.c: make struct rpcb_program static

Christoph Hellwig (1):
[NFS] [PATCH] nfs: tiny makefile cleanup

Chuck Lever (41):
SUNRPC: Fix a signed v. unsigned comparison in rpcbind's XDR routines
SUNRPC: Fix a signed v. unsigned comparison in net/sunrpc/xprtsock.c
SUNRPC: Use standard macros for printing IP addresses
SUNRPC: Free address buffers in a loop
SUNRPC: Add hex-formatted address support to rpc_peeraddr2str()
SUNRPC: Rename xs_format_peer_addresses
SUNRPC: add a function to format IPv6 addresses
SUNRPC: add support for IPv6 to the kernel's rpcbind client
SUNRPC: Introduce support for setting the port number in IPv6 addresses
SUNRPC: Rename xs_bind() to prepare for IPv6-specific bind method
SUNRPC: create an IPv6-savvy mechanism for binding to a reserved port
SUNRPC: Refactor a ...

To: Trond Myklebust <trond.myklebust@...>
Cc: <linux-kernel@...>, Andrew Morton <akpm@...>, <nfs@...>, <nfsv4@...>
Date: Thursday, October 4, 2007 - 2:52 am

On Wed, 03 Oct 2007 19:41:16 -0400

As has been pointed[1] out[2], this will cause regressions for non-LFS
applications (of which there are still lots and lots). This change
should be in feature-removal (the "feature" being removed is legacy
support for non-LFS applications using NFS servers that make full use
of the protocol) and preferably accompanied with appropriate user space
changes (e.g. compatibility option in glibc).

[1] https://bugzilla.redhat.com/show_bug.cgi?id=241348
[2] http://marc.info/?l=linux-nfs&m=118701088726477&w=2

Rgds
--
-- Pierre Ossman

Linux kernel, MMC maintainer http://www.kernel.org
PulseAudio, core developer http://pulseaudio.org
rdesktop, core developer http://www.rdesktop.org
-

To: Pierre Ossman <drzeus-list@...>, Peter Staubach <staubach@...>
Cc: <nfsv4@...>, Andrew Morton <akpm@...>, <nfs@...>, <linux-kernel@...>
Date: Thursday, October 4, 2007 - 10:00 am

How about a boot/module parameter to turn it on or off?

I don't see any point in having a sysctl for something like this: either
you have legacy applications or you don't. It is not something that you
switch off as you go off to lunch.
A compile parameter, OTOH, would be too restrictive since it would force
distros to choose just one behaviour (which would mean they would have
to choose the most conservative).

Trond

-

To: Trond Myklebust <trond.myklebust@...>
Cc: Pierre Ossman <drzeus-list@...>, Peter Staubach <staubach@...>, <nfsv4@...>, Andrew Morton <akpm@...>, <nfs@...>, <linux-kernel@...>
Date: Friday, October 5, 2007 - 1:30 pm

How does Joe Sysadmin tell if he has an affected legacy app or not?

(The obvious "try it and see what breaks" is a non-starter for many places,
because you too easily end up in a loop of "enable it, find 4-5 show stoppers,
turn it off, fix them, lather rinse repease". Been there, done that, got
the tshirt - a project I got dragged into involves a large storage array that
appears to insist on exporting 64-bit stuff, and a large farm of clients that
are very 64-bit unclean....)

To: <Valdis.Kletnieks@...>
Cc: Trond Myklebust <trond.myklebust@...>, Peter Staubach <staubach@...>, <nfsv4@...>, <linux-kernel@...>, <nfs@...>, Pierre Ossman <drzeus-list@...>, Andrew Morton <akpm@...>
Date: Friday, October 5, 2007 - 2:12 pm

On Fri, 05 Oct 2007 13:30:10 -0400

Note that "try it and see what breaks" isn't reliable either. If glibc
gets back a 64 bit inode number that just happens to fit in the 32-bit
field, then everything will work. You don't actually get an EOVERFLOW
until st_ino overflows the field, and that may not happen often enough
for testing this way to detect it...

--
Jeff Layton <jlayton@redhat.com>
-

To: Jeff Layton <jlayton@...>
Cc: <Valdis.Kletnieks@...>, Peter Staubach <staubach@...>, <Andrew@...>, <nfsv4@...>, <linux-kernel@...>, Trond Myklebust <trond.myklebust@...>, <nfs@...>, Pierre Ossman <drzeus-list@...>, Morton <akpm@...>
Date: Sunday, October 7, 2007 - 6:56 pm

There's a damn easy way of testing this.

Use XFS on a 64 bit Linux NFS server, mount is '-o inode64,ino64'
and then export it to you client that is going to have problems.
the "ino64" mount option guarantees that the userspace visible
inode number is always > 32 bits in length....

Cheers,

Dave.
--
Dave Chinner
Principal Engineer
SGI Australian Software Group
-

To: <Valdis.Kletnieks@...>
Cc: Trond Myklebust <trond.myklebust@...>, Peter Staubach <staubach@...>, <nfsv4@...>, <linux-kernel@...>, <nfs@...>, Pierre Ossman <drzeus-list@...>, Andrew Morton <akpm@...>
Date: Friday, October 5, 2007 - 2:00 pm

On Fri, 05 Oct 2007 13:30:10 -0400

In addition to Trond's suggestion, you might be able to use "nm" or
something like it and see if there are references to non-LFS (f)stat
calls in your binaries. For instance, if you see references to stat()
(and not stat64()), then the app is probably not built with 64-bit file
offsets.

This is probably not as reliable as Trond's method, but it might be
less invasive and reasonable for a first pass when looking for these
sorts of apps...

--
Jeff Layton <jlayton@redhat.com>
-

To: Jeff Layton <jlayton@...>
Cc: <Valdis.Kletnieks@...>, Peter Staubach <staubach@...>, <Andrew@...>, <nfsv4@...>, <linux-kernel@...>, Trond Myklebust <trond.myklebust@...>, <nfs@...>, Pierre Ossman <drzeus-list@...>, Morton <akpm@...>
Date: Monday, October 8, 2007 - 4:36 am

Attached is a Perl script I wrote a while back to scan directories
looking for old stat calls in binaries. Here's the output from
my laptop:

# ./summarise-stat64.pl /usr/bin
775 26.8% are scripts (shell, perl, whatever)
1404 48.5% don't use any stat() family calls at all
428 14.8% use 32-bit stat() family interfaces only
278 9.6% use 64-bit stat64() family interfaces only
11 0.4% use both 32-bit and 64-bit stat() family interfaces

# ./summarise-stat64.pl /usr/sbin
164 35.7% are scripts (shell, perl, whatever)
170 37.0% don't use any stat() family calls at all
78 17.0% use 32-bit stat() family interfaces only
46 10.0% use 64-bit stat64() family interfaces only
1 0.2% use both 32-bit and 64-bit stat() family interfaces

# ./summarise-stat64.pl -v /usr/bin
...
/usr/bin/vi use 32-bit stat() family interfaces only
/usr/bin/view use 32-bit stat() family interfaces only
/usr/bin/vim use 32-bit stat() family interfaces only
...
/usr/bin/Mail use 32-bit stat() family interfaces only
/usr/bin/mail use 32-bit stat() family interfaces only
/usr/bin/mailx use 32-bit stat() family interfaces only
...
/usr/bin/gdb use 32-bit stat() family interfaces only
/usr/bin/gdbtui use 32-bit stat() family interfaces only
/usr/bin/rpcgen use 32-bit stat() family interfaces only
...
/usr/bin/cc use 32-bit stat() family interfaces only
/usr/bin/gcc use 32-bit stat() family interfaces only
/usr/bin/gcov use 32-bit stat() family interfaces only
/usr/bin/unprotoize use 32-bit stat() family interfaces only
...
/usr/bin/git use 32-bit stat() family interfaces only
/usr/bin/git-check-ref-format use 32-bit stat() family interfaces only
/usr/bin/git-cat-file use 32-bit stat() family interfaces only
/usr/bin/git-checkout-index use 32-bit stat() family interfaces only
/usr/bin/git-clone-pack use 32-bit stat() family interfaces only
/usr/bin/git-commit-tree use 32-bit stat() family interfaces only
/usr/bin/git-convert-objects use 32-bit stat() family interfac...

To: <Valdis.Kletnieks@...>
Cc: Pierre Ossman <drzeus-list@...>, Peter Staubach <staubach@...>, <nfsv4@...>, Andrew Morton <akpm@...>, <nfs@...>, <linux-kernel@...>
Date: Friday, October 5, 2007 - 1:52 pm

If you're unsure, then set the bloody boot parameter. That's what it is
for...

Trond

-

To: Trond Myklebust <trond.myklebust@...>
Cc: Peter Staubach <staubach@...>, <nfsv4@...>, Andrew Morton <akpm@...>, <nfs@...>, <linux-kernel@...>
Date: Thursday, October 4, 2007 - 12:43 pm

On Thu, 04 Oct 2007 10:00:50 -0400

That would be perfect. It can even be in non-legacy mode by default,
just as long as you can go back to the old behaviour when/if you run

Agreed.

Rgds
--
-- Pierre Ossman

Linux kernel, MMC maintainer http://www.kernel.org
PulseAudio, core developer http://pulseaudio.org
rdesktop, core developer http://www.rdesktop.org
-

To: Pierre Ossman <drzeus-list@...>
Cc: <trond.myklebust@...>, <staubach@...>, <nfsv4@...>, <nfs@...>, <linux-kernel@...>
Date: Thursday, October 4, 2007 - 2:42 pm

On Thu, 4 Oct 2007 18:43:04 +0200

Wouldn't a mount option be better?
-

To: Andrew Morton <akpm@...>
Cc: Pierre Ossman <drzeus-list@...>, <staubach@...>, <nfsv4@...>, <nfs@...>, <linux-kernel@...>
Date: Thursday, October 4, 2007 - 3:16 pm

I suppose that might be OK if you know that the 32-bit legacy
applications will only touch one or two servers, but that sounds like a
niche thing.

On the downside, forcing all those people who have portable 64-bit aware
applications to upgrade their version of mount just in order to have
stat64() work correctly seems unnecessarily complicated. I'd prefer not
to have to do that unless someone comes up with a good reason why we
must.

Cheers
Trond

-

To: Trond Myklebust <trond.myklebust@...>
Cc: <drzeus-list@...>, <staubach@...>, <nfsv4@...>, <nfs@...>, <linux-kernel@...>
Date: Thursday, October 4, 2007 - 3:59 pm

On Thu, 04 Oct 2007 15:16:03 -0400

Confused. You don't need to modify mount(8) when adding a new mount option?
-

To: Andrew Morton <akpm@...>
Cc: <drzeus-list@...>, <staubach@...>, <nfsv4@...>, <nfs@...>, <linux-kernel@...>
Date: Thursday, October 4, 2007 - 8:58 pm

Prior to 2.6.22, the 'mount' program used a binary blob for passing the
NFS mount options to the kernel.
It is only very recently that we have started doing in-kernel parsing of
text strings, and in order to make use of that, people will need to
upgrade to the latest version of nfs-utils.

Trond

-

To: Trond Myklebust <trond.myklebust@...>
Cc: Andrew Morton <akpm@...>, Pierre Ossman <drzeus-list@...>, <nfsv4@...>, <nfs@...>, <linux-kernel@...>
Date: Thursday, October 4, 2007 - 3:41 pm

I would agree. The 64 bit fileids will only become visible when
the server is exporting file systems which contain fileids which
are bigger than 32 bits and then only when the application
encounters these files.

Also, these 32-bit legacy applications are going to have a
problem if they are ever run on a system which contains local
file systems which expose the large fileids.

It would be better to identify these applications and get them
fixed. The world is evolving and it is time for them to do so.

ps
-

To: Peter Staubach <staubach@...>
Cc: Trond Myklebust <trond.myklebust@...>, Andrew Morton <akpm@...>, <nfsv4@...>, <nfs@...>, <linux-kernel@...>
Date: Friday, October 5, 2007 - 2:25 am

On Thu, 04 Oct 2007 15:41:57 -0400

Or, as has been pointed out, when the server is not the Linux in-kernel

Agreed. And I'd probably like a way around that as well. But local
files have never worked, NFS has. So removing it from NFS (where it is

Print a warning or something so that they can be found. Don't go
breaking systems left and right. People have better things to do than
to fix the build systems for ever program they use.

Rgds
--
-- Pierre Ossman

Linux kernel, MMC maintainer http://www.kernel.org
PulseAudio, core developer http://pulseaudio.org
rdesktop, core developer http://www.rdesktop.org
-

To: Pierre Ossman <drzeus-list@...>
Cc: Peter Staubach <staubach@...>, Andrew Morton <akpm@...>, <nfsv4@...>, <nfs@...>, <linux-kernel@...>
Date: Friday, October 5, 2007 - 1:36 pm

The kernel knows bugger all about what glibc function your program is
calling. The problem here is precisely that newer versions of glibc will
transform legacy 32-bit stat() calls into 64-bit stat64() calls, then
will complain when the result overflows.

If you want to figure out which apps are broken, then you will have to
either do so in glibc or use a preloaded shared library to intercept the
32-bit stat() calls and print out a warning.

Trond

-

To: Trond Myklebust <trond.myklebust@...>
Cc: Peter Staubach <staubach@...>, Andrew Morton <akpm@...>, <nfsv4@...>, <nfs@...>, <linux-kernel@...>
Date: Friday, October 5, 2007 - 1:54 pm

On Fri, 05 Oct 2007 13:36:19 -0400

Right, I didn't suggest that this had to be done in the kernel. My
point was that first you mark something as deprecated, make a lot of
noise when someone uses it so that problems can be identified, and some
time later you remove it. You don't just remove it and let production
systems deal with the fallout.

Rgds
--
-- Pierre Ossman

Linux kernel, MMC maintainer http://www.kernel.org
PulseAudio, core developer http://pulseaudio.org
rdesktop, core developer http://www.rdesktop.org
-

To: Trond Myklebust <trond.myklebust@...>
Cc: <linux-kernel@...>, Andrew Morton <akpm@...>, <nfs@...>, <nfsv4@...>
Date: Wednesday, October 3, 2007 - 7:52 pm

The marketroids compel me to say: It is Red Hat, not RedHat :)

Jeff, looking forward to NFSv4 over IPv6

-

Previous thread: Re: + add-documentation-w1w1-masters-00-index.patch added to -mm tree by Randy Dunlap on Wednesday, October 3, 2007 - 5:38 pm. (3 messages)

Next thread: Re: Point of gpl-only modules (flame) by Robert Hancock on Wednesday, October 3, 2007 - 10:08 pm. (1 message)