(resending .. forgot to copy the lists originally)
We have a problem coming down the pipeline.
Practically all utilities that care about it,

use ioctl(fd, HDIO_GETGEO) to determine the starting

sector offset of a hard disk partition.
SCSI, libata, IDE, USB, Firewire.. you name it.
The return value uses "unsigned long",

which on a 32-bit system limits drive offsets to 2TB.
There will be single drives exceeding this limit within

the next 12 months or less, and we already have RAID arrays

that exceed 2TB.
So.. what's the replacement for HDIO_GETGEO on 32-bits ?
One candidate might seem to be the existing /sys/block/dev/partition/start

which I expect is already 64-bit friendly.
But this requires about 150 lines of somewhat complex C code to access,

using only the dev_t (from stat(2) on a file) as a starting point,

or less if one relies upon the udev device name matching the sysfs device name.
Is it time now for HDIO_GETGEO64 to make an appearance?

Similar to how the existing BLKGETSIZE64 is supplanting BLKGETSIZE ?
??

--

Perhaps I've missed something, but surely geometry doesn't make sense on

a >2TB drive does it?  The only reason we use it on modern disks (which

usually make it up specially for us) is that the DOS partition scheme

requires it.  Once we're over 2TB, isn't it impossible to use DOS

partitions (well, OK, unless you increase the sector size, but that's

only delaying the inevitable), so we can just go with a proper disk

labelling scheme and use BLKGETSIZE64 all the time.
James
--

On Tue, Mar 25, 2008 at 11:17 AM, James Bottomley
I believe GUID Partition Tables (GPTs) are the answer.
I believe one of the features of GPT is the elimination of the 32-bit

sector restrictions.
http://en.wikipedia.org/wiki/GUID_Partition_Table
Windows VISTA 64-bit supports GPTs on data disks and new Mac OS based

systems have been using it on internal drives for a couple years at

least.
GPTs are part of the Extensible Firmware Interface (EFI), so they

should be usable for PC bootable disks at some point.  (Maybe now in

some cases?)
I'm not sure what the Linux Kernel support is for GPTs.
Greg

--

Greg Freemyer

Litigation Triage Solutions Specialist

http://www.linkedin.com/in/gregfreemyer

First 99 Days Litigation White Paper -

http://www.norcrossgroup.com/forms/whitepapers/99%20Days%20whitepaper.pdf
The Norcross Group

The Intersection of Evidence & Technology

http://www.norcrossgroup.com

--

It has been supported since the first Itanium systems shipped.  It's

the first code I wrote 7+ years before it was really needed. :-)  Most

distributions have it enabled, as do userspace tools like GNU Parted.
--

Matt Domsch

Linux Technology Strategist, Dell Office of the CTO

linux.dell.com & www.dell.com/linux

--

It's implemented.  Not sure about how well used/tested it is.
config EFI_PARTITION

	bool "EFI GUID Partition support"

	depends on PARTITION_ADVANCED

	select CRC32

	help

	  Say Y here if you would like to use hard disks under Linux which

	  were partitioned using EFI GPT.
---

~Randy

--

ia64 uses it exclusively ... at least on discs that you want to use from

EFI.
--

Intel are signing my paycheques ... these opinions are still mine

"Bill, look, we understand that you're interested in selling us this

operating system, but compare it to ours.  We can't possibly take such

a retrograde step."

--

I thinks intel-Macs do too.
Boaz
--

..
I haven't thought much about problems with the virtual geometry,

because, as you say, we really don't care about it for the most part.

We use LBA values from the partition tables rather than CHS.

I suppose those also likely to be 32-bit limited.
The "partition offset", or "starting sector" is the important

bit of information for most things.  And that's currently available

from HDIO_GETGEO, and from /sys/block/XXX/XXXn/start, if sysfs is mounted.
We just need an easy way to get it, given a dev_t from stat(2).

Currently there isn't an easy way, and HDIO_GETGEO returns

only 32-bits on a 32-bit system.
Cheers
--

But I think where this is leading is that you've been using the geometry

call, but all you really want to know is the actual partition start in

sector units, so a new BLKGETPARTSTART (or something) ioctl that was

designed to return a u64 would work for you?  That sounds reasonable to

me; so not a HDIO_GETGEO64 which gets us into trouble with geometries,

but a simple ioctl that gives you exactly what you're looking for.
James
--

Probably a better thing to have would be a way to look up block devices

in sysfs by device number.
	-hpa

--

It shouldn't be under /sys/block... there are enough many things that

scan /sys/block and assume any directory underneath it has the current

format.
	-hpa

--

..
So long as we only add things, and not remove them, then any software

that scans /sys/block/ shouldn't care, really.
But yes, it could go elsewhere, too.

Perhaps a /sys/dev/ directory, populated with symbolic links

(or hard links?) back to the /sys/block/ entries, something like this:
   /sys/dev/block/8:0 -> ../../../block/sda

   /sys/dev/block/8:1 -> ../../../block/sda/sda1

   /sys/dev/block/8:2 -> ../../../block/sda/sda2

   ...
That's just a suggestion, really.

And what about character devices?
Perhaps Greg will chime in.

--

I've been waiting to see if sanity will take hold of anyone here.
Come on people, adding symlinks for device major:minor numbers in sysfs

to save a few 10s of lines of userspace code?  Can things get sillier?
You can add a single udev rule to probably build these in a tree in /dev

if you really need such a thing...
And what's wrong with your new ioctl recomendation?
greg k-h

--

..
So have we.  sysfs is a total nightmare to extract information from

under program / script control.  The idea presented in this thread,

is to have it cross-index the contents with a method that actually

makes it easy to access in many common scenarios, without requiring

huge gobs of code in user space.  Or in kernel space.
And it's not just a few 10s of lines of code currently,

but rather about 80-100 lines just to find the correct device subdir,

and *then* a few more 10s of lines of code to retrieve the value.
In a bulletproof fashion, that is.  Sure it can be slightly smaller

if niceties such as error checking/handling are omitted.
There's no guarantee that udev is present, and even if it were present,

there's no guarantee that the names in /dev/ will match /sysfs/ pathnames,

since udev is very configurable to do otherwise.
So lookups are by dev_t, which sysfs has no simple or even easy way

of accomplishing.  O(n) at a minimum.
If we make it easier to access, then more programs will use it

rather than us having to expand our tricky binary ioctl interfaces.
Isn't that part of the idea of sysfs -- to limit the need for new ioctls ?
Cheers

--

Hmm, 100 lines? What else do you need?
  $ grep -l 8:3 /sys/class/block/*/dev

  /sys/class/block/sdc/dev
Kay

--

That's particularly funny, because your very own example gives the wrong

result -- sdc is 8:32 not 8:3 (which is sdc3, which is also excluded by

your search.)
Not to mention the fact that it is still O(n).
	-hpa

--

Very true, but I guess you get the idea, and know how to add the proper
Any real numbers from a large setup, which show that we want to have a

reverse devnum map in sysfs?
Thanks,

Kay
--

Hello,
The questions are...
1. Are we gonna push sysfs as the primary interface and not provide an

alternative interface (ioctl here) which can provide equivalent

information?  There are people running their systems w/o sysfs but I

think we're getting closer to this everyday.
2. Is udev an essential part of all systems?  I'm not sure about this

one.  Lots of small machines run w/o udev and I think udev is a bit too

high level to depend on for every system.
If both #1 and #2 are true, I agree with Mark that we need an easy to

map from device number to matching sysfs nodes.  Tools which are used

early during boot and emergency sessions need this mapping and many of

them are minimal C program w/o much dependency for a good reason.

Requiring each of them to implement their own way to map device node to

sysfs node is too awkward.
Probably something like /sys/class/block/MAJ:MIN or

/sys/class/devnums/bMAJ:MIN?
--

tejun

--

"Devices directories" are not supposed to contain duplicate entries.
These are no devices belonging to the class "devnums", so it may

confuse things which crawl these directories to get "all devices".

Current coldplug-like setups will likely add duplicate devices with

the wrong subsystem. There are also bus-devices with have a dev_t, and

that will make them show up in /sys/class, which might confuse some

tools too.
I guess we will need to find some other solution as a /sys/class/ for

that. And we must prefix the links with 'c' and 'b' because dev_t is

not unique across char and block devices.
Thanks,

Kay

--

It doesn't really seem to be to belong under class at all.  I would

suggest /sys/dev/char/ and /sys/dev/block/, for char and block respectively.
	-hpa
--

This thread fizzled out without a patch... here goes:
[ note: I'm replying via gmail, so if it has whitespace mangled the

patch please see the attachment ]
-----snip---->

sysfs: add /sys/dev/{char,block} to lookup sysfs path by major:minor
From: Dan Williams <dan.j.williams@intel.com>
Why?:

There are occasions where userspace would like to access sysfs

attributes for a device but it may not know how sysfs has named the

device or the path.  For example what is the sysfs path for

/dev/disk/by-id/ata-ST3160827AS_5MT004CK?  With this change a call to

stat(2) returns the major:minor then userspace can see that

/sys/dev/block/8:32 links to /sys/block/sdc.
What are the alternatives?:

1/ Add an ioctl to return the path: Doable, but sysfs is meant to reduce

   the need to proliferate ioctl interfaces into the kernel, so this

   seems counter productive.
2/ Use udev to create these symlinks: Also doable, but it adds a

   udev dependency to utilities that might be running in a limited

   environment like an initramfs.
Cc: NeilBrown <neilb@suse.de>

Cc: Tejun Heo <htejun@gmail.com>

Cc: Kay Sievers <kay.sievers@vrfy.org>

Cc: Greg KH <gregkh@suse.de>

Cc: Mark Lord <lkml@rtr.ca>

Cc: H. Peter Anvin <hpa@zytor.com>

Signed-off-by: Dan Williams <dan.j.williams@intel.com>

---
 drivers/base/core.c |   37 ++++++++++++++++++++++++++++++++++++-

 1 files changed, 36 insertions(+), 1 deletions(-)
diff --git a/drivers/base/core.c b/drivers/base/core.c

index 24198ad..de925f8 100644

--- a/drivers/base/core.c

+++ b/drivers/base/core.c

@@ -27,6 +27,9 @@
 int (*platform_notify)(struct device *dev) = NULL;

 int (*platform_notify_remove)(struct device *dev) = NULL;

+static struct kobject *dev_kobj;

+static struct kobject *char_kobj;

+static struct kobject *block_kobj;
 #ifdef CONFIG_BLOCK

 static inline int device_is_not_partition(struct device *dev)

@@ -759,6 +762,11 @@ static void device_remove_class_symlinks(struct

device *dev...
[ message continues ]

Crickets are chirping and I can't remember what the conclusion to all this

was.  In fact the thread was more than ten-deep so I probably fell asleep.
I queued it up so that others cannot do the same ;)

--

The expressed preference was simply to expand the ioctl (or add a new

one that got the required information without having to go through the

old HDIOGETGEO path to extract the value from a fictitious geometry).
Greg was a bit sceptical of the value of the above proposal ...
James
--

However, you have to admit that kind of defeats the whole point of

having this information in sysfs.  IMNSHO, even scanning sysfs is better

than keep adding binary ioctls.
	-hpa
--

This is a note to let you know that I've just added the patch titled
     Subject: sysfs: add /sys/dev/{char,block} to lookup sysfs path by major:minor
to my gregkh-2.6 tree.  Its filename is
     sysfs-add-sys-dev-char-block-to-lookup-sysfs-path-by-major-minor.patch
This tree can be found at

    http://www.kernel.org/pub/linux/kernel/people/gregkh/gregkh-2.6/patches/
From dan.j.williams@intel.com  Wed Apr 16 13:49:38 2008

From: Dan Williams <dan.j.williams@intel.com>

Date: Tue, 15 Apr 2008 16:43:15 -0700

Subject: sysfs: add /sys/dev/{char,block} to lookup sysfs path by major:minor

To: Andrew Morton <akpm@linux-foundation.org>

Cc: James Bottomley <James.Bottomley@HansenPartnership.com>, Kay Sievers <kay.sievers@vrfy.org>, Tejun Heo <htejun@gmail.com>, Mark Lord <lkml@rtr.ca>, Greg KH <gregkh@suse.de>, Jens Axboe <axboe@kernel.dk>, Jeff Garzik <jgarzik@pobox.com>, Linus Torvalds <torvalds@linux-foundation.org>, Linux Kernel <linux-kernel@vger.kernel.org>, IDE/ATA development list <linux-ide@vger.kernel.org>, linux-scsi <linux-scsi@vger.kernel.org>, "H. Peter Anvin" <hpa@zytor.com>

Message-ID: <1208302995.21877.12.camel@dwillia2-linux.ch.intel.com>
From: Dan Williams <dan.j.williams@intel.com>
Why?:

There are occasions where userspace would like to access sysfs

attributes for a device but it may not know how sysfs has named the

device or the path.  For example what is the sysfs path for

/dev/disk/by-id/ata-ST3160827AS_5MT004CK?  With this change a call to

stat(2) returns the major:minor then userspace can see that

/sys/dev/block/8:32 links to /sys/block/sdc.
What are the alternatives?:

1/ Add an ioctl to return the path: Doable, but sysfs is meant to reduce

   the need to proliferate ioctl interfaces into the kernel, so this

   seems counter productive.
2/ Use udev to create these symlinks: Also doable, but it adds a

   udev dependency to utilities that might be running ...
[ message continues ]

..
Last I recall, Greg was vehemently opposed to having direct path access

by device number in sysfs, but many other people saw benefit.
Myself (the originator), I simply decided that my sysfs access code has

to work with older kernels too, so for now I'm just doing a brute force

tree search to find things in sysfs.  I did get the code size down smaller

for it, but it's still a pain.
When the direct access feature goes in, I'll just change my code to try it first,

..
Good!

--

I think you are using either the wrong programming language, or your

sysfs walking logic is quite convulted.  Look at the udev and HAL code
Exactly, originally you suggested a new ioctl, which would be trivial to

add, and trivial to switch any program that was currently using an ioctl

to get the disk size, to use it instead.
Since when is the major:minor view of devices the "standard" one that

userspace uses?  Last I looked, userspace uses symlinks and lots of

other ways of directly accessing block devices in /dev/, and does not

rely on major:minor.
And finally, I haven't seen a patch that implements this "shadow" tree,
My tiny little phone runs udev, I don't see why anyone wouldn't run it

these days, except in very limited embedded applications with no dynamic

devices.  But if you are in that situation, you aren't querying the size

of any random block device either :)
And heck, this phone is a very limited embedded application, with razor

thin margins, if it can use udev, I'd be interested in hearing the

justifications for anyone who says it is too large for their systems to
Why the preopcupation with major:minor?  Just because you are able to

grab it from an open file handle?  Heck, why not just an ioctl to get

the path within sysfs for the device currently open?  :)
thanks,
greg k-h

--

Hello, Greg.
The fact that major:minor is the unique identifier of a device makes it
It's possible, all that's needed are symlinks.  We do similar things all
I agree udev is affordable for most cases but it's still a major step to

require it for every system.  I would hate to hear that hdparm or fdisk

doesn't work unless udev is online.  These are tools which are used to
Because major:minor is the key attribute to devices?
Thanks.
--

tejun

--

In particular, stat() and friends returns the device number, not a

device name.
	-hpa

--

Ah, there's some sanity.  :)
---

~Randy

--

It's not so much an issue of a few tens of lines of user space code, but

rather the fact that something that should be O(1) is currently O(n).
	-hpa

--

"should"?  why?  Is this some new requirement that everyone needs?  I've

_never_ seen anyone ask for the ability to find sysfs devices by

major:minor number in O(1) time.  Is this somehow a place where such

optimization is warranted?
thanks,
greg k-h

--

Well, when dealing with shell scripts a O(n) very easily becomes O(n^2).

  For the stuff that I, personally, do, it's not a big deal, but people

with large number of disks have serious gripes with our boot times.
	-hpa

--

This should be a solved problem with scsi_mod.scan=async (or equivalent

compile option).  Are people still complaining about it, and if so, have

they tried this option?
--

Intel are signing my paycheques ... these opinions are still mine

"Bill, look, we understand that you're interested in selling us this

operating system, but compare it to ours.  We can't possibly take such

a retrograde step."

--

How does this have anything to do with boot times?  Do you really have a

foolish shell script that iteratorates over every single disk in the

sysfs tree for every disk?  What does it do that for?
I thought we were talking about 2TB disks here, with a proposed new

ioctl, not foolishness of boot scripts...

--

Any time you want to get the sysfs information for a filesystem which is 
I pointed out that having a way to map device numbers to sysfs

directories would have the same effect, *and* would be usable for other

purposes.  I'd rather see that than a new ioctl, and another, and another...
ioctl()s are also nasty since they're generally root-only (or rather,

device-owner only).  Since the information is already in sysfs, there is

no benefit to this hiding.  Otherwise one could consider a ioctl() "give

me the sysfs name of this device."
	-hpa

--

Again, a simple udev rule will give you that today if you really want

it...
And I think 'udevinfo' can be used to retrieve this information as well.
thanks,
greg k-h

--

That sounds useful.
But you're the one who has investigated this - please make a recommendation?

--