During a conversation about converting pci_drivers, a question was raised about adding user space interfaces into the kernel to support XFree86. Linux creator Linus Torvalds quickly replied, "There won't be any 'user space interfaces'. There are perfectly good in-kernel interfaces, and anybody who needs them needs to be in kernel space. Ie the kernel interfaces are for kernel modules, not for user space accesses."
Linus went on to note that he'd "be really happy to have real graphics drivers in the kernel", noting that this would mean a low-level driver doing only two things: basic hardware enumeration and setup, and serialization and arbitrary command queuing from a trusted party. He concludes, "Then, fbcon and DRI and X could all three use these basics - and they'd be _so_ basic that the hardware layer could be really stable (unlike the DRI code that tends to have to upgrade for each new type of command that DRI adds - since it has to take care of untrusted clients."
From: Egbert Eich [email blocked] To: Linus Torvalds [email blocked] Subject: Re: [Dri-devel] Re: [Linux-fbdev-devel] DRM and pci_driver conversion Date: Sat, 25 Oct 2003 19:29:18 +0200 Linus Torvalds writes: > > could lead to problems with hotplug. XFree is also mapping PCI ROMs in without > > informing the kernel and that can definitely cause problems. > > Absolutely. Changing PCI configurations without telling the kernel _will_ > cause problems. Especially for hotplug systems, but it's also very iffy to > do if the card is behind a PCI bridge, since you have to take bridge > resources into account (and know which bridges are transparent, which are > not, etc etc). Speaking of XFree86: when I developed the PCI resource stuff in XFree86 I was trying to get support from kernel folks to get the appropriate user space interfaces into the kernel. When I got nowhere I decided to do everything myself. XFree86 does PCI bridge tracking and one reason it does this is PCI ROM mapping. > > The kernel spends a lot of effort on getting this right, and even so it > fails every once in a while (devices that use IO or memory regions without > announcing them in the standard BAR's are quite common, and the kernel has > to have special quirk entries for things like that). Right. One reason why the PCI code in XFree86 looks so difficult is that old ATi Mach?? cards had their 8514 registers (and their mirror images) scattered all over the PIO space. > > Few enough drivers actually want to enable the roms, but the code should > look something like > > /* Assign space for ROM resource if not already assigned. Ugly. */ > if (!pci_resource_start(dev, PCI_ROM_RESOURCE)) > if (pci_assign_resource(dev, PCI_ROM_RESOURCE) < 0) > return -ENOMEM; [Stuff deleted] ..... Is there any API that allows one to do this from user space? There are plenty of XFree86 drivers around that don't have a DRM kernel module and it should be possible to run those which do without DRI support, therefore it would be a good if the XFree86 driver could do this registration itself. Cheers, Egbert.
From: Linus Torvalds [email blocked] Subject: Re: [Dri-devel] Re: [Linux-fbdev-devel] DRM and pci_driver conversion Date: Sat, 25 Oct 2003 11:37:05 -0700 (PDT) On Sat, 25 Oct 2003, Egbert Eich wrote: > > Speaking of XFree86: when I developed the PCI resource stuff in > XFree86 I was trying to get support from kernel folks to get the > appropriate user space interfaces into the kernel. When I got > nowhere I decided to do everything myself. There won't be any "user space interfaces". There are perfectly good in-kernel interfaces, and anybody who needs them needs to be in kernel space. Ie the kernel interfaces are for kernel modules, not for user space accesses. The kernel module can be a simple interface layer like DRI, but the thing is, it needs to be specific to some kind of hardware. I refuse to have some kind of crap "user space driver" interface - drivers in user space are a mistake. > Is there any API that allows one to do this from user space? No. And I don't really see any huge reason for it. > There are plenty of XFree86 drivers around that don't have a > DRM kernel module and it should be possible to run those which > do without DRI support, therefore it would be a good if the > XFree86 driver could do this registration itself. If you do this, do it _right_. Look at what X really needs, and make a driver for it. A _real_ driver. Not just a half-hearted "we want to do things in user space, but we can't". Face it, a good graphics driver needs more than just "set up the ROM". It needs DMA access, and the ability to use interrupts. It needs a real driver. It basically needs something like what the DRI modules tend to do. I'd be really happy to have real graphics drivers in the kernel, but quite frankly, so far the abstractions I've seen have sucked dead donkeys through a straw. "fbcon" does way too much (it's not a driver, it's more a text delivery system and a mode switcher). And DRI is too complex and fluid to be a good low-level driver. Quite frankly, I'd much rather see a low-level graphics driver that does _two_ things, and those things only: - basic hardware enumeration and setup (and no, "basic setup" does not mean "mode switching": it literally means things like doing the pci_enable_device() stuff. - serialization and arbitrary command queuing from a _trusted_ party (ie it could take command lists from the X server, but not from untrusted clients). This part basically boils down to "DMA and interrupts". This is the part that allows others to wait for command completion, "enough space in the ring buffers" etc. But it does _not_ know or care what the commands are. Then, fbcon and DRI and X could all three use these basics - and they'd be _so_ basic that the hardware layer could be really stable (unlike the DRI code that tends to have to upgrade for each new type of command that DRI adds - since it has to take care of untrusted clients. So DRI would basically use the low-level driver as a separate module, the way it already uses AGP). But I'm _not_ interested in some interfaces to let user mode just bypass the kernel. Because they will not solve any of the other problems that clearly _do_ need solving, and if the X server continues to believe that it can just access the hardware directly, it will never play well together with projects like fbcon/dri. Linus
From: Jeff Garzik [email blocked] Subject: Re: [Dri-devel] Re: [Linux-fbdev-devel] DRM and pci_driver conversion Date: Sat, 25 Oct 2003 15:17:44 -0400 Linus Torvalds wrote: > Quite frankly, I'd much rather see a low-level graphics driver that does > _two_ things, and those things only: > > - basic hardware enumeration and setup (and no, "basic setup" does not > mean "mode switching": it literally means things like doing the > pci_enable_device() stuff. > > - serialization and arbitrary command queuing from a _trusted_ party (ie > it could take command lists from the X server, but not from untrusted > clients). This part basically boils down to "DMA and interrupts". This > is the part that allows others to wait for command completion, "enough > space in the ring buffers" etc. But it does _not_ know or care what the > commands are. Thank you for saying it. This is what I have been preaching (quietly) for years -- command submission and synchronization (and thus, DMA/irq handling) needs to be in the kernel. Everything else can be in userspace (excluding hardware enable/enumerate, of course). Graphics processors are growing more general, too -- moving towards generic vector/data processing engines. I bet you'll see an optimal model emerge where you have some sort of "JIT" for GPU microcode in userspace. Multiple apps pipeline X/GL/hardware commands into the JIT, which in turn pipelines data and microcode commands to the GPU kernel driver. Jeff
| Easter Eggs in windows XP | 33 minutes ago | Windows |
| Kernel Module Oops! TRICKY PROBLEM! | 59 minutes ago | Linux kernel |
| usb mic not detected | 16 hours ago | Applications and Utilities |
| linux-2.6.10 | 22 hours ago | Linux kernel |
| How to detect usb device insertioin and removal event ? | 1 day ago | Linux general |
| Extended configuration Read/Write | 1 day ago | Windows |
| Add ext2 inode field | 1 day ago | Linux kernel |
| the kernel how to power off the machine | 1 day ago | Linux kernel |
| struct gendisk via request_queue | 2 days ago | Linux kernel |
| page initialization during kernel initialization | 3 days ago | Linux kernel |
KGI - Kernel Graphics Interface
"[R]eal graphics drivers in the kernel". Isn't that what the goal of KGI is? Disclaimer: I'm in no way a coder.
Benefits?
What benefits would KGI bring? I saw that you could run full-screen apps in console without X, but what else? What benefits would it bring to Xfree and general desktop-users for example?
You "always" could do graphics on console
SVGAlib from 1.0? or even previously?
/dev/fb from 2.2?
something missing?
KGI Benefits
The stated goal of KGI, where it differs from other solutions, is mainly all about all about security. (If your trusted application has holes in it, and access to a GPU DMA pipeline, that's practically as bad as having a hole in the kernel itself.) A secondary goal is virtualization of resources to be more friendly to multi-application environments (that includes serialization.) In addition portability across OS platforms is high on the list.
While the trusted-app approach has its benefits in keeping the trickier-to-develop kernel code minimal, it still faces the same challenges in userspace of implementing a barrier between untrusted applications and the display server. Linux is more evolved than most, perhaps all, other OSes with scheduler and VMA tweaks to get rid of the performance problems associated with funnelling everything through a trusted userspace application, so performance-wise the benefits to a different approach will be smaller under Linux than under other OSes.
KGI is trying to find a good middle ground where the kernel code is still relatively small but untrusted, even downright malevolent, apps can be run without a userspace middleman, e.g. it does not try to implement a generic API for userspace drawing -- chipset-dependent command construction is still up to userspace. We try to present to the application something as closely resembling the raw metal of the graphics hardware as can safely be permitted -- but that cannot really include mode setting, as that area is rife with opportunities to fry the hardware. KGI basically aims to prevent damage to data or hardware, and give apps that behave right a relative unencumbered shot at the pipeline. Apps that don't play nice are to get undefined, but nondestructive, results.
Noone (at least that I've heard) argues about certain things only the kernel can do properly being placed there (like enumeration and address table mapping.) KGI does *not* argue for "implementing GL in kernelspace" like some others have.
I think both trusted-app and KGI are valid and valuable approaches -- there will potentially always be a need for the types of tweaks being done in Linux to deal with X server performance, as they are generic and apply to many situations other than just an X server and its clients. But I think KGI's approach is worth investigation -- a graphics system that tries to put hardware features and kernelspace supervision to work to secure the graphics subsystem may prove more efficient even under Linux.
Regardless of who does what and where, graphics needs to get more attention than it does. I can totally understand that the main kernel developers don't necessarily want an additional burden of a high-volume development effort taking part in the main tree. ALSA was developed on the side and so can a graphics system be.
Someone does have to step up though and raise the bar on the development process applied to new graphics subsystems. No offerings are at the same time well and openly documented, following a well thought out and peer-reviewed agenda, and using a smooth release process and build environment that provides a positive experience to end-users, code tweakers, and developers alike. At the KGI project we try, but that's a mammoth task, and we are always pretty much crippled in the manpower department and never seem to quite get there.
Brian Julin, on-and-off KGI developer.
Thanks! and question?
Thaks for your reply Brian. Fairly informative.
Just to make sure I understand what's being discussed:
- KGI wants to present a generic API to userspace. All userspace apps would go through this API, which the kernel portion would check to make sure nothing "bad" is going to happen. You try and keep this API as simple as possible.
- What Linus is talking about is much simpler: just some basic support, then a pipeline from whatever userspace app wants it down to the GPU directly. The kernel would not really get involved in this pipeline at all, just serializes access between apps. So a userspace app would be able to "fry" the hardware if they wanted - or at least mess it up a lot.
The advantage of KGI is security: the userspace app cannot do anything bad. The disadvantage of KGI is that you need to define a "generic" API for all the cards out there, then map that generic API to each card - meaning you need a specific driver for each card. And that driver has to do checks and things to make sure nothing bad is going to happen. And Linus' and company don't want another large complex API to support.
The advantage of Linus' approach would be simplicity in the kernel and the API, leaving all the complexities in the Xserver or wherever. The disadvantage, as you say, is security - this approach has none. It still has to trust in the user space app to not screw anything up (as things do now with the X server). What this would give us over the current approach (which is basically everything in userspace, including device enumeration and address mappings and so on) is that some basic management stuff that's currently in the X server could be generalized and moved into the kernel, and other stuff (DRI) that's in the kernel, could be moved out of it.
Am I getting this right?
Pete
Re: Thanks! and question?
Pete writes:
'KGI wants to present a generic API to userspace. All userspace
apps would go through this API, which the kernel portion would check to make sure nothing "bad" is going to happen. You try and keep this API as simple as possible.'
Yes and no. KGI only wants the API to be "generic" to a certain extent: How to acquire (and gracefully share with other apps) the pipeline, and a generic system for managing the pipeline buffers. I use the term pipeline here loosly, because GPUs no longer resemble a plain FIFO. Today you have your vectors and textures and commands etc. The part of the userspace API that is visible simply tells the app what areas of GART and VRAM it has, and how to designate what it wants to use these areas for. Designating some commands to be executed after a VSYNC or other realtime event also falls in this category, since most chipsets cannot do this entirely through the command pipeline.
Most of the generic part of the API centers around permission systems and determining the location, endianness and purpose of the buffers. Basically just the stuff needed to give the proofreader enough clues to know where in the chipset and main RAM the app is allowed to tread.
KGI does NOT provide a generic API to do things like "draw a rectangle", or, as DRI does, to "load a vector table". User space code must be chipset-specific in order to do so. The proofreading
is done on raw command buffers in chipset-specific format. In
addition to (or actually, in order to cut down on the complexity of) proofreading the buffers, any hardware features that the chipset has that can be used for security purposes, like clip regions, etc, are kept confined to the areas of the GART and VRAM that the application is allowed to access. We hope to use MMU/0-copy when it works, but usually 1-copy is going to be faster given TLB and/or GTLB flushing inefficiencies. The hope is that normal drawing primitives will rarely require any complex proofreading operations and can just be glossed over, while most of the work happens when texture and surface strides/widths/offsets are changed (with code optimized for changing offsets.) This seems to be working out so far pretty well in what code I've managed to complete so far for my Radeon QD, and I hope it works as well for newer and older chipsets -- we won't know until we get a few more proofreaders under our belt.
As far as modes go, the only reason KGI provides a generic API for these to userspace is simply because it needs to provide one to the OS's kernel's console system anyway, so exporting it to userspace is just a little glue. We could just as easily demand userspace use chipset-specific code to set modes and proofread the settings, but since most OSes expect to have a graphics subsystem available to the kernel that can set modes, we probably will stick to providing one, and handing it over to userspace.
I suppose I should disclose that KGI does intend to keep the full
context of a GPU abstracted such that the state can be saved, another application given the CPU(s)/GPU, and the state then restored again.
This is harder than it may seem as the register count in modern GPUs
increases, and I could see where this could easily cause thrashing of "GPU application context switches" with SMP. The solution to reducing this effect might end up being a bit hairy and complicated, requiring us to keep track of what sections of the register space each application is actually using and what can be treated with a lazy policy. However if we don't do it all apps are going to have to be prepared to restore their contexts before any more of their (possibly already filled out) buffers are queued. That would mean the app would have to remember the context as it was between each buffer. We'd like to avoid that. That's a relatively far-future hurdle for us at this point, though, as now we're trying to settle on an internals rework and get our source tree out of the messy-bedroom state it is in right now.
Another complication is we have to do all the above in the face of safely dealing with monitor hotplugs and the prospect of migrating apps between surfaces on the same card, possibly surfaces on different GPUs and maybe even different *kinds* of GPUs someday. There will be a lot of work involved in correctly halting realtime accelerator queues and syncing with userspace in these instances.
The rest though is pretty straightforward: copy or map out a buffer so the app cannot munge it anymore, proofread it, check it for VSYNC qualifiers in a shared buffer control area (or at the chipset driver's author's discretion, use bogus commands as "escape sequences" embedded in the command stream), and if it passes, append it to the list of buffers to execute. On interrupt, manage the execution queues as necessary/appropriate, and return finished buffers to the app.
--
Brian S. Julin
Thanks
Thanks Brian - that was extremely informative!
Pete
Re: KGI - Kernel Graphics Interface
So, is KGI already merged with linux kernel 2.6 or will it be later, what are the plans because it is often a problem only having NVIDIA drivers or ATI drivers only for X.
What is the progress so far and will NVIDIA and ATI etc write drivers for KGI instead of X?