Timothy Miller offered a summary of the recent discussions surrounding his proposal to create a free software and open-source friendly graphics card [story], "I'm still trying to digest all the feedback I've been getting. It's overwhelming and gratifying, and I want to offer my gratitude for all the discussion and ideas." He noted that as the active discussion evolves toward the actual implementation of the card, his intention is to move it from the lkml to another more topical mailing list. He then went on to discuss the features that will most likley be in the new graphics card.

Timothy pointed out that the card would have a reprogrammable field-programmable gate array (FPGA) primarily to reduce non-recurring engineering (NRE) costs. He explains, "reprogramability is not its primary purpose. The primary reason to use an FPGA is to minimize NRE for manufacturing. However, as a result, users will be able to download updates. Additionally, those who are dedicated enough to reprogram it completely will find the necessary documentation to do so. Finally, it is my desire that we would release the source code to the FPGA for obsoleted products, however, it's too early to make promises."

He went on to further describe the Graphics Processor Unit (GPU), explaining, "what I'm thinking is that the setup engine will be [a] general-purpose-ish CPU with special vector and matrix instructions. This way, the transformation stage will occur in "software" executed by a specialized processor. Additionally, the lighting phase might be done here as well. The setup engine would produce triangle parameters which are fed to a rasterizer which does Gouraud shading and texture-mapping. That feeds pixels into something that handles antialiasing and alpha blending, etc." He noted that this offers three advantages: the engine is fully customizable by the user community simply by writing code, 2d emulation is included, and anything "missing" can be emulated. He also pointed out three disadvantages: the triangle rate would be limited by the speed of the processor, transform and lighting (T&L) functionality would be serialized, and phong shading and bump mapping "may be impossible or too slow". Read on for the full description.

From: Timothy Miller [email blocked]
To: Linux Kernel Mailing List [email blocked]
Subject: Some discussion points open source friendly graphics [was: HARDWARE: Open-Source-Friendly Graphics Cards -- Viable?]
Date: 	Mon, 25 Oct 2004 11:54:48 -0400

I'm still trying to digest all the feedback I've been getting.  It's 
overwhelming and gratifying, and I want to offer my gratitude for all 
the discussion and ideas.

It's been pointed out that some of this discussion may be getting 
off-topic for LKML.  My addition to that is that soon, it will get very 
specialized in graphics math, and it would be best to move to a list 
where people actually talk about this on a regular basis.

To that end, I'd appreciate it people could point me to some other 
appropriate mailing lists.  Not just the names, but URL's to the FAQ's 
which explain exactly how to subscribe, please.

Also, I'm thinking of starting my own yahoo groups list specifically for 
this chip.  Is that a good idea?

Next, I'm getting lots of ideas from people.  Some of them are core to 
the product, and some of them would be nice for follow-on products.  For 
instance, dual-video would not be on the first model released.  However, 
it is important that analog output always have crisp rise and fall times 
and be free of noise in order to maximize display quality.

The reprogramability of the FPGA has many advantages, but 
reprogramability is not its primary purpose.  The primary reason to use 
an FPGA is to minimize NRE for manufacturing.  However, as a result, 
users will be able to download updates.  Additionally, those who are 
dedicated enough to reprogram it completely will find the necessary 
documentation to do so.  Finally, it is my desire that we would release 
the source code to the FPGA for obsoleted products, however, it's too 
early to make promises.


Ok, now on to some design stuff:

The picture I have in my head at this time expands on the idea of the 
setup engine seen in most GPU's.  What I'm thinking is that the setup 
engine will be general-purpose-ish CPU with special vector and matrix 
instructions.  This way, the transformation stage will occur in 
"software" executed by a specialized processor.  Additionally, the 
lighting phase might be done here as well.

The setup engine would produce triangle parameters which are fed to a 
rasterizer which does Gouraud shading and texture-mapping.  That feeds 
pixels into something that handles antialiasing and alpha blending, etc.

The advantages are:

- The community can customize the setup engine as they please, just by 
writing code.
- This also includes the 2D emulation
- Anything "missing" can be emulated.

The disadvantages are:

- Triangle rate limited by speed of processor
- T&L is serialized, rather than being parallelized in dedicated hardware
- Phong shading and bump mapping may be impossible or too slow

It's been a while since I read about phong shading and bump mapping.  As 
I understand it, some of the lighting phase is pushed into the 
rasterizer.  With gouraud shading, colors are interpolated between the 
virtexes.  With phong shading, the surface normals are interpolated 
across the triangle, and that's used to compute lighting.  Bump mapping 
is like phong shading where the normals are specified in a bitmap.

What I don't know is how important bump-mapping and phong shading are.



From: karl.vogel@seagha.com
Subject: Re: Some discussion points open source friendly graphics
Date: 	Mon, 25 Oct 2004 22:31:59 +0200
 Obscurity, linux)

> It's been pointed out that some of this discussion may be getting
> off-topic for LKML.  My addition to that is that soon, it will get
> very specialized in graphics math, and it would be best to move to a
> list where people actually talk about this on a regular basis.
>
> To that end, I'd appreciate it people could point me to some other
> appropriate mailing lists.  Not just the names, but URL's to the FAQ's
> which explain exactly how to subscribe, please.

Something like this?
  http://www.opencores.org/forums.cgi/video/



From: Jeff Garzik [email blocked]
Subject: Re: Some discussion points open source friendly graphics [was: HARDWARE: Open-Source-Friendly Graphics Cards -- Viable?]
Date: 	Mon, 25 Oct 2004 16:34:45 -0400

Timothy Miller wrote:
> Also, I'm thinking of starting my own yahoo groups list specifically for 
> this chip.  Is that a good idea?

In theory it's useful; in reality at you'll have to balance what's 
public and what's not, and it's easy to allow yourself to get bogged 
down into "digesting", so much so that the "doing" is put off.

My only request is that you (a) make the graphics interface as simple 
and high level as feasible and (b) you publish the hardware interface 
specification as soon as possible, so that driver work can occur in 
parallel.


> Next, I'm getting lots of ideas from people.  Some of them are core to 
> the product, and some of them would be nice for follow-on products.  For 
> instance, dual-video would not be on the first model released.  However, 
> it is important that analog output always have crisp rise and fall times 
> and be free of noise in order to maximize display quality.

Once you have a core design, it's easier to dicker about specific 
features.  I would put this stuff on the "worry about it later" list.


> The reprogramability of the FPGA has many advantages, but 
> reprogramability is not its primary purpose.  The primary reason to use 
> an FPGA is to minimize NRE for manufacturing.  However, as a result, 
> users will be able to download updates.  Additionally, those who are 

Will the capability to apply these updates be included with the base card?
Will users need to purchase additional "update FPGA" hardware to do the 
reprogramming?


> Ok, now on to some design stuff:
> 
> The picture I have in my head at this time expands on the idea of the 
> setup engine seen in most GPU's.  What I'm thinking is that the setup 
> engine will be general-purpose-ish CPU with special vector and matrix 
> instructions.  This way, the transformation stage will occur in 
> "software" executed by a specialized processor.  Additionally, the 
> lighting phase might be done here as well.
> 
> The setup engine would produce triangle parameters which are fed to a 
> rasterizer which does Gouraud shading and texture-mapping.  That feeds 
> pixels into something that handles antialiasing and alpha blending, etc.
> 
> The advantages are:
> 
> - The community can customize the setup engine as they please, just by 
> writing code.
> - This also includes the 2D emulation
> - Anything "missing" can be emulated.
> 
> The disadvantages are:
> 
> - Triangle rate limited by speed of processor
> - T&L is serialized, rather than being parallelized in dedicated hardware
> - Phong shading and bump mapping may be impossible or too slow

Well, I certainly like it :)

I think that a more generic approach allows you to recognize performance 
bottlenecks, and update the IP core to support instructions specific to 
(say) triangles.

Random closing notes:

* data movement will be an everpresent issue

* in graphics you really have a number of data types (16-bit float, 
etc.) that are specific to graphics.  Supporting these natively should 
be quite helpful, if not an already-obvious prerequisite

	Jeff



From: Timothy Miller [email blocked]
Subject: Re: Some discussion points open source friendly graphics [was: HARDWARE: Open-Source-Friendly Graphics Cards -- Viable?]
Date: 	Mon, 25 Oct 2004 18:45:44 -0400



Jeff Garzik wrote:
> Timothy Miller wrote:
> 
>> Also, I'm thinking of starting my own yahoo groups list specifically 
>> for this chip.  Is that a good idea?
> 
> 
> In theory it's useful; in reality at you'll have to balance what's 
> public and what's not, and it's easy to allow yourself to get bogged 
> down into "digesting", so much so that the "doing" is put off.

Right.  I was just discussing this with the CEO, and he pointed out that 
I would get bogged down with too much online discussion.  At FIRST, I 
might be able to handle an open list, but pretty quickly, I'd have to 
create an invite-only list where a select group who reads the other list 
would digest things for me.

Yeah, if I'm going to design a large chunk of the chip and write a large 
chunk of the software, I can't spend all day reading a mailing list.  :)

> 
> My only request is that you (a) make the graphics interface as simple 
> and high level as feasible and (b) you publish the hardware interface 
> specification as soon as possible, so that driver work can occur in 
> parallel.

Yes, and what I need to do is find the right developers to talk to so 
that I can given them interfaces that they actually NEED, because right 
now, I'm just guessing.

> 
> 
>> Next, I'm getting lots of ideas from people.  Some of them are core to 
>> the product, and some of them would be nice for follow-on products.  
>> For instance, dual-video would not be on the first model released.  
>> However, it is important that analog output always have crisp rise and 
>> fall times and be free of noise in order to maximize display quality.
> 
> 
> Once you have a core design, it's easier to dicker about specific 
> features.  I would put this stuff on the "worry about it later" list.

Definately.  There's loads of features that are NICE, but not useful to 
most people.  I need to make the lowest common demoninator first.

> 
>> The reprogramability of the FPGA has many advantages, but 
>> reprogramability is not its primary purpose.  The primary reason to 
>> use an FPGA is to minimize NRE for manufacturing.  However, as a 
>> result, users will be able to download updates.  Additionally, those 
>> who are 
> 
> 
> Will the capability to apply these updates be included with the base card?
> Will users need to purchase additional "update FPGA" hardware to do the 
> reprogramming?

My intention is to include a bit of logic in the FPGA which can be used 
to reprogram the prom.  You would then cycle power to get the FPGA to 
load the new code.  Only if it gets corrupted would you have to solder 
on a header and reprogram it another way.

Also, if someone decides to reprogram it completely, and they haven't 
included a host interface and prom programmer in their design, then, 
well, they're on their own.  :)

> 
>> Ok, now on to some design stuff:
>>
>> The picture I have in my head at this time expands on the idea of the 
>> setup engine seen in most GPU's.  What I'm thinking is that the setup 
>> engine will be general-purpose-ish CPU with special vector and matrix 
>> instructions.  This way, the transformation stage will occur in 
>> "software" executed by a specialized processor.  Additionally, the 
>> lighting phase might be done here as well.
>>
>> The setup engine would produce triangle parameters which are fed to a 
>> rasterizer which does Gouraud shading and texture-mapping.  That feeds 
>> pixels into something that handles antialiasing and alpha blending, etc.
>>
>> The advantages are:
>>
>> - The community can customize the setup engine as they please, just by 
>> writing code.
>> - This also includes the 2D emulation
>> - Anything "missing" can be emulated.
>>
>> The disadvantages are:
>>
>> - Triangle rate limited by speed of processor
>> - T&L is serialized, rather than being parallelized in dedicated hardware
>> - Phong shading and bump mapping may be impossible or too slow
> 
> 
> Well, I certainly like it :)
> 
> I think that a more generic approach allows you to recognize performance 
> bottlenecks, and update the IP core to support instructions specific to 
> (say) triangles.
> 
> Random closing notes:
> 
> * data movement will be an everpresent issue
> 
> * in graphics you really have a number of data types (16-bit float, 
> etc.) that are specific to graphics.  Supporting these natively should 
> be quite helpful, if not an already-obvious prerequisite


Yeah, for the transformation phase, you need 32-bit floating point. 
After that, you can switch to 16.16 fixed point and get perfect results.


From: Timothy Miller [email blocked]
Subject: Need help and advice... [was: HARDWARE: Open-Source-Friendly Graphics Cards -- Viable?]
Date: 	Mon, 25 Oct 2004 19:06:41 -0400

I'm probably doing this too early, but it's been pointed out in so many 
words that I won't scale.  I've spent two full work days digesting all 
of the discussions on LKML, kerneltrap, and slashdot.  That's been as 
fun as you can imagine, but when the time comes for me to start working 
on the design, I can't do that at all anymore.

What I'll need are representatives of the community who work with me to 
make sure that my design does what everyone needs it to do, and I need 
representatives who engage in mailing lists and extract for me 
information that I should see.

I'm sure there are lots of people who would love to have that kind of 
influence over a company whose hardware they want to buy.  I need the 
subset of those people who have expertise in 3D graphics, most of whom 
I'm sure are on OpenGL and X.org mailing lists right now.  If there's 
anyone here who can get me connected with those people, I would 
appreciate it very much.


I feel like I'm being presumptuous.  Although everyone at Tech Source is 
excited by the idea, and there seems to be a ground-swell of interest, 
the economic feasibility of this is still in the air.  I don't want to 
waste anyone's time.  However, a lot of excellent ideas have been put 
forth, and I'd like discussion to continue, but in a way that I can handle.

I humbly request advice on how I might go about handling all of this in 
the best way.

I also think the discussion should transition off LKML and onto its own 
forum.

Also, I want to thank everyone for talking to me about this.  Members of 
this list and people posting to kerneltrap and slashdot have been very 
informative and helpful.  I also want to mention Jeremy Andrews of 
kerneltrap.org who first turned my posting into news.  I've wanted to do 
a project like this for a long time, but the past few days of online 
discussion have really helped push it towards becoming reality.

• Archive of above thread