IMHO...
The reason why there is no single 'format conversion library' that
everybody uses is because of the large differences between requirements
for such a thing. The line between 'format conversion' and things such
as a video codec, or image processing is very vague. For example: Some
apps just want compressed video format output. Would video compression
be part of such a 'format conversion library'? If so, then which
compressed video formats? Proprietary ones too? If not all formats that
exist are supported, it would not be complete for some or many apps, and
maybe not even useful at all (because integrating any necessary
pixel-format conversion into the compression routines may be more
efficient than making multiple passes over the data with separate
libraries). Will the library include resizing? If so, which resampling
algorithms? Then how about rotation? Then maybe geometrical mapping
(games could want that)? Will the library be able to adjust
brightness/contrast/saturation in software? If so, then what about noise
filtering (which algorithms?), etc... perhaps the library should go all
the way, bind to port 80 and serve-up a live video stream
'youtube-style'? ok, now that would definitely go too far...
The question is: Where exactly to put the boundary?
My point is that format conversion is not a video capture driver issue.
Sure, it is convenient for apps to be able to use standard libraries
that perform certain functions with optimized code, etc, but for the
purpose of capturing video (media) it's not always necessary to convert
the output into something different before the data is useful for an app.
Certain format conversions can be done very efficiently inside video
cards, for example, and an app may prefer to use that. If a video card
has such functionality, access to it should be part of its driver.
Applications needing format conversions would benefit tremendously from
a good, powerful, flexible, efficient, etc, library that removes the
necessity for each application to choose between using a video-card
accelerator, or MMX code, or SSE code, or another method to do the work.
Maybe it should allow applications to request a 'media stream' in a
given format and container, with conversion automagically happening when
needed, as efficiently as possible (or at the requested quality level).
But that doesn't mean that that library should be an integrated part of
the video capture interface. The boundary of the 'media capture
interface' driver should be on the data, as delivered by the video
capture interface.
The reason is that the 'best' way to do format conversions depend on the
application requirements (what formats can the app use directly?), on
the available hardware (is there an accelerator that can help?), and on
the quality expectation or system issues (is the CPU the bottleneck or
is the USB bus saturated? Do we request JPEG from the webcam and
decompress into YUV, or do we request the RAW bayer from the
high-quality image sensor and convert that into YUV?). Choices, choises...
That means that IMHO, to prevent having runaway complexity plus chasing
a moving target, format conversion should definitely not be embedded in
the API to access video capture hardware, unless the hardware itself
offers format conversion functions (for example, as it does now, by
allowing choice of the output format, but only for formats that the
hardware itself supports directly).
It may make things 'easy' if it makes it easy to use existing (?) code
from other sources, but that's not a reason why such an interface is
technically superior. For example, 'ndiswrapper' makes it very easy to
use certain wireless card drivers, but that doesn't mean (by far) that
ndiswrapper's method results in the best wireless card drivers for Linux.
It sounds to me that your approach is more similar to ndiswrapper, for
practical reasons 'this is how I can make it work with less effort' than
it is similar to a full-blown Linux driver 'this is the best way to make
a reliable and efficient driver for this kind of hardware'. Linux
_users_ may need to use ndiswrapper right now, but Linux itself needs
real drivers, not a compromise.
Just my 2cts...
Jelle.
-
