[vlc-devel] RFC: vout system (Also long)

[R. Bernstein]

Some random comments on recent subtitle experiences. Sorry this is
long.

First, I know I made it sound like some of the OGT routines were the
greatest thing thing since sliced Wonder(tm) bread (which I find
disgusting). There are still some problems with it namely: 
-  There are even more chromas, like the recent altivec work
-  There almost certainly are big-endian problems 
-  testing has been a difficult, so expect bugs
-  fullscreen on RGB16/24/32 doesn't always work.
-  RGB2 needs to have better access to the 8-bpp colormap

While not *strictly* a problem, there's a bit of cut-and paste code
that probably should be modularized better, and there's code that
probably could be used other places (like the routine to write a PNG
via libpng) or should be using code from other places (like the
yuv2rgb routines). 

It's just that at present I think what is in ogt is slightly better
than than spudec. Perhaps it's just because it is newer and draws on
that code base.

- - - - -

However after doing this much, while in some areas I have the same
beliefs about a redesign, in other areas I think my views have changed.

Here's what's *not* changed:

1. The subpicture data-structure while it might be fine for some
   things, isn't well suited as it is for extracting pixel/chroma
   information such as might be used in blending a subpictures and a
   picture together.

(A small change suggestion to the subpicture: add a boolean to
indicate whether the subpicture is intended to be scaled before
rendering or not.)

2. I do think it possible to have more video things reusable. Somehow
   more of a library there.

3. There are two approaches one can take, a "top-down" design,
   assuming as DJ said we really understand what we did wrong and have
   a vision as to how to fix. (But I'm more getting the sense that
   that may not in fact be the case). A "bottom-up" design which is
   what I've been doing. Here you get a real problem solved which is
   nice, and one begins to understand the complexities involved. Which
   leads to...

Here's some views that *have* changed:

1. The vout problem may be tougher than I had originally thought. A
   lot of that code is tricky and error prone which is why it would be
   great to have reusable modules. But I don't mean tricky necessarily
   in this sense.

   Before undertaking the code in codec/ogt/render.c, I looked at many
   other projects like libpng, transcode, mpeg2dec, xine, and vlc to
   see if some other project had yuv2rgb conversion routines, scaling
   and blending encapsulated. The answer is sort of (to varying
   degrees of success depending on the project) but not really, and
   not in a way that any other project might use. Nothing coming close
   to a library that I thought I could use. So it seems there's lots
   of cut and paste. I bet that's also true between vlc and libcaca
   even though these may have been largely (if not totally) written by
   the same person.

   At any rate, I wonder that if the problem were that simple why
   hasn't it been done? 

2. I thought there could be more of a "universal" format for say
   "subpictures". For example perhaps make each subtitle pixel YUV,
   and transparency. But it seems that the format of the video to be
   rendered can (and probably should) effect how to store the
   subtitle. In particular, you are going to blend to a palettized RGB
   chroma like RGB2, no sense in converting the subtitle which started
   with a colormap into an unpalettized form. So here you may want to
   pass along the pixels in colormap form plus the color map
   (containing transparency) rather than individual YUV + T pixels.

3. Sort of an extension of 2 above, I thought one could write some
   generic routines and through the magic of conversions there might
   not be that many of them. Think the compiler picture of multiple
   front-ends and multiple back-ends and one intermediate
   language. You see this in the transcode diagram here;
   http://www.theorie.physik.uni-goettingen.de/~ostreich/transcode/html/tc.png

   Well, as I've have alluded to in the past, blending VCD/DVD
   subtitles onto a video image are a special kind of blending. The
   palette for these "subpictures" is pretty small, even if you
   consider simple antialiasing or color averaging as a result of
   aspect-ratio scaling. Since they are text, you might want to use
   text-like algorithms on them (e.g. for antialiasing, but I've
   avoided that for now).

   I'm not suggesting the problem is hopeless. It isn't. It's just
   larger than I originally thought. 

   An example, Right now I have *two* scaling routines. One is used in
   YUY2 and I420 (actually to *undo* the effect of these) which is
   done before rendering. It downscales along the x direction and only
   works for an aspect ratio between 1/2 and 1 (the inverse of 4:3 or
   16:9). And there is another that's part of the RGB16/RGB24/RGB32,
   done *during* scaling which upscales everything. Oy.

-- 
This is the vlc-devel mailing-list, see http://www.videolan.org/vlc/
To unsubscribe, please read http://developers.videolan.org/lists.html
If you are in trouble, please contact <postmaster at videolan.org>