[vlc-devel] RFC: vout system

[Derk-Jan Hartman]

Hi rocky and others

Rocky, I'm starting to understand a bit better all the things you said 
when you started on your subtitles :)
It's becoming more and more obvious to me that we need to spend some 
serious time redesigning the entire 'advanced' features set of the 
video system.

As far as I can see almost all the API parts of VLC have had several 
major redesigns. input, demux, codec, aout etc. Even playlist is 
getting a redesign now (though not entirely successful yet :)
The thing that has been untouched for a long time now though is the 
entire vout system. Perhaps it is time to work on it again.

Some relevant points:
- users should be able to choose easily between aspect ratio's (4:3 
16:9 23:? and free aspect ratio), stretching, cropping and padding. 
This needs to be brought back to a more essential part of the drawing 
system. I know Gibalou actually already spent some time investigating 
this. Please enlighten us with your ideas and suggestions? :)

- everything spu related sucks... it has to change.
- we need to do blending at a higher level and have general methods for 
this. chroma issues need to be dealt with. We have chroma conversion 
routines. can't we use these on spu's as well?? rocky actually already 
largely did this at the level of his own modules.

- we should think about multiple 'layers' of spu. We could then draw 
multiple spu's on top of each other (prioritized)
- we should try to see if we can think of a way that would allow us to 
draw outside the real video. Also to make spu resolution independent 
from normal pictures. That way you could have full resolution subtitles 
in fullscreen for instance. Now they are blended into the actual video, 
so after scaling to fullscreen they can be quite pixelated, which is a 
bit of a waste.

- I know jack about it myself, but apparently the filters need some 
serious work? Let's make a list of what we would like it to be able to 
do. I know having the filters work on sout is on that list. what more?

- multiple vout's, multiple inputs, multiple playing tracks at once, 
visualizations. Much of this is a bit of a hacky job now (if possible 
at all). video integration into a intf window is also something to be 
kept in mind.

- next, besides our text rendering engine, we need a general image 
rendering routine. Probably png based. This engine should be able to 
handle scaling, aspect ratio's and other annoying stuff. It could be 
used for image based subtitles, as well as to create a TRUE OSD intf 
that renders menu's etc.

For this we need to collect as much data on every sort of subtitle and 
spu drawing system and make a list of all the requirements that DVD/VCD 
menus, MPEG4 menu's, text subtitles, rle based subtitles and for 
instance a OSD have.
I'm thinking palette's, scaling, positioning, timing, aligning, 
encoding, hi-lighting etc etc etc. Then define structures and locations 
to hold this information, while preventing duplicity.
The same goes for video-filter, goom, and spectrum analyzers etc etc 
etc.

At the moment there are more important things to handle, like fixing 
the playlist API and fen's work on demux/input/access/tracks
However do more people agree that this is something we need to work on 
for our 0.8.0 and 0.9.0 releases? I know parts of all this have already 
been on gibalou's, Dnumgis and rocky's wishlist. (sam, i know you can 
make some good contribution to this as well). I think we are now at 
that place where we can do a major redesign, because we KNOW what we 
want. We just need to put all the knowledge together.....
I'm thinking a wiki like design document might be a good idea. Then 
when the other API's have been trough their updates, we can take that 
and 'just' implement it.

DJ

P.S. hmm, this mail became a bit larger than i had expected... :D


On 28 jan 2004, at 01:22, R. Bernstein wrote:
> hartman suggests:
>>   Vobsub is working now. only track 0 will be played atm.
>>   spudec scaling will need to be implemented or vobsub is pretty 
>> useless.
>
> Actually, scaling is done in the VCD versions of this - see
> codec/ogt. If you need to reduce scale along the X axis (for I420 or
> YUY2) you can probably use the routine ScaleX from common.c. And the
> VCD version handles all the color planes rather than just the Y plane;
> also blending is done for some of the chromas. And more chromas are
> handled. (e.g. RGB2).
>
> And I bet those routines are little faster than than the ones in
> spudec. For cropped images we terminate loops earlier or start them
> later. And even when no cropped images, we loop over pixels in the
> input subtitle image rather than the output scaled (enlarged) image,
> so when a pixel is skipped over we are skipping over several pixels in
> the scaled image.
>
> A possibility then is to use the routines from ogt. But for this to
> happen, the spudec RLE should be expanded before blending. This is
> probably a good thing to do anyway; a subtitle is rendered several
> times per subtitle, but expansion need only be done once.
>
> -- 
> This is the vlc-devel mailing-list, see http://www.videolan.org/vlc/
> To unsubscribe, please read http://developers.videolan.org/lists.html
> If you are in trouble, please contact <postmaster at videolan.org>
>
>
---
Videolan - VLC media player
Derk-Jan Hartman (hartman at videolan dot org)
Developer of the MacOS X port of vlc
http://www.videolan.org/vlc

-- 
This is the vlc-devel mailing-list, see http://www.videolan.org/vlc/
To unsubscribe, please read http://developers.videolan.org/lists.html
If you are in trouble, please contact <postmaster at videolan.org>