[vlc-devel] [vlc-commits] avcodec: fix direct rendering with some codecs

[Laurent Aimar]

On Sun, Dec 11, 2011 at 01:03:33AM -0500, Rafaël Carré wrote:
> Le Sun, 11 Dec 2011 00:29:54 +0100,
> Laurent Aimar <fenrir at elivagar.org> a écrit :
> 
> > On Sat, Dec 10, 2011 at 06:07:21PM -0500, Rafaël Carré wrote:
> > > Le Sat, 10 Dec 2011 23:43:22 +0100,
> > > Laurent Aimar <fenrir at elivagar.org> a écrit :
> > > 
> > > > On Sat, Dec 10, 2011 at 11:00:07PM +0100, Rafaël Carré wrote:
> > > > > vlc | branch: master | Rafaël Carré <funman at videolan.org> | Sat Dec 10 16:58:35 2011 -0500| [fcc84608b67f3c3567b96604577bfdcc0de45b53] | committer: Rafaël Carré
> > > > > 
> > > > > avcodec: fix direct rendering with some codecs
> > > > > 
> > > > > Provide edge buffers when CODEC_FLAG_EMU_EDGE is set
> > > > > Fix ProRes playback with direct rendering
> > > > 
> > > >  From what I understand of avcodec, this is wrong. We need edge only when
> > > > CODEC_FLAG_EMU_EDGE isn't set while we always set it.
> 
> Reverted.
> 
> > > Hm right, EMU_EDGE shouldn't draw in this border.
> > > 
> > > I'm not sure why this fixes ProRes decoding when EMU_EDGE is set,
> > > perhaps libav doesn't respect the flag, or perhaps video buffers
> > > provide the edges already?
> > 
> >  Either the prores decoder doesn't respect EMU_EDGE or it expects a greater
> > buffer/pitch alignment than we provide.
> 
> We use different picture sizes for vout and decoder when using direct
> rendering.
No. The pictures than a decoder in VLC receives can comes from 2 sources:
 1. the vout.
 2. picture_New() (through a picture_pool I think)

In the case 1., we don't always control the properties of the buffers except
that it will be large enough for holding width/height (ie we may not have
sufficient data pointer/pitch alignment).
That's partially why we have a compatibility check in ffmpeg_GetFrameBuf().

> vout pictures are aligned on requested width/height, but decoder
> pictures width/height are modified by picture_Setup()
> 
>     /* We want V (width/height) to respect:
>         (V * p_dsc->p[i].w.i_num) % p_dsc->p[i].w.i_den == 0
>         (V * p_dsc->p[i].w.i_num/p_dsc->p[i].w.i_den * p_dsc->i_pixel_size) % 16 == 0
>        Which is respected if you have
>        V % lcm( p_dsc->p[0..planes].w.i_den * 16) == 0
>     */
 Not always, picture_Setup() set what VLC wants, but the vout may not be using
it or it can always force other values (ex: xvideo does not use it, see
PoolAlloc() in xcb/xvideo.c).

> IIUC we modify picture width so all planes are modulo 16 (IIUC), so
> when there is chroma subsampling, even if the width is modulo 16, the
> chroma planes are not necessary.
 I don't understand this part/what you mean.

> pointers welcome
 I am a bit dubious about the fact that the proress decoder in ffmpeg doesn't
seem to check the value of CODEC_FLAG_EMU_EDGE. (h264 decoder does for
example).

-- 
fenrir