[vlc-devel] Developing decode/encode plug-in: NV12/D3D

[Laurent Aimar]

n Fri, Dec 10, 2010 at 06:27:43AM +0200, Rémi Denis-Courmont wrote:
> On Friday 10 December 2010, Larsson, Petter wrote:
> > It seems the NV12 format is not fully supported by VLC. It's listed in
> > vlc_fourcc.h but it's not supported in vout_pictures.c which makes it
> > impossible to create (decoder_NewPicture(...)) a picture object for the
> > purposes of decode surface creation.
> 
> Yeah. Proper support for a new chroma involves quite a bit of work, and in 
> principles testing/validation.
> 
> > A non optimal solution is to convert NV12 to/from YV12, this works fine but
> > this is obviously a large CPU hit.
> 
> Yes :-( On the bright side, that stuff is very prone to vectorization. In any 
> case, slow software conversion is hardly avoidable until at least one video 
> output plugin provides native/hardware support for NV12.
 In DXVA2, I had to copy the surface from GPU to system memory. Doing a NV12 copy
or doing a copy while converting to YV12 is about the same CPU wise. So it was
not an issue. In avcodec/copy.c you have an optimized NV12->YV12 convert/copy
function (for reading from CPU memory, but it should not be bad when reading from
system).
 Besides, NV12 is not supported by our filters, nor when blending subtiles
and would need a lot of works. (Nothing wrong with the code itself, it's
just that you have to add support to every filter that you want).

> > Hacking vout_pictures.c I did add NV12 format as a new format with the same
> > planar setup as YV12, but this solution is not entirely correct since NV12
> > is a hybrid planar format (albeit the same size as YV12). This solution
> > works for the transcode case where both my decoder and encoder is used.
> > 
> > If an existing VLC encoder (such as x264) is used together with my NV12
> > type decoder for transcode it works but since the encoder does not expect
> > NV12 format resulting in an output has the UV plane colors all wrong.
> > 
> > What is the stance on NV12 support in VLC? Am I missing something here?
 Patch are welcome. As up to now it wasn't needed it wasn't done, but there
would be no probalem to integrate a patch supporting it. But take note of my
previous remark about video filters/subtitles.

> At this point in time, NV12 is only used with avcodec VAAPI/DxVA2 decoders. In 
> particular, it is not supported by any video outputs.
> 
> So I guess Laurent found it much simpler to convert on-the-fly to/from YV12.
> 
> Of course, if proper NV12 turns out to be useful, then it should be added to 
> the core.
> 
> > Regarding using the NV12 decoder for the playback case, it does appear that
> > NV12 rendering is understood (at least from the log). But I'm currently
> > not able to fully verify this due to some odd behavior (bitstream blocks
> > delivered for the playback case differs from the transcode case. For
> > playback the blocks are missing header info, and as it seems start codes)
 I will depend on the vout and the driver. I think you can hope that direct3d
support NV12 directly, but I have never tested it.

> > D3D frame surface support:
> > ---------------------------
> > Is there a way to use D3D surfaces instead of system memory surfaces for
> > decoder and encoder plugin frame buffers. I confess, I have not been
> > digging deep here but maybe some can give some pointers on where to find
> > more info or dismiss it altogether.
> 
> If you mean to ask whether you can fetch Direct3D handles to surfaces, then 
> no. VLC (currently) follows a rather strict separation and abstraction of 
> tasks. Using D3D surfaces would tangle decoding and output.
> 
> That said, if the active video output happens to be Direct3D, *and* if the 
> decoder raw video output format is supported by the Direct3D output, then the 
> picture buffers will be Direct3D surfaces.

 Not easy. One problem is that direct3d surfaces (it depends on the type of
surface used, I think) can be marked as invalid at any time by the OS (usually
when you switch of screen). In this case I think you loose the content of the
surface AND you have to close their handles and recreate them. I don't know
d3d very well, so I can be wrong.

Regards,

-- 
fenrir