[vlc-devel] [PATCH 0/2] Direct Rendering pool size

[Steve Lhomme]

With Direct3D11 it seems there's a hard limit of 30 buffer slices you can use
with ID3D11VideoDecoderOutputView. That's the idea behind this line:
http://git.videolan.org/?p=vlc.git;a=blob;f=modules/video_output/win32/direct3d11.c;h=f720da46adca440133234caee661edef05397e57;hb=HEAD#l637

I had crashes on AMD graphics card and now I have some with NVIDIA as well.
Anything up to 30 is fine.

We used to allocate the decoder buffer in the dxva code but it doesn't work
on some AMD cards when decoding 4K video (the ones where we really want hardware
decoding) due to internal memory limitations. So it's not possible to do that
as long as the vout wrapper will force the vout to allocate the whole pool for
decoders, filters, etc. I think this design should be changed but as there's
work planned for the picture pools for 4.0, this may be delayed.

For now I have 2 patches that allow playing some files, I mostly hit an issue
on HEVC files. Because the decoder pool has a certain size and the avcodec
decoder needs its whole pool in one time, we are only left with 3 pictures
available to send the decoed data to the vout. In many cases the decoder stalls
on picture_pool_Wait() because there's no picture available. At this point
nothing gets decoded and/or sent to the vout.

The directx_va patch reduces the number of buffers passed to the avcodec
hardware decoder. It's 2+spb_size+threads instead of 4+spb_size+threads. I
don't know where this 2 magic number comes from. But if that's some extra
buffering, 2 (input/output) should be enough.

The change in vout_wrapper assumes (correctly AFAIK) that there won't be any
filter used when using direct rendering. And so we don't need to reserve 3
buffers for that.

Steve Lhomme (2):
  directx_va: reduce the number of necessary decoding buffers
  core: don't reserve decoder pictures for filtering in direct
    renderering

 modules/codec/avcodec/directx_va.c | 2 +-
 src/video_output/vout_wrapper.c    | 3 ++-
 2 files changed, 3 insertions(+), 2 deletions(-)

-- 
2.10.1