[vlc-devel] [PATCH 00/31] video context in the decoder

Steve Lhomme robux4 at ycbcr.xyz
Fri Jul 5 16:19:39 CEST 2019

Here is a new patchset for the decoder device/video context usage.

Following the previous patchset and discussion with Thomas, the decoder device
is now created once when the window is enabled (and released when the window is
disabled). On Windows it's not necessary but on other systems it may be needed
to be attached to a particular display.

The "dec-device" module is now loaded before the VA and before the display
module. The plan is to remove the "avcodec-hw" variable and only allow VA
modules matching the "dec-device" they receive instead.

For the display it's more tricky as we may still want to be able to select
a different display module even if the dec-device is set or automatically
picked. In this case it should accept video context in other types that the
one it would expect (D3D11 and D3D11VA for example). In software decoding the
video context won't provide pictures from a HW pool anyway. So they still have
to accept and handle that.

And because of external rendering (D3D callbacks) we need to pass the decoder
device to the display module so it matches what the host requested (otherwise
it just won't work). For now it's passed inside the video context but it could
change and be passed within the vout since it keeps track of its decoder device

This patchset splits the vout/display module creation as discussed before so that
the video context can be created and provided to the display module, including
when it's created by the VA.

There are 2 decoder device for D3D11 and for D3D9 but I think they can be merged.

You can find the patches and more work on the video context in this branch:

This branch effectively removes the decoder pool from the display module.
Each decoder now has its pool. Either coming directly from the (avcodec)
VA pool, or using a local CPU-based pool for the decoder.

Steve Lhomme (31):
  vout: create/release the decoder device when the window is
  decoder device: the D3D9 device needs to store a IDirect3D9 and an
  direct3d9: add a decoder device when using external rendering
  direct3d9: create a decoder device when the local swapchain is used
  decoder device: use a structure to read the opaque of
  decoder: let the decoder device know it's force created
  direct3d11: add a decoder device when using external rendering
  direct3d11: create a decoder device when the local swapchain is used
  vout: add a getter for the decoder device of the vout
  decoder: get the decoder device from the vout
  decoder: split the decoder format update in 2 parts
  video_output: split the code after the window/decoder device creation
  input: resource: split the call to vout_Request()
  audio filters: split the call to vout_Request()
  vout: vout_Close() should be used when vout_Create() used
  vout: remove unused vout_Request
  input: resource: split input_resource_GetVout()
  decoder: move the video_format cleaning for the display in a function
  decoder: split the creation of the vout/decoder device and the display
  decoder: keep the decoder device in the decoder
  vout: request the display using a video context
  video context: make the video context structure private
  decoder: create a vlc_video_context in the decoder
  decoder: provide the vlc_decoder_device from the vout thread to the VA
  decoder: get the vlc_video_context created by the VA
  d3d11va: create a video context for the VA
  dxva2: create a video context for the VA
  video context: allow storing extra data in the video context
  dxva2: store the IDirect3DDevice9 in the video context
  d3d11va: store the ID3D11DeviceContext in the video context
  avcodec: split the calls to decoder_UpdateVideoFormat()

 include/vlc_codec.h                           |  56 ++++-
 include/vlc_picture.h                         |  34 ++-
 include/vlc_vout.h                            |   2 +-
 include/vlc_vout_display.h                    |   4 +-
 lib/media_player.c                            |   2 +
 modules/codec/avcodec/d3d11va.c               |  65 +++--
 modules/codec/avcodec/dxva2.c                 |  44 +++-
 modules/codec/avcodec/va.c                    |  11 +-
 modules/codec/avcodec/va.h                    |   4 +
 modules/codec/avcodec/vaapi.c                 |   7 +-
 modules/codec/avcodec/video.c                 |  28 ++-
 modules/hw/d3d11/Makefile.am                  |   1 +
 modules/hw/d3d11/d3d11_device.c               | 104 ++++++++
 modules/hw/d3d11/d3d11_filters.c              |  11 +
 modules/hw/d3d11/d3d11_filters.h              |   4 +
 modules/hw/d3d9/Makefile.am                   |   1 +
 modules/hw/d3d9/d3d9_device.c                 | 101 ++++++++
 modules/hw/d3d9/d3d9_filters.c                |   5 +
 modules/hw/d3d9/d3d9_filters.h                |   4 +
 modules/hw/vaapi/decoder_device.c             |   3 +-
 modules/hw/vdpau/avcodec.c                    |   3 +-
 modules/video_chroma/d3d11_fmt.h              |  39 +++
 modules/video_chroma/d3d9_fmt.h               |  39 +++
 modules/video_output/opengl/converter.h       |   4 +-
 modules/video_output/opengl/converter_vaapi.c |  24 +-
 modules/video_output/opengl/converter_vdpau.c |  28 ++-
 modules/video_output/opengl/vout_helper.c     |   2 +-
 modules/video_output/splitter.c               |   7 +-
 modules/video_output/win32/d3d11_swapchain.c  | 103 +++++---
 modules/video_output/win32/d3d11_swapchain.h  |   7 +-
 modules/video_output/win32/direct3d11.c       |  64 ++---
 modules/video_output/win32/direct3d9.c        | 141 +++++++----
 src/audio_output/filters.c                    |  24 +-
 src/input/decoder.c                           | 224 +++++++++++-------
 src/input/decoder_helpers.c                   |  94 +++++++-
 src/input/resource.c                          |  29 ++-
 src/input/resource.h                          |   4 +-
 src/libvlccore.sym                            |   9 +-
 src/video_output/display.c                    |  13 +-
 src/video_output/video_output.c               |  44 +++-
 src/video_output/vout_internal.h              |  25 +-
 src/video_output/vout_wrapper.c               |   7 +-
 42 files changed, 1108 insertions(+), 317 deletions(-)
 create mode 100644 modules/hw/d3d11/d3d11_device.c
 create mode 100644 modules/hw/d3d9/d3d9_device.c


More information about the vlc-devel mailing list