[vlc-devel] [PATCH] Added video bridge output plugin

Oliver Collyer ovcollyer at mac.com
Sun Jul 2 10:34:39 CEST 2017


Thanks for your polite response Thomas - no problem at all.

Just offering in case it was of interest :)

All the best

> On 2 Jul 2017, at 11:26, Thomas Guillem <thomas at gllm.fr> wrote:
> 
> 
> 
> On Sat, Jul 1, 2017, at 14:00, Oliver Collyer wrote:
>> Hello
>> 
>> A little out of left-field this one but I wrote this vout plugin for a
>> project I've been working on.
>> 
>> It's kind of similar to vmem in that an application can use this plugin
>> to retrieve the decoder/filtered surfaces, but rather than receive a
>> system memory copy of the surface (clean but slow) it instead allows
>> these surfaces to be passed through directly to the caller. More
>> importantly, it supports this for certain hardware formats too -
>> currently DXVA2, VideoToolbox and VAAPI. There is no reason why D3D11VA,
>> VDPAU etc couldn't be supported in future.
>> 
>> I see this plugin as filling a small niche where an application already
>> manages its own devices, contexts, textures etc and wants to embed video
>> into this, but without losing the benefits of hardware acceleration.
>> 
>> Im my video streaming application I take these hardware surfaces and use
>> them to fill out SDL textures, thus avoiding any GPU-CPU-GPU copies. For
>> DXVA2 this is done via StretchRect, for VideoToolbox I used the texture
>> cache on both iOS/tvOS and macOS. For VAAPI I use the GLX extension. I
>> had to add in a couple of functions to SDL to give deeper access to the
>> underling devices/textures/surfaces, but it works very well.
>> 
>> Initially I was hacking these changes into vmem but rather than break
>> that interface or get bogged-down trying to support two different methods
>> I thought it cleaner to write this as a new plugin.
>> 
>> I understand from jb that there are possible plans in future for a plugin
>> that writes to a texture, so maybe my plugin will be redundant, I'm not
>> sure, but I'm offering it anyway, in case you want to include it, or in
>> case someone wants to use it in their own project.
> 
> Thanks for offering it to the community. I'll keep this patch in a
> branch. A lot of people already asked us to do something like that. But
> unfortunately, we won't merge it. As jb already said, we'll add a way to
> get access to the opengl/direct3D texture for 4.0.
> 
> 
> Best regards,
> 
> Thomas.
> 
>> 
>> Cheers
>> 
>> Oliver
>> 
>> ---
>> include/vlc/libvlc_media_player.h | 108 +++++++
>> lib/libvlc.sym                    |   1 +
>> lib/media_player.c                |  32 ++
>> modules/MODULES_LIST              |   1 +
>> modules/video_output/Makefile.am  |  33 ++
>> modules/video_output/vbridge.c    | 615
>> ++++++++++++++++++++++++++++++++++++++
>> po/POTFILES.in                    |   1 +
>> 7 files changed, 791 insertions(+)
>> create mode 100644 modules/video_output/vbridge.c
>> 
>> diff --git a/include/vlc/libvlc_media_player.h
>> b/include/vlc/libvlc_media_player.h
>> index 160cdfbaa8..72be05bb9b 100644
>> --- a/include/vlc/libvlc_media_player.h
>> +++ b/include/vlc/libvlc_media_player.h
>> @@ -457,6 +457,114 @@ void libvlc_video_set_format_callbacks(
>> libvlc_media_player_t *mp,
>>                                         libvlc_video_cleanup_cb cleanup
>>                                         );
>> 
>> /**
>> + * Callback prototype to configure format for video bridge output.
>> + *
>> + * This callback gets the format of the video as output by the video
>> decoder
>> + * and the chain of video filters (if any). It can opt to change any
>> parameter
>> + * as it needs. In that case, LibVLC will attempt to convert the video
>> format
>> + * (rescaling and chroma conversion) but these operations can be CPU
>> intensive.
>> + *
>> + * The device pointer must be set to a valid D3D9 device in the case of
>> DXVA2 or
>> + * a valid VADisplay in the case of VAAPI. This is used by libVLC to
>> initialize
>> + * the hardware decoder and create the hardware surfaces that are later
>> passed
>> + * to the prepare and display callbacks.
>> + *
>> + * \param opaque pointer to the private pointer passed to
>> + *               libvlc_media_player_set_vbridge_callbacks() [IN/OUT]
>> + * \param chroma pointer to the 4 bytes video format identifier [IN/OUT]
>> + * \param width pointer to the pixel width [IN/OUT]
>> + * \param height pointer to the pixel height [IN/OUT]
>> + * \param x_offset pointer to the pixel x offset [IN/OUT]
>> + * \param y_offset pointer to the pixel y offset [IN/OUT]
>> + * \param visible_width pointer to the pixel visible width [IN/OUT]
>> + * \param visible_height pointer to the pixel visible height [IN/OUT]
>> + * \param device pointer to the platform-specific device pointer [OUT]
>> + *
>> + */
>> +typedef void (*libvlc_vbridge_format_cb)(void **opaque, char *chroma,
>> +                                         unsigned *width, unsigned
>> *height,
>> +                                         unsigned *x_offset, unsigned
>> *y_offset,
>> +                                         unsigned *visible_width,
>> +                                         unsigned *visible_height,
>> +                                         void **device);
>> +
>> +/**
>> + * Callback prototype to cleanup resources for video bridge output.
>> + *
>> + * \param opaque private pointer as passed to
>> + *               libvlc_media_player_set_vbridge_callbacks()
>> + *               (and possibly modified by @ref
>> libvlc_vbridge_format_cb) [IN]
>> + */
>> +typedef void (*libvlc_vbridge_cleanup_cb)(void *opaque);
>> +
>> +/**
>> + * Callback prototype to prepare a picture for video bridge output.
>> + *
>> + * When the video frame decoding is complete, the prepare callback is
>> invoked.
>> + * This callback might not be needed at all. It is only an indication
>> that the
>> + * application can now read the pixel values if it needs to.
>> + *
>> + * If the picture is a software surface then planes[] will point to
>> system
>> + * memory and pitches[] and lines[] will be valid. The number and layout
>> of the
>> + * planes will depend on the chroma.
>> + *
>> + * If the picture is a hardware surface then planes[0] should be cast to
>> the
>> + * appropriate type (IDirect3D9Surface, CVPixelBuffer or VASurfaceID)
>> and the
>> + * pitches[0] and lines[0] entries will be zero.
>> + *
>> + * \param opaque private pointer as passed to
>> + *               libvlc_media_player_set_vbridge_callbacks()
>> + *               (and possibly modified by @ref
>> libvlc_vbridge_format_cb) [IN]
>> + * \param planes start address of the pixel planes[IN]
>> + * \param pitches table of scanline pitches in bytes for each pixel
>> plane [IN]
>> + * \param lines table of scanlines count for each plane [IN]
>> + */
>> +typedef void (*libvlc_vbridge_prepare_cb)(void *opaque, void *const
>> *planes,
>> +                                          unsigned *pitches, unsigned
>> *lines);
>> +
>> +/**
>> + * Callback prototype to display a picture for video bridge output.
>> + *
>> + * When the video frame needs to be shown, as determined by the media
>> playback
>> + * clock, the display callback is invoked.
>> + *
>> + * If the picture is a software surface then planes[] will point to
>> system
>> + * memory and pitches[] and lines[] will be valid. The number and layout
>> of the
>> + * planes will depend on the chroma.
>> + *
>> + * If the picture is a hardware surface then planes[0] should be cast to
>> the
>> + * appropriate type (IDirect3D9Surface, CVPixelBuffer or VASurfaceID)
>> and the
>> + * pitches[0] and lines[0] entries will be zero.
>> + *
>> + * \param opaque private pointer as passed to
>> + *               libvlc_media_player_set_vbridge_callbacks()
>> + *               (and possibly modified by @ref
>> libvlc_vbridge_format_cb) [IN]
>> + * \param planes start address of the pixel planes[IN]
>> + * \param pitches table of scanline pitches in bytes for each pixel
>> plane [IN]
>> + * \param lines table of scanlines count for each plane [IN]
>> + */
>> +typedef void (*libvlc_vbridge_display_cb)(void *opaque, void *const
>> *planes,
>> +                                          unsigned *pitches, unsigned
>> *lines);
>> +
>> +/**
>> + * Set callbacks and private data for video bridge output.
>> + *
>> + * \param mp the media player
>> + * \param format callback to configure format
>> + * \param cleanup callback to cleanup resources
>> + * \param prepare callback to prepare a picture
>> + * \param display callback to display a picture
>> + * \param opaque private pointer
>> + */
>> +LIBVLC_API
>> +void libvlc_media_player_set_vbridge_callbacks(libvlc_media_player_t
>> *mp,
>> +                                               libvlc_vbridge_format_cb
>> format,
>> +                                               libvlc_vbridge_cleanup_cb
>> cleanup,
>> +                                               libvlc_vbridge_prepare_cb
>> prepare,
>> +                                               libvlc_vbridge_display_cb
>> display,
>> +                                               void *opaque);
>> +
>> +/**
>>  * Set the NSView handler where the media player should render its video
>>  output.
>>  *
>>  * Use the vout called "macosx".
>> diff --git a/lib/libvlc.sym b/lib/libvlc.sym
>> index caa55981bf..9fb3be8707 100644
>> --- a/lib/libvlc.sym
>> +++ b/lib/libvlc.sym
>> @@ -203,6 +203,7 @@ libvlc_media_player_stop
>> libvlc_media_player_will_play
>> libvlc_media_player_navigate
>> libvlc_media_player_set_video_title_display
>> +libvlc_media_player_set_vbridge_callbacks
>> libvlc_media_release
>> libvlc_media_retain
>> libvlc_media_save_meta
>> diff --git a/lib/media_player.c b/lib/media_player.c
>> index dc17f7765d..9d56402cd7 100644
>> --- a/lib/media_player.c
>> +++ b/lib/media_player.c
>> @@ -693,6 +693,12 @@ libvlc_media_player_new( libvlc_instance_t *instance
>> )
>>     var_Create (mp, "saturation", VLC_VAR_FLOAT | VLC_VAR_DOINHERIT);
>>     var_Create (mp, "gamma", VLC_VAR_FLOAT | VLC_VAR_DOINHERIT);
>> 
>> +    var_Create (mp, "vbridge-format", VLC_VAR_ADDRESS);
>> +    var_Create (mp, "vbridge-cleanup", VLC_VAR_ADDRESS);
>> +    var_Create (mp, "vbridge-prepare", VLC_VAR_ADDRESS);
>> +    var_Create (mp, "vbridge-display", VLC_VAR_ADDRESS);
>> +    var_Create (mp, "vbridge-opaque", VLC_VAR_ADDRESS);
>> +
>>      /* Audio */
>>     var_Create (mp, "aout", VLC_VAR_STRING | VLC_VAR_DOINHERIT);
>>     var_Create (mp, "audio-device", VLC_VAR_STRING);
>> @@ -1115,6 +1121,32 @@ void libvlc_video_set_format(
>> libvlc_media_player_t *mp, const char *chroma,
>> }
>> 
>> /**************************************************************************
>> + * set_vbridge_callbacks
>> +
>> **************************************************************************/
>> +void libvlc_media_player_set_vbridge_callbacks(libvlc_media_player_t
>> *mp,
>> +                                               libvlc_vbridge_format_cb
>> format,
>> +                                               libvlc_vbridge_cleanup_cb
>> cleanup,
>> +                                               libvlc_vbridge_prepare_cb
>> prepare,
>> +                                               libvlc_vbridge_display_cb
>> display,
>> +                                               void *opaque)
>> +{
>> +    var_SetAddress(mp, "vbridge-format", format);
>> +    var_SetAddress(mp, "vbridge-cleanup", cleanup);
>> +    var_SetAddress(mp, "vbridge-prepare", prepare);
>> +    var_SetAddress(mp, "vbridge-display", display);
>> +    var_SetAddress(mp, "vbridge-opaque", opaque);
>> +#ifdef _WIN32
>> +    var_SetString(mp, "avcodec-hw", "dxva2");
>> +#elif defined(__linux__)
>> +    var_SetString(mp, "avcodec-hw", "vaapi");
>> +#else
>> +    var_SetString(mp, "avcodec-hw", "none");
>> +#endif
>> +    var_SetString(mp, "vout", "vbridge");
>> +    var_SetString(mp, "window", "none");
>> +}
>> +
>> +/**************************************************************************
>>  * set_nsobject
>>  **************************************************************************/
>> void libvlc_media_player_set_nsobject( libvlc_media_player_t *p_mi,
>> diff --git a/modules/MODULES_LIST b/modules/MODULES_LIST
>> index 04468e438d..0db5467123 100644
>> --- a/modules/MODULES_LIST
>> +++ b/modules/MODULES_LIST
>> @@ -421,6 +421,7 @@ $Id$
>>  * vaapi_drm: VAAPI hardware-accelerated decoding with drm backend
>>  * vaapi_filters: VAAPI hardware-accelerated
>>  deinterlacing/adjust/sharpen/chroma filters
>>  * vaapi_x11: VAAPI hardware-accelerated decoding with x11 backend
>> + * vbridge: video bridge output
>>  * vc1: VC-1 Video demuxer
>>  * vcd: input module for accessing Video CDs
>>  * vdpau_adjust: VDPAU color adjust video filter
>> diff --git a/modules/video_output/Makefile.am
>> b/modules/video_output/Makefile.am
>> index 2e7c5d8ba7..7407896569 100644
>> --- a/modules/video_output/Makefile.am
>> +++ b/modules/video_output/Makefile.am
>> @@ -440,6 +440,39 @@ vout_LTLIBRARIES += libevas_plugin.la
>> endif
>> 
>> 
>> +### Video Bridge ###
>> +libvbridge_plugin_la_SOURCES = video_output/vbridge.c
>> +if HAVE_AVCODEC_DXVA2
>> +libvbridge_plugin_la_CFLAGS = $(AM_CFLAGS) -DUSE_DXVA2
>> +libvbridge_plugin_la_LDFLAGS = $(AM_LDFLAGS) -rpath '$(voutdir)'
>> +endif
>> +if HAVE_OSX
>> +libvbridge_plugin_la_SOURCES += codec/vt_utils.c codec/vt_utils.h
>> +libvbridge_plugin_la_CFLAGS = $(AM_CFLAGS) -DUSE_VTBOX
>> +libvbridge_plugin_la_LDFLAGS = $(AM_LDFLAGS) -rpath '$(voutdir)' \
>> +       -Wl,-framework,CoreFoundation,-framework,CoreVideo
>> +endif
>> +if HAVE_IOS
>> +libvbridge_plugin_la_SOURCES += codec/vt_utils.c codec/vt_utils.h
>> +libvbridge_plugin_la_CFLAGS = $(AM_CFLAGS) -DUSE_VTBOX
>> +libvbridge_plugin_la_LDFLAGS = $(AM_LDFLAGS) -rpath '$(voutdir)' \
>> +       -Wl,-framework,CoreFoundation,-framework,CoreVideo
>> +endif
>> +if HAVE_TVOS
>> +libvbridge_plugin_la_SOURCES += codec/vt_utils.c codec/vt_utils.h
>> +libvbridge_plugin_la_CFLAGS = $(AM_CFLAGS) -DUSE_VTBOX
>> +libvbridge_plugin_la_LDFLAGS = $(AM_LDFLAGS) -rpath '$(voutdir)' \
>> +       -Wl,-framework,CoreFoundation,-framework,CoreVideo
>> +endif
>> +if HAVE_AVCODEC_VAAPI
>> +libvbridge_plugin_la_SOURCES += hw/vaapi/vlc_vaapi.c
>> hw/vaapi/vlc_vaapi.h
>> +libvbridge_plugin_la_CFLAGS = $(AM_CFLAGS) -DUSE_VAAPI
>> +libvbridge_plugin_la_LDFLAGS = $(AM_LDFLAGS) -rpath '$(voutdir)'
>> +libvbridge_plugin_la_LIBADD = $(LIBVA_LIBS) libvlc_vaapi_instance.la
>> +endif
>> +vout_LTLIBRARIES += libvbridge_plugin.la
>> +
>> +
>> ### Common ###
>> 
>> libflaschen_plugin_la_SOURCES = video_output/flaschen.c
>> diff --git a/modules/video_output/vbridge.c
>> b/modules/video_output/vbridge.c
>> new file mode 100644
>> index 0000000000..839b51526e
>> --- /dev/null
>> +++ b/modules/video_output/vbridge.c
>> @@ -0,0 +1,615 @@
>> +/*****************************************************************************
>> + * vbridge.c: video bridge driver for vlc
>> +
>> *****************************************************************************
>> + * Copyright (C) 2017 VLC authors and VideoLAN
>> + *
>> + * Authors: Oliver Collyer <ovcollyer at mac.com>
>> + *
>> + * This program is free software; you can redistribute it and/or modify
>> it
>> + * under the terms of the GNU Lesser General Public License as published
>> by
>> + * the Free Software Foundation; either version 2.1 of the License, or
>> + * (at your option) any later version.
>> + *
>> + * This program is distributed in the hope that it will be useful,
>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
>> + * GNU Lesser General Public License for more details.
>> + *
>> + * You should have received a copy of the GNU Lesser General Public
>> License
>> + * along with this program; if not, write to the Free Software
>> Foundation,
>> + * Inc., 51 Franklin Street, Fifth Floor, Boston MA 02110-1301, USA.
>> +
>> *****************************************************************************/
>> +
>> +/*****************************************************************************
>> + * Preamble
>> +
>> *****************************************************************************/
>> +#ifdef HAVE_CONFIG_H
>> +# include "config.h"
>> +#endif
>> +
>> +#include <assert.h>
>> +
>> +#include <vlc_common.h>
>> +#include <vlc_plugin.h>
>> +#include <vlc_vout_display.h>
>> +#include <vlc_picture_pool.h>
>> +
>> +#ifdef USE_DXVA2
>> +#include <d3d9.h>
>> +#include "../video_chroma/d3d9_fmt.h"
>> +#elif defined(USE_VTBOX)
>> +#import <CoreVideo/CoreVideo.h>
>> +#include "../codec/vt_utils.h"
>> +#elif defined(USE_VAAPI)
>> +#include <va/va.h>
>> +#include "../hw/vaapi/vlc_vaapi.h"
>> +#endif
>> +
>> +/*****************************************************************************
>> + * Module descriptor
>> +
>> *****************************************************************************/
>> +static int  Open (vlc_object_t *);
>> +static void Close(vlc_object_t *);
>> +
>> +vlc_module_begin()
>> +    set_shortname(N_("Video bridge"))
>> +    set_description(N_("Video bridge output"))
>> +    set_category(CAT_VIDEO)
>> +    set_subcategory(SUBCAT_VIDEO_VOUT)
>> +    set_capability("vout display", 0)
>> +    set_callbacks(Open, Close)
>> +vlc_module_end()
>> +
>> +/*****************************************************************************
>> + * Local prototypes
>> +
>> *****************************************************************************/
>> +struct vout_display_sys_t
>> +{
>> +    picture_pool_t *pool;
>> +    picture_t *prev_pic;
>> +    bool hw_surfaces;
>> +
>> +#ifdef USE_VAAPI
>> +    VADisplay dpy;
>> +    VABufferID config_id;
>> +    VABufferID context_id;
>> +#endif
>> +
>> +    void (*cleanup)(void *);
>> +    void (*prepare)(void *, void *const *, unsigned *, unsigned *);
>> +    void (*display)(void *, void *const *, unsigned *, unsigned *);
>> +    void *opaque;
>> +    void *device;
>> +};
>> +
>> +typedef void (*vlc_format_cb)(void **, char *,
>> +                              unsigned *, unsigned *,
>> +                              unsigned *, unsigned *,
>> +                              unsigned *, unsigned *,
>> +                              void **);
>> +
>> +static picture_pool_t *Pool(vout_display_t *, unsigned);
>> +static void SoftwareSurfacePrepare(vout_display_t *, picture_t *,
>> subpicture_t *);
>> +static void SoftwareSurfaceDisplay(vout_display_t *, picture_t *,
>> subpicture_t *);
>> +static void HardwareSurfacePrepare(vout_display_t *, picture_t *,
>> subpicture_t *);
>> +static void HardwareSurfaceDisplay(vout_display_t *, picture_t *,
>> subpicture_t *);
>> +static int Control(vout_display_t *, int, va_list);
>> +
>> +/*****************************************************************************
>> + * Create the video bridge output
>> +
>> *****************************************************************************/
>> +static int Open(vlc_object_t *object)
>> +{
>> +    vout_display_t *vd = (vout_display_t *)object;
>> +
>> +    vlc_format_cb format = var_InheritAddress(vd, "vbridge-format");
>> +    if (!format)
>> +    {
>> +        msg_Err(vd, "Invalid vbridge-format callback");
>> +        return VLC_EGENERIC;
>> +    }
>> +
>> +    vout_display_sys_t *sys = calloc(1, sizeof(*sys));
>> +    if (unlikely(!sys))
>> +        return VLC_ENOMEM;
>> +
>> +    sys->cleanup = var_InheritAddress(vd, "vbridge-cleanup");
>> +    sys->prepare = var_InheritAddress(vd, "vbridge-prepare");
>> +    sys->display = var_InheritAddress(vd, "vbridge-display");
>> +    sys->opaque  = var_InheritAddress(vd, "vbridge-opaque");
>> +
>> +#ifdef USE_VAAPI
>> +    sys->config_id = VA_INVALID_ID;
>> +    sys->context_id = VA_INVALID_ID;
>> +#endif
>> +
>> +    video_format_t fmt;
>> +    video_format_ApplyRotation(&fmt, &vd->fmt);
>> +
>> +    char chroma[5];
>> +    memcpy(chroma, &fmt.i_chroma, 4);
>> +    chroma[4] = '\0';
>> +
>> +    format(&sys->opaque,
>> +          chroma,
>> +          &fmt.i_width, &fmt.i_height,
>> +          &fmt.i_x_offset, &fmt.i_y_offset,
>> +          &fmt.i_visible_width, &fmt.i_visible_height,
>> +          &sys->device);
>> +
>> +    fmt.i_chroma = vlc_fourcc_GetCodecFromString(VIDEO_ES, chroma);
>> +    if (!fmt.i_chroma)
>> +    {
>> +        msg_Err(vd, "Invalid chroma requested (%s)", chroma);
>> +        free(sys);
>> +        return VLC_EGENERIC;
>> +    }
>> +
>> +    switch (fmt.i_chroma)
>> +    {
>> +    case VLC_CODEC_RGB15:
>> +        fmt.i_rmask = 0x001f;
>> +        fmt.i_gmask = 0x03e0;
>> +        fmt.i_bmask = 0x7c00;
>> +        break;
>> +    case VLC_CODEC_RGB16:
>> +        fmt.i_rmask = 0x001f;
>> +        fmt.i_gmask = 0x07e0;
>> +        fmt.i_bmask = 0xf800;
>> +        break;
>> +    case VLC_CODEC_RGB24:
>> +    case VLC_CODEC_RGB32:
>> +        fmt.i_rmask = 0xff0000;
>> +        fmt.i_gmask = 0x00ff00;
>> +        fmt.i_bmask = 0x0000ff;
>> +        break;
>> +    default:
>> +        fmt.i_rmask = 0;
>> +        fmt.i_gmask = 0;
>> +        fmt.i_bmask = 0;
>> +        break;
>> +    }
>> +
>> +#ifdef USE_DXVA2
>> +    if (vd->fmt.i_chroma == VLC_CODEC_D3D9_OPAQUE ||
>> +        vd->fmt.i_chroma == VLC_CODEC_D3D9_OPAQUE_10B)
>> +        sys->hw_surfaces = true;
>> +#elif defined(USE_VTBOX)
>> +    if (vd->fmt.i_chroma == VLC_CODEC_CVPX_UYVY ||
>> +        vd->fmt.i_chroma == VLC_CODEC_CVPX_NV12 ||
>> +        vd->fmt.i_chroma == VLC_CODEC_CVPX_I420 ||
>> +        vd->fmt.i_chroma == VLC_CODEC_CVPX_BGRA)
>> +        sys->hw_surfaces = true;
>> +#elif defined(USE_VAAPI)
>> +    if (vd->fmt.i_chroma == VLC_CODEC_VAAPI_420)
>> +        sys->hw_surfaces = true;
>> +#endif
>> +    if (sys->hw_surfaces)
>> +        msg_Dbg(vd, "Using hardware surfaces");
>> +
>> +    vout_display_info_t info = vd->info;
>> +    info.has_hide_mouse = true;
>> +
>> +    vd->sys     = sys;
>> +    vd->fmt     = fmt;
>> +    vd->info    = info;
>> +    vd->pool    = Pool;
>> +    vd->prepare = sys->hw_surfaces ? HardwareSurfacePrepare :
>> SoftwareSurfacePrepare;
>> +    vd->display = sys->hw_surfaces ? HardwareSurfaceDisplay :
>> SoftwareSurfaceDisplay;
>> +    vd->control = Control;
>> +    vd->manage  = NULL;
>> +
>> +    vout_display_SendEventDisplaySize(vd, fmt.i_width, fmt.i_height);
>> +    vout_display_DeleteWindow(vd, NULL);
>> +
>> +    return VLC_SUCCESS;
>> +}
>> +
>> +/*****************************************************************************
>> + * Destroy the video bridge output
>> +
>> *****************************************************************************/
>> +static void Close(vlc_object_t *object)
>> +{
>> +    vout_display_t *vd = (vout_display_t *)object;
>> +    vout_display_sys_t *sys = vd->sys;
>> +
>> +    if (sys->prev_pic)
>> +        picture_Release(sys->prev_pic);
>> +
>> +#ifdef USE_VAAPI
>> +    if (sys->context_id != VA_INVALID_ID)
>> +        vlc_vaapi_DestroyContext(object, sys->dpy, sys->context_id);
>> +#endif
>> +    if (sys->pool != NULL)
>> +        picture_pool_Release(sys->pool);
>> +#ifdef USE_VAAPI
>> +    if (sys->config_id != VA_INVALID_ID)
>> +        vlc_vaapi_DestroyConfig(object, sys->dpy, sys->config_id);
>> +    if (sys->dpy != NULL)
>> +        vlc_vaapi_ReleaseInstance(sys->dpy);
>> +#endif
>> +
>> +    if (sys->cleanup)
>> +        sys->cleanup(sys->opaque);
>> +
>> +    free(sys);
>> +}
>> +
>> +#ifdef USE_DXVA2
>> +
>> +/*****************************************************************************
>> + * Destroy a single DXVA2 picture
>> +
>> *****************************************************************************/
>> +static void DestroyPoolPicture(picture_t *picture)
>> +{
>> +    picture_sys_t *p_sys = (picture_sys_t *)picture->p_sys;
>> +
>> +    if (p_sys->surface)
>> +        IDirect3DSurface9_Release(p_sys->surface);
>> +    free(p_sys);
>> +    free(picture);
>> +}
>> +
>> +/*****************************************************************************
>> + * Create the DXVA2 picture pool
>> +
>> *****************************************************************************/
>> +static picture_pool_t *Pool(vout_display_t *vd, unsigned count)
>> +{
>> +    vout_display_sys_t *sys = vd->sys;
>> +
>> +    if (sys->pool != NULL)
>> +        return sys->pool;
>> +
>> +    if (!sys->hw_surfaces)
>> +    {
>> +        sys->pool = picture_pool_NewFromFormat(&vd->fmt, count);
>> +        return sys->pool;
>> +    }
>> +
>> +    LPDIRECT3DDEVICE9 d3ddev = (LPDIRECT3DDEVICE9)sys->device;
>> +    if (!d3ddev)
>> +    {
>> +       msg_Err(vd, "Invalid D3D9 display device");
>> +       return NULL;
>> +    }
>> +
>> +    D3DFORMAT format;
>> +    switch (vd->fmt.i_chroma)
>> +    {
>> +    case VLC_CODEC_D3D9_OPAQUE_10B:
>> +        format = MAKEFOURCC('P','0','1','0');
>> +        break;
>> +    default:
>> +        format = MAKEFOURCC('N','V','1','2');
>> +        break;
>> +    }
>> +
>> +    picture_t** pictures = calloc(count, sizeof(*pictures));
>> +    if (unlikely(!pictures))
>> +        goto error;
>> +
>> +    for (unsigned int i = 0; i < count; i++)
>> +    {
>> +        picture_sys_t *picsys = calloc(1, sizeof(*picsys));
>> +        if (unlikely(!picsys))
>> +            goto error;
>> +
>> +        HRESULT hr =
>> IDirect3DDevice9_CreateOffscreenPlainSurface(d3ddev,
>> +                                                             
>> vd->fmt.i_width,
>> +                                                             
>> vd->fmt.i_height,
>> +                                                              format,
>> +                                                             
>> D3DPOOL_DEFAULT,
>> +                                                             
>> &picsys->surface,
>> +                                                              NULL);
>> +        if (FAILED(hr))
>> +        {
>> +            msg_Err(vd, "Failed to create D3D9 surface");
>> +            free(picsys);
>> +            goto error;
>> +        }
>> +
>> +        picture_resource_t picture_resource;
>> +        picture_resource.p_sys = picsys;
>> +        picture_resource.pf_destroy = DestroyPoolPicture;
>> +
>> +        picture_t *picture = picture_NewFromResource(&vd->fmt,
>> &picture_resource);
>> +        if (unlikely(picture == NULL))
>> +        {
>> +            free(picsys);
>> +            goto error;
>> +        }
>> +
>> +        pictures[i] = picture;
>> +    }
>> +
>> +    picture_pool_configuration_t pool_config;
>> +    memset(&pool_config, 0, sizeof(pool_config));
>> +    pool_config.picture_count = count;
>> +    pool_config.picture = pictures;
>> +
>> +    sys->pool = picture_pool_NewExtended(&pool_config);
>> +    if (!sys->pool)
>> +    {
>> +        msg_Err(vd, "Failed to create picture pool");
>> +        goto error;
>> +    }
>> +
>> +    return sys->pool;
>> +
>> +error:
>> +    if (pictures)
>> +    {
>> +        for (unsigned int i = 0; i < count; i++)
>> +            if (pictures[i])
>> +                DestroyPoolPicture(pictures[i]);
>> +        free(pictures);
>> +    }
>> +
>> +    return NULL;
>> +}
>> +
>> +#elif defined(USE_VAAPI)
>> +
>> +/*****************************************************************************
>> + * Create the VAAPI picture pool
>> +
>> *****************************************************************************/
>> +static picture_pool_t *Pool(vout_display_t *vd, unsigned count)
>> +{
>> +    vout_display_sys_t *sys = vd->sys;
>> +    vlc_object_t *o = VLC_OBJECT(vd);
>> +
>> +    if (sys->pool != NULL)
>> +        return sys->pool;
>> +
>> +    if (!sys->hw_surfaces)
>> +    {
>> +        sys->pool = picture_pool_NewFromFormat(&vd->fmt, count);
>> +        return sys->pool;
>> +    }
>> +
>> +    sys->dpy = (VADisplay)sys->device;
>> +    if (!sys->dpy)
>> +    {
>> +        msg_Err(vd, "Invalid VAAPI display device");
>> +        return NULL;
>> +    }
>> +
>> +    if (vlc_vaapi_Initialize(o, sys->dpy))
>> +    {
>> +        msg_Err(vd, "Failed to initialize VAAPI");
>> +        sys->dpy = NULL;
>> +        goto error;
>> +    }
>> +
>> +    if (vlc_vaapi_SetInstance(sys->dpy))
>> +    {
>> +        msg_Err(vd, "Failed to set VAAPI instance");
>> +        sys->dpy = NULL;
>> +        goto error;
>> +    }
>> +
>> +    sys->config_id = vlc_vaapi_CreateConfigChecked(o, sys->dpy,
>> VAProfileNone,
>> +                                                  
>> VAEntrypointVideoProc, 0);
>> +    if (sys->config_id == VA_INVALID_ID)
>> +    {
>> +        msg_Err(vd, "Failed to create VAAPI config");
>> +        goto error;
>> +    }
>> +
>> +    VASurfaceID *surfaces;
>> +    sys->pool = vlc_vaapi_PoolNew(o, sys->dpy, count, &surfaces,
>> &vd->fmt,
>> +                                  VA_RT_FORMAT_YUV420, 0);
>> +    if (!sys->pool)
>> +    {
>> +        msg_Err(vd, "Failed to create VAAPI pool");
>> +        goto error;
>> +    }
>> +
>> +    sys->context_id = vlc_vaapi_CreateContext(o, sys->dpy,
>> sys->config_id,
>> +                                              vd->fmt.i_width,
>> vd->fmt.i_height,
>> +                                              VA_PROGRESSIVE, surfaces,
>> count);
>> +    if (sys->context_id == VA_INVALID_ID)
>> +    {
>> +        msg_Err(vd, "Failed to create VAAPI context");
>> +        goto error;
>> +    }
>> +
>> +    return sys->pool;
>> +
>> +error:
>> +    if (sys->context_id != VA_INVALID_ID)
>> +        vlc_vaapi_DestroyContext(o, sys->dpy, sys->context_id);
>> +    if (sys->pool != NULL)
>> +        picture_pool_Release(sys->pool);
>> +    if (sys->config_id != VA_INVALID_ID)
>> +        vlc_vaapi_DestroyConfig(o, sys->dpy, sys->config_id);
>> +    if (sys->dpy != NULL)
>> +        vlc_vaapi_ReleaseInstance(sys->dpy);
>> +
>> +    return NULL;
>> +}
>> +
>> +#else
>> +
>> +/*****************************************************************************
>> + * Create the generic picture pool
>> +
>> *****************************************************************************/
>> +static picture_pool_t *Pool(vout_display_t *vd, unsigned count)
>> +{
>> +    vout_display_sys_t *sys = vd->sys;
>> +
>> +    if (sys->pool != NULL)
>> +        return sys->pool;
>> +
>> +    sys->pool = picture_pool_NewFromFormat(&vd->fmt, count);
>> +
>> +    return sys->pool;
>> +}
>> +
>> +#endif
>> +
>> +/*****************************************************************************
>> + * Call the software surface prepare
>> +
>> *****************************************************************************/
>> +static void SoftwareSurfacePrepare(vout_display_t *vd, picture_t *pic,
>> subpicture_t *subpic)
>> +{
>> +    vout_display_sys_t *sys = vd->sys;
>> +
>> +    if (sys->prepare != NULL)
>> +    {
>> +        void *planes[PICTURE_PLANE_MAX];
>> +        unsigned pitches[PICTURE_PLANE_MAX];
>> +        unsigned lines[PICTURE_PLANE_MAX];
>> +        for (unsigned int i = 0; i < PICTURE_PLANE_MAX; i++)
>> +        {
>> +            if (i < (unsigned int)pic->i_planes)
>> +            {
>> +                plane_t *p_src = pic->p+i;
>> +                planes[i] = p_src->p_pixels;
>> +                pitches[i] = p_src->i_pitch;
>> +                lines[i] = p_src->i_lines;
>> +            }
>> +            else
>> +            {
>> +                planes[i] = NULL;
>> +                pitches[i] = 0;
>> +                lines[i] = 0;
>> +            }
>> +        }
>> +        sys->prepare(sys->opaque, planes, pitches, lines);
>> +    }
>> +
>> +    VLC_UNUSED(subpic);
>> +}
>> +
>> +/*****************************************************************************
>> + * Call the software surface display
>> +
>> *****************************************************************************/
>> +static void SoftwareSurfaceDisplay(vout_display_t *vd, picture_t *pic,
>> subpicture_t *subpic)
>> +{
>> +    vout_display_sys_t *sys = vd->sys;
>> +
>> +    if (sys->display != NULL)
>> +    {
>> +        void *planes[PICTURE_PLANE_MAX];
>> +        unsigned pitches[PICTURE_PLANE_MAX];
>> +        unsigned lines[PICTURE_PLANE_MAX];
>> +        for (unsigned int i = 0; i < PICTURE_PLANE_MAX; i++)
>> +        {
>> +            if (i < (unsigned int)pic->i_planes)
>> +            {
>> +                plane_t *p_src = pic->p+i;
>> +                planes[i] = p_src->p_pixels;
>> +                pitches[i] = p_src->i_pitch;
>> +                lines[i] = p_src->i_lines;
>> +            }
>> +            else
>> +            {
>> +                planes[i] = NULL;
>> +                pitches[i] = 0;
>> +                lines[i] = 0;
>> +            }
>> +        }
>> +        sys->display(sys->opaque, planes, pitches, lines);
>> +    }
>> +
>> +    if (sys->prev_pic)
>> +        picture_Release(sys->prev_pic);
>> +    sys->prev_pic = pic;
>> +
>> +    VLC_UNUSED(subpic);
>> +}
>> +
>> +/*****************************************************************************
>> + * Call the hardware surface prepare
>> +
>> *****************************************************************************/
>> +static void HardwareSurfacePrepare(vout_display_t *vd, picture_t *pic,
>> subpicture_t *subpic)
>> +{
>> +    vout_display_sys_t *sys = vd->sys;
>> +
>> +    if (sys->prepare)
>> +    {
>> +        void *planes[PICTURE_PLANE_MAX];
>> +        unsigned pitches[PICTURE_PLANE_MAX];
>> +        unsigned lines[PICTURE_PLANE_MAX];
>> +        for (unsigned int i = 0; i < PICTURE_PLANE_MAX; i++)
>> +        {
>> +            if (i == 0)
>> +            {
>> +#ifdef USE_DXVA2
>> +                if (pic->context)
>> +                    planes[i] = (void*)((struct
>> va_pic_context*)pic->context)->picsys.surface;
>> +                else
>> +                    planes[i] = (void*)pic->p_sys->surface;
>> +#elif defined(USE_VTBOX)
>> +                planes[i] = (void*)cvpxpic_get_ref(pic);
>> +#elif defined(USE_VAAPI)
>> +                planes[i] =
>> (void*)(uintptr_t)vlc_vaapi_PicGetSurface(pic);
>> +#else
>> +                planes[i] = NULL;
>> +#endif
>> +            }
>> +            else
>> +                planes[i] = NULL;
>> +            pitches[i] = 0;
>> +            lines[i] = 0;
>> +        }
>> +        sys->prepare(sys->opaque, planes, pitches, lines);
>> +    }
>> +
>> +    VLC_UNUSED(subpic);
>> +}
>> +
>> +/*****************************************************************************
>> + * Call the hardware surface display
>> +
>> *****************************************************************************/
>> +static void HardwareSurfaceDisplay(vout_display_t *vd, picture_t *pic,
>> subpicture_t *subpic)
>> +{
>> +    vout_display_sys_t *sys = vd->sys;
>> +
>> +    if (sys->display)
>> +    {
>> +        void *planes[PICTURE_PLANE_MAX];
>> +        unsigned pitches[PICTURE_PLANE_MAX];
>> +        unsigned lines[PICTURE_PLANE_MAX];
>> +        for (unsigned int i = 0; i < PICTURE_PLANE_MAX; i++)
>> +        {
>> +            if (i == 0)
>> +            {
>> +#ifdef USE_DXVA2
>> +                if (pic->context)
>> +                    planes[i] = (void*)((struct
>> va_pic_context*)pic->context)->picsys.surface;
>> +                else
>> +                    planes[i] = (void*)pic->p_sys->surface;
>> +#elif defined(USE_VTBOX)
>> +                planes[i] = (void*)cvpxpic_get_ref(pic);
>> +#elif defined(USE_VAAPI)
>> +                planes[i] =
>> (void*)(uintptr_t)vlc_vaapi_PicGetSurface(pic);
>> +#else
>> +                planes[i] = NULL;
>> +#endif
>> +            }
>> +            else
>> +                planes[i] = NULL;
>> +            pitches[i] = 0;
>> +            lines[i] = 0;
>> +        }
>> +        sys->display(sys->opaque, planes, pitches, lines);
>> +    }
>> +
>> +    if (sys->prev_pic)
>> +        picture_Release(sys->prev_pic);
>> +    sys->prev_pic = pic;
>> +
>> +    VLC_UNUSED(subpic);
>> +}
>> +
>> +/* */
>> +static int Control(vout_display_t *vd, int query, va_list args)
>> +{
>> +    VLC_UNUSED(vd);
>> +    VLC_UNUSED(query);
>> +    VLC_UNUSED(args);
>> +
>> +    return VLC_EGENERIC;
>> +}
>> diff --git a/po/POTFILES.in b/po/POTFILES.in
>> index 66ea2ddf6d..3197106a26 100644
>> --- a/po/POTFILES.in
>> +++ b/po/POTFILES.in
>> @@ -1138,6 +1138,7 @@ modules/video_output/win32/events.c
>> modules/video_output/win32/glwin32.c
>> modules/video_output/win32/wingdi.c
>> modules/video_output/sdl.c
>> +modules/video_output/vbridge.c
>> modules/video_output/vdummy.c
>> modules/video_output/vmem.c
>> modules/video_output/wayland/shell.c
>> -- 
>> 2.11.0
>> 
>> _______________________________________________
>> vlc-devel mailing list
>> To unsubscribe or modify your subscription options:
>> https://mailman.videolan.org/listinfo/vlc-devel <https://mailman.videolan.org/listinfo/vlc-devel>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.videolan.org/pipermail/vlc-devel/attachments/20170702/5ed6d707/attachment-0001.html>


More information about the vlc-devel mailing list