[vlc-devel] Software decoding in Hardware buffers

[Thomas Guillem]

On Thu, Aug 8, 2019, at 15:46, Steve Lhomme wrote:
> On 2019-08-08 15:07, Thomas Guillem wrote:
> > 
> > On Thu, Aug 8, 2019, at 14:29, Steve Lhomme wrote:
> >> Hi,
> >>
> >> I'm looking at the display pool in the MMAL (Raspberry Pi) code and it
> >> seems that we currently decode in "hardware" buffers all the time.
> >> Either the opaque decoder output, or when the decoder outputs I420.
> >>
> >> In push we don't to use this pool anymore. The decoder will have its own
> >> pool and the display just deals with what it receives. In most cases
> >> that means copy from CPU memory to GPU memory. This doesn't work with a
> >> SoC like on the Raspberry Pi where the memory is the same and can be
> >> used directly from both sides.
> >>
> >> The idea was that current decoders continue to use decoder_NewPicture()
> >> as they used to. The pictures will come from the decoder video context,
> >> if there's one (hardware decoding) or from picture_NewFromFormat() if
> >> there's none. That means for MMAL we would need to copy this CPU
> >> allocated memory to the "port allocated" memory (the mechanism to get
> >> buffers from the display). Given the limited resources that's something
> >> we should avoid.
> >>
> >> I think we should have a third way to provide pictures: from the decoder
> >> device. In case of software decoding there is no video context, but
> >> there is a decoder device (A MMAL one in this case).
> > 
> > Small reminder: There is ONE decoder device for several decoder hw + video_context.
> > A decoder device hold the X11/WL and Vaapi Display instance for example.
> > I don't see how we can put a pool on it.
> > 
> > Why not using a video context for the MMAL case too ?
> 
> Originally I went through a video context for everything. Then we 
> decided NULL is allowed for software decoders (and it does make the code 
> simpler).
> 
> Creating one for software decoders that use MMAL would solve the issue 
> as it would use its video context allocator normally. The problem is how 
> to decide to create a video context in that case. Right now only VA 
> decoders modules create one. It could be a flag in the MMAL decoder 
> device that tells it needs one for software decoding. The others 
> wouldn't set that flag.

Currently, only a hw decoder create a video context ? Maybe we should add the possibility to create one for SW case too, created from the decoder device. That's way the video context for MMAL could take care of pool allocation.

BTW:  MMAL need its own window even if it's a dummy one. Because, if there is a window module for MMAL, there can be a decoder device.

> 
> >>
> >> So I suggests the decoders (and filters) get their output picture from:
> >> - the video context if there is one
> >> - the decoder device if there is one and it has an allocator
> >> - picture_NewFromFormat() otherwise
> >>
> >> Any opinion ?
> >> _______________________________________________
> >> vlc-devel mailing list
> >> To unsubscribe or modify your subscription options:
> >> https://mailman.videolan.org/listinfo/vlc-devel
> > _______________________________________________
> > vlc-devel mailing list
> > To unsubscribe or modify your subscription options:
> > https://mailman.videolan.org/listinfo/vlc-devel
> > 
> _______________________________________________
> vlc-devel mailing list
> To unsubscribe or modify your subscription options:
> https://mailman.videolan.org/listinfo/vlc-devel