[vlc-devel] [PATCH 1/2] nvdec: add chroma and depth fallback

[Thomas Guillem]

MediaCodec also use that delayed open hack. I would love to remove it from VLC 4.0

As Rémi said, it's not the good solution. Currently, most format changes are detected by the packetizer. In that case, the decoder is restarted using the fmt_out of the packetizer as the fmt_in, therefore, the chroma is known from Open directly.

If you have an issue with a sample, the correct fix is to fix the packetizer or the demuxer.
 - If no hevc packetizers is used, then the demuxer is lying and a packetizer is needed (cf. b_packetized).
 - If a hevc packetizer is used, you need to propagate the format change to the packetizer fmt_out.

On Tue, Apr 7, 2020, at 10:10, Quentin Chateau wrote:
> Hi,

> I'm not sure how it is supposed to work, but in nvdec's case, the format we read while opening the decoder is only used to verify that the decoder will actually be able to decode the video. In any case, you can start the decoder and it will provide you with the actual video format once it started to decode. As Steve said, the issue is that if we start the decoder without verifying the format first, we may run into the situation where the decoder cannot actually decode the video, and we are stuck.

> On 07/04/2020 09:48, Rémi Denis-Courmont wrote:
>> Hi,
>> 
>>  I don't follow. Normally, you can't start any decoder if you don't know the format. It's even so now that if the packetiser finds a format change, we restart the decoder. That was specifically added (by Thomas?) to address switching between software and hardware decoding.
>> 
>>  I don't see how you can even end up in the described situation, unless the demuxer lies about the ES being packetised.
>> 
>> 
>> Le 6 avril 2020 16:02:06 GMT+03:00, quentin.chateau at deepskycorp.com a écrit : 
>>> From: Quentin Chateau <quentin.chateau at deepskycorp.com>

When the chroma and depth is unknown to the hxxx_helper,
fallback to the common 4:2:0 8bits format as initial value.
The decoder may change these values later on. modules/hw/nvdec/nvdec.c | 20 ++++++++++++--------
 1 file changed, 12 insertions(+), 8 deletions(-)

diff --git a/modules/hw/nvdec/nvdec.c b/modules/hw/nvdec/nvdec.c
index e3aeff2355..ca28522d32 100644
--- a/modules/hw/nvdec/nvdec.c
+++ b/modules/hw/nvdec/nvdec.c
@@ -923,15 +923,19 @@ static int OpenDecoder(vlc_object_t *p_this)
     // try different output
     if (p_sys->b_is_hxxx)
     {
-        uint8_t i_chroma_idc, i_depth_chroma;
-        result = hxxx_helper_get_chroma_chroma(&p_sys->hh, &i_chroma_idc,
-                                            &i_depth_luma, &i_depth_chroma);
-        if (result != VLC_SUCCESS) {
-            if (p_sys->b_is_hxxx)
-                hxxx_helper_clean(&p_sys->hh);
-            return VLC_EGENERIC;
+        uint8_t hxxx_chroma_idc, hxxx_depth_chroma, hxxx_depth_luma;
+        result = hxxx_helper_get_chroma_chroma(&p_sys->hh, &hxxx_chroma_idc,
+                                            &hxxx_depth_luma, &hxxx_depth_chroma);
+        if (result == VLC_SUCCESS)
+        {
+            i_depth_luma = hxxx_depth_luma;
+            cudaChroma = MapChomaIDC(hxxx_chroma_idc);
+        }
+        else
+        {
+            cudaChroma = cudaVideoChromaFormat_420;
+            i_depth_luma = 8;
         }
-        cudaChroma = MapChomaIDC(i_chroma_idc);
 
         unsigned i_w, i_h, i_vw, i_vh;
         result = hxxx_helper_get_current_picture_size(&p_sys->hh, &i_w, &i_h, &i_vw, &i_vh);
>> 
>> --
>>  Envoyé de mon appareil Android avec Courriel K-9 Mail. Veuillez excuser ma brièveté. 
>> 
>> _______________________________________________
vlc-devel mailing list
To unsubscribe or modify your subscription options:
>> https://mailman.videolan.org/listinfo/vlc-devel
> _______________________________________________
> vlc-devel mailing list
> To unsubscribe or modify your subscription options:
> https://mailman.videolan.org/listinfo/vlc-devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.videolan.org/pipermail/vlc-devel/attachments/20200407/b7ad01db/attachment.html>