[vlc-commits] demux:mkv: fix hvcC detection with mkvmerge

[Alexandre Janniaux]

vlc | branch: master | Alexandre Janniaux <alexandre.janniaux at gmail.com> | Wed Feb 21 18:35:06 2018 +0100| [70cd50797cca3c176885a0c944b42c0271c8d7f4] | committer: Steve Lhomme

demux:mkv: fix hvcC detection with mkvmerge

mkvmerge had an issue with the first reserved bits and fixed it in v16.0.0
in the commit 4bb8ad6f5565e87ad6d6a8e7e9d453e64985344e. Some files done
with anterior versions were not played by VLC with mediacodec.

See the changelog of mkvmerge for version 16.0.0, especially the
following:
* mkvmerge: HEVC/h.265: the generation of the HEVCC structure stored in
  `CodecPrivate` was wrong in two places: 1. the position of the number of
  sub-layers was swapped with reserved bits and 2. the VPS/SPS/PPS/SEI lists
  did not start with a reserved 1 bit.

See also https://code.videolan.org/videolan/vlc-android/issues/466 for issue
and sample.

Signed-off-by: Steve Lhomme <robux4 at ycbcr.xyz>

> http://git.videolan.org/gitweb.cgi/vlc.git/?a=commit;h=70cd50797cca3c176885a0c944b42c0271c8d7f4
---

 modules/demux/mkv/matroska_segment_parse.cpp | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/modules/demux/mkv/matroska_segment_parse.cpp b/modules/demux/mkv/matroska_segment_parse.cpp
index 8ec4f17aca..430d86bb2b 100644
--- a/modules/demux/mkv/matroska_segment_parse.cpp
+++ b/modules/demux/mkv/matroska_segment_parse.cpp
@@ -1586,6 +1586,25 @@ bool matroska_segment_c::TrackInit( mkv_track_t * p_tk )
         }
         S_CASE("V_MPEGH/ISO/HEVC") {
             vars.p_tk->fmt.i_codec = VLC_CODEC_HEVC;
+
+            uint8_t* p_extra = (uint8_t*) vars.p_tk->p_extra_data;
+
+            /* HACK: if we found invalid format, made by mkvmerge < 16.0.0,
+             *       we try to fix it. They fixed it in 16.0.0. */
+            const char* app = vars.obj->psz_writing_application;
+            if( p_extra && p_extra[0] == 0 && app != NULL &&
+                    strncmp(app, "mkvmerge", strlen("mkvmerge")) == 0 )
+            {
+                int major_version;
+                if( sscanf(app, "mkvmerge v%d.", &major_version) && major_version < 16 )
+                {
+                    msg_Dbg(vars.p_demuxer,
+                            "Invalid HEVC reserved bits in mkv file"
+                            "made by mkvmerge < v16.0.0 detected, fixing it");
+                    p_extra[0] = 0x01;
+                }
+            }
+
             fill_extra_data( vars.p_tk, 0 );
         }
         S_CASE("V_QUICKTIME") {