[vlc-devel] [PATCH] hxx_sei: Fix improper counting of remaining bytes due to 3-byte emulation

[Devin Heitmueller]

The routine which iterates over the RBSP to process the SEIs
attempts to accommodate the individual parsers not reading all
the bytes (i.e. by reading any remaining bytes before restarting
the loop), as well as detecting if the parser for a given type
read past the end of SEI.  However the implementation does this
by noting the position of the start of the NAL, getting the
position after parsing, and computing the difference.  This does
not take into account that the bitstream parser has a pf_forward
routine to strip out 3-byte emulation.

Hence in cases where an emulation sequence is found, the number
of bytes processed by the parser don't match how many bytes were
actually consumed in the stream.  The failure occurs at the bottom
of the loop where either it fails to read out extra bytes if the
parser didn't process the entire SEI, or aborting prematurely
thinking that the parser processed too many bytes.

To avoid this issue, clone the bitstream into a second instance
which already has the three byte emulation stripped, and use that
with the existing parser routines.

The use case where this problem manifested was a low latency
stream where it failed to find the SEI recovery point because
there was an emulation sequence in the preceding picture timing
SEI section, and this caused the loop to bail out because the
bs_position was past the size of the picture_timing SEI length.

Signed-off-by: Devin Heitmueller <dheitmueller at ltnglobal.com>
---
 modules/packetizer/hxxx_sei.c | 20 +++++++++++++++++---
 1 file changed, 17 insertions(+), 3 deletions(-)

diff --git a/modules/packetizer/hxxx_sei.c b/modules/packetizer/hxxx_sei.c
index 15791c3750..18a249cdaa 100644
--- a/modules/packetizer/hxxx_sei.c
+++ b/modules/packetizer/hxxx_sei.c
@@ -38,16 +38,30 @@ void HxxxParse_AnnexB_SEI(const uint8_t *p_buf, size_t i_buf,
 void HxxxParseSEI(const uint8_t *p_buf, size_t i_buf,
                   uint8_t i_header, pf_hxxx_sei_callback pf_callback, void *cbdata)
 {
+    bs_t s_ep3b;
     bs_t s;
     unsigned i_bitflow = 0;
     bool b_continue = true;
+    char buf[256];
+    int i = 0;
 
     if( i_buf <= i_header )
         return;
 
-    bs_init( &s, &p_buf[i_header], i_buf - i_header ); /* skip nal unit header */
-    s.p_fwpriv = &i_bitflow;
-    s.pf_forward = hxxx_bsfw_ep3b_to_rbsp;  /* Does the emulated 3bytes conversion to rbsp */
+    bs_init( &s_ep3b, &p_buf[i_header], i_buf - i_header ); /* skip nal unit header */
+    s_ep3b.p_fwpriv = &i_bitflow;
+    s_ep3b.pf_forward = hxxx_bsfw_ep3b_to_rbsp;  /* Does the emulated 3bytes conversion to rbsp */
+
+    /* While a NAL can technically be up to 65535 bytes, an SEI NAL
+       will never be anywhere near that size */
+    if (bs_remain(&s_ep3b) / 8 > sizeof(buf))
+        return;
+
+    while (bs_remain(&s_ep3b) > 0)
+        buf[i++] = bs_read(&s_ep3b, 8);
+
+    /* Parse the resulting RBSP bytes as SEI */
+    bs_init( &s, buf, i);
 
     while( bs_remain( &s ) >= 8 && bs_aligned( &s ) && b_continue )
     {
-- 
2.13.2