[vlc-devel] [PATCH] demux: ts: fix MPGA and ADTS probing

[Michael Kaufmann]

MPEG-1 Audio Layer II (MP2) was erroneously detected as ADTS.

MPGA: The previous check used wrong bitmasks (1 bit too far to the right).
The layer check was also wrong, the layer must not be 0.

ADTS: Previously only one bit of the layer was checked. Check both bits.
The check is now identical to the one in HasADTSHeader() in mpeg4audio.c.

References:
- http://mpgedit.org/mpgedit/mpeg_format/mpeghdr.htm
- https://wiki.multimedia.cx/index.php/ADTS
---
 modules/demux/mpeg/ts_hotfixes.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/modules/demux/mpeg/ts_hotfixes.c b/modules/demux/mpeg/ts_hotfixes.c
index 1af301c801..1593eec4e3 100644
--- a/modules/demux/mpeg/ts_hotfixes.c
+++ b/modules/demux/mpeg/ts_hotfixes.c
@@ -171,11 +171,11 @@ void ProbePES( demux_t *p_demux, ts_pid_t *pid, const uint8_t *p_pesstart, size_
     {
         pid->probed.i_cat = AUDIO_ES;
         if( p_data[0] == 0xFF && (p_data[1] & 0xE0) == 0xE0 &&
-           (p_data[1] & 0x0C) != 0x04 && (p_data[1] & 0x03) == 0x00 )
+           (p_data[1] & 0x18) != 0x08 && (p_data[1] & 0x06) != 0x00 )
         {
             pid->probed.i_fourcc = VLC_CODEC_MPGA;
         }
-        else if( p_data[0] == 0xFF && (p_data[1] & 0xF2) == 0xF0 )
+        else if( p_data[0] == 0xFF && (p_data[1] & 0xF6) == 0xF0 )
         {
             pid->probed.i_fourcc = VLC_CODEC_MP4A; /* ADTS */
             pid->probed.i_original_fourcc = VLC_FOURCC('A','D','T','S');
-- 
2.20.1