[x264-devel] Behaviour of Annex B encoding: bug or not?

[Alex Giladi]

You're right. Will read again before answering e-mails ;-)

On Wed, Jan 6, 2010 at 4:09 AM, Gil Pedersen <gil at cmi.aau.dk> wrote:
> On 06/01/2010, at 01.36, Alex Giladi wrote:
>
>> Gil,
>> Quoting section 3 of the RFC:  "This payload specification can only be
>> used to carry the "naked" H.264 NAL unit stream over RTP, and not the
>> bitstream format discussed in Annex B of H.264". Escaping is defined
>> in Annex B (section B.1).
>> Haven't played with this (I work with plain old MPEG-2 TS), but this
>> is my interpretation of the specs.
>
> You're reading it wrong then. Escaping (by inserting emulation prevention bytes) is defined for NAL units in section 7.3.1. Annex B only defines further encapsulation of these bits but doesn't change the actual representation.
> /Gil
>
>> Also, there is no "de-facto standard": there is only compliance or
>> lack of compliance.
>> Alex.
>>
>> On Tue, Jan 5, 2010 at 5:44 PM, Gil Pedersen <gil at cmi.aau.dk> wrote:
>>> On 05/01/2010, at 20.49, Alex Giladi wrote:
>>>
>>>> RTP payload (at RFC 3984) uses RBSP. Maybe it's worth adding "rtp" as
>>>> an output option?
>>>> --ag
>>>
>>> Is this true? I see no mention of RBSP in the RFC. The RFC 3984 payload is based on NAL units, which according to the H.264 spec consists of a 1-byte header followed by RBSP _and_ the necessary emulation bytes. Further, as mentioned, the H.264 reference encoder seems to output RTP packets with the emulation bytes.
>>>
>>> As far as I can tell any decoder that expects raw RBSP + 1 byte header is broken and x264 should never allow this to be generated unless it can be proven that it's the de-facto standard.
>>>
>>> /Gil
>>>
>>>> On Tue, Jan 5, 2010 at 2:04 PM, Jason Garrett-Glaser
>>>> <darkshikari at gmail.com> wrote:
>>>>> On Tue, Jan 5, 2010 at 1:41 PM, Philip Spencer
>>>>> <pspencer at fields.utoronto.ca> wrote:
>>>>>> First a disclaimer: I know very little about the H.264 spec, and am just
>>>>>> trying to resolve an interoperability issue we are having, so please bear
>>>>>> with me if I misstate or misunderstand something.
>>>>>>
>>>>>> In the routine x264_nal_encode (in common/common.c), the flag b_annexb
>>>>>> controls whether or not to add a NAL start code (00 00 00 01) to the
>>>>>> beginning of the packet, but does not control whether or not to do the
>>>>>> escaping of sequences of the form 00 00 00/1/2/3 by inserting a 03 byte into
>>>>>> the third position -- that escaping is ALWAYS done, even if b_annexb is not
>>>>>> set.
>>>>>>
>>>>>> Is this a bug, or is it meant to be that way? (I don't have access to the
>>>>>> text of Annex B).
>>>>>
>>>>> We didn't make it an option because we didn't know of any devices that
>>>>> expected non-escaped bytestreams.  All containers we knew that didn't
>>>>> want Annex-B startcodes still expected escaped NAL units.
>>>>>
>>>>>> It certainly breaks interoperability with several devices. In particular,
>>>>>> x264 cannot be used with the Ekiga softphone application to communicate with
>>>>>> the "LifeSize Room" brand of videoconferencing equipment: that device
>>>>>> expects non-Annex-B packets over RTP, and cannot handle the extra 03 byte
>>>>>> that is inserted. The result is that the session parameters (like stream
>>>>>> resolution and other such settings, which often contain multiple zero bytes
>>>>>> in a row) get completely garbled because of the Annex-B bytestream encoding.
>>>>>>
>>>>>> Also, if one attempts to sniff the network traffic with software such as
>>>>>> WireShark, it too chokes on the unexpected 03 bytes in RTP packets.
>>>>>>
>>>>>> It would seem to me that this is a bug: if Annex B bytestream encoding is
>>>>>> not desired, such as for an RTP packet, then the extra escape bytes should
>>>>>> not be inserted.
>>>>>>
>>>>>> On our system, I have applied the patch below to common.c and then
>>>>>> H.264 connectivity to the LifeSize Room videoconferencing equipment works
>>>>>> just fine.
>>>>>>
>>>>>> On the other hand, from a quick glance at the source code of the reference
>>>>>> encoder/decoder, it seems that it behaves the same way as x264: always
>>>>>> inserts the escape byte. Is this a bug in the reference encoder/decoder too,
>>>>>> or does the text of Annex B specify that ALL H.264 streams should have the
>>>>>> extra bytes inserted, even when bytestream encoding is not being used?
>>>>>>
>>>>>> In the latter case, then obviously the LifeSize brand videoconference units
>>>>>> are buggy, but since I know they interperate will over H.264 with a wide
>>>>>> range of units from other manufacturers there must be a lot of buggy devices
>>>>>> out there -- would it be worth adding an extra flag to x264 that says "do
>>>>>> bytestream encoding only in Annex B mode, for compatibility with
>>>>>> devices that cannot handle it in RTP mode"?
>>>>>
>>>>> The best solution here is probably to add another parameter to control
>>>>> the escaping.  b_escaped_nals or similar.  In the case of
>>>>> b_escaped_nals == 0, a straight memcpy would be used.
>>>>>
>>>>> Dark Shikari
>>>>> _______________________________________________
>>>>> x264-devel mailing list
>>>>> x264-devel at videolan.org
>>>>> http://mailman.videolan.org/listinfo/x264-devel
>>>>>
>>>> _______________________________________________
>>>> x264-devel mailing list
>>>> x264-devel at videolan.org
>>>> http://mailman.videolan.org/listinfo/x264-devel
>>>
>>> _______________________________________________
>>> x264-devel mailing list
>>> x264-devel at videolan.org
>>> http://mailman.videolan.org/listinfo/x264-devel
>>>
>> _______________________________________________
>> x264-devel mailing list
>> x264-devel at videolan.org
>> http://mailman.videolan.org/listinfo/x264-devel
>
> _______________________________________________
> x264-devel mailing list
> x264-devel at videolan.org
> http://mailman.videolan.org/listinfo/x264-devel
>