[x264-devel] Behaviour of Annex B encoding: bug or not?

[Philip Spencer]

On Tue, 5 Jan 2010, Alex Giladi wrote:

> RTP payload (at RFC 3984) uses RBSP. Maybe it's worth adding "rtp" as
> an output option?
> --ag

That would solve our problems, certainly (once I hack ekiga/opal to pass 
that option to x264, which is not a big deal since I already had to make 
considerable changes to it to get it to use the x264 libraries correctly 
in the first place.)

I guess it makes sense to call the option "rtp" if you know for sure RTP 
payloads are the only thing that requires unescaped RBSP, or to call it 
by a more general name if there's the possibility of other formats needing 
it too. (I'm at the limit of my knowledge here!)

Thanks,

Philip

>
> On Tue, Jan 5, 2010 at 2:04 PM, Jason Garrett-Glaser
> <darkshikari at gmail.com> wrote:
>> On Tue, Jan 5, 2010 at 1:41 PM, Philip Spencer
>> <pspencer at fields.utoronto.ca> wrote:
>>> First a disclaimer: I know very little about the H.264 spec, and am just
>>> trying to resolve an interoperability issue we are having, so please bear
>>> with me if I misstate or misunderstand something.
>>>
>>> In the routine x264_nal_encode (in common/common.c), the flag b_annexb
>>> controls whether or not to add a NAL start code (00 00 00 01) to the
>>> beginning of the packet, but does not control whether or not to do the
>>> escaping of sequences of the form 00 00 00/1/2/3 by inserting a 03 byte into
>>> the third position -- that escaping is ALWAYS done, even if b_annexb is not
>>> set.
>>>
>>> Is this a bug, or is it meant to be that way? (I don't have access to the
>>> text of Annex B).
>>
>> We didn't make it an option because we didn't know of any devices that
>> expected non-escaped bytestreams.  All containers we knew that didn't
>> want Annex-B startcodes still expected escaped NAL units.
>>
>>> It certainly breaks interoperability with several devices. In particular,
>>> x264 cannot be used with the Ekiga softphone application to communicate with
>>> the "LifeSize Room" brand of videoconferencing equipment: that device
>>> expects non-Annex-B packets over RTP, and cannot handle the extra 03 byte
>>> that is inserted. The result is that the session parameters (like stream
>>> resolution and other such settings, which often contain multiple zero bytes
>>> in a row) get completely garbled because of the Annex-B bytestream encoding.
>>>
>>> Also, if one attempts to sniff the network traffic with software such as
>>> WireShark, it too chokes on the unexpected 03 bytes in RTP packets.
>>>
>>> It would seem to me that this is a bug: if Annex B bytestream encoding is
>>> not desired, such as for an RTP packet, then the extra escape bytes should
>>> not be inserted.
>>>
>>> On our system, I have applied the patch below to common.c and then
>>> H.264 connectivity to the LifeSize Room videoconferencing equipment works
>>> just fine.
>>>
>>> On the other hand, from a quick glance at the source code of the reference
>>> encoder/decoder, it seems that it behaves the same way as x264: always
>>> inserts the escape byte. Is this a bug in the reference encoder/decoder too,
>>> or does the text of Annex B specify that ALL H.264 streams should have the
>>> extra bytes inserted, even when bytestream encoding is not being used?
>>>
>>> In the latter case, then obviously the LifeSize brand videoconference units
>>> are buggy, but since I know they interperate will over H.264 with a wide
>>> range of units from other manufacturers there must be a lot of buggy devices
>>> out there -- would it be worth adding an extra flag to x264 that says "do
>>> bytestream encoding only in Annex B mode, for compatibility with
>>> devices that cannot handle it in RTP mode"?
>>
>> The best solution here is probably to add another parameter to control
>> the escaping.  b_escaped_nals or similar.  In the case of
>> b_escaped_nals == 0, a straight memcpy would be used.
>>
>> Dark Shikari
>> _______________________________________________
>> x264-devel mailing list
>> x264-devel at videolan.org
>> http://mailman.videolan.org/listinfo/x264-devel
>>
> _______________________________________________
> x264-devel mailing list
> x264-devel at videolan.org
> http://mailman.videolan.org/listinfo/x264-devel
>

--------------------------------------------+-------------------------------
Philip Spencer  pspencer at fields.utoronto.ca | Director of Computing Services
Room 336        (416)-348-9710  ext3036     | The Fields Institute for
222 College St, Toronto ON M5T 3J1 Canada   | Research in Mathematical Sciences