[x264-devel] Re: H.264 (over MPEG2-TS) over packet-lossy networks

[Alex Izvorski]

On Tue, 2007-02-13 at 19:42 +0100, Gonzalo Berdeal wrote:
> Hello everybody,
> 
> I am currently developing a packetizer and depacketizer of H.264 over
> TS.  For this task I have the Amendment 3 of MPEG-2 Systems.
> I am using ffmpeg/libavcodec for coding and decoding to H.264 (coding
> is actually done by x264).
> 
> Once done the transmission the depacketizer will have as an input the
> TS packets, but eventually some packets may have been lost (due to UDP
> protocol) or may have some bit errors (wireless).
> Does anybody have any sugestion about how to handle these problems?
> For example, if the continuity_counter of the TS packets has a
> discontinuity, may be I could send to the decoder the last error-safe
> received Slice (frame).
> 
> Could anybody give me some ideas?
> Does anybody know some good references on this topic?
> 
> Thank you very much in advance.
> 
> Gonzalo Berdeal.

Hello,

Yes, a few random ideas...

* you could put slice support back in x264.  this existed in prior
revisions (before r609) but it was very closely tied to threading, so it
got ripped out during the threading overhaul.  it shouldn't be too hard
to put it back in.

* you could tweak the x264 encoder so that slice boundaries match packet
boundaries, i.e. each N network packets are (almost exactly) a slice.
that would be some kind of option that ends a slice and starts a new one
once they it is close to a certain size.

* you could tweak the decoder you're using to replace a lost slice with
the same image data as in the previous picture (optionally shifted by
the previous picture's motion vectors).

* instead of tossing a slice with errors completely, you could decode it
up to the beginning of bad data (ts cc discontinuity etc), and only
replace the remainder with data from the previous picture (not sure this
will add much at low bitrates, at high bitrates it will roughly halve
the visible distortion).  naturally you can't rely on errors in decoding
the h.264 stream since those are likely to occur some distance after the
start of bad data, and you'll get a few macroblocks of essentially
random stuff.

* ts continuity counters don't always work, since there is only a small
number of values.  however if do you see a discontinuity, that's
probably either a gap or out-of-order.  bearing in mind that network
packets in ts over udp typically have 7 ts packets, if you know they're
packed sequentially you can have a pretty good way to detect gaps... for
example, if the last cc was 3 and you see 11 or 1, it's a good bet you
lost one or two network packets... however your video ts packets may
also be mixed with audio etc so that gets a little complicated.  does
your network deliver packets out of order?  it may be possible to fix
out of order by keeping a queue of recent transport packets and
reordering them (the reordering criteria is presumably that you can
remove gaps without creating new gaps).

* do you want to use pes packets with a defined length, or with a length
of 0? (as the standard says you should)  if the former, you may have to
break up nals between packets (depends on bitrate).  if the latter, you
need some way of making sure your received pes packet is the right size.

* you could calculate checksums in pes packets, and check them on the
receiving end.  pretty sure the ffmpeg's ts packetizer/depacketizer
doesn't handle that now.  one tricky detail is that checksums for each
packet are sent in the header of the next packet with that pid.

* is the network you're using burst-lossy?  i.e. does it tend to lose
several packets in a row?  this affects what the best strategy is.  i'd
think that udp packets come with checksums already so you're unlikely to
have bit errors from wireless, but you may have to do something to turn
them on.

I'd be interested to see what you come up with.  Is this a research
project?  Open-source?  Commercial?  Academia?

Regards,
--Alex


-- 
This is the x264-devel mailing-list
To unsubscribe, go to: http://developers.videolan.org/lists.html