[x264-devel] Re: AW: Re: Error Concealment

[Ryan Dalzell]

On Wednesday 17 January 2007 14:22, List, Peter wrote:
> > On Wednesday 17 January 2007 12:38, List, Peter wrote:
> > > > Betreff: [x264-devel] Re: Error Concealment
> > > >
> > > > On Wednesday 17 January 2007 09:53, Awadh Bihari wrote:
> > > > > Please let me know how to detect the slice loss at the decoder
> > > > > side.
> > > > > It can only be detected if active sps/pps is not equal to sps/pps
> > > > > ?
> > > > Due to arbitrary slice order and the fact that slice headers do not
> > > > tell
> > > > you how many macroblocks are in the slice (and assuming you are
> > > > receiving
> > > > the video data over a network), I believe that the only way is to
> > > > actually
> > > > decode each slice and count the number of macroblocks that it
> > > > contributes
> > > > to a picture. Then, after a suitable period of time to allow late
> > > > packets
> > > > to arrive, if you still haven't decoded all the macroblocks of a
> > > > picture
> > > > then you have lost some slices.
> > >
> > > I dont see why you need to count anything. The slice always tells you
> > > where it's locates in the frame, so you just dump the macroblocks to
> > > that place. If you have arbitrary slice order, things get more
> > difficult,
> > > but in principle it should be the same. You should be able to detect
> > > when the next frame starts (by POC and/or frame_number). If that
> > > happens
> > > you can finalize the previous frame, and that's it. Am I missing
> > > something
> > > here?
> >
> > Yes, I think you are. Let's say that your receive six slices for a
> > picture
> > before a slice with a different POC arrives. Is this all the slices for
> > that picture, or are there supposed to be seven and one got lost? You
> > can't tell, short of decoding all the slices and counting the number of
> > macroblocks provided.
>
> I was expecting that my network interface would sort the packets for me.
> That may be naïve though. A delay would be involved as well in this case,
> but I need a receiving buffer anyways, right?

In reality I think you are right, either the packets will be put into the 
correct order by TCP or if using UDP, then you can do it with sequence 
numbers in RTP. I guess I was being very theoretical in my statements 
because, as I understand it, ASO permits this reordering to be removed from 
the system and the slices decoded as they come in, even with network 
jitter, thus reducing the overall latency in the system.

But if you do take the ASO from the network out of the equation you still 
have the ASO from the encoder, and the problem of knowing when you have all 
your slices for a picture remains the same.

> > This gets even worse when slices can be out of order across picture
> > boundaries. I have searched high and low the spec and I cannot find a
> > statement that precludes this from happening. (Of course, due to the
> > lack
> > of plain English in the spec it could be there.)
>
> I will check if that is possible, but I definitely do not believe it is!
> And yes, this standard is not that easy to read :-)

Please do check as I would love to be proven wrong on this one. Although I 
did an experiment once and reordered one of the conformance bitstreams to 
have slices out of order across picture boundaries, and it crashed the 
reference decoder (JM 10.2). So, based the fact that the reference decoder 
is normative we could interpret that as saying ASO across picture 
boundaries is not supported. Or it could be a bug :).

> > Also this could happen
> > when packets become reordered due to network jitter or taking different
> > routes. Then you don't even know that you have the last slice you are
> > going
> > to get when the POC changes. So you have to wait a while to see if
> > there
> > are more packets to arrive containing slices for the current picture.
> >
> >
> >
> > IMO, it's pretty horrible
>
> I have heard a lot of different things about FMO (Flexible Macroblock
> Ordering). Some said it was absolutely easy to implement (on PC: simply
> have more than one input-buffer, one for every slice-group. Then, while
> decoding from left to right, top to bottom simply switch to the right
> buffer). Some others said it was horrible.
>
> Could you comment?

To be honest I somehow avoided being the person to implement FMO in any of 
our projects (maybe I avoided it on purpose ;)), but from what I can gather 
from others it isn't as horrible as it first appears. There are two nasty 
bits that I am aware of:

1. We write hardware decoders and the amount of expensive on-chip RAM 
required to calculate the availability of macroblock neighbours is just too 
much to justify for this one task. So this calculation has to be done in 
software and beforehand and downloaded to the chip, which is a complexity 
that we don't like.

2. Without FMO the loop filter can work in the pipeline filtering each 
macroblock as it is decoded and without requiring too much RAM. With FMO 
this cannot be done and solving this really messes up the design.

Cheers,
-Ryan
-- 
Ryan Dalzell
4i2i Communications Ltd.
39 Palmerston Place
Edinburgh
EH12 5AU
Scotland, UK
http://www.4i2i.com

-- 
This is the x264-devel mailing-list
To unsubscribe, go to: http://developers.videolan.org/lists.html