[vlc-devel] Re: Problems with using FFMPEG's "h264.c" to decode H264/RTP streams

[Michael Niedermayer]

Hi

On Tuesday 19 July 2005 04:24, Ross Finlayson wrote:
> Michael (cc. "vlc-devel"),
>
> I have recently been trying to update VLC (and the appropriate LIVE.COM
> RTSP/RTP client code) to receive and render incoming H.264 RTP streams,
> using your FFMPEG H.264 decoder ("libavcodec/h264.c").  Unfortunately I
> have run into a snag that - to be solved cleanly - may require a minor
> change to your code.
>
> The problem is that your function "decode_nal_units()" does not like to be
> passed a buffer that contains just a single NAL unit; instead, it seems to
> want a buffer that contains all NAL units that make up an 'access unit'
> (i.e., frame).  If "decode_nal_units()" is fed just a single NAL unit, then
> - at the time that "ff_er_frame_end()" is called, "error_count" has not yet
> been decremented to zero, and so we get errors about 'concealing' data.
>
> However, the H.264/RTP client code (in the LIVE.COM libraries and VLC)
> wants to feed the decoder one NAL unit at a time.  There are two reasons
> for this.  First, it improves resiliency to packet loss.  (If a RTP packet
> is lost, then this will cause the loss only of one NAL - not of the entire
> 'access unit' (frame).)  

how does sending all the available NAL units for a access unit make this 
worse? you just send what you have


> Second, your code (quite reasonably) expects a 
> "nal_size" field to precede each NAL unit in the buffer.  The cleanest
> place to prepend this size field is in VLC's RTP interface code
> ("modules/demux/livedotcom.cpp"), but unfortunately this can be done only
> if this code is delivered one NAL unit at a time - not an entire 'access
> unit' at a time.

hmm, the nal_size field is only required if is_avc==1 which is true only if 
the first byte of extradata is 1 which i thought although iam not sure is 
only set by some commercial h264-in-avi encoder


>
> Given this, would it be possible for you to update your code so that
> "decode_nal_units()" can properly handle a buffer that contains just a
> single NAL unit?

it shouldnt be too difficult to support this except that the end of a access 
unit needs to be detected somehow, 
possibility 1 would be to detect the last slice but if that where lost due to 
packet loss it would mean loosing the whole frame, which is not really the 
thing anyone would want
possibility 2 would be to wait for the first NAL unit of the next frame, but 
that would add a one frame delay and in general seems a little tricky and 
complicated
possibility 3 would be a CODEC_FLAG_NOTEND or so that would be set by the user 
app if a complete NAL unit/slice which isnt the last one for the current 
frame were feeded 
possibility 4 would be to pass the NAL units over 
AVCodecContext.slice_count/offset like its done with real media by mplayer 
IIRC

[...]
-- 
Michael

-- 
This is the vlc-devel mailing-list, see http://www.videolan.org/vlc/
To unsubscribe, please read http://developers.videolan.org/lists.html