[vlc-devel] Transcoding bug

[Anatoly madRat L.Berenblit]

Hello developers. I've using vlc for transcoding in the chain " 
internet-url --> vlc (for transcode from mms to rcp with mp4v/mp4a) --> 
DarwinStreamingServer (stream to my local network)

Some mms channels can be opened only with new vlc 0.9.8a (i've using 
0.8.6d - because it can do everythink that I need). I've donwload neew 
one and install it. So I face the follows problem:

when I try to stream with b-frames' 
'#transcode{venc=avcodec{bframes=1},vcodec=mp4v,vb=400,fps=20,width=320,height=240,acodec=mp4a,channels=1,ab=48}:rtp{port-audio=20000,port-video=20002,dst=127.0.0.1,sdp=file:///usr/local/movies/1.sdp}
no video sending to DarwinStreaming Server, but If i try to disable 
bframes or transcode to file (std instead rcp) - everything works.

I've research the problem and want to report to you.

The problem is in encoder module(modules/codec/avcodec/encoder.c), it 
generates invalid dts. and that wrong value sending into 
modules/stream_out/rtp.c.

rtp.c sends data to socket in special thread, that mwait(sleep) if frame 
has come to rpc-streamer much earlier, but with wrong dts is waits about 
290000 years

*rtp.c near line 1524*
mtime_t  i_date = out->i_dts + i_caching;
ssize_t  len = out->i_buffer;

mwait( i_date ); // i_date = "i_caching-0x8000000000000000" and systems 
must wait about 290000 years

the problem is in modules/codec/avcodec/encoder.c
in function: static block_t *EncodeVideo( encoder_t *p_enc, picture_t 
*p_pict ) (line ~ 849)

/* Set the pts of the frame being encoded (segfaults with mpeg4!)*/
*if( p_enc->fmt_out.i_codec != VLC_FOURCC( 'm', 'p', '4', 'v' ) )*
{
...
}
else
{
    frame.pts = (int64_t)AV_NOPTS_VALUE;
}

in the follows code system suppose that frame.pts have valid value, not 
AV_NOPTS_VALUE (==0x8000000000000000)

in VLC 0.8.6 was
#if LIBAVCODEC_BUILD < 4702
    /* Set the pts of the frame being encoded (segfaults with mpeg4!)*/
    if( p_enc->fmt_out.i_codec == VLC_FOURCC( 'm', 'p', 'g', 'v' ) ||
        p_enc->fmt_out.i_codec == VLC_FOURCC( 'm', 'p', '1', 'v' ) ||
        p_enc->fmt_out.i_codec == VLC_FOURCC( 'm', 'p', '2', 'v' ) )
#else
    if( 1 )
#endif
    {
...
}
else
{
    frame.pts = (int64_t)AV_NOPTS_VALUE;
}

So, looks that bug appears where code was ported.

Best regards, madRat.

PS. Also not fixed a Bug with wrong mp4 video transcoding: vlc generates 
mpeg4 video with long sequences with 0xFF 0xFF  0xFF ...0xFF values.
Correct video stream should be not have any regular sequence. (if it is 
=> it may be compressed => it should not be in data where bitrate is 
limited and every saved bit is important! I not belive that mpeg4 team 
suppose that is a good idea to have some reqular sequences.)

*The best player should be the best in all, that it does!!!

*