[vlc-devel] [PATCH] Assertion error while streaming using filters

[Frederik Vestre]

Hi

I get an assertion error while streaming using vlc from git (commit 
f043aa0e11894803f8b0f6f9f940a660d3a36069 ). The stacktrace produced follows:

#0  0x00007ffff7163405 in raise () from /lib/x86_64-linux-gnu/libc.so.6
#1  0x00007ffff7166680 in abort () from /lib/x86_64-linux-gnu/libc.so.6
#2  0x00007ffff715c5b1 in __assert_fail () from /lib/x86_64-linux-gnu/libc.so.6
#3  0x00007ffff795c23e in plane_CopyPixels (p_dst=0xc5a5b58, p_src=0xbe19308) at misc/picture.c:326
#4  0x00007ffff795c2bf in picture_CopyPixels (p_dst=0xc5a5ad0, p_src=0xbe19280) at misc/picture.c:354
#5  0x00007ffff795c2f9 in picture_Copy (p_dst=0xc5a5ad0, p_src=0xbe19280) at misc/picture.c:359
#6  0x00007fffe8331205 in transcode_video_process (p_stream=0xc15398, id=0xba1c0c0, in=0x0, out=0x7fffe6ac0d58) at video.c:750
#7  0x00007fffe832d21d in Send (p_stream=0xc15398, id=0xba1c0c0, p_buffer=<optimized out>) at transcode.c:660
#8  0x00007ffff1860f8e in sout_StreamIdSend (b=0xbe19da0, id=<optimized out>, s=0xc15398) at ../../include/vlc_sout.h:225
#9  Send (p_stream=<optimized out>, id=0xba12cc0, p_buffer=<optimized out>) at duplicate.c:287
#10 0x00007ffff797d532 in sout_InputSendBuffer (p_input=0xba0be50, p_buffer=0x7fffecfe24e0) at stream_output/stream_output.c:237
#11 0x00007ffff79179e4 in DecoderPlaySout (p_sout_block=0x7fffecfe24e0, p_dec=0x10e7958) at input/decoder.c:1681
#12 DecoderProcessSout (p_block=0x0, p_dec=0x10e7958) at input/decoder.c:1803
#13 DecoderProcess (p_dec=0x10e7958, p_block=<optimized out>) at input/decoder.c:2004
#14 0x00007ffff79182fb in DecoderThread (p_data=0x10e7958) at input/decoder.c:938
#15 0x00007ffff76bfb50 in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#16 0x00007ffff72064ad in clone () from /lib/x86_64-linux-gnu/libc.so.6
#17 0x0000000000000000 in ?? ()


The error appears because transcode_video_process tries to copy a video 
buffer (picture) to a duplicate video buffer of a different format. Most 
likely this is a result of a copy-paste from code earlier in the 
function. The destination format used in the original code was the 
format produced by the video decoder, but the source format was the 
format used after applying filters. By using the format of the source 
picture the assertion error disappear, and as far as i have experienced 
the code works perfectly.

I have attached a patch implementing the solution outlined in the 
paragraph above. Could you review and apply the patch in to the main 
repository for VLC?

--

Frederik M. J. Vestre
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: patch
URL: <http://mailman.videolan.org/pipermail/vlc-devel/attachments/20120128/c14b0630/attachment.ksh>