[vlc-devel] [vlc-commits] avcodec: fix direct rendering with some codecs
Rafaël Carré
funman at videolan.org
Mon Dec 12 01:21:01 CET 2011
Le Sun, 11 Dec 2011 23:44:33 +0100,
Laurent Aimar <fenrir at elivagar.org> a écrit :
> On Sun, Dec 11, 2011 at 03:50:33PM -0500, Rafaël Carré wrote:
> > So chroma is I2AL
> >
> > The picture is here:
> > width/height = visible width/height = 720x486
> > For the 3 planes though:
> > width/height = 736x498
> > visible width/height = 720x486
> That's normal.
> For the width:
> U/V -> 720/2 -> 360 -> 368 (mod 16)
> Y -> 2 * U/V (I422° -> 2*368 -> 736 (and already mod 16)
>
> For the height:
> Y/U/V -> 486 -> 496 (mod 16)
> and there is 2 extra hardcoded lines to ensure that the pictures will
> be compatible with avcodec_align_dimensions() for H264 (otherwise
> dr will be disabled by ffmpeg_GetFrameBuf()).
>
> >
> > [0x6a18628] avcodec decoder debug: using direct rendering
> > ==2248== Invalid write of size 8
> > ==2248== at 0x145B00A8: ??? (proresdsp.asm:428)
> > ==2248== by 0x14453BB0: decode_slice_plane (proresdec.c:492)
> > ==2248== by 0x1445403F: decode_slice (proresdec.c:587)
> > ==2248== by 0x144D690E: avcodec_default_execute (utils.c:568)
> > ==2248== by 0x1445315C: decode_frame (proresdec.c:624)
> > ==2248== by 0x144D7277: avcodec_decode_video2 (utils.c:866)
> > ==2248== by 0x13EAEF70: DecodeVideo (video.c:590)
> > ==2248== by 0x5085966: DecoderDecodeVideo (decoder.c:1512)
> > ==2248== by 0x508567F: DecoderProcess (decoder.c:1878)
> > ==2248== by 0x508577A: DecoderThread (decoder.c:938)
> > ==2248== by 0x532AEFB: start_thread (pthread_create.c:304)
> > ==2248== by 0x582585C: clone (clone.S:112)
> > ==2248== Address 0x96c5650 is 0 bytes after a block of size 1,466,112 alloc'd
> > ==2248== at 0x4C278E8: memalign (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
> > ==2248== by 0x4C27997: posix_memalign (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
> > ==2248== by 0x50C05AF: picture_NewFromResource (vlc_common.h:935)
> > ==2248== by 0x50C1267: picture_pool_NewFromFormat (picture_pool.c:138)
> > ==2248== by 0x50AF3DE: vout_InitWrapper (vout_wrapper.c:153)
> > ==2248== by 0x50A78FB: ThreadStart (video_output.c:1360)
> > ==2248== by 0x50A8085: Thread (video_output.c:1499)
> > ==2248== by 0x532AEFB: start_thread (pthread_create.c:304)
> > ==2248== by 0x582585C: clone (clone.S:112)
> > ==2248==
> > ==2248==
> >
> >
> > ff_prores_idct_put_10_sse4 () at libavcodec/x86/proresdsp.asm:428
> > 428 idct_put_fn sse4, 16
> > (gdb) bt
> > #0 ff_prores_idct_put_10_sse4 () at libavcodec/x86/proresdsp.asm:428
> > #1 0x0000000014453bb1 in decode_slice_plane (ctx=<optimized out>, td=0x2e00, buf=<optimized out>, data_size=<optimized out>,
> > out_ptr=0x96c2290, linesize=1472, mbs_per_slice=8, blocks_per_mb=2, plane_size_factor=4, qmat=0x66e1270)
> > at libavcodec/proresdec.c:492
> > #2 0x0000000014454040 in decode_slice (avctx=<optimized out>, td=0xc328f20) at libavcodec/proresdec.c:587
> > #3 0x00000000144d690f in avcodec_default_execute (c=0x66df560, func=0x14453d10 <decode_slice>, arg=<optimized out>,
> > ret=<optimized out>, count=112, size=4128) at libavcodec/utils.c:568
> > #4 0x000000001445315d in decode_picture (avctx=0x66df560, pic_num=1, ctx=0x66e0f20) at libavcodec/proresdec.c:624
> > #5 decode_frame (avctx=0x66df560, data=0x66dff60, data_size=0x8383cec, avpkt=0x8383c90) at libavcodec/proresdec.c:669
> > #6 0x00000000144d7278 in avcodec_decode_video2 (avctx=0x66df560, picture=0x66dff60, got_picture_ptr=0x8383cec,
> > avpkt=0x8383c90) at libavcodec/utils.c:866
> > #7 0x0000000013eaef71 in DecodeVideo (p_dec=0x6a18628, pp_block=<optimized out>)
> > at ../../../../vlc/modules/codec/avcodec/video.c:590
> > #8 0x0000000005085967 in DecoderDecodeVideo (p_dec=0x6a18628, p_block=0x7291190) at ../../vlc/src/input/decoder.c:1512
> > #9 0x0000000005085680 in DecoderProcessVideo (b_flush=false, p_block=0x7291190, p_dec=0x6a18628)
> > at ../../vlc/src/input/decoder.c:1878
> > #10 DecoderProcess (p_dec=0x6a18628, p_block=<optimized out>) at ../../vlc/src/input/decoder.c:2052
> > #11 0x000000000508577b in DecoderThread (p_data=0x6a18628) at ../../vlc/src/input/decoder.c:938
> > #12 0x000000000532aefc in start_thread (arg=0x8384700) at pthread_create.c:304
> > #13 0x000000000582585d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
> > #14 0x0000000000000000 in ?? ()
> > (gdb)
>
> Did yoy try the decoder from ffmpeg iof libav? Just in case.
> Also, did you
> try disabling sse4 and see if the invalid write still happen?
==25067== Invalid write of size 2
==25067== at 0x15FC7790: put_pixels (proresdsp.c:43)
==25067== by 0x15FC77F6: prores_idct_put_c (proresdsp.c:51)
==25067== by 0x15FC6AE4: decode_slice_plane (proresdec.c:492)
==25067== by 0x15FC7054: decode_slice (proresdec.c:587)
==25067== by 0x1605E399: avcodec_default_execute (utils.c:569)
==25067== by 0x15FC71E3: decode_picture (proresdec.c:624)
==25067== by 0x15FC7372: decode_frame (proresdec.c:669)
==25067== by 0x1605F198: avcodec_decode_video2 (utils.c:867)
==25067== by 0x15AAEF70: DecodeVideo (video.c:590)
==25067== by 0x5085966: DecoderDecodeVideo (decoder.c:1512)
==25067== by 0x508567F: DecoderProcess (decoder.c:1878)
==25067== by 0x508577A: DecoderThread (decoder.c:938)
==25067== Address 0xfd50460 is not stack'd, malloc'd or (recently)
free'd ==25067==
==25067==
==25067== ---- Attach to debugger ? --- [Return/N/n/Y/y/C/c] ----
Same bug with C instead of asm.
0x0000000015fc7790 in put_pixels (dst=0xfd4fea0, stride=736,
in=0xc8665a0) at libavcodec/proresdsp.c:43
43 dst[dst_offset + x] =
CLIP_AND_BIAS(in[src_offset]); (gdb) print dst
$1 = (uint16_t *) 0xfd4fea0
(gdb) print dst_offset
$2 = 736
(gdb) print x
$3 = 0
(gdb)
(gdb) frame 1
#1 0x0000000015fc77f7 in prores_idct_put_c (out=0xfd4fea0,
linesize=1472, block=0xc8665a0, qmat=0x69db6d0) at
libavcodec/proresdsp.c:51 51 put_pixels(out, linesize >> 1,
block); (gdb) print out
$4 = (uint16_t *) 0xfd4fea0
(gdb) print linesize
$5 = 1472
(gdb) frame 2
#2 0x0000000015fc6ae5 in decode_slice_plane (ctx=0x69db380,
td=0xc866500, buf=0xca7620b
"\212_D\261,S\024\306Ѵ_\027\305\321te\031rH\n\210̄R<#\242?8\212\222'I5$GZ%\254\212\243$\201\322&#\",\"\262/\n\220\342P\231%\311r`\322t\221:I\250Mbjvd\205&)jH)\022̱3Ŋo\026r\205-\231l\301>\026l\361b\343~qg\036au\364\245#_H\331\r\363\002\242_3\204O0\352J\230\245):RȂki\032\352\352\373jg\b\274X\250\063\023ҏ\363\006\213\226\023\b\304\325\301\213\302G\374\016w\006/\006B=\300\066a\317\060\004",
<incomplete sequence \333>, data_size=167, out_ptr=0xfd4d0a0,
linesize=1472, mbs_per_slice=8, blocks_per_mb=2, plane_size_factor=4,
qmat=0x69db6d0) at libavcodec/proresdec.c:492 492
ctx->dsp.idct_put(out_ptr + linesize * 4, linesize, block_ptr,
qmat); (gdb) print linesize $6 = 1472 (gdb) print mb_num
$7 = 0
(gdb) print mbs_per_slice
$8 = 8
(gdb) print blocks_per_mb
$9 = 2
(gdb)
(gdb) frame 3
#3 0x0000000015fc7055 in decode_slice (avctx=0x699ebe0, td=0xc866500)
at libavcodec/proresdec.c:587
587 decode_slice_plane(ctx, td, buf + hdr_size + y_data_size
+ u_data_size, (gdb) print pic
$10 = (AVFrame *) 0x69db3d0
(gdb) print *pic
$11 = {data = {0xfbea280 "", 0xfc9d200 "", 0xfcf69c0 "", 0x0}, linesize
= { 1472, 736, 736, 0}, base = {0x0, 0x0, 0x0, 0x0}, key_frame = 1,
pict_type = 0, pts = -9223372036854775808, coded_picture_number = 0,
display_picture_number = 0, quality = 0, age = 1073741824, reference
= 0, qscale_table = 0x0, qstride = 0, mbskip_table = 0x0, motion_val =
{0x0, 0x0}, mb_type = 0x0, motion_subsample_log2 = 0 '\000',
opaque = 0xab67880, error = {0, 0, 0, 0}, type = 2, repeat_pict = 0,
qscale_type = 0, interlaced_frame = 1, top_field_first = 0, pan_scan
= 0x0, palette_has_changed = 0, buffer_hints = 0, dct_coeff = 0x0,
ref_index = { 0x0, 0x0}, reordered_opaque = 3, hwaccel_picture_private
= 0x0, pkt_pts = 0, pkt_dts = 0, owner = 0x0, thread_opaque = 0x0,
nb_samples = 0, extended_data = 0x0}
(gdb)
V plane is at 0xfcf69c0
(gdb) frame 0
(gdb) print &dst[dst_offset+x]
$13 = (uint16_t *) 0xfd50460
invalid access is at V plane + 1472
BTW, pic is interlaced.
And FFmpeg doesn't have the bug.
avcodec_align_dimensions2() was changed when ProRes decoder was
committed in FFmpeg.
height is aligned on 32 pixels (= 2 blocks) for a bunch of codecs (to
account for interlacing), and ProRes is missing from the list in libav.
The side effect is that it disables direct rendering because the VLC
picture is too small, unless we align on 32 pixels too.
However I think in picture_Setup() we don't know the video codec, nor
if the stream is interlaced.
--
Rafaël Carré
More information about the vlc-devel
mailing list