[vlc-devel] [WIP/PATCH] avcodec: ensure correct buffer size

[Tristan Matthews]

On Wed, Oct 23, 2013 at 6:08 AM, Ilkka Ollakka <ileoo at videolan.org> wrote:
> On Wed, Oct 23, 2013 at 01:04:09AM -0400, Tristan Matthews wrote:
>> Given this sample:
>> https://trac.videolan.org/vlc/attachment/ticket/6505/amono8000sps_10sec.wav
>> transcoding to ulaw (via avcodec) fails with repeated "filling error on encode"
>> messages.
>
> Hi,
>
>> This patch tries to address this by resizing the target buffers since
>> avcodec_fill_audio_frame will reject buffers that are too small.
>
> Could be that my code just gives wrong buffers, it doesn't seem correct
> to realloc input buffer size in here.

Yes, in this situation p_aout_buf's buffer size is 800, which
libavcodec rejects.

>
>
>> +        /* ensure that buffer has acceptable size for avcodec */
>> +        const int needed_size = av_samples_get_buffer_size(NULL,
>> +                p_sys->p_context->channels, p_sys->frame->nb_samples,
>> +                p_sys->p_context->sample_fmt, align);
>
> This should be done in Open maybe, when p_sys->i_buffer_out is
> calculated and set p_sys->i_buffer_out to this size?

As I understand it, i_buffer_out is just used to deinterleave buffers, so for
non-planar audio (as in this case) it is not used.

>
>> +        if( unlikely( buf_size < needed_size ) )
>> +        {
>> +            if( p_sys->b_planar)
>> +            {
>> +                p_sys->p_buffer = realloc(p_sys->p_buffer, needed_size);
>> +                if( !p_sys->p_buffer )
>> +                    return p_chain;
>
> You should update p_sys->i_buffer here if you have changed it size,
> otherwise it most likely just comes back in realloc on next turn and
> other issues could happen.
>
> Does the ulaw set p_sys->i_frame_size?

This case is for libavcodec's pcm ulaw encoder ("mulaw"), which sets
context->frame_size to 0.

I believe the proper solution is probably to do some buffering of the
input to ensure that avcodec_fill_audio_frame is always called with a
buffer size that it likes. I will revise my patch and resend.

Best,
Tristan

-- 
Tristan Matthews
web: http://tristanswork.blogspot.com