[vlc-devel] [WIP/PATCH] avcodec: ensure correct buffer size

[Ilkka Ollakka]

On Wed, Oct 23, 2013 at 01:04:09AM -0400, Tristan Matthews wrote:
> Given this sample:
> https://trac.videolan.org/vlc/attachment/ticket/6505/amono8000sps_10sec.wav
> transcoding to ulaw (via avcodec) fails with repeated "filling error on encode"
> messages.

Hi,

> This patch tries to address this by resizing the target buffers since
> avcodec_fill_audio_frame will reject buffers that are too small.

Could be that my code just gives wrong buffers, it doesn't seem correct
to realloc input buffer size in here.


> +        /* ensure that buffer has acceptable size for avcodec */
> +        const int needed_size = av_samples_get_buffer_size(NULL,
> +                p_sys->p_context->channels, p_sys->frame->nb_samples,
> +                p_sys->p_context->sample_fmt, align);

This should be done in Open maybe, when p_sys->i_buffer_out is
calculated and set p_sys->i_buffer_out to this size?

> +        if( unlikely( buf_size < needed_size ) )
> +        {
> +            if( p_sys->b_planar)
> +            {
> +                p_sys->p_buffer = realloc(p_sys->p_buffer, needed_size);
> +                if( !p_sys->p_buffer )
> +                    return p_chain;

You should update p_sys->i_buffer here if you have changed it size,
otherwise it most likely just comes back in realloc on next turn and
other issues could happen.

Does the ulaw set p_sys->i_frame_size?

-- 
Ilkka Ollakka
Nobody wants constructive criticism.  It's all we can do to put up with
constructive praise.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 490 bytes
Desc: Digital signature
URL: <http://mailman.videolan.org/pipermail/vlc-devel/attachments/20131023/5e3cb4ea/attachment.sig>