[vlc-devel] [PATCH 1/1] [transcode] Fix allocation of picture_t without full initialization

[David Flynn]

On 2008-11-04, Laurent Aimar <fenrir at via.ecp.fr> wrote:
> On Tue, Nov 04, 2008, David Flynn wrote:
>> -    p_pic = malloc( sizeof(picture_t) );
>> +    p_pic = picture_New( p_dec->fmt_out.video.i_chroma,
>> +                         p_dec->fmt_out.video.i_width,
>> +                         p_dec->fmt_out.video.i_height,
>> +                         p_dec->fmt_out.video.i_aspect );
>>      if( !p_pic ) return NULL;
>>      p_dec->fmt_out.video.i_chroma = p_dec->fmt_out.i_codec;
>> -    vout_AllocatePicture( VLC_OBJECT(p_dec), p_pic,
>> -                          p_dec->fmt_out.video.i_chroma,
>> -                          p_dec->fmt_out.video.i_width,
>> -                          p_dec->fmt_out.video.i_height,
>> -                          p_dec->fmt_out.video.i_aspect );
>> -
>> -    if( !p_pic->i_planes )
>> -    {
>> -        free( p_pic );
>> -        return NULL;
>> -    }
>> -
>> -    p_pic->pf_release = video_release_buffer;
>> -    p_pic->p_sys = malloc( sizeof(picture_sys_t) );
>> +    p_pic->p_sys = calloc( 1, sizeof(picture_sys_t) );
>>      if( !p_pic->p_sys )
>>      {
>> -        free( p_pic );
>> +        p_pic->pf_release( p_pic );
>  Using picture_Release would be cleaner.
> Otherwise, it seems fine and thanks for your patches.
>

Hmm, doing this (as opposed to the calloc one) sems to guarantee:
 [0x818d870] stream_out_transcode decoder error: decoder/filter is leaking pictures, resetting its ring buffer

Ah, i see why, video_new_buffer +firends assume i_refcount starts at
zero, whereas picture_New initializes it to one.  Is the right thing to
fix video_new_buffer +firends to work that way?  should just be
video_release_buffer that needs fixing?

..david