[vlc-devel] Setup of filter chain picture output size

[Tobias Rapp]

Hi,

when implementing a filter chain for video downscaling inside my custom 
filter module I stumbled over the problem that the width/height of the 
output picture changes depending on whether I stop the chain after 
downscaling or pass that picture on to some other filter.

Initialization of the conversion filter looks like this inside the 
constructor method of my custom filter:

"""
static int Create( vlc_object_t *p_this )
{
     filter_t *p_filter = (filter_t *)p_this;
     filter_sys_t *p_sys;
     es_format_t scale_fmt;

...

     es_format_Init( &scale_fmt, VIDEO_ES, 
p_filter->fmt_out.video.i_chroma );
     video_format_Setup( &scale_fmt.video, p_filter->fmt_out.video.i_chroma,
         200, 100, 200, 100, 1, 1 );
     filter_chain_AppendConverter( p_sys->p_chain, &p_filter->fmt_in, 
&scale_fmt );
     es_format_Clean(&scale_fmt);

...
}
"""

And in the filter processing hook:

"""
static picture_t *Filter( filter_t *p_filter, picture_t *p_pic )
{
     filter_sys_t *p_sys = p_filter->p_sys;
     picture_t *p_outpic;

     p_outpic = filter_chain_VideoFilter( p_sys->p_chain, p_pic );

     if (1) {
         // in this case a scaled-down picture is returned
         return p_outpic;
     } else {
         // in this case p_outpic has the same size as p_pic

         // ... some processing on p_outpic ...
         picture_Release(p_outpic);
         picture_Hold(p_pic);
         return p_pic;
     }
}
"""

When I look at the debug message I can see a line like this in both 
cases, which seems to indicate that initialization of the filter chain 
works:

"""
swscale filter debug: 720x608 (720x608) chroma: I422 -> 200x100 
(200x100) chroma: I422 with scaling using Bicubic (good quality) 
Process HelloVlc.exe (16748)
"""

Is there something missing to actually get a scaled-down picture when 
just processing the data without passing it to the outside?

Regards,
Tobias