[x264-devel] [PATCH] Fix index out-of-bound

[Vitaly Buka]

I would like to re-ask for the review of this trivial patch again.

On Wed, 15 Apr 2020 at 22:01, Vitaly Buka <vitalybuka at google.com> wrote:

> From: Vitaly Buka <vitalybuka at gmail.com>
>
> rcc->last_non_b_pict_type can be -1 in cases when last_non_b_q even
> not used.
> ---
>  encoder/ratecontrol.c | 3 +--
>  1 file changed, 1 insertion(+), 2 deletions(-)
>
> diff --git a/encoder/ratecontrol.c b/encoder/ratecontrol.c
> index 1598b508..06359442 100644
> --- a/encoder/ratecontrol.c
> +++ b/encoder/ratecontrol.c
> @@ -2031,7 +2031,6 @@ static double get_diff_limited_q(x264_t *h,
> ratecontrol_entry_t *rce, double q,
>
>      // force I/B quants as a function of P quants
>      const double last_p_q    = rcc->last_qscale_for[SLICE_TYPE_P];
> -    const double last_non_b_q=
> rcc->last_qscale_for[rcc->last_non_b_pict_type];
>      if( pict_type == SLICE_TYPE_I )
>      {
>          double iq = q;
> @@ -2050,7 +2049,7 @@ static double get_diff_limited_q(x264_t *h,
> ratecontrol_entry_t *rce, double q,
>      else if( pict_type == SLICE_TYPE_B )
>      {
>          if( h->param.rc.f_pb_factor > 0 )
> -            q = last_non_b_q;
> +            q = rcc->last_qscale_for[rcc->last_non_b_pict_type];
>          if( !rce->kept_as_ref )
>              q *= fabs( h->param.rc.f_pb_factor );
>      }
> --
> 2.26.0.110.g2183baf09c-goog
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.videolan.org/pipermail/x264-devel/attachments/20200505/889ffbbf/attachment.html>