[x264-devel] Integer overflow in x264_weights_analyse
Xuezhi Yan
xuezhiy at google.com
Tue Apr 23 01:18:22 CEST 2019
Yes, confirm that this fix works. Thanks for making this fix.
On Mon, Apr 22, 2019 at 3:21 PM BugMaster <BugMaster at narod.ru> wrote:
> On Thu, 18 Apr 2019 16:43:05 -0700, Xuezhi Yan wrote:
> > Hi, while running x264 under Clang's UndefinedBehaviorSanitizer,
> > I'm hitting integer overflow in the following line:
>
>
>
> >
> https://code.videolan.org/videolan/x264/blob/master/encoder/slicetype.c#L408
>
>
>
> > where "ref_mean[plane]" is 0.000003 because the reference frame's
> > "i_pixel_sum[plane]" is 0 ("ref_mean[plane]" is not zero because
> > zero_bias is added). With the following clamping using float I am able
> to fix the problem locally:
>
>
> > 408,409c408,409 encoder/slicetype.c
> > < cur_scale = (1 << mindenom) * (fenc_mean[plane] - cur_offset) /
> ref_mean[plane] + 0.5f;
> > < cur_scale = x264_clip3( cur_scale, 0, 127 );
> > ---
> >> float temp = (1 << mindenom) * (fenc_mean[plane] - cur_offset) /
> ref_mean[plane] + 0.5f;
> >> cur_scale = temp > 127.0f ? 127 : (temp < 0.0f ? 0 : x264_clip3(
> temp, 0, 127 ));
>
>
> > If this looks good, can someone help me merge it? Otherwise is there a
> more appropriate fix?
>
> Hi.
>
> Thanks for bug report. I have attached patch with more correct/clean fix.
> Check that it works for you._______________________________________________
> x264-devel mailing list
> x264-devel at videolan.org
> https://mailman.videolan.org/listinfo/x264-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.videolan.org/pipermail/x264-devel/attachments/20190422/b3a36313/attachment.html>
More information about the x264-devel
mailing list