[x265] [PATCH] Fix bug in reading 32x32 scalinglists
Vignesh V Menon
vignesh at multicorewareinc.com
Fri Sep 9 10:55:50 CEST 2016
Thanks for pointing out.
Working on the fix.
Thanks
Vignesh
On Fri, Sep 9, 2016 at 1:40 PM, Mateusz Brzostek <mateusz at msystem.waw.pl>
wrote:
> For command
> x265 --vbv-maxrate 10000 --vbv-bufsize 10000 720p50_parkrun_ter.y4m w.hevc
> in function Encoder::create() in line #278
> double start = quantF / (m_scalingList.m_quantCoef[trSize][numList][QP_MAX_SPEC
> % 6][i]);
> trSize is 0, 1, 2 and 3
> numList is 1
> so with trSize == 3 && numList == 1 there is crash -- in this patch there
> is only memory allocation for [3][0] and [3][3] indexes but not for [3][1].
>
>
> W dniu 2016-09-07 o 08:55, vignesh at multicorewareinc.com pisze:
> > # HG changeset patch
> > # User vignesh
> > # Date 1472642767 -19800
> > # Wed Aug 31 16:56:07 2016 +0530
> > # Node ID e61403d77a36ad1da3d9ea50f68c12c9a6d2026a
> > # Parent e6ab763efa9509e6212b9018da7889a20676a69c
> > Fix bug in reading 32x32 scalinglists
> >
> > diff -r e6ab763efa95 -r e61403d77a36 source/common/scalinglist.cpp
> > --- a/source/common/scalinglist.cpp Wed Jun 22 12:02:06 2016 +0530
> > +++ b/source/common/scalinglist.cpp Wed Aug 31 16:56:07 2016 +0530
> > @@ -141,7 +141,7 @@
> > bool ok = true;
> > for (int sizeId = 0; sizeId < NUM_SIZES; sizeId++)
> > {
> > - for (int listId = 0; listId < NUM_LISTS; listId++)
> > + for (int listId = 0; listId < NUM_LISTS; listId += (sizeId ==
> 3) ? 3 : 1)
> > {
> > m_scalingListCoef[sizeId][listId] = X265_MALLOC(int32_t,
> X265_MIN(MAX_MATRIX_COEF_NUM, s_numCoefPerSize[sizeId]));
> > ok &= !!m_scalingListCoef[sizeId][listId];
> > @@ -160,7 +160,7 @@
> > {
> > for (int sizeId = 0; sizeId < NUM_SIZES; sizeId++)
> > {
> > - for (int listId = 0; listId < NUM_LISTS; listId++)
> > + for (int listId = 0; listId < NUM_LISTS; listId += (sizeId ==
> 3) ? 3 : 1)
> > {
> > X265_FREE(m_scalingListCoef[sizeId][listId]);
> > for (int rem = 0; rem < NUM_REM; rem++)
> > @@ -198,7 +198,7 @@
> > int defaultCounter = 0;
> >
> > for (int s = 0; s < NUM_SIZES; s++)
> > - for (int l = 0; l < NUM_LISTS; l++)
> > + for (int l = 0; l < NUM_LISTS; l += (s == 3) ? 3 : 1)
> > if (!memcmp(m_scalingListCoef[s][l],
> getScalingListDefaultAddress(s, l),
> > sizeof(int32_t) * X265_MIN(MAX_MATRIX_COEF_NUM,
> s_numCoefPerSize[s])) &&
> > ((s < BLOCK_16x16) || (m_scalingListDC[s][l] == 16)))
> > @@ -237,7 +237,7 @@
> > void ScalingList::setDefaultScalingList()
> > {
> > for (int sizeId = 0; sizeId < NUM_SIZES; sizeId++)
> > - for (int listId = 0; listId < NUM_LISTS; listId++)
> > + for (int listId = 0; listId < NUM_LISTS; listId += (sizeId ==
> 3) ? 3 : 1)
> > processDefaultMarix(sizeId, listId);
> > m_bEnabled = true;
> > m_bDataPresent = false;
> > @@ -332,7 +332,7 @@
> > int stride = X265_MIN(MAX_MATRIX_SIZE_NUM, width);
> > int count = s_numCoefPerSize[size];
> >
> > - for (int list = 0; list < NUM_LISTS; list++)
> > + for (int list = 0; list < NUM_LISTS; list += (size == 3) ? 3 :
> 1)
> > {
> > int32_t *coeff = m_scalingListCoef[size][list];
> > int32_t dc = m_scalingListDC[size][list];
> > _______________________________________________
> > x265-devel mailing list
> > x265-devel at videolan.org
> > https://mailman.videolan.org/listinfo/x265-devel
>
> _______________________________________________
> x265-devel mailing list
> x265-devel at videolan.org
> https://mailman.videolan.org/listinfo/x265-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.videolan.org/pipermail/x265-devel/attachments/20160909/e7fd2fa9/attachment-0001.html>
More information about the x265-devel
mailing list