[x265] [PATCH] quant[BUG]: correct scaling list type

[Deepthi Nandakumar]

Yeah - beats me as well, but this is clearly a bug. The output has no
change, I can only surmise it's some lucky permutation of those constant
arrays in ScalingList.

On Wed, Mar 18, 2015 at 9:39 PM, Steve Borho <steve at borho.org> wrote:

> On 03/18, deepthi at multicorewareinc.com wrote:
> > # HG changeset patch
> > # User Deepthi Nandakumar <deepthi at multicorewareinc.com>
> > # Date 1426664785 -19800
> > #      Wed Mar 18 13:16:25 2015 +0530
> > # Node ID 3a913041edebc6198c78800b63337d3c5afe8f1e
> > # Parent  14aec6dc4d3b6c7b295fe1ead34c0f9f89a08cc4
> > quant[BUG]: correct scaling list type
>
> how did this not cause serious hash mismatches before?
>
> > diff -r 14aec6dc4d3b -r 3a913041edeb source/common/quant.cpp
> > --- a/source/common/quant.cpp Wed Mar 18 12:41:16 2015 +0530
> > +++ b/source/common/quant.cpp Wed Mar 18 13:16:25 2015 +0530
> > @@ -422,7 +422,7 @@
> >      {
> >          int deltaU[32 * 32];
> >
> > -        int scalingListType = ttype + (isLuma ? 3 : 0);
> > +        int scalingListType = (cu.isIntra(absPartIdx) ? 0 : 3) + ttype;
> >          int rem = m_qpParam[ttype].rem;
> >          int per = m_qpParam[ttype].per;
> >          const int32_t* quantCoeff =
> m_scalingList->m_quantCoef[log2TrSize - 2][scalingListType][rem];
> > _______________________________________________
> > x265-devel mailing list
> > x265-devel at videolan.org
> > https://mailman.videolan.org/listinfo/x265-devel
>
> --
> Steve Borho
> _______________________________________________
> x265-devel mailing list
> x265-devel at videolan.org
> https://mailman.videolan.org/listinfo/x265-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.videolan.org/pipermail/x265-devel/attachments/20150318/c0521214/attachment.html>