[x265] fix lossless
Steve Borho
steve at borho.org
Mon Aug 25 18:20:38 CEST 2014
On 08/25, Satoshi Nakagawa wrote:
> Ø Negative shift or bts cause unexpected behavior.
>
> Negative shift is used for round value calculation.
>
> int round = 1 << (shift - 1);
Ah, we need this operation to be a copy, there shouldn't be any rounding
> At 2014-08-25 10:06:23,"Steve Borho" <steve at borho.org> wrote:
> >On 08/23, Satoshi Nakagawa wrote:
> >> # HG changeset patch
> >> # User Satoshi Nakagawa <nakagawa424 at oki.com>
> >> # Date 1408803114 -32400
> >> # Sat Aug 23 23:11:54 2014 +0900
> >> # Node ID 218b9ddfe148c55afccc2c5a073858692e09f5c6
> >> # Parent 6e6756f94b27c3ef30f6159f1880112a7ff978e3
> >> fix lossless
> >
> >do you have a repro case that this fixes? I tried Main and Main10
> >encodes with --lossless --hash 1 and they were both fine prior to this
> >patch.
> >
> >> diff -r 6e6756f94b27 -r 218b9ddfe148 source/common/quant.cpp
> >> --- a/source/common/quant.cpp Fri Aug 22 15:53:34 2014 -0500
> >> +++ b/source/common/quant.cpp Sat Aug 23 23:11:54 2014 +0900
> >> @@ -399,7 +399,7 @@
> >> {
> >> if (transQuantBypass)
> >> {
> >> - primitives.cvt32to16_shr(residual, coeff, stride, 0, 1 <<
> log2TrSize);
> >> + primitives.cvt32to16_shl[log2TrSize - 2](residual, coeff,
> stride, 0);
> >> return;
> >> }
> >>
> >> @@ -430,7 +430,7 @@
> >> #if X265_DEPTH <= 10
> >> primitives.cvt32to16_shr(residual, m_resiDctCoeff, stride,
> shift, trSize);
> >> #else
> >> - if (shift >= 0)
> >> + if (shift > 0)
> >> primitives.cvt32to16_shr(residual, m_resiDctCoeff, stride,
> shift, trSize);
> >> else
> >> primitives.cvt32to16_shl[log2TrSize - 2](residual,
> m_resiDctCoeff, stride, -shift);
> >> _______________________________________________
> >> x265-devel mailing list
> >> x265-devel at videolan.org
> >> https://mailman.videolan.org/listinfo/x265-devel
> >
> >--
> >Steve Borho
> >_______________________________________________
> >x265-devel mailing list
> >x265-devel at videolan.org
> >https://mailman.videolan.org/listinfo/x265-devel
> _______________________________________________
> x265-devel mailing list
> x265-devel at videolan.org
> https://mailman.videolan.org/listinfo/x265-devel
--
Steve Borho
More information about the x265-devel
mailing list