[x265] [PATCH] quant: remove extra upshift by 4 in s_invQuantScales
deepthi at multicorewareinc.com
deepthi at multicorewareinc.com
Wed Aug 13 10:58:23 CEST 2014
# HG changeset patch
# User Deepthi Nandakumar <deepthi at multicorewareinc.com>
# Date 1407919768 -19800
# Wed Aug 13 14:19:28 2014 +0530
# Node ID 41df534b9605806520c8ff23348e15aff793f8dc
# Parent d43e9a6a7cced5b60284c25bd987c55c522c1212
quant: remove extra upshift by 4 in s_invQuantScales
Adding 4 to unQuantPer was a bug, since unQuantScale was already upshifted by 4.
diff -r d43e9a6a7cce -r 41df534b9605 source/common/quant.cpp
--- a/source/common/quant.cpp Wed Aug 13 00:19:31 2014 -0500
+++ b/source/common/quant.cpp Wed Aug 13 14:19:28 2014 +0530
@@ -516,18 +516,11 @@
int32_t *unquantScale = m_scalingList->m_dequantCoef[log2TrSize - 2][scalingListType][rem];
int unquantShift = QUANT_IQUANT_SHIFT - QUANT_SHIFT - transformShift;
int unquantRound, unquantPer;
- unquantShift += 4;
+ unquantPer = per;
if (unquantShift > per)
- {
unquantRound = 1 << (unquantShift - per - 1);
- unquantPer = per;
- }
else
- {
- unquantPer = per + 4;
- unquantShift += 4;
unquantRound = 0;
- }
#define SIGCOST(bits) ((lambda2 * (bits)) >> 8)
#define RDCOST(d, bits) ((((int64_t)d * d) << scaleBits) + ((lambda2 * (bits)) >> 8))
diff -r d43e9a6a7cce -r 41df534b9605 source/common/scalinglist.cpp
--- a/source/common/scalinglist.cpp Wed Aug 13 00:19:31 2014 -0500
+++ b/source/common/scalinglist.cpp Wed Aug 13 14:19:28 2014 +0530
@@ -349,7 +349,7 @@
for (int i = 0; i < count; i++)
{
quantCoeff[i] = s_quantScales[rem];
- dequantCoeff[i] = s_invQuantScales[rem] << 4;
+ dequantCoeff[i] = s_invQuantScales[rem];
}
}
}
More information about the x265-devel
mailing list