[x265-commits] [x265] search: remove curUseRDOQ argument from encodeResAndCalcR...
Steve Borho
steve at borho.org
Mon Aug 4 01:25:24 CEST 2014
details: http://hg.videolan.org/x265/rev/dcf8948beac2
branches:
changeset: 7698:dcf8948beac2
user: Steve Borho <steve at borho.org>
date: Sun Aug 03 14:16:33 2014 -0500
description:
search: remove curUseRDOQ argument from encodeResAndCalcRdInterCU, always true
Subject: [x265] search: hard-code curUseRDOQ
details: http://hg.videolan.org/x265/rev/2ea423991b4a
branches:
changeset: 7699:2ea423991b4a
user: Steve Borho <steve at borho.org>
date: Sun Aug 03 14:17:55 2014 -0500
description:
search: hard-code curUseRDOQ
Subject: [x265] search: remove curUseRDOQ arguments from two more methods
details: http://hg.videolan.org/x265/rev/02ac2e9324b5
branches:
changeset: 7700:02ac2e9324b5
user: Steve Borho <steve at borho.org>
date: Sun Aug 03 14:23:41 2014 -0500
description:
search: remove curUseRDOQ arguments from two more methods
residualTransformQuantInter and xEstimateResidualQT were always called with
curUseRDOQ = true, and the flag was never changed
Subject: [x265] quant: remove curUseRDOQ argument from transformNxN, always true
details: http://hg.videolan.org/x265/rev/c01f8fd0985f
branches:
changeset: 7701:c01f8fd0985f
user: Steve Borho <steve at borho.org>
date: Sun Aug 03 14:30:31 2014 -0500
description:
quant: remove curUseRDOQ argument from transformNxN, always true
Subject: [x265] quant: comment nits
details: http://hg.videolan.org/x265/rev/c97672cee4c3
branches:
changeset: 7702:c97672cee4c3
user: Steve Borho <steve at borho.org>
date: Sun Aug 03 15:04:53 2014 -0500
description:
quant: comment nits
Subject: [x265] quant: disallow psy-rdoq if transformSkip was used
details: http://hg.videolan.org/x265/rev/ede3a4679d98
branches:
changeset: 7703:ede3a4679d98
user: Steve Borho <steve at borho.org>
date: Sun Aug 03 15:19:52 2014 -0500
description:
quant: disallow psy-rdoq if transformSkip was used
psy-rdoq is not possible without a transform. Or said another way, we would need
a different psy-rdoq approach for transform skip
Subject: [x265] quant: inline the quant method, only called from transformNxN
details: http://hg.videolan.org/x265/rev/f55d750331aa
branches:
changeset: 7704:f55d750331aa
user: Steve Borho <steve at borho.org>
date: Sun Aug 03 15:27:43 2014 -0500
description:
quant: inline the quant method, only called from transformNxN
Subject: [x265] quant: don't pass member variable to class method
details: http://hg.videolan.org/x265/rev/7d727cd9bc63
branches:
changeset: 7705:7d727cd9bc63
user: Steve Borho <steve at borho.org>
date: Sun Aug 03 15:36:02 2014 -0500
description:
quant: don't pass member variable to class method
Subject: [x265] quant: denoise nits
details: http://hg.videolan.org/x265/rev/42b1d7c17510
branches:
changeset: 7706:42b1d7c17510
user: Steve Borho <steve at borho.org>
date: Sun Aug 03 15:52:00 2014 -0500
description:
quant: denoise nits
Subject: [x265] quant: sign hide nits
details: http://hg.videolan.org/x265/rev/59398cdff016
branches:
changeset: 7707:59398cdff016
user: Steve Borho <steve at borho.org>
date: Sun Aug 03 16:45:20 2014 -0500
description:
quant: sign hide nits
Subject: [x265] quant: cleanup coeff group zero-check code
details: http://hg.videolan.org/x265/rev/545d1ae4375b
branches:
changeset: 7708:545d1ae4375b
user: Steve Borho <steve at borho.org>
date: Sun Aug 03 16:57:44 2014 -0500
description:
quant: cleanup coeff group zero-check code
Subject: [x265] quant: rdStats -> cgRdStats
details: http://hg.videolan.org/x265/rev/4addf282bf76
branches:
changeset: 7709:4addf282bf76
user: Steve Borho <steve at borho.org>
date: Sun Aug 03 16:58:35 2014 -0500
description:
quant: rdStats -> cgRdStats
Subject: [x265] quant: assign a signal cost of zero to zero coeff after lastNZ
details: http://hg.videolan.org/x265/rev/7f550011c40f
branches:
changeset: 7710:7f550011c40f
user: Steve Borho <steve at borho.org>
date: Sun Aug 03 18:01:59 2014 -0500
description:
quant: assign a signal cost of zero to zero coeff after lastNZ
This stack variable was uninitialized otherwised, and could have been used
during the optimization of lastNZ
Subject: [x265] entropy: cleanup calcCost, add EMMS for float safety
details: http://hg.videolan.org/x265/rev/b2ee9de29995
branches:
changeset: 7711:b2ee9de29995
user: Steve Borho <steve at borho.org>
date: Sun Aug 03 18:02:34 2014 -0500
description:
entropy: cleanup calcCost, add EMMS for float safety
Subject: [x265] quant: nit
details: http://hg.videolan.org/x265/rev/774dc8b6a535
branches:
changeset: 7712:774dc8b6a535
user: Steve Borho <steve at borho.org>
date: Sun Aug 03 18:09:12 2014 -0500
description:
quant: nit
diffstat:
source/Lib/TLibEncoder/TEncSearch.cpp | 43 +++---
source/Lib/TLibEncoder/TEncSearch.h | 6 +-
source/common/quant.cpp | 202 +++++++++++++++++----------------
source/common/quant.h | 7 +-
source/encoder/analysis.cpp | 13 +-
source/encoder/entropy.cpp | 19 ++-
6 files changed, 151 insertions(+), 139 deletions(-)
diffs (truncated from 734 to 300 lines):
diff -r 01116c890510 -r 774dc8b6a535 source/Lib/TLibEncoder/TEncSearch.cpp
--- a/source/Lib/TLibEncoder/TEncSearch.cpp Sun Aug 03 12:56:29 2014 -0500
+++ b/source/Lib/TLibEncoder/TEncSearch.cpp Sun Aug 03 18:09:12 2014 -0500
@@ -430,7 +430,7 @@ void TEncSearch::xIntraCodingLumaBlk(TCo
m_entropyCoder->estBit(m_quant.m_estBitsSbac, log2TrSize, TEXT_LUMA);
//--- transform and quantization ---
- uint32_t numSig = m_quant.transformNxN(cu, fenc, stride, residual, stride, coeff, log2TrSize, TEXT_LUMA, absPartIdx, useTransformSkip, true);
+ uint32_t numSig = m_quant.transformNxN(cu, fenc, stride, residual, stride, coeff, log2TrSize, TEXT_LUMA, absPartIdx, useTransformSkip);
//--- set coded block flag ---
cbf = numSig ? 1 : 0;
@@ -499,7 +499,7 @@ void TEncSearch::xIntraCodingChromaBlk(T
m_entropyCoder->estBit(m_quant.m_estBitsSbac, log2TrSizeC, TEXT_CHROMA);
//--- transform and quantization ---
- uint32_t numSig = m_quant.transformNxN(cu, fenc, stride, residual, stride, coeff, log2TrSizeC, ttype, absPartIdx, useTransformSkipC, true);
+ uint32_t numSig = m_quant.transformNxN(cu, fenc, stride, residual, stride, coeff, log2TrSizeC, ttype, absPartIdx, useTransformSkipC);
//--- set coded block flag ---
cbf = numSig ? 1 : 0;
@@ -859,7 +859,7 @@ void TEncSearch::residualTransformQuantI
primitives.calcresidual[sizeIdx](fenc, pred, residual, stride);
//===== transform and quantization =====
- uint32_t numSig = m_quant.transformNxN(cu, fenc, stride, residual, stride, coeff, log2TrSize, TEXT_LUMA, absPartIdx, useTransformSkip, true);
+ uint32_t numSig = m_quant.transformNxN(cu, fenc, stride, residual, stride, coeff, log2TrSize, TEXT_LUMA, absPartIdx, useTransformSkip);
//--- set coded block flag ---
cu->setCbfSubParts((numSig ? 1 : 0) << trDepth, TEXT_LUMA, absPartIdx, fullDepth);
@@ -1332,7 +1332,7 @@ void TEncSearch::residualQTIntrachroma(T
primitives.calcresidual[sizeIdxC](fenc, pred, residual, stride);
//--- transform and quantization ---
- uint32_t numSig = m_quant.transformNxN(cu, fenc, stride, residual, stride, coeff, log2TrSizeC, ttype, absPartIdxC, useTransformSkipC, true);
+ uint32_t numSig = m_quant.transformNxN(cu, fenc, stride, residual, stride, coeff, log2TrSizeC, ttype, absPartIdxC, useTransformSkipC);
//--- set coded block flag ---
cu->setCbfPartRange((((numSig > 0) ? 1 : 0) << trDepth), ttype, absPartIdxC, tuIterator.absPartIdxStep);
@@ -2341,7 +2341,7 @@ void TEncSearch::encodeResAndCalcRdSkipC
* \returns void
*/
void TEncSearch::encodeResAndCalcRdInterCU(TComDataCU* cu, TComYuv* fencYuv, TComYuv* predYuv, ShortYuv* outResiYuv,
- ShortYuv* outBestResiYuv, TComYuv* outReconYuv, bool curUseRDOQ)
+ ShortYuv* outBestResiYuv, TComYuv* outReconYuv)
{
X265_CHECK(!cu->isIntra(0), "intra CU not expected\n");
@@ -2383,7 +2383,7 @@ void TEncSearch::encodeResAndCalcRdInter
uint32_t distortion = 0;
m_entropyCoder->load(m_rdEntropyCoders[depth][CI_CURR_BEST]);
- xEstimateResidualQT(cu, 0, fencYuv, predYuv, outResiYuv, depth, cost, bits, distortion, &zeroDistortion, curUseRDOQ);
+ xEstimateResidualQT(cu, 0, fencYuv, predYuv, outResiYuv, depth, cost, bits, distortion, &zeroDistortion);
m_entropyCoder->resetBits();
m_entropyCoder->codeQtRootCbfZero(cu);
@@ -2482,7 +2482,7 @@ void TEncSearch::generateCoeffRecon(TCom
if (cu->getPredictionMode(0) == MODE_INTER)
{
- residualTransformQuantInter(cu, 0, fencYuv, resiYuv, cu->getDepth(0), true);
+ residualTransformQuantInter(cu, 0, fencYuv, resiYuv, cu->getDepth(0));
if (cu->getQtRootCbf(0))
reconYuv->addClip(predYuv, resiYuv, cu->getLog2CUSize(0));
else
@@ -2501,7 +2501,7 @@ void TEncSearch::generateCoeffRecon(TCom
}
}
-void TEncSearch::residualTransformQuantInter(TComDataCU* cu, uint32_t absPartIdx, TComYuv* fencYuv, ShortYuv* resiYuv, const uint32_t depth, bool curuseRDOQ)
+void TEncSearch::residualTransformQuantInter(TComDataCU* cu, uint32_t absPartIdx, TComYuv* fencYuv, ShortYuv* resiYuv, const uint32_t depth)
{
X265_CHECK(cu->getDepth(0) == cu->getDepth(absPartIdx), "invalid depth\n");
const uint32_t trMode = depth - cu->getDepth(0);
@@ -2553,7 +2553,7 @@ void TEncSearch::residualTransformQuantI
const uint32_t strideResiC = resiYuv->m_cwidth;
uint32_t numSigY = m_quant.transformNxN(cu, fencYuv->getLumaAddr(absPartIdx), fencYuv->getStride(), curResiY, strideResiY, coeffCurY,
- log2TrSize, TEXT_LUMA, absPartIdx, false, curuseRDOQ);
+ log2TrSize, TEXT_LUMA, absPartIdx, false);
cu->setCbfSubParts(numSigY ? setCbf : 0, TEXT_LUMA, absPartIdx, depth);
@@ -2578,9 +2578,9 @@ void TEncSearch::residualTransformQuantI
cu->setTransformSkipPartRange(0, TEXT_CHROMA_V, absPartIdxC, tuIterator.absPartIdxStep);
uint32_t numSigU = m_quant.transformNxN(cu, fencYuv->getCbAddr(absPartIdxC), fencYuv->getCStride(), curResiU, strideResiC, coeffCurU + subTUOffset,
- log2TrSizeC, TEXT_CHROMA_U, absPartIdxC, false, curuseRDOQ);
+ log2TrSizeC, TEXT_CHROMA_U, absPartIdxC, false);
uint32_t numSigV = m_quant.transformNxN(cu, fencYuv->getCrAddr(absPartIdxC), fencYuv->getCStride(), curResiV, strideResiC, coeffCurV + subTUOffset,
- log2TrSizeC, TEXT_CHROMA_V, absPartIdxC, false, curuseRDOQ);
+ log2TrSizeC, TEXT_CHROMA_V, absPartIdxC, false);
cu->setCbfPartRange(numSigU ? setCbf : 0, TEXT_CHROMA_U, absPartIdxC, tuIterator.absPartIdxStep);
cu->setCbfPartRange(numSigV ? setCbf : 0, TEXT_CHROMA_V, absPartIdxC, tuIterator.absPartIdxStep);
@@ -2611,7 +2611,7 @@ void TEncSearch::residualTransformQuantI
{
const uint32_t qPartNumSubdiv = cu->m_pic->getNumPartInCU() >> ((depth + 1) << 1);
for (uint32_t i = 0; i < 4; ++i)
- residualTransformQuantInter(cu, absPartIdx + i * qPartNumSubdiv, fencYuv, resiYuv, depth + 1, curuseRDOQ);
+ residualTransformQuantInter(cu, absPartIdx + i * qPartNumSubdiv, fencYuv, resiYuv, depth + 1);
uint32_t ycbf = 0;
uint32_t ucbf = 0;
@@ -2641,8 +2641,7 @@ void TEncSearch::xEstimateResidualQT(TCo
uint64_t & rdCost,
uint32_t & outBits,
uint32_t & outDist,
- uint32_t * outZeroDist,
- bool curuseRDOQ)
+ uint32_t * outZeroDist)
{
X265_CHECK(cu->getDepth(0) == cu->getDepth(absPartIdx), "depth not matching\n");
const uint32_t trMode = depth - cu->getDepth(0);
@@ -2714,11 +2713,11 @@ void TEncSearch::xEstimateResidualQT(TCo
cu->setTransformSkipSubParts(0, TEXT_LUMA, absPartIdx, depth);
- if (m_bEnableRDOQ && curuseRDOQ)
+ if (m_bEnableRDOQ)
m_entropyCoder->estBit(m_quant.m_estBitsSbac, log2TrSize, TEXT_LUMA);
numSigY = m_quant.transformNxN(cu, fencYuv->getLumaAddr(absPartIdx), fencYuv->getStride(), resiYuv->getLumaAddr(absPartIdx), resiYuv->m_width, coeffCurY,
- log2TrSize, TEXT_LUMA, absPartIdx, false, curuseRDOQ);
+ log2TrSize, TEXT_LUMA, absPartIdx, false);
cu->setCbfSubParts(numSigY ? setCbf : 0, TEXT_LUMA, absPartIdx, depth);
@@ -2742,13 +2741,13 @@ void TEncSearch::xEstimateResidualQT(TCo
cu->setTransformSkipPartRange(0, TEXT_CHROMA_U, absPartIdxC, tuIterator.absPartIdxStep);
cu->setTransformSkipPartRange(0, TEXT_CHROMA_V, absPartIdxC, tuIterator.absPartIdxStep);
- if (m_bEnableRDOQ && curuseRDOQ)
+ if (m_bEnableRDOQ)
m_entropyCoder->estBit(m_quant.m_estBitsSbac, log2TrSizeC, TEXT_CHROMA);
numSigU[tuIterator.section] = m_quant.transformNxN(cu, fencYuv->getCbAddr(absPartIdxC), fencYuv->getCStride(), resiYuv->getCbAddr(absPartIdxC), resiYuv->m_cwidth, coeffCurU + subTUOffset,
- log2TrSizeC, TEXT_CHROMA_U, absPartIdxC, false, curuseRDOQ);
+ log2TrSizeC, TEXT_CHROMA_U, absPartIdxC, false);
numSigV[tuIterator.section] = m_quant.transformNxN(cu, fencYuv->getCrAddr(absPartIdxC), fencYuv->getCStride(), resiYuv->getCrAddr(absPartIdxC), resiYuv->m_cwidth, coeffCurV + subTUOffset,
- log2TrSizeC, TEXT_CHROMA_V, absPartIdxC, false, curuseRDOQ);
+ log2TrSizeC, TEXT_CHROMA_V, absPartIdxC, false);
cu->setCbfPartRange(numSigU[tuIterator.section] ? setCbf : 0, TEXT_CHROMA_U, absPartIdxC, tuIterator.absPartIdxStep);
cu->setCbfPartRange(numSigV[tuIterator.section] ? setCbf : 0, TEXT_CHROMA_V, absPartIdxC, tuIterator.absPartIdxStep);
@@ -3077,7 +3076,7 @@ void TEncSearch::xEstimateResidualQT(TCo
m_entropyCoder->estBit(m_quant.m_estBitsSbac, log2TrSize, TEXT_LUMA);
uint32_t numSigTSkipY = m_quant.transformNxN(cu, fencYuv->getLumaAddr(absPartIdx), fencYuv->getStride(), resiYuv->getLumaAddr(absPartIdx), resiYuv->m_width, tsCoeffY,
- log2TrSize, TEXT_LUMA, absPartIdx, true, curuseRDOQ);
+ log2TrSize, TEXT_LUMA, absPartIdx, true);
cu->setCbfSubParts(numSigTSkipY ? setCbf : 0, TEXT_LUMA, absPartIdx, depth);
if (numSigTSkipY)
@@ -3158,9 +3157,9 @@ void TEncSearch::xEstimateResidualQT(TCo
m_entropyCoder->estBit(m_quant.m_estBitsSbac, log2TrSizeC, TEXT_CHROMA);
uint32_t numSigTSkipU = m_quant.transformNxN(cu, fencYuv->getCbAddr(absPartIdxC), fencYuv->getCStride(), resiYuv->getCbAddr(absPartIdxC), resiYuv->m_cwidth, tsCoeffU,
- log2TrSizeC, TEXT_CHROMA_U, absPartIdxC, true, curuseRDOQ);
+ log2TrSizeC, TEXT_CHROMA_U, absPartIdxC, true);
uint32_t numSigTSkipV = m_quant.transformNxN(cu, fencYuv->getCrAddr(absPartIdxC), fencYuv->getCStride(), resiYuv->getCrAddr(absPartIdxC), resiYuv->m_cwidth, tsCoeffV,
- log2TrSizeC, TEXT_CHROMA_V, absPartIdxC, true, curuseRDOQ);
+ log2TrSizeC, TEXT_CHROMA_V, absPartIdxC, true);
cu->setCbfPartRange(numSigTSkipU ? setCbf : 0, TEXT_CHROMA_U, absPartIdxC, tuIterator.absPartIdxStep);
cu->setCbfPartRange(numSigTSkipV ? setCbf : 0, TEXT_CHROMA_V, absPartIdxC, tuIterator.absPartIdxStep);
diff -r 01116c890510 -r 774dc8b6a535 source/Lib/TLibEncoder/TEncSearch.h
--- a/source/Lib/TLibEncoder/TEncSearch.h Sun Aug 03 12:56:29 2014 -0500
+++ b/source/Lib/TLibEncoder/TEncSearch.h Sun Aug 03 18:09:12 2014 -0500
@@ -150,7 +150,7 @@ public:
/// encode residual and compute rd-cost for inter mode
void encodeResAndCalcRdInterCU(TComDataCU* cu, TComYuv* fencYuv, TComYuv* predYuv, ShortYuv* resiYuv, ShortYuv* bestResiYuv,
- TComYuv* reconYuv, bool curUseRDOQ);
+ TComYuv* reconYuv);
void encodeResAndCalcRdSkipCU(TComDataCU* cu, TComYuv* fencYuv, TComYuv* predYuv, TComYuv* reconYuv);
void xRecurIntraCodingQT(TComDataCU* cu, uint32_t trDepth, uint32_t absPartIdx, TComYuv* fencYuv,
@@ -161,10 +161,10 @@ public:
void generateCoeffRecon(TComDataCU* cu, TComYuv* fencYuv, TComYuv* predYuv, ShortYuv* resiYuv, TComYuv* reconYuv);
void xEstimateResidualQT(TComDataCU* cu, uint32_t absPartIdx, TComYuv* fencYuv, TComYuv* predYuv, ShortYuv* resiYuv, uint32_t depth,
- uint64_t &rdCost, uint32_t &outBits, uint32_t &outDist, uint32_t *puiZeroDist, bool curUseRDOQ = true);
+ uint64_t &rdCost, uint32_t &outBits, uint32_t &outDist, uint32_t *puiZeroDist);
void xSetResidualQTData(TComDataCU* cu, uint32_t absPartIdx, ShortYuv* resiYuv, uint32_t depth, bool bSpatial);
- void residualTransformQuantInter(TComDataCU* cu, uint32_t absPartIdx, TComYuv* fencYuv, ShortYuv* resiYuv, uint32_t depth, bool curUseRDOQ = true);
+ void residualTransformQuantInter(TComDataCU* cu, uint32_t absPartIdx, TComYuv* fencYuv, ShortYuv* resiYuv, uint32_t depth);
// -------------------------------------------------------------------------------------------------------------------
// compute symbol bits
diff -r 01116c890510 -r 774dc8b6a535 source/common/quant.cpp
--- a/source/common/quant.cpp Sun Aug 03 12:56:29 2014 -0500
+++ b/source/common/quant.cpp Sun Aug 03 18:09:12 2014 -0500
@@ -49,9 +49,9 @@ inline int fastMin(int x, int y)
return y + ((x - y) & ((x - y) >> (sizeof(int) * CHAR_BIT - 1))); // min(x, y)
}
-inline void denoiseDct(coeff_t* dctCoef, uint32_t* resSum, uint16_t* offset, int size)
+inline void denoiseDct(coeff_t* dctCoef, uint32_t* resSum, uint16_t* offset, int numCoeff)
{
- for (int i = 0; i < size; i++)
+ for (int i = 0; i < numCoeff; i++)
{
int level = dctCoef[i];
int sign = level >> 31;
@@ -213,19 +213,20 @@ void Quant::setQPforQuant(int qpy, TextT
m_qpParam[ttype].setQpParam(qp + QP_BD_OFFSET);
}
-// To minimize the distortion only. No rate is considered.
-uint32_t Quant::signBitHidingHDQ(coeff_t* qCoef, coeff_t* coef, int32_t* deltaU, uint32_t numSig, const TUEntropyCodingParameters &codingParameters)
+/* To minimize the distortion only. No rate is considered */
+uint32_t Quant::signBitHidingHDQ(coeff_t* coeff, int32_t* deltaU, uint32_t numSig, const TUEntropyCodingParameters &codingParameters)
{
const uint32_t log2TrSizeCG = codingParameters.log2TrSizeCG;
+ const uint16_t *scan = codingParameters.scan;
bool lastCG = true;
- for (int subSet = (1 << log2TrSizeCG * 2) - 1; subSet >= 0; subSet--)
+ for (int cg = (1 << log2TrSizeCG * 2) - 1; cg >= 0; cg--)
{
- int subPos = subSet << LOG2_SCAN_SET_SIZE;
+ int cgStartPos = cg << LOG2_SCAN_SET_SIZE;
int n;
for (n = SCAN_SET_SIZE - 1; n >= 0; --n)
- if (qCoef[codingParameters.scan[n + subPos]])
+ if (coeff[scan[n + cgStartPos]])
break;
if (n < 0)
continue;
@@ -233,18 +234,18 @@ uint32_t Quant::signBitHidingHDQ(coeff_t
int lastNZPosInCG = n;
for (n = 0;; n++)
- if (qCoef[codingParameters.scan[n + subPos]])
+ if (coeff[scan[n + cgStartPos]])
break;
int firstNZPosInCG = n;
if (lastNZPosInCG - firstNZPosInCG >= SBH_THRESHOLD)
{
- uint32_t signbit = (qCoef[codingParameters.scan[subPos + firstNZPosInCG]] > 0 ? 0 : 1);
+ uint32_t signbit = coeff[scan[cgStartPos + firstNZPosInCG]] > 0 ? 0 : 1;
uint32_t absSum = 0;
for (n = firstNZPosInCG; n <= lastNZPosInCG; n++)
- absSum += qCoef[codingParameters.scan[n + subPos]];
+ absSum += coeff[scan[n + cgStartPos]];
if (signbit != (absSum & 0x1)) // compare signbit with sum_parity
{
@@ -252,8 +253,8 @@ uint32_t Quant::signBitHidingHDQ(coeff_t
for (n = (lastCG ? lastNZPosInCG : SCAN_SET_SIZE - 1); n >= 0; --n)
{
- uint32_t blkPos = codingParameters.scan[n + subPos];
- if (qCoef[blkPos])
+ uint32_t blkPos = scan[n + cgStartPos];
+ if (coeff[blkPos])
{
if (deltaU[blkPos] > 0)
{
@@ -262,7 +263,7 @@ uint32_t Quant::signBitHidingHDQ(coeff_t
}
else
{
- if (n == firstNZPosInCG && abs(qCoef[blkPos]) == 1)
+ if (n == firstNZPosInCG && abs(coeff[blkPos]) == 1)
curCost = MAX_INT;
else
{
@@ -275,7 +276,7 @@ uint32_t Quant::signBitHidingHDQ(coeff_t
{
if (n < firstNZPosInCG)
{
- uint32_t thisSignBit = coef[blkPos] >= 0 ? 0 : 1;
+ uint32_t thisSignBit = m_resiDctCoeff[blkPos] >= 0 ? 0 : 1;
if (thisSignBit != signbit)
curCost = MAX_INT;
else
@@ -299,18 +300,19 @@ uint32_t Quant::signBitHidingHDQ(coeff_t
}
}
- if (qCoef[minPos] == 32767 || qCoef[minPos] == -32768)
+ /* do not allow change to violate coeff clamp */
+ if (coeff[minPos] == 32767 || coeff[minPos] == -32768)
finalChange = -1;
- if (!qCoef[minPos])
+ if (!coeff[minPos])
numSig++;
- else if (finalChange == -1 && abs(qCoef[minPos]) == 1)
+ else if (finalChange == -1 && abs(coeff[minPos]) == 1)
numSig--;
More information about the x265-commits
mailing list