[x265-commits] [x265] motion: nit
Steve Borho
steve at borho.org
Wed Dec 11 22:14:11 CET 2013
details: http://hg.videolan.org/x265/rev/3bf6be9d766b
branches:
changeset: 5689:3bf6be9d766b
user: Steve Borho <steve at borho.org>
date: Tue Dec 10 11:05:28 2013 -0600
description:
motion: nit
Subject: [x265] cmake: Add LOG_CU_STATISTICS macro to enable/disable CU mode decision statistics.
details: http://hg.videolan.org/x265/rev/a1e7aac486a7
branches:
changeset: 5690:a1e7aac486a7
user: Deepthi Nandakumar <deepthi at multicorewareinc.com>
date: Wed Dec 11 14:25:52 2013 +0530
description:
cmake: Add LOG_CU_STATISTICS macro to enable/disable CU mode decision statistics.
Subject: [x265] log: fix memory overread issue
details: http://hg.videolan.org/x265/rev/40895c94da21
branches:
changeset: 5691:40895c94da21
user: Kavitha Sampath <kavitha at multicorewareinc.com>
date: Wed Dec 11 14:52:38 2013 +0530
description:
log: fix memory overread issue
Subject: [x265] TEncSearch: remove unused functions
details: http://hg.videolan.org/x265/rev/b938ac5cef81
branches:
changeset: 5692:b938ac5cef81
user: Deepthi Devaki <deepthidevaki at multicorewareinc.com>
date: Tue Dec 10 17:38:52 2013 +0530
description:
TEncSearch: remove unused functions
Subject: [x265] add function for Inter residual tranform and quantization without RD
details: http://hg.videolan.org/x265/rev/9b7f7303cdb9
branches:
changeset: 5693:9b7f7303cdb9
user: Deepthi Devaki <deepthidevaki at multicorewareinc.com>
date: Tue Dec 10 17:40:00 2013 +0530
description:
add function for Inter residual tranform and quantization without RD
Subject: [x265] add functions for Intra residual transform and quantization without rd
details: http://hg.videolan.org/x265/rev/14ddc3ed548a
branches:
changeset: 5694:14ddc3ed548a
user: Deepthi Devaki <deepthidevaki at multicorewareinc.com>
date: Tue Dec 10 17:40:42 2013 +0530
description:
add functions for Intra residual transform and quantization without rd
Subject: [x265] rdlevel: implement rdlevel 1
details: http://hg.videolan.org/x265/rev/298470fa2d19
branches:
changeset: 5695:298470fa2d19
user: Deepthi Devaki <deepthidevaki at multicorewareinc.com>
date: Tue Dec 10 17:42:00 2013 +0530
description:
rdlevel: implement rdlevel 1
At each depth, dct, quant, iquant, idct is done on best mode. No RD done during TU selection. Recon is generated for intra prediction in neighbouring quadtree is not broken
Subject: [x265] tcomtrquant: when useRDOQ is disabled, it recomputes qpScaled, the qpScaled for trQuant is already set
details: http://hg.videolan.org/x265/rev/c5f84164ef36
branches:
changeset: 5696:c5f84164ef36
user: Gopu Govindaswamy <gopu at multicorewareinc.com>
date: Wed Dec 11 16:31:28 2013 +0530
description:
tcomtrquant: when useRDOQ is disabled, it recomputes qpScaled, the qpScaled for trQuant is already set
Subject: [x265] assembly code for intra_pred_planar_16x16 for 10 and 12-bit
details: http://hg.videolan.org/x265/rev/2b618395a586
branches:
changeset: 5697:2b618395a586
user: Dnyaneshwar G <dnyaneshwar at multicorewareinc.com>
date: Tue Dec 10 21:20:05 2013 +0550
description:
assembly code for intra_pred_planar_16x16 for 10 and 12-bit
Subject: [x265] 16bpp: assembly code for intra_pred_planar_32x32
details: http://hg.videolan.org/x265/rev/771e657a11fa
branches:
changeset: 5698:771e657a11fa
user: Dnyaneshwar G <dnyaneshwar at multicorewareinc.com>
date: Tue Dec 10 22:46:59 2013 +0550
description:
16bpp: assembly code for intra_pred_planar_32x32
Subject: [x265] asm: fix hash miss match due to intra_pred_ang4_17
details: http://hg.videolan.org/x265/rev/8f634eac2ca0
branches:
changeset: 5699:8f634eac2ca0
user: Yuvaraj Venkatesh <yuvaraj at multicorewareinc.com>
date: Wed Dec 11 14:12:04 2013 +0550
description:
asm: fix hash miss match due to intra_pred_ang4_17
Subject: [x265] asm: fix miss EMMS in pixel_ssd_ss_4xN
details: http://hg.videolan.org/x265/rev/3e6b5f65f47c
branches:
changeset: 5700:3e6b5f65f47c
user: Min Chen <chenm003 at 163.com>
date: Wed Dec 11 14:30:43 2013 +0800
description:
asm: fix miss EMMS in pixel_ssd_ss_4xN
Subject: [x265] asm: fix hash mismatch in intra_pred_planar_32x32 16bpp
details: http://hg.videolan.org/x265/rev/0d2771085aec
branches:
changeset: 5701:0d2771085aec
user: Dnyaneshwar G <dnyaneshwar at multicorewareinc.com>
date: Wed Dec 11 14:50:36 2013 +0550
description:
asm: fix hash mismatch in intra_pred_planar_32x32 16bpp
Subject: [x265] asm: remove reduce operators in intra_pred_planar32
details: http://hg.videolan.org/x265/rev/5d21f5c91495
branches:
changeset: 5702:5d21f5c91495
user: Min Chen <chenm003 at 163.com>
date: Wed Dec 11 15:11:52 2013 +0800
description:
asm: remove reduce operators in intra_pred_planar32
Subject: [x265] asm: 10bpp test bench code for calcrecon
details: http://hg.videolan.org/x265/rev/99f3527f9b7b
branches:
changeset: 5703:99f3527f9b7b
user: Murugan Vairavel <murugan at multicorewareinc.com>
date: Wed Dec 11 15:16:46 2013 +0550
description:
asm: 10bpp test bench code for calcrecon
Subject: [x265] testbench: fix invalid memory compare for 16bpp primitives
details: http://hg.videolan.org/x265/rev/da972cf802fb
branches:
changeset: 5704:da972cf802fb
user: Dnyaneshwar G <dnyaneshwar at multicorewareinc.com>
date: Wed Dec 11 15:19:20 2013 +0550
description:
testbench: fix invalid memory compare for 16bpp primitives
Subject: [x265] asm: 10bpp code for calcrecon_4x4 and 8x8
details: http://hg.videolan.org/x265/rev/59c09449211d
branches:
changeset: 5705:59c09449211d
user: Murugan Vairavel <murugan at multicorewareinc.com>
date: Wed Dec 11 15:59:58 2013 +0550
description:
asm: 10bpp code for calcrecon_4x4 and 8x8
Subject: [x265] asm: 10bpp code for calcrecon_16x16 and 32x32
details: http://hg.videolan.org/x265/rev/350b0757d1a0
branches:
changeset: 5706:350b0757d1a0
user: Murugan Vairavel <murugan at multicorewareinc.com>
date: Wed Dec 11 16:49:06 2013 +0550
description:
asm: 10bpp code for calcrecon_16x16 and 32x32
Subject: [x265] asm : Hook up chroma_hps with encoder.
details: http://hg.videolan.org/x265/rev/93a92dafb667
branches:
changeset: 5707:93a92dafb667
user: Nabajit Deka <nabajit at multicorewareinc.com>
date: Wed Dec 11 19:11:32 2013 +0550
description:
asm : Hook up chroma_hps with encoder.
Subject: [x265] ssd: backport EMMS fix for SSD_4xN
details: http://hg.videolan.org/x265/rev/d2385a789c8a
branches: stable
changeset: 5708:d2385a789c8a
user: Steve Borho <steve at borho.org>
date: Wed Dec 11 14:48:28 2013 -0600
description:
ssd: backport EMMS fix for SSD_4xN
Subject: [x265] merge with stable
details: http://hg.videolan.org/x265/rev/25f412ecaba2
branches:
changeset: 5709:25f412ecaba2
user: Steve Borho <steve at borho.org>
date: Wed Dec 11 15:12:02 2013 -0600
description:
merge with stable
diffstat:
source/CMakeLists.txt | 7 +
source/Lib/TLibCommon/TComPrediction.cpp | 16 +-
source/Lib/TLibCommon/TComTrQuant.cpp | 38 +-
source/Lib/TLibEncoder/TEncCu.cpp | 21 +-
source/Lib/TLibEncoder/TEncCu.h | 2 +
source/Lib/TLibEncoder/TEncSearch.cpp | 661 ++++++++++++++++++++++++++----
source/Lib/TLibEncoder/TEncSearch.h | 21 +-
source/common/lowres.cpp | 6 +
source/common/x86/asm-primitives.cpp | 6 +
source/common/x86/intrapred16.asm | 270 ++++++++++++-
source/common/x86/pixel-util.h | 2 +
source/common/x86/pixel-util8.asm | 365 +++++++++++++++++-
source/common/x86/ssd-a.asm | 4 +-
source/encoder/compress.cpp | 82 ++-
source/encoder/encoder.cpp | 3 +-
source/encoder/frameencoder.cpp | 3 +-
source/encoder/motion.cpp | 2 +-
source/test/intrapredharness.cpp | 8 +-
source/test/pixelharness.cpp | 30 +-
19 files changed, 1343 insertions(+), 204 deletions(-)
diffs (truncated from 2034 to 300 lines):
diff -r dcef9f3bca1e -r 25f412ecaba2 source/CMakeLists.txt
--- a/source/CMakeLists.txt Tue Dec 10 11:04:41 2013 -0600
+++ b/source/CMakeLists.txt Wed Dec 11 15:12:02 2013 -0600
@@ -101,6 +101,13 @@ else(HIGH_BIT_DEPTH)
add_definitions(-DHIGH_BIT_DEPTH=0)
endif(HIGH_BIT_DEPTH)
+option(LOG_CU_STATISTICS "Log Mode Decision Statistics at the CU Level" OFF)
+if(LOG_CU_STATISTICS)
+ add_definitions(-DLOG_CU_STATISTICS=1)
+else(LOG_CU_STATISTICS)
+ add_definitions(-DLOG_CU_STATISTICS=0)
+endif(LOG_CU_STATISTICS)
+
find_package(Yasm)
if(YASM_FOUND)
if (YASM_VERSION_STRING VERSION_LESS "1.2.0")
diff -r dcef9f3bca1e -r 25f412ecaba2 source/Lib/TLibCommon/TComPrediction.cpp
--- a/source/Lib/TLibCommon/TComPrediction.cpp Tue Dec 10 11:04:41 2013 -0600
+++ b/source/Lib/TLibCommon/TComPrediction.cpp Wed Dec 11 15:12:02 2013 -0600
@@ -548,10 +548,10 @@ void TComPrediction::xPredInterChromaBlk
int filterSize = NTAPS_CHROMA;
int halfFilterSize = (filterSize >> 1);
- primitives.ipfilter_ps[FILTER_H_P_S_4](refCb - (halfFilterSize - 1) * refStride, refStride, m_immedVals, extStride, cxWidth, cxHeight + filterSize - 1, g_chromaFilter[xFrac]);
+ primitives.chroma[csp].filter_hps[partEnum](refCb, refStride, m_immedVals, extStride, xFrac, 1);
primitives.chroma_vsp(m_immedVals + (halfFilterSize - 1) * extStride, extStride, dstCb, dstStride, cxWidth, cxHeight, yFrac);
- primitives.ipfilter_ps[FILTER_H_P_S_4](refCr - (halfFilterSize - 1) * refStride, refStride, m_immedVals, extStride, cxWidth, cxHeight + filterSize - 1, g_chromaFilter[xFrac]);
+ primitives.chroma[csp].filter_hps[partEnum](refCr, refStride, m_immedVals, extStride, xFrac, 1);
primitives.chroma_vsp(m_immedVals + (halfFilterSize - 1) * extStride, extStride, dstCr, dstStride, cxWidth, cxHeight, yFrac);
}
}
@@ -572,6 +572,10 @@ void TComPrediction::xPredInterChromaBlk
int xFrac = mv->x & 0x7;
int yFrac = mv->y & 0x7;
+
+ int partEnum = partitionFromSizes(width, height);
+ int csp = X265_CSP_I420;
+
uint32_t cxWidth = width >> 1;
uint32_t cxHeight = height >> 1;
@@ -585,8 +589,8 @@ void TComPrediction::xPredInterChromaBlk
}
else if (yFrac == 0)
{
- primitives.ipfilter_ps[FILTER_H_P_S_4](refCb, refStride, dstCb, dstStride, cxWidth, cxHeight, g_chromaFilter[xFrac]);
- primitives.ipfilter_ps[FILTER_H_P_S_4](refCr, refStride, dstCr, dstStride, cxWidth, cxHeight, g_chromaFilter[xFrac]);
+ primitives.chroma[csp].filter_hps[partEnum](refCb, refStride, dstCb, dstStride, xFrac, 0);
+ primitives.chroma[csp].filter_hps[partEnum](refCr, refStride, dstCr, dstStride, xFrac, 0);
}
else if (xFrac == 0)
{
@@ -598,9 +602,9 @@ void TComPrediction::xPredInterChromaBlk
int extStride = cxWidth;
int filterSize = NTAPS_CHROMA;
int halfFilterSize = (filterSize >> 1);
- primitives.ipfilter_ps[FILTER_H_P_S_4](refCb - (halfFilterSize - 1) * refStride, refStride, m_immedVals, extStride, cxWidth, cxHeight + filterSize - 1, g_chromaFilter[xFrac]);
+ primitives.chroma[csp].filter_hps[partEnum](refCb, refStride, m_immedVals, extStride, xFrac, 1);
primitives.ipfilter_ss[FILTER_V_S_S_4](m_immedVals + (halfFilterSize - 1) * extStride, extStride, dstCb, dstStride, cxWidth, cxHeight, yFrac);
- primitives.ipfilter_ps[FILTER_H_P_S_4](refCr - (halfFilterSize - 1) * refStride, refStride, m_immedVals, extStride, cxWidth, cxHeight + filterSize - 1, g_chromaFilter[xFrac]);
+ primitives.chroma[csp].filter_hps[partEnum](refCr, refStride, m_immedVals, extStride, xFrac, 1);
primitives.ipfilter_ss[FILTER_V_S_S_4](m_immedVals + (halfFilterSize - 1) * extStride, extStride, dstCr, dstStride, cxWidth, cxHeight, yFrac);
}
}
diff -r dcef9f3bca1e -r 25f412ecaba2 source/Lib/TLibCommon/TComTrQuant.cpp
--- a/source/Lib/TLibCommon/TComTrQuant.cpp Tue Dec 10 11:04:41 2013 -0600
+++ b/source/Lib/TLibCommon/TComTrQuant.cpp Wed Dec 11 15:12:02 2013 -0600
@@ -273,42 +273,6 @@ uint32_t TComTrQuant::xQuant(TComDataCU*
int deltaU[32 * 32];
- QpParam cQpBase;
- int qpbase = cu->getSlice()->getSliceQpBase();
-
- int qpScaled;
- int qpBDOffset = (ttype == TEXT_LUMA) ? cu->getSlice()->getSPS()->getQpBDOffsetY() : cu->getSlice()->getSPS()->getQpBDOffsetC();
-
- if (ttype == TEXT_LUMA)
- {
- qpScaled = qpbase + qpBDOffset;
- }
- else
- {
- int chromaQPOffset;
- if (ttype == TEXT_CHROMA_U)
- {
- chromaQPOffset = cu->getSlice()->getPPS()->getChromaCbQpOffset() + cu->getSlice()->getSliceQpDeltaCb();
- }
- else
- {
- chromaQPOffset = cu->getSlice()->getPPS()->getChromaCrQpOffset() + cu->getSlice()->getSliceQpDeltaCr();
- }
- qpbase = qpbase + chromaQPOffset;
-
- qpScaled = Clip3(-qpBDOffset, 57, qpbase);
-
- if (qpScaled < 0)
- {
- qpScaled = qpScaled + qpBDOffset;
- }
- else
- {
- qpScaled = g_chromaScale[qpScaled] + qpBDOffset;
- }
- }
- cQpBase.setQpParam(qpScaled);
-
uint32_t log2TrSize = g_convertToBit[width] + 2;
int scalingListType = (cu->isIntra(absPartIdx) ? 0 : 3) + g_eTTable[(int)ttype];
assert(scalingListType < 6);
@@ -317,7 +281,7 @@ uint32_t TComTrQuant::xQuant(TComDataCU*
int transformShift = MAX_TR_DYNAMIC_RANGE - X265_DEPTH - log2TrSize; // Represents scaling through forward transform
- int qbits = QUANT_SHIFT + cQpBase.m_per + transformShift;
+ int qbits = QUANT_SHIFT + m_qpParam.m_per + transformShift;
add = (cu->getSlice()->getSliceType() == I_SLICE ? 171 : 85) << (qbits - 9);
int numCoeff = width * height;
diff -r dcef9f3bca1e -r 25f412ecaba2 source/Lib/TLibEncoder/TEncCu.cpp
--- a/source/Lib/TLibEncoder/TEncCu.cpp Tue Dec 10 11:04:41 2013 -0600
+++ b/source/Lib/TLibEncoder/TEncCu.cpp Wed Dec 11 15:12:02 2013 -0600
@@ -356,16 +356,20 @@ void TEncCu::compressCU(TComDataCU* cu)
m_temporalSAD = 0;
// analysis of CU
+#if LOG_CU_STATISTICS
+ int numPartition = cu->getTotalNumPart();
+#endif
if (m_bestCU[0]->getSlice()->getSliceType() == I_SLICE)
{
xCompressIntraCU(m_bestCU[0], m_tempCU[0], 0);
+#if LOG_CU_STATISTICS
int i = 0, part;
- part = cu->getDepth(i);
do
{
m_log->totalCu++;
- int next = cu->getTotalNumPart() >> (part * 2);
+ part = cu->getDepth(i);
+ int next = numPartition >> (part * 2);
if (part == g_maxCUDepth - 1 && cu->getPartitionSize(i) != SIZE_2Nx2N)
{
m_log->cntIntraNxN++;
@@ -379,9 +383,9 @@ void TEncCu::compressCU(TComDataCU* cu)
m_log->cuIntraDistribution[part][cu->getLumaIntraDir()[i]]++;
}
i += next;
- part = cu->getDepth(i);
}
- while (part < g_maxCUDepth);
+ while (i < numPartition);
+#endif
}
else
{
@@ -395,12 +399,13 @@ void TEncCu::compressCU(TComDataCU* cu)
}
else
xCompressCU(m_bestCU[0], m_tempCU[0], 0);
+#if LOG_CU_STATISTICS
int i = 0, part;
- part = cu->getDepth(i);
do
{
+ part = cu->getDepth(i);
m_log->cntTotalCu[part]++;
- int next = cu->getTotalNumPart() >> (part * 2);
+ int next = numPartition >> (part * 2);
if (cu->isSkipped(i))
{
m_log->cntSkipCu[part]++;
@@ -433,9 +438,9 @@ void TEncCu::compressCU(TComDataCU* cu)
}
}
i = i + next;
- part = cu->getDepth(i);
}
- while (part < g_maxCUDepth);
+ while (i < numPartition);
+#endif
}
}
diff -r dcef9f3bca1e -r 25f412ecaba2 source/Lib/TLibEncoder/TEncCu.h
--- a/source/Lib/TLibEncoder/TEncCu.h Tue Dec 10 11:04:41 2013 -0600
+++ b/source/Lib/TLibEncoder/TEncCu.h Wed Dec 11 15:12:02 2013 -0600
@@ -132,8 +132,10 @@ private:
public:
+#if LOG_CU_STATISTICS
StatisticLog m_sliceTypeLog[3];
StatisticLog* m_log;
+#endif
TEncCu();
void init(Encoder* top);
diff -r dcef9f3bca1e -r 25f412ecaba2 source/Lib/TLibEncoder/TEncSearch.cpp
--- a/source/Lib/TLibEncoder/TEncSearch.cpp Tue Dec 10 11:04:41 2013 -0600
+++ b/source/Lib/TLibEncoder/TEncSearch.cpp Wed Dec 11 15:12:02 2013 -0600
@@ -983,6 +983,142 @@ void TEncSearch::xRecurIntraCodingQT(TCo
rdCost += singleCost;
}
+void TEncSearch::residualTransformQuantIntra(TComDataCU* cu,
+ uint32_t trDepth,
+ uint32_t absPartIdx,
+ bool bLumaOnly,
+ TComYuv* fencYuv,
+ TComYuv* predYuv,
+ TShortYUV* resiYuv,
+ TComYuv* reconYuv)
+{
+ uint32_t fullDepth = cu->getDepth(0) + trDepth;
+ uint32_t trSizeLog2 = g_convertToBit[cu->getSlice()->getSPS()->getMaxCUWidth() >> fullDepth] + 2;
+ bool bCheckFull = (trSizeLog2 <= cu->getSlice()->getSPS()->getQuadtreeTULog2MaxSize());
+ bool bCheckSplit = (trSizeLog2 > cu->getQuadtreeTULog2MinSizeInCU(absPartIdx));
+
+ int maxTuSize = cu->getSlice()->getSPS()->getQuadtreeTULog2MaxSize();
+ int isIntraSlice = (cu->getSlice()->getSliceType() == I_SLICE);
+
+ // don't check split if TU size is less or equal to max TU size
+ bool noSplitIntraMaxTuSize = bCheckFull;
+
+ if (m_cfg->param.rdPenalty && !isIntraSlice)
+ {
+ // in addition don't check split if TU size is less or equal to 16x16 TU size for non-intra slice
+ noSplitIntraMaxTuSize = (trSizeLog2 <= X265_MIN(maxTuSize, 4));
+
+ // if maximum RD-penalty don't check TU size 32x32
+ if (m_cfg->param.rdPenalty == 2)
+ {
+ bCheckFull = (trSizeLog2 <= X265_MIN(maxTuSize, 4));
+ }
+ }
+ if (bCheckFull)
+ {
+ cu->setTransformSkipSubParts(0, TEXT_LUMA, absPartIdx, fullDepth);
+
+ //----- code luma block with given intra prediction mode and store Cbf-----
+ uint32_t lumaPredMode = cu->getLumaIntraDir(absPartIdx);
+ uint32_t fullDepth = cu->getDepth(0) + trDepth;
+ uint32_t width = cu->getWidth(0) >> trDepth;
+ uint32_t height = cu->getHeight(0) >> trDepth;
+ uint32_t stride = fencYuv->getStride();
+ Pel* fenc = fencYuv->getLumaAddr(absPartIdx);
+ Pel* pred = predYuv->getLumaAddr(absPartIdx);
+ int16_t* residual = resiYuv->getLumaAddr(absPartIdx);
+ Pel* recon = reconYuv->getLumaAddr(absPartIdx);
+
+ uint32_t trSizeLog2 = g_convertToBit[cu->getSlice()->getSPS()->getMaxCUWidth() >> fullDepth] + 2;
+ uint32_t qtLayer = cu->getSlice()->getSPS()->getQuadtreeTULog2MaxSize() - trSizeLog2;
+ uint32_t numCoeffPerInc = cu->getSlice()->getSPS()->getMaxCUWidth() * cu->getSlice()->getSPS()->getMaxCUHeight() >> (cu->getSlice()->getSPS()->getMaxCUDepth() << 1);
+ TCoeff* coeff = m_qtTempCoeffY[qtLayer] + numCoeffPerInc * absPartIdx;
+
+ int16_t* reconQt = m_qtTempTComYuv[qtLayer].getLumaAddr(absPartIdx);
+ assert(m_qtTempTComYuv[qtLayer].m_width == MAX_CU_SIZE);
+
+ uint32_t zorder = cu->getZorderIdxInCU() + absPartIdx;
+ Pel* reconIPred = cu->getPic()->getPicYuvRec()->getLumaAddr(cu->getAddr(), zorder);
+ uint32_t reconIPredStride = cu->getPic()->getPicYuvRec()->getStride();
+
+ bool useTransformSkip = cu->getTransformSkip(absPartIdx, TEXT_LUMA);
+
+ //===== init availability pattern =====
+
+ cu->getPattern()->initPattern(cu, trDepth, absPartIdx);
+ cu->getPattern()->initAdiPattern(cu, absPartIdx, trDepth, m_predBuf, m_predBufStride, m_predBufHeight, refAbove, refLeft, refAboveFlt, refLeftFlt);
+ //===== get prediction signal =====
+ predIntraLumaAng(lumaPredMode, pred, stride, width);
+
+ //===== get residual signal =====
+ assert(!((uint32_t)(size_t)fenc & (width - 1)));
+ assert(!((uint32_t)(size_t)pred & (width - 1)));
+ assert(!((uint32_t)(size_t)residual & (width - 1)));
+ primitives.calcresidual[(int)g_convertToBit[width]](fenc, pred, residual, stride);
+
+ //===== transform and quantization =====
+ uint32_t absSum = 0;
+ int lastPos = -1;
+ cu->setTrIdxSubParts(trDepth, absPartIdx, fullDepth);
+
+ m_trQuant->setQPforQuant(cu->getQP(0), TEXT_LUMA, cu->getSlice()->getSPS()->getQpBDOffsetY(), 0);
+ m_trQuant->selectLambda(TEXT_LUMA);
+ absSum = m_trQuant->transformNxN(cu, residual, stride, coeff, width, height, TEXT_LUMA, absPartIdx, &lastPos, useTransformSkip);
+
+ //--- set coded block flag ---
+ cu->setCbfSubParts((absSum ? 1 : 0) << trDepth, TEXT_LUMA, absPartIdx, fullDepth);
+
+ //--- inverse transform ---
+ int size = g_convertToBit[width];
+ if (absSum)
+ {
+ int scalingListType = 0 + g_eTTable[(int)TEXT_LUMA];
+ assert(scalingListType < 6);
More information about the x265-commits
mailing list