[x265-commits] [x265] quant: returns numSig instead of absSum and lastPos
Satoshi Nakagawa
nakagawa424 at oki.com
Thu Jul 10 03:47:28 CEST 2014
details: http://hg.videolan.org/x265/rev/65ac66dc89b6
branches:
changeset: 7242:65ac66dc89b6
user: Satoshi Nakagawa <nakagawa424 at oki.com>
date: Mon Jul 07 17:00:26 2014 +0900
description:
quant: returns numSig instead of absSum and lastPos
Subject: [x265] TEncCU: save CABAC context missing after merge
details: http://hg.videolan.org/x265/rev/c269f73b94c9
branches:
changeset: 7243:c269f73b94c9
user: Min Chen <chenm003 at 163.com>
date: Wed Jul 09 11:23:44 2014 +0530
description:
TEncCU: save CABAC context missing after merge
Also removed an extra context save
Subject: [x265] TEncCU: save CABAC context missing after merge
details: http://hg.videolan.org/x265/rev/8a5b02464813
branches: stable
changeset: 7244:8a5b02464813
user: Min Chen <chenm003 at 163.com>
date: Wed Jul 09 11:23:44 2014 +0530
description:
TEncCU: save CABAC context missing after merge
Also removed an extra context save
Subject: [x265] Merge with stable
details: http://hg.videolan.org/x265/rev/a1e46d813642
branches:
changeset: 7245:a1e46d813642
user: Deepthi Nandakumar <deepthi at multicorewareinc.com>
date: Wed Jul 09 12:57:19 2014 +0530
description:
Merge with stable
Subject: [x265] TEncCU: bug fix introduced in commit 8a5b02464813
details: http://hg.videolan.org/x265/rev/d6257335c537
branches: stable
changeset: 7246:d6257335c537
user: Deepthi Nandakumar <deepthi at multicorewareinc.com>
date: Wed Jul 09 13:55:15 2014 +0530
description:
TEncCU: bug fix introduced in commit 8a5b02464813
Remove the xCheckBestMode call, add in a ccontext save.
Subject: [x265] Merge with stable
details: http://hg.videolan.org/x265/rev/644773b85329
branches:
changeset: 7247:644773b85329
user: Deepthi Nandakumar <deepthi at multicorewareinc.com>
date: Wed Jul 09 13:55:42 2014 +0530
description:
Merge with stable
Subject: [x265] port TEncEntropy and TEncSbac into internal Entropy and SBac classes
details: http://hg.videolan.org/x265/rev/53fcddea0959
branches:
changeset: 7248:53fcddea0959
user: Steve Borho <steve at borho.org>
date: Tue Jul 08 22:00:37 2014 -0500
description:
port TEncEntropy and TEncSbac into internal Entropy and SBac classes
The Entropy class will be short-lived
Subject: [x265] entropy: remove all data members except m_entropyCoder
details: http://hg.videolan.org/x265/rev/5321d02d9703
branches:
changeset: 7249:5321d02d9703
user: Steve Borho <steve at borho.org>
date: Tue Jul 08 22:44:39 2014 -0500
description:
entropy: remove all data members except m_entropyCoder
The three variables were only used to maintain state through recursive calls
to encodeTransform() and could be just as easily kept on the stack.
The class is now a pure reflector to the SBac class and can be merge into it
piece-wise.
Subject: [x265] entropy: hoist all non-trivial Entropy methods into SBac
details: http://hg.videolan.org/x265/rev/e4a21cfa8206
branches:
changeset: 7250:e4a21cfa8206
user: Steve Borho <steve at borho.org>
date: Tue Jul 08 23:07:30 2014 -0500
description:
entropy: hoist all non-trivial Entropy methods into SBac
Now Entropy only exists in entropy.h, making further cleanups possible
Subject: [x265] entropy: fix msvc warnings about signed/unsigned operations
details: http://hg.videolan.org/x265/rev/a25a3782e51c
branches:
changeset: 7251:a25a3782e51c
user: Steve Borho <steve at borho.org>
date: Tue Jul 08 23:24:25 2014 -0500
description:
entropy: fix msvc warnings about signed/unsigned operations
Subject: [x265] entropy: remove default argument to encodeIntraDirModeLuma()
details: http://hg.videolan.org/x265/rev/436bd8207469
branches:
changeset: 7252:436bd8207469
user: Steve Borho <steve at borho.org>
date: Tue Jul 08 23:10:48 2014 -0500
description:
entropy: remove default argument to encodeIntraDirModeLuma()
Subject: [x265] entropy: reorder SBac methods for more clarity
details: http://hg.videolan.org/x265/rev/01e20a16e890
branches:
changeset: 7253:01e20a16e890
user: Steve Borho <steve at borho.org>
date: Tue Jul 08 23:32:30 2014 -0500
description:
entropy: reorder SBac methods for more clarity
Subject: [x265] entropy: remove unnecessary API differences between Entropy and SBac
details: http://hg.videolan.org/x265/rev/ca2b4259c44f
branches:
changeset: 7254:ca2b4259c44f
user: Steve Borho <steve at borho.org>
date: Wed Jul 09 00:30:31 2014 -0500
description:
entropy: remove unnecessary API differences between Entropy and SBac
These would only trip us up as we try to remove Entropy entirely
Subject: [x265] remove m_entropyCoder from FrameFilter and SAO code
details: http://hg.videolan.org/x265/rev/767a10ce1a64
branches:
changeset: 7255:767a10ce1a64
user: Steve Borho <steve at borho.org>
date: Wed Jul 09 01:40:38 2014 -0500
description:
remove m_entropyCoder from FrameFilter and SAO code
Lo and Behold! the SAO code suddenly makes a tad more sense when this useless
redirection is removed
Subject: [x265] sao: remove redundant braces and other nits
details: http://hg.videolan.org/x265/rev/1429920a7420
branches:
changeset: 7256:1429920a7420
user: Steve Borho <steve at borho.org>
date: Wed Jul 09 01:41:13 2014 -0500
description:
sao: remove redundant braces and other nits
Subject: [x265] frameencoder: do not use Entropy object to code stream headers
details: http://hg.videolan.org/x265/rev/919c77a86b29
branches:
changeset: 7257:919c77a86b29
user: Steve Borho <steve at borho.org>
date: Wed Jul 09 12:18:27 2014 -0500
description:
frameencoder: do not use Entropy object to code stream headers
Subject: [x265] entropy: remove m_slice member from SBac class
details: http://hg.videolan.org/x265/rev/576bb702f286
branches:
changeset: 7258:576bb702f286
user: Steve Borho <steve at borho.org>
date: Wed Jul 09 02:21:38 2014 -0500
description:
entropy: remove m_slice member from SBac class
The slice object was only needed in resetEntropy() and determineCabacInitIdx()
and could be passed to those two methods easily.
It was also being used in two functions to gain access to the scaling list, but
this was actually a problem for us because headers can be queuried before any
slice objects are initialized so it was better to pass in the scaling lists
directly from the top Encoder. Note this fixes a bug in scaling list support -
we were previously incapable of signaling them properly in our stream headers.
Removing this member variable lets us clean up a lot of redundant setSlice()
calls and other cruft
Subject: [x265] frameencoder: do not pass member variable to its own function
details: http://hg.videolan.org/x265/rev/6589396373de
branches:
changeset: 7259:6589396373de
user: Steve Borho <steve at borho.org>
date: Wed Jul 09 12:37:43 2014 -0500
description:
frameencoder: do not pass member variable to its own function
We were still passing an array of Bitstream objects to encodeSlice() even though
the array was made a member of the class so it was available via m_outStreams.
Within the function is was using both pointers, unhelfpully.
Subject: [x265] frameencoder: remove most traces of the Encoder class
details: http://hg.videolan.org/x265/rev/02f8c0af20f2
branches:
changeset: 7260:02f8c0af20f2
user: Steve Borho <steve at borho.org>
date: Wed Jul 09 01:46:46 2014 -0500
description:
frameencoder: remove most traces of the Encoder class
Subject: [x265] remove Entropy from TEncCu, TEncSearch, and CTURow
details: http://hg.videolan.org/x265/rev/ff987b44de91
branches:
changeset: 7261:ff987b44de91
user: Steve Borho <steve at borho.org>
date: Wed Jul 09 01:18:56 2014 -0500
description:
remove Entropy from TEncCu, TEncSearch, and CTURow
Instead, give each an SBac pointer that they are supposed to use to code the
bitstream.
Subject: [x265] entropy: remove now unused Entropy class
details: http://hg.videolan.org/x265/rev/461e3e940b0c
branches:
changeset: 7262:461e3e940b0c
user: Steve Borho <steve at borho.org>
date: Wed Jul 09 14:06:44 2014 -0500
description:
entropy: remove now unused Entropy class
Subject: [x265] entropy: make m_cabac member of SBac class a non-pointer instance
details: http://hg.videolan.org/x265/rev/f60a2f9a88ad
branches:
changeset: 7263:f60a2f9a88ad
user: Steve Borho <steve at borho.org>
date: Wed Jul 09 16:15:25 2014 -0500
description:
entropy: make m_cabac member of SBac class a non-pointer instance
This forces a 1::1 correspondence between SBac and CABAC instances, which cleans
up a mess of confusing allocations and assignments.
This moves the CABAC class into entropy.h and it is no longer referenced outside
of this header and entropy.cpp, allowing us to further combine them.
Subject: [x265] framefilter: rename m_rdGoOnSbacCoder to m_sbacCoder - there is only one
details: http://hg.videolan.org/x265/rev/878e6f228fe6
branches:
changeset: 7264:878e6f228fe6
user: Steve Borho <steve at borho.org>
date: Wed Jul 09 16:29:36 2014 -0500
description:
framefilter: rename m_rdGoOnSbacCoder to m_sbacCoder - there is only one
Subject: [x265] entropy: allocate RDO coders as dual-dimensioned array
details: http://hg.videolan.org/x265/rev/c49b61c5e6bb
branches:
changeset: 7265:c49b61c5e6bb
user: Steve Borho <steve at borho.org>
date: Wed Jul 09 17:41:08 2014 -0500
description:
entropy: allocate RDO coders as dual-dimensioned array
In order to make this change even less intrusive as it was going to be, I
simultaneously changed the load/store functions to take references instead of
pointers (since NULL pointers are not supported).
This further simplified setup/tear-down code and should improve data locality
Subject: [x265] ctu: rename m_sbacCoder to m_rowEntropyCoder
details: http://hg.videolan.org/x265/rev/6a1370596ede
branches:
changeset: 7266:6a1370596ede
user: Steve Borho <steve at borho.org>
date: Wed Jul 09 18:05:08 2014 -0500
description:
ctu: rename m_sbacCoder to m_rowEntropyCoder
Subject: [x265] frameencoder: remove barely use helper function getRDGoOnSbacCoder
details: http://hg.videolan.org/x265/rev/ddf49ea5073c
branches:
changeset: 7267:ddf49ea5073c
user: Steve Borho <steve at borho.org>
date: Wed Jul 09 18:06:07 2014 -0500
description:
frameencoder: remove barely use helper function getRDGoOnSbacCoder
Subject: [x265] frameencoder: remove rarely used getRowCoder() and getBufferSBac()
details: http://hg.videolan.org/x265/rev/22ac140e0da8
branches:
changeset: 7268:22ac140e0da8
user: Steve Borho <steve at borho.org>
date: Wed Jul 09 18:12:40 2014 -0500
description:
frameencoder: remove rarely used getRowCoder() and getBufferSBac()
Subject: [x265] ctu: inline setThreadLocalData() for better clarity
details: http://hg.videolan.org/x265/rev/6c3a4a5498f1
branches:
changeset: 7269:6c3a4a5498f1
user: Steve Borho <steve at borho.org>
date: Wed Jul 09 18:23:51 2014 -0500
description:
ctu: inline setThreadLocalData() for better clarity
Subject: [x265] cu: pass bit counting flag as argument to encodeCU
details: http://hg.videolan.org/x265/rev/4c98884a7729
branches:
changeset: 7270:4c98884a7729
user: Steve Borho <steve at borho.org>
date: Wed Jul 09 18:28:33 2014 -0500
description:
cu: pass bit counting flag as argument to encodeCU
Subject: [x265] cu: assign m_rdGoOnSbacCoder without helper methods
details: http://hg.videolan.org/x265/rev/a44b2c12fe04
branches:
changeset: 7271:a44b2c12fe04
user: Steve Borho <steve at borho.org>
date: Wed Jul 09 18:30:10 2014 -0500
description:
cu: assign m_rdGoOnSbacCoder without helper methods
Subject: [x265] cu: nit
details: http://hg.videolan.org/x265/rev/c78b7cca188a
branches:
changeset: 7272:c78b7cca188a
user: Steve Borho <steve at borho.org>
date: Wed Jul 09 18:30:19 2014 -0500
description:
cu: nit
Subject: [x265] cu: remove more set methods
details: http://hg.videolan.org/x265/rev/cd93637ce951
branches:
changeset: 7273:cd93637ce951
user: Steve Borho <steve at borho.org>
date: Wed Jul 09 18:46:26 2014 -0500
description:
cu: remove more set methods
Subject: [x265] cu: remove get/set methods for its own member variable
details: http://hg.videolan.org/x265/rev/b1d077221c25
branches:
changeset: 7274:b1d077221c25
user: Steve Borho <steve at borho.org>
date: Wed Jul 09 18:51:24 2014 -0500
description:
cu: remove get/set methods for its own member variable
Subject: [x265] entropy: nits
details: http://hg.videolan.org/x265/rev/a86cd034e254
branches:
changeset: 7275:a86cd034e254
user: Steve Borho <steve at borho.org>
date: Wed Jul 09 20:08:59 2014 -0500
description:
entropy: nits
Subject: [x265] entropy: remove temporary helper function
details: http://hg.videolan.org/x265/rev/b7934e58ea40
branches:
changeset: 7276:b7934e58ea40
user: Steve Borho <steve at borho.org>
date: Wed Jul 09 20:12:22 2014 -0500
description:
entropy: remove temporary helper function
Subject: [x265] csv: add slice-type stats to encode summary, avoid dup copies of summary string
details: http://hg.videolan.org/x265/rev/5c7da1e88682
branches:
changeset: 7277:5c7da1e88682
user: Steve Borho <steve at borho.org>
date: Thu Jul 03 14:40:06 2014 -0500
description:
csv: add slice-type stats to encode summary, avoid dup copies of summary string
Subject: [x265] Added tag 1.2 for changeset d6257335c537
details: http://hg.videolan.org/x265/rev/7ea0ba364367
branches: stable
changeset: 7278:7ea0ba364367
user: Steve Borho <steve at borho.org>
date: Wed Jul 09 20:19:21 2014 -0500
description:
Added tag 1.2 for changeset d6257335c537
Subject: [x265] Merge with stable
details: http://hg.videolan.org/x265/rev/56c4719ae735
branches:
changeset: 7279:56c4719ae735
user: Steve Borho <steve at borho.org>
date: Wed Jul 09 20:19:35 2014 -0500
description:
Merge with stable
Subject: [x265] search: remove braces around single line expressions, improve code flow
details: http://hg.videolan.org/x265/rev/30e20a7b6fdd
branches:
changeset: 7280:30e20a7b6fdd
user: Steve Borho <steve at borho.org>
date: Wed Jul 09 20:33:52 2014 -0500
description:
search: remove braces around single line expressions, improve code flow
diffstat:
.hgtags | 1 +
source/Lib/TLibCommon/CommonDef.h | 2 -
source/Lib/TLibCommon/ContextTables.h | 1 -
source/Lib/TLibCommon/TComSlice.cpp | 1 -
source/Lib/TLibCommon/TComSlice.h | 16 +-
source/Lib/TLibCommon/TComTrQuant.cpp | 192 +-
source/Lib/TLibCommon/TComTrQuant.h | 17 +-
source/Lib/TLibEncoder/TEncBinCoderCABAC.cpp | 363 --
source/Lib/TLibEncoder/TEncBinCoderCABAC.h | 97 -
source/Lib/TLibEncoder/TEncCu.cpp | 128 +-
source/Lib/TLibEncoder/TEncCu.h | 37 +-
source/Lib/TLibEncoder/TEncEntropy.cpp | 468 --
source/Lib/TLibEncoder/TEncEntropy.h | 158 -
source/Lib/TLibEncoder/TEncSampleAdaptiveOffset.cpp | 190 +-
source/Lib/TLibEncoder/TEncSampleAdaptiveOffset.h | 35 +-
source/Lib/TLibEncoder/TEncSbac.cpp | 2512 ---------------
source/Lib/TLibEncoder/TEncSbac.h | 156 -
source/Lib/TLibEncoder/TEncSearch.cpp | 788 +---
source/Lib/TLibEncoder/TEncSearch.h | 27 +-
source/common/dct.cpp | 11 +-
source/common/primitives.cpp | 1 -
source/common/primitives.h | 2 +-
source/common/x86/pixel-util.h | 2 +-
source/common/x86/pixel-util8.asm | 104 +-
source/encoder/CMakeLists.txt | 7 +-
source/encoder/compress.cpp | 39 +-
source/encoder/cturow.cpp | 96 +-
source/encoder/cturow.h | 44 +-
source/encoder/encoder.cpp | 59 +-
source/encoder/encoder.h | 2 +
source/encoder/entropy.cpp | 3003 +++++++++++++++++++
source/encoder/entropy.h | 217 +
source/encoder/frameencoder.cpp | 176 +-
source/encoder/frameencoder.h | 13 +-
source/encoder/framefilter.cpp | 25 +-
source/encoder/framefilter.h | 9 +-
source/test/mbdstharness.cpp | 11 +-
37 files changed, 3985 insertions(+), 5025 deletions(-)
diffs (truncated from 11806 to 300 lines):
diff -r 6325261d393d -r 30e20a7b6fdd .hgtags
--- a/.hgtags Mon Jul 07 17:31:20 2014 -0500
+++ b/.hgtags Wed Jul 09 20:33:52 2014 -0500
@@ -13,3 +13,4 @@ 527d03c56d6860dc979ddea1196f7e94d13d3e82
82bbd2bf3b49ba086be0f0922f91fe0084896351 0.9
cea97c4d79456842e00ade6be6fd5ec34610e5f8 1.0
ae9609aeebdc3271114168ece003679e9b1dca1b 1.1
+d6257335c5370ee54317a0426a12c1f0724b18b9 1.2
diff -r 6325261d393d -r 30e20a7b6fdd source/Lib/TLibCommon/CommonDef.h
--- a/source/Lib/TLibCommon/CommonDef.h Mon Jul 07 17:31:20 2014 -0500
+++ b/source/Lib/TLibCommon/CommonDef.h Wed Jul 09 20:33:52 2014 -0500
@@ -118,8 +118,6 @@
#define LOG2_MAX_COLUMN_WIDTH 13
#define LOG2_MAX_ROW_HEIGHT 13
-#define REG_DCT 65535
-
#define CABAC_INIT_PRESENT_FLAG 1
#define MAX_GOP 64 ///< max. value of hierarchical GOP size
diff -r 6325261d393d -r 30e20a7b6fdd source/Lib/TLibCommon/ContextTables.h
--- a/source/Lib/TLibCommon/ContextTables.h Mon Jul 07 17:31:20 2014 -0500
+++ b/source/Lib/TLibCommon/ContextTables.h Wed Jul 09 20:33:52 2014 -0500
@@ -138,7 +138,6 @@ typedef struct ContextModel
extern const uint32_t g_entropyBits[128];
extern const uint8_t g_nextState[128][2];
-uint8_t sbacInit(int qp, int initValue); ///< initialize state with initial probability
#define sbacGetMps(S) ((S) & 1)
#define sbacGetState(S) ((S) >> 1)
diff -r 6325261d393d -r 30e20a7b6fdd source/Lib/TLibCommon/TComSlice.cpp
--- a/source/Lib/TLibCommon/TComSlice.cpp Mon Jul 07 17:31:20 2014 -0500
+++ b/source/Lib/TLibCommon/TComSlice.cpp Wed Jul 09 20:33:52 2014 -0500
@@ -476,7 +476,6 @@ TComSPS::TComSPS()
, m_qpBDOffsetC(0)
, m_bitsForPOC(8)
, m_numLongTermRefPicSPS(0)
- , m_maxTrSize(32)
, m_bUseSAO(false)
, m_bTemporalIdNestingFlag(false)
, m_scalingListEnabledFlag(false)
diff -r 6325261d393d -r 30e20a7b6fdd source/Lib/TLibCommon/TComSlice.h
--- a/source/Lib/TLibCommon/TComSlice.h Mon Jul 07 17:31:20 2014 -0500
+++ b/source/Lib/TLibCommon/TComSlice.h Wed Jul 09 20:33:52 2014 -0500
@@ -170,14 +170,14 @@ public:
void setUseTransformSkip(bool b) { m_useTransformSkip = b; }
- int32_t* getScalingListAddress(uint32_t sizeId, uint32_t listId) { return m_scalingListCoef[sizeId][listId]; } //!< get matrix coefficient
+ int32_t* getScalingListAddress(uint32_t sizeId, uint32_t listId) { return m_scalingListCoef[sizeId][listId]; } //!< get matrix coefficient
- bool checkPredMode(uint32_t sizeId, uint32_t listId);
+ bool checkPredMode(uint32_t sizeId, int listId);
void setRefMatrixId(uint32_t sizeId, uint32_t listId, uint32_t u) { m_refMatrixId[sizeId][listId] = u; } //!< set reference matrix ID
- uint32_t getRefMatrixId(uint32_t sizeId, uint32_t listId) { return m_refMatrixId[sizeId][listId]; } //!< get reference matrix ID
+ uint32_t getRefMatrixId(uint32_t sizeId, uint32_t listId) { return m_refMatrixId[sizeId][listId]; } //!< get reference matrix ID
- int32_t* getScalingListDefaultAddress(uint32_t sizeId, uint32_t listId); //!< get default matrix coefficient
+ int32_t* getScalingListDefaultAddress(uint32_t sizeId, uint32_t listId); //!< get default matrix coefficient
void processDefaultMarix(uint32_t sizeId, uint32_t listId);
void setScalingListDC(uint32_t sizeId, uint32_t listId, uint32_t u) { m_scalingListDC[sizeId][listId] = u; } //!< set DC value
@@ -825,9 +825,6 @@ private:
uint32_t m_ltRefPicPocLsbSps[33];
bool m_usedByCurrPicLtSPSFlag[33];
- // Max physical transform size
- uint32_t m_maxTrSize;
-
int m_iAMPAcc[MAX_CU_DEPTH];
bool m_bUseSAO;
@@ -954,11 +951,6 @@ public:
void setTMVPFlagsPresent(bool b) { m_TMVPFlagsPresent = b; }
- // physical transform
- void setMaxTrSize(uint32_t u) { m_maxTrSize = u; }
-
- uint32_t getMaxTrSize() const { return m_maxTrSize; }
-
// AMP accuracy
int getAMPAcc(uint32_t depth) const { return m_iAMPAcc[depth]; }
diff -r 6325261d393d -r 30e20a7b6fdd source/Lib/TLibCommon/TComTrQuant.cpp
--- a/source/Lib/TLibCommon/TComTrQuant.cpp Mon Jul 07 17:31:20 2014 -0500
+++ b/source/Lib/TLibCommon/TComTrQuant.cpp Wed Jul 09 20:33:52 2014 -0500
@@ -143,7 +143,7 @@ void TComTrQuant::setQPforQuant(int qpy,
}
// To minimize the distortion only. No rate is considered.
-void TComTrQuant::signBitHidingHDQ(coeff_t* qCoef, coeff_t* coef, int32_t* deltaU, const TUEntropyCodingParameters &codingParameters)
+uint32_t TComTrQuant::signBitHidingHDQ(coeff_t* qCoef, coeff_t* coef, int32_t* deltaU, uint32_t numSig, const TUEntropyCodingParameters &codingParameters)
{
const uint32_t log2TrSizeCG = codingParameters.log2TrSizeCG;
@@ -249,6 +249,11 @@ void TComTrQuant::signBitHidingHDQ(coeff
finalChange = -1;
}
+ if (qCoef[minPos] == 0)
+ numSig++;
+ else if (finalChange == -1 && abs(qCoef[minPos]) == 1)
+ numSig--;
+
if (coef[minPos] >= 0)
{
qCoef[minPos] += finalChange;
@@ -261,12 +266,13 @@ void TComTrQuant::signBitHidingHDQ(coeff
}
lastCG = 0;
} // TU loop
+
+ return numSig;
}
-uint32_t TComTrQuant::xQuant(TComDataCU* cu, int32_t* coef, coeff_t* qCoef, int trSize,
- TextType ttype, uint32_t absPartIdx, int32_t *lastPos)
+uint32_t TComTrQuant::xQuant(TComDataCU* cu, int32_t* coef, coeff_t* qCoef, uint32_t log2TrSize,
+ TextType ttype, uint32_t absPartIdx)
{
- const uint32_t log2TrSize = g_convertToBit[trSize] + 2;
TUEntropyCodingParameters codingParameters;
getTUEntropyCodingParameters(cu, codingParameters, absPartIdx, log2TrSize, ttype);
int deltaU[32 * 32];
@@ -281,13 +287,13 @@ uint32_t TComTrQuant::xQuant(TComDataCU*
int add = (cu->getSlice()->getSliceType() == I_SLICE ? 171 : 85) << (qbits - 9);
int numCoeff = 1 << log2TrSize * 2;
- uint32_t acSum = primitives.quant(coef, quantCoeff, deltaU, qCoef, qbits, add, numCoeff, lastPos);
+ uint32_t numSig = primitives.quant(coef, quantCoeff, deltaU, qCoef, qbits, add, numCoeff);
- if (acSum >= 2 && cu->getSlice()->getPPS()->getSignHideFlag())
+ if (numSig >= 2 && cu->getSlice()->getPPS()->getSignHideFlag())
{
- signBitHidingHDQ(qCoef, coef, deltaU, codingParameters);
+ return signBitHidingHDQ(qCoef, coef, deltaU, numSig, codingParameters);
}
- return acSum;
+ return numSig;
}
void TComTrQuant::init(bool useRDOQ)
@@ -299,73 +305,65 @@ uint32_t TComTrQuant::transformNxN(TComD
int16_t* residual,
uint32_t stride,
coeff_t* coeff,
- uint32_t trSize,
+ uint32_t log2TrSize,
TextType ttype,
uint32_t absPartIdx,
- int32_t* lastPos,
bool useTransformSkip,
bool curUseRDOQ)
{
if (cu->getCUTransquantBypass(absPartIdx))
{
- uint32_t absSum = 0;
- for (uint32_t k = 0; k < trSize; k++)
+ uint32_t numSig = 0;
+ int trSize = 1 << log2TrSize;
+ for (int k = 0; k < trSize; k++)
{
- for (uint32_t j = 0; j < trSize; j++)
+ for (int j = 0; j < trSize; j++)
{
coeff[k * trSize + j] = ((int16_t)residual[k * stride + j]);
- absSum += abs(residual[k * stride + j]);
+ numSig += (residual[k * stride + j] != 0);
}
}
- return absSum;
+ return numSig;
}
- uint32_t mode; //luma intra pred
- if (ttype == TEXT_LUMA && cu->getPredictionMode(absPartIdx) == MODE_INTRA)
+ X265_CHECK((cu->getSlice()->getSPS()->getQuadtreeTULog2MaxSize() >= log2TrSize), "transform size too large\n");
+ if (!useTransformSkip)
{
- mode = cu->getLumaIntraDir(absPartIdx);
+ // TODO: this may need larger data types for X265_DEPTH > 8
+ const uint32_t sizeIdx = log2TrSize - 2;
+ int useDST = (sizeIdx == 0 && ttype == TEXT_LUMA && cu->getPredictionMode(absPartIdx) == MODE_INTRA);
+ int index = DCT_4x4 + sizeIdx - useDST;
+ primitives.dct[index](residual, m_tmpCoeff, stride);
+ if (m_nr->bNoiseReduction)
+ {
+ if (index > 0)
+ {
+ denoiseDct(m_tmpCoeff, m_nr->residualSum[sizeIdx], m_nr->offset[sizeIdx], (16 << sizeIdx * 2));
+ m_nr->count[sizeIdx]++;
+ }
+ }
}
else
{
- mode = REG_DCT;
- }
-
- X265_CHECK((cu->getSlice()->getSPS()->getMaxTrSize() >= trSize), "transform size too large\n");
- if (useTransformSkip)
- {
- xTransformSkip(residual, stride, m_tmpCoeff, trSize);
- }
- else
- {
- // TODO: this may need larger data types for X265_DEPTH > 8
- const uint32_t log2BlockSize = g_convertToBit[trSize];
- primitives.dct[DCT_4x4 + log2BlockSize - ((trSize == 4) && (mode != REG_DCT))](residual, m_tmpCoeff, stride);
- if (m_nr->bNoiseReduction)
- {
- int index = (DCT_4x4 + log2BlockSize - ((trSize == 4) && (mode != REG_DCT)));
- if (index > 0 && index < 5)
- {
- denoiseDct(m_tmpCoeff, m_nr->residualSum[index - 1], m_nr->offset[index - 1], (16 << (index - 1) * 2));
- m_nr->count[index - 1]++;
- }
- }
+ xTransformSkip(residual, stride, m_tmpCoeff, log2TrSize);
}
if (m_useRDOQ && curUseRDOQ)
{
- return xRateDistOptQuant(cu, m_tmpCoeff, coeff, trSize, ttype, absPartIdx, lastPos);
+ return xRateDistOptQuant(cu, m_tmpCoeff, coeff, log2TrSize, ttype, absPartIdx);
}
- return xQuant(cu, m_tmpCoeff, coeff, trSize, ttype, absPartIdx, lastPos);
+ return xQuant(cu, m_tmpCoeff, coeff, log2TrSize, ttype, absPartIdx);
}
-void TComTrQuant::invtransformNxN(bool transQuantBypass, uint32_t mode, int16_t* residual, uint32_t stride, coeff_t* coeff, uint32_t trSize, int scalingListType, bool useTransformSkip, int lastPos)
+void TComTrQuant::invtransformNxN(bool transQuantBypass, int16_t* residual, uint32_t stride, coeff_t* coeff, uint32_t log2TrSize, TextType ttype, bool bIntra, bool useTransformSkip, uint32_t numSig)
{
if (transQuantBypass)
{
- for (uint32_t k = 0; k < trSize; k++)
+ int trSize = 1 << log2TrSize;
+ for (int k = 0; k < trSize; k++)
{
- for (uint32_t j = 0; j < trSize; j++)
+ for (int j = 0; j < trSize; j++)
{
residual[k * stride + j] = (int16_t)(coeff[k * trSize + j]);
}
@@ -377,37 +375,34 @@ void TComTrQuant::invtransformNxN(bool t
// Values need to pass as input parameter in dequant
int per = m_qpParam.m_per;
int rem = m_qpParam.m_rem;
- bool useScalingList = getUseScalingList();
- const uint32_t log2TrSize = g_convertToBit[trSize] + 2;
int transformShift = MAX_TR_DYNAMIC_RANGE - X265_DEPTH - log2TrSize;
int shift = QUANT_IQUANT_SHIFT - QUANT_SHIFT - transformShift;
- int32_t *dequantCoef = getDequantCoeff(scalingListType, m_qpParam.m_rem, log2TrSize - 2);
+ int numCoeff = 1 << log2TrSize * 2;
- if (!useScalingList)
+ if (!getUseScalingList())
{
static const int invQuantScales[6] = { 40, 45, 51, 57, 64, 72 };
int scale = invQuantScales[rem] << per;
- primitives.dequant_normal(coeff, m_tmpCoeff, trSize * trSize, scale, shift);
+ primitives.dequant_normal(coeff, m_tmpCoeff, numCoeff, scale, shift);
}
else
{
// CHECK_ME: the code is not verify since this is DEAD path
- primitives.dequant_scaling(coeff, dequantCoef, m_tmpCoeff, trSize * trSize, per, shift);
+ int scalingListType = (!bIntra ? 3 : 0) + ttype;
+ X265_CHECK(scalingListType < 6, "scalingListType invalid %d\n", scalingListType);
+ int32_t *dequantCoef = getDequantCoeff(scalingListType, m_qpParam.m_rem, log2TrSize - 2);
+ primitives.dequant_scaling(coeff, dequantCoef, m_tmpCoeff, numCoeff, per, shift);
}
- if (useTransformSkip == true)
+ if (!useTransformSkip)
{
- xITransformSkip(m_tmpCoeff, residual, stride, trSize);
- }
- else
- {
- // CHECK_ME: we can't here when no any coeff
- X265_CHECK(lastPos >= 0, "lastPos negative\n");
+ const uint32_t sizeIdx = log2TrSize - 2;
+ int useDST = (sizeIdx == 0 && ttype == TEXT_LUMA && bIntra);
- const uint32_t log2BlockSize = log2TrSize - 2;
+ X265_CHECK(numSig == primitives.count_nonzero(coeff, 1 << log2TrSize * 2), "numSig differ\n");
// DC only
- if (lastPos == 0 && !((trSize == 4) && (mode != REG_DCT)))
+ if (numSig == 1 && coeff[0] != 0 && !useDST)
{
const int shift_1st = 7;
const int add_1st = 1 << (shift_1st - 1);
@@ -415,13 +410,17 @@ void TComTrQuant::invtransformNxN(bool t
const int add_2nd = 1 << (shift_2nd - 1);
More information about the x265-commits
mailing list