<div dir="auto"><div>Thanks Chen, will apply these suggestions as soon as I get past the cabac issues. As I mentioned, the patch is missing some fixes and results in CABAC errors whenever intra NxN is chosen in case that MinCbSizeY != 8.</div><div dir="auto"><br></div><div dir="auto">Any suggestions on where else to look is greatly appreciated</div><div><br></div><div data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><div><b>Qiwei Wen</b></div><div><b>Software Developer</b></div><div><b>Exablaze Pty Ltd</b></div><div><span style="color:rgb(76,76,76);font-family:AppleSDGothicNeo-Regular,"lucida grande",tahoma,verdana,arial,sans-serif,"Segoe UI Emoji","Segoe UI Symbol",NotoColorEmoji,EmojiSymbols,Symbola,Noto,"Android Emoji",AndroidEmoji,"Arial Unicode MS","Zapf Dingbats",AppleColorEmoji,"Apple Color Emoji";font-size:13px;text-align:justify;text-indent:27px">✆</span> 0430872689</div><div><a href="https://www.linkedin.com/in/qiwei-wen-9905578a/" target="_blank">LinkedIn</a></div><div><br></div><div><br><br></div></div></div></div></div></div></div></div></div></div></div><br><div class="gmail_quote gmail_quote_container"><div dir="ltr" class="gmail_attr">On Fri, Dec 19, 2025, 2:58 AM chen <<a href="mailto:chenm003@163.com">chenm003@163.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="line-height:1.7;color:#000000;font-size:14px;font-family:Arial"><div id="m_-2904158189910759774spnEditorContent"><div style="margin:0">Hi Wen,</div><div style="margin:0"><br></div><div style="margin:0">Thank for the patch.</div><div style="margin:0"><br></div><div style="margin:0">I have some comments,</div><div style="margin:0"><br></div><div style="margin:0">(1) access global variant is higher cost, how about change below?</div><div style="margin:0">cuGeom.log2CUSize == g_log2Size[m_param->minCUSize]</div><div style="margin:0">(1 << cuGeom.log2CUSize) == m_param->minCUSize</div><div style="margin:0"><br></div><div style="margin:0">(2) looks mixed another fix on array index typo, suggest separate patch</div><div style="margin:0">m_lumaIntraDir[absPartIdx]</div><div style="margin:0"><br></div><div style="margin:0">Regards,<br>Chen</div></div><div style="zoom:1"></div><div id="m_-2904158189910759774divNeteaseMailCard"></div><div style="margin:0"><br></div><pre><br>At 2025-12-18 21:26:30, "Qiwei Wen" <<a href="mailto:wenqiweiabcd@gmail.com" target="_blank" rel="noreferrer">wenqiweiabcd@gmail.com</a>> wrote:
>Dear Gurus,
>
>As you know, the HEVC spec allows NxN partition mode for intra-coded
>CUs when the CU is of size MinCbSizeY. I assume either for coding
>efficiency or historical reasons, x265 won't explore PART_NxN if the
>minimum CU size is not 8. I'm working on a project that involves using
>x265 to prototype intra-coding algorithms and it'd be ideal for my use
>case if the limitation could be lifted.
>
>I realise that there are a few places in the code base that assume
>that intra NxN implies a CU size of 8. Below is my attempt at fixing
>it, the result of which is a non-local CABAC error, where the first
>few intra NxN CU is coded correctly, but the CABAC state starts to
>diverge to the point of making the bitstream undecodeable a few CUs
>further down.
>
>I'd appreciate any pointers on if there are more places in the code
>that makes the assumption and what the possible causes could be.
>
>Thanks,
>Qiwei
>
>> diff --git a/source/encoder/analysis.cpp b/source/encoder/analysis.cpp
>> index b219d5da4..4c42faba6 100644
>> --- a/source/encoder/analysis.cpp
>> +++ b/source/encoder/analysis.cpp
>> @@ -580,7 +580,7 @@ uint64_t Analysis::compressIntraCU(const CUData& parentCTU, const CUGeom& cuGeom
>> checkIntra(md.pred[PRED_INTRA], cuGeom, SIZE_2Nx2N);
>> checkBestMode(md.pred[PRED_INTRA], depth);
>>
>> - if (cuGeom.log2CUSize == 3 && m_slice->m_sps->quadtreeTULog2MinSize < 3)
>> + if (cuGeom.log2CUSize == g_log2Size[m_param->minCUSize] && m_slice->m_sps->quadtreeTULog2MinSize < g_log2Size[m_param->minCUSize])
>> {
>> md.pred[PRED_INTRA_NxN].cu.initSubCU(parentCTU, cuGeom, qp);
>> checkIntra(md.pred[PRED_INTRA_NxN], cuGeom, SIZE_NxN);
>> diff --git a/source/encoder/entropy.cpp b/source/encoder/entropy.cpp
>> index d4bf73d53..9aecbb5e5 100644
>> --- a/source/encoder/entropy.cpp
>> +++ b/source/encoder/entropy.cpp
>> @@ -1328,7 +1328,7 @@ void Entropy::encodeTransform(const CUData& cu, uint32_t absPartIdx, uint32_t cu
>>
>> /* in each of these conditions, the subdiv flag is implied and not signaled,
>> * so we have checks to make sure the implied value matches our intentions */
>> - if (cu.isIntra(absPartIdx) && cu.m_partSize[absPartIdx] != SIZE_2Nx2N && log2CurSize == MIN_LOG2_CU_SIZE)
>> + if (cu.isIntra(absPartIdx) && cu.m_partSize[absPartIdx] != SIZE_2Nx2N && log2CurSize == g_log2Size[cu.m_encData->m_param->minCUSize])
>> {
>> X265_CHECK(subdiv, "intra NxN requires TU depth below CU depth\n");
>> }
>> @@ -1463,7 +1463,7 @@ void Entropy::encodeTransformLuma(const CUData& cu, uint32_t absPartIdx, uint32_
>>
>> /* in each of these conditions, the subdiv flag is implied and not signaled,
>> * so we have checks to make sure the implied value matches our intentions */
>> - if (cu.isIntra(absPartIdx) && cu.m_partSize[absPartIdx] != SIZE_2Nx2N && log2CurSize == MIN_LOG2_CU_SIZE)
>> + if (cu.isIntra(absPartIdx) && cu.m_partSize[absPartIdx] != SIZE_2Nx2N && log2CurSize == g_log2Size[cu.m_encData->m_param->minCUSize])
>> {
>> X265_CHECK(subdiv, "intra NxN requires TU depth below CU depth\n");
>> }
>> diff --git a/source/encoder/search.cpp b/source/encoder/search.cpp
>> index 0522f52cc..3d61caabf 100644
>> --- a/source/encoder/search.cpp
>> +++ b/source/encoder/search.cpp
>> @@ -1541,8 +1541,8 @@ sse_t Search::estIntraPredQT(Mode &intraMode, const CUGeom& cuGeom, const uint32
>> {
>> uint32_t bmode = 0;
>>
>> - if (intraMode.cu.m_lumaIntraDir[puIdx] != (uint8_t)ALL_IDX)
>> - bmode = intraMode.cu.m_lumaIntraDir[puIdx];
>> + if (intraMode.cu.m_lumaIntraDir[absPartIdx] != (uint8_t)ALL_IDX)
>> + bmode = intraMode.cu.m_lumaIntraDir[absPartIdx];
>> else
>> {
>> uint64_t candCostList[MAX_RD_INTRA_MODES];
>
>--
>Qiwei Wen
>Software Developer
>Exablaze Pty Ltd
>✆ 0430872689
>LinkedIn
>_______________________________________________
>x265-devel mailing list
><a href="mailto:x265-devel@videolan.org" target="_blank" rel="noreferrer">x265-devel@videolan.org</a>
><a href="https://mailman.videolan.org/listinfo/x265-devel" target="_blank" rel="noreferrer">https://mailman.videolan.org/listinfo/x265-devel</a>
</pre></div></blockquote></div>