[x265] [PATCH] limitTU: fix energy calculation used in limiting TU recursion

[Bhavna Hariharan]

On Mon, Oct 17, 2016 at 2:57 PM, Pradeep Ramachandran <
pradeep at multicorewareinc.com> wrote:

>
> On Fri, Oct 14, 2016 at 7:20 PM, <bhavna at multicorewareinc.com> wrote:
>
>> # HG changeset patch
>> # User Bhavna Hariharan <bhavna at multicorewareinc.com>
>> # Date 1476275329 -19800
>> #      Wed Oct 12 17:58:49 2016 +0530
>> # Node ID 854149baceefa075c3b1af12433680ffda2e3b64
>> # Parent  c97805dad9148ad3cddba10a67ed5596508e8f86
>> limitTU: fix energy calculation used in limiting TU recursion
>>
>> This commit changes the output of limit TU
>>
>> diff -r c97805dad914 -r 854149baceef source/encoder/search.cpp
>> --- a/source/encoder/search.cpp Thu Oct 13 17:53:48 2016 +0800
>> +++ b/source/encoder/search.cpp Wed Oct 12 17:58:49 2016 +0530
>> @@ -3420,14 +3420,15 @@
>>          if (m_param->limitTU && bCheckSplit)
>>          {
>>              // Stop recursion if the TU's energy level is minimal
>> +            uint32_t numCoeff = trSize * trSize;
>>              if (cbfFlag[TEXT_LUMA][0] == 0)
>>                  bCheckSplit = false;
>> -            else if (numSig[TEXT_LUMA][0] < (cuGeom.numPartitions / 16))
>> +            else if (numSig[TEXT_LUMA][0] < (numCoeff / 64))
>>              {
>>                  uint32_t energy = 0;
>> -                for (uint32_t i = 0; i < cuGeom.numPartitions; i++)
>> +                for (uint32_t i = 0; i < numCoeff; i++)
>>                      energy += abs(coeffCurY[i]);
>> -                if (energy < numSig[TEXT_LUMA][0])
>> +                if (energy == numSig[TEXT_LUMA][0])
>>                      bCheckSplit = false;
>>
>
> Can you give an example where CheckSplit is disabled here? I am finding it
> hard to reason conditions under which this condition is satisfied.
>

Energy will be equal to the number of significant coefficients when each of
the non-zero coefficients is one.


>
>
>>              }
>>          }
>> _______________________________________________
>> x265-devel mailing list
>> x265-devel at videolan.org
>> https://mailman.videolan.org/listinfo/x265-devel
>>
>
>
> _______________________________________________
> x265-devel mailing list
> x265-devel at videolan.org
> https://mailman.videolan.org/listinfo/x265-devel





Regards,

Bhavna Hariharan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.videolan.org/pipermail/x265-devel/attachments/20161017/74592b96/attachment.html>