[x265] [PATCH] limitTU: fix energy calculation used in limiting TU recursion

[Ashok Kumar Mishra]

On Mon, Oct 17, 2016 at 3:14 PM, Bhavna Hariharan <
bhavna at multicorewareinc.com> wrote:

>
> On Mon, Oct 17, 2016 at 2:57 PM, Pradeep Ramachandran <
> pradeep at multicorewareinc.com> wrote:
>
>>
>> On Fri, Oct 14, 2016 at 7:20 PM, <bhavna at multicorewareinc.com> wrote:
>>
>>> # HG changeset patch
>>> # User Bhavna Hariharan <bhavna at multicorewareinc.com>
>>> # Date 1476275329 -19800
>>> #      Wed Oct 12 17:58:49 2016 +0530
>>> # Node ID 854149baceefa075c3b1af12433680ffda2e3b64
>>> # Parent  c97805dad9148ad3cddba10a67ed5596508e8f86
>>> limitTU: fix energy calculation used in limiting TU recursion
>>>
>>> This commit changes the output of limit TU
>>>
>>> diff -r c97805dad914 -r 854149baceef source/encoder/search.cpp
>>> --- a/source/encoder/search.cpp Thu Oct 13 17:53:48 2016 +0800
>>> +++ b/source/encoder/search.cpp Wed Oct 12 17:58:49 2016 +0530
>>> @@ -3420,14 +3420,15 @@
>>>          if (m_param->limitTU && bCheckSplit)
>>>          {
>>>              // Stop recursion if the TU's energy level is minimal
>>> +            uint32_t numCoeff = trSize * trSize;
>>>              if (cbfFlag[TEXT_LUMA][0] == 0)
>>>                  bCheckSplit = false;
>>> -            else if (numSig[TEXT_LUMA][0] < (cuGeom.numPartitions / 16))
>>> +            else if (numSig[TEXT_LUMA][0] < (numCoeff / 64))
>>>              {
>>>                  uint32_t energy = 0;
>>> -                for (uint32_t i = 0; i < cuGeom.numPartitions; i++)
>>> +                for (uint32_t i = 0; i < numCoeff; i++)
>>>                      energy += abs(coeffCurY[i]);
>>> -                if (energy < numSig[TEXT_LUMA][0])
>>> +                if (energy == numSig[TEXT_LUMA][0])
>>>                      bCheckSplit = false;
>>>
>>
>> Can you give an example where CheckSplit is disabled here? I am finding
>> it hard to reason conditions under which this condition is satisfied.
>>
>
> Energy will be equal to the number of significant coefficients when each
> of the non-zero coefficients is one.
>
>

I feel this condition may not be satisfied(very rare). You are calculating
energy as sum of abs values of transform coefficients and checking with a
threshold(number of coefficients).
There is a very less chance that both will be same. Either it should be
less than equal to or greater than equal to. This may be a test to
replicate a zero coefficient TU block. We
should check the subjective quality particularly for limit TU, considering
larger TUs produce more ringing artifacts.

>
>>
>>>              }
>>>          }
>>> _______________________________________________
>>> x265-devel mailing list
>>> x265-devel at videolan.org
>>> https://mailman.videolan.org/listinfo/x265-devel
>>>
>>
>>
>> _______________________________________________
>> x265-devel mailing list
>> x265-devel at videolan.org
>> https://mailman.videolan.org/listinfo/x265-devel
>
>
>
>
>
> Regards,
>
> Bhavna Hariharan
>
>
> _______________________________________________
> x265-devel mailing list
> x265-devel at videolan.org
> https://mailman.videolan.org/listinfo/x265-devel
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.videolan.org/pipermail/x265-devel/attachments/20161017/a823c9d8/attachment-0001.html>