[x265] [PATCH] limitTU: fix energy calculation used in limiting TU recursion

[Ashok Kumar Mishra]

Logically checking the condition  if (energy < numSig[TEXT_LUMA][0]) is
correct, but not  if (energy == numSig[TEXT_LUMA][0]).
I have sent two pdfs today and check how they have achieved a lot more than
9% what we achieved with almost applying similar concept.

On Tue, Oct 18, 2016 at 5:01 PM, Bhavna Hariharan <
bhavna at multicorewareinc.com> wrote:

>
> On Mon, Oct 17, 2016 at 8:30 PM, Ashok Kumar Mishra <
> ashok at multicorewareinc.com> wrote:
>
>>
>>
>> On Mon, Oct 17, 2016 at 3:14 PM, Bhavna Hariharan <
>> bhavna at multicorewareinc.com> wrote:
>>
>>>
>>> On Mon, Oct 17, 2016 at 2:57 PM, Pradeep Ramachandran <
>>> pradeep at multicorewareinc.com> wrote:
>>>
>>>>
>>>> On Fri, Oct 14, 2016 at 7:20 PM, <bhavna at multicorewareinc.com> wrote:
>>>>
>>>>> # HG changeset patch
>>>>> # User Bhavna Hariharan <bhavna at multicorewareinc.com>
>>>>> # Date 1476275329 -19800
>>>>> #      Wed Oct 12 17:58:49 2016 +0530
>>>>> # Node ID 854149baceefa075c3b1af12433680ffda2e3b64
>>>>> # Parent  c97805dad9148ad3cddba10a67ed5596508e8f86
>>>>> limitTU: fix energy calculation used in limiting TU recursion
>>>>>
>>>>> This commit changes the output of limit TU
>>>>>
>>>>> diff -r c97805dad914 -r 854149baceef source/encoder/search.cpp
>>>>> --- a/source/encoder/search.cpp Thu Oct 13 17:53:48 2016 +0800
>>>>> +++ b/source/encoder/search.cpp Wed Oct 12 17:58:49 2016 +0530
>>>>> @@ -3420,14 +3420,15 @@
>>>>>          if (m_param->limitTU && bCheckSplit)
>>>>>          {
>>>>>              // Stop recursion if the TU's energy level is minimal
>>>>> +            uint32_t numCoeff = trSize * trSize;
>>>>>              if (cbfFlag[TEXT_LUMA][0] == 0)
>>>>>                  bCheckSplit = false;
>>>>> -            else if (numSig[TEXT_LUMA][0] < (cuGeom.numPartitions /
>>>>> 16))
>>>>> +            else if (numSig[TEXT_LUMA][0] < (numCoeff / 64))
>>>>>              {
>>>>>                  uint32_t energy = 0;
>>>>> -                for (uint32_t i = 0; i < cuGeom.numPartitions; i++)
>>>>> +                for (uint32_t i = 0; i < numCoeff; i++)
>>>>>                      energy += abs(coeffCurY[i]);
>>>>> -                if (energy < numSig[TEXT_LUMA][0])
>>>>> +                if (energy == numSig[TEXT_LUMA][0])
>>>>>                      bCheckSplit = false;
>>>>>
>>>>
>>>> Can you give an example where CheckSplit is disabled here? I am finding
>>>> it hard to reason conditions under which this condition is satisfied.
>>>>
>>>
>>> Energy will be equal to the number of significant coefficients when each
>>> of the non-zero coefficients is one.
>>>
>>>
>>
>> I feel this condition may not be satisfied(very rare). You are
>> calculating energy as sum of abs values of transform coefficients and
>> checking with a threshold(number of coefficients).
>> There is a very less chance that both will be same. Either it should be
>> less than equal to or greater than equal to. This may be a test to
>> replicate a zero coefficient TU block. We
>> should check the subjective quality particularly for limit TU,
>> considering larger TUs produce more ringing artifacts.
>>
>
>
> The number of significant coefficients can never be less than the energy,
> if the energy is greater than the number of coefficients we don't want to
> limit the TU depth. We noticed a performance improvement up to 9% for
> certain videos. We checked the visual quality for the videos and have not
> observed any ringing artifacts so far.
>
> croud run:
>
> with energy check - encoded 500 frames in 545.41s (0.93 fps), 8878.75
> kb/s, Avg QP:38.00, SSIM Mean Y: 0.8659922 ( 8.729 dB)
> without energy check - encoded 500 frames in 568.05s (0.85 fps), 8879.27
> kb/s, Avg QP:38.00, SSIM Mean Y: 0.8660624 ( 8.731 dB)
> Improvement in performance - 8.6%
> Hit percentage of energy check - 61%
>
> kimono:
>
> with energy check - encoded 240 frames in 286.38s (0.84 fps), 4361.14
> kb/s, Avg QP:26.83, SSIM Mean Y: 0.9579188 (13.759 dB)
> without energy check - encoded 240 frames in 312.36s (0.77 fps), 4361.61
> kb/s, Avg QP:26.83, SSIM Mean Y: 0.9579364 (13.761 dB)
> Improvement in performance - 8.3%
> Hit percentage of energy check - 70%
>
>
>
>
>>
>>>>
>>>>>              }
>>>>>          }
>>>>> _______________________________________________
>>>>> x265-devel mailing list
>>>>> x265-devel at videolan.org
>>>>> https://mailman.videolan.org/listinfo/x265-devel
>>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> x265-devel mailing list
>>>> x265-devel at videolan.org
>>>> https://mailman.videolan.org/listinfo/x265-devel
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> x265-devel mailing list
>>> x265-devel at videolan.org
>>> https://mailman.videolan.org/listinfo/x265-devel
>>>
>>>
>>
>> _______________________________________________
>> x265-devel mailing list
>> x265-devel at videolan.org
>> https://mailman.videolan.org/listinfo/x265-devel
>>
>>
>
> _______________________________________________
> x265-devel mailing list
> x265-devel at videolan.org
> https://mailman.videolan.org/listinfo/x265-devel
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.videolan.org/pipermail/x265-devel/attachments/20161018/2ad66766/attachment-0001.html>