[x265] [PATCH] rd level: fix for output mismatch

[Sumalatha Polureddy]

On Mon, Dec 16, 2013 at 2:49 PM, Deepthi Nandakumar <
deepthi at multicorewareinc.com> wrote:

>
>
>
> On Mon, Dec 16, 2013 at 1:58 PM, <sumalatha at multicorewareinc.com> wrote:
>
>> # HG changeset patch
>> # User Sumalatha Polureddy
>> # Date 1387182478 -19800
>> # Node ID 70a5357626fe2f45f0c462871c26b468a023a322
>> # Parent  238d7f272d1e5c49d90bb15f40b2d7e90be78ba1
>> rd level: fix for output mismatch
>>
>> For boundary conditions, cost calculation was wrong.
>>
>> diff -r 238d7f272d1e -r 70a5357626fe source/encoder/compress.cpp
>> --- a/source/encoder/compress.cpp       Mon Dec 16 10:57:06 2013 +0530
>> +++ b/source/encoder/compress.cpp       Mon Dec 16 13:57:58 2013 +0530
>> @@ -768,11 +768,12 @@
>>                  m_entropyCoder->resetBits();
>>                  m_entropyCoder->encodeSplitFlag(outTempCU, 0, depth,
>> true);
>>                  outTempCU->m_totalBits +=
>> m_entropyCoder->getNumberOfWrittenBits(); // split bits
>> -                outTempCU->m_totalCost =
>> m_rdCost->calcRdCost(outTempCU->m_totalDistortion, outTempCU->m_totalBits);
>>              }
>> -            else
>> -                outTempCU->m_totalCost =
>> m_rdCost->calcRdSADCost(outTempCU->m_totalDistortion,
>> outTempCU->m_totalBits);
>>          }
>> +        if (m_cfg->param.rdLevel > 1)
>> +            outTempCU->m_totalCost =
>> m_rdCost->calcRdCost(outTempCU->m_totalDistortion, outTempCU->m_totalBits);
>> +        else
>> +            outTempCU->m_totalCost =
>> m_rdCost->calcRdSADCost(outTempCU->m_totalDistortion,
>> outTempCU->m_totalBits);
>>
>> Whats the logic behind this? If it's a valid output change for rd >1, we
> dont have to revert.
>

 Output change is not valid. It was mistake that cost was calculated only
if it is not boundary.

         if ((g_maxCUWidth >> depth) ==
>> outTempCU->getSlice()->getPPS()->getMinCuDQPSize() &&
>> outTempCU->getSlice()->getPPS()->getUseDQP())
>>          {
>> _______________________________________________
>> x265-devel mailing list
>> x265-devel at videolan.org
>> https://mailman.videolan.org/listinfo/x265-devel
>>
>
>
> _______________________________________________
> x265-devel mailing list
> x265-devel at videolan.org
> https://mailman.videolan.org/listinfo/x265-devel
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.videolan.org/pipermail/x265-devel/attachments/20131216/c746bcad/attachment.html>