[x265] [PATCH] rd level: fix for output mismatch

[Deepthi Nandakumar]

On Mon, Dec 16, 2013 at 1:58 PM, <sumalatha at multicorewareinc.com> wrote:

> # HG changeset patch
> # User Sumalatha Polureddy
> # Date 1387182478 -19800
> # Node ID 70a5357626fe2f45f0c462871c26b468a023a322
> # Parent  238d7f272d1e5c49d90bb15f40b2d7e90be78ba1
> rd level: fix for output mismatch
>
> For boundary conditions, cost calculation was wrong.
>
> diff -r 238d7f272d1e -r 70a5357626fe source/encoder/compress.cpp
> --- a/source/encoder/compress.cpp       Mon Dec 16 10:57:06 2013 +0530
> +++ b/source/encoder/compress.cpp       Mon Dec 16 13:57:58 2013 +0530
> @@ -768,11 +768,12 @@
>                  m_entropyCoder->resetBits();
>                  m_entropyCoder->encodeSplitFlag(outTempCU, 0, depth,
> true);
>                  outTempCU->m_totalBits +=
> m_entropyCoder->getNumberOfWrittenBits(); // split bits
> -                outTempCU->m_totalCost =
> m_rdCost->calcRdCost(outTempCU->m_totalDistortion, outTempCU->m_totalBits);
>              }
> -            else
> -                outTempCU->m_totalCost =
> m_rdCost->calcRdSADCost(outTempCU->m_totalDistortion,
> outTempCU->m_totalBits);
>          }
> +        if (m_cfg->param.rdLevel > 1)
> +            outTempCU->m_totalCost =
> m_rdCost->calcRdCost(outTempCU->m_totalDistortion, outTempCU->m_totalBits);
> +        else
> +            outTempCU->m_totalCost =
> m_rdCost->calcRdSADCost(outTempCU->m_totalDistortion,
> outTempCU->m_totalBits);
>
> Whats the logic behind this? If it's a valid output change for rd >1, we
dont have to revert.


>          if ((g_maxCUWidth >> depth) ==
> outTempCU->getSlice()->getPPS()->getMinCuDQPSize() &&
> outTempCU->getSlice()->getPPS()->getUseDQP())
>          {
> _______________________________________________
> x265-devel mailing list
> x265-devel at videolan.org
> https://mailman.videolan.org/listinfo/x265-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.videolan.org/pipermail/x265-devel/attachments/20131216/784b432e/attachment-0001.html>