[x265] [PATCH 1 of 3] compress: Fixes fudging in merge costs, stores sa8d costs for bestMergeCU in m_sa8dCost

[Sumalatha Polureddy]

On Sun, Jul 13, 2014 at 6:51 PM, <deepthi at multicorewareinc.com> wrote:

> # HG changeset patch
> # User Deepthi Nandakumar <deepthi at multicorewareinc.com>
> # Date 1405255445 -19800
> #      Sun Jul 13 18:14:05 2014 +0530
> # Node ID 0fe178bf1f2d53636eb84cd0848e95b9b9cc4292
> # Parent  454a2fc37fee4ba7010d0044553575c6a9b81442
> compress: Fixes fudging in merge costs, stores sa8d costs for bestMergeCU
> in m_sa8dCost
>
> diff -r 454a2fc37fee -r 0fe178bf1f2d source/encoder/compress.cpp
> --- a/source/encoder/compress.cpp       Sun Jul 13 02:32:31 2014 -0500
> +++ b/source/encoder/compress.cpp       Sun Jul 13 18:14:05 2014 +0530
> @@ -269,7 +269,7 @@
>              outTempCU->m_totalBits = bitsCand;
>              outTempCU->m_totalDistortion =
> primitives.sa8d[sizeIdx](m_origYuv[depth]->getLumaAddr(),
> m_origYuv[depth]->getStride(),
>
>  m_tmpPredYuv[depth]->getLumaAddr(), m_tmpPredYuv[depth]->getStride());
> -            outTempCU->m_totalRDCost =
> m_rdCost.calcRdSADCost(outTempCU->m_totalDistortion,
> outTempCU->m_totalBits);
> +            outTempCU->m_sa8dCost =
> m_rdCost.calcRdSADCost(outTempCU->m_totalDistortion,
> outTempCU->m_totalBits);
>

Here we are calculating the SAD cost and saving in m_sa8dCost. But it is
not being used for selecting the best merge candidate. The code still
uses  "m_totalRDCost" for comparison which always has "MAX_INT64". when we
run it in debug mode , it hits X265_CHECK(outBestCU->m_totalRDCost !=
MAX_INT64, "no best prediction cost\n"); and throws error

>
>              if (outTempCU->m_totalRDCost < outBestCU->m_totalRDCost)
>              {
> @@ -294,10 +294,8 @@
>
>  outTempCU->getCUMvField(REF_PIC_LIST_1)->setAllMvField(mvFieldNeighbours[bestMergeCand][1],
> SIZE_2Nx2N, 0, 0);
>          outTempCU->m_totalBits = outBestCU->m_totalBits;
>          outTempCU->m_totalDistortion = outBestCU->m_totalDistortion;
> -        outTempCU->m_totalBits = 0;
> -        outTempCU->m_totalRDCost =
> m_rdCost.calcRdSADCost(outTempCU->m_totalDistortion,
> outTempCU->m_totalBits);
> -        outTempCU->m_sa8dCost = outTempCU->m_totalRDCost;
> -        outBestCU->m_sa8dCost = outTempCU->m_sa8dCost;
> +        outTempCU->m_sa8dCost = outBestCU->m_sa8dCost;
> +
>          if (m_param->rdLevel >= 1)
>          {
>              //calculate the motion compensation for chroma for the best
> mode selected
> _______________________________________________
> x265-devel mailing list
> x265-devel at videolan.org
> https://mailman.videolan.org/listinfo/x265-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.videolan.org/pipermail/x265-devel/attachments/20140714/5ba2eba3/attachment.html>