[x265] [PATCH] analysis: bidir/inter choice using sa8dCost

[Steve Borho]

On 03/11, deepthi at multicorewareinc.com wrote:
> # HG changeset patch
> # User Deepthi Nandakumar <deepthi at multicorewareinc.com>
> # Date 1426065176 -19800
> #      Wed Mar 11 14:42:56 2015 +0530
> # Node ID 7d8f7aadfb6d0af071d804a3eec1511cd00818c5
> # Parent  37ec8002eba15ec3b7bf2b16a792af572b099078
> analysis: bidir/inter choice using sa8dCost
> 
> diff -r 37ec8002eba1 -r 7d8f7aadfb6d source/encoder/analysis.cpp
> --- a/source/encoder/analysis.cpp	Wed Mar 11 14:31:57 2015 +0530
> +++ b/source/encoder/analysis.cpp	Wed Mar 11 14:42:56 2015 +0530
> @@ -749,7 +749,7 @@
>          /* Compute Merge Cost */
>          md.pred[PRED_MERGE].cu.initSubCU(parentCTU, cuGeom);
>          md.pred[PRED_SKIP].cu.initSubCU(parentCTU, cuGeom);
> -        checkMerge2Nx2N_rd0_4(md.pred[PRED_SKIP], md.pred[PRED_MERGE], cuGeom);
> +        checkMerge2Nx2N_rd0_4(md.pred[PRED_SKIP], md.pred[PRED_MERGE], cuGeom); /* writes into md.bestMode */
>  
>          bool earlyskip = false;
>          if (m_param->rdLevel)
> @@ -822,24 +822,26 @@
>  
>              if (m_param->rdLevel >= 3)
>              {
> -                /* Calculate RD cost of best inter option */
> -                if (!m_bChromaSa8d) /* When m_bChromaSa8d is enabled, chroma MC has already been done */
> -                {
> -                    for (uint32_t puIdx = 0; puIdx < bestInter->cu.getNumPartInter(); puIdx++)
> -                    {
> -                        PredictionUnit pu(bestInter->cu, cuGeom, puIdx);
> -                        motionCompensation(bestInter->cu, pu, bestInter->predYuv, false, true);
> -                    }
> -                }
> -                encodeResAndCalcRdInterCU(*bestInter, cuGeom);
> -                checkBestMode(*bestInter, depth);
> -
> -                /* If BIDIR is available and within 17/16 of best inter option, choose by RDO */
> +                /* If BIDIR is available and within 17/16 of best inter option, choose bidir */
>                  if (m_slice->m_sliceType == B_SLICE && md.pred[PRED_BIDIR].sa8dCost != MAX_INT64 &&
>                      md.pred[PRED_BIDIR].sa8dCost * 16 <= bestInter->sa8dCost * 17)
>                  {
>                      encodeResAndCalcRdInterCU(md.pred[PRED_BIDIR], cuGeom);
> -                    checkBestMode(md.pred[PRED_BIDIR], depth);
> +                    checkBestMode(md.pred[PRED_BIDIR], depth); /* compare against Merge by RDO*/
> +                }
> +                else
> +                {
> +                    /* Calculate RD cost of best inter option */
> +                    if (!m_bChromaSa8d) /* When m_bChromaSa8d is enabled, chroma MC has already been done */
> +                    {
> +                        for (uint32_t puIdx = 0; puIdx < bestInter->cu.getNumPartInter(); puIdx++)
> +                        {
> +                            PredictionUnit pu(bestInter->cu, cuGeom, puIdx);
> +                            motionCompensation(bestInter->cu, pu, bestInter->predYuv, false, true);
> +                        }
> +                    }
> +                    encodeResAndCalcRdInterCU(*bestInter, cuGeom);
> +                    checkBestMode(*bestInter, depth);
>                  }

If bidir sa8d is within 17/16 of skip/merge sa8d, then compare bidir RD
cost with merge/skip RD cost, else compare merge/skip RD with best inter
RD cost?

It's giving up the chance to compare inter (psy-)RD cost against bidir
(psy-)RD cost. It might be appropriate for RD levels 0..2, but the case
for 3 and 4 is less solid.

-- 
Steve Borho