[x265] [PATCH 1 of 2] rdcost: auto down-scale psy-rd at higher QPs

[Steve Borho]

On 01/29, Steve Borho wrote:
> # HG changeset patch
> # User Steve Borho <steve at borho.org>
> # Date 1422549267 21600
> #      Thu Jan 29 10:34:27 2015 -0600
> # Node ID e91e608e751339b28746f928835fb4c1e3b2d4fb
> # Parent  5e5dc3763f6386da9722903033a2b9dd263a5226
> rdcost: auto down-scale psy-rd at higher QPs
> 
> When QP gets above 42, turn down psy-rd by half. When it gets to 50 disable it
> outright. Note that we're not mucking with psy-rdoq at this time.
> 
> diff -r 5e5dc3763f63 -r e91e608e7513 source/encoder/rdcost.h
> --- a/source/encoder/rdcost.h	Thu Jan 29 10:37:54 2015 -0600
> +++ b/source/encoder/rdcost.h	Thu Jan 29 10:34:27 2015 -0600
> @@ -48,10 +48,17 @@
>      {
>          m_qp = qp;
>          int qpCb, qpCr;
> +
>          /* Scale PSY RD factor by a slice type factor */
>          static const uint32_t psyScaleFix8[3] = { 300, 256, 96 }; /* B, P, I */
>          m_psyRd = (m_psyRdBase * psyScaleFix8[slice.m_sliceType]) >> 8;
>  
> +        /* Scale PSY RD factor by QP, at high QP psy-rd can cause artifacts */
> +        if (qp >= 42)
> +            m_psyRd >>= 1;
> +        else if (qp >= 50)
> +            m_psyRd = 0;

obviously, this won't work right unless the checks are reversed

>          setLambda(x265_lambda2_tab[qp], x265_lambda_tab[qp]);
>          if (slice.m_sps->chromaFormatIdc == X265_CSP_I420)
>              qpCb = x265_clip3(QP_MIN, QP_MAX_MAX, (int)g_chromaScale[qp + slice.m_pps->chromaQpOffset[0]]);

-- 
Steve Borho