[x265] [PATCH 1 of 2] rdcost: auto down-scale psy-rd at higher QPs

[Steve Borho]

# HG changeset patch
# User Steve Borho <steve at borho.org>
# Date 1422549267 21600
#      Thu Jan 29 10:34:27 2015 -0600
# Node ID e91e608e751339b28746f928835fb4c1e3b2d4fb
# Parent  5e5dc3763f6386da9722903033a2b9dd263a5226
rdcost: auto down-scale psy-rd at higher QPs

When QP gets above 42, turn down psy-rd by half. When it gets to 50 disable it
outright. Note that we're not mucking with psy-rdoq at this time.

diff -r 5e5dc3763f63 -r e91e608e7513 source/encoder/rdcost.h
--- a/source/encoder/rdcost.h	Thu Jan 29 10:37:54 2015 -0600
+++ b/source/encoder/rdcost.h	Thu Jan 29 10:34:27 2015 -0600
@@ -48,10 +48,17 @@
     {
         m_qp = qp;
         int qpCb, qpCr;
+
         /* Scale PSY RD factor by a slice type factor */
         static const uint32_t psyScaleFix8[3] = { 300, 256, 96 }; /* B, P, I */
         m_psyRd = (m_psyRdBase * psyScaleFix8[slice.m_sliceType]) >> 8;
 
+        /* Scale PSY RD factor by QP, at high QP psy-rd can cause artifacts */
+        if (qp >= 42)
+            m_psyRd >>= 1;
+        else if (qp >= 50)
+            m_psyRd = 0;
+
         setLambda(x265_lambda2_tab[qp], x265_lambda_tab[qp]);
         if (slice.m_sps->chromaFormatIdc == X265_CSP_I420)
             qpCb = x265_clip3(QP_MIN, QP_MAX_MAX, (int)g_chromaScale[qp + slice.m_pps->chromaQpOffset[0]]);