[x265] [PATCH] rd Level: improve bitrate and psnr in rd level 2

[sumalatha at multicorewareinc.com]

# HG changeset patch
# User Sumalatha Polureddy
# Date 1389343212 -19800
# Node ID 104fb077a9813f0f3bb94c370e134d0d6d180809
# Parent  80b63c3ee144e6edbafbbe281ad3d1d8505be1f6
rd Level: improve bitrate and psnr in rd level 2

a. Always allow intra mode for mode decision
b. increase more merge skips at each depth

Results are below
CLI: input.y4m -o bitstream.hevc -r recon.y4m --preset veryfast --rd 2

rd3 / previous rd2 / latest rd2
BasketballDrive_1920x1080_50
fps: 4.24/5.615.7
psnr: 33.67/33.95/33.677
birate: 2211/2272/2188

Kimono1_1920x1080_24
fps: 5.45/6.11/6.1
psnr: 35.004/35.144/34.835
birate: 713/740/692

FourPeople_1280x720_60
fps: 7.99/10.86/11.46
psnr: 34.93/35.17/34.931
birate: 320/341/341

big_buck_bunny_360p24
fps: 38.98/49.32/44.46
psnr: 33.21/33.36/33.167
birate: 56/60/56

Johnny_1280x720_60
fps: 8.21/9.58/11.32
psnr: 36.74/37.238/37.01
birate: 203/204/206

diff -r 80b63c3ee144 -r 104fb077a981 source/encoder/compress.cpp
--- a/source/encoder/compress.cpp	Thu Jan 09 12:50:16 2014 +0550
+++ b/source/encoder/compress.cpp	Fri Jan 10 14:10:12 2014 +0530
@@ -517,12 +517,6 @@
                         bdoIntra = (outBestCU->getCbf(0, TEXT_LUMA) ||  outBestCU->getCbf(0, TEXT_CHROMA_U) ||
                                     outBestCU->getCbf(0, TEXT_CHROMA_V));
                     }
-                    else
-                    {
-                        uint32_t threshold[4] = { 20000, 6000, 1600, 500 };
-                        int index = 4 - g_convertToBit[outBestCU->getWidth(0)];
-                        bdoIntra = (outBestCU->m_totalDistortion > threshold[index]);
-                    }
                     if (bdoIntra)
                     {
                         xComputeCostIntraInInter(m_intraInInterCU[depth], SIZE_2Nx2N);
@@ -587,8 +581,12 @@
                         }
                         m_search->encodeResAndCalcRdInterCU(m_mergeCU[depth], m_origYuv[depth], bestMergePred, m_tmpResiYuv[depth],
                                                             m_bestResiYuv[depth], m_tmpRecoYuv[depth], true);
-
-                        if (m_mergeCU[depth]->m_totalCost < outBestCU->m_totalCost)
+                        double lambda[4];
+                        lambda[0] = 1.06;
+                        lambda[1] = 1.5;
+                        lambda[2] = 1.1;
+                        lambda[3] = 1.0;
+                        if (m_mergeCU[depth]->m_totalCost < lambda[depth] * outBestCU->m_totalCost)
                         {
                             outBestCU = m_mergeCU[depth];
                             tempYuv = m_bestRecoYuv[depth];