[x265] [PATCH] lambda: New 8-bit lambda2 table

[Tom Vaughan]

Note that constant qp (--qp) and CRF (--crf) encodes will have a different
bit rate than before (higher bit rates / larger files for a given
setting).  Average Bit Rate (--bitrate) encodes will end up with the same
bit rate as before, but the initial bit rate will have more variation than
before, until we are able to make some adjustments to our ABR rate control
algorithm.  So it is recommended to do 2 pass encodes to compare visual
quality with previous builds of x265.

-----Original Message-----
From: x265-devel [mailto:x265-devel-bounces at videolan.org] On Behalf Of
Pradeep Ramachandran
Sent: Monday, March 13, 2017 1:31 AM
To: x265-devel at videolan.org
Subject: [x265] [PATCH] lambda: New 8-bit lambda2 table

# HG changeset patch
# User Pradeep Ramachandran <pradeep at multicorewareinc.com> # Date
1489382943 -19800
#      Mon Mar 13 10:59:03 2017 +0530
# Node ID 080c27ef4bb8d9a51cac2f4e9e12ad4b3cf9bae1
# Parent  88fd9082764c7d7b4668e30763a93215980efd70
lambda: New 8-bit lambda2 table.

Considerably improves subjective visual quality.

diff -r 88fd9082764c -r 080c27ef4bb8 source/common/constants.cpp
--- a/source/common/constants.cpp	Wed Mar 08 17:15:59 2017 +0530
+++ b/source/common/constants.cpp	Mon Mar 13 10:59:03 2017 +0530
@@ -129,24 +129,24 @@
     456.1401, 512.0000, 574.7006, 645.0796, 724.0773  };

-// lambda2 = pow(lambda, 2) * scale (0.85);
+// lambda2 = 0.038 * pow(0.234, QP)
 double x265_lambda2_tab[QP_MAX_MAX + 1] =  {
-    0.0531, 0.0669, 0.0843, 0.1063, 0.1339,
-    0.1687, 0.2125, 0.2677, 0.3373, 0.4250,
-    0.5355, 0.6746, 0.8500, 1.0709, 1.3493,
-    1.7000, 2.1419, 2.6986, 3.4000, 4.2837,
-    5.3970, 6.8000, 8.5675, 10.7943, 13.6000,
-    17.1345, 21.5887, 27.2004, 34.2699, 43.1773,
-    54.4000, 68.5397, 86.3551, 108.7998, 137.0792,
-    172.7097, 217.6000, 274.1590, 345.4172, 435.1993,
-    548.3169, 690.8389, 870.4000, 1096.6362, 1381.6757,
-    1740.7974, 2193.2676, 2763.3460, 3481.6000, 4386.5446,
-    5526.6890, 6963.2049, 8773.0879, 11053.3840, 13926.4000,
-    17546.1542, 22106.7835, 27852.7889, 35092.3170, 44213.5749,
-    55705.6000, 70184.6657, 88427.1342, 111411.2172, 140369.3373,
-    176854.2222, 222822.4000, 280738.6627, 353708.5368, 445644.7459
-};
+    0.0380, 0.0480, 0.0606, 0.0766, 0.0968,
+    0.1224, 0.1547, 0.1955, 0.2470, 0.3121,
+    0.3944, 0.4984, 0.6299, 0.7959, 1.0058,
+    1.2710, 1.6061, 2.0295, 2.5646, 3.2408,
+    4.0952, 5.1749, 6.5393, 8.2633, 10.4419,
+    13.1949, 16.6736, 21.0695, 26.6244, 33.6438,
+    42.5138, 53.7224, 67.8860, 85.7838, 108.4003,
+    136.9794, 173.0933, 218.7284, 276.3949, 349.2649,
+    441.3467, 557.7054, 704.7413, 890.5425, 1125.3291,
+    1422.0160, 1796.9227, 2270.6714, 2869.3215, 3625.8023,
+    4581.7251, 5789.6717, 7316.0868, 9244.9328, 11682.3084,
+    14762.2847, 18654.2798, 23572.3779, 29787.1055, 37640.3119,
+    47563.9728, 60103.9523, 75950.0283, 95973.8349, 121276.8079,
+    153250.7703, 193654.4919, 244710.4321, 309226.9897, 390752.9823 };

 #endif