[x265] [PATCH] level: signal profile/level NONE if non-compliant configuration is found

[Steve Borho]

# HG changeset patch
# User Steve Borho <steve at borho.org>
# Date 1407706598 18000
#      Sun Aug 10 16:36:38 2014 -0500
# Node ID 7965aacd35ae87defeb0475ff4602394d494cd83
# Parent  6e4eb854220350cf0c980fc02cc11109c506585f
level: signal profile/level NONE if non-compliant configuration is found

Once you get above a certain resolution, the CTU size must be at least 32. There
is no level that supports a smaller CTU, so signal level NONE.  It's my
understanding that you cannot signal a profile if the level is NONE, so we
reset profile to NONE as well.

The same is true if NumPocTotalCurr is greater than 8; there are no levels which
support values that high.  Instead of signaling the closest level, we should
signal profile/level NONE

diff -r 6e4eb8542203 -r 7965aacd35ae source/encoder/level.cpp
--- a/source/encoder/level.cpp	Sat Aug 09 19:43:23 2014 -0500
+++ b/source/encoder/level.cpp	Sun Aug 10 16:36:38 2014 -0500
@@ -134,12 +134,26 @@
 
         /* For level 5 and higher levels, the value of CtbSizeY shall be equal to 32 or 64 */
         if (levels[i].levelEnum >= Level::LEVEL5 && param.maxCUSize < 32)
-            x265_log(&param, X265_LOG_WARNING, "CTU size is too small, stream will be non-compliant for level %s\n", levels[i].name);
+        {
+            x265_log(&param, X265_LOG_WARNING, "level %s detected, but CTU size 16 is non-compliant\n", levels[i].name);
+            vps.ptl.profileIdc = Profile::NONE;
+            vps.ptl.levelIdc = Level::NONE;
+            vps.ptl.tierFlag = Level::MAIN;
+            x265_log(&param, X265_LOG_INFO, "NONE profile, Level-NONE (Main tier)\n");
+            return;
+        }
 
         /* The value of NumPocTotalCurr shall be less than or equal to 8 */
-        int numPocTotalCurr = param.maxNumReferences + !!param.bframes;
+        int numPocTotalCurr = param.maxNumReferences + vps.numReorderPics;
         if (numPocTotalCurr > 8)
-            x265_log(&param, X265_LOG_WARNING, "Too many reference frames, stream will be non-compliant for level %s\n", levels[i].name);
+        {
+            x265_log(&param, X265_LOG_WARNING, "level %s detected, but NumPocTotalCurr (total references) is non-compliant\n", levels[i].name);
+            vps.ptl.profileIdc = Profile::NONE;
+            vps.ptl.levelIdc = Level::NONE;
+            vps.ptl.tierFlag = Level::MAIN;
+            x265_log(&param, X265_LOG_INFO, "NONE profile, Level-NONE (Main tier)\n");
+            return;
+        }
 
         vps.ptl.levelIdc = levels[i].levelEnum;
         if (bitrate > levels[i].maxBitrateMain && bitrate <= levels[i].maxBitrateHigh &&