[x265] [PATCH] SAO: avoid negative indexes in 'x265_lambda2_tab' table

[Mateusz Brzostek]

This patch fixes issue #323 -- crash when encoding to 422. The 'qpCb' index is -2 on Selur's sample file and options.

Please review.

Mateusz

# HG changeset patch
# User Ma0 <mateuszb at poczta.onet.pl>
# Date 1487183578 -3600
#      Wed Feb 15 19:32:58 2017 +0100
# Node ID 7b55d81f0677b7cbef5a490d90bd9cbbf96127cb
# Parent  912dd749bdb53cdd1e251bc3a69e4c41ece3b308
SAO: avoid negative indexes in 'x265_lambda2_tab' table

diff -r 912dd749bdb5 -r 7b55d81f0677 source/encoder/sao.cpp
--- a/source/encoder/sao.cpp	Wed Feb 15 12:04:41 2017 +0530
+++ b/source/encoder/sao.cpp	Wed Feb 15 19:32:58 2017 +0100
@@ -1234,7 +1234,7 @@
     if (m_param->internalCsp == X265_CSP_I420)
         qpCb = x265_clip3(m_param->rc.qpMin, m_param->rc.qpMax, (int)g_chromaScale[qp + slice->m_pps->chromaQpOffset[0]]);
     else
-        qpCb = X265_MIN(qp + slice->m_pps->chromaQpOffset[0], QP_MAX_SPEC);
+        qpCb = x265_clip3(0, QP_MAX_SPEC, qp + slice->m_pps->chromaQpOffset[0]);
 
     lambda[0] = (int64_t)floor(256.0 * x265_lambda2_tab[qp]);
     lambda[1] = (int64_t)floor(256.0 * x265_lambda2_tab[qpCb]); // Use Cb QP for SAO chroma 

-------------- next part --------------
# HG changeset patch
# User Ma0 <mateuszb at poczta.onet.pl>
# Date 1487183578 -3600
#      Wed Feb 15 19:32:58 2017 +0100
# Node ID 7b55d81f0677b7cbef5a490d90bd9cbbf96127cb
# Parent  912dd749bdb53cdd1e251bc3a69e4c41ece3b308
SAO: avoid negative indexes in 'x265_lambda2_tab' table

diff -r 912dd749bdb5 -r 7b55d81f0677 source/encoder/sao.cpp
--- a/source/encoder/sao.cpp	Wed Feb 15 12:04:41 2017 +0530
+++ b/source/encoder/sao.cpp	Wed Feb 15 19:32:58 2017 +0100
@@ -1234,7 +1234,7 @@
     if (m_param->internalCsp == X265_CSP_I420)
         qpCb = x265_clip3(m_param->rc.qpMin, m_param->rc.qpMax, (int)g_chromaScale[qp + slice->m_pps->chromaQpOffset[0]]);
     else
-        qpCb = X265_MIN(qp + slice->m_pps->chromaQpOffset[0], QP_MAX_SPEC);
+        qpCb = x265_clip3(0, QP_MAX_SPEC, qp + slice->m_pps->chromaQpOffset[0]);
 
     lambda[0] = (int64_t)floor(256.0 * x265_lambda2_tab[qp]);
     lambda[1] = (int64_t)floor(256.0 * x265_lambda2_tab[qpCb]); // Use Cb QP for SAO chroma