[x265] [PATCH] SAO: avoid negative indexes in 'x265_lambda2_tab' table

[Pradeep Ramachandran]

On Thu, Feb 16, 2017 at 12:11 AM, Mateusz Brzostek <mateusz at msystem.waw.pl>
wrote:

> This patch fixes issue #323 -- crash when encoding to 422. The 'qpCb'
> index is -2 on Selur's sample file and options.
>
> Please review.
>
> Mateusz
>
> # HG changeset patch
> # User Ma0 <mateuszb at poczta.onet.pl>
> # Date 1487183578 -3600
> #      Wed Feb 15 19:32:58 2017 +0100
> # Node ID 7b55d81f0677b7cbef5a490d90bd9cbbf96127cb
> # Parent  912dd749bdb53cdd1e251bc3a69e4c41ece3b308
> SAO: avoid negative indexes in 'x265_lambda2_tab' table
>
> diff -r 912dd749bdb5 -r 7b55d81f0677 source/encoder/sao.cpp
> --- a/source/encoder/sao.cpp    Wed Feb 15 12:04:41 2017 +0530
> +++ b/source/encoder/sao.cpp    Wed Feb 15 19:32:58 2017 +0100
> @@ -1234,7 +1234,7 @@
>      if (m_param->internalCsp == X265_CSP_I420)
>          qpCb = x265_clip3(m_param->rc.qpMin, m_param->rc.qpMax,
> (int)g_chromaScale[qp + slice->m_pps->chromaQpOffset[0]]);
>      else
> -        qpCb = X265_MIN(qp + slice->m_pps->chromaQpOffset[0],
> QP_MAX_SPEC);
> +        qpCb = x265_clip3(0, QP_MAX_SPEC, qp +
> slice->m_pps->chromaQpOffset[0]);
>

This clip should be between m_param->rc.qpMin and m_param->rc.qpMax, as per
what is done for the 4:2:0 videos.


>
>      lambda[0] = (int64_t)floor(256.0 * x265_lambda2_tab[qp]);
>      lambda[1] = (int64_t)floor(256.0 * x265_lambda2_tab[qpCb]); // Use Cb
> QP for SAO chroma
>
>
> _______________________________________________
> x265-devel mailing list
> x265-devel at videolan.org
> https://mailman.videolan.org/listinfo/x265-devel
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.videolan.org/pipermail/x265-devel/attachments/20170216/0189e1d6/attachment.html>