[x265] [PATCH] rdcost: fix cut-paste bugs in chroma QP offset R-D calculations

[Steve Borho]

On 05/04, Steve Borho wrote:
> # HG changeset patch
> # User Steve Borho <steve at borho.org>
> # Date 1430767137 18000
> #      Mon May 04 14:18:57 2015 -0500
> # Node ID 4c0648e4c3cc75cafa6b525c0e79dd16b023cdae
> # Parent  28d7e7582028b76afb9faf885a35e293d77e437f
> rdcost: fix cut-paste bugs in chroma QP offset R-D calculations
> 
> diff -r 28d7e7582028 -r 4c0648e4c3cc source/encoder/rdcost.h
> --- a/source/encoder/rdcost.h	Mon May 04 12:57:08 2015 -0500
> +++ b/source/encoder/rdcost.h	Mon May 04 14:18:57 2015 -0500
> @@ -71,9 +71,9 @@
>          m_chromaDistWeight[0] = lambdaOffset;
>  
>          if (slice.m_sps->chromaFormatIdc == X265_CSP_I420)
> -            qpCr = x265_clip3(QP_MIN, QP_MAX_MAX, (int)g_chromaScale[qp + slice.m_pps->chromaQpOffset[0]]);
> +            qpCr = x265_clip3(QP_MIN, QP_MAX_MAX, (int)g_chromaScale[qp + slice.m_pps->chromaQpOffset[1]]);
>          else
> -            qpCr = X265_MIN(qp + slice.m_pps->chromaQpOffset[0], QP_MAX_SPEC);
> +            qpCr = X265_MIN(qp + slice.m_pps->chromaQpOffset[1], QP_MAX_SPEC);

I've pushed this obvious bug fix, but looking at this again I wonder why
i444 and i422 chroma QPs are not clamped between QP_MIN and QP_MAX_MAX
as i420 chroma QPs are.

It would be pretty easy to make a negative qpCr or qpCb which might be
dangerous for the table below (or at least cause unexpected results).

>          chroma_offset_idx = X265_MIN(qp - qpCr + 12, MAX_CHROMA_LAMBDA_OFFSET);
>          lambdaOffset = m_psyRd ? x265_chroma_lambda2_offset_tab[chroma_offset_idx] : 256;
>          m_chromaDistWeight[1] = lambdaOffset;

-- 
Steve Borho