[x265] [PATCH] rc: bug fix - clip qp before setting into TComPic:m_avgQpRc.

[Aarthi Priya Thirumalai]

# HG changeset patch
# User Aarthi Thirumalai
# Date 1393263092 -19800
#      Mon Feb 24 23:01:32 2014 +0530
# Node ID 2f813eb983817a48aeb483ba2ac28303d5b4d451
# Parent  dce74082c20eea1f7ef9eb10f9a9addc5e7c7bb7
rc: bug fix - clip qp before setting into TComPic:m_avgQpRc.

This fix resolves Encoder crash caused due to invalid qp being used in each
CU.

diff -r dce74082c20e -r 2f813eb98381 source/encoder/ratecontrol.cpp
--- a/source/encoder/ratecontrol.cpp Mon Feb 24 17:50:55 2014 +0530
+++ b/source/encoder/ratecontrol.cpp Mon Feb 24 23:01:32 2014 +0530
@@ -409,7 +409,8 @@
         /* Update rce for use in rate control VBV later */
         rce->lastSatd = currentSatd;
         double q = qScale2qp(rateEstimateQscale(pic, rce));
-        qp = Clip3(MIN_QP, MAX_MAX_QP, (int)(q + 0.5));
+        q = Clip3((double)MIN_QP, (double)MAX_MAX_QP, q);
+        qp = int(q + 0.5);
         rce->qpaRc = pic->m_avgQpRc = q;
         /* copy value of lastRceq into thread local rce struct *to be used
in RateControlEnd() */
         rce->qRceq = lastRceq;
@@ -585,10 +586,10 @@
             if (qCompress != 1 && framesDone == 0)
                 q = qp2qScale(ABR_INIT_QP) / fabs(cfg->param.rc.ipFactor);
         }
-        qpNoVbv = qScale2qp(q);
         double lmin1 = lmin[sliceType];
         double lmax1 = lmax[sliceType];
         q = Clip3(lmin1, lmax1, q);
+        qpNoVbv = qScale2qp(q);

         q = clipQscale(pic, q);




On Mon, Feb 24, 2014 at 10:38 PM, Derek Buitenhuis <
derek.buitenhuis at gmail.com> wrote:

> On 2/24/2014 12:56 PM, aarthi at multicorewareinc.com wrote:
> > # HG changeset patch
> > # User Aarthi Thirumalai
> > # Date 1393244587 -19800
> > #      Mon Feb 24 17:53:07 2014 +0530
> > # Node ID 3abef12d5b47106005c813bfd60ea49c31048f12
> > # Parent  dce74082c20eea1f7ef9eb10f9a9addc5e7c7bb7
> > rc: bug fix for encoder crash
>
> Please state *what* was fixed. It's not obvious from the commit.
>
> - Derek
> _______________________________________________
> x265-devel mailing list
> x265-devel at videolan.org
> https://mailman.videolan.org/listinfo/x265-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.videolan.org/pipermail/x265-devel/attachments/20140224/d31c98d1/attachment-0001.html>