[x265] [PATCH] Quant: fix for decoding hash mismatch in lossless mode

[Ashok Kumar Mishra]

On Wed, Aug 12, 2015 at 1:05 PM, Steve Borho <steve at borho.org> wrote:

> On 08/11, ashok at multicorewareinc.com wrote:
> > # HG changeset patch
> > # User Ashok Kumar Mishra<ashok at multicorewareinc.com>
> > # Date 1439283627 -19800
> > #      Tue Aug 11 14:30:27 2015 +0530
> > # Node ID 854f0f9aabf7ec45da3cb301e0f78a5bd1e96391
> > # Parent  3eb2ec5922be1cd934dec7f7ed886d03c0125ef5
> > Quant: fix for decoding hash mismatch in lossless mode
> >
> > diff -r 3eb2ec5922be -r 854f0f9aabf7 source/common/quant.cpp
> > --- a/source/common/quant.cpp Wed Aug 05 12:20:01 2015 +0530
> > +++ b/source/common/quant.cpp Tue Aug 11 14:30:27 2015 +0530
> > @@ -228,9 +228,6 @@
> >
> >  void Quant::setQPforQuant(const CUData& ctu, int qp)
> >  {
> > -    m_tqBypass = !!ctu.m_tqBypass[0];
> > -    if (m_tqBypass)
> > -        return;
>
> that's a bit unexpected. what field needs to be initialized for
> cu-lossless to work right?
>
> >      m_nr = m_frameNr ? &m_frameNr[ctu.m_encData->m_frameEncoderID] :
> NULL;
> >      m_qpParam[TEXT_LUMA].setQpParam(qp + QP_BD_OFFSET);
> >      setChromaQP(qp + ctu.m_slice->m_pps->chromaQpOffset[0],
> TEXT_CHROMA_U, ctu.m_chromaFormat);
> > _______________________________________________
> > x265-devel mailing list
> > x265-devel at videolan.org
> > https://mailman.videolan.org/listinfo/x265-devel
>
> --
> Steve Borho
> _______________________________________________
> x265-devel mailing list
> x265-devel at videolan.org
> https://mailman.videolan.org/listinfo/x265-devel


When cu-lossless mode is set, we are comparing rate distortion cost for
normal encoding and lossless encoding.
For that tq_bypss flag is OFF for normal encoding and ON for lossless
encoding. But since the qgSize is set to 32,
qp value will change for depth 1 (32x32 CU). So for normal encoding we need
to set the new qp value. The reason for hash mismatch
was we were encoding the CU with the old qp value and signalling the
decoder new qp value.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.videolan.org/pipermail/x265-devel/attachments/20150812/cc3677dd/attachment.html>