[x265] [PATCH] Quant: fix for decoding hash mismatch in lossless mode

[Ashok Kumar Mishra]

Yes.this flag is used to decide whether to perform transform or not in
cu-lossless and lossless mode.
But it seems this is redundant. We can directly read the value from
cu.m_tqBypass[] and decide whether to perform transform or not.

On Wed, Aug 12, 2015 at 4:23 PM, Steve Borho <steve at borho.org> wrote:

> On 08/12, Ashok Kumar Mishra wrote:
> > On Wed, Aug 12, 2015 at 1:05 PM, Steve Borho <steve at borho.org> wrote:
> >
> > > On 08/11, ashok at multicorewareinc.com wrote:
> > > > # HG changeset patch
> > > > # User Ashok Kumar Mishra<ashok at multicorewareinc.com>
> > > > # Date 1439283627 -19800
> > > > #      Tue Aug 11 14:30:27 2015 +0530
> > > > # Node ID 854f0f9aabf7ec45da3cb301e0f78a5bd1e96391
> > > > # Parent  3eb2ec5922be1cd934dec7f7ed886d03c0125ef5
> > > > Quant: fix for decoding hash mismatch in lossless mode
> > > >
> > > > diff -r 3eb2ec5922be -r 854f0f9aabf7 source/common/quant.cpp
> > > > --- a/source/common/quant.cpp Wed Aug 05 12:20:01 2015 +0530
> > > > +++ b/source/common/quant.cpp Tue Aug 11 14:30:27 2015 +0530
> > > > @@ -228,9 +228,6 @@
> > > >
> > > >  void Quant::setQPforQuant(const CUData& ctu, int qp)
> > > >  {
> > > > -    m_tqBypass = !!ctu.m_tqBypass[0];
> > > > -    if (m_tqBypass)
> > > > -        return;
> > >
> > > that's a bit unexpected. what field needs to be initialized for
> > > cu-lossless to work right?
> > >
> > > >      m_nr = m_frameNr ? &m_frameNr[ctu.m_encData->m_frameEncoderID] :
> > > NULL;
> > > >      m_qpParam[TEXT_LUMA].setQpParam(qp + QP_BD_OFFSET);
> > > >      setChromaQP(qp + ctu.m_slice->m_pps->chromaQpOffset[0],
> > > TEXT_CHROMA_U, ctu.m_chromaFormat);
> >
> > When cu-lossless mode is set, we are comparing rate distortion cost for
> > normal encoding and lossless encoding.
> > For that tq_bypss flag is OFF for normal encoding and ON for lossless
> > encoding. But since the qgSize is set to 32,
> > qp value will change for depth 1 (32x32 CU). So for normal encoding we
> need
> > to set the new qp value. The reason for hash mismatch
> > was we were encoding the CU with the old qp value and signalling the
> > decoder new qp value.
>
> Ahh, Analysis::tryLossless() is setting m_quant.m_tqBypass directly. Ok,
> this makes sense then.
>
> Any idea why Search::checkIntra() does this:
>
>   m_quant.m_tqBypass = !!cu.m_tqBypass[0];
>
> it doesn't seem helpful
>
> --
> Steve Borho
> _______________________________________________
> x265-devel mailing list
> x265-devel at videolan.org
> https://mailman.videolan.org/listinfo/x265-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.videolan.org/pipermail/x265-devel/attachments/20150812/937afd5e/attachment.html>