[x265] [PATCH] fix for crash when scalingList feature is enabled

[Sumalatha Polureddy]

On Tue, Jul 15, 2014 at 12:29 PM, Steve Borho <steve at borho.org> wrote:

> On 07/15, sumalatha at multicorewareinc.com wrote:
> > # HG changeset patch
> > # User Sumalatha Polureddy<sumalatha at multicorewareinc.com>
> > # Date 1405401248 -19800
> > # Node ID 52aa7ca073cf83ddde782175dd8c488b712f05c2
> > # Parent  51b3ff5e88e24c4a29cf422434f2e2a38f99d862
> > fix for crash when scalingList feature is enabled
> >
> > diff -r 51b3ff5e88e2 -r 52aa7ca073cf source/Lib/TLibCommon/TComSlice.cpp
> > --- a/source/Lib/TLibCommon/TComSlice.cpp     Mon Jul 14 13:42:37 2014
> -0500
> > +++ b/source/Lib/TLibCommon/TComSlice.cpp     Tue Jul 15 10:44:08 2014
> +0530
> > @@ -715,19 +715,6 @@
> >      destroy();
> >  }
> >
> > -/** set default quantization matrix to array
> > -*/
> > -void TComSlice::setDefaultScalingList()
> > -{
> > -    for (uint32_t sizeId = 0; sizeId < SCALING_LIST_SIZE_NUM; sizeId++)
> > -    {
> > -        for (uint32_t listId = 0; listId < g_scalingListNum[sizeId];
> listId++)
> > -        {
> > -            getScalingList()->processDefaultMarix(sizeId, listId);
> > -        }
> > -    }
> > -}
> > -
> >  /** check if use default quantization matrix
> >   * \returns true if use default quantization matrix in all size
> >  */
> > diff -r 51b3ff5e88e2 -r 52aa7ca073cf source/Lib/TLibCommon/TComSlice.h
> > --- a/source/Lib/TLibCommon/TComSlice.h       Mon Jul 14 13:42:37 2014
> -0500
> > +++ b/source/Lib/TLibCommon/TComSlice.h       Tue Jul 15 10:44:08 2014
> +0530
> > @@ -1484,7 +1484,6 @@
> >
> >      TComScalingList*   getScalingList()        { return m_scalingList; }
> >
> > -    void  setDefaultScalingList();
> >      bool  checkDefaultScalingList();
>
> does anything call this checkDefaultScalingList() method? I imagine we
> need to, particularly after loading one from a file.
>
In HM, checkDefaultScalingList() is called when scalinglistid
is SCALING_LIST_FILE_READ


>
> >      void      setCabacInitFlag(bool val)   { m_cabacInitFlag = val; }
> //!< set CABAC initial flag
> >
> > diff -r 51b3ff5e88e2 -r 52aa7ca073cf source/Lib/TLibEncoder/TEncCu.cpp
> > --- a/source/Lib/TLibEncoder/TEncCu.cpp       Mon Jul 14 13:42:37 2014
> -0500
> > +++ b/source/Lib/TLibEncoder/TEncCu.cpp       Tue Jul 15 10:44:08 2014
> +0530
> > @@ -83,6 +83,7 @@
> >      }
> >      else if (top->m_useScalingListId == SCALING_LIST_DEFAULT)
> >      {
> > +        top->setDefaultScalingList();
>
> this isn't the place to initialize the scaling list (especially not when
> this function can run many times); it should be done at about the same
> time the SPS and PPS are configured at startup.
>

will change and the patch

>
> >          m_trQuant.setScalingList(top->getScalingList());
> >          m_trQuant.setUseScalingList(true);
> >      }
> > diff -r 51b3ff5e88e2 -r 52aa7ca073cf source/encoder/encoder.cpp
> > --- a/source/encoder/encoder.cpp      Mon Jul 14 13:42:37 2014 -0500
> > +++ b/source/encoder/encoder.cpp      Tue Jul 15 10:44:08 2014 +0530
> > @@ -232,6 +232,19 @@
> >      }
> >  }
> >
> > +/** set default quantization matrix to array
> > +*/
> > +void Encoder::setDefaultScalingList()
> > +{
> > +    for (uint32_t sizeId = 0; sizeId < SCALING_LIST_SIZE_NUM; sizeId++)
> > +    {
> > +        for (uint32_t listId = 0; listId < g_scalingListNum[sizeId];
> listId++)
> > +        {
> > +            getScalingList()->processDefaultMarix(sizeId, listId);
> > +        }
> > +    }
> > +}
> > +
> >  /**
> >   \param   pic_in              input original YUV picture or NULL
> >   \param   pic_out             pointer to reconstructed picture struct
> > diff -r 51b3ff5e88e2 -r 52aa7ca073cf source/encoder/encoder.h
> > --- a/source/encoder/encoder.h        Mon Jul 14 13:42:37 2014 -0500
> > +++ b/source/encoder/encoder.h        Tue Jul 15 10:44:08 2014 +0530
> > @@ -202,6 +202,8 @@
> >
> >      TComScalingList* getScalingList() { return &m_scalingList; }
> >
> > +    void  setDefaultScalingList();
> > +
> >      void setThreadPool(ThreadPool* p) { m_threadPool = p; }
> >
> >      void configure(x265_param *param);
> > _______________________________________________
> > x265-devel mailing list
> > x265-devel at videolan.org
> > https://mailman.videolan.org/listinfo/x265-devel
>
> --
> Steve Borho
> _______________________________________________
> x265-devel mailing list
> x265-devel at videolan.org
> https://mailman.videolan.org/listinfo/x265-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.videolan.org/pipermail/x265-devel/attachments/20140715/226cd6be/attachment-0001.html>