[x265] [PATCH] frame: initialize recon to avoid SAO read of uninitialized pixels beyond picture end

[Steve Borho]

On Wed, Jul 2, 2014 at 7:08 AM,  <kavitha at multicorewareinc.com> wrote:
> # HG changeset patch
> # User Kavitha Sampath <kavitha at multicorewareinc.com>
> # Date 1404297383 -19800
> #      Wed Jul 02 16:06:23 2014 +0530
> # Node ID 731068beca7e5e1403c39c5ede610101b3e6e69f
> # Parent  a18972fd05b1d6242a881bef979b9e1ff17543d9
> frame: initialize recon to avoid SAO read of uninitialized pixels beyond picture end
>
> diff -r a18972fd05b1 -r 731068beca7e source/common/frame.cpp
> --- a/source/common/frame.cpp   Tue Jul 01 14:58:35 2014 -0500
> +++ b/source/common/frame.cpp   Wed Jul 02 16:06:23 2014 +0530
> @@ -106,14 +106,24 @@
>  {
>      m_picSym = new TComPicSym;
>      m_reconPicYuv = new TComPicYuv;
> +    bool picCreate = false, reconCreate = false;

I don't like this style of declaring variables up here and giving them
initial values, and then unconditionally re-assigning them later.  It
adds needless work to the function, and doesn't make it any easier to
read.

>      if (m_picSym && m_reconPicYuv)
>      {
>          m_picSym->m_reconPicYuv = m_reconPicYuv;
> -        return m_picSym->create(param) &&
> -          m_reconPicYuv->create(param->sourceWidth, param->sourceHeight, param->internalCsp, g_maxCUSize, g_maxCUDepth);
> +        picCreate = m_picSym->create(param);
> +        reconCreate = m_reconPicYuv->create(param->sourceWidth, param->sourceHeight,
> +                                            param->internalCsp, g_maxCUSize, g_maxCUDepth);
> +

if reconCreate was false, there is no point in doing any memsets

> +        // initialize m_reconpicYuv as SAO may read beyond the end of the picture accessing uninitialized pixels
> +        int maxHeight = m_reconPicYuv->m_numCuInHeight * g_maxCUSize;
> +        memset(m_reconPicYuv->m_picBuf[0], 0, m_reconPicYuv->m_stride * (maxHeight +
> +               (m_reconPicYuv->m_lumaMarginY * 2)));
> +        memset(m_reconPicYuv->m_picBuf[1], 0, m_reconPicYuv->m_strideC * ((maxHeight >>
> +               m_reconPicYuv->m_vChromaShift) + (m_reconPicYuv->m_chromaMarginY * 2)));
> +        memset(m_reconPicYuv->m_picBuf[2], 0, m_reconPicYuv->m_strideC * ((maxHeight >>
> +               m_reconPicYuv->m_vChromaShift) + (m_reconPicYuv->m_chromaMarginY * 2)));
>      }
> -    else
> -        return false;
> +    return picCreate && reconCreate;
>  }
>
>  void Frame::reinit(x265_param *param)
> _______________________________________________
> x265-devel mailing list
> x265-devel at videolan.org
> https://mailman.videolan.org/listinfo/x265-devel



-- 
Steve Borho