[x265] [PATCH] psyrd: fix for inconsistent output

[Deepthi Nandakumar]

Thanks, good catch.


On Fri, Jun 27, 2014 at 2:16 PM, <sumalatha at multicorewareinc.com> wrote:

> # HG changeset patch
> # User Sumalatha Polureddy<sumalatha at multicorewareinc.com>
> # Date 1403858781 -19800
> # Node ID a789870889fcc9a31deff7fc6961d143b0db86c1
> # Parent  1b669c33ff3a8d8f6c9bd1e18979c009baed2433
> psyrd: fix for inconsistent output
>
> maximum buffer size for zeropel is MAX_CU_SIZExMAX_CU_SIZE. since stride
> was wrong,
> it was accessing out of boundary memory which was different for each run,
> so inconsistent output
>
> diff -r 1b669c33ff3a -r a789870889fc source/Lib/TLibEncoder/TEncSearch.cpp
> --- a/source/Lib/TLibEncoder/TEncSearch.cpp     Wed Jun 25 22:46:45 2014
> +0530
> +++ b/source/Lib/TLibEncoder/TEncSearch.cpp     Fri Jun 27 14:16:21 2014
> +0530
> @@ -3035,7 +3035,7 @@
>          {
>              int size = g_convertToBit[trSize];
>              psyEnergyY = m_rdCost->psyCost(size,
> fencYuv->getLumaAddr(absPartIdx), fencYuv->getStride(),
> -               (pixel*)RDCost::zeroPel,
> cu->getPic()->getPicYuvRec()->getStride()); // need to check whether zero
> distortion is similar to psyenergy of fenc
> +                (pixel*)RDCost::zeroPel, trSize); // need to check
> whether zero distortion is similar to psyenergy of fenc
>          }
>          int16_t *curResiY =
> m_qtTempShortYuv[qtLayer].getLumaAddr(absPartIdx);
>          X265_CHECK(m_qtTempShortYuv[qtLayer].m_width == MAX_CU_SIZE,
> "width not full CU\n");
> _______________________________________________
> x265-devel mailing list
> x265-devel at videolan.org
> https://mailman.videolan.org/listinfo/x265-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.videolan.org/pipermail/x265-devel/attachments/20140627/298d63d8/attachment.html>