[x265] [PATCH] fix assert crash

[Steve Borho]

On Thu, Oct 24, 2013 at 7:11 AM, Min Chen <chenm003 at 163.com> wrote:

> # HG changeset patch
> # User Min Chen <chenm003 at 163.com>
> # Date 1382616670 -28800
> # Node ID 31485d1c19051ac633bda3298f8b9c2dc762d928
> # Parent  8430c3486b416687a25e37c60090a2358ceec47e
> fix assert crash
>

Folded into the patch which introduced the asserts


>
> diff -r 8430c3486b41 -r 31485d1c1905 source/Lib/TLibCommon/TComTrQuant.cpp
> --- a/source/Lib/TLibCommon/TComTrQuant.cpp     Thu Oct 24 19:55:58 2013
> +0800
> +++ b/source/Lib/TLibCommon/TComTrQuant.cpp     Thu Oct 24 20:11:10 2013
> +0800
> @@ -1032,8 +1032,11 @@
>      const int size = (1 << log2BlockSize);
>      const UInt* sigPos = &sigCoeffGroupFlag[(posYCG << log2BlockSize) +
> posXCG];
>
> -    assert(sigPos[1] <= 1);
> -    assert(sigPos[size] <= 1);
> +    if (posXCG < size - 1)
> +        assert(sigPos[1] <= 1);
> +
> +    if (posYCG < size - 1)
> +        assert(sigPos[size] <= 1);
>
>      UInt sigRight = (sigPos[1]);
>      UInt sigLower = (sigPos[size]);
> @@ -1358,8 +1361,11 @@
>      const int size = (1 << log2BlockSize);
>      const UInt* sigPos = &sigCoeffGroupFlag[(cgPosY << log2BlockSize) +
> cgPosX];
>
> -    assert(sigPos[1] <= 1);
> -    assert(sigPos[size] <= 1);
> +    if (cgPosX < size - 1)
> +        assert(sigPos[1] <= 1);
> +
> +    if (cgPosY < size - 1)
> +        assert(sigPos[size] <= 1);
>
>      UInt sigRight = (cgPosX == size - 1) ? 0 : (sigPos[1]);
>      UInt sigLower = (cgPosY == size - 1) ? 0 : (sigPos[size]);
>
> _______________________________________________
> x265-devel mailing list
> x265-devel at videolan.org
> https://mailman.videolan.org/listinfo/x265-devel
>



-- 
Steve Borho
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.videolan.org/pipermail/x265-devel/attachments/20131024/ae5f8c81/attachment.html>