[x265] [PATCH] count_nonzero primitive, downscaling quantCoeff from int32_t* to int16_t*

[Praveen Tiwari]

Thanks, just sent a fix for it.


Regards,
Praveen


On Tue, Aug 12, 2014 at 7:18 PM, chen <chenm003 at 163.com> wrote:

> >-        X265_CHECK((int)numSig == primitives.count_nonzero(coeff, 1 << log2TrSize * 2), "numSig differ\n");
> >+        /* This section of code is to safely convert int32_t coefficients to int16_t, once the caller function is
> >+         * optimize to take coefficients as int16_t*, it will be cleanse.*/
> >+        int numCoeff = (1 << (log2TrSize * 2));
> >+        assert(numCoeff <= 1024);
> >+        ALIGN_VAR_16(int16_t, qCoeff[32 * 32]);
> >+        for (int i = 0; i < numCoeff; i++)
> >+        {
> >+            qCoeff[i] = (
> coeff[i] & 0xFFFF);
> >+        }
> I suggest use clip on it, to avoid value problem (eg: 0x10000 become zero) and asm instruction match to clip
>
>
> _______________________________________________
> x265-devel mailing list
> x265-devel at videolan.org
> https://mailman.videolan.org/listinfo/x265-devel
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.videolan.org/pipermail/x265-devel/attachments/20140813/306f62ad/attachment-0001.html>