[x265] [PATCH] asm : saturation bug fix for luma_vss asm routine

[chen]

>@@ -5105,8 +5108,9 @@
>     pmaddwd    m5, [r6 + 3 * 16]
>     paddd      m1, m5                          ;m1=[1+2+3+4+5+6+7+8]  Row2 end
>     psrad      m1, 6
>-
>-    packssdw   m0, m1
>+    pand       m1, m7
>+
>+    packusdw   m0, m1
> 
>     movlps     [r2], m0
>     movhps     [r2 + r3], m0

PAND + PACKUSDW may avoid overflow problem, but it is wrong way here
as you said, you got a result value 0x8D84, it is overflow on 16bits, so we need to find really reason,
I check the HM code, it use Short, so I suggest you catch input data and put into HM to check HM's output.
 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.videolan.org/pipermail/x265-devel/attachments/20140129/ef4c03f9/attachment.html>