[x265] [BUG] Failed CHECK at calcRdSADCost when using SSE2

den c dncodr at gmail.com
Wed May 21 01:11:32 CEST 2014 

>Satoshi Nakagawa
>emms ?

>Steve Borho
>The fact that it goes away when SSE2 is disabled points at a simple
EMMS problem; the check macro uses floats to check for integer
overflow.

Get it. Thanks.

On 5/18/14, Steve Borho <steve at borho.org> wrote:
> On Sat, May 17, 2014 at 12:50 PM, den c <dncodr at gmail.com> wrote:
>>  --asm=0x017002F works
>>  --asm=0x017001F fails
>> #define X265_CPU_SSE2            0x0000010
>
>
> Thanks, I was looking at this a couple of days ago, but then had to
> jump onto a plane and didn't get to push the fixes I had, and I hadn't
> figured out why these check macros were triggering.
>
> The fact that it goes away when SSE2 is disabled points at a simple
> EMMS problem; the check macro uses floats to check for integer
> overflow.
>
>> x265_6872_8.exe --crf 26.4  --y4m --preset=veryslow  --asm=0x017001F
>> --keyint 400 --scenecut 23  --frame-threads 1  --b-adapt 2 --weightp
>> --rc-lookahead 60 --bframes 6 --ref 4 --aq-mode 2 --aq-strength 1.0
>> --rd 6 --subme 4 --max-merge=4 --no-tskip  --ssim --psnr --csv
>> 1_26.4.csv --output 1_26.4_6872_8.265
>> x265 [info]: HEVC encoder version +-
>> x265 [info]: build info [Windows][GCC 4.8.2][32 bit][CHECKED] 8bpp
>> x265 [info]: using cpu capabilities: MMX2 SSE2 Cache32 Cache64
>> x265 [info]: WPP streams / pool / frames         : 6 / 2 / 1
>> x265 [info]: Main profile, Level-2.1 (Main tier)
>> x265 [info]: CU size                             : 64
>> x265 [info]: Max RQT depth inter / intra         : 3 / 3
>> x265 [info]: ME / range / subpel / merge         : star / 57 / 4 / 4
>> x265 [info]: Keyframe min / max / scenecut       : 25 / 400 / 23
>> x265 [info]: Lookahead / bframes / badapt        : 60 / 6 / 2
>> x265 [info]: b-pyramid / weightp / weightb / refs: 1 / 1 / 0 / 4
>> x265 [info]: Rate Control / AQ-Strength / CUTree : CRF-26.4 / 1.0 / 1
>> x265 [info]: tools: rect amp rd=6 psyrd=0.0 lft sao-lcu signhide
>> x265 [error]: calcRdSADCost wrap detected dist: 958, bits 1, lambda: 1290
>> x265 [error]: calcRdSADCost wrap detected dist: 676, bits 1, lambda: 1290
>> x265 [error]: calcRdSADCost wrap detected dist: 196, bits 2, lambda: 1290
>> x265 [error]: calcRdSADCost wrap detected dist: 394, bits 2, lambda: 1290
>> x265 [error]: calcRdSADCost wrap detected dist: 16, bits 2, lambda: 1290
>> x265 [error]: calcRdSADCost wrap detected dist: 22, bits 2, lambda: 1290
>> x265 [error]: calcRdSADCost wrap detected dist: 28, bits 6, lambda: 1290
>> x265 [error]: calcRdSADCost wrap detected dist: 30, bits 6, lambda: 1290
>> x265 [error]: calcRdSADCost wrap detected dist: 206, bits 2, lambda: 1290
>> x265 [error]: calcRdSADCost wrap detected dist: 399, bits 6, lambda: 1290
>> x265 [error]: calcRdSADCost wrap detected dist: 198, bits 2, lambda: 1290
>> x265 [error]: calcRdSADCost wrap detected dist: 408, bits 6, lambda: 1290
>> x265 [error]: calcRdSADCost wrap detected dist: 24, bits 7, lambda: 1290
>> x265 [error]: calcRdSADCost wrap detected dist: 22, bits 7, lambda: 1290
>> x265 [error]: calcRdSADCost wrap detected dist: 25, bits 7, lambda: 1290
>> x265 [error]: calcRdSADCost wrap detected dist: 16, bits 7, lambda: 1290
>> x265 [error]: calcRdSADCost wrap detected dist: 17, bits 3, lambda: 1290
>> x265 [error]: calcRdSADCost wrap detected dist: 14, bits 3, lambda: 1290
>> x265 [error]: calcRdSADCost wrap detected dist: 22, bits 7, lambda: 1290
>> x265 [error]: calcRdSADCost wrap detected dist: 14, bits 7, lambda: 1290
>> x265 [error]: calcRdSADCost wrap detected dist: 8, bits 3, lambda: 1290
>> x265 [error]: calcRdSADCost wrap detected dist: 14, bits 3, lambda: 1290
>> x265 [error]: calcRdSADCost wrap detected dist: 16, bits 7, lambda: 1290
>> x265 [error]: calcRdSADCost wrap detected dist: 16, bits 7, lambda: 1290
>> x265 [error]: calcRdSADCost wrap detected dist: 16, bits 7, lambda: 1290
>> x265 [error]: calcRdSADCost wrap detected dist: 8, bits 7, lambda: 1290
>> x265 [error]: calcRdSADCost wrap detected dist: 23, bits 7, lambda: 1290
>> x265 [error]: calcRdSADCost wrap detected dist: 22, bits 7, lambda: 1290
>> x265 [error]: calcRdSADCost wrap detected dist: 12, bits 7, lambda: 1290
>> x265 [error]: calcRdSADCost wrap detected dist: 8, bits 7, lambda: 1290
>> x265 [error]: calcRdSADCost wrap detected dist: 15, bits 7, lambda: 1290
>> x265 [error]: calcRdSADCost wrap detected dist: 8, bits 7, lambda: 1290
>> x265 [error]: calcRdSADCost wrap detected dist: 210, bits 2, lambda: 1290
>> x265 [error]: calcRdSADCost wrap detected dist: 402, bits 6, lambda: 1290
>> x265 [error]: calcRdSADCost wrap detected dist: 204, bits 2, lambda: 1290
>> x265 [error]: calcRdSADCost wrap detected dist: 414, bits 6, lambda: 1290
>> x265 [error]: calcRdSADCost wrap detected dist: 22, bits 6, lambda: 1290
>> x265 [error]: calcRdSADCost wrap detected dist: 16, bits 6, lambda: 1290
>> x265 [error]: calcRdSADCost wrap detected dist: 22, bits 6, lambda: 1290
>> x265 [error]: calcRdSADCost wrap detected dist: 16, bits 6, lambda: 1290
>> x265 [error]: calcRdSADCost wrap detected dist: 204, bits 3, lambda: 1290
>> x265 [error]: calcRdSADCost wrap detected dist: 410, bits 7, lambda: 1290
>> x265 [error]: calcRdSADCost wrap detected dist: 194, bits 3, lambda: 1290
>> x265 [error]: calcRdSADCost wrap detected dist: 410, bits 7, lambda: 1290
>> x265 [error]: calcRdSADCost wrap detected dist: 26, bits 6, lambda: 1290
>> x265 [error]: calcRdSADCost wrap detected dist: 25, bits 6, lambda: 1290
>> x265 [error]: calcRdSADCost wrap detected dist: 26, bits 6, lambda: 1290
>> x265 [error]: calcRdSADCost wrap detected dist: 16, bits 6, lambda: 1290
>> [...]
>> x265 [error]: calcRdSADCost wrap detected dist: 30, bits 2, lambda: 1149
>> x265 [error]: calcRdSADCost wrap detected dist: 249, bits 2, lambda: 1149
>> x265 [error]: calcRdSADCost wrap detected dist: 31, bits 7, lambda: 1149
>> x265 [error]: calcRdSADCost wrap detected dist: 193, bits 2, lambda: 1149
>> x265 [error]: calcRdSADCost wrap detected dist: 146, bits 2, lambda: 1149
>> x265 [error]: calcRdSADCost wrap detected dist: 228, bits 2, lambda: 1149
>> x265 [error]: calcRdSADCost wrap detected dist: 150, bits 7, lambda: 1149
>> x265 [error]: calcRdSADCost wrap detected dist: 172, bits 7, lambda: 1149
>> x265 [error]: calcRdSADCost wrap detected dist: 31, bits 8, lambda: 1149
>> x265 [error]: calcRdSADCost wrap detected dist: 184, bits 8, lambda: 1149
>> x265 [error]: calcRdSADCost wrap detected dist: 14, bits 8, lambda: 1149
>> x265 [error]: calcRdSADCost wrap detected dist: 174, bits 3, lambda: 1149
>> x265 [error]: calcRdSADCost wrap detected dist: 127, bits 7, lambda: 1149
>> x265 [error]: calcRdSADCost wrap detected dist: 163, bits 7, lambda: 1149
>> x265 [error]: calcRdSADCost wrap detected dist: 128, bits 7, lambda: 1149
>> x265 [error]: calcRdSADCost wrap detected dist: 167, bits 7, lambda: 1149
>> x265 [error]: calcRdSADCost wrap detected dist: 323, bits 2, lambda: 1149
>> x265 [error]: calcRdSADCost wrap detected dist: 200, bits 2, lambda: 1149
>> x265 [error]: calcRdSADCost wrap detected dist: 231, bits 7, lambda: 1149
>> x265 [error]: calcRdSADCost wrap detected dist: 82, bits 2, lambda: 1149
>> x265 [error]: calcRdSADCost wrap detected dist: 446, bits 2, lambda: 1149
>> x265 [error]: calcRdSADCost wrap detected dist: 44, bits 1, lambda: 1149
>> x265 [error]: calcRdSADCost wrap detected dist: 213, bits 2, lambda: 1149
>> x265 [error]: calcRdSADCost wrap detected dist: 28, bits 2, lambda: 1149
>> x265 [error]: calcRdSADCost wrap detected dist: 192, bits 8, lambda: 1149
>> x265 [error]: calcRdSADCost wrap detected dist: 118, bits 2, lambda: 1149
>> x265 [error]: calcRdSADCost wrap detected dist: 216, bits 2, lambda: 1149
>> x265 [error]: calcRdSADCost wrap detected dist: 66, bits 2, lambda: 1149
>> x265 [error]: calcRdSADCost wrap detected dist: 190, bits 2, lambda: 1149
>> x265 [error]: calcRdSADCost wrap detected dist: 22, bits 8, lambda: 1149
>> x265 [error]: calcRdSADCost wrap detected dist: 198, bits 8, lambda: 1149
>> x265 [error]: calcRdSADCost wrap detected dist: 43, bits 8, lambda: 1149
>> x265 [error]: calcRdSADCost wrap detected dist: 30, bits 7, lambda: 1149
>> x265 [error]: calcRdSADCost wrap detected dist: 224, bits 7, lambda: 1149
>> x265 [error]: calcRdSADCost wrap detected dist: 36, bits 7, lambda: 1149
>> x265 [error]: calcRdSADCost wrap detected dist: 240, bits 7, lambda: 1149
>> x265 [error]: calcRdSADCost wrap detected dist: 169, bits 8, lambda: 1149
>> x265 [error]: calcRdSADCost wrap detected dist: 204, bits 8, lambda: 1149
>> x265 [error]: calcRdSADCost wrap detected dist: 246, bits 8, lambda: 1149
>> x265 [error]: calcRdSADCost wrap detected dist: 136, bits 3, lambda: 1149
>> x265 [error]: calcRdSADCost wrap detected dist: 84, bits 2, lambda: 1149
>> x265 [error]: calcRdSADCost wrap detected dist: 458, bits 8, lambda: 1149
>> x265 [error]: calcRdSADCost wrap detected dist: 288, bits 2, lambda: 1149
>> x265 [error]: calcRdSADCost wrap detected dist: 293, bits 3, lambda: 1149
>> x265 [error]: calcRdSADCost wrap detected dist: 264, bits 2, lambda: 1149
>> x265 [error]: calcRdSADCost wrap detected dist: 296, bits 1, lambda: 1149
>> x265 [error]: calcRdSADCost wrap detected dist: 164, bits 2, lambda: 1149
>> x265 [error]: calcRdSADCost wrap detected dist: 62, bits 2, lambda: 1149
>> [.....]
>> It seems logger sync is b0rken, at some point it starts print crazy
>> mixed output:
>>
>> x265 [error]: calcRdSADxCost2 w6rap d5etec ted d[ist:e r17, brits 3o,
>> lamrbd]a::912 calcRdSADCost wrap detected dist: 221, bits 1, lambda:
>> 1290
>> x265 [error]: calcRdSADCxo265 [error]: calcRdSADCosst wrap detected
>> dist: 412,t wrapb deteits 2, lamctebdd ad:i s1t2:9 020, bits 3,
>> lambda: 912
>> x265 [error]: calcx265R [errdor]:S AcalcDRdSACDCosto wraps dettecte d
>> diswt: 12r, baipts 6,  lambdda: 9e12tected dist: 225, bits 2, lambda:
>> 1290
>> x265 [error]: calcRdSADCost wrap detected dist: 12, bxits 32,
>> lam6bda:5 912 [error]: calcRdSADCost wrap detected dist: 399, bits 6,
>> lambda: 1x2652 [err9o0r]: calcRdSADCost wrap detected dist: 14, bits
>> 3, lambda: 912
>> x265 [error]: calcRdSADCost wrap detected x265d [eriror]:s tcalcR:dSAD
>> Cos1t wra5p det,ected  dibst: 8i, bitts s2, la m3bda: ,912 lambda:
>> 1290
>> x265 [error]: calcRdSAxDC2ost w6rap d5etec ted [dist: 8, bits 3,
>> elambda: 912rror]: calcRdSADCost wrap detected dist: 14, bits 3,
>> lamx2b6d5a :[ e1290rxror]2: 6calc5RdSA
>> _______________________________________________
>> x265-devel mailing list
>> x265-devel at videolan.org
>> https://mailman.videolan.org/listinfo/x265-devel
>
>
>
> --
> Steve Borho
> _______________________________________________
> x265-devel mailing list
> x265-devel at videolan.org
> https://mailman.videolan.org/listinfo/x265-devel
>

More information about the x265-devel mailing list