[x265] [BUG] Failed CHECK at calcRdSADCost when using SSE2
Steve Borho
steve at borho.org
Sun May 18 20:20:01 CEST 2014
On Sat, May 17, 2014 at 12:50 PM, den c <dncodr at gmail.com> wrote:
> --asm=0x017002F works
> --asm=0x017001F fails
> #define X265_CPU_SSE2 0x0000010
Thanks, I was looking at this a couple of days ago, but then had to
jump onto a plane and didn't get to push the fixes I had, and I hadn't
figured out why these check macros were triggering.
The fact that it goes away when SSE2 is disabled points at a simple
EMMS problem; the check macro uses floats to check for integer
overflow.
> x265_6872_8.exe --crf 26.4 --y4m --preset=veryslow --asm=0x017001F
> --keyint 400 --scenecut 23 --frame-threads 1 --b-adapt 2 --weightp
> --rc-lookahead 60 --bframes 6 --ref 4 --aq-mode 2 --aq-strength 1.0
> --rd 6 --subme 4 --max-merge=4 --no-tskip --ssim --psnr --csv
> 1_26.4.csv --output 1_26.4_6872_8.265
> x265 [info]: HEVC encoder version +-
> x265 [info]: build info [Windows][GCC 4.8.2][32 bit][CHECKED] 8bpp
> x265 [info]: using cpu capabilities: MMX2 SSE2 Cache32 Cache64
> x265 [info]: WPP streams / pool / frames : 6 / 2 / 1
> x265 [info]: Main profile, Level-2.1 (Main tier)
> x265 [info]: CU size : 64
> x265 [info]: Max RQT depth inter / intra : 3 / 3
> x265 [info]: ME / range / subpel / merge : star / 57 / 4 / 4
> x265 [info]: Keyframe min / max / scenecut : 25 / 400 / 23
> x265 [info]: Lookahead / bframes / badapt : 60 / 6 / 2
> x265 [info]: b-pyramid / weightp / weightb / refs: 1 / 1 / 0 / 4
> x265 [info]: Rate Control / AQ-Strength / CUTree : CRF-26.4 / 1.0 / 1
> x265 [info]: tools: rect amp rd=6 psyrd=0.0 lft sao-lcu signhide
> x265 [error]: calcRdSADCost wrap detected dist: 958, bits 1, lambda: 1290
> x265 [error]: calcRdSADCost wrap detected dist: 676, bits 1, lambda: 1290
> x265 [error]: calcRdSADCost wrap detected dist: 196, bits 2, lambda: 1290
> x265 [error]: calcRdSADCost wrap detected dist: 394, bits 2, lambda: 1290
> x265 [error]: calcRdSADCost wrap detected dist: 16, bits 2, lambda: 1290
> x265 [error]: calcRdSADCost wrap detected dist: 22, bits 2, lambda: 1290
> x265 [error]: calcRdSADCost wrap detected dist: 28, bits 6, lambda: 1290
> x265 [error]: calcRdSADCost wrap detected dist: 30, bits 6, lambda: 1290
> x265 [error]: calcRdSADCost wrap detected dist: 206, bits 2, lambda: 1290
> x265 [error]: calcRdSADCost wrap detected dist: 399, bits 6, lambda: 1290
> x265 [error]: calcRdSADCost wrap detected dist: 198, bits 2, lambda: 1290
> x265 [error]: calcRdSADCost wrap detected dist: 408, bits 6, lambda: 1290
> x265 [error]: calcRdSADCost wrap detected dist: 24, bits 7, lambda: 1290
> x265 [error]: calcRdSADCost wrap detected dist: 22, bits 7, lambda: 1290
> x265 [error]: calcRdSADCost wrap detected dist: 25, bits 7, lambda: 1290
> x265 [error]: calcRdSADCost wrap detected dist: 16, bits 7, lambda: 1290
> x265 [error]: calcRdSADCost wrap detected dist: 17, bits 3, lambda: 1290
> x265 [error]: calcRdSADCost wrap detected dist: 14, bits 3, lambda: 1290
> x265 [error]: calcRdSADCost wrap detected dist: 22, bits 7, lambda: 1290
> x265 [error]: calcRdSADCost wrap detected dist: 14, bits 7, lambda: 1290
> x265 [error]: calcRdSADCost wrap detected dist: 8, bits 3, lambda: 1290
> x265 [error]: calcRdSADCost wrap detected dist: 14, bits 3, lambda: 1290
> x265 [error]: calcRdSADCost wrap detected dist: 16, bits 7, lambda: 1290
> x265 [error]: calcRdSADCost wrap detected dist: 16, bits 7, lambda: 1290
> x265 [error]: calcRdSADCost wrap detected dist: 16, bits 7, lambda: 1290
> x265 [error]: calcRdSADCost wrap detected dist: 8, bits 7, lambda: 1290
> x265 [error]: calcRdSADCost wrap detected dist: 23, bits 7, lambda: 1290
> x265 [error]: calcRdSADCost wrap detected dist: 22, bits 7, lambda: 1290
> x265 [error]: calcRdSADCost wrap detected dist: 12, bits 7, lambda: 1290
> x265 [error]: calcRdSADCost wrap detected dist: 8, bits 7, lambda: 1290
> x265 [error]: calcRdSADCost wrap detected dist: 15, bits 7, lambda: 1290
> x265 [error]: calcRdSADCost wrap detected dist: 8, bits 7, lambda: 1290
> x265 [error]: calcRdSADCost wrap detected dist: 210, bits 2, lambda: 1290
> x265 [error]: calcRdSADCost wrap detected dist: 402, bits 6, lambda: 1290
> x265 [error]: calcRdSADCost wrap detected dist: 204, bits 2, lambda: 1290
> x265 [error]: calcRdSADCost wrap detected dist: 414, bits 6, lambda: 1290
> x265 [error]: calcRdSADCost wrap detected dist: 22, bits 6, lambda: 1290
> x265 [error]: calcRdSADCost wrap detected dist: 16, bits 6, lambda: 1290
> x265 [error]: calcRdSADCost wrap detected dist: 22, bits 6, lambda: 1290
> x265 [error]: calcRdSADCost wrap detected dist: 16, bits 6, lambda: 1290
> x265 [error]: calcRdSADCost wrap detected dist: 204, bits 3, lambda: 1290
> x265 [error]: calcRdSADCost wrap detected dist: 410, bits 7, lambda: 1290
> x265 [error]: calcRdSADCost wrap detected dist: 194, bits 3, lambda: 1290
> x265 [error]: calcRdSADCost wrap detected dist: 410, bits 7, lambda: 1290
> x265 [error]: calcRdSADCost wrap detected dist: 26, bits 6, lambda: 1290
> x265 [error]: calcRdSADCost wrap detected dist: 25, bits 6, lambda: 1290
> x265 [error]: calcRdSADCost wrap detected dist: 26, bits 6, lambda: 1290
> x265 [error]: calcRdSADCost wrap detected dist: 16, bits 6, lambda: 1290
> [...]
> x265 [error]: calcRdSADCost wrap detected dist: 30, bits 2, lambda: 1149
> x265 [error]: calcRdSADCost wrap detected dist: 249, bits 2, lambda: 1149
> x265 [error]: calcRdSADCost wrap detected dist: 31, bits 7, lambda: 1149
> x265 [error]: calcRdSADCost wrap detected dist: 193, bits 2, lambda: 1149
> x265 [error]: calcRdSADCost wrap detected dist: 146, bits 2, lambda: 1149
> x265 [error]: calcRdSADCost wrap detected dist: 228, bits 2, lambda: 1149
> x265 [error]: calcRdSADCost wrap detected dist: 150, bits 7, lambda: 1149
> x265 [error]: calcRdSADCost wrap detected dist: 172, bits 7, lambda: 1149
> x265 [error]: calcRdSADCost wrap detected dist: 31, bits 8, lambda: 1149
> x265 [error]: calcRdSADCost wrap detected dist: 184, bits 8, lambda: 1149
> x265 [error]: calcRdSADCost wrap detected dist: 14, bits 8, lambda: 1149
> x265 [error]: calcRdSADCost wrap detected dist: 174, bits 3, lambda: 1149
> x265 [error]: calcRdSADCost wrap detected dist: 127, bits 7, lambda: 1149
> x265 [error]: calcRdSADCost wrap detected dist: 163, bits 7, lambda: 1149
> x265 [error]: calcRdSADCost wrap detected dist: 128, bits 7, lambda: 1149
> x265 [error]: calcRdSADCost wrap detected dist: 167, bits 7, lambda: 1149
> x265 [error]: calcRdSADCost wrap detected dist: 323, bits 2, lambda: 1149
> x265 [error]: calcRdSADCost wrap detected dist: 200, bits 2, lambda: 1149
> x265 [error]: calcRdSADCost wrap detected dist: 231, bits 7, lambda: 1149
> x265 [error]: calcRdSADCost wrap detected dist: 82, bits 2, lambda: 1149
> x265 [error]: calcRdSADCost wrap detected dist: 446, bits 2, lambda: 1149
> x265 [error]: calcRdSADCost wrap detected dist: 44, bits 1, lambda: 1149
> x265 [error]: calcRdSADCost wrap detected dist: 213, bits 2, lambda: 1149
> x265 [error]: calcRdSADCost wrap detected dist: 28, bits 2, lambda: 1149
> x265 [error]: calcRdSADCost wrap detected dist: 192, bits 8, lambda: 1149
> x265 [error]: calcRdSADCost wrap detected dist: 118, bits 2, lambda: 1149
> x265 [error]: calcRdSADCost wrap detected dist: 216, bits 2, lambda: 1149
> x265 [error]: calcRdSADCost wrap detected dist: 66, bits 2, lambda: 1149
> x265 [error]: calcRdSADCost wrap detected dist: 190, bits 2, lambda: 1149
> x265 [error]: calcRdSADCost wrap detected dist: 22, bits 8, lambda: 1149
> x265 [error]: calcRdSADCost wrap detected dist: 198, bits 8, lambda: 1149
> x265 [error]: calcRdSADCost wrap detected dist: 43, bits 8, lambda: 1149
> x265 [error]: calcRdSADCost wrap detected dist: 30, bits 7, lambda: 1149
> x265 [error]: calcRdSADCost wrap detected dist: 224, bits 7, lambda: 1149
> x265 [error]: calcRdSADCost wrap detected dist: 36, bits 7, lambda: 1149
> x265 [error]: calcRdSADCost wrap detected dist: 240, bits 7, lambda: 1149
> x265 [error]: calcRdSADCost wrap detected dist: 169, bits 8, lambda: 1149
> x265 [error]: calcRdSADCost wrap detected dist: 204, bits 8, lambda: 1149
> x265 [error]: calcRdSADCost wrap detected dist: 246, bits 8, lambda: 1149
> x265 [error]: calcRdSADCost wrap detected dist: 136, bits 3, lambda: 1149
> x265 [error]: calcRdSADCost wrap detected dist: 84, bits 2, lambda: 1149
> x265 [error]: calcRdSADCost wrap detected dist: 458, bits 8, lambda: 1149
> x265 [error]: calcRdSADCost wrap detected dist: 288, bits 2, lambda: 1149
> x265 [error]: calcRdSADCost wrap detected dist: 293, bits 3, lambda: 1149
> x265 [error]: calcRdSADCost wrap detected dist: 264, bits 2, lambda: 1149
> x265 [error]: calcRdSADCost wrap detected dist: 296, bits 1, lambda: 1149
> x265 [error]: calcRdSADCost wrap detected dist: 164, bits 2, lambda: 1149
> x265 [error]: calcRdSADCost wrap detected dist: 62, bits 2, lambda: 1149
> [.....]
> It seems logger sync is b0rken, at some point it starts print crazy
> mixed output:
>
> x265 [error]: calcRdSADxCost2 w6rap d5etec ted d[ist:e r17, brits 3o,
> lamrbd]a::912 calcRdSADCost wrap detected dist: 221, bits 1, lambda:
> 1290
> x265 [error]: calcRdSADCxo265 [error]: calcRdSADCosst wrap detected
> dist: 412,t wrapb deteits 2, lamctebdd ad:i s1t2:9 020, bits 3,
> lambda: 912
> x265 [error]: calcx265R [errdor]:S AcalcDRdSACDCosto wraps dettecte d
> diswt: 12r, baipts 6, lambdda: 9e12tected dist: 225, bits 2, lambda:
> 1290
> x265 [error]: calcRdSADCost wrap detected dist: 12, bxits 32,
> lam6bda:5 912 [error]: calcRdSADCost wrap detected dist: 399, bits 6,
> lambda: 1x2652 [err9o0r]: calcRdSADCost wrap detected dist: 14, bits
> 3, lambda: 912
> x265 [error]: calcRdSADCost wrap detected x265d [eriror]:s tcalcR:dSAD
> Cos1t wra5p det,ected dibst: 8i, bitts s2, la m3bda: ,912 lambda:
> 1290
> x265 [error]: calcRdSAxDC2ost w6rap d5etec ted [dist: 8, bits 3,
> elambda: 912rror]: calcRdSADCost wrap detected dist: 14, bits 3,
> lamx2b6d5a :[ e1290rxror]2: 6calc5RdSA
> _______________________________________________
> x265-devel mailing list
> x265-devel at videolan.org
> https://mailman.videolan.org/listinfo/x265-devel
--
Steve Borho
More information about the x265-devel
mailing list