[x264-devel] crash in latest version

[David Sullivan]

It crashes with the patch applied with the following parameters.
c:\mobile_cif.y4m --pass 1 --bitrate 100 --stats "mobile.stats" --ref 8
--mixed-refs --bframes 3
--b-adapt 2 --b-pyramid --weightb --direct auto --filter -1:-1 --subme 8
--trellis 2 --partitions p8x8,b8x8,i4x4,i8x8 --8x8dct
--me umh --threads auto --thread-input --progress --no-psnr --no-ssim -o
c:\mob.mkv

The crash happens always in ssd_plane function  after the patch is applied
but this time the function ssd_plane crashes
when it' is called from x264_rd_cost_i8x8 function and not from
x264_rd_cost_part.

It seems that the patch is useful only if the call to ssd_plane comes from
the function ssd_mb.
In this case the variable named "size" ( static inline int ssd_plane( x264_t
*h, int size, int p, int x, int y )  )
has always (in my case) the value 0 that is h->pixf.hadamard_ac[size] =
0x0045db60 _x264_pixel_hadamard_ac_16x16_mmxext
and no crash appens.

If the call comes from x264_rd_cost_part function, the variable named "size"
is set to 3 and the corresponding value of
h->pixf.hadamard_ac[size] = 0x0045db60 _x264_pixel_hadamard_ac_16x16_ssse3
and so it crashes.

I see more.

Tnx
David



2008/10/21 Gabriel Bouvigne <gabriel.bouvigne at joost.com>

> David Sullivan a écrit :
>
> > 2) If I try a compression in two passes, (c:\mobile_cif.y4m --pass 1
> > --bitrate 100 --stats "mobile.stats" --ref 8 --mixed-refs --bframes 3
> > --b-adapt 2 --b-pyramid --weightb --direct auto --filter -1:-1 --subme
> > 8                                                              --trellis
> > 2 --partitions p8x8,b8x8,i4x4,i8x8 --8x8dct --me umh --threads auto
> > --thread-input --progress --no-psnr --no-ssim -o c:\mob.mkv)
> >
> > on the first pass (frame 12) and the first time the execution reaches
> > the instruction uint64_t acs = h->pixf.hadamard_ac[size]( fdec,
> > FDEC_STRIDE ); , it crashes.
>
> I'm not sure about what you meant there: does is still crashes with the
> patch applied, or does it crashes only without the patch?
>
> --
> Gabriel
> _______________________________________________
> x264-devel mailing list
> x264-devel at videolan.org
> http://mailman.videolan.org/listinfo/x264-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.videolan.org/pipermail/x264-devel/attachments/20081021/96b841c1/attachment.htm