[x264-devel] crash in latest version

[David Sullivan]

Hi,
I tried the two patches posted by Gabriel and now x264 r999 works fine but
in slow_mod for my Intel Core 2 Extreme X9770 supporting the following
instructions: MMX SSE SSE2 SSE3 SSSE3 SSE4.1 EM64T .
So subme crash came from that.

The problem was something I can't explain:
i had to add brackets to the ifs of the patch *hadamar_slowmod4.patch*

+        if (!(cpu&X264_CPU_STACK_MOD4))
+        {
+            INIT4( hadamard_ac, _ssse3 );
+        }

and this

+        if (!(cpu&X264_CPU_STACK_MOD4))
+        {
+            INIT4( hadamard_ac, _sse2 );
+        }

Don't ask me why... I know it sound crazy.
David

2008/10/24 Gabriel Bouvigne <gabriel.bouvigne at joost.com>

> David Sullivan a écrit :
>
> > It seems that the patch is useful only if the call to ssd_plane comes
> from
> > the function ssd_mb.
> > In this case the variable named "size" ( static inline int ssd_plane(
> x264_t
> > *h, int size, int p, int x, int y )  )
> > has always (in my case) the value 0 that is h->pixf.hadamard_ac[size] =
> > 0x0045db60 _x264_pixel_hadamard_ac_16x16_mmxext
> > and no crash appens.
> >
> > If the call comes from x264_rd_cost_part function, the variable named
> "size"
> > is set to 3 and the corresponding value of
> > h->pixf.hadamard_ac[size] = 0x0045db60
> _x264_pixel_hadamard_ac_16x16_ssse3
> > and so it crashes.
>
> Which imply that there is likely a compilation issue on your side, as
> the patch is disabling SSE2/3 versions of hadamard_ac.
>
> I tested on my side, and to me hadamard_slowmod4 patch and updated msvc
> workspace are all that are necessary to have trunk version working on an
> SSE2 processor when compiling with msvc.
>
>
> --
> Gabriel
> _______________________________________________
> x264-devel mailing list
> x264-devel at videolan.org
> http://mailman.videolan.org/listinfo/x264-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.videolan.org/pipermail/x264-devel/attachments/20081024/04d28f41/attachment.htm