[vlc-devel] [PATCH] Fixed a crash caused by yadif deinterlacer on Windows XP

Rafaël Carré funman at videolan.org
Tue Jan 10 03:04:38 CET 2012

Le 2012-01-09 08:10, Naohiro KORIYAMA a écrit :
> Today, I missed so often.
>>> I suggest reverting the patch and doing something like:
>>> uint8_t tmp[5*16];
>>> uint8_t *tmpA= (uint8_t*)(((uintptr_t)(tmp+15)) & ~15);
>>> uint8_t *tmp0 = &tmpA[0]; uint8_t *tmp1 = &tmpA[16] ...
>>> what do you think?
>>> It would use the old version of the code and ensure that tmp* are aligned.
>> I also think the last patch must be reverted.
>> I tried your suggestion and linux 32/64 bit build crashed when
>> enabling yadif, another builds, win32/64 osx64,  seemed to work.
> not only linux 32/64 bit build but also win32/64 osx64 crashed.

I don't know why but maybe my patch is just bad.

I have another idea for the original win32-only crash though:

Looking at gcc docs:

Realign the stack at entry. On the Intel x86, the -mstackrealign option
will generate an alternate prologue and epilogue that realigns the
runtime stack if necessary. This supports mixing legacy codes that keep
a 4-byte aligned stack with modern codes that keep a 16-byte stack for
SSE compatibility. See also the attribute force_align_arg_pointer,
applicable to individual functions.

Since we do 'generic' (compatible with pentium-1 ?) builds on win32,
there should be no stack alignement requirement, afaik.

But we build these functions (and only them) with SSE enabled.
=> Is there a mismatch somewhere? GCC assuming that stack is aligned on
16bytes and not issuing realignement code?

Does a build with CFLAGS='-msse -mmmx' work?

If it does, that maybe a good occasion to enable MMX and SSE
optimizations in VLC by default and wish "good bye, use VLC 1.1 if you
want" to these old CPUs.

More information about the vlc-devel mailing list