[vlc-devel] [PATCH] Fixed a crash caused by yadif deinterlacer on Windows XP

[Rafaël Carré]

Hello,

Le 12-01-08 00:52, Naohiro KORIYAMA a écrit :
> The old patch, I sent to vlc-devel ML yesterday, didn't help the crash
> on Windows XP 32bit.
> I was wrong that I tested with the module that was compiled with a
> different patch.
> 
> So I made a new patch and tested again
>  (before applying old patch, after applying old patch, after applying
> new patch):
> - Windows XP 32bit using XP mode on Windows 7 (win32)
>   NG->NG->OK
> - Windows 7 64bit (win32/win64)
>   OK->OK->OK
> - Mac OS X Lion (intel64)
>   OK->OK->OK
> - Ubuntu 11.10 32bit using VirtualBox (32)
>   OK->OK->OK
> - Ubuntu 11.10 64bit using VirtualBox (64)
>   OK->OK->OK
> 
> It's all OK with the new patch.
> But as jb mentioned on the ML few days ago, compiler warnings appear again
> on 32bit builds both Windows and Linux.

These warnings might be fixed by using uintptr_t instead of uint64_t.

Perhaps you noticed but this patch broke compile on OSX 32 bits and
FreeBSD 32 bits : http://buildbot.videolan.org/builders/

OSX says:

../../../modules/video_filter/deinterlace/yadif_template.h:245: error:
can't find a register in class 'GENERAL_REGS' while reloading 'asm'
../../../modules/video_filter/deinterlace/yadif_template.h:251: error:
can't find a register in class 'GENERAL_REGS' while reloading 'asm'
../../../modules/video_filter/deinterlace/yadif_template.h:245: error:
'asm' operand has impossible constraints
../../../modules/video_filter/deinterlace/yadif_template.h:251: error:
'asm' operand has impossible constraints

and FreeBSD:
cc1: error in backend: Ran out of registers during register allocation!
and other stuff.

I suggest reverting the patch and doing something like:

uint8_t tmp[5*16];
uint8_t *tmpA= (uint8_t*)(((uintptr_t)(tmp+15)) & ~15);
uint8_t *tmp0 = &tmpA[0]; uint8_t *tmp1 = &tmpA[16] ...

what do you think?
It would use the old version of the code and ensure that tmp* are aligned.

Btw any idea why alignement doesn't work? Is it only for static data and
not for stack variables?

Other functions use (and rely on?) alignement in VLC so it should be
important to know what's going on there