[vlc-devel] [PATCH] Fixed a crash caused by yadif deinterlacer on Windows XP

[Naohiro KORIYAMA]

Hello,

2012/1/9 Rafaël Carré <funman at videolan.org>:
> Hello,
>
> Le 12-01-08 00:52, Naohiro KORIYAMA a écrit :
>> The old patch, I sent to vlc-devel ML yesterday, didn't help the crash
>> on Windows XP 32bit.
>> I was wrong that I tested with the module that was compiled with a
>> different patch.
>>
>> So I made a new patch and tested again
>>  (before applying old patch, after applying old patch, after applying
>> new patch):
>> - Windows XP 32bit using XP mode on Windows 7 (win32)
>>   NG->NG->OK
>> - Windows 7 64bit (win32/win64)
>>   OK->OK->OK
>> - Mac OS X Lion (intel64)
>>   OK->OK->OK
>> - Ubuntu 11.10 32bit using VirtualBox (32)
>>   OK->OK->OK
>> - Ubuntu 11.10 64bit using VirtualBox (64)
>>   OK->OK->OK
>>
>> It's all OK with the new patch.
>> But as jb mentioned on the ML few days ago, compiler warnings appear again
>> on 32bit builds both Windows and Linux.
>
> These warnings might be fixed by using uintptr_t instead of uint64_t.
>
> Perhaps you noticed but this patch broke compile on OSX 32 bits and
> FreeBSD 32 bits : http://buildbot.videolan.org/builders/
>
> OSX says:
>
> ../../../modules/video_filter/deinterlace/yadif_template.h:245: error:
> can't find a register in class 'GENERAL_REGS' while reloading 'asm'
> ../../../modules/video_filter/deinterlace/yadif_template.h:251: error:
> can't find a register in class 'GENERAL_REGS' while reloading 'asm'
> ../../../modules/video_filter/deinterlace/yadif_template.h:245: error:
> 'asm' operand has impossible constraints
> ../../../modules/video_filter/deinterlace/yadif_template.h:251: error:
> 'asm' operand has impossible constraints
>
> and FreeBSD:
> cc1: error in backend: Ran out of registers during register allocation!
> and other stuff.
>
> I suggest reverting the patch and doing something like:
>
> uint8_t tmp[5*16];
> uint8_t *tmpA= (uint8_t*)(((uintptr_t)(tmp+15)) & ~15);
> uint8_t *tmp0 = &tmpA[0]; uint8_t *tmp1 = &tmpA[16] ...
>
> what do you think?
> It would use the old version of the code and ensure that tmp* are aligned.

I also think the last patch must be reverted.
I tried your suggestion and linux 32/64 bit build crashed when
enabling yadif, another builds, win32/64 osx64,  seemed to work.

So, I want to revert the modification done in the last patch, and
apply manually alignment only on win32 builds.
I want to know whether yadif worked on Mac OS X/FreeBSD 32bit build
before applying a patch (577b183d295adbea85e03502d73d04fd3f90ef90) or
not.

>
> Btw any idea why alignement doesn't work? Is it only for static data and
> not for stack variables?
>
> Other functions use (and rely on?) alignement in VLC so it should be
> important to know what's going on there
> _______________________________________________
> vlc-devel mailing list
> To unsubscribe or modify your subscription options:
> http://mailman.videolan.org/listinfo/vlc-devel



-- 
KORIYAMA, Naohiro
nkoriyama at gmail.com