[vlc-devel] [PATCH] Fixed a crash caused by yadif deinterlacer on Windows XP

[Naohiro KORIYAMA]

I made a patch for Mac OS X and FreeBSD 32bit builds.

Only on win32 builds, use manual alignment.
On other builds, use gcc's attribute aligned.


2012/1/9 Naohiro KORIYAMA <nkoriyama at gmail.com>:
> Hello,
>
> 2012/1/9 Rafaël Carré <funman at videolan.org>:
>> Hello,
>>
>> Le 12-01-08 00:52, Naohiro KORIYAMA a écrit :
>>> The old patch, I sent to vlc-devel ML yesterday, didn't help the crash
>>> on Windows XP 32bit.
>>> I was wrong that I tested with the module that was compiled with a
>>> different patch.
>>>
>>> So I made a new patch and tested again
>>>  (before applying old patch, after applying old patch, after applying
>>> new patch):
>>> - Windows XP 32bit using XP mode on Windows 7 (win32)
>>>   NG->NG->OK
>>> - Windows 7 64bit (win32/win64)
>>>   OK->OK->OK
>>> - Mac OS X Lion (intel64)
>>>   OK->OK->OK
>>> - Ubuntu 11.10 32bit using VirtualBox (32)
>>>   OK->OK->OK
>>> - Ubuntu 11.10 64bit using VirtualBox (64)
>>>   OK->OK->OK
>>>
>>> It's all OK with the new patch.
>>> But as jb mentioned on the ML few days ago, compiler warnings appear again
>>> on 32bit builds both Windows and Linux.
>>
>> These warnings might be fixed by using uintptr_t instead of uint64_t.
>>
>> Perhaps you noticed but this patch broke compile on OSX 32 bits and
>> FreeBSD 32 bits : http://buildbot.videolan.org/builders/
>>
>> OSX says:
>>
>> ../../../modules/video_filter/deinterlace/yadif_template.h:245: error:
>> can't find a register in class 'GENERAL_REGS' while reloading 'asm'
>> ../../../modules/video_filter/deinterlace/yadif_template.h:251: error:
>> can't find a register in class 'GENERAL_REGS' while reloading 'asm'
>> ../../../modules/video_filter/deinterlace/yadif_template.h:245: error:
>> 'asm' operand has impossible constraints
>> ../../../modules/video_filter/deinterlace/yadif_template.h:251: error:
>> 'asm' operand has impossible constraints
>>
>> and FreeBSD:
>> cc1: error in backend: Ran out of registers during register allocation!
>> and other stuff.
>>
>> I suggest reverting the patch and doing something like:
>>
>> uint8_t tmp[5*16];
>> uint8_t *tmpA= (uint8_t*)(((uintptr_t)(tmp+15)) & ~15);
>> uint8_t *tmp0 = &tmpA[0]; uint8_t *tmp1 = &tmpA[16] ...
>>
>> what do you think?
>> It would use the old version of the code and ensure that tmp* are aligned.
>
> I also think the last patch must be reverted.
> I tried your suggestion and linux 32/64 bit build crashed when
> enabling yadif, another builds, win32/64 osx64,  seemed to work.
>
> So, I want to revert the modification done in the last patch, and
> apply manually alignment only on win32 builds.
> I want to know whether yadif worked on Mac OS X/FreeBSD 32bit build
> before applying a patch (577b183d295adbea85e03502d73d04fd3f90ef90) or
> not.
>
>>
>> Btw any idea why alignement doesn't work? Is it only for static data and
>> not for stack variables?
>>
>> Other functions use (and rely on?) alignement in VLC so it should be
>> important to know what's going on there
>> _______________________________________________
>> vlc-devel mailing list
>> To unsubscribe or modify your subscription options:
>> http://mailman.videolan.org/listinfo/vlc-devel
>
>
>
> --
> KORIYAMA, Naohiro
> nkoriyama at gmail.com



-- 
KORIYAMA, Naohiro
nkoriyama at gmail.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-Fix-for-comilation-problem-on-Mac-OS-X-32bit-and-Fre.patch
Type: application/octet-stream
Size: 5324 bytes
Desc: not available
URL: <http://mailman.videolan.org/pipermail/vlc-devel/attachments/20120110/543f8f38/attachment.obj>