[vlc-devel] [PATCH] Fixed a crash caused by yadif deinterlacer on Windows XP

[Kaarlo Räihä]

2012/1/9 Rafaël Carré <funman at videolan.org>

> Hello,
>
> Le 12-01-08 00:52, Naohiro KORIYAMA a écrit :
> > The old patch, I sent to vlc-devel ML yesterday, didn't help the crash
> > on Windows XP 32bit.
> > I was wrong that I tested with the module that was compiled with a
> > different patch.
> >
> > So I made a new patch and tested again
> >  (before applying old patch, after applying old patch, after applying
> > new patch):
> > - Windows XP 32bit using XP mode on Windows 7 (win32)
> >   NG->NG->OK
> > - Windows 7 64bit (win32/win64)
> >   OK->OK->OK
> > - Mac OS X Lion (intel64)
> >   OK->OK->OK
> > - Ubuntu 11.10 32bit using VirtualBox (32)
> >   OK->OK->OK
> > - Ubuntu 11.10 64bit using VirtualBox (64)
> >   OK->OK->OK
> >
> > It's all OK with the new patch.
> > But as jb mentioned on the ML few days ago, compiler warnings appear
> again
> > on 32bit builds both Windows and Linux.
>
> These warnings might be fixed by using uintptr_t instead of uint64_t.
>
> Perhaps you noticed but this patch broke compile on OSX 32 bits and
> FreeBSD 32 bits : http://buildbot.videolan.org/builders/
>
> OSX says:
>
> ../../../modules/video_filter/deinterlace/yadif_template.h:245: error:
> can't find a register in class 'GENERAL_REGS' while reloading 'asm'
> ../../../modules/video_filter/deinterlace/yadif_template.h:251: error:
> can't find a register in class 'GENERAL_REGS' while reloading 'asm'
> ../../../modules/video_filter/deinterlace/yadif_template.h:245: error:
> 'asm' operand has impossible constraints
> ../../../modules/video_filter/deinterlace/yadif_template.h:251: error:
> 'asm' operand has impossible constraints
>
> and FreeBSD:
> cc1: error in backend: Ran out of registers during register allocation!
> and other stuff.
>
> I suggest reverting the patch and doing something like:
>
> uint8_t tmp[5*16];
> uint8_t *tmpA= (uint8_t*)(((uintptr_t)(tmp+15)) & ~15);
> uint8_t *tmp0 = &tmpA[0]; uint8_t *tmp1 = &tmpA[16] ...
>
> what do you think?
> It would use the old version of the code and ensure that tmp* are aligned.
>
> Btw any idea why alignement doesn't work? Is it only for static data and
> not for stack variables?
>
> Other functions use (and rely on?) alignement in VLC so it should be
> important to know what's going on there
>

And it would be nice to have a Wiki page about these, since it seems every
x86 ASM patch causes some issues.


> _______________________________________________
> vlc-devel mailing list
> To unsubscribe or modify your subscription options:
> http://mailman.videolan.org/listinfo/vlc-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.videolan.org/pipermail/vlc-devel/attachments/20120109/74e33c89/attachment.html>