[x264-devel] Failure to build x264 with ASM on i386

[Martin Storsjö]

On Mon, 3 Feb 2014, Dimitry Andric wrote:

> On 03 Feb 2014, at 18:39, Loren Merritt <lorenm at u.washington.edu> wrote:
> ...
>> Otoh, gcc works with -fPIC. I can confirm that 5 registers is enough for
>> the inline asm blocks in question. If clang thinks they need more than 5
>> registers, that's a bug in clang's register allocator.
>
> No, gcc uses 7 registers.  For example, with -fPIC and gcc 4.8, the
> allocation for x264_predictor_clip_mmx2() is as follows:
>
> %0 = %eax
> %1 = %edx
> %2 = %ecx
> %4 = %ebp
> %5 = %esi
> %6 = %edi
> %7 = %ebx
>
> The reason is that gcc assumes a 16 byte stack alignment on i386, which
> is only valid for Linux after ~2006, not most BSDs.  If you force gcc to
> assume a 4 byte stack alignment, it also cannot compile the inline
> assembly:

FWIW, the similar cases within libav are handled by adding 
__attribute__((force_align_arg_pointer)) to all public entry points into 
the libraries, which adds a special prologue to these functions that 
realign the stack to 16 bytes, and adding -mincoming-stack-boundary=4 to 
the cflags, telling the compiler to assume a 16 byte aligned stack in all 
functions, so only the public ones need to care about fixing the 
alignment.

// Martin