[x264-devel] [PATCH] Fix build on Solaris 10 SPARC

[Sean McGovern]

2010/11/20 Måns Rullgård <mans at mansr.com>
>
> Sean McGovern <gseanmcg at gmail.com> writes:
>
> > 2010/11/20 Måns Rullgård <mans at mansr.com>
> >> Sean McGovern <gseanmcg at gmail.com> writes:
> >>
> >> >  echo "int i = 0x42494745; double f = 0x1.0656e6469616ep+102;" > conftest.c
> >> >  $CC $CFLAGS conftest.c -c -o conftest.o 2>$DEVNULL || die "endian test failed"
> >> > -if grep -q BIGE conftest.o && grep -q FPendian conftest.o ; then
> >> > +if (strings -a conftest.o | grep -q BIGE) && (strings -a conftest.o |
> >> grep -q FPendian) ; then
> >> >      define WORDS_BIGENDIAN
> >> > -elif !(grep -q EGIB conftest.o && grep -q naidnePF conftest.o) ; then
> >> > +elif !(strings -a conftest.o | grep -q EGIB && strings -a conftest.o |
> >> grep -q naidnePF) ; then
> >> >      die "endian test failed"
> >> >  fi
> >>
> >> I remember encountering some system where this didn't work.
> >> Unfortunately, I don't remember which one.  It has to do with this
> >> part of the 'strings' spec:
> >>
> >>  A printable string is any sequence of four (by default) or more
> >>  printable characters terminated by a <newline> or NUL character.
> >>
> >> You can't be certain that the test word will be followed by a zero.
> >
> > Oh, ugh. Was it anything modern where this failed? Please don't tell
> > me this is on Irix or Ultrix or something equally obscure.
>
> It was something someone saw fit to run FFmpeg on.  Stock FFmpeg
> doesn't build at all on Irix, so it can't have been that.  It was
> probably some BSD or other.
>
> I suppose if you made the test variables 2-element arrays with the
> second entry zero, it would be safer.
>
> However, this entire approach can still fail in some cases.  With
> link-time optimisation enabled, the Clang compiler doesn't output
> normal object files with machine code but rather an intermediate
> "bitcode" in which endianness isn't necessarily the same as the final
> target.
>
> --
> Måns Rullgård
> mans at mansr.com
> _______________________________________________
> x264-devel mailing list
> x264-devel at videolan.org
> http://mailman.videolan.org/listinfo/x264-devel


How about replacing the current endian test with this snippet from
autoconf's AC_C_BIGENDIAN test and then just testing against the
return code?

	     union
	     {
	       long int l;
	       char c[sizeof (long int)];
	     } u;
	     u.l = 1;
	     return u.c[sizeof (long int) - 1] == 1;

-- Sean McG