[x264-devel] ppc64: versions of x264 post r901 generate wrong output

[Hans J. Harff]

hi list,

sorry for reporting my findings late, but it has been a while since i pulled git.

i am running linux ppc64 with 64bit ul on an ibm power5 box (no altivec)
and all versions of x264 up to and including r901 have been working fine.

but it appears that since the following commits:

[x264-devel] commit: faster bs_write
(Jason Garrett-Glaser ), Fri Jul  4 18:32:32 2008 -0600|
[8951797a04c4069e1926b5c4bcd2c39eb3b3a266]

[x264-devel] commit: Fix compilation on PPC systems (borked in r903)
( Jason Garrett-Glaser ), Fri Jul 11 15:45:54 2008 -0600|
[8cd5d60fcc23edcbdb97a190d8ac926db4161eb1]

the file output generated by x264 is not recognized by any decoder. this
is true no matter what output format i select - raw, or muxed to either
mkv or mp4 (using either x264's mkv/mp4 muxing or muxing with external
apps like mp4box)

i have taken a look at a few raw output files from *pre* r902 versions, and
they usually start out like this:

0000:0010 b7 96 2c d8 20 d9 23 ee ef 78 32 36 34 20 2d 20 ·.,Ø Ù#îïx264 - 
0000:0020 63 6f 72 65 20 36 30 20 72 30 2b 39 30 31 20 35 core 60 r0+901 5
0000:0030 38 64 37 64 30 36 20 2d 20 48 2e 32 36 34 2f 4d 8d7d06 - H.264/M
0000:0040 50 45 47 2d 34 20 41 56 43 20 63 6f 64 65 63 20 PEG-4 AVC codec 
0000:0050 2d 20 43 6f 70 79 6c 65 66 74 20 32 30 30 33 2d - Copyleft 2003-
0000:0060 32 30 30 38 20 2d 20 68 74 74 70 3a 2f 2f 77 77 2008 - http://ww
0000:0070 77 2e 76 69 64 65 6f 6c 61 6e 2e 6f 72 67 2f 78 w.videolan.org/x
0000:0080 32 36 34 2e 68 74 6d 6c 20 2d 20 6f 70 74 69 6f 264.html - optio
0000:0090 6e 73 3a 20 63 61 62 61 63 3d 31 20 72 65 66 3d ns: cabac=1 ref=
0000:00a0 33 20 64 65 62 6c 6f 63 6b 3d 31 3a 2d 32 3a 2d 3 deblock=1:-2:-
(snip)

but *post* r901 versions start like this:

0000:0000 00 00 00 01 06 00 00 03 00 00 03 00 00 03 00 00 ................
0000:0010 03 00 00 03 00 00 03 00 00 03 00 00 03 00 00 03 ................
0000:0020 00 00 03 00 00 03 00 00 03 00 00 03 00 00 03 00 ................
0000:0030 00 03 00 00 03 00 00 03 00 00 03 00 00 03 00 00 ................
0000:0040 03 00 00 03 00 00 03 00 00 03 00 00 03 00 00 03 ................
0000:0050 00 00 03 00 00 03 00 00 03 00 00 03 00 00 03 00 ................
0000:0060 00 03 00 00 03 00 00 03 00 00 03 00 00 03 00 00 ................
0000:0070 03 00 00 03 00 00 03 00 00 03 00 00 03 00 00 03 ................
0000:0080 00 00 03 00 00 03 00 00 03 00 00 03 00 00 03 00 ................
0000:0090 00 03 00 00 03 00 00 03 00 00 03 00 00 03 00 00 ................
0000:00a0 03 00 00 03 00 00 03 00 00 03 00 00 03 00 00 03 ................
(eventually followed by other random-looking data)
(snip)


i am thinking there is something going on with little-/big-endian
conversions not working out as intended, so i took the freedom of commenting
out the -DWORDS_BIGENDIAN flag before compilation (which i believe disables
a few things in /common/osdep.h and some other locations). which generated
output like this:

0000:0000 00 00 00 01 06 2a ff ff 05 bd e9 45 dc b7 48 d9 .....*ÿÿ.½éEÜ·HÙ
0000:0010 e6 20 d8 2c 96 ef ee 23 d9 34 36 32 78 63 20 2d æ Ø,.ïî#Ù462xc -
0000:0020 20 20 65 72 6f 72 20 34 36 4d 39 37 39 34 64 36   eror 46M9794d6
0000:0030 20 64 38 66 61 48 20 2d 20 34 36 32 2e 45 50 4d  d8faH - 462.EPM
0000:0040 2f 20 34 2d 47 20 43 56 41 65 64 6f 63 20 2d 20 / 4-G CVAedoc - 
0000:0050 63 79 70 6f 43 74 66 65 6c 30 30 32 20 30 32 2d cypoCtfel002 02-
0000:0060 33 2d 20 38 30 74 74 68 20 2f 2f 3a 70 2e 77 77 3- 80tth //:p.ww
0000:0070 77 65 64 69 76 6e 61 6c 6f 67 72 6f 2e 36 32 78 wedivnalogro.62x
0000:0080 2f 74 68 2e 34 2d 20 6c 6d 74 70 6f 20 73 6e 6f /th.4- lmtpo sno
0000:0090 69 61 63 20 3a 3d 63 61 62 65 72 20 31 20 33 3d iac :=caber 1 3=
0000:00a0 66 6c 62 65 64 3d 6b 63 6f 32 2d 3a 31 20 31 2d flbed=kco2-:1 1-
(snip)


sure... everthing is swapped around, but it's more than just 00 00 03...


i know that my particular platform isn't terribly popular, but i would
love to stay up to date with all the exciting developments lately... i
am by no means an expert, but it seems that this is an easy one to fix.

i am happy to provide any additional information if necessary or offering
up my box as a guinea pig for trying out any patches related to this.



cheers :)

hans.

.