[x264-devel] x264 encoding corruption bug w/ AltiVec enabled

[Timothy Pearson]

On 01/30/2019 04:52 PM, Timothy Pearson wrote:
> On 01/30/2019 04:18 PM, Timothy Pearson wrote:
>> On 01/30/2019 03:17 PM, Timothy Pearson wrote:
>>> On 01/30/2019 03:08 PM, Timothy Pearson wrote:
>>>> On 01/30/2019 07:08 AM, Luca Barbato wrote:
>>>>> On 30/01/2019 12:10, Timothy Pearson wrote:
>>>>>> When attempting to use the latest libx264 with the latest ffmpeg sources
>>>>>> on Talos II (ppc64le), a color / blocking artifact is introduced in the
>>>>>> source video.  These artifacts disappear entirely when libx264 is
>>>>>> rebuilt with asm support (AltiVec) disabled, but the machine is then no
>>>>>> longer able to maintain 60fps framerate on the sample live stream.
>>>>>
>>>>> Please test with the x264 cli and, if possible, provide me the sample
>>>>> triggering it.
>>>>>
>>>>> All the altivec code is tested on checkasm so sounds quite strange there
>>>>> is a problem there.
>>>>>
>>>>> lu
>>>>
>>>> After a bit of work I was able to put together a direct x264 command
>>>> that produces corrupted videos, based on the video output settings used
>>>> for ffmpeg.  I should note that for various technical reasons related to
>>>> downstream consumer devices, a few of the settings (FPS, bitrate,
>>>> keyint) are mandatory for a proper experience, and that they work
>>>> without AltiVec enabled.
>>>>
>>>> Pull the Big Buck Bunny 23 second file from here:
>>>>
>>>> https://commons.wikimedia.org/wiki/File:Big_Buck_Bunny_first_23_seconds_1080p.ogv
>>>>
>>>> Encode with:
>>>>
>>>> x264 --threads 31 --fps 30 --keyint 30 --min-keyint 30 --scenecut 0
>>>> --preset ultrafast --tune grain --bitrate 25600000 -o
>>>> Big_Buck_Bunny_encoding_failure.ogv
>>>> Big_Buck_Bunny_first_23_seconds_1080p.ogv
>>>>
>>>> I'm noticing with some interest that the resulting file is unplayable
>>>> (black screen) in VLC while mplayer shows intermittent artifacts like
>>>> the one I attached.  mplayer also complains about numerous errors, which
>>>> would probably explain why each video seems to have different artifacts:
>>>
>>> <snip>
>>>
>>> Digging a bit further the artifacting is also present when rebuilt with
>>> --disable-asm.  It seems disable-asm might have made the bug shift
>>> somewhere else under ffmpeg but not disappear when using the direct x264
>>> application.  Also, the bug is readily apparent when using "-threads 16"
>>> instead of "threads 31", so it doesn't appear related to the threading
>>> support.
>>>
>>
>> After fixing the bitrate (was off by a factor of 1000 due to differences
>> between ffmpeg and x264 command line parameters) I'm unable to replicate
>> the issue with the standalone x264 application.  This ffmpeg command
>> line replicates the issue perfectly however:
>>
>> ffmpeg -r 30 -i Big_Buck_Bunny_first_23_seconds_1080p.ogv -r 30 -pix_fmt
>> yuv422p -vcodec libx264 -preset ultrafast -x264opts
>> keyint=30:min-keyint=30:scenecut=0 -b:v 25600k -bufsize 25600k -tune
>> grain -threads 16 Big_Buck_Bunny_encoding_failure.mp4
>>
>> I'm assuming something might be off in the interface between libx264 and
>> ffmpeg?  --disable-vsx did not help, --disable-asm completely resolved
>> the problems.
>>
>> Screenshot of a corrupted frame:
>> http://imgbin.ca/image.uploads/30-01-2019/original-206092b96120b5f748e06bd9e58e6df5.png
>>
> 
> I've tracked this down to the following two AltiVec functions:
> x264_add8x8_idct_dc_altivec;
> x264_add16x16_idct_dc_altivec;
> 
> Disabling both of them, while leaving all other AltiVec functions
> active, resolves the corruption.
> 

Trying to speed up encode I found that disabling trellis corrupted the
output beyond recognition.  I also found that setting that flag in the
native x264 application reproduced one of my original encoding problems
seen under ffmpeg:

x264 --threads 16 --fps 30 --keyint 30 --min-keyint 30 --scenecut 0
--preset ultrafast --tune grain --bitrate 25600 --trellis 0 -o
Big_Buck_Bunny_encoding_failure.ogv
Big_Buck_Bunny_first_23_seconds_1080p.ogv

Note that this test was run with the two problematic AltiVec functions
above disabled.  Disabling all asm with --disable-asm fixes the problems
with trellis=0.

Screenshot:
http://imgbin.ca/image.uploads/30-01-2019/original-57d9eb4308cbdcbd46bc56b38bbae722.png

Hopefully this will be somewhat easier to debug now that ffmpeg is out
of the picture!

-- 
Timothy Pearson
Raptor Engineering
+1 (415) 727-8645 (direct line)
+1 (512) 690-0200 (switchboard)
https://www.raptorengineering.com