[x264-devel] x264 segmentation fault when 4:4:4 10 bits cavlc encode.

[Henrik Gramner]

On Wed, Aug 15, 2018 at 5:55 AM, Masaharu Tamura <tamura at pegasys-inc.com> wrote:
> Hi,
>
> I have a problem when I try to encode random pixel video.
>
> Using x264 options is:
>
> --input-res 704x480 --fps 30/1 --profile high444 --input-csp i444
> --input-depth 10 --input-range tv --output-csp i444 --output-depth 10
> --no-cabac --crf 1 --keyint 1 --subme 11 --trellis 2 -o output.264
> output.yuv
>
> I attached back trace, disassemble and register information.

The GDB dump indicates that all 4x4 DCT coefficients are zero, in
which case the code path that resulted in the crashing function being
called should never have been taken in the first place.

Haven't had time to look into it more than that.