[x264-devel] core dump on solaris 10/space with x264-snapshot-20100112-2245

[Håkan Hjort]

On Fri, Jan 15, 2010 at 22:40, Chris Brookes <cbrookes at gmail.com> wrote:
> 2010/1/14 Jason Garrett-Glaser <darkshikari at gmail.com>:
>
>>>> I'm going to guess int dmf = dequant_mf[i_qp%6][0], but that makes no
>>>> sense whatsoever.
>>>
>>> Current function is x264_mb_encode_8x8_chroma
>>>   55           int d0 = dct[0] + dct[1]; \
>>
>> That's impossible.  There's no way it can crash on that line without
>> an utterly horrific compiler failure.
>>
>> At this point, I would be inclined to blame GCC.
>
> OK, an eagle eyed mailing list reader pointed out that the \
> characters I left in the code I pasted would mean the pre-processor
> would merge the lines and make the crash point as unfathomable as
> having the macro. I took those out and back with gcc 3.4.6 sure enough
> it cores at the point you guessed.
>
> Current function is x264_mb_encode_8x8_chroma
>   59       int dmf = dequant_mf[i_qp%6][0];
> (dbx) where
> current thread: t at 1
> =>[1] x264_mb_encode_8x8_chroma(h = 0xab340, b_inter = 0, i_qp = 19),
> line 59 in "macroblock.c"

I think interesting info would be the the register values and the
surrounding machine code.
Not sure how to do it in dbx, using gdb it would be something like;
info regs
disassemble $pc-32 $pc+16
p dequant_mf

/Håkan