[x264-devel] Potential bug in transform bypass 8x8

[Pavel Koshevoy]

On 06/21/2013 06:38 PM, Jason Garrett-Glaser wrote:
> On Fri, Jun 21, 2013 at 3:22 PM, Alain Champenois <achampen at matrox.com> wrote:
>>> This is a known bug, but unfortunately there's not much that can be
>>> done at this point.
>> I agree. At this stage changing both encoder/decoder to become compliant
>> would be a big hassle.
>>
>> I would however suggest a way to make x264 compliant with the spec while
>> keeping compatibility with current libavcodec/coreavec decoders.
>>
>> If intra8x8 horizontal and vertical modes are disabled in x264 lossless,
>> streams encoded would become compliant with the spec while staying compatible
>> with any decoder supporting lossless (including libavcodec/coreavc). libavcodec
>> wouldn't have to be changed, and existing streams will continu to be deocded
>> properly.
>>
>> I did a small evaluation using the HEVC test sequences, and the size saving
>> went from 65.769% to 65.748% when those 2 modes were disabled in x264 using
>> the patch below (full results included attachment).
>>
>> So with negligible impact on compression and no impact on existing decoders,
>> x264 can be made compliant with the spec, which would garanty compatibility
>> with any current or future decoders (compliant or not).
> This can already be done with --no-8x8dct, so I'm not sure it's worth
> bothering hacking into x264.
>

Would it not be more correct to handle this within libx264 rather than 
externally?  The API user (like myself) may not know anything about H.264 
lossless spec and the need to disable 8x8dct for lossless compression.

Pavel.