[x264-devel] Potential bug in transform bypass 8x8

[Alain Champenois]

> This is a known bug, but unfortunately there's not much that can be
> done at this point.

I agree. At this stage changing both encoder/decoder to become compliant
would be a big hassle.

I would however suggest a way to make x264 compliant with the spec while 
keeping compatibility with current libavcodec/coreavec decoders.

If intra8x8 horizontal and vertical modes are disabled in x264 lossless, 
streams encoded would become compliant with the spec while staying compatible 
with any decoder supporting lossless (including libavcodec/coreavc). libavcodec
wouldn't have to be changed, and existing streams will continu to be deocded
properly.

I did a small evaluation using the HEVC test sequences, and the size saving
went from 65.769% to 65.748% when those 2 modes were disabled in x264 using 
the patch below (full results included attachment).

So with negligible impact on compression and no impact on existing decoders, 
x264 can be made compliant with the spec, which would garanty compatibility
with any current or future decoders (compliant or not).


A side question concerning lossless. Was there a reasons to not support 
B frame in lossless (beside limiting workload/complexity) ? Depending on  
the sequence, reduction in size can be noticeable when B frames are supported 
in lossless. Support of lossless B frame is present in libavcodec.

  Thanks
  Alain

diff --Naur a/encoder/analyse.c b/encoder/analyse.c
--- a/encoder/analyse.c 
+++ b/encoder/analyse.c
@@ -913,7 +913,12 @@
                     int i_mode = *predict_mode;

                     if( h->mb.b_lossless )
+                    {
+                        /* Skip modes not compliant with the H.264 spec */
+                        if(( i_mode == I_PRED_8x8_V || i_mode == I_PRED_8x8_H ) )
+                           continue;
                         x264_predict_lossless_8x8( h, p_dst_by, 0, idx, i_mode, edge );
+                    }
                     else
                         h->predict_8x8[i_mode]( p_dst_by, edge );

-------------- next part --------------
A non-text attachment was scrubbed...
Name: x264_lossless_analysis.xls
Type: application/msexcel
Size: 32768 bytes
Desc: not available
URL: <http://mailman.videolan.org/pipermail/x264-devel/attachments/20130621/400a42cb/attachment-0001.bin>