[x264-devel] fix x264_macroblock_analyse and x264_mb_analyse_p_rd functions

[Noboru Asai]

Hi,

These patches fix x264_macroblock_analyse and x264_mb_analyse_p_rd functions.

(1) patch-set-i_sub_partition-before-qpel_rd

Callgraph of x264_me_refine_qpel_rd function is:

x264_me_refine_qpel_rd
  -> COST_MV_RD
     -> x264_rd_cost_part
        -> x264_macroblock_encode_p8x8
           -> x264_mb_mc_8x8

Looking at COST_MV_RD macro and x264_mb_mc_8x8 function,
h->mb.i_sub_partition[] should be "D_L0_8x8" before calling
x264_me_refine_qpel_rd function when partition is
16x8 or 8x16.  But if 8x4, 4x8 or 4x4 partitions
are enabled, h->mb.i_sub_partition[] may not be "D_L0_8x8".

(2) patch-fix-x264_mb_analyse_p_rd-1

Callgraph of x264_rd_cost_mb function is

x264_rd_cost_mb
  -> x264_macroblock_size_cabac (x264_macroblock_write_cabac)
     -> x264_cabac_mb8x8_mvd
        -> x264_cabac_mb_mvd
           -> x264_mb_predict_mv

Since h->mb.i_partition is referenced in x264_mb_predict_mv function,
it should be "D_8x8" before calling x264_rd_cost_mb function.

(3) patch-fix-x264_mb_analyse_p_rd-2

Similarly (1), if 8x4, 4x8 or 4x4 partitions are enabled, h->mb.i_partition and
h->mb.cache.mv should be updated.


Any comments are welcome (I'm not sure they are right fixes).

Noboru Asai
-------------- next part --------------
A non-text attachment was scrubbed...
Name: patch-set-i_sub_partition-before-qpel_rd
Type: application/octet-stream
Size: 1334 bytes
Desc: not available
Url : http://mailman.videolan.org/pipermail/x264-devel/attachments/20080129/844cf02e/attachment.obj 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: patch-fix-x264_mb_analyse_p_rd-1
Type: application/octet-stream
Size: 451 bytes
Desc: not available
Url : http://mailman.videolan.org/pipermail/x264-devel/attachments/20080129/844cf02e/attachment-0001.obj 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: patch-fix-x264_mb_analyse_p_rd-2
Type: application/octet-stream
Size: 579 bytes
Desc: not available
Url : http://mailman.videolan.org/pipermail/x264-devel/attachments/20080129/844cf02e/attachment-0002.obj