[x264-devel] fix rd related functions

[Noboru Asai]

Hi,

These patches fix rd related functions.

(1) patch-fix-x264_mb_cache_mv_p8x8

Callgraph of x264_me_refine_qpel_rd function is:

x264_me_refine_qpel_rd
  -> COST_MV_RD
     -> x264_rd_cost_part
        -> x264_partition_size_cabac
           -> x264_cabac_mb8x8_mvd
              -> x264_cabac_mb_mvd
                 -> x264_cabac_mb_mvd_cpn

h->mb.cache.{mv,mvd} of left and top partition is referenced
in x264_cabac_mb_mvd_cpn function, but they are not updated
when left or top partition is 8x4, 4x8 or 4x4.

(2) patch-fix-x264_macroblock_encode_p8x8

Another callgraph of x264_me_refine_qpel_rd function is:

x264_me_refine_qpel_rd
  -> COST_MV_RD
     -> x264_rd_cost_part
        -> x264_partition_size_cabac
           -> block_residual_write_cabac
              -> x264_cabac_mb_cbf_ctxidxinc

h->mb.cache.non_zero_count of left and top partition is referenced
in x264_cabac_mb_cbf_ctxidxinc function, but they are not updated
when left or top partition is not 16x16.

(3) patch-fix-x264_intra_rd_refine-nnz

Callgraph of x264_intra_rd_refine function is:

x264_intra_rd_refine
  -> x264_rd_cost_i8x8 (or x264_rd_cost_i4x4)
     -> x264_partition_i8x8_size_cabac (or x264_partition_i4x4_size_cabac)
        -> block_residual_write_cabac
           -> x264_cabac_mb_cbf_ctxidxinc

Similarly (2), h->mb.cache.non_zero_count of left or top partition
are not updated when partition is i8x8 or i4x4.
(Updating non_zero_count may not be needed when partition is i8x8.)


If you try to apply these patches, you should apply another patches
attached the following:

o [x264-devel] fix x264_macroblock_analyse and x264_mb_analyse_p_rd functions
http://mailman.videolan.org/pipermail/x264-devel/2008-January/004016.html


Any comments are welcome.

Noboru Asai
-------------- next part --------------
A non-text attachment was scrubbed...
Name: patch-fix-x264_mb_cache_mv_p8x8
Type: application/octet-stream
Size: 5286 bytes
Desc: not available
Url : http://mailman.videolan.org/pipermail/x264-devel/attachments/20080305/4d02f8b1/attachment.obj 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: patch-fix-x264_macroblock_encode_p8x8
Type: application/octet-stream
Size: 3462 bytes
Desc: not available
Url : http://mailman.videolan.org/pipermail/x264-devel/attachments/20080305/4d02f8b1/attachment-0001.obj 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: patch-fix-x264_intra_rd_refine-nnz
Type: application/octet-stream
Size: 1071 bytes
Desc: not available
Url : http://mailman.videolan.org/pipermail/x264-devel/attachments/20080305/4d02f8b1/attachment-0002.obj