[x264-devel] commit: Fix satd predictors with high bit depth (Jason Garrett-Glaser )

[git at videolan.org]

x264 | branch: master | Jason Garrett-Glaser <darkshikari at gmail.com> | Fri Dec 24 17:28:42 2010 -0500| [a4406c19e75eda4ee5754827cc9381276be403ab] | committer: Jason Garrett-Glaser 

Fix satd predictors with high bit depth
Resulted in odd CRF-mode results with --no-mbtree, as well as suboptimal VBV handling.

> http://git.videolan.org/gitweb.cgi/x264.git/?a=commit;h=a4406c19e75eda4ee5754827cc9381276be403ab
---

 encoder/slicetype.c |    6 +++++-
 1 files changed, 5 insertions(+), 1 deletions(-)

diff --git a/encoder/slicetype.c b/encoder/slicetype.c
index 7eff570..9d5718c 100644
--- a/encoder/slicetype.c
+++ b/encoder/slicetype.c
@@ -1647,5 +1647,9 @@ int x264_rc_analyse_slice( x264_t *h )
         }
     }
 
-    return cost;
+    if( BIT_DEPTH > 8 )
+        for( int y = 0; y < h->mb.i_mb_height; y++ )
+            h->fdec->i_row_satd[y] >>= (BIT_DEPTH - 8);
+
+    return cost >> (BIT_DEPTH - 8);
 }