[vlc-commits] video_format_TransformBy: fix and preserve video offsets

[Rémi Denis-Courmont]

vlc | branch: master | Rémi Denis-Courmont <remi at remlab.net> | Sat Mar 15 14:38:53 2014 +0200| [3013997d801170ddb66668c73be3bbc7faa71676] | committer: Rémi Denis-Courmont

video_format_TransformBy: fix and preserve video offsets

In general, the picture buffer contains, in memory order:
- cropped top scan lines,
- visible scan lines,
- cropped bottom scan lines,
- extra codec lines,
- padding lines.

This order needs to be preserved when flipping vertically or rotating
180 degrees, since filters and displays rely on this. In particular,
the picture plane structure so far has no provisions for non-zero
offsets, so filters expect visible pixels at the top left of each
pixels plane. Effectively, non-zero offsets only work for cropping
after filtering. Lets assume that the number of cropped lines are
identical at the top and bottom, and keep the X and Y offsets as are.

This fixes green lines at the bottom on R180 and VFLIP.

If support for non-centered cropping is ever required, I believe
offsets will need to be added to plane_t. This would break many video
filters though.

> http://git.videolan.org/gitweb.cgi/vlc.git/?a=commit;h=3013997d801170ddb66668c73be3bbc7faa71676
---

 src/misc/es_format.c |   44 +++++---------------------------------------
 1 file changed, 5 insertions(+), 39 deletions(-)

diff --git a/src/misc/es_format.c b/src/misc/es_format.c
index 46094c5..9028847 100644
--- a/src/misc/es_format.c
+++ b/src/misc/es_format.c
@@ -339,54 +339,20 @@ void video_format_TransformBy( video_format_t *fmt, video_transform_t transform
     }
 
     /* Apply transform */
-
-    video_format_t scratch = *fmt;
-
-    if( ORIENT_IS_SWAP( fmt->orientation ) != ORIENT_IS_SWAP( dst_orient )) {
+    if( ORIENT_IS_SWAP( fmt->orientation ) != ORIENT_IS_SWAP( dst_orient ) )
+    {
+        video_format_t scratch = *fmt;
 
         fmt->i_width = scratch.i_height;
         fmt->i_visible_width = scratch.i_visible_height;
         fmt->i_height = scratch.i_width;
         fmt->i_visible_height = scratch.i_visible_width;
+        fmt->i_x_offset = scratch.i_y_offset;
+        fmt->i_y_offset = scratch.i_x_offset;
         fmt->i_sar_num = scratch.i_sar_den;
         fmt->i_sar_den = scratch.i_sar_num;
     }
 
-    unsigned int delta_y = scratch.i_height - scratch.i_visible_height - scratch.i_y_offset;
-    unsigned int delta_x = scratch.i_width - scratch.i_visible_width - scratch.i_x_offset;
-
-    switch ( transform )
-    {
-        case TRANSFORM_IDENTITY:
-            break;
-        case TRANSFORM_R90:
-            fmt->i_x_offset = delta_y;
-            fmt->i_y_offset = scratch.i_x_offset;
-            break;
-        case TRANSFORM_R180:
-            fmt->i_x_offset = delta_x;
-            fmt->i_y_offset = delta_y;
-            break;
-        case TRANSFORM_R270:
-            fmt->i_x_offset = scratch.i_y_offset;
-            fmt->i_y_offset = delta_x;
-            break;
-        case TRANSFORM_HFLIP:
-            fmt->i_x_offset = delta_x;
-            break;
-        case TRANSFORM_VFLIP:
-            fmt->i_y_offset = delta_y;
-            break;
-        case TRANSFORM_TRANSPOSE:
-            fmt->i_x_offset = scratch.i_y_offset;
-            fmt->i_y_offset = scratch.i_x_offset;
-            break;
-        case TRANSFORM_ANTI_TRANSPOSE:
-            fmt->i_x_offset = delta_y;
-            fmt->i_y_offset = delta_x;
-            break;
-    }
-
     fmt->orientation = dst_orient;
 }