[vlc-commits] picture: fix the order of operation for the visible pitch

[Steve Lhomme]

vlc/vlc-3.0 | branch: master | Steve Lhomme <robux4 at ycbcr.xyz> | Wed Apr 18 11:47:08 2018 +0200| [77e27eb67e38c55f5100fd95924ff2a95d4dd81b] | committer: Thomas Guillem

picture: fix the order of operation for the visible pitch

We round the value to the upper multiple first and then multiply and divide it.
Otherwise with semiplanar we end up with odd values for an even source.

(cherry picked from commit edc139506f9c5c226a874e98c63e9f9fd982d1ae)
Signed-off-by: Thomas Guillem <thomas at gllm.fr>

> http://git.videolan.org/gitweb.cgi/vlc/vlc-3.0.git/?a=commit;h=77e27eb67e38c55f5100fd95924ff2a95d4dd81b
---

 src/misc/picture.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/src/misc/picture.c b/src/misc/picture.c
index be51a46d6a..b7eecb4486 100644
--- a/src/misc/picture.c
+++ b/src/misc/picture.c
@@ -192,9 +192,9 @@ int picture_Setup( picture_t *p_picture, const video_format_t *restrict fmt )
         plane_t *p = &p_picture->p[i];
 
         p->i_lines         = (i_height_aligned + i_height_extra ) * p_dsc->p[i].h.num / p_dsc->p[i].h.den;
-        p->i_visible_lines = (fmt->i_visible_height + (p_dsc->p[i].h.den - 1)) * p_dsc->p[i].h.num / p_dsc->p[i].h.den;
+        p->i_visible_lines = (fmt->i_visible_height + (p_dsc->p[i].h.den - 1)) / p_dsc->p[i].h.den * p_dsc->p[i].h.num;
         p->i_pitch         = i_width_aligned * p_dsc->p[i].w.num / p_dsc->p[i].w.den * p_dsc->pixel_size;
-        p->i_visible_pitch = (fmt->i_visible_width + (p_dsc->p[i].w.den - 1)) * p_dsc->p[i].w.num / p_dsc->p[i].w.den * p_dsc->pixel_size;
+        p->i_visible_pitch = (fmt->i_visible_width + (p_dsc->p[i].w.den - 1)) / p_dsc->p[i].w.den * p_dsc->p[i].w.num * p_dsc->pixel_size;
         p->i_pixel_pitch   = p_dsc->pixel_size;
 
         assert( (p->i_pitch % 16) == 0 );