[vlc-commits] [Git][videolan/vlc][3.0.x] rawvid: fix rounding division when calculating pitch

[Jean-Baptiste Kempf (@jbk)]

Jean-Baptiste Kempf pushed to branch 3.0.x at VideoLAN / VLC


Commits:
4b2e6df3 by Pierre Lamot at 2023-02-17T17:19:13+01:00
rawvid: fix rounding division when calculating pitch

formats like NV12 where num and den are 2 had the wrong pitch

size is divided then multiplied, consistently with `rawvideo` decoder

(cherry-picked from commit 128f7ff6e9ee3c33098c2cc9896116374f04eb56)

- - - - -


1 changed file:

- modules/demux/rawvid.c


Changes:

=====================================
modules/demux/rawvid.c
=====================================
@@ -356,10 +356,10 @@ valid:
     p_sys->frame_size = 0;
     for (unsigned i=0; i<dsc->plane_count; i++)
     {
-        unsigned pitch = (i_width + (dsc->p[i].w.den - 1))
-                         * dsc->p[i].w.num / dsc->p[i].w.den * dsc->pixel_size;
-        unsigned lines = (i_height + (dsc->p[i].h.den - 1))
-                         * dsc->p[i].h.num / dsc->p[i].h.den;
+        unsigned pitch = ((i_width + (dsc->p[i].w.den - 1)) / dsc->p[i].w.den)
+                          * dsc->p[i].w.num * dsc->pixel_size;
+        unsigned lines = ((i_height + (dsc->p[i].h.den - 1)) / dsc->p[i].h.den)
+                         * dsc->p[i].h.num;
         p_sys->frame_size += pitch * lines;
     }
     p_sys->p_es_video = es_out_Add( p_demux->out, &p_sys->fmt_video );



View it on GitLab: https://code.videolan.org/videolan/vlc/-/commit/4b2e6df38979f3082a25781893baae75bef79693

-- 
View it on GitLab: https://code.videolan.org/videolan/vlc/-/commit/4b2e6df38979f3082a25781893baae75bef79693
You're receiving this email because of your account on code.videolan.org.


VideoLAN code repository instance