[vlc-commits] commit: transcode: reduce decoder aspect ratio fraction ( Rafaël Carré )

[git at videolan.org]

vlc/vlc-1.1 | branch: master | Rafaël Carré <rafael.carre at gmail.com> | Thu Nov 18 11:25:58 2010 +0100| [39caba70757051594d7aa876bd0673248b618f0c] | committer: Rafaël Carré 

transcode: reduce decoder aspect ratio fraction

avoids integer overflow with aspect ratio = BIG:BIG
fixes #4312
(cherry picked from commit 4cc2404ca5426f522822e4906572bfa7813669cd)

Signed-off-by: Rafaël Carré <rafael.carre at gmail.com>

> http://git.videolan.org/gitweb.cgi/vlc/vlc-1.1.git/?a=commit;h=39caba70757051594d7aa876bd0673248b618f0c
---

 modules/stream_out/transcode/video.c |    6 ++++++
 1 files changed, 6 insertions(+), 0 deletions(-)

diff --git a/modules/stream_out/transcode/video.c b/modules/stream_out/transcode/video.c
index 4ec192c..91b1045 100644
--- a/modules/stream_out/transcode/video.c
+++ b/modules/stream_out/transcode/video.c
@@ -435,6 +435,12 @@ static void transcode_video_encoder_init( sout_stream_t *p_stream,
     if( id->p_encoder->fmt_out.video.i_sar_num <= 0 ||
         id->p_encoder->fmt_out.video.i_sar_den <= 0 )
     {
+        vlc_ureduce( &id->p_decoder->fmt_out.video.i_sar_num,
+                     &id->p_decoder->fmt_out.video.i_sar_den,
+                     id->p_decoder->fmt_out.video.i_sar_num,
+                     id->p_decoder->fmt_out.video.i_sar_den,
+                     0 );
+
         id->p_encoder->fmt_out.video.i_sar_num = id->p_decoder->fmt_out.video.i_sar_num * i_src_width / i_dst_width;
         id->p_encoder->fmt_out.video.i_sar_den = id->p_decoder->fmt_out.video.i_sar_den * i_src_height / i_dst_height;
     }