[vlc-commits] commit: transcode: reduce decoder aspect ratio fraction ( Rafaël Carré )

[git at videolan.org]

vlc | branch: master | Rafaël Carré <rafael.carre at gmail.com> | Thu Nov 18 11:25:58 2010 +0100| [4cc2404ca5426f522822e4906572bfa7813669cd] | committer: Rafaël Carré 

transcode: reduce decoder aspect ratio fraction

avoids integer overflow with aspect ratio = BIG:BIG
fixes #4312

> http://git.videolan.org/gitweb.cgi/vlc.git/?a=commit;h=4cc2404ca5426f522822e4906572bfa7813669cd
---

 modules/stream_out/transcode/video.c |    6 ++++++
 1 files changed, 6 insertions(+), 0 deletions(-)

diff --git a/modules/stream_out/transcode/video.c b/modules/stream_out/transcode/video.c
index fcbde83..34a0087 100644
--- a/modules/stream_out/transcode/video.c
+++ b/modules/stream_out/transcode/video.c
@@ -495,6 +495,12 @@ static void transcode_video_encoder_init( sout_stream_t *p_stream,
     if( id->p_encoder->fmt_out.video.i_sar_num <= 0 ||
         id->p_encoder->fmt_out.video.i_sar_den <= 0 )
     {
+        vlc_ureduce( &id->p_decoder->fmt_out.video.i_sar_num,
+                     &id->p_decoder->fmt_out.video.i_sar_den,
+                     id->p_decoder->fmt_out.video.i_sar_num,
+                     id->p_decoder->fmt_out.video.i_sar_den,
+                     0 );
+
         id->p_encoder->fmt_out.video.i_sar_num = id->p_decoder->fmt_out.video.i_sar_num * i_src_width / i_dst_width;
         id->p_encoder->fmt_out.video.i_sar_den = id->p_decoder->fmt_out.video.i_sar_den * i_src_height / i_dst_height;
     }