[vlc-commits] [Git][videolan/vlc][master] es_format: Prevent integer overflow in video_format_IsSimilar

[Steve Lhomme (@robUx4)]

Steve Lhomme pushed to branch master at VideoLAN / VLC


Commits:
904fbeb9 by Mathias APARICIO at 2026-01-27T09:54:55+00:00
es_format: Prevent integer overflow in video_format_IsSimilar

Oss-fuzz 6290126878867456 highlights a possible integer overflow in
video_format_IsSimilar.

The i_sar_num and i_sar_den are stored as 32 bit unsigned integers
(unsigned int).

The multiplication of theses two terms can exceed the maximum value of a
signed 64-bit integer.

The maximal possible value UINT32_MAX * UINT32_MAX fits in an unsigned
64 bit integer.

Tackles https://code.videolan.org/videolan/vlc/-/issues/29562

- - - - -


1 changed file:

- src/misc/es_format.c


Changes:

=====================================
src/misc/es_format.c
=====================================
@@ -226,8 +226,8 @@ bool video_format_IsSimilar( const video_format_t *f1,
         f1->i_visible_height != f2->i_visible_height ||
         f1->i_x_offset != f2->i_x_offset || f1->i_y_offset != f2->i_y_offset )
         return false;
-    if( (int64_t)f1->i_sar_num * f2->i_sar_den !=
-        (int64_t)f2->i_sar_num * f1->i_sar_den )
+    if( (uint64_t)f1->i_sar_num * f2->i_sar_den !=
+        (uint64_t)f2->i_sar_num * f1->i_sar_den )
         return false;
 
     if( f1->orientation != f2->orientation)



View it on GitLab: https://code.videolan.org/videolan/vlc/-/commit/904fbeb9fae8b921b708004130293136b9918790

-- 
View it on GitLab: https://code.videolan.org/videolan/vlc/-/commit/904fbeb9fae8b921b708004130293136b9918790
You're receiving this email because of your account on code.videolan.org.


VideoLAN code repository instance