[vlc-devel] [PATCH] core: fix vlc_alloc() overflow detection

[Romain Vimont]

Le dimanche 12 novembre 2017 à 10:35 +0200, Rémi Denis-Courmont a écrit :
> On dimanche 12 novembre 2017 00:10:27 EET Romain Vimont wrote:
> > Note that the new condition is false when size is 0, even if there is no
> > overflow. In that case, directly returning NULL is ok.
> 
> Yes, the formula works fine with addition, not with multiplication.
> 
> > Signed-off-by: Romain Vimont <rom at rom1v.com>
> > ---
> >  include/vlc_common.h | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/include/vlc_common.h b/include/vlc_common.h
> > index ed5ff77730..d860d31bd9 100644
> > --- a/include/vlc_common.h
> > +++ b/include/vlc_common.h
> > @@ -847,7 +847,7 @@ VLC_API bool vlc_ureduce( unsigned *, unsigned *,
> > uint64_t, uint64_t, uint64_t ) VLC_USED VLC_MALLOC
> >  static inline void *vlc_alloc(size_t count, size_t size)
> >  {
> > -    return likely(count * size >= size) ? malloc(count * size) : NULL;
> > +    return likely(size && count <= SIZE_MAX / size) ? malloc(count * size)
> > : NULL; }
> 
> It should be size == 0 || ...

The only difference is that when size is 0, you call malloc(0) instead
of returning NULL directly.

But yes, it might be better semantically, so that the condition tests
only the overflow.

> >  /**************************************************************************
> > ***
> 
> -- 
> Rémi Denis-Courmont
> _______________________________________________
> vlc-devel mailing list
> To unsubscribe or modify your subscription options:
> https://mailman.videolan.org/listinfo/vlc-devel