[vlc-devel] [patch] i420_rgb: buffer overflow

[Francois Cartegnie]

Le 22/01/2019 à 18:44, jnqnfe at gmail.com a écrit :
> patch attached
> 
> Incorrect pointer offset calculation in SSE2 (non-assembly version)
> RGB15 unpacking.
> 
> Could, I believe, allow almost 128 bytes to be written past the end of
> the end of the buffer on last loop iteration.

So after investigating,
the only way to trigger that code path is (and probably why it never
happened):

- Build without swscale
- Build without asm tool (CAN_COMPILE_SSE2) but with intrinsics

In the use
- Have some I420 to RV15 conversion (unlikely in display)
- Have unaligned pixels

And it will overflow by 16 bytes at the end of the buffer, only if there
is no alignment.

-- 
Francois Cartegnie
VideoLAN - VLC Developer