[vlc-devel] [patch] i420_rgb: buffer overflow

[Steve Lhomme]

On 24/01/2019 18:07, Francois Cartegnie wrote:
> Le 22/01/2019 à 18:44, jnqnfe at gmail.com a écrit :
>> patch attached
>>
>> Incorrect pointer offset calculation in SSE2 (non-assembly version)
>> RGB15 unpacking.
>>
>> Could, I believe, allow almost 128 bytes to be written past the end of
>> the end of the buffer on last loop iteration.
> So after investigating,
> the only way to trigger that code path is (and probably why it never
> happened):
>
> - Build without swscale
> - Build without asm tool (CAN_COMPILE_SSE2) but with intrinsics

So not in our builds

>
> In the use
> - Have some I420 to RV15 conversion (unlikely in display)

With ultra ancient (if any) graphics card (16 bits ones ?).

> - Have unaligned pixels

Which can happen with some GPU driver allocated memory on Windows. But 
we don't use that anymore in the display.

>
> And it will overflow by 16 bytes at the end of the buffer, only if there
> is no alignment.

Which can happen on 3.x the visible area is at least a multiple of 16x2 
pixels (width alignment x line alignment). In 4:2:0 that's divided by 4 
for U and V planes, so 8 bytes.

On 4.x the pixel padding is 16x16 so no worries.

Is there a real case use of this issue ?