[vlc-devel] [patch] i420_rgb: buffer overflow

[Steve Lhomme]

In Direct3D11 there's a possible fallback to RGB16 if the source is not 
one of the formats it supports natively and it has less than 16M colors.

Apparently RGB15 is supported by D3D9 too.  But it's only picked if 
YUV/RGB32 and RGB24 fail, so very unlikely.

On 25/01/2019 09:03, Rémi Denis-Courmont wrote:
> Hi,
>
> Is there a use case for this code at all anymore? Do we even actually 
> support rendering onto 8/15/16 bits RGB anymore? Aren't 24-bits the 
> practical minimum nowadays?
>
> Le 25 janvier 2019 09:42:23 GMT+02:00, Steve Lhomme <robux4 at ycbcr.xyz> 
> a écrit :
>
>     On 24/01/2019 18:07, Francois Cartegnie wrote:
>
>         Le 22/01/2019 à 18:44, jnqnfe at gmail.com a écrit :
>
>             patch attached Incorrect pointer offset calculation in
>             SSE2 (non-assembly version) RGB15 unpacking. Could, I
>             believe, allow almost 128 bytes to be written past the end
>             of the end of the buffer on last loop iteration. 
>
>         So after investigating, the only way to trigger that code path
>         is (and probably why it never happened): - Build without
>         swscale - Build without asm tool (CAN_COMPILE_SSE2) but with
>         intrinsics 
>
>
>     So not in our builds
>
>     >
>
>         In the use - Have some I420 to RV15 conversion (unlikely in
>         display) 
>
>
>     With ultra ancient (if any) graphics card (16 bits ones ?).
>
>         - Have unaligned pixels 
>
>
>     Which can happen with some GPU driver allocated memory on Windows. But
>     we don't use that anymore in the display.
>
>     >
>
>         And it will overflow by 16 bytes at the end of the buffer,
>         only if there is no alignment. 
>
>
>     Which can happen on 3.x the visible area is at least a multiple of 16x2
>     pixels (width alignment x line alignment). In 4:2:0 that's divided by 4
>     for U and V planes, so 8 bytes.
>
>     On 4.x the pixel padding is 16x16 so no worries.
>
>     Is there a real case use of this issue ?
>     ------------------------------------------------------------------------
>     vlc-devel mailing list
>     To unsubscribe or modify your subscription options:
>     https://mailman.videolan.org/listinfo/vlc-devel
>
>
> -- 
> Envoyé de mon appareil Android avec Courriel K-9 Mail. Veuillez 
> excuser ma brièveté.
>
> _______________________________________________
> vlc-devel mailing list
> To unsubscribe or modify your subscription options:
> https://mailman.videolan.org/listinfo/vlc-devel