[vlc-commits] [Git][videolan/vlc][3.0.x] 4 commits: picture: add plane_/picture_CopyVisiblePixels inline helpers
Steve Lhomme (@robUx4)
gitlab at videolan.org
Tue Jul 7 16:51:42 UTC 2026
Steve Lhomme pushed to branch 3.0.x at VideoLAN / VLC
Commits:
587b4d69 by Thomas Guillem at 2026-07-07T16:22:53+00:00
picture: add plane_/picture_CopyVisiblePixels inline helpers
Move it to vlc_picture.h as static inline so swscale and wall can share
it.
Ideally, we should revert 4594dedf117e40a1ba783ef55df995e12ea8e5fe like
we did in 4.0, but this have many known and unknown consequences, so
leave it for 3.0.
No functional changes.
- - - - -
3212ab77 by Thomas Guillem at 2026-07-07T16:22:53+00:00
wall: copy only the visible tile lines
Similar to 7da4777400949df7388875b6864a2040e6df296a
- - - - -
c812d8f0 by Thomas Guillem at 2026-07-07T16:22:53+00:00
swscale: fix OOB read when scaling a cropped alpha plane
The alpha plane is extracted into a visible-sized GREY picture
(p_src_a), but the shared Convert() re-applied the source crop offset to
it. With a cropped input (e.g. an aspect-ratio thumbnail), sws_scale
started reading p_src_a at that offset and ran lines past the buffer:
SIGSEGV on Windows.
Scale the extracted plane from its own origin via a dedicated ConvertA()
instead of routing it through Convert(), which is meant for the full-size
colour pictures.
Command:
```
./bin/vlc-preparser-static --verbose 2 alpha_codecPNGImage.mov \
--type thumbnail_to_files --seek-pos 0.1 --output-path out.rgba \
--output-width 128 --output-height 64 --output-format rgba --output-crop
```
gdb backtrace on Windows:
```
[11:39:19] "VLC [level 0]: 1920x960 (1920x1080) chroma: RGBA colorspace: BT709 -> 128x64 (128x64) chroma: RGBA colorspace: BT709 with scaling using Bicubic (good quality)"
warning: main generic debug: using video converter module "swscale"
[11:39:19] "VLC [level 0]: using video converter module \"swscale\""
Thread 44 "vlc-run-thumb" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 15320.0x2e10]
0x00007ffbff38c616 in ff_hscale8to15_X4_avx2 () from /c/Users/videolabs/Desktop/tguillem/vlc-4.0.0-dev/plugins/video_chroma/libswscale_plugin.dll
(gdb) bt
```
Valgrind log on Linux:
```
[0000000004e9b170] swscale filter debug: 1920x960 (1920x1080) chroma: RGBA colorspace: BT709 -> 128x64 (128x64) chroma: RGBA colorspace: BT709 with scaling using Bicubic (good quality)
[00000000068e69d0] main generic debug: using video converter module "swscale"
==3471261== Thread 3 vlc-run-thfil:
==3471261== Invalid read of size 8
==3471261== at 0x714A01A: ??? (in /home/tom/work/git/vlc/build/modules/.libs/libswscale_plugin.so)
==3471261== by 0x385: ???
==3471261== by 0x71574BD: lum_h_scale (hscale.c:57)
==3471261== by 0x6FD056E: ff_swscale (swscale.c:501)
==3471261== by 0x6FD1A20: scale_internal (swscale.c:1179)
==3471261== by 0x6FD2A16: sws_scale (swscale.c:1535)
==3471261== by 0x6FA8AC9: Convert (swscale.c:719)
==3471261== by 0x6FA9E67: Filter (swscale.c:787)
==3471261== by 0x4957674: ImageWrite (image.c:431)
==3471261== by 0x49500FC: picture_Export (picture.c:648)
==3471261== by 0x48D59A5: ThumbnailerToFilesRun (internal.c:451)
==3471261== by 0x494A38D: ThreadRun (executor.c:134)
==3471261== Address 0x73eafff is in a rw- mapped file /memfd:vlc-memfd (deleted) segment
==3471261==
==3471261==
==3471261== Process terminating with default action of signal 11 (SIGSEGV)
==3471261== Access not within mapped region at address 0x73EB000
==3471261== at 0x714A01A: ??? (in /home/tom/work/git/vlc/build/modules/.libs/libswscale_plugin.so)
==3471261== by 0x385: ???
==3471261== by 0x71574BD: lum_h_scale (hscale.c:57)
==3471261== by 0x6FD056E: ff_swscale (swscale.c:501)
==3471261== by 0x6FD1A20: scale_internal (swscale.c:1179)
==3471261== by 0x6FD2A16: sws_scale (swscale.c:1535)
==3471261== by 0x6FA8AC9: Convert (swscale.c:719)
==3471261== by 0x6FA9E67: Filter (swscale.c:787)
==3471261== by 0x4957674: ImageWrite (image.c:431)
==3471261== by 0x49500FC: picture_Export (picture.c:648)
==3471261== by 0x48D59A5: ThumbnailerToFilesRun (internal.c:451)
==3471261== by 0x494A38D: ThreadRun (executor.c:134)
```
(cherry picked from commit c6d12114c4c41719271f4110afc04530335f07b6)
Signed-off-by: Thomas Guillem <thomas at gllm.fr>
- - - - -
e927f06d by Thomas Guillem at 2026-07-07T16:22:53+00:00
swscale: extract the alpha plane from the cropped origin
ExtractA() read the source alpha from (0,0), grabbing the uncropped
top-left region, while the colour plane is scaled from the crop offset.
The rescaled alpha therefore did not line up with the colour for cropped
input. Start the extraction (and the planar-alpha copy) at the crop
origin so the alpha matches the colour plane.
(cherry picked from commit bba0df115e9fff686f48b294b5ca803e1144f031)
Signed-off-by: Thomas Guillem <thomas at gllm.fr>
Copy only the visible lines (to behave like 4.0 and fix a potential
other OOB).
- - - - -
4 changed files:
- include/vlc_picture.h
- modules/video_chroma/swscale.c
- modules/video_filter/magnify.c
- modules/video_splitter/wall.c
Changes:
=====================================
include/vlc_picture.h
=====================================
@@ -188,6 +188,61 @@ VLC_API void plane_CopyPixels( plane_t *p_dst, const plane_t *p_src );
*/
VLC_API void picture_Copy( picture_t *p_dst, const picture_t *p_src );
+/**
+ * Copy the visible pixels of a plane.
+ *
+ * Unlike plane_CopyPixels(), which copies whole (padded) i_lines, this copies
+ * only i_visible_lines / i_visible_pitch. Use it when the source p_pixels has
+ * been advanced to a sub-region (a crop or tile origin), where copying the
+ * padding lines would read past the source plane.
+ */
+static inline void plane_CopyVisiblePixels( plane_t *p_dst, const plane_t *p_src )
+{
+ const unsigned i_width = __MIN( p_dst->i_visible_pitch,
+ p_src->i_visible_pitch );
+ const unsigned i_height = __MIN( p_dst->i_visible_lines,
+ p_src->i_visible_lines );
+
+ /* The 2x visible pitch check does two things:
+ 1) Makes field plane_t's work correctly (see the deinterlacer module)
+ 2) Moves less data if the pitch and visible pitch differ much.
+ */
+ if( p_src->i_pitch == p_dst->i_pitch &&
+ p_src->i_pitch < 2*p_src->i_visible_pitch )
+ {
+ memcpy( p_dst->p_pixels, p_src->p_pixels,
+ p_src->i_pitch * i_height );
+ }
+ else
+ {
+ uint8_t *p_in = p_src->p_pixels;
+ uint8_t *p_out = p_dst->p_pixels;
+
+ for( unsigned i_line = i_height; i_line--; )
+ {
+ memcpy( p_out, p_in, i_width );
+ p_in += p_src->i_pitch;
+ p_out += p_dst->i_pitch;
+ }
+ }
+}
+
+/**
+ * Copy the visible pixels and the context of a picture.
+ *
+ * Like picture_CopyPixels() but restricted to the visible pixels (see
+ * plane_CopyVisiblePixels()). Dynamic properties are not copied; use
+ * picture_CopyProperties() for those.
+ */
+static inline void picture_CopyVisiblePixels( picture_t *p_dst, const picture_t *p_src )
+{
+ for( int i = 0; i < p_src->i_planes; i++ )
+ plane_CopyVisiblePixels( &p_dst->p[i], &p_src->p[i] );
+
+ if( p_src->context != NULL )
+ p_dst->context = p_src->context->copy( p_src->context );
+}
+
/**
* Perform a shallow picture copy
*
=====================================
modules/video_chroma/swscale.c
=====================================
@@ -529,14 +529,16 @@ static void GetPixels( uint8_t *pp_pixel[4], int pi_pitch[4],
}
static void ExtractA( picture_t *p_dst, const picture_t *restrict p_src,
- unsigned offset )
+ unsigned x_offset, unsigned y_offset, unsigned offset )
{
plane_t *d = &p_dst->p[0];
const plane_t *s = &p_src->p[0];
+ const uint8_t *src = s->p_pixels + y_offset * s->i_pitch
+ + x_offset * s->i_pixel_pitch + offset;
for( unsigned y = 0; y < p_dst->format.i_height; y++ )
for( unsigned x = 0; x < p_dst->format.i_width; x++ )
- d->p_pixels[y*d->i_pitch+x] = s->p_pixels[y*s->i_pitch+4*x+offset];
+ d->p_pixels[y*d->i_pitch+x] = src[y*s->i_pitch+x*s->i_pixel_pitch];
}
static void InjectA( picture_t *p_dst, const picture_t *restrict p_src,
@@ -557,6 +559,20 @@ static void FillA( plane_t *d, unsigned i_offset )
d->p_pixels[y*d->i_pitch+x+i_offset] = 0xff;
}
+static void ConvertA( struct SwsContext *ctx, plane_t *dst_plane,
+ const plane_t *src_plane, int i_height )
+{
+ /* src/dst are GREY planes that already hold the cropped,
+ * de-offset alpha. Feed swscale their raw pixels. */
+ const uint8_t *src[4] = { src_plane->p_pixels, NULL, NULL, NULL };
+ uint8_t *dst[4] = { dst_plane->p_pixels, NULL, NULL, NULL };
+ const int src_stride[4] = { src_plane->i_pitch, 0, 0, 0 };
+ const int dst_stride[4] = { dst_plane->i_pitch, 0, 0, 0 };
+
+ sws_scale( ctx, src, src_stride, 0, i_height, dst, dst_stride );
+}
+
+
static void CopyPad( picture_t *p_dst, const picture_t *p_src )
{
picture_Copy( p_dst, p_src );
@@ -673,14 +689,21 @@ static picture_t *Filter( filter_t *p_filter, picture_t *p_pic )
{
/* We extract the A plane to rescale it, and then we reinject it. */
if( p_fmti->i_chroma == VLC_CODEC_RGBA || p_fmti->i_chroma == VLC_CODEC_BGRA )
- ExtractA( p_sys->p_src_a, p_src, OFFSET_A );
+ ExtractA( p_sys->p_src_a, p_src, p_fmti->i_x_offset, p_fmti->i_y_offset, OFFSET_A );
else if( p_fmti->i_chroma == VLC_CODEC_ARGB )
- ExtractA( p_sys->p_src_a, p_src, 0 );
+ ExtractA( p_sys->p_src_a, p_src, p_fmti->i_x_offset, p_fmti->i_y_offset, 0 );
else
- plane_CopyPixels( p_sys->p_src_a->p, p_src->p+A_PLANE );
+ {
+ plane_t a = p_src->p[A_PLANE];
+ a.p_pixels += p_fmti->i_y_offset * a.i_pitch
+ + p_fmti->i_x_offset * a.i_pixel_pitch;
+ /* p_pixels was advanced to the cropped origin, so the padding
+ * lines are past the source: copy only the visible lines. */
+ plane_CopyVisiblePixels( p_sys->p_src_a->p, &a );
+ }
- Convert( p_filter, p_sys->ctxA, p_sys->p_dst_a, p_sys->p_src_a,
- p_fmti->i_visible_height, 1, false, false );
+ ConvertA( p_sys->ctxA, &p_sys->p_dst_a->p[0], &p_sys->p_src_a->p[0],
+ p_fmti->i_visible_height );
if( p_fmto->i_chroma == VLC_CODEC_RGBA || p_fmto->i_chroma == VLC_CODEC_BGRA )
InjectA( p_dst, p_sys->p_dst_a, OFFSET_A );
else if( p_fmto->i_chroma == VLC_CODEC_ARGB )
=====================================
modules/video_filter/magnify.c
=====================================
@@ -148,53 +148,6 @@ static void Destroy( vlc_object_t *p_this )
free( p_sys );
}
-static void plane_CopyVisiblePixels( plane_t *p_dst, const plane_t *p_src )
-{
- const unsigned i_width = __MIN( p_dst->i_visible_pitch,
- p_src->i_visible_pitch );
- const unsigned i_height = __MIN( p_dst->i_visible_lines,
- p_src->i_visible_lines );
-
- /* The 2x visible pitch check does two things:
- 1) Makes field plane_t's work correctly (see the deinterlacer module)
- 2) Moves less data if the pitch and visible pitch differ much.
- */
- if( p_src->i_pitch == p_dst->i_pitch &&
- p_src->i_pitch < 2*p_src->i_visible_pitch )
- {
- /* There are margins, but with the same width : perfect ! */
- memcpy( p_dst->p_pixels, p_src->p_pixels,
- p_src->i_pitch * i_height );
- }
- else
- {
- /* We need to proceed line by line */
- uint8_t *p_in = p_src->p_pixels;
- uint8_t *p_out = p_dst->p_pixels;
-
- assert( p_in );
- assert( p_out );
-
- for( int i_line = i_height; i_line--; )
- {
- memcpy( p_out, p_in, i_width );
- p_in += p_src->i_pitch;
- p_out += p_dst->i_pitch;
- }
- }
-}
-
-static void picture_CopyVisiblePixels( picture_t *p_dst, const picture_t *p_src )
-{
- for( int i = 0; i < p_src->i_planes ; i++ )
- plane_CopyVisiblePixels( p_dst->p+i, p_src->p+i );
-
- assert( p_dst->context == NULL );
-
- if( p_src->context != NULL )
- p_dst->context = p_src->context->copy( p_src->context );
-}
-
/*****************************************************************************
* Render: displays previously rendered output
*****************************************************************************/
=====================================
modules/video_splitter/wall.c
=====================================
@@ -418,7 +418,10 @@ static int Filter( video_splitter_t *p_splitter, picture_t *pp_dst[], picture_t
p->p_pixels += i_y * p->i_pitch + i_x * p->i_pixel_pitch;
}
- picture_Copy( p_dst, &tmp );
+ /* Each plane's p_pixels was advanced to the tile origin, so the
+ * padding lines are past the source: copy only the visible tile. */
+ picture_CopyVisiblePixels( p_dst, &tmp );
+ picture_CopyProperties( p_dst, &tmp );
}
}
View it on GitLab: https://code.videolan.org/videolan/vlc/-/compare/468a41f57a630abc00ba926a612b6d5e900eac33...e927f06dd85d4288352056f4dbd8e6460f1f8ae2
--
View it on GitLab: https://code.videolan.org/videolan/vlc/-/compare/468a41f57a630abc00ba926a612b6d5e900eac33...e927f06dd85d4288352056f4dbd8e6460f1f8ae2
You're receiving this email because of your account on code.videolan.org. Manage all notifications: https://code.videolan.org/-/profile/notifications | Help: https://code.videolan.org/help
More information about the vlc-commits
mailing list