[x265] [PATCH] Fix Segfault during Reconfigure

[Rob Arrow]

Hi,

We've encountered an issue whereby following the addition of reconfigure in v4.1, it seems that segfaults may occur randomly when performing reconfigure. The issue seems to be caused by uninitialized variables during copying the paramters in the reconfigure. We have debugged the issue and come up with the a patch to fix it. The patch is included at the end of this message.

The issue is that the values of `dst->filmGrain` and `dst-aomFilmGrain` may be uninitialized in `x265_copy_params` if the respective values in `src` are null. When the `src` values are null, no value is assigned the destination leading to uninitialized pointers which may be used later. To fix this, we have taken two steps; firstly we removed the check on the `src` value as this was not adding any value as the pointers are being copied by value rather than being dereferenced. This ensures that even if the `src` values are null, the `dst` values are not left uninitialized. Secondly, we have made sure that the destination is already zero initialized before the copy in `x265_encoder_reconfig`. This seems to be the only location that uses a stack variable, all others appear to be heap allocated with zero memory.

Please let me know if there's anything else I can do to help.

Thanks,
Rob

diff --git a/source/common/param.cpp b/source/common/param.cpp
index a35b06339..7eea045d2 100755
--- a/source/common/param.cpp
+++ b/source/common/param.cpp
@@ -3017,11 +3017,9 @@ void x265_copy_params(x265_param* dst, x265_param* src)
     memcpy(dst->svtHevcParam, src->svtHevcParam, sizeof(EB_H265_ENC_CONFIGURATION));
 #endif
     /* Film grain */
-    if (src->filmGrain)
-        dst->filmGrain = src->filmGrain;
+    dst->filmGrain = src->filmGrain;
     /* Aom Film grain*/
-    if (src->aomFilmGrain)
-        dst->aomFilmGrain = src->aomFilmGrain;
+    dst->aomFilmGrain = src->aomFilmGrain;
     dst->bEnableSBRC = src->bEnableSBRC;
     dst->bConfigRCFrame = src->bConfigRCFrame;
     dst->isAbrLadderEnable = src->isAbrLadderEnable;
diff --git a/source/encoder/api.cpp b/source/encoder/api.cpp
index e89f0cf8d..0a06c6eb3 100644
--- a/source/encoder/api.cpp
+++ b/source/encoder/api.cpp
@@ -309,7 +309,7 @@ int x265_encoder_reconfig(x265_encoder* enc, x265_param* param_in)
 {
     if (!enc || !param_in)
         return -1;
-    x265_param save;
+    x265_param save = {};
     Encoder* encoder = static_cast<Encoder*>(enc);
     if (strlen(encoder->m_param->csvfn) && param_in->csvfpt != NULL)
          encoder->m_param->csvfpt = param_in->csvfpt;



[cid:thumbnail_bfa8ec31-1546-45d1-990c-0de39ea526e7.jfif]<http://www.v-nova.com/><https://www.v-nova.com/perseus-video-compression-technology/#whatis>


Rob Arrow
Software Engineer
T: +44 203 481 4300


20 Eastbourne Terrace
London W2 6LG
www.v-nova.com<http://www.v-nova.com>

[cid:adrianaslinkedin_1ea25bda-8391-4a31-8dce-d6ca3ce29f1f.png]<https://www.linkedin.com/company/2760764/admin/> [cid:AdrianaTwitter_5893815c-bebb-47eb-adf5-461be8d5de73.png] <https://twitter.com/VNovaVideo>


This email and any attachments are sent in strictest confidence for the sole use of the addressee and may contain legally privileged, confidential, proprietary and personal data. If you are not the intended recipient, please advise the sender by replying promptly to this email and then delete and destroy this email and any attachments without any further use, copying or forwarding.

For more information on how we use and protect your information, please review our Privacy Policy here<https://www.v-nova.com/privacy-policy/>

Please consider your environmental responsibility before printing this e-mail
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.videolan.org/pipermail/x265-devel/attachments/20250318/76c95fab/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: thumbnail_bfa8ec31-1546-45d1-990c-0de39ea526e7.jfif
Type: application/octet-stream
Size: 40523 bytes
Desc: thumbnail_bfa8ec31-1546-45d1-990c-0de39ea526e7.jfif
URL: <http://mailman.videolan.org/pipermail/x265-devel/attachments/20250318/76c95fab/attachment-0001.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: adrianaslinkedin_1ea25bda-8391-4a31-8dce-d6ca3ce29f1f.png
Type: image/png
Size: 1488 bytes
Desc: adrianaslinkedin_1ea25bda-8391-4a31-8dce-d6ca3ce29f1f.png
URL: <http://mailman.videolan.org/pipermail/x265-devel/attachments/20250318/76c95fab/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: AdrianaTwitter_5893815c-bebb-47eb-adf5-461be8d5de73.png
Type: image/png
Size: 1471 bytes
Desc: AdrianaTwitter_5893815c-bebb-47eb-adf5-461be8d5de73.png
URL: <http://mailman.videolan.org/pipermail/x265-devel/attachments/20250318/76c95fab/attachment-0003.png>