[vlc-devel] [PATCH] mediacodec: fix crash when color-format is invalid

[Martin Storsjö]

On Thu, 23 Oct 2014, Thomas Guillem wrote:

> Fail if GetVlcChromaFormat return 0.
> (It'll throw a HardwareAccelerationError)
> ---
> modules/codec/omxil/android_mediacodec.c | 10 ++++++++--
> 1 file changed, 8 insertions(+), 2 deletions(-)
>
> diff --git a/modules/codec/omxil/android_mediacodec.c b/modules/codec/omxil/android_mediacodec.c
> index 0f291b7..a5e7025 100644
> --- a/modules/codec/omxil/android_mediacodec.c
> +++ b/modules/codec/omxil/android_mediacodec.c
> @@ -842,8 +842,14 @@ static void GetOutput(decoder_t *p_dec, JNIEnv *env, picture_t **pp_pic, jlong t
>                     sar_den = p_dec->fmt_in.video.i_sar_den;
>                 }
>                 jni_SetAndroidSurfaceSizeEnv(env, width, height, width, height, sar_num, sar_den);
> -            } else
> -                GetVlcChromaFormat(p_sys->pixel_format, &p_dec->fmt_out.i_codec, &name);
> +            } else {
> +                if (!GetVlcChromaFormat(p_sys->pixel_format,
> +                                        &p_dec->fmt_out.i_codec, &name)) {
> +                    msg_Err(p_dec, "color-format not recognized");
> +                    p_sys->error_state = true;
> +                    return;
> +                }
> +            }

Looks ok assuming you've tested it.

// Martin