[vlc-devel] [PATCH] omxil: Keep the old frame height as plane size (nSliceHeight)

[Martin Storsjö]

If the decoder returns a new size via the crop rect, but hasn't
explicitly indicated a plane height (via the nSliceHeight parameter),
keep the frame height as plane height instead.

Also do the same check for stride vs width. The decoders that had
problems that the nSliceHeight fixes do set nStride properly, so
that part won't be needed there, but might be needed on some
platform.
---
 modules/codec/omxil/omxil.c |    4 ++++
 1 files changed, 4 insertions(+), 0 deletions(-)

diff --git a/modules/codec/omxil/omxil.c b/modules/codec/omxil/omxil.c
index 80d327e..f7e3c48 100644
--- a/modules/codec/omxil/omxil.c
+++ b/modules/codec/omxil/omxil.c
@@ -460,6 +460,10 @@ static OMX_ERRORTYPE GetPortDefinition(decoder_t *p_dec, OmxPort *p_port,
         omx_error = OMX_GetConfig(p_port->omx_handle, OMX_IndexConfigCommonOutputCrop, &crop_rect);
         if (omx_error == OMX_ErrorNone)
         {
+            if (!def->format.video.nSliceHeight)
+                def->format.video.nSliceHeight = def->format.video.nFrameHeight;
+            if (!def->format.video.nStride)
+                def->format.video.nStride = def->format.video.nFrameWidth;
             p_fmt->video.i_width = crop_rect.nWidth;
             p_fmt->video.i_visible_width = crop_rect.nWidth;
             p_fmt->video.i_height = crop_rect.nHeight;
-- 
1.7.9