[Android] Green line in output

[Felix Abecassis]

Hi everyone,

An infamous problem recently in VLC is green lines at the bottom of videos.
After investigation, it appears we still have such an issue with RGB
and YUV outputs.

Like all green lines problem, this is again related to visible/hidden
dimensions. With SW decoding they are different since avcodec needs a
few lines of padding at the bottom of a video for direct rendering.

At first I thought the problem was only with only with YUV outputs,
because this is not well supported. I started by increasing hidden
dimension excessively. Hidden dimension was made twice the visible
dimension (i.e. the dimension of the video). I had awful results like
this one on Nokia X:
https://docs.google.com/file/d/0B-uo-FB7egGkXzFUOGFUNmkxQ00/
This is a picture of the device because when capturing with vol down +
power, the large green line is gone. However with a Nexus 5, the green
line is still here after capture:
https://docs.google.com/file/d/0B-uo-FB7egGkdmVMbmkyYzdCX1k/

I switched to RGB, keeping the huge hidden dimension, but I added code
to set the whole surface buffer to blue after getting the
NativeWindow. I got the same result as above but with blue instead of
green. Thus we have the same problem with RGB and YUV, except that the
problem is hidden because if the buffer is filled with 0, the lines
are black in RGB mode, but they are green with YUV.
I guess it could also happen to the RGB output mode if the buffer is
not initialized to 0 anymore.

Doubling the hidden dimension might sound unrealistic, but if I keep
only the avcodec padding and my blue fill, we still have a blue line
on some videos. *However*, the blue line disappears when doing screen
even with Nexus 5 whereas it worked before.

It looks like we might be doing something not standard. Or we have a
bug, I went through the code in VideoPlayerActivity but everything
looked correct, any help is welcome here. I suspect this is related to
how we handle surfaces and frames in the Java code. But I might be
wrong.

-- 
Félix Abecassis
http://felix.abecassis.me