[x264-devel] Help to debug h264 headers (or video) generation in kernel driver

[Andrey Utkin]

I'm working on Linux kernel driver for multimedia grabber and H264
encoder based on Techwell/Intersil 5864 chip. Of course it will be
open and pushed into upstream kernel.

The device produces H264 encoded data, but it lacks any headers. The
reference driver generates headers and glues frames together in code,
but I failed both reverse-engineering and porting that code. The code
of reference driver is overwhelmingly complicated, and I have no
knowledge how H264 bitstream is formed, that's why my attempts failed.

I have reached a limited success with both setting up the hardware
encoder and forming the headers. You can see the samples of produced
video here:
http://lizard.bluecherry.net/~autkin/tw5864_tiled_video/ntsc_cif.h264
http://lizard.bluecherry.net/~autkin/tw5864_tiled_video/ntsc_d1.h264
http://lizard.bluecherry.net/~autkin/tw5864_tiled_video/pal_cif.h264
http://lizard.bluecherry.net/~autkin/tw5864_tiled_video/pal_d1.h264

Currently it produces "tiled" picture. The width of tiles on these
sample videos is 256 pixels.
I am not sure whether the issue is in hardware settings, or incorrect
headers which lead to partial failure of decoding.
Here is ffplay log for playing back such video:
https://gist.github.com/krieger-od/5269c762a36bcb52b93a . Obviously
some enhancements to headers generation are needed.

Playing with hardware mirroring flags proves that the raw frame gets
caught fully and correctly. But for h264 encoding, it gets garbled.
Also please notice that frame order is incorrect, the motion goes back
and forth.

You can see our latest code for this development at
https://github.com/bluecherrydvr/linux/tree/tw5864/drivers/staging/media/tw5864
(branch tw5864, subdirectory drivers/staging/media/tw5864).

Any input is extremely appreciated.

-- 
Bluecherry developer.