[x264-devel] Opaque pointer for nalu processing?

[Killian De Smedt]

On 8 August 2012 00:13, Jason Garrett-Glaser <jason at x264.com> wrote:

> On Tue, Aug 7, 2012 at 3:00 PM, Killian De Smedt <killiands at gmail.com>
> wrote:
> >
> >
> > On 7 August 2012 22:24, Jason Garrett-Glaser <jason at x264.com> wrote:
> >>
> >> On Tue, Aug 7, 2012 at 12:07 PM, Killian De Smedt <killiands at gmail.com>
> >> wrote:
> >> >
> >> >
> >> > On 7 August 2012 19:53, Jason Garrett-Glaser <jason at x264.com> wrote:
> >> >>
> >> >> On Tue, Aug 7, 2012 at 9:06 AM, Killian De Smedt <
> killiands at gmail.com>
> >> >> wrote:
> >> >> >
> >> >> > On 7 August 2012 18:00, Jason Garrett-Glaser <jason at x264.com>
> wrote:
> >> >> >>
> >> >> >> On Tue, Aug 7, 2012 at 4:59 AM, Killian De Smedt
> >> >> >> <killiands at gmail.com>
> >> >> >> wrote:
> >> >> >> > Hello,
> >> >> >> >
> >> >> >> > Today I tried to setup my encoding system using nalu_process to
> >> >> >> > handle
> >> >> >> > NAL
> >> >> >> > unit encoding and transmission. However, my application can
> handle
> >> >> >> > multiple
> >> >> >> > (dynamic) streams, but I have no way to differentiate between
> them
> >> >> >> > in
> >> >> >> > the
> >> >> >> > actual nalu_process call. After some discussion on IRC it turned
> >> >> >> > out
> >> >> >> > I
> >> >> >> > could
> >> >> >> > only do this by creating a new function per stream. While I can
> >> >> >> > use
> >> >> >> > this
> >> >> >> > as
> >> >> >> > a workaround (nr of streams = low), this does seem like a dirty
> >> >> >> > solution
> >> >> >> > and
> >> >> >> > it leads to code bloat because I have to create systems to
> >> >> >> > allocate
> >> >> >> > streams
> >> >> >> > to a static callback.
> >> >> >>
> >> >> >> Have you tried using the x264_t pointer to indicate, or is that
> too
> >> >> >> hacky? I'd be happy to add an extra way if that wasn't enough, but
> >> >> >> I'd
> >> >> >> like to know why that isn't sufficient first.
> >> >> >>
> >> >> >
> >> >> > I thought of this also (mapping the x264 pointer to my stream), but
> >> >> > someone
> >> >> > on IRC (BugMaster) told me that the x264_t pointer I receive in
> >> >> > nalu_process
> >> >> > could differ from the main one used to call encoder_encode, due to
> >> >> > worker
> >> >> > threads. If this is not the case and this value uniquely identifies
> >> >> > my
> >> >> > stream, this solution would be sufficient for me.
> >> >>
> >> >> You're right, that's a problem. It could be solved by violating the
> >> >> encapsulation and looking inside x264_t, but that's not a pretty
> >> >> solution and violates the ABI. How would you prefer I modify the
> >> >> callback so that you know which one it is? I could pass the original
> >> >> x264_t as an argument, for example.
> >> >>
> >> >
> >> > Ideally, it would be great if the `x264_nal_t` structure gets expanded
> >> > with
> >> > a `void* opaque` pointer, that gets copied over from the pic_in (which
> >> > is
> >> > also copied to fenc). In this case the callback interface itself would
> >> > stay
> >> > the same and 'any" data can be passed.
> >> >
> >> > But if that's not possible, any field I can consistently use as a key
> >> > suffices and the original x264_t pointer is as good as any in that
> case.
> >> >
> >> > Thank you,
> >> >
> >> > Killian
> >>
> >> Try this:
> >>
> https://github.com/DarkShikari/x264-devel/commit/e4b8ee321131c5da61569ae538b69bfc6aa9404d
> >>
> >> By the way, thanks for doing testing with this feature -- I only got
> >> to do basic functionality testing because I never got to use it in a
> >> real application, so it's really good to see it actually getting used
> >> and improved to do what people need.
> >>
> >
> > It seems that the opaque pointer I get passed is always set to NULL. I
> saw
> > in the diff that you get the opaque from fdec, but when I take a look at
> how
> > opaque is propagated, I only find these two calls:
>
> Whoops, thanks for catching that, fixed.
>
> This seems to work fine, thank you.

Now I am trying to do the reordering of the nalu's, I assume this is
necessary (annex b used)? Currently (for debugging) I am doing this in the
same app and write the final stream to a file that i try to play with the
totem movie player. This works fine for the encoder_encode version.

My current algorithm is as follows:
if type is not 1,2,3,4,5 (slices) : write to file immediately, I assume
headers are sent synchronously in-order before other nalu's, otherwise this
could not work, because the mb_start/finish values make no sense for them.
Otherwise, buffer until all chunks are received, order by i_mb_start and
write away to file in that order.

This however does not seem to work. The video is started and the correct
length is detected, so it seems like the stream format is recognized, but
the image is unrecognizable, I often end up with just a blank (usually
grey) screen with from time to time a ghost of the image I am encoding.

I tried to reverse the slice nalu's (so from last to first), which
performed a little better, I see more or less the image i want, but with a
srong ghosting effect. It appears as if one horizontal bar (I assume one of
the 5 slices) is decoded correctly and then smeared over the other 4 slices
(I'll try to upload a screenshot). The only thing that seems to solve it a
bit is motion compensation. Most slices stay bad and even IDR's don't solve
that.

Do you have any idea what I am doing wrong?

Thank you,

Killian
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.videolan.org/pipermail/x264-devel/attachments/20120808/9e4800d9/attachment.html>