[x264-devel] Opaque pointer for nalu processing?

[Killian De Smedt]

On 8 August 2012 11:52, Jason Garrett-Glaser <jason at x264.com> wrote:

> > Now I am trying to do the reordering of the nalu's, I assume this is
> > necessary (annex b used)?
>
> Technically yes, but some decoders might work anyways.  I don't think
> libavcodec does, but it shouldn't be horribly difficult to improve it
> to support that.
>
> Even if you don't reorder them server-side, for example, you could
> send out the NALUs one by one, then reorder them on the client and
> send the whole frame to decode at once. Some example guessy numbers:
>
> Example without nalu_process: 20ms encode, 30ms network send, 5ms decode
> (55ms)
> Example with nalu_process: 2ms encode before NALUs are output, 30ms
> network send, 5ms decode (37ms)
> Example with nalu_process, no reordering: 2ms encode before NALUs are
> output, 30ms network send (decode done at the same time) 0.5ms decode
> finish after network send (32.5ms)
>

My original idea was to send out nalu's unordered and reorder them on the
client before decoding, but it would be even better if the decoder could
manage that for me, I might look into libavcodec for that, thank you.

>
> > Currently (for debugging) I am doing this in the
> > same app and write the final stream to a file that i try to play with the
> > totem movie player. This works fine for the encoder_encode version.
> >
> > My current algorithm is as follows:
> > if type is not 1,2,3,4,5 (slices) : write to file immediately, I assume
> > headers are sent synchronously in-order before other nalu's, otherwise
> this
> > could not work, because the mb_start/finish values make no sense for
> them.
> > Otherwise, buffer until all chunks are received, order by i_mb_start and
> > write away to file in that order.
>
> Are you sure all the slices are for the same frame?  If you end up
> with slices from two different frames, this could happen.
>
>
Yes, but I just found my (quite stupid) error, I created a buffer of the
recommended size in x264.h (nal->i_payload*3/2 + 5 + 16.), but forget to
update the size I pass to my serializer after the nal_encode, so I ended up
writing a lot of rubbish padding. I now send the nal->i_payload (after
encoding) to my serializer and everything works as expected.

I had one last question. this is a low latency project for
computer-generated frames, and it could be that after a frame gets encoded
it takes some time before I receive another frame. Therefore it is
essential for me to know (asap) when a frame is fully encoded/received
without having to wait on another frame.

Currently I still look at the nals returned by encoder_encode and use the
largest last_mb value from that to signal what is the last slice of my
frame. This works for now, but is problematic for two reasons, it does not
really follow x264 rules (nals could be invalid when using nalu_process)
and it introduces a further useless message. Is there a way I could know if
a specific nalu is the last slice in a frame? It is certainly not direct
present in the nal_t, but could I possibly derive it from the last_mb value
or the startcode?

Thank you,

Killian
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.videolan.org/pipermail/x264-devel/attachments/20120808/5f8869f2/attachment.html>