[x264-devel] nalu_process callback does not work when b_sliced_threads=0

[BugMaster]

On Sat, 23 Feb 2019 17:33:45 +0900, Duc Nguyen wrote:
> Dear developers,


> I am trying to use the nalu_process callback to send out NAL units
> as soon as they are encoded for low-latency streaming applications. My implementation is as follows.
> void nalu_process(x264_t *h, x264_nal_t* nal, void* opaque){
>   int first_mb, last_mb;
>   unsigned char* dst = (unsigned char*) malloc(nal->i_payload*3/2 + 5 + 64);
>   x264_nal_encode(h, dst, nal);
>   if(nal->i_type == 6 || nal->i_type == 7 || nal->i_type == 8){
>     first_mb = 0;
>     last_mb = 0;
>   }else{
>     first_mb = nal->i_first_mb;
>     last_mb = nal->i_last_mb;
>   }
>   send_packet_to_receiver(sock_send, sock_send_si_other,
> nal->p_payload, nal->i_payload, fof, first_mb, last_mb);
>   if(fof == 1)
>     fof = 0;
> }


> The callback is called four times and I can get 4 NAL units
> consequentially with the following settings. I guess it is 4 because my PC's CPU has 4 cores.
> x264_param_default_preset(param, "ultralow", "zerolatency");
> x264_param_apply_profile(param, "main");
param->>b_sliced_threads=1;


> However, I do not want to use sliced-based threading in my app. So, I change the settings to:
> x264_param_default_preset(param, "ultralow", "zerolatency");
> x264_param_apply_profile(param, "main");
param->>b_sliced_threads=0;
param->>i_sliced_count = 2;


> In this case, the callback is not called at all. How should I
> change the settings to get the callback called if I still want b_sliced_threads = 0.


> Best regards,

Hi. As is said in API documentation (comments in x264.h) for
nalu_process param:
     * This callback does not work with frame-based threads; threads must be disabled
     * or sliced-threads enabled.  This callback also does not work as one would expect
     * with HRD -- since the buffering period SEI cannot be calculated until the frame
     * is finished encoding, it will not be sent via this callback.

This limitation is because frame-threads wouldn't be zero latency
while nalu_process API is mostly oriented to minimize latency even
more. And because with frame-threads you wouldn't be able to know from
which frame this NAL is from (and how they should be in decoder order
due frame reordering).