[vlc-commits] [Git][videolan/vlc][master] 16 commits: transcode: encoder: close encoder with ops
Felix Paul Kühne (@fkuehne)
gitlab at videolan.org
Wed May 11 07:41:54 UTC 2022
Felix Paul Kühne pushed to branch master at VideoLAN / VLC
Commits:
f7be6711 by Alexandre Janniaux at 2022-05-11T07:03:05+00:00
transcode: encoder: close encoder with ops
Missing part from 5140b25e88d1d4fff014192f5653e981828895fb.
- - - - -
11ee3d5f by Alexandre Janniaux at 2022-05-11T07:03:05+00:00
test: modules: add transcoder test infrastructure
A unit-test for the transcoder code currently is hard to achieve, as it
involves decoders, encoders, filters, sout_stream_t object, and general
transcode code.
This commit introduces an integration test which mocks the extern
components (filters, decoders, encoders) used by the transcode pipeline
and starts this pipeline using the usual input item properties.
The main goals are to:
- check whether no part of the stream is dropped.
- check whether push model with video context is supported everywhere.
- check the different kind of decoder behaviour, in particular those
for which decode is asynchronous with the decoder_QueueVideo call,
those for which Open() is asynchronous with the decoder_UpdateVideo*
call and their synchronous variants.
- check memory leaks and use-after-free in the pipeline (through asan).
- check threading within the pipeline (through tsan).
- check decoder, encoder and filter loading failure.
- check format adaptation through filters before the encoder.
- check format conversion (called final_conv_static) before the
encoder, needed for encoders requesting a conversion.
- - - - -
39fd9245 by Alexandre Janniaux at 2022-05-11T07:03:05+00:00
transcode: video: use vlc_object_t for debug format
- - - - -
f84b3212 by Alexandre Janniaux at 2022-05-11T07:03:05+00:00
transcode: forward the sout_stream_t object
The object is needed to forward the elementary streams and data to the
sout stream components following the transcode.
Note that having the stream object here means that we must ensure we
don't call it after the transcode is supposed to be closed, so we must
ensure every asynchronous components are drained and destroyed before
leaving the close functions.
The constraint was already there though, given p_obj actually pointed to
the p_stream object too.
- - - - -
b1278274 by Alexandre Janniaux at 2022-05-11T07:03:05+00:00
transcode: video: move some definition up
They will be reused differently when refactoring the initialization
later at the format update site.
- - - - -
7ed34d69 by Alexandre Janniaux at 2022-05-11T07:03:05+00:00
transcode: handle id->b_error
If transcode failed, we want to forward the error to the stream output
pipeline, so we need to store it for later and to abort the transcoding
process.
- - - - -
387a691a by Alexandre Janniaux at 2022-05-11T07:03:05+00:00
transcode: video: check before cleaning
In future commits, we'll create the filter chain and encoder as soon as
the decoder outputs the format it will generate through
decoder_UpdateVideoOutput. It means that if the decoder doesn't report
any format, we won't have an encoder anymore.
- - - - -
8177a1d6 by Alexandre Janniaux at 2022-05-11T07:03:05+00:00
transcode: video: refactor initialization to format update
Some decoders are pushing format update synchronously during their
activation but others have an asynchronous behaviour and may push it
before their activation function has finished, or even after the first
decode call when the decoder is asynchronously configured and ready.
Thus, previously, the decoder format from the decoder was not
necessarily known when opening the filters and the encoder. This
patchset shifts the initialization of previously mentioned objects to
the decoder update callback.
In addition, encoders will now need to get the final video context to
configure and it cannot open correctly without it, so the encoder test
typically makes no sense anymore for video. In this patch, since we
create the encoder only when the final format is finally supplied by the
decoder, we can directly create the encoder that will be used for
transcoding.
In addition, since format update would still happen at activation for
the synchronous decoders, the format and encoders will already be
available, effectively replacing the encoder test afterwards.
- - - - -
79902c34 by Alexandre Janniaux at 2022-05-11T07:03:05+00:00
test: transcode: add error checking case
This test scenario for the transcode checks that the error state is
correctly reported back to the stream output pipeline even if it
happened asynchronously.
- - - - -
b04216b2 by Alexandre Janniaux at 2022-05-11T07:03:05+00:00
transcode: encoder: avoid format update after open
When the encoder is opened, it cannot change format anymore, and we
cannot change it behind its back.
- - - - -
9722e11c by Alexandre Janniaux at 2022-05-11T07:03:05+00:00
transcode: video: remove unused functions
- - - - -
d8c883e2 by Alexandre Janniaux at 2022-05-11T07:03:05+00:00
transcode: fix asynchronous decoder encoding
Some decoders have the decoder_QueueVideo called asynchronously wrt. the
decode() call, which is allowed and expected by the API. The MediaCodec
decoder also has a limited number of input buffers and output buffers
when mediacodec-dr is enabled.
This creates a strong backpressure linking the output buffers to the
input ones, meaning that no input buffer can be decoded unless the
already submitted output buffers are processed and released.
By running the decode() call followed by the processing of output
buffers at the same location, we're linking the output buffers
processing to the execution of the decode() call: no processing of
outputs can happen until decode() hasn't returned.
Except that decode() can be called to saturate the input buffers while
the output was already saturated, leading it to a non deterministic
deadlock.
By processing the buffer synchronously with the decoder_QueueVideo()
call, we ensure that the processing of the picture happens regardless of
the decode() blocking state. It will typically call the filter and
encode callbacks from the thread of the asynchronous module or from the
decode callback itself for synchronous decoders, leading to either the
same behaviour or difference wrt. depth-first vs breadth-first
processing at the decoder level (do we decode all the pictures we can or
do we process every new picture before decoding the others).
The refactor is made so that the actual processing function is isolated
and can be reused in a latter refactor involving a new thread if needed
to bring those difference in execution.
The previous transcode function is still handling the decoder and
encoder drain(), waiting for the decoder to queue all the video frames
and then waiting for the encoder to output every packets.
It fixes a systematic non-deterministic deadlock happening when having
the MediaCodec decoder in direct rendering mode within the transcode
pipeline, when implementing #25153.
- - - - -
41600043 by Alexandre Janniaux at 2022-05-11T07:03:05+00:00
transcode: remove unused transcode_dequeue_all_pics
Pictures are now pushed towards the encoder directly and not queued for
encoding.
- - - - -
77cfe38a by Alexandre Janniaux at 2022-05-11T07:03:05+00:00
transcode: encoder: remove encoder_test function
The encoder test is not used anymore, we'll try to open the encoder at
the moment we have something to encode and using the output of filters.
- - - - -
a685b7ea by Alexandre Janniaux at 2022-05-11T07:03:05+00:00
test: modules: add transcoder test
A unit-test for the transcoder code currently is hard to achieve, as it
involves decoders, encoders, filters, sout_stream_t object, and general
transcode code.
This commit introduces an integration test which mocks the extern
components (filters, decoders, encoders) used by the transcode pipeline
and starts this pipeline using the usual input item properties.
The main end goals are to:
- check whether no part of the stream is dropped.
- check whether push model with video context is supported everywhere.
- check the different kind of decoder behaviour, in particular those
for which decode is asynchronous with the decoder_QueueVideo call,
those for which Open() is asynchronous with the decoder_UpdateVideo*
call and their synchronous variants.
- check memory leaks and use-after-free in the pipeline (through asan).
- check threading within the pipeline (through tsan).
- check decoder, encoder and filter loading failure.
- check format adaptation through filters before the encoder.
- check format conversion (called final_conv_static) before the
encoder, needed for encoders requesting a conversion.
It's currently written in an asynchronous state way spread across
multiple feature functions to handle the different stage of the pipeline
without interaction, like it was done for the video output test, but we
might want to find a way to rewrite those tests in a more sequential way
if it were to get more complex even.
- - - - -
5e202180 by Alexandre Janniaux at 2022-05-11T07:03:05+00:00
transcode_scenario: add encoder_close entrypoint
The endpoint checks that the encoder is correctly closed when reaching
the end of the test. Such failure typically happened during the refactor
of encoders function pointers into a separate virtual table.
- - - - -
9 changed files:
- modules/stream_out/transcode/encoder/encoder.c
- modules/stream_out/transcode/encoder/video.c
- modules/stream_out/transcode/transcode.c
- modules/stream_out/transcode/transcode.h
- modules/stream_out/transcode/video.c
- test/Makefile.am
- + test/modules/stream_out/transcode.c
- + test/modules/stream_out/transcode.h
- + test/modules/stream_out/transcode_scenarios.c
Changes:
=====================================
modules/stream_out/transcode/encoder/encoder.c
=====================================
@@ -127,15 +127,18 @@ const es_format_t *transcode_encoder_format_out( const transcode_encoder_t *p_en
void transcode_encoder_update_format_in( transcode_encoder_t *p_enc, const es_format_t *fmt,
const transcode_encoder_config_t *p_cfg )
{
- es_format_Clean( &p_enc->p_encoder->fmt_in );
- es_format_Copy( &p_enc->p_encoder->fmt_in, fmt );
- switch (fmt->i_cat)
+ if ( !transcode_encoder_opened( p_enc ) )
{
- case VIDEO_ES:
- transcode_encoder_video_set_src(p_enc->p_encoder, &fmt->video, p_cfg);
- break;
- default:
- break;
+ es_format_Clean( &p_enc->p_encoder->fmt_in );
+ es_format_Copy( &p_enc->p_encoder->fmt_in, fmt );
+ switch (fmt->i_cat)
+ {
+ case VIDEO_ES:
+ transcode_encoder_video_set_src(p_enc->p_encoder, &fmt->video, p_cfg);
+ break;
+ default:
+ break;
+ }
}
}
@@ -239,9 +242,6 @@ int transcode_encoder_test( encoder_t *p_encoder,
switch ( p_dec_fmtin->i_cat )
{
- case VIDEO_ES:
- return transcode_encoder_video_test( p_encoder, p_cfg, p_dec_fmtin,
- i_codec_in, p_enc_wanted_in );
case AUDIO_ES:
return transcode_encoder_audio_test( p_encoder, p_cfg, p_dec_fmtin,
i_codec_in, p_enc_wanted_in );
=====================================
modules/stream_out/transcode/encoder/video.c
=====================================
@@ -295,58 +295,6 @@ void transcode_encoder_video_configure( vlc_object_t *p_obj,
(const char *)&p_enc_in->i_chroma);
}
-int transcode_encoder_video_test( encoder_t *p_encoder,
- const transcode_encoder_config_t *p_cfg,
- const es_format_t *p_dec_fmtin,
- vlc_fourcc_t i_codec_in,
- es_format_t *p_enc_wanted_in )
-{
- p_encoder->i_threads = p_cfg->video.threads.i_count;
- p_encoder->p_cfg = p_cfg->p_config_chain;
- p_encoder->ops = NULL;
-
- es_format_Init( &p_encoder->fmt_in, VIDEO_ES, i_codec_in );
- es_format_Init( &p_encoder->fmt_out, VIDEO_ES, p_cfg->i_codec );
-
- const video_format_t *p_dec_in = &p_dec_fmtin->video;
- video_format_t *p_vfmt_in = &p_encoder->fmt_in.video;
- video_format_t *p_vfmt_out = &p_encoder->fmt_out.video;
-
- /* Requested output */
- p_encoder->fmt_out.i_bitrate = p_cfg->video.i_bitrate;
-
- /* The dimensions will be set properly later on.
- * Just put sensible values so we can test an encoder is available. */
- /* Input */
- p_vfmt_in->i_chroma = i_codec_in;
- transcode_video_size_config_apply(VLC_OBJECT(p_encoder), p_dec_in, p_cfg, p_vfmt_in);
- p_vfmt_in->i_frame_rate = ENC_FRAMERATE;
- p_vfmt_in->i_frame_rate_base = ENC_FRAMERATE_BASE;
-
- p_vfmt_out->i_width = p_vfmt_in->i_width & ~1;
- p_vfmt_out->i_height = p_vfmt_in->i_height & ~1;
-
- module_t *p_module = module_need( p_encoder, "video encoder", p_cfg->psz_name, true );
- if( !p_module )
- {
- msg_Err( p_encoder, "cannot find video encoder (module:%s fourcc:%4.4s). "
- "Take a look few lines earlier to see possible reason.",
- p_cfg->psz_name ? p_cfg->psz_name : "any",
- (char *)&p_cfg->i_codec );
- }
-
- if( likely(!p_encoder->fmt_in.video.i_chroma) ) /* always missing, and required by filter chain */
- p_encoder->fmt_in.video.i_chroma = p_encoder->fmt_in.i_codec;
-
- /* output our requested format */
- es_format_Copy( p_enc_wanted_in, &p_encoder->fmt_in );
- video_format_FixRgb( &p_enc_wanted_in->video ); /* set masks when RGB */
-
- vlc_encoder_Destroy(p_encoder);
-
- return p_module != NULL ? VLC_SUCCESS : VLC_EGENERIC;
-}
-
static void* EncoderThread( void *obj )
{
transcode_encoder_t *p_enc = obj;
@@ -437,6 +385,8 @@ void transcode_encoder_video_close( transcode_encoder_t *p_enc )
}
/* Close encoder */
+ if (p_enc->p_encoder->ops->close)
+ p_enc->p_encoder->ops->close(p_enc->p_encoder);
module_unneed( p_enc->p_encoder, p_enc->p_encoder->p_module );
p_enc->p_encoder->p_module = NULL;
}
@@ -470,6 +420,8 @@ int transcode_encoder_video_open( transcode_encoder_t *p_enc,
{
if( vlc_clone( &p_enc->thread, EncoderThread, p_enc ) )
{
+ if (p_enc->p_encoder->ops->close)
+ p_enc->p_encoder->ops->close(p_enc->p_encoder);
module_unneed( p_enc->p_encoder, p_enc->p_encoder->p_module );
p_enc->p_encoder->p_module = NULL;
return VLC_EGENERIC;
=====================================
modules/stream_out/transcode/transcode.c
=====================================
@@ -579,7 +579,7 @@ static void *Add( sout_stream_t *p_stream, const es_format_t *p_fmt )
struct decoder_owner * p_owner = vlc_object_create( p_stream, sizeof( *p_owner ) );
if( !p_owner )
goto error;
- p_owner->p_obj = VLC_OBJECT(p_stream);
+ p_owner->p_stream = p_stream;
id->p_decoder = &p_owner->dec;
decoder_Init( id->p_decoder, p_fmt );
@@ -676,7 +676,10 @@ static void Del( sout_stream_t *p_stream, void *_id )
transcode_audio_clean( p_stream, id );
break;
case VIDEO_ES:
- Send( p_stream, id, NULL );
+ /* Drain if we didn't receive an error, otherwise the
+ * decoder/encoder might not even exist. */
+ if(!id->b_error)
+ Send( p_stream, id, NULL );
decoder_Destroy( id->p_decoder );
vlc_mutex_lock( &p_sys->lock );
if( id == p_sys->id_video )
@@ -738,6 +741,9 @@ static int Send( sout_stream_t *p_stream, void *_id, block_t *p_buffer )
sout_StreamIdSend( p_stream->p_next, id->downstream_id, p_out ) )
i_ret = VLC_EGENERIC;
+ if (i_ret != VLC_SUCCESS)
+ id->b_error = true;
+
return i_ret;
error:
if( p_buffer )
=====================================
modules/stream_out/transcode/transcode.h
=====================================
@@ -79,6 +79,8 @@ struct sout_stream_id_sys_t
const es_format_t *orig,
const es_format_t *current );
+ vlc_fifo_t *output_fifo;
+
/* Decoder */
decoder_t *p_decoder;
@@ -150,7 +152,7 @@ struct sout_stream_id_sys_t
struct decoder_owner
{
decoder_t dec;
- vlc_object_t *p_obj;
+ sout_stream_t *p_stream;
sout_stream_id_sys_t *id;
};
=====================================
modules/stream_out/transcode/video.c
=====================================
@@ -80,9 +80,9 @@ static vlc_decoder_device * video_get_decoder_device( decoder_t *p_dec )
return TranscodeHoldDecoderDevice(&p_dec->obj, p_owner->id);
}
-static void debug_format( sout_stream_t *p_stream, const es_format_t *fmt )
+static void debug_format( vlc_object_t *p_obj, const es_format_t *fmt )
{
- msg_Dbg( p_stream, "format now %4.4s/%4.4s %dx%d(%dx%d) ø%d",
+ msg_Dbg( p_obj, "format now %4.4s/%4.4s %dx%d(%dx%d) ø%d",
(const char *) &fmt->i_codec,
(const char *) &fmt->video.i_chroma,
fmt->video.i_visible_width, fmt->video.i_visible_height,
@@ -90,50 +90,157 @@ static void debug_format( sout_stream_t *p_stream, const es_format_t *fmt )
fmt->video.orientation );
}
+static picture_t *transcode_video_filter_buffer_new( filter_t *p_filter )
+{
+ assert(p_filter->fmt_out.video.i_chroma == p_filter->fmt_out.i_codec);
+ return picture_NewFromFormat( &p_filter->fmt_out.video );
+}
+
static vlc_decoder_device * transcode_video_filter_hold_device(vlc_object_t *o, void *sys)
{
sout_stream_id_sys_t *id = sys;
return TranscodeHoldDecoderDevice(o, id);
}
+static const struct filter_video_callbacks transcode_filter_video_cbs =
+{
+ transcode_video_filter_buffer_new, transcode_video_filter_hold_device,
+};
+
+static int transcode_video_filters_init( sout_stream_t *p_stream,
+ const sout_filters_config_t *p_cfg,
+ const es_format_t *p_src,
+ vlc_video_context *src_ctx,
+ const es_format_t *p_dst,
+ sout_stream_id_sys_t *id );
+
static int video_update_format_decoder( decoder_t *p_dec, vlc_video_context *vctx )
{
struct decoder_owner *p_owner = dec_get_owner( p_dec );
sout_stream_id_sys_t *id = p_owner->id;
- vlc_object_t *p_obj = p_owner->p_obj;
- filter_chain_t *test_chain;
- vlc_mutex_lock( &id->fifo.lock );
-
- const es_format_t *p_enc_in = transcode_encoder_format_in( id->encoder );
+ vlc_mutex_lock(&id->fifo.lock);
+ if( id->encoder != NULL && transcode_encoder_opened( id->encoder ) )
+ {
+ if( video_format_IsSimilar( &p_dec->fmt_out.video, &id->decoder_out.video ) )
+ {
+ vlc_mutex_unlock(&id->fifo.lock);
+ goto end;
+ }
- if( p_enc_in->i_codec == p_dec->fmt_out.i_codec ||
- video_format_IsSimilar( &id->decoder_out.video, &p_dec->fmt_out.video ) )
+ transcode_remove_filters( &id->p_final_conv_static );
+ transcode_remove_filters( &id->p_uf_chain );
+ transcode_remove_filters( &id->p_f_chain );
+ }
+ else if( id->encoder == NULL )
{
- vlc_mutex_unlock( &id->fifo.lock );
- return 0;
+ struct encoder_owner *p_enc_owner =
+ (struct encoder_owner *)sout_EncoderCreate( VLC_OBJECT(p_owner->p_stream), sizeof(struct encoder_owner) );
+ if ( unlikely(p_enc_owner == NULL))
+ return VLC_EGENERIC;
+
+ id->encoder = transcode_encoder_new( &p_enc_owner->enc, &p_dec->fmt_out );
+ if( !id->encoder )
+ {
+ vlc_object_delete( &p_enc_owner->enc );
+ return VLC_EGENERIC;
+ }
+
+ p_enc_owner->id = id;
+ p_enc_owner->enc.cbs = &encoder_video_transcode_cbs;
}
+
es_format_Clean( &id->decoder_out );
es_format_Copy( &id->decoder_out, &p_dec->fmt_out );
-
/* crap, decoders resetting the whole fmtout... */
es_format_SetMeta( &id->decoder_out, &p_dec->fmt_in );
- vlc_mutex_unlock( &id->fifo.lock );
+ if( transcode_video_filters_init( p_owner->p_stream,
+ id->p_filterscfg,
+ &id->decoder_out,
+ vctx,
+ &id->decoder_out, id) != VLC_SUCCESS )
+ {
+ msg_Err(p_dec, "Could not update transcode chain to new format");
+ goto error;
+ }
- msg_Dbg( p_obj, "Checking if filter chain %4.4s -> %4.4s is possible",
- (char *)&p_dec->fmt_out.i_codec, (char*)&p_enc_in->i_codec );
- test_chain = filter_chain_NewVideo( p_obj, false, NULL );
- filter_chain_Reset( test_chain, &p_dec->fmt_out, vctx, p_enc_in );
+ struct vlc_video_context *enc_vctx = NULL;
+ const es_format_t *out_fmt;
- int chain_works = filter_chain_AppendConverter( test_chain, p_enc_in );
- filter_chain_Delete( test_chain );
+ if( id->p_uf_chain )
+ {
+ enc_vctx = filter_chain_GetVideoCtxOut( id->p_uf_chain );
+ out_fmt = filter_chain_GetFmtOut( id->p_uf_chain );
+ }
+ else if( id->p_f_chain )
+ {
+ enc_vctx = filter_chain_GetVideoCtxOut( id->p_f_chain );
+ out_fmt = filter_chain_GetFmtOut( id->p_f_chain );
+ }
+ else
+ {
+ enc_vctx = vctx; /* Decoder video context */
+ out_fmt = &id->decoder_out;
+ }
+
+ if( !transcode_encoder_opened( id->encoder ) )
+ {
+ transcode_encoder_video_configure( VLC_OBJECT(p_owner->p_stream),
+ &id->p_decoder->fmt_out.video,
+ id->p_enccfg,
+ &out_fmt->video,
+ enc_vctx,
+ id->encoder);
+
+ if( transcode_encoder_open( id->encoder, id->p_enccfg ) != VLC_SUCCESS )
+ goto error;
+ }
+
+ const es_format_t *encoder_fmt = transcode_encoder_format_in( id->encoder );
+
+ if( !video_format_IsSimilar(&encoder_fmt->video, &out_fmt->video) )
+ {
+ filter_owner_t chain_owner = {
+ .video = &transcode_filter_video_cbs,
+ .sys = id,
+ };
+
+ if ( !id->p_final_conv_static )
+ id->p_final_conv_static =
+ filter_chain_NewVideo( p_owner->p_stream, false, &chain_owner );
+ filter_chain_Reset( id->p_final_conv_static,
+ out_fmt,
+ enc_vctx,
+ encoder_fmt);
+ if( filter_chain_AppendConverter( id->p_final_conv_static, NULL ) != VLC_SUCCESS )
+ goto error;
+ }
+ vlc_mutex_unlock(&id->fifo.lock);
+
+ if( !id->downstream_id )
+ id->downstream_id =
+ id->pf_transcode_downstream_add( p_owner->p_stream,
+ &id->p_decoder->fmt_in,
+ transcode_encoder_format_out( id->encoder ) );
+ msg_Info( p_dec, "video format update succeed" );
+
+end:
+ return VLC_SUCCESS;
+
+error:
+ transcode_remove_filters( &id->p_final_conv_static );
+
+ if( transcode_encoder_opened( id->encoder ) )
+ transcode_encoder_close( id->encoder );
+
+ transcode_remove_filters( &id->p_uf_chain );
+ transcode_remove_filters( &id->p_f_chain );
- msg_Dbg( p_obj, "Filter chain testing done, input chroma %4.4s seems to be %s for transcode",
- (char *)&p_dec->fmt_out.video.i_chroma,
- chain_works == 0 ? "possible" : "not possible");
- return chain_works;
+ vlc_mutex_unlock( &id->fifo.lock );
+
+ return VLC_EGENERIC;
}
static picture_t *video_new_buffer_encoder( transcode_encoder_t *p_enc )
@@ -141,31 +248,31 @@ static picture_t *video_new_buffer_encoder( transcode_encoder_t *p_enc )
return picture_NewFromFormat( &transcode_encoder_format_in( p_enc )->video );
}
-static picture_t *transcode_video_filter_buffer_new( filter_t *p_filter )
-{
- assert(p_filter->fmt_out.video.i_chroma == p_filter->fmt_out.i_codec);
- return picture_NewFromFormat( &p_filter->fmt_out.video );
-}
+static int transcode_process_picture( sout_stream_id_sys_t *id,
+ picture_t *p_pic, block_t **out);
static void decoder_queue_video( decoder_t *p_dec, picture_t *p_pic )
{
struct decoder_owner *p_owner = dec_get_owner( p_dec );
sout_stream_id_sys_t *id = p_owner->id;
- assert(!picture_HasChainedPics(p_pic));
- vlc_mutex_lock(&id->fifo.lock);
- vlc_picture_chain_Append( &id->fifo.pic, p_pic );
- vlc_mutex_unlock(&id->fifo.lock);
-}
+ block_t *p_block = NULL;
+ int ret = transcode_process_picture( id, p_pic, &p_block );
-static vlc_picture_chain_t transcode_dequeue_all_pics( sout_stream_id_sys_t *id )
-{
- vlc_picture_chain_t p_pics;
- vlc_mutex_lock(&id->fifo.lock);
- vlc_picture_chain_GetAndClear(&id->fifo.pic, &p_pics);
- vlc_mutex_unlock(&id->fifo.lock);
+ if( p_block == NULL )
+ return;
- return p_pics;
+ vlc_fifo_Lock( id->output_fifo );
+ id->b_error |= ret != VLC_SUCCESS;
+ if( id->b_error )
+ {
+ vlc_fifo_Unlock( id->output_fifo );
+ block_ChainRelease( p_block );
+ return;
+ }
+
+ vlc_fifo_QueueUnlocked( id->output_fifo, p_block );
+ vlc_fifo_Unlock( id->output_fifo );
}
int transcode_video_init( sout_stream_t *p_stream, const es_format_t *p_fmt,
@@ -176,6 +283,10 @@ int transcode_video_init( sout_stream_t *p_stream, const es_format_t *p_fmt,
(char*)&p_fmt->i_codec, (char*)&id->p_enccfg->i_codec );
vlc_picture_chain_Init( &id->fifo.pic );
+ id->output_fifo = block_FifoNew();
+ if( id->output_fifo == NULL )
+ return VLC_ENOMEM;
+
id->b_transcode = true;
es_format_Init( &id->decoder_out, VIDEO_ES, 0 );
@@ -211,62 +322,7 @@ int transcode_video_init( sout_stream_t *p_stream, const es_format_t *p_fmt,
es_format_Copy( &id->decoder_out, &id->p_decoder->fmt_out );
}
- /*
- * Open encoder.
- * Because some info about the decoded input will only be available
- * once the first frame is decoded, we actually only test the availability
- * of the encoder here.
- */
-
- /* Should be the same format until encoder loads */
- es_format_t encoder_tested_fmt_in;
- es_format_Init( &encoder_tested_fmt_in, id->decoder_out.i_cat, 0 );
-
- struct encoder_owner *p_enc_owner = (struct encoder_owner*)sout_EncoderCreate(p_stream, sizeof(struct encoder_owner));
- if ( unlikely(p_enc_owner == NULL))
- goto error;
-
- p_enc_owner->id = id;
- p_enc_owner->enc.cbs = &encoder_video_transcode_cbs;
-
- if( transcode_encoder_test( &p_enc_owner->enc,
- id->p_enccfg,
- &id->p_decoder->fmt_in,
- id->p_decoder->fmt_out.i_codec,
- &encoder_tested_fmt_in ) )
- goto error;
-
- p_enc_owner = (struct encoder_owner *)sout_EncoderCreate(p_stream, sizeof(struct encoder_owner));
- if ( unlikely(p_enc_owner == NULL))
- goto error;
-
- id->encoder = transcode_encoder_new( &p_enc_owner->enc, &encoder_tested_fmt_in );
- if( !id->encoder )
- goto error;
-
- p_enc_owner->id = id;
- p_enc_owner->enc.cbs = &encoder_video_transcode_cbs;
-
- es_format_Clean( &encoder_tested_fmt_in );
-
return VLC_SUCCESS;
-
-error:
- module_unneed( id->p_decoder, id->p_decoder->p_module );
- id->p_decoder->p_module = NULL;
- es_format_Clean( &encoder_tested_fmt_in );
- es_format_Clean( &id->decoder_out );
- return VLC_EGENERIC;
-}
-
-static const struct filter_video_callbacks transcode_filter_video_cbs =
-{
- transcode_video_filter_buffer_new, transcode_video_filter_hold_device,
-};
-
-static inline bool transcode_video_filters_configured( const sout_stream_id_sys_t *id )
-{
- return !!id->p_f_chain;
}
static int transcode_video_filters_init( sout_stream_t *p_stream,
@@ -322,8 +378,8 @@ static int transcode_video_filters_init( sout_stream_t *p_stream,
filter_chain_Reset( id->p_uf_chain, p_src, src_ctx, p_dst );
filter_chain_AppendFromString( id->p_uf_chain, p_cfg->psz_filters );
p_src = filter_chain_GetFmtOut( id->p_uf_chain );
- debug_format( p_stream, p_src );
- }
+ debug_format( VLC_OBJECT(p_stream), p_src );
+ }
/* Update encoder so it matches filters output */
transcode_encoder_update_format_in( id->encoder, p_src, id->p_enccfg );
@@ -340,9 +396,12 @@ static int transcode_video_filters_init( sout_stream_t *p_stream,
void transcode_video_clean( sout_stream_id_sys_t *id )
{
- /* Close encoder */
- transcode_encoder_close( id->encoder );
- transcode_encoder_delete( id->encoder );
+ /* Close encoder, but only if one was opened. */
+ if ( id->encoder )
+ {
+ transcode_encoder_close( id->encoder );
+ transcode_encoder_delete( id->encoder );
+ }
es_format_Clean( &id->decoder_out );
@@ -356,6 +415,8 @@ void transcode_video_clean( sout_stream_id_sys_t *id )
spu_Destroy( id->p_spu );
if ( id->dec_dev )
vlc_decoder_device_Release( id->dec_dev );
+
+ block_FifoRelease(id->output_fifo);
}
void transcode_video_push_spu( sout_stream_t *p_stream, sout_stream_id_sys_t *id,
@@ -440,211 +501,71 @@ static void tag_last_block_with_flag( block_t **out, int i_flag )
}
}
-int transcode_video_process( sout_stream_t *p_stream, sout_stream_id_sys_t *id,
- block_t *in, block_t **out )
+static int transcode_process_picture( sout_stream_id_sys_t *id,
+ picture_t *p_pic, block_t **out)
{
- *out = NULL;
-
- bool b_eos = in && (in->i_flags & BLOCK_FLAG_END_OF_SEQUENCE);
-
- int ret = id->p_decoder->pf_decode( id->p_decoder, in );
- if( ret != VLCDEC_SUCCESS )
- return VLC_EGENERIC;
-
- vlc_picture_chain_t p_pics = transcode_dequeue_all_pics( id );
-
- while( !vlc_picture_chain_IsEmpty( &p_pics ) )
+ /* Run the filter and output chains; first with the picture,
+ * and then with NULL as many times as we need until they
+ * stop outputting frames.
+ */
+ for ( picture_t *p_in = p_pic ;; p_in = NULL /* drain second time */ )
{
- picture_t *p_pic = vlc_picture_chain_PopFront( &p_pics );
+ /* Run filter chain */
+ if( id->p_f_chain )
+ p_in = filter_chain_VideoFilter( id->p_f_chain, p_in );
- if( id->b_error && p_pic )
- {
- picture_Release( p_pic );
- continue;
- }
+ if( !p_in )
+ break;
- if( p_pic && ( unlikely(!transcode_encoder_opened(id->encoder)) ||
- !video_format_IsSimilar( &id->decoder_out.video, &p_pic->format ) ) )
+ for( ;; p_in = NULL /* drain second time */ )
{
- if( !transcode_encoder_opened(id->encoder) ) /* Configure Encoder input/output */
+ /* Run user specified filter chain */
+ filter_chain_t * secondary_chains[] = { id->p_uf_chain,
+ id->p_final_conv_static };
+ for( size_t i=0; i<ARRAY_SIZE(secondary_chains); i++ )
{
- assert( !id->p_f_chain && !id->p_uf_chain );
- transcode_encoder_video_configure( VLC_OBJECT(p_stream),
- &id->p_decoder->fmt_out.video,
- id->p_enccfg,
- &p_pic->format,
- picture_GetVideoContext(p_pic),
- id->encoder );
- /* will be opened below */
+ if( !secondary_chains[i] )
+ continue;
+ p_in = filter_chain_VideoFilter( secondary_chains[i], p_in );
}
- else /* picture format has changed */
- {
- msg_Info( p_stream, "aspect-ratio changed, reiniting. %i -> %i : %i -> %i.",
- id->decoder_out.video.i_sar_num, p_pic->format.i_sar_num,
- id->decoder_out.video.i_sar_den, p_pic->format.i_sar_den
- );
- /* Close filters, encoder format input can't change */
- transcode_remove_filters( &id->p_f_chain );
- transcode_remove_filters( &id->p_uf_chain );
- transcode_remove_filters( &id->p_final_conv_static );
- if( id->p_spu_blender )
- filter_DeleteBlend( id->p_spu_blender );
- id->p_spu_blender = NULL;
-
- video_format_Clean( &id->decoder_out.video );
- }
-
- video_format_Copy( &id->decoder_out.video, &p_pic->format );
- transcode_video_framerate_apply( &p_pic->format, &id->decoder_out.video );
- transcode_video_sar_apply( &p_pic->format, &id->decoder_out.video );
-
- if( !transcode_video_filters_configured( id ) )
- {
- if( transcode_video_filters_init( p_stream,
- id->p_filterscfg,
- &id->decoder_out,
- picture_GetVideoContext(p_pic),
- transcode_encoder_format_in( id->encoder ),
- id ) != VLC_SUCCESS )
- goto error;
- }
-
- /* Store the current encoder input chroma to detect whether we need
- * a converter in p_final_conv_static. The encoder will override it
- * if it needs any different format or chroma. */
- es_format_t filter_fmt_out;
- es_format_Copy( &filter_fmt_out, transcode_encoder_format_in( id->encoder ) );
- bool is_encoder_open = transcode_encoder_opened( id->encoder );
- /* Start missing encoder */
- if( !is_encoder_open &&
- transcode_encoder_open( id->encoder, id->p_enccfg ) != VLC_SUCCESS )
- {
- msg_Err( p_stream, "cannot find video encoder (module:%s fourcc:%4.4s). "
- "Take a look few lines earlier to see possible reason.",
- id->p_enccfg->psz_name ? id->p_enccfg->psz_name : "any",
- (char *)&id->p_enccfg->i_codec );
- goto error;
- }
+ if( !p_in )
+ break;
- /* The fmt_in may have been overridden by the encoder. */
- const es_format_t *encoder_fmt_in = transcode_encoder_format_in( id->encoder );
+ /* Blend subpictures */
+ p_in = RenderSubpictures( id, p_in );
- /* check if we need to add a converter between last user filter and encoder. */
- if( filter_fmt_out.i_codec != encoder_fmt_in->i_codec ||
- id->decoder_out.video.i_width != encoder_fmt_in->video.i_width ||
- id->decoder_out.video.i_height != encoder_fmt_in->video.i_height ||
- id->decoder_out.video.i_visible_width != encoder_fmt_in->video.i_visible_width ||
- id->decoder_out.video.i_visible_height != encoder_fmt_in->video.i_visible_height )
- {
- if ( !id->p_final_conv_static )
- id->p_final_conv_static =
- filter_chain_NewVideo( p_stream, false, NULL );
-
- const es_format_t *p_fmt_filtered = &filter_fmt_out;
- es_format_t tmpdst;
- if ( id->p_filterscfg->video.b_reorient &&
- filter_fmt_out.video.orientation != ORIENT_NORMAL )
- {
- es_format_Init( &tmpdst, VIDEO_ES, p_fmt_filtered->video.i_chroma );
- video_format_ApplyRotation( &tmpdst.video, &p_fmt_filtered->video );
- p_fmt_filtered = &tmpdst;
- }
-
- filter_chain_Reset( id->p_final_conv_static,
- &id->decoder_out,
- picture_GetVideoContext(p_pic),
- encoder_fmt_in );
- filter_chain_AppendConverter( id->p_final_conv_static, NULL );
- }
- es_format_Clean(&filter_fmt_out);
-
- msg_Dbg( p_stream, "destination (after video filters) %ux%u",
- transcode_encoder_format_in( id->encoder )->video.i_width,
- transcode_encoder_format_in( id->encoder )->video.i_height );
-
- if( !id->downstream_id )
- id->downstream_id =
- id->pf_transcode_downstream_add( p_stream,
- &id->p_decoder->fmt_in,
- transcode_encoder_format_out( id->encoder ) );
- if( !id->downstream_id )
+ if( p_in )
{
- msg_Err( p_stream, "cannot output transcoded stream %4.4s",
- (char *) &id->p_enccfg->i_codec );
- goto error;
+ /* If a packetizer is used, multiple blocks might be returned, in w */
+ block_t *p_encoded = transcode_encoder_encode( id->encoder, p_in );
+ picture_Release( p_in );
+ block_ChainAppend( out, p_encoded );
}
}
+ }
- /* Run the filter and output chains; first with the picture,
- * and then with NULL as many times as we need until they
- * stop outputting frames.
- */
- for ( picture_t *p_in = p_pic; ; p_in = NULL /* drain second time */ )
- {
- /* Run filter chain */
- if( id->p_f_chain )
- p_in = filter_chain_VideoFilter( id->p_f_chain, p_in );
-
- if( !p_in )
- break;
+ return VLC_SUCCESS;
+}
- for ( ;; p_in = NULL /* drain second time */ )
- {
- /* Run user specified filter chain */
- filter_chain_t * secondary_chains[] = { id->p_uf_chain,
- id->p_final_conv_static };
- for( size_t i=0; p_in && i<ARRAY_SIZE(secondary_chains); i++ )
- {
- if( !secondary_chains[i] )
- continue;
- p_in = filter_chain_VideoFilter( secondary_chains[i], p_in );
- }
-
- if( !p_in )
- break;
-
- /* Blend subpictures */
- p_in = RenderSubpictures( id, p_in );
-
- if( p_in )
- {
- block_t *p_encoded = transcode_encoder_encode( id->encoder, p_in );
- if( p_encoded )
- block_ChainAppend( out, p_encoded );
- picture_Release( p_in );
- }
- }
- }
+int transcode_video_process( sout_stream_t *p_stream, sout_stream_id_sys_t *id,
+ block_t *in, block_t **out )
+{
+ *out = NULL;
- if( b_eos )
- {
- msg_Info( p_stream, "Drain/restart on EOS" );
- if( transcode_encoder_drain( id->encoder, out ) != VLC_SUCCESS )
- goto error;
- transcode_encoder_close( id->encoder );
- /* Close filters */
- transcode_remove_filters( &id->p_f_chain );
- transcode_remove_filters( &id->p_uf_chain );
- transcode_remove_filters( &id->p_final_conv_static );
- tag_last_block_with_flag( out, BLOCK_FLAG_END_OF_SEQUENCE );
- b_eos = false;
- }
+ bool b_eos = in && (in->i_flags & BLOCK_FLAG_END_OF_SEQUENCE);
- continue;
-error:
- if( p_pic )
- picture_Release( p_pic );
- id->b_error = true;
- }
+ int ret = id->p_decoder->pf_decode( id->p_decoder, in );
+ if( ret != VLCDEC_SUCCESS )
+ return VLC_EGENERIC;
- if( id->p_enccfg->video.threads.i_count >= 1 )
- {
- /* Pick up any return data the encoder thread wants to output. */
- block_ChainAppend( out, transcode_encoder_get_output_async( id->encoder ) );
- }
+ /* Only drain if we drained the decoder too. */
+ if (in != NULL)
+ return VLC_SUCCESS;
/* Drain encoder */
+ vlc_fifo_Lock( id->output_fifo );
+ assert(id->encoder);
if( unlikely( !id->b_error && in == NULL ) && transcode_encoder_opened( id->encoder ) )
{
msg_Dbg( p_stream, "Flushing thread and waiting that");
@@ -653,9 +574,16 @@ error:
else
msg_Warn( p_stream, "Flushing failed");
}
+ bool has_error = id->b_error;
+ if( !has_error )
+ {
+ vlc_frame_t *pendings = vlc_fifo_DequeueAllUnlocked( id->output_fifo );
+ block_ChainAppend(out, pendings);
+ }
+ vlc_fifo_Unlock( id->output_fifo );
if( b_eos )
tag_last_block_with_flag( out, BLOCK_FLAG_END_OF_SEQUENCE );
- return id->b_error ? VLC_EGENERIC : VLC_SUCCESS;
+ return has_error ? VLC_EGENERIC : VLC_SUCCESS;
}
=====================================
test/Makefile.am
=====================================
@@ -46,6 +46,7 @@ check_PROGRAMS = \
test_modules_demux_timestamps_filter \
test_modules_demux_ts_pes \
test_modules_playlist_m3u \
+ test_modules_stream_out_transcode \
$(NULL)
if ENABLE_SOUT
@@ -177,6 +178,12 @@ test_src_video_output_LDADD = $(LIBVLCCORE) $(LIBVLC)
test_src_video_output_opengl_SOURCES = src/video_output/opengl.c
test_src_video_output_opengl_LDADD = $(LIBVLCCORE) $(LIBVLC)
+test_modules_stream_out_transcode_SOURCES = \
+ modules/stream_out/transcode.c \
+ modules/stream_out/transcode.h \
+ modules/stream_out/transcode_scenarios.c
+test_modules_stream_out_transcode_LDADD = $(LIBVLCCORE) $(LIBVLC)
+
checkall:
$(MAKE) check_PROGRAMS="$(check_PROGRAMS) $(EXTRA_PROGRAMS)" check
=====================================
test/modules/stream_out/transcode.c
=====================================
@@ -0,0 +1,381 @@
+/*****************************************************************************
+ * transcode.c: test for transcoding pipeline
+ *****************************************************************************
+ * Copyright (C) 2021 VideoLabs
+ *
+ * Author: Alexandre Janniaux <ajanni at videolabs.io>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU Lesser General Public License as published
+ * by the Free Software Foundation; either version 2.1 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public License
+ * along with this program; if not, write to the Free Software Foundation,
+ * Inc., 51 Franklin Street, Fifth Floor, Boston MA 02110-1301, USA.
+ *****************************************************************************/
+
+#ifdef HAVE_CONFIG_H
+# include "config.h"
+#endif
+
+/* Define a builtin module for mocked parts */
+#define MODULE_NAME test_transcode_mock
+#define MODULE_STRING "test_transcode_mock"
+#undef __PLUGIN__
+
+const char vlc_module_name[] = MODULE_STRING;
+
+#include "../../libvlc/test.h"
+#include <vlc_common.h>
+#include <vlc_plugin.h>
+#include <vlc_access.h>
+#include <vlc_demux.h>
+#include <vlc_codec.h>
+#include <vlc_vout_window.h>
+#include <vlc_interface.h>
+#include <vlc_player.h>
+#include <vlc_filter.h>
+#include <vlc_threads.h>
+#include <vlc_sout.h>
+
+#include <limits.h>
+
+#include "transcode.h"
+static size_t current_scenario = 0;
+
+static vlc_cond_t player_cond = VLC_STATIC_COND;
+
+static void DecoderDeviceClose(struct vlc_decoder_device *device)
+ { VLC_UNUSED(device); }
+
+static const struct vlc_decoder_device_operations decoder_device_ops =
+{
+ .close = DecoderDeviceClose,
+};
+
+static int OpenDecoderDevice(
+ struct vlc_decoder_device *device,
+ vout_window_t *window
+) {
+ VLC_UNUSED(window);
+ device->ops = &decoder_device_ops;
+ /* Pick any valid one, we'll not use the module which can make use of
+ * the private parts. */
+ device->type = VLC_DECODER_DEVICE_VAAPI;
+ return VLC_SUCCESS;
+}
+
+static int DecoderDecode(decoder_t *dec, block_t *block)
+{
+ if (block == NULL)
+ return VLC_SUCCESS;
+
+ const picture_resource_t resource = {
+ .p_sys = NULL,
+ };
+ picture_t *pic = picture_NewFromResource(&dec->fmt_out.video, &resource);
+ assert(pic);
+ pic->date = block->i_pts;
+ block_Release(block);
+
+ struct transcode_scenario *scenario = &transcode_scenarios[current_scenario];
+ assert(scenario->decoder_decode != NULL);
+ return scenario->decoder_decode(dec, pic);
+}
+
+static void CloseDecoder(vlc_object_t *obj)
+{
+ decoder_t *dec = (decoder_t*)obj;
+ struct vlc_video_context *vctx = dec->p_sys;
+ if (vctx)
+ vlc_video_context_Release(vctx);
+}
+
+static int OpenDecoder(vlc_object_t *obj)
+{
+ decoder_t *dec = (decoder_t*)obj;
+
+ struct vlc_decoder_device *device = decoder_GetDecoderDevice(dec);
+ assert(device);
+ vlc_decoder_device_Release(device);
+
+ dec->pf_decode = DecoderDecode;
+ // Necessary ?
+ es_format_Clean(&dec->fmt_out);
+ es_format_Copy(&dec->fmt_out, &dec->fmt_in);
+
+ struct transcode_scenario *scenario = &transcode_scenarios[current_scenario];
+ assert(scenario->decoder_setup != NULL);
+ scenario->decoder_setup(dec);
+
+ msg_Dbg(obj, "Decoder chroma %4.4s -> %4.4s size %ux%u",
+ (const char *)&dec->fmt_in.i_codec,
+ (const char *)&dec->fmt_out.i_codec,
+ dec->fmt_out.video.i_width, dec->fmt_out.video.i_height);
+
+ return VLC_SUCCESS;
+}
+
+static int OpenFilter(vlc_object_t *obj)
+{
+ filter_t *filter = (filter_t *)obj;
+
+ static const struct vlc_filter_operations ops = {
+ .filter_video = NULL,
+ .close = NULL,
+ };
+ filter->ops = &ops;
+
+ return VLC_SUCCESS;
+}
+
+static picture_t *ConverterFilter(filter_t *filter, picture_t *input)
+{
+ video_format_Clean(&input->format);
+ video_format_Copy(&input->format, &filter->fmt_out.video);
+ return input;
+}
+
+static int OpenConverter(vlc_object_t *obj)
+{
+ filter_t *filter = (filter_t *)obj;
+
+ msg_Dbg(obj, "converter chroma %4.4s -> %4.4s size %ux%u -> %ux%u",
+ (const char *)&filter->fmt_in.i_codec,
+ (const char *)&filter->fmt_out.i_codec,
+ filter->fmt_in.video.i_width, filter->fmt_in.video.i_height,
+ filter->fmt_out.video.i_width, filter->fmt_out.video.i_height);
+
+ struct transcode_scenario *scenario = &transcode_scenarios[current_scenario];
+ assert(scenario->converter_setup != NULL);
+ scenario->converter_setup(filter);
+
+ static const struct vlc_filter_operations ops = {
+ .filter_video = ConverterFilter,
+ .close = NULL,
+ };
+ filter->ops = &ops;
+
+ return VLC_SUCCESS;
+}
+
+static block_t *EncodeVideo(encoder_t *enc, picture_t *pic)
+{
+ if (pic == NULL)
+ return NULL;
+
+ assert(pic->format.i_chroma == enc->fmt_in.video.i_chroma);
+ block_t *block = block_Alloc(4);
+
+ struct transcode_scenario *scenario = &transcode_scenarios[current_scenario];
+ if (scenario->encoder_encode != NULL)
+ scenario->encoder_encode(enc, pic);
+ return block;
+}
+
+static void CloseEncoder(encoder_t *enc)
+{
+ struct transcode_scenario *scenario = &transcode_scenarios[current_scenario];
+ if (scenario->encoder_close != NULL)
+ scenario->encoder_close(enc);
+}
+
+static int OpenEncoder(vlc_object_t *obj)
+{
+ encoder_t *enc = (encoder_t *)obj;
+ enc->p_sys = NULL;
+
+ struct transcode_scenario *scenario = &transcode_scenarios[current_scenario];
+ assert(scenario->encoder_setup != NULL);
+ scenario->encoder_setup(enc);
+
+ msg_Dbg(obj, "Encoder chroma %4.4s -> %4.4s size %ux%u -> %ux%u",
+ (const char *)&enc->fmt_in.i_codec,
+ (const char *)&enc->fmt_out.i_codec,
+ enc->fmt_in.video.i_width, enc->fmt_in.video.i_height,
+ enc->fmt_out.video.i_width, enc->fmt_out.video.i_height);
+
+ static const struct vlc_encoder_operations ops =
+ {
+ .encode_video = EncodeVideo,
+ .close = CloseEncoder,
+ };
+ enc->ops = &ops;
+
+ return VLC_SUCCESS;
+}
+
+static int ErrorCheckerSend(sout_stream_t *stream, void *id, block_t *b)
+{
+ struct transcode_scenario *scenario = &transcode_scenarios[current_scenario];
+
+ int ret = sout_StreamIdSend(stream->p_next, id, b);
+ if (ret != VLC_SUCCESS)
+ scenario->report_error(stream);
+ return VLC_SUCCESS;
+}
+
+static void* ErrorCheckerAdd(sout_stream_t *stream, const es_format_t *fmt)
+ { return sout_StreamIdAdd(stream->p_next, fmt); }
+
+static void ErrorCheckerDel(sout_stream_t *stream, void *id)
+ { sout_StreamIdDel(stream->p_next, id); };
+
+static int OpenErrorChecker(vlc_object_t *obj)
+{
+ sout_stream_t *stream = (sout_stream_t *)obj;
+ static const struct sout_stream_operations ops = {
+ .add = ErrorCheckerAdd,
+ .del = ErrorCheckerDel,
+ .send = ErrorCheckerSend,
+ };
+ stream->ops = &ops;
+ return VLC_SUCCESS;
+}
+
+static void on_state_changed(vlc_player_t *player, enum vlc_player_state state, void *opaque)
+{
+ (void)player; (void)state; (void) opaque;
+ vlc_cond_signal(&player_cond);
+}
+
+static void play_scenario(intf_thread_t *intf, struct transcode_scenario *scenario)
+{
+ transcode_scenario_init();
+ input_item_t *media = input_item_New(scenario->source, "dummy");
+ assert(media);
+
+ /* TODO: Codec doesn't seem to have effect in transcode:
+ * - add a test that --codec works?
+ * - do not use --codec at all here? */
+ input_item_AddOption(media, scenario->sout, VLC_INPUT_OPTION_TRUSTED);
+
+ var_Create(intf, "codec", VLC_VAR_STRING);
+ var_SetString(intf, "codec", MODULE_STRING);
+
+ var_Create(intf, "sout-transcode-venc", VLC_VAR_STRING);
+ var_SetString(intf, "sout-transcode-venc", MODULE_STRING);
+
+ var_Create(intf, "sout-transcode-vcodec", VLC_VAR_STRING);
+ var_SetString(intf, "sout-transcode-vcodec", "test");
+
+ vlc_player_t *player = vlc_player_New(&intf->obj,
+ VLC_PLAYER_LOCK_NORMAL, NULL, NULL);
+ assert(player);
+
+ static const struct vlc_player_cbs player_cbs = {
+ .on_state_changed = on_state_changed,
+ };
+
+ vlc_player_Lock(player);
+ vlc_player_listener_id *listener =
+ vlc_player_AddListener(player, &player_cbs, NULL);
+ vlc_player_SetCurrentMedia(player, media);
+ vlc_player_Start(player);
+ vlc_player_Unlock(player);
+
+ transcode_scenario_wait(scenario);
+
+ vlc_player_Lock(player);
+ vlc_player_Stop(player);
+
+ while (vlc_player_GetState(player) != VLC_PLAYER_STATE_STOPPED)
+ vlc_player_CondWait(player, &player_cond);
+
+ vlc_player_RemoveListener(player, listener);
+ vlc_player_Unlock(player);
+
+ transcode_scenario_check(scenario);
+
+ vlc_player_Delete(player);
+ input_item_Release(media);
+
+ var_Destroy(intf, "sout-transcode-vcodec");
+ var_Destroy(intf, "sout-transcode-venc");
+}
+
+static int OpenIntf(vlc_object_t *obj)
+{
+ intf_thread_t *intf = (intf_thread_t*)obj;
+
+ while (current_scenario < transcode_scenarios_count)
+ {
+ msg_Info(intf, " - Running transcode scenario %zu", current_scenario);
+ play_scenario(intf, &transcode_scenarios[current_scenario]);
+ current_scenario++;
+ }
+
+ return VLC_SUCCESS;
+}
+
+/**
+ * Inject the mocked modules as a static plugin:
+ * - access for triggering the correct decoder
+ * - decoder for generating video format and context
+ * - filter for generating video format and context
+ * - encoder to check the previous video format and context
+ **/
+vlc_module_begin()
+ set_callbacks(OpenDecoder, CloseDecoder)
+ set_capability("video decoder", INT_MAX)
+
+ add_submodule()
+ set_callback(OpenErrorChecker)
+ set_capability("sout filter", 0)
+ add_shortcut("error_checker")
+
+ add_submodule()
+ set_callback(OpenDecoderDevice)
+ set_capability("decoder device", 0)
+
+ add_submodule()
+ set_callback(OpenFilter)
+ set_capability("video filter", 0)
+
+ add_submodule()
+ set_callback(OpenConverter)
+ set_capability("video converter", INT_MAX)
+
+ add_submodule()
+ set_callback(OpenEncoder)
+ set_capability("video encoder", 0)
+
+ add_submodule()
+ set_callback(OpenIntf)
+ set_capability("interface", 0)
+
+vlc_module_end()
+
+/* Helper typedef for vlc_static_modules */
+typedef int (*vlc_plugin_cb)(vlc_set_cb, void*);
+
+VLC_EXPORT vlc_plugin_cb vlc_static_modules[] = {
+ VLC_SYMBOL(vlc_entry),
+ NULL
+};
+
+int main( int argc, char **argv )
+{
+ (void)argc; (void)argv;
+ test_init();
+
+ const char * const args[] = {
+ "-vvv", "--vout=dummy", "--aout=dummy", "--text-renderer=dummy",
+ "--no-auto-preparse", "--dec-dev=" MODULE_STRING,
+ };
+
+ libvlc_instance_t *vlc = libvlc_new(ARRAY_SIZE(args), args);
+
+ libvlc_add_intf(vlc, MODULE_STRING);
+ libvlc_playlist_play(vlc);
+
+ libvlc_release(vlc);
+ assert(transcode_scenarios_count == current_scenario);
+ return 0;
+}
=====================================
test/modules/stream_out/transcode.h
=====================================
@@ -0,0 +1,50 @@
+/*****************************************************************************
+ * transcode.h: test for transcoding pipeline
+ *****************************************************************************
+ * Copyright (C) 2021 VideoLabs
+ *
+ * Author: Alexandre Janniaux <ajanni at videolabs.io>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU Lesser General Public License as published
+ * by the Free Software Foundation; either version 2.1 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public License
+ * along with this program; if not, write to the Free Software Foundation,
+ * Inc., 51 Franklin Street, Fifth Floor, Boston MA 02110-1301, USA.
+ *****************************************************************************/
+
+#define MODULE_NAME test_transcode_mock
+#define MODULE_STRING "test_transcode_mock"
+#undef __PLUGIN__
+
+#include <vlc_fourcc.h>
+
+#define TEST_FLAG_CONVERTER 0x01
+#define TEST_FLAG_FILTER 0x02
+
+struct transcode_scenario {
+ const char *source;
+ const char *sout;
+ void (*decoder_setup)(decoder_t *);
+ int (*decoder_decode)(decoder_t *, picture_t *);
+ void (*encoder_setup)(encoder_t *);
+ void (*encoder_close)(encoder_t *);
+ void (*encoder_encode)(encoder_t *, picture_t *);
+ void (*filter_setup)(filter_t *);
+ void (*converter_setup)(filter_t *);
+ void (*report_error)(sout_stream_t *);
+};
+
+
+void transcode_scenario_init(void);
+void transcode_scenario_wait(struct transcode_scenario *scenario);
+void transcode_scenario_check(struct transcode_scenario *scenario);
+extern size_t transcode_scenarios_count;
+extern struct transcode_scenario transcode_scenarios[];
=====================================
test/modules/stream_out/transcode_scenarios.c
=====================================
@@ -0,0 +1,381 @@
+/*****************************************************************************
+ * transcode_scenario.c: testflight for transcoding pipeline
+ *****************************************************************************
+ * Copyright (C) 2021 VideoLabs
+ *
+ * Author: Alexandre Janniaux <ajanni at videolabs.io>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU Lesser General Public License as published
+ * by the Free Software Foundation; either version 2.1 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public License
+ * along with this program; if not, write to the Free Software Foundation,
+ * Inc., 51 Franklin Street, Fifth Floor, Boston MA 02110-1301, USA.
+ *****************************************************************************/
+
+#ifdef HAVE_CONFIG_H
+# include "config.h"
+#endif
+
+#define MODULE_NAME test_transcode_mock
+#define MODULE_STRING "test_transcode_mock"
+#undef __PLUGIN__
+
+#include <vlc_common.h>
+#include "transcode.h"
+
+#include <vlc_filter.h>
+
+static struct scenario_data
+{
+ vlc_sem_t wait_stop;
+ struct vlc_video_context *decoder_vctx;
+ unsigned encoder_picture_count;
+ bool converter_opened;
+ bool encoder_opened;
+ bool encoder_closed;
+ bool error_reported;
+} scenario_data;
+
+static void decoder_fixed_size(decoder_t *dec, vlc_fourcc_t chroma,
+ unsigned width, unsigned height)
+{
+ dec->fmt_out.video.i_chroma
+ = dec->fmt_out.i_codec
+ = chroma;
+ dec->fmt_out.video.i_visible_width
+ = dec->fmt_out.video.i_width
+ = width;
+ dec->fmt_out.video.i_visible_height
+ = dec->fmt_out.video.i_height
+ = height;
+}
+
+static void decoder_i420_800_600(decoder_t *dec)
+ { decoder_fixed_size(dec, VLC_CODEC_I420, 800, 600); }
+
+static void decoder_nv12_800_600(decoder_t *dec)
+ { decoder_fixed_size(dec, VLC_CODEC_NV12, 800, 600); }
+
+static void decoder_i420_800_600_vctx(decoder_t *dec)
+{
+ /* We use VLC_VIDEO_CONTEXT_VAAPI here but it could be any other kind of
+ * video context type since we prevent the usual plugins from loading. */
+ struct vlc_video_context *vctx = vlc_video_context_Create(
+ NULL, VLC_VIDEO_CONTEXT_VAAPI, 0, NULL);
+ assert(vctx);
+ dec->p_sys = vctx;
+ decoder_i420_800_600(dec);
+}
+
+static int decoder_decode_dummy(decoder_t *dec, picture_t *pic)
+{
+ int ret = decoder_UpdateVideoOutput(dec, NULL);
+ assert(ret == VLC_SUCCESS);
+ decoder_QueueVideo(dec, pic);
+ return VLC_SUCCESS;
+}
+
+/* Picture context implementation */
+static void picture_context_destroy(struct picture_context_t *ctx)
+ { free(ctx); }
+
+static struct picture_context_t *
+picture_context_copy(struct picture_context_t *ctx)
+{
+ struct picture_context_t *copy = malloc(sizeof *copy);
+ copy = ctx;
+ copy->vctx = vlc_video_context_Hold(ctx->vctx);
+ return copy;
+}
+
+static int decoder_decode_vctx(decoder_t *dec, picture_t *pic)
+{
+ struct vlc_video_context *vctx = dec->p_sys;
+ assert(vctx);
+ scenario_data.decoder_vctx = vctx;
+
+ int ret = decoder_UpdateVideoOutput(dec, vctx);
+ assert(ret == VLC_SUCCESS);
+
+ picture_context_t *context = malloc(sizeof *context);
+ assert(context);
+ context->destroy = picture_context_destroy;
+ context->copy = picture_context_copy;
+ context->vctx = vlc_video_context_Hold(vctx);
+ pic->context = context;
+ pic->format.i_chroma = dec->fmt_out.video.i_chroma;
+ decoder_QueueVideo(dec, pic);
+ return VLC_SUCCESS;
+}
+
+static int decoder_decode_vctx_update(decoder_t *dec, picture_t *pic)
+{
+ bool should_switch = scenario_data.decoder_vctx != NULL;
+
+ if (should_switch)
+ {
+ switch (dec->fmt_out.i_codec)
+ {
+ case VLC_CODEC_I420:
+ msg_Dbg(dec, "Switching from I420 to NV12");
+ dec->fmt_out.video.i_chroma
+ = dec->fmt_out.i_codec
+ = VLC_CODEC_NV12;
+ break;
+ default:
+ break;
+ }
+ }
+ decoder_decode_vctx(dec, pic);
+ return VLC_SUCCESS;
+}
+
+static int decoder_decode_error(decoder_t *dec, picture_t *pic)
+{
+ (void)dec;
+ picture_Release(pic);
+ return VLC_EGENERIC;
+}
+
+static void wait_error_reported(sout_stream_t *stream)
+{
+ (void)stream;
+ vlc_sem_post(&scenario_data.wait_stop);
+}
+
+static void encoder_fixed_size(encoder_t *enc, vlc_fourcc_t chroma,
+ unsigned width, unsigned height)
+{
+ assert(!scenario_data.encoder_opened);
+ msg_Info(enc, "Setting up the encoder %4.4s: %ux%u",
+ (const char *)&chroma, width, height);
+ enc->fmt_in.video.i_chroma
+ = enc->fmt_in.i_codec
+ = chroma;
+ enc->fmt_in.video.i_visible_width
+ = enc->fmt_in.video.i_width
+ = width;
+ enc->fmt_in.video.i_visible_height
+ = enc->fmt_in.video.i_height
+ = height;
+ scenario_data.encoder_opened = true;
+}
+
+static void encoder_i420_800_600(encoder_t *enc)
+ { encoder_fixed_size(enc, VLC_CODEC_I420, 800, 600); }
+
+static void encoder_nv12_800_600(encoder_t *enc)
+ { encoder_fixed_size(enc, VLC_CODEC_NV12, 800, 600); }
+
+static void encoder_i420_800_600_vctx(encoder_t *enc)
+{
+ encoder_fixed_size(enc, VLC_CODEC_I420, 800, 600);
+ assert(scenario_data.decoder_vctx != NULL);
+ assert(enc->vctx_in == scenario_data.decoder_vctx);
+}
+
+#if 0
+static void encoder_nv12_800_600_no_vctx(encoder_t *enc)
+{
+ encoder_fixed_size(enc, VLC_CODEC_NV12, 800, 600);
+ assert(enc->vctx_in == NULL);
+}
+
+static void encoder_i420_800_600_no_vctx(encoder_t *enc)
+{
+ encoder_fixed_size(enc, VLC_CODEC_I420, 800, 600);
+ assert(enc->vctx_in == NULL);
+}
+#endif
+
+static void encoder_encode_dummy(encoder_t *enc, picture_t *pic)
+{
+ (void)enc; (void)pic;
+ msg_Info(enc, "Encode");
+ vlc_sem_post(&scenario_data.wait_stop);
+}
+
+static void encoder_encode_wait_10_images(encoder_t *enc, picture_t *pic)
+{
+ (void)enc; (void)pic;
+ if (scenario_data.encoder_picture_count++ == 10)
+ vlc_sem_post(&scenario_data.wait_stop);
+}
+
+static void encoder_close(encoder_t *enc)
+{
+ (void)enc;
+ scenario_data.encoder_closed = true;
+}
+
+static void converter_fixed_size(filter_t *filter, vlc_fourcc_t chroma_in,
+ vlc_fourcc_t chroma_out, unsigned width, unsigned height)
+{
+ assert(filter->fmt_in.video.i_width == width);
+ assert(filter->fmt_in.video.i_visible_width == width);
+ assert(filter->fmt_in.video.i_height == height);
+ assert(filter->fmt_in.video.i_visible_height == height);
+
+ assert(filter->fmt_out.video.i_width == width);
+ assert(filter->fmt_out.video.i_visible_width == width);
+ assert(filter->fmt_out.video.i_height == height);
+ assert(filter->fmt_out.video.i_visible_height == height);
+
+ assert(filter->fmt_in.video.i_chroma == chroma_in);
+ assert(filter->fmt_out.video.i_chroma == chroma_out);
+
+ scenario_data.converter_opened = true;
+}
+
+static void converter_i420_to_nv12_800_600(filter_t *filter)
+ { converter_fixed_size(filter, VLC_CODEC_I420, VLC_CODEC_NV12, 800, 600); }
+
+static void converter_nv12_to_i420_800_600(filter_t *filter)
+ { converter_fixed_size(filter, VLC_CODEC_NV12, VLC_CODEC_I420, 800, 600); }
+
+static void converter_nv12_to_i420_800_600_vctx(filter_t *filter)
+{
+ converter_fixed_size(filter, VLC_CODEC_NV12, VLC_CODEC_I420, 800, 600);
+ assert(filter->vctx_in == scenario_data.decoder_vctx);
+}
+
+static void converter_i420_to_nv12_800_600_vctx(filter_t *filter)
+{
+ converter_fixed_size(filter, VLC_CODEC_I420, VLC_CODEC_NV12, 800, 600);
+ assert(filter->vctx_in == scenario_data.decoder_vctx);
+}
+
+const char source_800_600[] = "mock://video_track_count=1;length=100000000000;video_width=800;video_height=600";
+struct transcode_scenario transcode_scenarios[] =
+{{
+ .source = source_800_600,
+ .sout = "sout=#transcode:dummy",
+ .decoder_setup = decoder_i420_800_600,
+ .decoder_decode = decoder_decode_dummy,
+ .encoder_setup = encoder_i420_800_600,
+ .encoder_encode = encoder_encode_dummy,
+ .encoder_close = encoder_close,
+},{
+ .source = source_800_600,
+ .sout = "sout=#transcode:dummy",
+ .decoder_setup = decoder_nv12_800_600,
+ .decoder_decode = decoder_decode_dummy,
+ .encoder_setup = encoder_nv12_800_600,
+ .encoder_encode = encoder_encode_dummy,
+ .encoder_close = encoder_close,
+},{
+ .source = source_800_600,
+ .sout = "sout=#transcode:dummy",
+ .decoder_setup = decoder_i420_800_600,
+ .decoder_decode = decoder_decode_dummy,
+ .encoder_setup = encoder_nv12_800_600,
+ .encoder_encode = encoder_encode_dummy,
+ .encoder_close = encoder_close,
+ .converter_setup = converter_i420_to_nv12_800_600,
+},{
+ .source = source_800_600,
+ .sout = "sout=#transcode:dummy",
+ .decoder_setup = decoder_nv12_800_600,
+ .decoder_decode = decoder_decode_dummy,
+ .encoder_setup = encoder_i420_800_600,
+ .encoder_encode = encoder_encode_dummy,
+ .encoder_close = encoder_close,
+ .converter_setup = converter_nv12_to_i420_800_600,
+},{
+ .source = source_800_600,
+ .sout = "sout=#transcode:dummy",
+ .decoder_setup = decoder_i420_800_600_vctx,
+ .decoder_decode = decoder_decode_vctx,
+ .encoder_setup = encoder_i420_800_600_vctx,
+ .encoder_encode = encoder_encode_dummy,
+ .encoder_close = encoder_close,
+},{
+ .source = source_800_600,
+ .sout = "sout=#transcode:dummy",
+ .decoder_setup = decoder_i420_800_600_vctx,
+ .decoder_decode = decoder_decode_vctx,
+ .encoder_setup = encoder_nv12_800_600,
+ .encoder_encode = encoder_encode_dummy,
+ .encoder_close = encoder_close,
+ .converter_setup = converter_i420_to_nv12_800_600,
+},{
+ /* Make sure fps filter in transcode will forward the video context */
+ .source = source_800_600,
+ .sout = "sout=#transcode{fps=1}:dummy",
+ .decoder_setup = decoder_i420_800_600_vctx,
+ .decoder_decode = decoder_decode_vctx,
+ .encoder_setup = encoder_i420_800_600_vctx,
+ .encoder_encode = encoder_encode_dummy,
+ .encoder_close = encoder_close,
+},{
+ // - Decoder format with video context
+ // - Encoder format request a different chroma
+ // - Converter must convert from one to the other
+ // but it doesn't forward any video context
+ /* Make sure converter will receive the video context */
+ .source = source_800_600,
+ .sout = "sout=#transcode:dummy",
+ .decoder_setup = decoder_i420_800_600_vctx,
+ .decoder_decode = decoder_decode_vctx,
+ .encoder_setup = encoder_nv12_800_600,
+ .encoder_encode = encoder_encode_dummy,
+ .encoder_close = encoder_close,
+ .converter_setup = converter_i420_to_nv12_800_600_vctx,
+},{
+ /* Make sure a change in format will lead to the addition of a converter.
+ * Here, decoder_decode_vctx_update will change format after the first
+ * frame. */
+ .source = source_800_600,
+ .sout = "sout=#transcode:dummy",
+ .decoder_setup = decoder_i420_800_600_vctx,
+ .decoder_decode = decoder_decode_vctx_update,
+ .encoder_setup = encoder_i420_800_600,
+ .encoder_encode = encoder_encode_wait_10_images,
+ .encoder_close = encoder_close,
+ .converter_setup = converter_nv12_to_i420_800_600_vctx,
+},{
+ /* Ensure that error are correctly forwarded back to the stream output
+ * pipeline. */
+ .source = source_800_600,
+ .sout = "sout=#error_checker:transcode:dummy",
+ .decoder_setup = decoder_i420_800_600,
+ .decoder_decode = decoder_decode_error,
+ .report_error = wait_error_reported,
+ .encoder_close = encoder_close,
+}};
+size_t transcode_scenarios_count = ARRAY_SIZE(transcode_scenarios);
+
+void transcode_scenario_init(void)
+{
+ scenario_data.decoder_vctx = NULL;
+ scenario_data.encoder_picture_count = 0;
+ scenario_data.converter_opened = false;
+ scenario_data.encoder_opened = false;
+ vlc_sem_init(&scenario_data.wait_stop, 0);
+}
+
+void transcode_scenario_wait(struct transcode_scenario *scenario)
+{
+ (void)scenario;
+ vlc_sem_wait(&scenario_data.wait_stop);
+}
+
+void transcode_scenario_check(struct transcode_scenario *scenario)
+{
+ if (scenario->converter_setup != NULL)
+ assert(scenario_data.converter_opened);
+
+ if (scenario->encoder_setup != NULL)
+ assert(scenario_data.encoder_opened);
+
+ if (scenario_data.encoder_opened && scenario->encoder_close != NULL)
+ assert(scenario_data.encoder_closed);
+}
View it on GitLab: https://code.videolan.org/videolan/vlc/-/compare/24246e4efb1730cfa1919faec2056c6e6f2bfc54...5e202180e0a44084473f500d0ae55ac827d845b4
--
View it on GitLab: https://code.videolan.org/videolan/vlc/-/compare/24246e4efb1730cfa1919faec2056c6e6f2bfc54...5e202180e0a44084473f500d0ae55ac827d845b4
You're receiving this email because of your account on code.videolan.org.
VideoLAN code repository instance
More information about the vlc-commits
mailing list