[vlc-devel] Soft field repeat?
juha.jeronen at jyu.fi
Fri Jan 7 06:16:28 CET 2011
I've been planning this a bit and I think I now mostly know what needs
to be done on a general level to get it working. (Where to look in the
code is another matter...)
The core should do whatever the stream requests - including computing
PTSs to account for soft field repeat. This needs to be checked that it
I'm currently having a problem with spurious PTS diff variation for some
streams (i.e. variation larger than explained by nb_fields). This needs
to be investigated, as the problem makes correct field rendering
impossible and also affects the smoothness of progressive playback. Some
other streams work just fine, so right now I have no idea...
There are two rendering modes: field-based and progressive. This
distinction basically boils down to whether there is a framerate
doubling deinterlacer in the filter chain. The core doesn't need to know
The field rendering mode is rather simple: we just generate the output
fields in the deinterlacer. If the stream is correctly authored, this
will always do the right thing, as field rendering is the primary way
how interlaced signals were designed to be handled.
To fix the soft field repeat issue, all the framerate doublers must be
taught to handle nb_fields correctly. The private_picture buffer needs
to have at least three frames for the filter (to account for one
repeated field; if we need more, then more). I think it should be
assumed, for generality, that the output fields are unique. (This allows
for temporal visual effects in the deinterlacer.)
As for progressive mode, I suppose I could update my IVTC filter to undo
soft telecine, too, which I imagine is the most common use for soft
field repeat. This is something I overlooked, as I thought no one
actually did soft telecine :)
I think both cases definitely belong in the same filter.
Details: soft telecine can be detected from the flag pattern: nb_fields
= 3,2,3,2,... and TFF = true, false, false, true (TFF telecine) or
false, true, true, false (BFF telecine). Hard telecine usually has
nb_fields = 2 and TFF = constant (i.e. the flags look as if it was an
interlaced stream), although I'm not 110% sure whether this holds for
The cadence detector is easy to update to detect soft telecine, too. The
filter will require two operation modes, since the operations required
to undo hard and soft telecine are different.
I think soft telecine should take precedence if detected, because it's a
clear-cut stream flag thing, and the hard case is based on analyzing the
picture. But to make a final decision, I need to gather TFF/BFF and
nb_fields information from a number of hard-telecined streams, to see
whether real-world usage won't accidentally trigger the soft detector
(which would then do the wrong thing).
In the soft case we don't need to make 5 into 4, because we have the
original film frames already, but the timings should be evened out to
2.5 times original field duration each.
I suppose I could compute the approximate correct frame duration by
taking the mean of the PTS diffs next-curr and curr-prev. For a steady
3:2 sequence, this should be about right. Then I just need to check the
flags for whether the input frame is a "3" or a "2", mangle its
timestamp accordingly ("3": do nothing, "2": make earlier by 0.5 fields
= mean/5), and output the picture as-is.
Note that this too needs the PTS diffs to be about correct in order to
play back smoothly. So getting that right is important.
Streams which have no field repeat, i.e. for which nb_fields is always
2, require no special handling. The existing code works fine for this case.
What about more exotic cases? As I explained, Sol Bianca does the
occasional TFF/BFF flip. It's not strictly speaking a soft telecine. I
still need to debug it, but I suspect the stream wants to repeat
individual fields every now and then.
The field rendering mode, once it understands nb_fields will handle this
case correctly automatically. (My latest Phosphor already does this, but
as I posted, it still stutters due to the spurious PTS variation issue.)
In the progressive mode, we could maybe handle this by lengthening the
duration of the whole frame by repeated_fields/2. If the PTSs are
computed correctly, that should actually already happen automatically.
I'll look into this further when I have the time.
More information about the vlc-devel