[vlc-devel] Soft field repeat?

[Juha Jeronen]

Hi all,

I've been planning this a bit and I think I now mostly know what needs 
to be done on a general level to get it working. (Where to look in the 
code is another matter...)


The core should do whatever the stream requests - including computing 
PTSs to account for soft field repeat. This needs to be checked that it 
works correctly.

I'm currently having a problem with spurious PTS diff variation for some 
streams (i.e. variation larger than explained by nb_fields). This needs 
to be investigated, as the problem makes correct field rendering 
impossible and also affects the smoothness of progressive playback. Some 
other streams work just fine, so right now I have no idea...


There are two rendering modes: field-based and progressive. This 
distinction basically boils down to whether there is a framerate 
doubling deinterlacer in the filter chain. The core doesn't need to know 
this.


The field rendering mode is rather simple: we just generate the output 
fields in the deinterlacer. If the stream is correctly authored, this 
will always do the right thing, as field rendering is the primary way 
how interlaced signals were designed to be handled.

To fix the soft field repeat issue, all the framerate doublers must be 
taught to handle nb_fields correctly. The private_picture buffer needs 
to have at least three frames for the filter (to account for one 
repeated field; if we need more, then more). I think it should be 
assumed, for generality, that the output fields are unique. (This allows 
for temporal visual effects in the deinterlacer.)


As for progressive mode, I suppose I could update my IVTC filter to undo 
soft telecine, too, which I imagine is the most common use for soft 
field repeat. This is something I overlooked, as I thought no one 
actually did soft telecine :)

I think both cases definitely belong in the same filter.

Details: soft telecine can be detected from the flag pattern: nb_fields 
= 3,2,3,2,... and TFF = true, false, false, true (TFF telecine) or 
false, true, true, false (BFF telecine). Hard telecine usually has 
nb_fields = 2 and TFF = constant (i.e. the flags look as if it was an 
interlaced stream), although I'm not 110% sure whether this holds for 
all cases.

The cadence detector is easy to update to detect soft telecine, too. The 
filter will require two operation modes, since the operations required 
to undo hard and soft telecine are different.

I think soft telecine should take precedence if detected, because it's a 
clear-cut stream flag thing, and the hard case is based on analyzing the 
picture. But to make a final decision, I need to gather TFF/BFF and 
nb_fields information from a number of hard-telecined streams, to see 
whether real-world usage won't accidentally trigger the soft detector 
(which would then do the wrong thing).

In the soft case we don't need to make 5 into 4, because we have the 
original film frames already, but the timings should be evened out to 
2.5 times original field duration each.

I suppose I could compute the approximate correct frame duration by 
taking the mean of the PTS diffs next-curr and curr-prev. For a steady 
3:2 sequence, this should be about right. Then I just need to check the 
flags for whether the input frame is a "3" or a "2", mangle its 
timestamp accordingly ("3": do nothing, "2": make earlier by 0.5 fields 
= mean/5), and output the picture as-is.

Note that this too needs the PTS diffs to be about correct in order to 
play back smoothly. So getting that right is important.


Streams which have no field repeat, i.e. for which nb_fields is always 
2, require no special handling. The existing code works fine for this case.


What about more exotic cases? As I explained, Sol Bianca does the 
occasional TFF/BFF flip. It's not strictly speaking a soft telecine. I 
still need to debug it, but I suspect the stream wants to repeat 
individual fields every now and then.

The field rendering mode, once it understands nb_fields will handle this 
case correctly automatically. (My latest Phosphor already does this, but 
as I posted, it still stutters due to the spurious PTS variation issue.)

In the progressive mode, we could maybe handle this by lengthening the 
duration of the whole frame by repeated_fields/2. If the PTSs are 
computed correctly, that should actually already happen automatically.


I'll look into this further when I have the time.

  -J