[vlc-devel] IVTC, Phosphor: 4:2:0 chroma revisited

[Juha Jeronen]

Hi all,

Just something I've been thinking about.

Examining the output from Phosphor with the dimmer off, frame by frame,
seems to indicate that in some cases, the output stream doesn't contain
the (rare) messed-up frames that my traditional IVTC filter outputs.
Where do those come from? At the moment, I suspect a chroma handling bug
somewhere in my original code.

Consider a TFF telecine (these diagrams were written with a fixed-width
font):

 0  2  4  6  8    time (fields)
aa bb cc dd ee    frames as stored on DVD
TB TB TB TB TB    field
11 12 23 33 44    content (film frames "1", "2", "3", and "4")

Note: in 4:2:0 format, each chroma line = 2 luma lines. This includes
how the telecine is done!

E.g. frame "bb" with "TB" = "12" looks like this:

  luma         4:2:0 chroma (represented here in 4:2:2 for easy comparison)
T 11111111     11111111 "T"
B 22222222     11111111 "T"
T 11111111     22222222 "B"
B 22222222     22222222 "B"
T 11111111     11111111 "T"
B 22222222     11111111 "T"
  ...          ...

The "top field" of the chroma for the film frame "2" is in the frame
"cc", just like the true top field of the luma.

Now, consider framerate doubling. Let's assume that we start rendering
at the bottom field for frame "aa":

 B  T  B  T  B  T  B  T  B  T   field being rendered
 0  1  2  3  4  5  6  7  8  9   time (fields)
aa ab bb bc cc cd dd de ee ea   source (DVD frames)
TB BT TB BT TB BT TB BT TB BT   old field / new field
11 11 12 22 23 33 33 34 44 41   content of each field (film frames)
11 MM 12 MM 23 MM 33 MM 44 MM   chroma according to "Merge"
11 12 12 23 23 33 33 44 44 11   chroma according to "Latest"
11 11 12 22 23 33 33 34 44 41   chroma according to "AltLine"
      xx    xx       xx    xx
 p  p  i  p  i  p  p  i  p  i   ...cadence!

Merge: not good, chroma will look messed up at each T field. (Here MM
means to take the full-frame chromas on the left and right of the
symbol, and average them.)

Latest: not good, sometimes chroma is one field ahead.

AltLine: always correct! (Note that actually it alternates every two
lines for 4:2:0, as mentioned above.)

Thus, AltLine is the correct 4:2:0 chroma handling algorithm for
telecined input.

The interlaced frames form a possibility for catching the cadence. The
"p p i p i" sequence is a half-cadence, with a period of five fields.
Every other repetition ends right in the middle of a video frame :)

To do at some point: figure out if this can be used to improve
Phosphate/IVTC2x, and by how much.


Finally, consider pure interlaced 4:2:0 input:

 0  2  4  6  8    time (fields)
aa bb cc dd ee    frames
TB TB TB TB TB    field

Render this with Phosphor (with the darkening filter on). What should we
put into the output chroma?

 B  T  B  T  B  T  B  T  B  T   field being rendered
 0  1  2  3  4  5  6  7  8  9   time (fields)
aa ab bb bc cc cd dd de ee ef   source (frames)
TB BT TB BT TB BT TB BT TB BT   old field / new field
aa MM bb MM cc MM dd MM ee MM   chroma according to "Merge"
aa bb bb cc cc dd dd ee ee ff   chroma according to "Latest"
aa ab bb bc cc cd dd de ee ef   chroma according to "AltLine"

For this case, the algorithm Latest seems the sanest choice, at least if
the dimmer is set to strong enough so that the wrong chroma in the
darkened field won't bother. If it does, then Upconvert - leading to
independent chroma for each field in the output frame - is the ideal
choice (if the machine can keep up).

AltLine is just wrong for this sort of material. (Note that it's still
alternating every two lines, whereas in the mixed frames the luma
alternates every line. So every other bright line will have the wrong
colours.)

This leaves me at a bit of a loss as to which one to set as the default
for Phosphor - Latest or AltLine - since each is correct for a different
kind of input. In practice, Merge often seems to look acceptable, too,
although it really shouldn't.

 -J