[vlc-devel] Deinterlacer TODO proposal / JJ

[Juha Jeronen]

Hi all,

Based on the recent discussions on this list (thanks Laurent, j-b and Rémi), here's what I'm planning to do at the moment.

The list is presented in two levels - main task plus work estimate, and details. Many of the details are of the "note to self" type, but please see the proposal under item 2).

In the plan, there is a potential wait after each group of patches (while they are being reviewed and reworked), but I think this is necessary, because most of the changes concern the same file and cannot be made code-orthogonal. This will be fixed eventually. See item 5). :)


0) Split the private_picture buffer increase into a separate patch. (Work estimate: a few minutes)

1) Fix the soft field repeat / repeat_pict patch (Laurent's review + my own observations). Aiming to make this the final round of changes for this patch. (Work estimate: 1-2 h)
    - Use a struct for the history stuff (last_dates et al.)
    - Don't reuse "iend"
    - Render as much as possible upon alloc failure (don't release successfully allocated pictures)
    - Check p_dst[i] != NULL in PTS generation
    - Initialize pb_top_field_firsts properly
    - Use a loop for setting b_progressive to true
    - Use a loop for picture_Release() upon failure
    - Format patches and repost.

Once the repeat_pict patch is final, then:

2) Refactor the input frame history mechanism. The exact same history mechanism that was originally developed for RenderYadif() will be needed by IVTC and Phosphor, so it makes sense to refactor it first. (Work estimate: 1-2 h)
    - Move the history update mechanism into Deinterlace()
        - In order not to take a performance hit when the history is not needed, I propose adding a "b_requires_input_history" field to filter_sys_t, which would be filled in SetFilterMethod() (like e.g. b_double_rate currently) depending on whether each algorithm needs it or not. This would also self-document which algorithms use the history.
    - Format patch, post
    - Fix according to review etc.

Once the history patch is final, then:

3) Rework IVTC. (Work estimate: 1-2 days)
    - Take the latest IVTC as base (yes, the fearful hydra including up to patch #7...)
    - Paste changes manually and selectively into a tree with the above patches applied. Refactor on the go as needed.
    - Use refactored history mechanism
    - Refactor cadence detection:
        - One function per detector (no detection code in RenderIVTC() itself)
        - Detectors produce raw data streams (and optionally reliability information)
        - Raw data is filtered and combined by the next layer (RenderIVTC()?) to produce the cadence position
    - Keep other three cadence detectors, but remove the one that may lock on incorrectly (interlace scores based attempt for new "Vektor-like" method).
    - Remove unnecessary debug messages (but keep the useful ones)
    - Review all comments; make sure they match the updated implementation
    - Extend ComposeFrame() to work better with 4:2:0 chroma (in preparation for Phosphor):
        - Allow the caller to specify what to do in case ComposeFrame() detects that the chroma is 4:2:0: whether to use the alternate line copy for chroma, or to copy completely from one input (and specify which). IVTC should use alternate line copy, since it works correctly. Phosphor should always use the input where the currently bright field comes from (this is especially important at the first field of each frame, because then the fields come from different frames).
    - Format a new IVTC patch that applies on top of all the above patches, and repost.

4) Update Phosphor. (Work estimate: 1-2 h)
    - Delegate PTS computation to the new general logic that was added in the updated repeat_pict patch
    - Use updated ComposeFrame() from the updated IVTC patch
    - Use refactored history mechanism from the history patch
    - Format patch, repost.

Once the above two patches are final, then:

5) Split the deinterlacer subsystem sensibly into separate files.
    - Make a subdirectory (in video_filter), move deinterlace.c there
    - Update existing Makefile
    - Keep general stuff in deinterlace.c
    - Keep general-use data in filter_sys_t
    - Move filter_sys_t to header
    - Move each algorithm into its own file
    - Move private data into private structs; add init/uninit API
    - Make a file for general-use helper functions
        - ComposeFrame()
        - Motion detector
        - Blend routines?
    - Test that nothing broke ;)
    - Format patch, post.
(Work estimate: a week?)


 -J