[vlc-devel] uncertain: whether to develop plugin for a custom codec.

[BGB]

my use-case I suspect is a little non-standard, so I may try to explain 
what it is I am doing here, as to why I would want to write a codec 
plugin for VLC (and not just use standard codecs).

sorry in advance if this is seen as inappropriate.


basically, for my own uses (mostly for use inside a 3D game engine, as 
animated / video textures and similar), I have ended up developing some 
fairly specialized codecs. (pretty much all this is custom-written code).

it is, however, often fairly convenient to be able to quickly view their 
contents in a video player, which is fairly limiting if stuck only with 
building on preexisting codecs (namely: since a standard video player 
will normally only be able to read standard codecs, or at least, ones 
which have visible codec drivers...).

I already tried implementing some of this as a VfW codec driver, but 
couldn't get it to work (no idea even if the driver was being invoked), 
and eventually got frustrated.

started looking into maybe trying to implement it as a VLC plugin 
instead (not done so yet, still don't know if doing so would "actually 
make sense").

the main alternative being to throw together a more specialized video 
player.



side information, or the "what and why" of the matter:

generally, bitrate isn't a particularly huge concern, at least relative 
to speed.

so, many of the codecs are more speed-centric, at the expense of worse 
compression. generally, this is because code is working with multiple 
video streams at a time, and needs to operate in real-time (hurting the 
framerate is ideally something to be avoided).

several of them decode fairly directly into DXTn (compressed textures), 
and generally need to make use of things like alpha channels, extended 
component images (such as normal and specular maps) and layers (in my 
implementation, these are called "component layers" and "tag layers"). 
some of the codecs also support HDR / floating-point textures, ... as well.

generally, pretty much all this ends up mostly handed off to the codec 
(currently with AVI as the primary container format).

(basically, every video frame is sort of like a PhotoShop PSD or GIMP 
XCF file or similar...)..

however, thus far, it has also turned out to be fairly convenient to 
view these files in a video player, even if a lot of the information is 
absent, since basically I can verify the "general contents", and often 
the base-RGB(A) layer is the primary (or sole) layer in the video (or 
contains a "rendered down" version).

this has generally led to sticking with hacked versions of existing 
formats (particularly Motion-JPEG and Apple Video / RPZA), with all the 
extended functionality being built in a way that a normal decoder will 
ignore (APPn markers for JPEG, unknown bytes for RPZA, ...).

(generally, a TLV format is built, and all of the extensions are built 
in terms of this TLV format).


however, in some cases, I am limited to some extent by the capabilities 
of the base-format.

likewise, (after some initial testing) some formats (notably Theora and 
XviD), were not particularly flexible nor well suited to the use-case. 
(yes, they do well at what they do well, still doesn't mean they fit 
this use case particularly well...).

(left out a much longer analysis on the various codecs and my 
experiences with them...).


I was then like, "you know what, I will just devise something more 
specialized" (mostly for the use case of intermediate storage of frame 
data to the HDD), and proceeded to design and implement a codec with a 
design sort of like RPZA mixed with Cinepak, but with more color bits 
(11,9,9 for LDR, and 9,8,8 for HDR with a 4-bit exponent, using a YCoCg 
colorspace with 4:2:0 chroma subsampling, and basic support for an alpha 
channel).

it is mostly tuned for speed and image quality, at the cost of a fairly 
high bitrate.
basically, a fairly niche use-case...


but, then the general issue once again pops up (if using a format like 
this):
I can no longer view the video in a normal video player (which is 
inconvenient), and also lose the thumbnail images (also annoying...).

and, hence, the issue of how to best deal with this?...