[vlc-devel] uncertain: whether to develop plugin for a custom codec.
cr88192 at gmail.com
Fri Oct 18 10:53:55 CEST 2013
my use-case I suspect is a little non-standard, so I may try to explain
what it is I am doing here, as to why I would want to write a codec
plugin for VLC (and not just use standard codecs).
sorry in advance if this is seen as inappropriate.
basically, for my own uses (mostly for use inside a 3D game engine, as
animated / video textures and similar), I have ended up developing some
fairly specialized codecs. (pretty much all this is custom-written code).
it is, however, often fairly convenient to be able to quickly view their
contents in a video player, which is fairly limiting if stuck only with
building on preexisting codecs (namely: since a standard video player
will normally only be able to read standard codecs, or at least, ones
which have visible codec drivers...).
I already tried implementing some of this as a VfW codec driver, but
couldn't get it to work (no idea even if the driver was being invoked),
and eventually got frustrated.
started looking into maybe trying to implement it as a VLC plugin
instead (not done so yet, still don't know if doing so would "actually
the main alternative being to throw together a more specialized video
side information, or the "what and why" of the matter:
generally, bitrate isn't a particularly huge concern, at least relative
so, many of the codecs are more speed-centric, at the expense of worse
compression. generally, this is because code is working with multiple
video streams at a time, and needs to operate in real-time (hurting the
framerate is ideally something to be avoided).
several of them decode fairly directly into DXTn (compressed textures),
and generally need to make use of things like alpha channels, extended
component images (such as normal and specular maps) and layers (in my
implementation, these are called "component layers" and "tag layers").
some of the codecs also support HDR / floating-point textures, ... as well.
generally, pretty much all this ends up mostly handed off to the codec
(currently with AVI as the primary container format).
(basically, every video frame is sort of like a PhotoShop PSD or GIMP
XCF file or similar...)..
however, thus far, it has also turned out to be fairly convenient to
view these files in a video player, even if a lot of the information is
absent, since basically I can verify the "general contents", and often
the base-RGB(A) layer is the primary (or sole) layer in the video (or
contains a "rendered down" version).
this has generally led to sticking with hacked versions of existing
formats (particularly Motion-JPEG and Apple Video / RPZA), with all the
extended functionality being built in a way that a normal decoder will
ignore (APPn markers for JPEG, unknown bytes for RPZA, ...).
(generally, a TLV format is built, and all of the extensions are built
in terms of this TLV format).
however, in some cases, I am limited to some extent by the capabilities
of the base-format.
likewise, (after some initial testing) some formats (notably Theora and
XviD), were not particularly flexible nor well suited to the use-case.
(yes, they do well at what they do well, still doesn't mean they fit
this use case particularly well...).
(left out a much longer analysis on the various codecs and my
experiences with them...).
I was then like, "you know what, I will just devise something more
specialized" (mostly for the use case of intermediate storage of frame
data to the HDD), and proceeded to design and implement a codec with a
design sort of like RPZA mixed with Cinepak, but with more color bits
(11,9,9 for LDR, and 9,8,8 for HDR with a 4-bit exponent, using a YCoCg
colorspace with 4:2:0 chroma subsampling, and basic support for an alpha
it is mostly tuned for speed and image quality, at the cost of a fairly
basically, a fairly niche use-case...
but, then the general issue once again pops up (if using a format like
I can no longer view the video in a normal video player (which is
inconvenient), and also lose the thumbnail images (also annoying...).
and, hence, the issue of how to best deal with this?...
More information about the vlc-devel