[x264-devel] Trellis RD

[Loren Merritt]

Does anyone know whether trellis quantization should be applicable to
encoding with CABAC? I tried a reasonably direct port of lavc's trellis
algorithm, and it seems to be working, but doesn't noticably improve
quality/bitrate.

Possible issues:
CABAC doesn't gain anything from a run of zeros (the zero flags each use a
separate context.)
I had to calculate rate based on the entropy added to the CABAC model,
not on the actual number of bits output (as that would just bias towards
delayed outputs).

--Loren Merritt

-- 
This is the x264-devel mailing-list
If you are in trouble, please contact <postmaster at videolan.org>