[x264-devel] [PATCH]adaptive lowpass

[Loren Merritt]

On Wed, 27 Jun 2007, Eric Viscito wrote:
> On Wed, June 27, 2007 4:51 pm, Loren Merritt wrote:
>
>> Is it better to multiply the coeffs by some fraction, rather than
>> subtracting a constant (in the case of deadzone) or increasing lambda (in
>> the case of trellis)? That wouldn't be strictly a lowpass, but it would
>> also reduce the bits in high frequencies by changing coefficient magnitude
>> rather than step size, and would be closer to RD optimal.
>
> A deadzone does not subtract a constant from every coefficient, but rather
> sends a larger bucket of coefficients to zero and leaves the quantizer
> characteristic alone for coefficients outside that big bucket and the two
> that surround it.  So it reduces the magnitude of small coefficients,
> independent of their location in the spectrum.  However, using a weighting
> matrix, together with the naturally lowpass nature of images makes the
> deadzone preferentially zero out higher frequencies.  It will act like a
> lowpass filter or even an "all-reject" filter, but spatially variant, and
> only in places and at frequencies where the prediction is good.

Right, I don't mean to imply that "deadzone" means subtract a constant. 
Rather, x264's implementation of deadzone does subtract a constant. 
Because not only is 0 cheaper than 1, but also every coefficient value 
is cheaper than the next larger value. While the amount of 
difference is not uniform over all values, it's much faster to use 
a constant rather than some table.
And from an RD perspective the LPF gets it backwards, it subtracts more 
from large coefficients than from small, whereas you would save more bits 
per distortion by reducing small coefficients.

> Raising lambda in the trellis quant algorithm will reduce the juiciest
> coefficients from an R/D perspective, considering all non-zeros in one
> block at a time.  That seems pretty powerful.  This is still spatially
> variant but not as obviously dependent on prediction quality since it
> operates on all magnitudes of coefficients.

Here I meant raise the lambda just for the HF coefficients. While the 
current trellis uses one lambda for all coefficients, there's no reason 
they have to be the same. Just use distortion/lambda[i]+rate instead of 
distortion+rate*lambda.

--Loren Merritt
_______________________________________________
x264-devel mailing list
x264-devel at videolan.org
http://mailman.videolan.org/listinfo/x264-devel