[x264-devel] x264 quantization and AIC quastions

[Loren Merritt]

On Thu, 8 Nov 2007, math-BeSSeL at yandex.ru wrote:

> 1) For quantization process in the h.264 standard the following
> formula is used
>
> coef_q = (coef * mf + f)>> (15 + Qp/6) (*)  (for example for intra mb
> and coef>=0)
>
> In the x264 codec other formula is used
>
> coef_q" = ((coef + f") * mf")>> 16 (**)
>
> After the certain transformations from one it is possible to receive 
> another: (*) <-> (**).
> But there are conditions when the quantization result of the second
> formula (**) does not coincide with result of the first formula (*).
> Why this formula (**) is used?

Because (*) requires 32-bit math, whereas (**) can be implemented with 
only a 16x16>>16 multiply instruction (PMULHUW). That makes a SIMD 
implementation much faster.

> What for a variable "deadzone"?

I don't understand the question.

> 2) I_PRED_4x4_DC_LEFT / TOP, I_PRED_CHROMA_DC_LEFT / TOP,
> I_PRED_16x16_DC_LEFT / TOP.  I can is wrong, but the given modes are
> not present in the h.264 standard.

The standard describes a single DC function for each size, but that 
function does 4 different things depending on which neighbors are present. 
x264 splits them into 4 functions.

--Loren Merritt