[x264-devel] x264 quantization and AIC quastions

[math-BeSSeL at yandex.ru]

09.11.07, 10:23, Sergey A. Sablin (sergey.sablin at elecard.com):

> decoder doesn't know anything about rounding parameters used in encoder, 
>   which means encoder may use whatever it needs (check out trellis 
> quantization, which uses different rounding for each coefficient)

Ok. Thanks for the answer.

> about intra modes - that's internal encoder constants, they're not 
> written into the bitstream.

yes, i have already found function x264_mb_pred_*_fix :)


> math-BeSSeL at yandex.ru wrote:
> > 
> > 09.11.07, 06:15, Loren Merritt (lorenm at u.washington.edu):
> > 
> >> On Thu, 8 Nov 2007, math-BeSSeL at yandex.ru wrote:
> >>> 1) For quantization process in the h.264 standard the following
> >>> formula is used
> >>>
> >>> coef_q = (coef * mf + f)>> (15 + Qp/6) (*)  (for example for intra mb
> >>> and coef>=0)
> >>>
> >>> In the x264 codec other formula is used
> >>>
> >>> coef_q" = ((coef + f") * mf")>> 16 (**)
> >>>
> >>> After the certain transformations from one it is possible to receive 
> >>> another: (*) <-> (**).
> >>> But there are conditions when the quantization result of the second
> >>> formula (**) does not coincide with result of the first formula (*).
> >>> Why this formula (**) is used?
> >> Because (*) requires 32-bit math, whereas (**) can be implemented with 
> >> only a 16x16>>16 multiply instruction (PMULHUW). That makes a SIMD 
> >> implementation much faster.
> > 
> > Ок. I had such idea (4 coef in xmm register). 
> > But (**) using gives an inaccuracy of calculations.
> > 
> > For example luma dc quantization: Qp = 29, coef(0,0) = coef_dc = 48.
> > (*) -> coef_q = 1
> > (**) -> coef_q" = 0.
> > 
> > I think, that the inaccuracy contains in f" parameter.
> > For calculation f" (or "quant4_bias" in the code) the "deadzone" is used.
> > I use value luma_deadzone[2] = {21, 11} -> deadzone[4] = {21, 11, 21, 11}.
> > What do you think about this?
> > 
> > Certainly (**) using does not give visual artefacts.
> > But if to compare results idct+dequantization of (*) and (**) in standart decoder, the luma component difference can <= 4 pixels (for the case coef in [-2048;2048]).  
> > 
> >>> 2) I_PRED_4x4_DC_LEFT / TOP, I_PRED_CHROMA_DC_LEFT / TOP,
> >>> I_PRED_16x16_DC_LEFT / TOP.  I can is wrong, but the given modes are
> >>> not present in the h.264 standard.
> >> The standard describes a single DC function for each size, but that 
> >> function does 4 different things depending on which neighbors are present. 
> >> x264 splits them into 4 functions.
> > 
> > I understand. But the standart decoder incorrectly decodes a stream for the given modes.
> > 
> > _______________________________________________
> > x264-devel mailing list
> > x264-devel at videolan.org
> > http://mailman.videolan.org/listinfo/x264-devel
> _______________________________________________
> x264-devel mailing list
> x264-devel at videolan.org
> http://mailman.videolan.org/listinfo/x264-devel