[x264-devel] Re: Rate-Distortion Optimization in H.264

[Måns Rullgård]

"Mohsen Shaaban" <mmm5554 at louisiana.edu> writes:

> Hello,
>
> I have a question in Rate Distortion Optimization (RDO) in H.264;
>
> Is always choosing a prediction mode with a finer partitions gives less
> distortion than a coarser mode? Like will Inter_8x8 always gives less
> distortion than Inter_16x16?

Using smaller prediction blocks increases the chance of finding of a
good match requiring fewer bits to encode.  OTOH, using small blocks
means increased overhead in block headers, so large blocks are
preferred until the reduced size of the residual compensates for the
increased overhead.

> Now for RDO, I need to minimize (D + lambda*R) 
>
> and I know that large QP results in a large lambda thus low bit-rate
> and large distortion and small QP results in small lambda thus high
> bit-rate and small distortion
>
> Does this mean that I'll always need to choose the prediction mode
> with finer partitions to guarantee a lower distortion and what's the
> rule of the Quantization parameter (QP) based on that???
>
> I'm confused between the prediction mode decision and the changing
> of the QP to minimize the RDO equation.

RD optimization is an NP-class problem.  There is no known method for
finding the best solution in a reasonable amount of time.

-- 
Måns Rullgård
mans at mansr.com

-- 
This is the x264-devel mailing-list
To unsubscribe, go to: http://developers.videolan.org/lists.html