[x264-devel] Re: Defining video quality

[Loren Merritt]

On Thu, 30 Nov 2006, Alex Izvorski wrote:

> About PSNR: if you encode the same continuous (no scenecuts) sequence
> with the same algorithm with several different settings, the version
> with the lower PSNR is almost always visually better.  Thus PSNR does a
            ^^^^^ higher
> reasonably good job of allowing you to optimise codec parameters that
> you can vary during an encode (like RDO or rate control).
>
> However, PSNR fails on several counts:
> 1) The absolute value of PSNR means nothing.  In other words, looking at
> two pairs of original/encoded images both with PSNR of 40 (for example),
> one pair may look identical and the other may have gross artifacts.  It
> all depends on the source footage and on the type of algorithm.  This
> means that PSNR is useless for comparing significantly different
> algorithms.  It also means that using PSNR to allocate bits accross
> scenecuts will result in bad decisions.
> 2) Some artifacts which are essentially invisible have a huge PSNR
> impact (try scaling Y uniformly or adding a small amount of gaussian
> noise to each pixel for example).  Other things which are extremely
> visible have an unreasonably low PSNR impact (like blocking).
>
> SSIM fixes these problems to a significant degree.  It is not *the best*
> video/image quality metric, but it is the best that can be computed
> quickly.  SSIM absolute values can be reasonably compared: as a very
> rough guide, below 0.7 is barely watchable, 0.8-0.85 has some visible
> distortion but ok for most people, and 0.9 and higher are
> indistinguishable from the original.

Really? imo, SSIM=0.98+ (qp=20) is indistinguishable from the original, 
SSIM=0.95 (qp=30) is barely watchable, SSIM=0.9 (qp=40) is really ugly, 
and there could be nothing left of the content (qp=51) and still produce 
SSIM=0.8. I failed to generate any videos with SSIM<0.7 except by adding 
tons of noise to the source and then setting qp high enough that all the 
noise is quantized away.
With both PSNR and SSIM, the absolute value of the metric is more a 
measure of how much noise was in the source than it is of how much 
quality is left in the compressed stream. Though perhaps SSIM does better 
at comparing difference scenes within one movie, when they can be assumed 
to have the same noise.

--Loren Merritt

-- 
This is the x264-devel mailing-list
To unsubscribe, go to: http://developers.videolan.org/lists.html