[x264-devel] [PATCH 2/3] RFC: checkasm: Warn if a better SIMD function is slower than the simpler one

[Loren Merritt]

On Fri, 14 Aug 2015, Martin Storsjö wrote:

> On Thu, 13 Aug 2015, Henrik Gramner wrote:
>
> > On Thu, Aug 13, 2015 at 11:00 PM, Martin Storsjö <martin at martin.st> wrote:
> > > ---
> > > This naively assumes that a later tested SIMD function is supposed
> > > to be better than the earlier ones - this probably doesn't
> > > hold for all x86 SIMD flags.
> >
> > This would most likely result in a huge amount of false positives.
> > There are plenty of AVX functions for example that are neither slower
> > nor faster than non-AVX functions on many CPUs which would often
> > trigger the warning since the cycle counter can drift a bit from run
> > to run for multiple reasons.
>
> Yeah, I guess so. With some amount of margin it might be more useful though
> (e.g. N * nop?). Even though it's prone to false positives, it can also be a
> useful hint to investigate things - at least for arm I found a few surprises
> where the C versions were faster than the asm ones. I guess it shouldn't be
> enabled by default at least.

The margin should just be a constant. The overhead of calling rdtsc varies
widely from one cpu to another (like, a factor of 10), and I haven't
noticed any tendency for benchmarking noise to scale with it.

And you're right about testing order being an inaccurate assumption. The
obvious way to fix that is to check which version of the function actually
gets selected on your cpu, and compare only that version.

--Loren Merritt