[x264-devel] Fix incorrect zero-extension assumptions in x86_64 asm

Henrik Gramner git at videolan.org
Wed Mar 7 03:20:13 CET 2012


x264 | branch: master | Henrik Gramner <hengar-6 at student.ltu.se> | Wed Feb  1 23:52:48 2012 +0100| [02c3d5ec58d6bcbc5e22715ae80d53d8556f3c8f] | committer: Jason Garrett-Glaser

Fix incorrect zero-extension assumptions in x86_64 asm
Some x264 asm assumed that the high 32 bits of registers containing "int" values would be zero.
This is almost always the case, and it seems to work with gcc, but it is *not* guaranteed by the ABI.
As a result, it breaks with some other compilers, like Clang, that take advantage of this in optimizations.
Accordingly, fix all x86 code by using intptr_t instead of int or using movsxd where neccessary.
Also add checkasm hack to detect when assembly functions incorrectly assumes that 32-bit integers are zero-extended to 64-bit.

> http://git.videolan.org/gitweb.cgi/x264.git/?a=commit;h=02c3d5ec58d6bcbc5e22715ae80d53d8556f3c8f
---

 common/arm/mc-a.S        |   26 +++---
 common/arm/mc-c.c        |   82 ++++++++++----------
 common/arm/pixel.h       |   32 ++++----
 common/deblock.c         |  144 +++++++++++++++++-----------------
 common/frame.c           |    4 +-
 common/frame.h           |    6 +-
 common/macroblock.c      |    2 +-
 common/mc.c              |   94 ++++++++++++-----------
 common/mc.h              |   69 +++++++---------
 common/pixel.c           |   61 ++++++++------
 common/pixel.h           |   33 ++++----
 common/ppc/deblock.c     |    4 +-
 common/ppc/mc.c          |   92 ++++++++++-----------
 common/ppc/pixel.c       |   97 ++++++++++++-----------
 common/sparc/pixel.h     |    8 +-
 common/x86/deblock-a.asm |   82 ++++++++++----------
 common/x86/mc-a.asm      |   44 +++++------
 common/x86/mc-a2.asm     |  108 ++++++++++---------------
 common/x86/mc-c.c        |  197 +++++++++++++++++++++++-----------------------
 common/x86/pixel-32.asm  |    6 +-
 common/x86/pixel-a.asm   |   38 +++++----
 common/x86/pixel.h       |   76 +++++++++---------
 common/x86/quant-a.asm   |    2 +
 common/x86/sad-a.asm     |   23 ++----
 common/x86/sad16-a.asm   |    9 +--
 common/x86/x86inc.asm    |    5 -
 encoder/analyse.c        |   10 +-
 encoder/me.c             |   10 +-
 encoder/slicetype.c      |    2 +-
 tools/checkasm-a.asm     |  120 +++++++++++++++++++---------
 tools/checkasm.c         |  170 ++++++++++++++++++++++------------------
 31 files changed, 843 insertions(+), 813 deletions(-)

Diff:   http://git.videolan.org/gitweb.cgi/x264.git/?a=commitdiff;h=02c3d5ec58d6bcbc5e22715ae80d53d8556f3c8f


More information about the x264-devel mailing list