Fix incorrect zero-extension assumptions in x86_64 asm
authorHenrik Gramner <hengar-6@student.ltu.se>
Wed, 1 Feb 2012 22:52:48 +0000 (23:52 +0100)
committerFiona Glaser <fiona@x264.com>
Tue, 6 Mar 2012 18:37:53 +0000 (10:37 -0800)
commit3131a19cabcdca221ce4cd61a3cff68d99f1a517
tree14d71531382f0ced24709d1895486ca530e09108
parentd52d0b1e6a9323911818c2a89764f6827974e0f7
Fix incorrect zero-extension assumptions in x86_64 asm
Some x264 asm assumed that the high 32 bits of registers containing "int" values would be zero.
This is almost always the case, and it seems to work with gcc, but it is *not* guaranteed by the ABI.
As a result, it breaks with some other compilers, like Clang, that take advantage of this in optimizations.
Accordingly, fix all x86 code by using intptr_t instead of int or using movsxd where neccessary.
Also add checkasm hack to detect when assembly functions incorrectly assumes that 32-bit integers are zero-extended to 64-bit.
31 files changed:
common/arm/mc-a.S
common/arm/mc-c.c
common/arm/pixel.h
common/deblock.c
common/frame.c
common/frame.h
common/macroblock.c
common/mc.c
common/mc.h
common/pixel.c
common/pixel.h
common/ppc/deblock.c
common/ppc/mc.c
common/ppc/pixel.c
common/sparc/pixel.h
common/x86/deblock-a.asm
common/x86/mc-a.asm
common/x86/mc-a2.asm
common/x86/mc-c.c
common/x86/pixel-32.asm
common/x86/pixel-a.asm
common/x86/pixel.h
common/x86/quant-a.asm
common/x86/sad-a.asm
common/x86/sad16-a.asm
common/x86/x86inc.asm
encoder/analyse.c
encoder/me.c
encoder/slicetype.c
tools/checkasm-a.asm
tools/checkasm.c