vp9: fix overflow in 8x8 topleft 32x32 idct ssse3 version.
authorRonald S. Bultje <rsbultje@gmail.com>
Wed, 9 Sep 2015 18:10:41 +0000 (14:10 -0400)
committerRonald S. Bultje <rsbultje@gmail.com>
Thu, 10 Sep 2015 11:51:16 +0000 (07:51 -0400)
commitfd8b90f5f63de12c1ee1ec1cbe99791c5629c582
treebdf50fda06657232f3c54265cb7fe3681567dd97
parent4bb9dbe4d7952d82bddb23c45889ab1550897397
vp9: fix overflow in 8x8 topleft 32x32 idct ssse3 version.

Also disable the mmx/iwht optimization when the bitexact flag is set.
With synthetically coded coefficients (i.e. these that lead to a
residual well outside the [-255,255] range), our optimizations will
overflow. It doesn't make sense to fix the overflows, since they can
only occur on synthetic input, not on real fwht-generated input. Thus,
add a bitexact flag that disables this optimization.
libavcodec/vp9.c
libavcodec/vp9dsp.c
libavcodec/vp9dsp.h
libavcodec/x86/vp9dsp_init.c
libavcodec/x86/vp9itxfm.asm