avfilter/vf_convolution: fix undefined behaviour