avfilter/vf_nnedi: fix some compilation errors and warnings