avfilter: Properly check for failed format query