avfilter/formats: proper error handling in ff_set_common_*() functions