avcodec/libopenh264dec: fix return error value when h264_mp4toannexb_bsf is not found