return AVERROR(ENOMEM) if stream could not be allocated