avformat/ffmdec: Only return EAGAIN if a server is attached