Only call inflateEnd when we were actually using the zlib code.