build: Only clean the architecture subdirectory we build for.