poll: deal with invalid file descriptors more like specified