Abstract

This paper responds to Nick Bostrom’s suggestion that the threat of a human-unfriendly superintelligenceshould lead us to delay or rethink progress in AI. I allow that progress in AI presents problems that we are currently unable to solve. However; we should distinguish between currently unsolved problems for which there are rational expectations of solutions and currently unsolved problems for which no such expectation is appropriate. The problem of a human-unfriendly superintelligence belongs to the first category. It is rational to proceed on that assumption that we will solve it. These observations do not reduce to zero the existential threat from superintelligence. But we should not permit fear of very improbable negative outcomes to delay the arrival of the expected benefits from AI.