Prof Nick Bostrom, Dr Anders Sandberg, Dr Tom Douglas
Authors: Professor Nick Bostrom, Director of the Oxford Martin Programme on the Impacts of Future Technology and the Future of Humanity Institute; Dr Anders Sandberg, James Martin Fellow of the Oxford Martin Programme on the Impacts of Future Technology and the Future of Humanity Institute and Dr Tom Douglas, Future of Humanity Institute.View Journal Article / Working Paper
In some situations a number of agents each have the ability to undertake an initiative that would have significant effects on the others. Suppose that each of these agents is purely motivated by an altruistic concern for the common good. We show that if each agent acts on her own personal judgment as to whether the initiative should be undertaken, then the initiative will move forward more often than is optimal. We suggest that this phenomenon, which we call the unilateralist’s curse, arises in many contexts, including some that are important for public policy. To lift the curse, we propose a principle of conformity, which would discourage unilateralist action. We consider three different models for how this principle could be implemented, and respond to some objections that could be raised against it.