Hail Mary, Value Porosity, and Utility Diversification

19 December 2014

Professor Nick Bostrom, Director, Oxford Martin Programme on the Impacts of Future Technology

View Journal Article / Working Paper

This paper introduces some new ideas related to the challenge of endowing a hypothetical future superintelligent AI with values that would cause it to act in ways that are beneficial. Since candidates for first-best solutions to this problem (e.g. coherent extrapolated volition) may be very difficult to implement, it is worth also looking for less-ideal solutions that may be more easily implementable, such as the Hail Mary approach. The paper introduces a novel concept—value porosity—for implementing a Hail Mary pass. It also discusses a possibly wider role of utility diversification in tackling the value-loading problem.