Abstract: Many scientists expect the eventual development of intelligent software programs capable of substituting for human labor in almost every economic niche. As software, such intelligences could be cheaply copied, with copies subsequently diverging and interacting with their copy-relatives. This talk examines a set of evolutionary pressures on interaction between related intelligent programs, pressures favoring the emergence of software superorganisms, comparable to the social insects or the organization of multicellular life. These would be groups of software entities ready to self-sacrifice in service of the aims of the superorganism of related entities. I argue that the capacities and internal coordination of such superorganisms could both pose increased risks of overriding human values, and also facilitate the solution of global coordination problems.

Carl Shulman is an independent scholar supported by the Singularity Institute for Artificial Intelligence (Canada). Previously, he held a position at Clarium Capital Management, a global macro hedge fund managed by PayPal founder and Facebook investor Peter Thiel, and attended New York University School of Law. He holds a B.A. in philosophy from Harvard University.