Humanities and Social Sciences Colloquium - Johanna Thoma

Risk Aversion and Rationality: Dilemmas for the Design of Autonomous Artificial Agents

The ambition for the design of autonomous artificial agents is that they can make decisions at least as good, or better, than those humans would make in the relevant decision context. For instance, the hope for autonomous vehicles is that they can drive safer and more efficiently than human-driven vehicles, as well as help prevent traffic jams. The theory of rational choice designers of artificial agents usually aim to implement, in the context of uncertainty, is expected utility theory, the standard theory or rationality in economics and the decision sciences. This talk will present two features of the attitudes to risk that human agents tend to display that defy any simple expected utility analysis: We tend to be differently risk averse with regard to different kinds of goods, and we tend to have inconsistent risk attitudes to small stakes and large stakes gambles. Unlike standard ‘anomalies’ of choice, these attitudes are not obviously irrational. In fact, the decisions of an agent who does not display them will often strike us as unintuitive, and even morally problematic when the risks involved are morally relevant (as they often are in the case of autonomous vehicles). I will show that this poses difficult dilemmas for the design of autonomous artificial agents.