What is a “Decision” in an Artificial Intelligence System?

In the context of human decision making, a decision is a commitment to a course of action (see the note here); it involves mental states that lead to specific actions. An AI system, as long as it is a combination of statistical learning algorithms and/or logic, and data, cannot have mental states in the same sense as people do. Consequently, this definition of “decision” does not work in the context of AI. 

If we take, for example, the EU AI Act, the term “decision” is not defined. It is mentioned in the definition of “AI system”, as follows.

“‘artificial intelligence system’ (AI system) means software that is developed with one or more of the techniques and approaches listed in Annex I and can, for a given set of human-defined objectives, generate outputs such as content, predictions, recommendations, or decisions influencing the environments they interact with;”

According to the quote, an AI system is able to decide. I wrote in another note, here, that AI cannot decide autonomously, which is coherent with how the AI system is defined in the EU AI Act. I argued in that note that the only thing that’s missing for autonomous decision making are preferences, and the EU AI Act definition solves this problem by stating clearly that human-defined objectives are provided to the system. 

That an AI system decides, then, means that it is able to conceive more than the single course of action in a situation where it is triggered to act, that it can compare these alternative courses of action prior to choosing one, and that it likes one over all others as a result of that comparison. In this context, a decision will be the selection of actions that have expected outcomes which score highest in terms of their proximity to human-defined objectives that were given.

Decision governance for AI systems consequently involves knowing how scoring is performed, what measures are used to score, how these are assigned, how expected outcomes of different actions are predicted, and many more. It is not much different from governing human decisions, except that in the case of an AI system, these do not need to be reified to be governed – see the note here – instead, the system needs to be able to generate explanations of its decisions (which may note be easy – see the note here).