What is AI Governance for?

When products, services, and processes use artificial intelligence, it is necessary to make sure that the AI does exactly what it is intended to do, and nothing else. This is what AI governance is for. AI governance has a role to play throughout the lifecycle of an AI, from requirements, design, through development, testing, release, monitoring, iterative improvement, until decommissioning. It shapes how all these activities are done, in order to have AI work as intended.

If the AI is designed to learn, then the range of all possible outputs it may give is not going to be completely predictable. Simply put, over time, the AI will not give the same answer to the same question, for at least some questions it is asked. 

If an AI is not predictable by design, then the purpose of governing it is to ensure that it gives the right answers (actions) most of the time, and that when it fails, the consequences are negligible, or that it can only fail on inconsequential questions, goals, or tasks.

This principal challenge for AI governance is not very different from what governance needs to do in other contexts – governance of companies, commercial relationships, business processes, etc. Something unpredictable behavior, which creates risks, and governing it is about how to design it and its context over time, to steer behavior in the desired direction.

The difference with AI governance is that the subject being governed is new and at the moment at least, requires knowledge that is not yet broadly available and brought to the level that is accessible to many. 

The above will resonate with anyone who tried to build and use AI for commercial purposes. I did this as early as 2011, and it was clear that no product or service that uses AI can go to market without an AI governance framework, clear roles and responsibilities, processes, risk management strategies, as well as tools deeply integrated with the AI to monitor and control its operation. It was also clear that absence of an understanding of the need for governance was a signal of a team that should be helped with mentoring and resource planning to put an AI governance capability in place. It was a criterion when comparing investment opportunities in companies integrating AI into their processes, products, and services. 

Understanding AI governance was already back then absolutely necessary for every executive in a company that used or planned to use AI.