Asking the right questions

Rather than leaping right in to solving problems, we take a step back and make sure we are asking the right questions. This is doubly important when contemplating emerging Ai technologies where the implications and consequences of those technologies at scale in society has never been explored.

Since our inception in 2017, we have been exploring and refining the questions that are central to a new branch of engineering to ensure the safe, sustainable and responsible development of cyber-physical systems (CPS). These questions look beyond the software and hardware in the lab, and contemplate what happens when technologies are let loose on the world – are taken up by many different organisations and governments, are connected to other intelligent systems, and go to other places in the world from where they were designed.

Autonomy

Questions about autonomy aren’t just technical questions like “how do we make the car brake when it needs to?” – they are also social, regulatory and public policy questions, which will be different for each use and context. Just because we can make a system autonomous, should we? How do we think about degrees of autonomy? How is an autonomous system different to an automated system? When we translate human processes into autonomous processes, what do we need to consider?

Agency

As the ability for machines to act independently of human oversight increases, questions need to be asked about how much agency we give cyber-physical systems. To what degree should the system be able to make decisions without human intervention? If a system learns, should we allow it to independently change its behaviour? And if so, does the system bear any responsibility for its actions? What is the override system and who has access?

Assurance

Technological progress necessitates new regulatory tools and processes, as systems designed in one place are introduced into different settings around the world, with access to new data. How do we preserve our safety and values? What are the mechanisms for assessing and managing safety, security and policing concerns? How do we reassure users and the wider community that the system is safe? To what laws and regulations is the system subject to, and are they adequate?

3Ai was founded on these 3 questions – we have since imagined more. And will likely imagine more still! We explore each of these questions above and below in our Masters and PhD programs.

Interfaces: throughout the 20th century we used keyboards and screens as interfaces with computers. What interfaces will new CPS use? How will people know they are interacting with an Ai system? Does the type of interface change our relationship with technology? What’s next beyond the field of Human-Computer Interaction?  

Indicators: the world is changing. Commercial measures of success such as productivity and efficiency used without reference to safety, sustainability and responsibility rarely cut it anymore. What measures of performance and success should we use for CPS and Ai? How do we account for unintended consequences? How might we change the intent of the system through our choice of metrics?

Intent: the last set of questions, but in some ways the first. A focus on safety, sustainability and responsibility means that we must deeply interrogate the intent – explicit and implicit – in building a system. Why does this CPS exist? Who does it serve? What is the power relationship between those who commission it, those who design and implement it and those whose lives are impacted by it?