The pages of The Policymaker are filling with contributions on Artificial Intelligence (AI) and public policy from some of Australia’s leading researchers in this field. This is a great sign that scholars and policymakers alike are starting to pay attention to the advantages, risks and challenges of AI systems – and automation more widely – in the public sector. Most contributions consider the very important question of the regulation and governance of AI as it is used now or is likely to be used in the immediate future (over a two-to-three-year horizon): as a tool to execute and implement public policies.  But we need to consider AI systems not simply in terms of the automation of menial administrative tasks, but also about their use to design and decide on public policies.

In a recently published chapter in Money, Power and AI (open access), I call this “a tale of two automated states”. The “easy” tale refers to automation in the execution of policies: things human can generally do in the current state of affairs, but could be done better (or more efficiently) through automation. This automation could be more or less sophisticated. For instance, following a flowchart to see who is entitled to park on a given street, or detecting possible fraud among millions of data points, are very different tasks. And yet, both are essentially automations of tasks public officials are already doing.

The “challenging” tale takes us to a world in a not-too-distant future when AI is used at the level of policy design. To illustrate it, I use the example of digital twins. Digital twins are data driven copies of existing real-life environments or organisms. Although attention has primarily focused on digital twins of living organisms – for instance, for testing drugs on those data-driven twins – promising work is being undertaken in other types of real-life twins, such as factories or cities. One leading example is the work in Barcelona to create a digital twin of the city that can, amongst other purposes, assist policymakers in their decisions on urban policy. Early pilots and proof of concept testing are also being developed for similar purposes in NSW.

If an organism as complex as a city can be mimicked by a digital twin, it would be possible to anticipate how a city would react to a given policy over time. Conversely, once a policy is introduced in real life, it would be possible to observe in the digital twin how the city would have fared if the status quo remained. This is precisely what happened in Barcelona with the introduction of a traffic-mitigation policy called the “superilles”.

The main declared purpose of the superilles policy was to reduce air pollution. Traditional modelling had offered promising perspectives based on certain assumptions about traffic patterns. As COVID-19 related mobility restrictions were first imposed and then lifted, the habits of residents and tourists changed, making the initial modelling irrelevant. Yet, thanks to the digital twin, it was still possible to assess what would have happened if the status quo remained – i.e., in a parallel world without superilles. With some caveats, the digital twin showed that the superilles had made no relevant difference in terms of air quality. There may have been positive effects in other aspects of city life, but, as a policy tool to fight air pollution, its impact was negligible at best.

Here things get interesting. The superilles policy was heavily fought socially and legally by some sectors of the city. The request for an injunction to stop initial public works was rejected in Spanish courts based on the principle that the expected harm suffered by the parties requesting the injunction – for instance, a likely decrease in value for their commercial properties at least in the immediate years – was justified by the public interest, namely the expected improvement for health and living conditions in the city.

Now, if the digital twin outcomes are to be believed, the court’s reasoning did not hold. With the benefit of hindsight, the case for the parties requesting the injunction was much stronger. Perhaps not strong enough to change policy, but at least enough to justify compensation.

Yet, the bigger question remains: what would the consequences be for policy design if the digital twin was used ex ante, before the policy was implemented to anticipate the negligible effects? Could the court have requested that analysis? Should policymakers take the digital twin into consideration as part of the decision-making procedure (e.g., as environment impact assessments are often mandatory)? Should such an output restrict the policy discretion of the decision-maker? How much is an AI-driven system to be trusted for these purposes? And, perhaps most importantly, how much testing against real-life conditions needs to happen before such a system is given a role in the policymaking process at all?

The answer – like anything related to AI – is not obvious, but we do know that regulation of public authorities will need to adapt to the policymaking application of AI. It is not too early to start testing these tools in parallel to existing procedures and analyse differences in performance over the years, so, when the time comes, we have the evidence to get the regulation right.

A good place to start is directly involving policymakers and administrative lawyers in these pilot initiatives and to test these new tools against existing methods. For example, the current gold standard in evidence for policymaking – randomised control trials or RCTs – have very limited applicability in large infrastructure projects or where network effects make it difficult to isolate interventions or compare environments (e.g., in traffic-related policies). Policy decisions are then based on subjective considerations or, if evidence based, on limited-value impact assessments.

Initiatives such as the one currently developed by CSIRO for Western Sydney are brilliant developments of the technology, but it remains uncertain whether they are being assessed from the point of view suggested in this article. Are they being tested against other options for policy decisions (e.g., local consultations)? If one of these systems determines that a given policy intervention could cause problems, is there a proper record of that output that can be tracked and reassessed in the future, so the next time the system generates a similar warning is it adequately valued? Are proposed policy decisions being tested at all with the systems?

To make this happen, there are essentially two routes. On the one hand, we could trust that this applied research is designed in a way that takes these elements into account. This would provide greater flexibility needed for the development of pilot programs.

On the other hand, though, the better approach is to make it formal policy. The NSW Government can use ongoing pilots as miniature “regulatory sandboxes” where  approval comes with conditions about policy-oriented, real-time testing. While this would likely increase costs, this would be outweighed by the value of that data for the future governance of AI-assisted policymaking.

José-Miguel Bello y Villarino is Senior Research Fellow at the University of Sydney Law School and Research Fellow at the Institutions program of the ARC Centre of Excellence for Automated Decision-Making and Society (ADM+S). He is a member of the Diplomatic Corps of Spain (on leave) and previously worked in different capacities for the European Union. His current research focuses on regulatory approaches to ADM and AI, especially on how to deal with risks derived from the operation of AI systems from a comparative approach and the use of AI in government. He is a recipient of the 2023 JMI Policy Challenge grant as part of a University of Sydney and University of Technology Sydney collaboration researching the governance of AI education and equity.

Image credit: Getty Images