“AlphaGo is the first computer program to defeat a professional human Go player, a landmark achievement that experts believe was a decade ahead of its time” AlphaGo | DeepMind.
The way governments are now operating is being transformed by technological innovation in ways that were not contemplated when State and Commonwealth parliaments established Australia’s administrative law systems. Transformation brings significant opportunities to improve public administration and make defensible data-driven decisions. One of those opportunities is the use of machines to assist in decision-making. However, as has been discussed in one of our earlier articles, there are challenges in implementing machine-assisted decision-making processes in a legally compliant manner.
At the end of last year, the NSW Ombudsman released a report titled “The new machinery of government: using machine technology in administrative decision-making” (Ombudsman Report) which outlines some of the challenges facing government. The Ombudsman Report uses “machine technology” to refer to a broad cluster of systems and processes which operate with limited or no human involvement to assist or displace human decision-making. These are generally associated with the terms “artificial intelligence” or “automated decision-making”.
While the Ombudsman Report is focused on NSW case studies, its findings are of broader relevance to all levels of government across Australia. We set out the key findings of the Ombudsman Report before outlining some key principles relevant to designing robust machine-assisted decision-making systems.
What prompted the Ombudsman Report?
The Ombudsman Report was prompted by an investigation undertaken by the NSW Ombudsman into Revenue NSW, following complaints regarding Revenue NSW’s use of garnishee orders. Under statute, the Commissioner of Fines Administration (Commissioner) has the discretion to issue garnishee orders, if satisfied that certain circumstances exist. Garnishee orders allow a debt collector (the Commissioner) to recover debts directly from a financial institution (such as a bank).
Revenue NSW used automated technology to analyse and profile customers to enforce fines.
Between 2010 and 2019, the number of garnishee orders issued by Revenue NSW increased from 6,905 to 1.6 million. Its reported impact included completely depleting people’s accounts, which at times held the complainant’s only source of income from Centrelink. Prior notice was not given to account holders.
Although Revenue NSW cooperated with the NSW Ombudsman and made a number of modifications to its process (such as excluding particularly vulnerable persons from the process), external advice sought by the NSW Ombudsman confirmed that the automated process was not lawful because:
- no authorised person engaged in a mental process of reasoning to reach the state of satisfaction prescribed by statute in order to issue a garnishee order; and
- discretionary power was not being exercised by the authorised person.
Revenue NSW’s process is only one of a number of recent examples where insufficient attention has been paid to administrative law principles in designing automated systems.
The public sector context
The Ombudsman Report emphasises that the powers and actions of government agencies and officials are “constitutionally unique” from that of the private sector. As such, although the use of machine technology raises a range of ethical, legal and privacy concerns, its use in the public sector must be assessed primarily from an administrative law perspective.
It is imperative to design systems which meet the essential requirements of good administrative decision-making. Those requirements, as summarised by the Ombudsman, are:
- Proper authorisation: the decision-maker has legal authority and their decision is within scope of decision-making power;
- Appropriate procedures: the process of decision-making is fair, legal and ethical, and reasons for the decision are given to affected persons;
- Appropriate assessment: the decision answers the right question, is based on a proper analysis of relevant material and is defensible; and
- Adequate documentation: the circumstances surrounding the making of the decision are adequately recorded.
Takeaways for government
- Seek advice: Engage a multidisciplinary team to meet the challenge of translating law into code. Be aware of the limitations of your legislative framework and whether legislative amendment is required. Consider how an affected person would participate in the decision-making process and the practical implications of challenging decisions.
- Buy smart: Observe responsible procurement practices that emphasise transparency over decision-making processes, especially regarding copyright and intellectual property rights of suppliers.
- Test, test, test: prior to implementation and when the system is established. Test to ensure the correctness of machine process and legal interpretation, and to ensure that machines maintain an audit trail that satisfies principles of transparency and accountability.
- Be transparent: Government should consider releasing source codes and technical information of machine processes to the public. Greater transparency has the benefit of engaging external expertise and providing greater oversight through both public and parliamentary scrutiny.
Evolving law and policy: Resources for government
The legal and policy frameworks governing the implementation and use of machine-assisted decision-making systems are continually evolving. The challenge facing government is to establish legal frameworks that are future-focused and drafted in a manner that facilitates technological iterations.
While attention must be paid to the particular legal constraints that apply to each public sector function or program in designing systems, the following resources serve as helpful guidance:
