The use of Artificial Intelligence or automated processes in Commonwealth decision-making

Overview

This article was co-authored with Imogen Thompson.

This article outlines the issues arising in respect of Commonwealth decision-making by Artificial Intelligence (AI) commonly referred to as automated decision-making processes (ADMPs). This includes the effect ADMPs can have on the making of a valid decision and some ways that agencies may be able to limit the risk of a decision being invalid when ADMPs are involved.

ADMPs are currently used by the Commonwealth for a variety of decisions – commonly high volume, routine administrative decisions. They may be used across the entire decision-making process or for only a part of the process (for example, to sort information for a human decision-maker to make the decision).

To understand how ADMPs affect the decision-making process, it is necessary to consider the kinds of decisions that government can make.

The first kind of decision is administrative. The principles that underpin administrative decisions are lawfulness, fairness, rationality, openness or transparency and efficiency (see p. 3 Automated Assistance in Administrative Decision Making Report published by the former Administrative Review Council).

Decisions of an administrative character must be made in accordance with the power granted by legislation or subordinate law. Relevant criteria can include that there is a human decision-maker or that the decision be made in a certain way. Such criteria can often be difficult to satisfy. In response to this issue, some departments have recently enacted legislation to expressly authorise making decisions through computers or ADMPs.

Other than administrative decisions, government can exercise powers arising from normal administrative and other functions including entering into a contract, acquiring and managing property, publishing guides or advice to the public and conducting lawsuits (see p. 2 of Decision Making: Lawfulness, published by the former Administrative Review Council). The risk in using ADMPs for these kinds of decisions is often less, as there are not strict legislative requirements involved.

Recent relevant cases

The case of Pintarich v Deputy Commissioner of Taxation [2018] FCAFC 79 (Pintarich) illustrated issues that can arise with ADMPs where there has been no mental process of reaching a conclusion in relation to an administrative decision. The case considered whether a letter issued by the Australian Taxation Office (ATO), which was issued by a computer generated system and had not been reviewed by the delegate before it was sent, amounted to a decision of the ATO.

The majority of the Full Federal Court concluded that no decision was made, because ‘there needs to be both a mental process of reaching a conclusion and an objective manifestation of that conclusion’. As the decision had not been reviewed by the delegate before it was sent, no mental process of reaching a conclusion had occurred, and therefore no decision had been made.

The dissenting judgement in Pintarich, however, notes that automated systems are increasingly being used by a number of Australian government departments for bulk decisions, and that the legal conception of what constitutes a decision cannot be static and must comprehend that technology has altered how decisions are made. The dissenting judge also makes the point that the majority decision creates administrative uncertainty, and that those dealing with government should be able to rely on letters from government agencies communicating decisions.

As a result of this case, there is now a degree of ambiguity on what constitutes a decision when ADMPs are used as part of the decision-making process.

While there have been no significant judgments on ADMPs since Pintarich, in the ‘Robodebt’ case the government ultimately conceded that the alleged debt was not validly claimed as the information before the decision-maker was not capable of satisfying the decision-maker that a debt was owed under the relevant provision of the Social Security Act 1991.

Government-issued guidance

In response to an increase in ADMPs being used by agencies, various government bodies have issued guidance and commentary on automated decision-making. We have briefly summarised this guidance and commentary below.

Automated Decision-making Better Practice Guide, published by the Commonwealth Ombudsman (Guide)

• This is the primary Commonwealth guidance on the use of ADMPs. It is particularly concerned with ensuring compliance with administrative law and privacy principles. The Guide notes that ADMPs are not suitable for any part of a decision-making process where it would:

• contravene administrative law requirements of legality, fairness, rationality and transparency;
• contravene privacy, data security or other legal requirements (including human rights obligations);
• compromise accuracy in decision-making; or
• significantly undermine public confidence in government administration (see p. 8 of the Guide).

• The Guide also notes that departments considering using ADMPs should be particularly careful when the decision-making process will involve the exercise of judgement or discretion. The Guide notes that there is not clear and definitive guidance from the courts about whether it is necessary for all discretions to be exercised personally by a decision-maker (see p.9 of the Guide).

Future of Law Reform Report, published by the Australian Law Reform Commission (Report)

• The Report provides commentary on concerns regarding the future development of ADMPs. The Report notes that even when automated systems are deployed carefully and effectively by government agencies, questions remain as to their compatibility with core administrative law principles and the rule of law.
• The Report raises concerns with administrative law principles and the use of ADMPs. These issues include:

• how decision-makers can be identified and disclosed;
• the identification of the precise tasks being undertaken by an automated system;
• appropriate review mechanisms for decisions; and
• how freedom of information obligations should apply (see p. 28 of the Report).

Artificial Intelligence: Australia’s Ethics Framework, published by CSIRO Data61 (Discussion Paper)

• The Discussion Paper provides ethical commentary on the use of ADMPs. It notes that existing legislation suggests that ADMPs are suitable when there are a large volume of decisions to be made, based on relatively uniform, uncontested criteria.
• When discretion and exceptions are required, the Discussion Paper notes that ADMPs are best only used as a tool to assist human decision-makers. It also suggests that there must be a clear chain of accountability for decisions made by an automated system, with a clear understanding of who is responsible for decisions made by the system (see p. 10 of the Discussion Paper).

Approach to legislative amendments

To avoid the risk of government decisions involving ADMP being declared invalid, legislation is increasingly being amended to authorise computer programs to make administrative decisions.

Examples of legislation with such provisions includes the Export Control Act 2020 (ECA) (s286), Migration Act 1958 (MA) (ss495A and s495B), Australian Citizenship Act 2007 (ACA) (ss48 and 49), A New Tax System (Family Assistance) (Administration) Act 1999 (s223) and Social Security (Administration) Act 1999 (s6A).

These provisions generally apply to decisions involving non-discretionary elements. The provisions often enable the relevant decision-maker (typically, either the Secretary or the Minister) to arrange for the use of computer programs to make decisions under the relevant legislation. A decision made under such an arrangement is deemed to be a decision by the Secretary or Minister.

The provisions also often allow for a decision to be substituted by the human decision-maker. Under the ECA, the Secretary may substitute a decision if he or she is satisfied that the decision made by operation of the computer program is incorrect. Other legislation, such as the MA and ACA only allows substitution of a decision in specific circumstances, which include the computer program not functioning properly or the substituted decision being more favourable to the applicant (s495B(1) MA; s49(3) ACA).

Practical guidance for the use of AI or automation in the decision-making process

Where ADMPs are used by government agencies, the following matters should be considered:

1. What is the decision?
2. Is there a legislative provision that authorises the decision? If yes, who is the specified decision-maker?
3. Is there legislation in place that authorises the use of ADMPs?
4. Does the decision involve the exercise of a discretion or a judgement call on the part of the decision-maker?

These considerations should be balanced against how the ADMP is being used in the decision-making process. For example, whether the ADMP is being used:

1. for the whole decision process or only a part;
2. to make a decision or recommend a decision;
3. to guide a decision-maker through relevant facts and provide support systems;
4. to provide a preliminary assessment; or
5. to automate the fact-finding process.

Lastly, it is prudent to consider whether the use of an ADMP is ethical, transparent and in line with community expectation.