A Commonwealth Ombudsman investigation has found that automated government systems unlawfully cancelled income support payments for 9,642 job seekers over more than two years due to failures in updating computer systems after legislative changes.

The investigation into Australia's Targeted Compliance Framework (TCF) revealed that between April 2022 and July 2024, automated decisions cancelled welfare payments without the human discretion required under 2022 legislative amendments.

The Department of Employment and Workplace Relations (DEWR) and Services Australia failed to update their automated decision-making systems when Parliament changed the law to require consideration of individual circumstances before cancelling payments.

Commonwealth Ombudsman Iain Anderson found the agencies' actions were "contrary to law" and highlighted systemic failures in governance of automated decision-making processes.

"Automatic cancellation of vital income support for these job seekers is likely to have a profound, if not catastrophic impact," Anderson said in the report released this week.

The investigation uncovered multiple breakdowns in oversight and quality assurance. An independent review by Deloitte found the TCF computer system had become "increasingly unstable" with "volatility directly impacting compliance function operation."

DEWR's Secretary paused cancellation decisions in July 2024 after becoming aware of the unlawful automated processes. The department is now reviewing each affected case and making compensation payments.

The report made seven recommendations including establishing better consultation processes for legislative changes affecting automated systems, annual training for staff on administrative law requirements, and proactive identification of automation errors.

Of particular concern to compliance and risk managers, the investigation found that quality assurance activities failed to identify the unlawful cancellations for over two years. The automated system continued making decisions without human discretion despite legislative requirements.

The case highlights critical risks in automated decision-making systems, particularly the need to align technology with legal frameworks. The Ombudsman noted this was especially concerning given lessons from the Robodebt Royal Commission about automated processes affecting vulnerable people.

DEWR Secretary Natalie James accepted all recommendations and established a TCF Integrity Assurance Program with formal governance structures. The department committed to not resuming automated cancellations until systems comply with legal requirements.

The investigation also found DEWR failed to establish a Digital Protections Framework required by Parliament in 2022, more than three years after the legislative mandate.

Services Australia accepted recommendations for improved consultation processes, staff training, and ongoing assurance systems for automated decision-making compliance.

The case serves as a warning for organisations implementing automated decision-making systems about the critical importance of governance, oversight, and alignment with regulatory requirements.

The full report is available here.