Critical gaps in the Federal Government’s recent announcements on artificial intelligence (AI) raise an important question: will Aboriginal and Torres Strait Islander people have genuine decision-making authority in Australia’s AI future?
The Government should explain how Aboriginal and Torres Strait Islander people will participate in its new Office of AI, and how Indigenous data and cultural knowledge will be protected, says Bronwyn Carlson, a Distinguished Professor at Macquarie University and an internationally recognised scholar on Indigenous digital life, data sovereignty, and the social impacts of emerging technologies.
“This cannot become another government body that makes decisions about us without us,” Carlson writes below.
Bronwyn Carlson writes:
Prime Minister Anthony Albanese’s recent speech, “AI in Australia’s interests”, set out an ambitious vision for Australia’s technological future. The Government will introduce mandatory Australian Standards for AI, regulate large data centres and has established an Office of AI within the Department of the Prime Minister and Cabinet.
Yet the speech presented AI primarily as a matter of investment, productivity, national security and Australian sovereignty. It had remarkably little to say about health, inequality or human rights. Aboriginal and Torres Strait Islander peoples were almost entirely absent.
This is more than an oversight. AI is already influencing decisions across health, welfare, education, policing and child protection. These are institutions in which Aboriginal and Torres Strait Islander peoples are already over-surveilled, over-classified and subject to disproportionate intervention.
If AI is to be developed in “Australia’s interests”, we need to ask who is included in Australia, whose interests will be protected and who will have the power to decide.
Producing real harm
The speech began by presenting Medicare as an example of Australian innovation grounded in social justice. Yet health received only a brief reference to AI-assisted cancer screening. There was no serious consideration of its use in diagnosis, clinical decisions, mental health, disability services, aged care or health administration.
AI may support earlier diagnosis, improve access to services and reduce administrative workloads. But these benefits are not automatic or equally distributed.
AI systems learn from existing data, and Aboriginal and Torres Strait Islander health data have been shaped by undercounting, misclassification, deficit framing, racism and unequal access to care. Systems trained on these data may reproduce these patterns while presenting their decisions as objective.
Some of the most consequential AI systems are also the least visible. Automated decision-making is already used to assess eligibility, calculate risk, identify supposed fraud, prioritise cases and recommend institutional action.
These systems do not simply analyse people. They affect whether someone receives income support, is investigated by child protection, gains access to housing or disability assistance, is identified as a policing risk or receives medical treatment.
When these systems are wrong, the consequences are not abstract technical errors. They can include debt, poverty, family separation, loss of housing, denial of care, increased surveillance and serious psychological distress.
Robodebt showed what happens when flawed automated processes are given the authority of government. Its lesson is not simply that Australia needs better algorithms. It demonstrated the danger of allowing automated systems to exercise power over people while making their decisions difficult to understand or challenge.
These dangers are particularly acute for Aboriginal and Torres Strait Islander peoples.
A community that has been over-policed will generate more police data. An automated system may interpret those data as evidence that the community requires even more policing. A family already subjected to intensive child protection surveillance may appear in government data as presenting greater risk, leading to further intervention.
Structural racism becomes a feedback loop and is relabelled as prediction. The promise that a human will remain involved is not enough. If a worker accepts an automated recommendation without understanding it or lacks the authority to change it, there is no meaningful human oversight.
People must have the right to know when an automated system has influenced a decision, understand the reasons, correct inaccurate data, obtain human review and seek a remedy.
Some uses of automated decision-making may be so dangerous that they should not be permitted.
Responsibility matters
Our Relational Futures research starts from a different position to much of the current AI debate. Rather than beginning with what AI can do, we ask what relationships it creates, who holds power within those relationships and what responsibilities follow.
AI connects people, data, governments, corporations, communities, Country and environmental resources. These relationships distribute power, benefits and harm. They are never neutral.
In our recent article, ‘Indigenous Data Sovereignty and the Governance of Artificial Intelligence: Toward Equitable Benefit for Indigenous Peoples‘, we argue that AI will not benefit Aboriginal and Torres Strait Islander peoples equitably unless we have authority over the data and systems that affect us.
Indigenous data sovereignty means that Indigenous peoples have the right to govern data about our peoples, communities, lands, cultures and knowledges. Decisions about our data should not be made solely by governments, researchers or companies.
We must be able to determine what is collected, how it is interpreted, who can use it, for what purposes and who benefits. This is especially important in health.
Aboriginal and Torres Strait Islander health data may be collected to provide care, then later reused for research, product development or AI training. Access to data does not provide an automatic right to use it for another purpose.
We are also more than data. Aboriginal and Torres Strait Islander languages, images, stories and cultural expressions cannot be reduced to content available for extraction.
The Prime Minister promised to protect Australian writers, artists, musicians and journalists from the unauthorised use of their work in AI training. That protection must extend explicitly to Indigenous Cultural and Intellectual Property.
Australia is not empty space
The Prime Minister described Australia as having “a continent to ourselves” and “the space to build”.
This language is troubling. Australia is not empty space waiting for data centres. It is made up of Aboriginal and Torres Strait Islander Countries, each with its own peoples, histories and systems of responsibility.
Data centres occupy land, consume water and energy, rely on mineral extraction and generate waste. Aboriginal and Torres Strait Islander communities already experience disproportionate exposure to climate change, water insecurity, environmental degradation and extractive development.
The Government’s proposed regulation of data centres must consider more than investment and infrastructure. It must address cumulative environmental and health effects and recognise the authority of communities whose Country will carry the burden of Australia’s AI ambitions.
No data centre should be built on or affect Country without the informed participation and consent of the relevant Aboriginal or Torres Strait Islander peoples. Communities must also share in the material benefits.
Will we be included?
The creation of an Office of AI raises an immediate political question – will Aboriginal and Torres Strait Islander people be included?
The Prime Minister provided no indication that the Office would include Aboriginal and Torres Strait Islander leadership or expertise. There was no commitment to Indigenous representation, no recognition of Indigenous data sovereignty and no explanation of how the Office would work with Aboriginal Community Controlled Organisations.
This cannot become another government body that makes decisions about us without us. Aboriginal and Torres Strait Islander people must hold senior decision-making positions within the Office, including people with expertise in data governance, health, technology, law and community-controlled services.
The Government should explain how Aboriginal and Torres Strait Islander people will participate in the Office, how Indigenous data and cultural knowledge will be protected, and what power the Office will have to stop harmful systems.
The new Australian Standards should prohibit fully automated adverse decisions in essential public services, require Indigenous-led assessments of high-risk systems, establish a public register of automated systems and guarantee explanation, human review and effective remedies.
Governments and companies must remain legally responsible for harms. Responsibility cannot be shifted to an algorithm.
The public health community must engage before the standards are settled through negotiations dominated by governments, technology companies and investors. Health equity, anti-racism, Indigenous rights and environmental justice cannot be added later.
A relational approach asks not only what can be built, but what relationships we are creating and whether we are prepared to accept responsibility for them.
The Prime Minister has called for AI to be developed in Australia’s interests. Yet his speech gave no indication that Aboriginal and Torres Strait Islander peoples are included in that vision or will have a place in the new institution created to deliver it.
The Government must answer a straightforward question – will Aboriginal and Torres Strait Islander people have genuine decision-making authority in Australia’s AI future, or will decisions about our data, communities and Country once again be made without us?
We should not have to ask whether we are included. We should be helping to decide.
Author details
Bronwyn Carlson is a Distinguished Professor at Macquarie University and an internationally recognised scholar in Critical Indigenous Studies. Her research examines Indigenous digital life, data sovereignty, and the social impacts of emerging technologies, with a strong focus on institutional accountability and gendered violence. She is Deputy Director Indigenous at the ARC Centre of Excellence for the Elimination of Violence Against Women, where she leads Indigenous research and training initiatives across Australia and the Indo Pacific.
Carlson’s work shapes national policy, supports community led governance, and advances the next generation of Indigenous researchers through mentorship, collaboration, and globally influential scholarship programmes. She has advised government, industry, and community organisations, and her work continues to influence debates on digital governance, safety, and Indigenous futures. She is also the founding and managing editor of the Journal of Global Indigeneity and the Director of The Macquarie University Global Indigenous Futures Research Centre. In 2020 she was elected as a Fellow of the Australian Academy of the Humanities for her research focused on Indigenous peoples and digital technologies.
See Croakey’s archive of articles on AI and health
Add Comment