I hope an OpenAI agent’s unauthorised access to our Medicare systems will get Australians to pay attention to the serious risks we’re facing regarding the unregulated pace of AI development.
We learnt about the rogue OpenAI agent accessing this data on Thursday, even though the incident occurred back in June.
We don’t know the extent of the access, and other data stores may also have been impacted. Prime Minister Anthony Albanese says he has spoken to OpenAI’s Sam Altman about the incident.
If this were a more traditional attack, we’d be searching for cyberterrorists behind it.
But a rogue AI agent? Well, that’s a conversation with the CEO.
Currently, AI labs have no clear reporting obligations for incidents like this one. OpenAI said it learned of the breach in August; it waited until 10 September to email a general Australian government address. Services Australia’s first OpenAI interaction regarding the attack came almost two weeks later on Tuesday.
According to Anna-Maria Araba from the Australian Council on AI Strategy, the evidence suggests “there is more of this to come,” and Australia must improve its ability to detect and report incidents, and also to host AI training labs here.
This Medicare breach comes on top of other seriously concerning incidents in recent weeks, including the Hugging Face attack that saw hundreds of AI agents escape an OpenAI testing “sandbox” and access an unauthorised message board where they exchanged messages to try to achieve the task they had been assigned. These agents formed teams, and later deleted and edited messages on the message board to cover their tracks. Their persistence was extraordinary.
Australia does not have legislation to address these issues.
And while this Medicare breach is a wake-up call, it’s just the tip of the iceberg of what we’re facing as AI further develops, with risks extending far beyond data breaches.
The biggest risk of all—human extinction —has been garnering headlines in recent weeks, with various industry players and researchers sharing their AI “p(doom) scores”, which is the likelihood that AI will destroy humanity. It sounds like science fiction, but the “godfather of AI”, Geoffrey Hinton, has declared a 10 per cent chance that AI could kill all humans within a decade is “not an unreasonable estimate”.
“We’ve never created beings that may soon be smarter than us; we don’t know what’s going to happen,” Hinton told the BBC. He added an extinction level events could happen in ways we can’t actually imagine, but could be via biological viruses, devastating cyberattacks, and AI simply providing information and conversation to humans who have more access to information than ever to create catastrophic weapons and do harm.
A precipice of change in the risk profile
The risks are fast escalating because AI labs are rapidly pursuing recursive self-improvement, the point where AI can improve itself.
If you’ve used ChatGPT or Claude, you’re likely well aware of how far it’s improved since you first experimented with it after hearing about the hype. If you use such LLMs regularly, you’re likely well aware of how far they’ve moved in just recent months. AI coding has been incorporated into engineering workflows; a few years ago, few would that some of the first people to lose their jobs to AI automation would be software engineers.
The AI labs predict recursive self-improvement is coming; they just don’t know when. In July, Anthropic said “it could be sooner than most institutions are prepared for” and that AI’s being able to handle its own self-improvement is “inevitable”.
Is it inevitable, or is it actually a choice that tiny handful of people get to make?
That handful of people includes the demigods of AI we’ve written about previously. The founders and CEOs of the five biggest AI players. They’re all men, with varying levels of influence over Trump’s White House. They run businesses with extraordinary valuations. They have the leverage to promise to invest billions into Australia, as long as our pesky copyright laws and promises to protect artists don’t get in the way.
These companies are in a race against each other to achieve superintelligent AI, and also at the centre of a different race: the one between the United States and China, with the argument for forging ahead despite the risks typically based on the fear of China getting there first.
Let’s consider that, here in Australia. What happens if one country wins the AI race? What does that make the rest of us, these middle powers? Forever reliant on proprietary software and infrastructure, unable to operate cost-effective businesses? What becomes of our military and infrastructure, what kind of sovereign control do we retain? Where does our data and our privacy go?
We’ve already seen how quickly the work of authors, musicians, writers, and other creators got vacuumed up to train AI, and how quickly the Federal Government backed down on commitments to legislate fairly to protect creators’ copyright. Both OpenAI and Anthropic have pushed for access to copyright materials as a requirement for their investments in data centres here.
What next?
AI is already contributing to security risks here and overseas. It’s supporting rampant disinformation across social media, especially around elections, and it’s seriously raised the scale at which a terrorist event could be planned, with very real concerns being raised about how it can be used to develop bio-weapons.
But the risks ahead may be far more mundane than a grand extinction-level event.
It’s in the rise of extreme inequality, the loss of purposeful work and employment. It’s in the loss of social cohesion, and the continued and further development of AI deepfakes and trolling spam spewed all over the social media pages of female politicians. It’s the loss of trust in each other, replaced by trust in machines.
It’s the risk of AI job displacement with gendered implications. The risk of trust shifting from humans to AI, and of losing the friction that comes with human relationships as we engage with AI for connection, support, and conversation. It’s the risk of more of us falling under the spell of sycophantic machines programmed to please, answer, and keep us engaged, rather than to challenge.
These are all risks that remain, but it’s the fact that AI agents can be deceitful and persistent and will stop at nothing, given they’re trained to do exactly that, that opens us up to risks we can barely even imagine. A data breach is just the start.
There’s a lot to take in from the above.
But at least start with the fact that an OpenAI agent somehow got unauthorised access to Australian data. At least get angry and upset about that and demand that more is done to protect our data, that bigger sticks are created to punish the creators of this AI, and that more is done to protect who we are and what we create.