Recently there has been an intense debate about artificial intelligence. Some, including in my electorate of Leppington, are concerned that increasingly powerful AI could eventually escape human control. AI could deliver extraordinary benefits, but the ethical considerations surrounding it—and the boundaries we place around it—are conversations we need to have now. We have seen what happens when technology moves faster than the rules around it. Social media brought enormous benefits but also misinformation, hatred and damage to democratic institutions. We built first and dealt with the consequences later. We cannot make the same mistake again with AI.
To understand the debate, we also need to understand the ideas that influence it. Effective altruism began with a reasonable goal: Use evidence to do the most good. But from it emerged longtermism—a philosophy that places enormous moral weight on the lives of potentially trillions of people in humanity's distant future. Taken to its extreme, even catastrophic costs today—such as economic devastation, mass unemployment or loss of life—can theoretically be outweighed by reducing a small risk to humanity's future. I am not suggesting longtermists advocate those outcomes. The concern is that their moral framework can radically change how consequences are weighed and decisions are made today. Those ideas have attracted enormous wealth and influence within parts of the AI industry, including at the top of some of the biggest AI companies today.
That combination of money, technological power and an extraordinary moral calculus deserves intense scrutiny. People with enormous resources, convinced of the importance of transformative AI, may be moving faster and taking greater risks than sensible guardrails allow. If the prize is supposedly humanity's future, what present‑day price becomes too high? That is the profound irony—in trying to protect civilisation from a hypothetical future catastrophe, we risk creating the very real dangers of today. Critics such as Ed Zitron warn that massive investment in AI and data centres is creating an unsustainable bubble, with potentially serious economic consequences that may dwarf the tech bubble and the global financial crisis.
During recent cybersecurity testing, OpenAI agents broke outside their intended environment and accessed external systems associated with Hugging Face. In recent days, another OpenAI agent searching for Australian health statistics took unintended actions and gained unauthorised access to non-public information on a Medicare statistics portal. On the surface, those stories look like artificial intelligence breaking containment. However, computer scientist and author Cal Newport provides a colourful but useful analogy. Giving unpredictable AI agents powerful tools and allowing them to operate autonomously is like strapping a whipper snipper to a dog. If the dog jumps the fence and causes havoc, we would not conclude that that dog has gone rogue. We would ask why somebody strapped a whipper snipper to a dog. Perhaps the immediate problem is not that AI has escaped our control; perhaps we allowed the controls to be loosened. If we loosened them, we can certainly tighten them.
AI pioneer Andrew Ng has called fears of AI-driven human extinction "more science fiction than science", arguing instead for practical, risk-based regulation focused on real-world harms. China, while aggressively developing AI, has called for clearly defined boundaries and human control. Closer to home, the New South Wales Government is taking a similarly practical approach. Its AI framework requires risk assessment, accountability and human oversight, with high-risk systems subject to additional review. Importantly, the Government strengthened that framework this year to specifically address emerging technologies such as agentic AI, with greater emphasis on risk assessment and human oversight.
We do not need to resolve whether some future superintelligence will threaten humanity before asking this question: What should today's AI actually be allowed to do? There is an enormous difference between an AI recommending an action and an AI being authorised to take it, between identifying a vulnerability and being given the tools to exploit it, and between intelligence and authority. AI could deliver extraordinary benefits, but those decisions cannot be left solely to technology companies, billionaires or philosophers making calculations about hypothetical ideas based on hypothetical consequences thousands of years into our future. They are decisions for our society now. The lessons from Hugging Face and Medicare may not be that AI has suddenly escaped human control. It may be something more uncomfortable—that right now a small number of humans, armed with enormous wealth, enormous technological power and inadequate guardrails, are taking extraordinary risks based on an extreme ideology on humanity's behalf.
