In her Echo column last week, Mandy Nolan recommended literature for her daughter, including classics like Catcher in the Rye and To Kill a Mockingbird.
At risk of impertinence, I want to add to her reading list:
E.M. Forster’s The Machine Stops describes a world where humans spend their days communicating with others via instant messaging devices, alone in tiny individual dwellings. Technology meeting their every need. Until it doesn’t. The short story was published in 1909.
AI study released
Last week, a group of academics and pollsters released a fascinating study on how Australian workers are experiencing the latest wave of technology: artificial intelligence, or AI.
The five-month qualitative study involved in-depth interviews and multiple focus groups with nurses, retail employees and public servants. It found workers being treated as ‘invisible bystanders’ regarding AI, ‘leaving organisations, employees, and the public at risk of missed opportunities and significant harms.’
Workers were generally not opposed to AI. They saw both opportunities and dangers. Nurses were worried about AI impacts on dispensing drugs and diagnoses.
Retail workers were concerned by automated checkouts and surveillance. Public servants saw threats to trust through scandals like Robodebt.
The study authors – from the University of Technology and the Essential polling company – made several recommendations. Embedding workers’ voices in AI development. Limiting the use of AI for surveillance. New workplace safety rules to protect people from the dangers of the new technology.
Malicious malgorithms
The Royal Commission into the infamous Robodebt tax scandal similarly made far-reaching recommendations. After finding widespread use of secret, inaccurate, and illegal algorithms, the commissioner recommended new powers to scrutinise automated decision-making, to ensure fairness and avoid bias.
Robodebt is a well-known case of AI harming many lives, but it’s not the only one.
zIn the UK, more than 900 post-office workers were wrongly prosecuted for fraud and theft in one of greatest miscarriages of justice ever. Over a decade of official cover-ups and denials, many people faced bankruptcy, some went to jail, and some even died prematurely.
The reason? Flaws in an automated IT system.
In 2023, the Australian Information Commissioner made damning findings against Serco, the company which runs immigration detention centres. It found Serco’s ‘Security Risk Assessment Reports’ produced inaccurate and incomplete information that caused harm to some detainees. As with the Robodebt and post-office scandals, a key problem with Serco’s inaccurate risk assessments was the algorithm.
Of course, algorithms and artificial intelligence can bring us benefits, but only if we can much better prevent their potential harms.
In the medical world, there’s much hype about what AI can deliver. But a recent review of all the relevant studies, published in The Lancet, is sobering. Most studies in the review reported positive findings, with many showing AI was better at picking up potential disease.
But The Lancet review raised concerns about the scientific quality of the studies of AI, and how applicable they were in the real world. Moreover, it found little strong evidence that AI was leading to any improvement in actual health outcomes for people.
Importantly, the studies of AI were not tending to investigate whether the enhanced ‘diagnosis yield’ – which arose when algorithms read X-rays for example – was identifying genuine disease, or simply finding potential problems that would never actually cause any harm.
Managing AI risks
Barely a week passes without news of AI’s risks, and how we might best manage them.
Last month, leading AI scientist, Max Tegmark, reiterated calls for a pause on advanced AI research, and like other experts, he’s demanding much stronger regulation to rein in out-of-control AI.
Last week, former and current employees at high-profile AI companies, including Open AI and Google, released an open letter about the technology’s opportunities and dangers. Arguing AI can deliver ‘unprecedented benefits’ the group also warned of ‘serious risks’ including entrenching inequality, misinformation, and ‘loss of control of autonomous AI systems potentially resulting in human extinction.’
The open letter rejected the idea that industry could govern these risks itself, and it argued for stronger government oversight and protections for whistle-blowers.
Not everyone is an invisible bystander.
The queer visionary, E.M. Forster, is best known for his classic novels, like A Passage to India and A Room with a View, both transformed into lush cinematic dramas. But one of the lesser-known themes bubbling through his work over a century ago was how to maintain vital human connection in the face of a growing technological imperialism. Forster may ultimately be better remembered for The Machine Stops.
♦ Ray Moynihan is an academic working at Bond University and the University of Sydney, currently coordinating a Junior Landcare project with Brunswick Valley Landcare.
For four decades The Echo has printed the stories some people loved, some people hated, and some pretended not to read. If you want us to keep telling the truth, the real truth, not the sugar-coated version. We’ll need your support to keep the presses rolling. 
