Introduction by Croakey: An investigation by The Guardian recently revealed that Federal Parliamentary inquiry processes are being “flooded with AI-generated material”, with dozens of policy submissions found to incorrectly summarise real research and invent or wrongly cite sources.
“In some instances, Google’s AI summary and ChatGPT will also cite the inquiry submission with the fake reference as a source, creating an ongoing cycle of misinformation,” reported The Guardian.
This growing risk to the integrity of our democratic processes and to areas of urgent public interest comes as a detailed analysis of the economic implications of AI has been prepared for the Treasurer to inform the 2026 Intergenerational Report and work on an overarching AI national strategy.
While reporting on the Treasury analysis has focused on implications for productivity, it adds to concerns about misinformation in noting that AI will likely be adopted most by information, telecommunications, professional services, finance and insurance sectors, affecting jobs of course but also information and communication.
That has critical implications for the health sector, which will have to keep abreast of the ever increasing complexity of forces undermining the integrity of online information, with implications for policy, practice, education, communications, and public understanding of health issues.
Meanwhile, RMIT’s Dr Dana McKay and Dr Damiano Spina explain below why the quality of AI summaries is set to become even more unreliable. Their article was first published at The Conversation.
Dana McKay and Damiano Spina write:
For 30 years, the world wide web has run on a surprisingly profound social contract: most sites are free for search engines to access, but if you use their content you give credit by linking to the source.
Recently, that social contract has begun to collapse. Artificial intelligence (AI) tools are crawling sites not to link to them, but to train models and generate answers – which may or may not be accurate.
When you search for something, ChatGPT’s response or Google’s AI Overviews may still include links to sources, but they’re a kind of optional extra to the main answer.
This has triggered a bad dynamic for website owners, the public, and even AI companies themselves: as websites lose traffic (and revenue), many are beginning to block AI scraping tools, meaning AI results depend more on low-quality websites (many of which are also generated by AI).
As a result, good information can be harder than ever to find.
Changing traffic patterns
In the early days of the world wide web, search engines and content creators came to an agreement about crawling (the practice of technologically examining a site to index it, so it can be served up in search results). Content creators would provide access to their sites for free, and even allow search engines to reproduce small snippets of text.
In return, search engines provided links to the sites owned by content creators, who benefited from that web traffic. If content creators didn’t like the deal, they could prevent search engines from crawling their site with instructions in a file called robots.txt.
But if AI tools no longer provide web traffic, it cuts content creators out of the economic loop.
There are also other costs associated with each visit to a website, so AI crawling can cost website providers money while not giving them any of the ad or other revenue that would come from human traffic. AI crawlers also crawl more deeply and more intensely than traditional web crawlers, magnifying that cost.
This change in traffic patterns isn’t a small or hypothetical problem. Cloudflare, a web hosting and service company that manages 30 percent or more of the top 10,000 sites on the internet, estimates over half of all web traffic is now AI bots.
Some of this will be AI agents supervised directly by people, but the majority will be crawlers. Site owners can use robots.txt to ask AI crawlers to stay off their sites – but some AI companies may ignore this polite request.
If the AI companies do honour the request, that can create a different problem. Sites containing misinformation are far less likely to ban AI crawlers, so the AI answers won’t be informed by high-quality sources.
D-day
On the horizon is an event dubbed “Google Zero” – the day when through-traffic from Google drops to nothing. While some grey-haired diehards (like one of the authors of this piece) might still click through to verify AI answers, this traffic is rapidly dwindling, as a direct result of AI summaries.
A study of Wikipedia confirms this, showing that traffic in the English language version of the site dropped off quickly with the launch of AI summaries on Google in English, and that the same pattern occurred in other languages as AI summaries were rolled out.
Never having to click through to get an answer might seem great for information seekers, but the reality is more complex.
Many sites are now blocking AI crawlers altogether. Site owners who decide to block AI crawlers are less likely to be linked in AI Overviews answers, even when the AI tool can still access the content to ground its answers (using a technique called retrieval-augmented generation).
Alternative “pay to crawl” models have been suggested as a way to compensate content creators, but haven’t gained traction.
Come 15 September, Cloudflare sites will block AI crawlers by default on pages that contain advertising (and therefore make money for content creators).
This means up to 30 percent of the world’s top sites will no longer appear in Google AI Overviews summaries. It also means that much of what AI is being trained on will itself be AI-generated text.
Quality suffers
So what does this mean when you’re looking for information?
The quality of AI summaries is likely to go down, at least in the short term, while the new economics of the web get sorted out.
This will happen for two reasons. The first is that high-quality content is less likely to go into those AI summaries – one recent study found that already, around one in six sources used by AI search tools is itself an AI-generated website.
The second reason is that, as AI models are trained on more AI text, their output may degrade (a phenomenon known as model collapse).
As a result, search engines that depend less on AI may become more reliable. The challenge is finding one that doesn’t use an AI-based crawler.
They do exist: ZDNet recommends Mojeek, PCMag recommends Brave, and Ban the Bots lists several, including one specifically for “small producer” content such as blogs.
For now, whatever search engine you’re using, the best thing you can do is to scroll down and click on some actual search results.
This benefits content creators, and is also more likely to give you more accurate information.
Author details
Dr Dana McKay is associate dean, interaction, technology and information at RMIT’s School of Computing Technologies. She has received funding from the Australian Research Council, and from Google. Her primary interest is in ensuring that advances in information technology make the world fairer and more equitable. To do this she focuses on how people interact with information, and how technology shapes that interaction.
Dr Damiano Spina is a Senior Lecturer at RMIT University (School of Computing Technologies), an Associate Investigator at the ARC Centre of Excellence for Automated Decision-Making and Society, RMIT Research Leader at the Australian Internet Observatory, and a Member of the Association for Computing Machinery (ACM).
See Croakey’s archive of articles on misinformation and disinformation; and AI and health
Add Comment