A recent study by the BBC found that many popular chatbots struggle to summarize news articles accurately. The BBC tested four major systems—ChatGPT, Copilot, Gemini, and Perplexity AI—by asking them to summarize 100 articles from its website.
More than half of the chatbot-generated summaries contained mistakes. Some errors were minor, like incorrect dates or numbers, but others were more serious, misrepresenting key facts or mixing opinions with factual reporting.
One major mistake came from ChatGPT, which incorrectly stated that the UK’s National Health Service (NHS) does not support vaping to help people quit smoking. In reality, the NHS does recognize vaping as an option for smokers trying to quit.
Deborah Turness, CEO of BBC News and Current Affairs, warned that unreliable news summaries could mislead the public and damage trust in journalism. She called for closer collaboration between news organizations and technology companies to improve the accuracy of these tools.
Key Takeaways
A BBC study reveals major chatbots struggle with accurate news reporting, leading to calls for better collaboration between tech companies and news organizations.
- More than half of the chatbot-generated summaries contained mistakes ranging from minor errors to serious misrepresentations.
- Microsoft’s new Copilot Vision tool can analyze images and provide real-time insights on webpages without storing user data.
- OpenAI’s Deep Research offers detailed reports for professionals but faces concerns over potential bias in its source selection.
Microsoft introduces Copilot Vision
Microsoft is rolling out a new feature called Copilot Vision, designed to help users interact with online content more effectively. Unlike traditional chat assistants that process only text, Copilot Vision can also analyze images and other visual content on a webpage.
The tool is designed to give real-time explanations and insights as users browse. Microsoft has stated that Copilot Vision does not store user data or track browsing history, addressing privacy concerns often associated with digital assistants.
The feature is still in the testing phase, with an initial rollout planned for select high-traffic websites before expanding further. Microsoft aims to make online research more intuitive, helping users quickly interpret complex information without needing to leave the page they are viewing.
AI assistants do not discern between facts and opinion in news coverage; do not make a distinction between current and archive material; and tend to inject opinions into their answers.
Deborah Turness
OpenAI introduces Deep Research for detailed reports
OpenAI has launched Deep Research, a tool designed for professionals who require more in-depth analysis. Unlike traditional chat assistants that provide quick answers, Deep Research takes up to 30 minutes to gather and analyze information from multiple sources before delivering a report.
This tool is intended for industries such as finance, engineering, and policy analysis, where detailed and accurate information is crucial. OpenAI has hinted that future updates will include visual aids like charts and data visualizations to make reports easier to understand.
However, some experts have raised concerns about how Deep Research selects its sources. If the tool pulls from inaccurate or biased data, it could still produce misleading reports.
World leaders disagree on regulations at Paris summit
Global leaders gathered at a recent summit in Paris to discuss how to regulate emerging technologies. While countries such as France, China, and India signed an agreement calling for stricter rules, the United States and the United Kingdom refused to join.
U.S. Vice President J.D. Vance argued that excessive regulations could slow innovation and put businesses at a disadvantage. On the other hand, French President Emmanuel Macron pushed for stricter oversight, warning that without clear rules, technology could pose ethical and security risks. Some governments believe stronger oversight is necessary to prevent misuse, while others fear that too many restrictions could limit growth.
It was fascinating to be part of this research into AI and news content at the BBC.
Top line findings include: 51% of all AI answers to questions about the news were judged to have significant issues of some formhttps://t.co/qkD1GwJ203
— George Bowden (@georgebowden) February 11, 2025
The next round of discussions will take place in India, where world leaders will attempt to find a compromise. As technology continues to develop rapidly, countries, businesses, and regulators will need to work together to ensure it is used responsibly and ethically.
Collaboration between tech & news companies
The chatbots and automated tools have become more common, it is becoming clear that technology companies and news organizations must work together to improve accuracy. Misinformation can spread quickly when summaries contain errors, and without proper oversight, people may begin to lose trust in digital sources.
News organizations have expertise in verifying facts and ensuring balanced reporting. By collaborating, they can help prevent misleading summaries and develop better ways to share news with the public. As the debate continues, companies that offer human edited content, such as Monok Content Service, can help bridge the gap by ensuring that regulatory discussions, industry reports, and public updates are presented accurately and clearly.
The way people consume news is changing, and ensuring that information remains accurate and trustworthy should be a shared goal among developers, journalists, and regulators.
