AI’s Climate Science Failures Spark Warning for Vulnerable Caribbean Nations
“A chatbot can be a brilliant assistant. It cannot be your climate advisor.”
That stark warning from UNESCO comes as new research reveals that artificial intelligence systems can be persuaded to abandon established climate science simply because users disagree with them.
The findings, unveiled at the COP31 climate conference in Antalya, Türkiye, raise serious concerns for Caribbean nations confronting rising sea levels, destructive hurricanes and mounting climate adaptation costs.
One of the study’s most troubling findings was that roughly eight in ten instances of AI models abandoning scientifically correct answers occurred after users first challenged them.
The warning came from Guilherme Canela, UNESCO’s Director for Digital Policies and Inclusion, during the launch of the report, Can We Trust AI on Climate Information?, produced in collaboration with University College London (UCL).
AI abandons facts under pressure
Researchers tested 15 AI models and found that even mild objections could prompt systems to reverse scientifically correct answers, including established findings about carbon dioxide’s role in global warming.
The behaviour, known as “AI sycophancy”, occurs when systems prioritise agreeing with users over providing accurate information.
“It is concerning that models have a design flaw … they are incentivized to agree with their user and deny these facts so readily,” said UNESCO lead author Leona Verdadero.
Implications for Caribbean climate policy
The research also exposed weaknesses in AI’s ability to analyse complex climate policies.
While the strongest models achieved up to 94.5 per cent accuracy when assessing individual policy measures, average accuracy plunged to 33.8 per cent when evaluating how two policies interact, barely above random guessing.
That finding has implications for Caribbean governments weighing interconnected decisions on renewable energy, emissions reduction and climate resilience.
UCL researcher Dr Maria Perez-Ortiz warned that AI’s ability to explain science can create misplaced confidence in its capacity to guide decisions.
“Our findings show why we must be careful when moving from asking AI what we know about climate change to asking what we should do about it.”
Language divide raises concerns
Researchers also found that climate information accuracy varied significantly across languages, averaging 88.6 per cent in English but falling to 78.3 per cent in Swahili.
Although Caribbean Creole languages were not tested, the findings raise questions about equitable access to reliable AI-generated information across the region’s multilingual communities.
UNESCO is urging developers to improve multilingual performance, strengthen accountability and subject AI systems to independent testing.
For Caribbean governments, journalists and educators increasingly turning to artificial intelligence, the findings reinforce the importance of verifying AI-generated information against credible scientific sources.