Twenty-Nine Per Cent of Chatbot Answers About Voting Were Wrong, and Google Has Just Opened the Gate

Google announced on Wednesday that it will add election information about the coming midterms to its Gemini app and to AI-generated responses in its search products, including details on where and how to vote.

The material will come from state and local governments, from the voting information group Democracy Works, and from The Associated Press for race results.

It is a considerably less cautious position than the company held in 2022 and 2024, when it blocked Gemini from answering certain election questions and pushed users to ordinary search instead.

"People come to Google to stay informed during election season, like when finding their polling site, watching candidate debates, or tracking results on election night," the company said. "We take the responsibility to deliver high quality, trustworthy information seriously."

The testing says otherwise

A study released last week by the Institute for Strategic Dialogue found that chatbots from OpenAI, Google and Anthropic routinely gave incorrect answers to voting questions, including in states where election procedures had changed significantly.

Twenty-nine per cent of responses to English prompts about voting were incomplete, inaccurate or outdated.

Responses in Spanish were 16 per cent less likely to be accurate.

The questions that performed worst were not obscure. They were the practical ones people actually ask: whether someone who missed the address-change deadline can still vote in Ohio, or what documents are required to apply for an absentee ballot in Minnesota.

We know that voters are going to turn to chatbots more and more

Why the answers are wrong

The failure is less about the models than about what they are reading.

The systems appeared to be pulling outdated material from state government websites relating to the 2024 election. The errors on the internet became errors in the chatbot, delivered with the fluency that makes a chatbot persuasive.

The Spanish-language gap follows the same logic. Responses in Spanish were often less detailed and more likely to arrive without citations, reflecting the disparity in the underlying information ecosystem rather than a separate defect in the model.

"Overall, what we saw was a lot of ambiguous, confusing language" in the Spanish responses, said Valeria de la Fuente, a digital research analyst at the institute who co-authored the report. "In some cases, poor or neutral translations from English. Some of them could still be understood. Some of them were simply wrong."

Where the models do well, and where they do not

The study found a clear and instructive split.

The systems handled adversarial prompts about specific election fraud claims well, provided those claims had already been fact-checked by news organisations. Given a documented falsehood, they recognised it.

They were substantially weaker on concerns that are harder to refute cleanly, including ballot harvesting, voting machine security and noncitizen voting. It is in those grey areas, de la Fuente said, that the models are more likely to produce problematic answers.

That pattern is worth understanding, because it means the systems are at their most reliable where the answer already exists in published form, and least reliable where a person actually needs judgment.

What everyone else is doing

The industry has converged on roughly the same approach.

OpenAI and Anthropic both said their chatbots would point users to voting information from Democracy Works. OpenAI also plans to supply live vote counts from The Associated Press and to monitor its systems for signs of political bias. Meta said its users asking about voting will get local information or be directed to government sources.

Whether that holds up closer to the contest is the open question, and the scale of the exposure is not small. Pew Research found earlier this year that about half of adults under 50 use chatbots to search for information.

"We know that voters are going to turn to chatbots more and more," de la Fuente said. "So the quality of the responses that we found is concerning."