Towergate Insurance has issued a warning over the growing use of artificial intelligence in business, highlighting the risk of so-called “AI hallucinations” and the potential for misinformation, data exposure and cyber-related incidents.
Marc Rocker, head of cyber at Towergate, said that AI systems do not interpret information independently but rely on existing data to generate responses. If that data is inaccurate or manipulated, the output can be flawed.
“An AI hallucination is when the AI generator produces an answer that is completely incorrect but may appear plausible,” he said. Rocker added that the confident tone often used by AI chatbots can make such misinformation appear credible.
He noted that while many of these errors are unintentional, there is also the potential for deliberate manipulation of AI systems through targeted attacks. In such cases, businesses could be misled into believing and acting on false information.
Towergate is advising organisations to take a cautious approach to AI-generated content, particularly when it is used for customer communications, public materials or in sensitive sectors, such as healthcare. Rocker recommended verifying AI-generated information using independent, reputable sources and advised against relying solely on outputs from generative tools.
“Human fact-checking remains crucial when handling any information supplied by AI,” he said.
The company also raised concerns around data security, particularly when business information is input into publicly available AI tools. Rocker said that users may not always be aware of where that data is processed, how it is stored, or who has access to it.
In addition to internal safeguards, Towergate is encouraging firms to consider cyber insurance as part of their overall risk management strategy. In addition to internal safeguards, Towergate is encouraging firms to consider cyber insurance as part of their overall risk management strategy. “Using AI for your business is not without risk,” Rocker said, adding that cyber cover may provide support if something goes wrong.
Towergate is currently assessing how AI exposure may affect other insurance lines and said it is working with clients to help identify and manage potential risks linked to the use of emerging technologies.
