We have all been turning on and tuning in to artificial intelligence since OpenAI released its first ChatGPT large language model search almost four years ago. Indeed, it’s been a far-out trip that has increasingly improved with time—essentially offering a search engine on steroids.
Yet, hallucinations have also been a part of that trip in its entirety (including all previously released versions of large language models, or LLMs), which involve highly convincing yet erroneous insights, leading users down a path of misinformation.
“One commenter referred to it as ‘competent bullshit,’” says David R. Canton, a lawyer at Harrison Pensa in London, Ont., who specializes in technology and privacy law.
He wrote a 2025 post discussing how hallucinations cost Canadian businesses money. In it, he points to a study finding generalized LLM tools like ChatGPT achieved an average of 50 per cent accuracy on simple tasks for financial analysts. In short, hallucinations present risks for family offices using AI, he adds.
AI hallucinations and family offices
Despite ChatGPT, Google Gemini, DeepSeek, Microsoft CoPilot, Grok, Anthropic’s Claude AI and other LLM providers improving their accuracy, hallucinations remain common. For instance, the 2026 Stanford HAI AI Index found hallucination rates for 26 top generative AI models range from 22 to 94 per cent.
The challenge for family offices—as for all AI users—is discerning truth from hallucination, says Spencer Clark, head of private markets and thematic investments at Richter Family Office.
“Some queries can lead to straight-up errors, but the challenge is that these often sound very plausible,” he says, noting Richter uses AI tools vetted by its information technology team. Richter—like many other multi-family offices—uses generative AI for preliminary research, while recognizing hallucinations are a significant risk.
“From our side of investment research, it is high stakes where you want to make sure that everything you’re doing is verified,” Clark says. “So, we have been using it [AI] to help parse through investment research information, but we’re always double-checking information.”
For family office professionals and the families that they represent, understanding what AI hallucinations are and their cause can be helpful in spotting the junk. AI platform provider supermind.ai defines a hallucination in its simplest terms: when “an AI system confidently makes something up.”
Faithfulness hallucinations versus factual hallucinations
Hallucinations can be broken down into more technical categories, it notes. These include “faithfulness hallucinations,” whereby a model adds unsupported information when summarizing a document. Another type more commonly recognized by users is a “factual hallucination,” involving claims not grounded in reality at all.
Their cause stems from LLM tools being prediction systems, designed to generate plausible text based on patterns learned from training data and context, supermind.ai notes.
AI chatbots seem like they’re conversing with us. Yet underneath, the word salad gets tossed differently to create meaning.
“We may call it ‘artificial intelligence,’ but it’s not true intelligence,” Canton explains. “When you ask it a question, AI models don’t really understand the question.”
The answer results from its complex algorithms trained on large datasets to predict the next most likely word or phrase in relation to the user prompt.
“AI is more like a parrot than Data from Star Trek,” he says, referring to the hyper-intelligent fictional android from Star Trek: The Next Generation. He adds that it doesn’t quite understand what it’s doing; it’s more just repeating stuff it has seen.
In turn, the datasets AI models use are critically important to reducing hallucinations. When they use a well-defined dataset with established, verified information, they can be very useful. “I use an AI-based trademark search tool in my trademark practice, and it’s marvelous compared to the old way of doing it, but it’s created specifically for that use,” Canton says.
Even then, the need to double-check findings remains. Generative AI leveraging large, public datasets like the Internet should be viewed with much more skepticism. That’s especially when information is limited on specialized subject like private investments, Clark says.
These models tend to “grasp at straws within the public domain to find information that is generally sparse,” he adds, noting that Richter clients have used generative AI searches to research private funds. “What gets spit out tends to be a mix of accurate stuff with anecdotal, and other elements that are completely wrong.”
Investment professionals typically recognize AI’s potential along with its hallucinogenic perils, says Milla Craig, president and chief executive officer of Millani Inc., a responsible investing and corporate sustainability advisory services provider based in Montreal.
“One of the key elements is that there has to be human oversight,” she says. Institutional investors recently surveyed by Millani grasp the need for verifying AI-generated insights due to the high possibility of hallucinations. She points to evolving governance ensuring AI hallucinations do not end up being incorporated into research and decisions.
The wealthy have different concerns of privacy and are higher-profile, so how they are being portrayed by generative AI to the world is something they should be concerned about.
Ronn Torossian of 5W AI Communications
Risks for family offices
The risk for family offices doesn’t just pertain to investments; its increased use in the legal world is also potentially problematic. “There have been a huge number of instances where lawyers have gone to court and filed legal briefs that have been created by AI and they’ve gotten into massive trouble,” Canton says.
One lawyer based in Paris has created a database compiling AI hallucinations in legal cases globally, including 175 instances in Canada. AI hallucinations also present reputational risk for family offices. Generative AI searches may be spewing erroneous summaries about families and their businesses, says Ronn Torossian, founder and chairman of 5W AI Communications in New York.
“The wealthy have different concerns of privacy and are higher-profile, so how they are being portrayed by generative AI to the world is something they should be concerned about.”
5W recently published a Hallucination Index examining AI responses for five major fashion brands to 25 core business questions. It found about 20 per cent of the answers contained material errors.
Among the mistakes were founder erasure, whereby co-founders disappeared from company descriptions. Another was entity confusion—conflating information about two different companies with similar names.
Torossian suggests family offices must be proactive in curating their reputation online in this new search engine era. “SEO is dead,” he says of search engine optimization for tools like Google and Bing to ensure a relevant website appears at or near the top of a web search.
Family offices should now focus on “GEO,” or generation engine optimization, he says, pointing to tech firms providing a service that rectifies hallucinations in searches for clients.
Canton says family offices can start curating on their own, simply by doing generative AI searches on their family members and businesses to at least know if they are victims of hallucinations.
As well, he recommends using pay-for-use LLM tools for research, which are based on better datasets than free tools. Even then, professionals serving institutional and ultra-high-net-worth clients must understand hallucinations—even as they become less common over time—remain a concern, Craig says.
“You always have to keep saying to yourself, ‘Trust, but always verify.’”
Joel Schlesinger is a Winnipeg-based freelance writer who has written for Canadian Family Offices since 2021. Specializing in investment, wealth advice, real estate and personal finance, he is also a regular columnist for the Winnipeg Free Press, and his work regularly appears in The Globe and Mail, Calgary Herald and Edmonton Journal.
The Canadian Family Offices newsletter comes out on Sundays and Wednesdays. If you are interested in stories about Canadian enterprising families, family offices and the professionals who work with them, but like your content aggregated, you can sign up for our free newsletter here.
Please visit here to see information about our standards of journalistic excellence.