20 Questions to Ask in Every WealthTech AI Demo

O'Connell, J. (2026, July 24). 20 questions to ask in every wealthtech AI demo. WealthManagement.com. https://www.wealthmanagement.com/financial-technology/20-questions-to-ask-in-every-wealthtech-ai-demo

• The article presents a 20 question due diligence framework organized into five levels, to be worked through in order, on the premise that a vendor who cannot answer Level 1 makes Level 2 irrelevant.

• Buyers are advised to prepare their own requirements list before the demo, documenting firm specific use cases, client workflows needing improvement, and compliance constraints, rather than letting the vendor set the agenda.

• Level 1 covers governance: what data the platform captures and where it goes, how personally identifiable information and client financial data are controlled against model or software team training use, data retention and post termination handling, and what compliance certifications or third party audits the AI system has undergone in the past twelve months.

• The 2024 Regulation S-P amendments impose documented vendor oversight obligations on every software tool that touches client data, with a compliance deadline for smaller RIAs of June 2026.

• Level 2 probes architecture transparency, including whether the models are proprietary, licensed, or built on a third party API such as OpenAI or Anthropic, what training data was used, where processing occurs, how hallucinations and bias are handled, whether temperature or confidence settings are configurable, and whether the system has ever produced incorrect output affecting client communications.

• Level 3 is the "turn it off" test, asking what functionality remains without the AI, for a side by side workflow comparison, what the system does that a rules based engine could not, and where the human sits in the loop, with Advisor360 research cited finding that 93% of advisors want final human review authority over any AI influenced output.

• Levels 4 and 5 require AI specific performance data rather than platform metrics, a measured error rate, third party validation, a reference client running the feature in production for at least twelve months, and a clear split between production, beta, and planned capabilities as well as between contractually guaranteed and aspirational roadmap items.

Knote: There you have it from The Oasis Group, which is rapidly becoming an authority on WealthTech AI. Note that Level 1 is all about governance. If that level does not pass, no need to proceed any further. It may be a long read but it is a very short masterclass.