A CTO's AI Buyer's Guide: Twelve Questions to Ask Every Vendor
Most AI vendor pitches sound the same. The differences hide in twelve questions about sovereignty, cost, operations, and real capability.
Most AI vendor pitches sound the same
Sit through three vendor demos in a row and you start to notice the pattern. The same hero claims, the same dashboard screenshots, the same "secure by design" badges, the same promise of department-level control. The slides are almost interchangeable.
The differences hide in the answers vendors give to specific questions. Twelve of them, grouped into the four areas where the trade-offs actually live: sovereignty, cost and lock-in, operations, and real capability.
Use this as a pre-call checklist. The list is the same whether you are evaluating us or somebody else.
Twelve questions, four buckets
Sovereignty
Where does my data live, what model touches it, and who can theoretically reach it.
- 01Where exactly is our data stored, and which legal jurisdictions can compel access to it?
- 02Which model providers will process our prompts, and can we choose between them per department?
- 03Who at your company has technical access to our environment, and under what conditions?
Cost and lock-in
What we will actually pay, what changes that, and how easy it is to leave.
- 04Is pricing per seat, per usage, or flat platform? What does the total bill look like at our scale?
- 05If we want to move to a different LLM provider in twelve months, what does that cost and how long does it take?
- 06What is our exit cost, in time and money, if we want to migrate to a different platform entirely?
Operations
How we run this in production once the demo is over.
- 07What does the audit trail show, and how quickly can we answer a question like "what did Finance ask AI last week"?
- 08Who manages users, departments, and permissions, and how does that integrate with our identity provider?
- 09When something goes wrong (bad answer, wrong action, cost spike), how do we investigate and roll back?
Real capability
Beyond the chat window, what can the platform actually do.
- 10How does the assistant use our internal documents, and how confident can we be in the citations?
- 11Which business systems can it actually act on (M365, ticketing, CRM), and how is permission scoped?
- 12Is this a chatbot wrapping a model, or a multi-agent platform with versioned prompts and tested behaviour?
How to use this list
Send four or five of these to a vendor before the first call. Watch how they handle the ones that do not have a clean answer.
Vague answers ("we can do that"), redirects to a future roadmap, or flat refusal to put numbers on cost and exit are all telling. So is over-confidence on questions that should have nuance.
Good vendors give you specific answers, name the trade-offs, and tell you which questions depend on your situation. Honest answers about what is hard are usually a better signal than polished answers about what is easy.
If you want to run the list past us first, that is what the demo button at the top of the page is for.
Related articles
Ready to see it in action?
Schedule a personalised demo and see how the Plainsight AI Assistant fits your organisation.