Per-Department LLM Selection: Different Brains for Different Jobs
Not every department needs the same model. Run a reasoning-heavy model for R&D, a fast cheap one for customer service, a compliant one for legal.
Different work, different brains
The instinct to standardise on a single model across the organisation is understandable. Procurement is simpler, support is simpler, and there is one bill to track. The problem is that the work different teams do is not similar enough for one model to be the right answer everywhere.
An R&D team chewing through technical specifications needs different capabilities than a customer service team handling hundreds of short conversations a day. Legal needs a different compliance posture than marketing. Forcing all of them onto the same model means at least one of them is paying too much, getting too little, or both.
Per-department model selection is what stops "AI strategy" from being a single procurement decision and starts treating it like the operational decision it actually is.
Three department archetypes
Most organisations have at least these three. Your structure may split or combine them differently, and the model fit will shift with it. The principle stays the same: pick the model that fits the work, not the org chart.
R&D and engineering
Long technical documents, code reasoning, multi-step problem solving. The work justifies a top-tier reasoning model and is usually low volume per user, so cost per token is less critical than capability.
Frontier reasoning model
Customer service
High volume, low variance, often multilingual. Speed and cost per response matter more than the deepest reasoning. A mid-tier model usually outperforms a frontier model on response time and stays well within budget.
Fast mid-tier model
Legal, HR and compliance
Sensitive data, factual grounding, sometimes strict residency requirements. The right pick is often a model with strong RAG behaviour and a regional or self-hosted deployment that satisfies the compliance team.
Compliant or self-hosted model
Switching feels like a toggle, not a project
The reason per-department selection only works in practice is that switching has to be cheap. If changing models means re-ingesting documents or retraining users, nobody does it.
What stays the same when you switch
- Pick the model per department from the admin dashboard
- Each department keeps its own knowledge base, prompts, and feature toggles
- Switching a model leaves all of that intact
- Test a new model in one department before rolling it out anywhere else
- Track usage and cost per department to spot the model that is overkill or underpowered
A practical pattern
A common starting point we see in the field:
- R&D on a frontier reasoning model
- Customer service on a fast mid-tier model
- Legal on a regionally hosted compliant model
- Everyone else on a sensible default
After three months, the data tells you whether to upgrade or downgrade each department. Not your gut.
Related articles
Ready to see it in action?
Schedule a personalised demo and see how the Plainsight AI Assistant fits your organisation.