AI Governance: 4 Essential Bank AI Tools

AI governance, not model benchmarks, is what determines which AI tools a bank can actually put into production. The short answer: use Microsoft 365 Copilot for knowledge work inside an existing Microsoft 365 tenant, use Azure AI Foundry or Anthropic's Claude for document-heavy reasoning and controlled application development, use OpenAI where breadth of capability and speed matter more than deep Microsoft integration, and reserve open-source models like Llama for cases where data residency, air-gapped operation or extreme token volume make hosted services untenable. Everything below explains how to make that call defensibly, with an evidence trail your model risk and audit teams will accept.

What AI governance actually requires before you pick a tool

Banks do not get to evaluate AI tools the way a marketing agency does. Your selection has to survive three separate reviews: model risk management under SR 11-7 or its local equivalent, third-party and concentration risk, and privacy or data protection review. If your AI governance framework cannot produce a written answer to each of the following, procurement should not proceed.

• Data privacy and residency: where prompts, completions and embeddings are processed and stored, and whether the vendor trains on your data.
• Explainability and audit trail: can you reconstruct, months later, what was asked, what sources were used and what was returned?
• Integration: how the tool reaches your core banking platform, loan origination system, data warehouse and Microsoft 365 content.
• Customisation: grounding, retrieval, prompt libraries, fine-tuning and evaluation harnesses.
• Cost at scale: per-seat versus per-token versus per-GPU, and what happens at 10x usage.
• Regulatory fit: EU AI Act classification, DORA, GLBA, GDPR, and your regulator's expectations on human review.

The NIST AI Risk Management Framework is the most practical spine we have seen banks map their AI governance policy to, largely because examiners recognise it and it translates cleanly into control language.

Microsoft 365 Copilot and Azure AI Foundry

Microsoft 365 Copilot is the lowest-friction option for a bank already standardised on Microsoft 365. Prompts and responses stay within the Microsoft 365 service boundary, tenant data is not used to train foundation models, and the assistant honours existing SharePoint, Exchange and Teams permissions. Microsoft documents this in detail in its Copilot data, privacy and security documentation.

The AI governance advantage is real: Microsoft Purview gives you audit logs, sensitivity labels, DLP and eDiscovery over Copilot interactions using tooling your compliance team already runs. The AI governance risk is equally real. Copilot inherits your permission model, so a decade of oversharing in SharePoint becomes discoverable in seconds. Fix access before you deploy, not after.

Where Azure AI Foundry fits

Foundry is the build surface rather than the productivity tool. It gives you a model catalogue that includes OpenAI models, Meta Llama, Mistral, Cohere and increasingly Anthropic models, plus regional deployment, content filtering, evaluation tooling and private networking. For a bank, the appeal is control: you choose the region, the network path, the retention setting and the logging depth. That is the difference between an AI governance story you can present to an examiner and one you cannot.

OpenAI's enterprise offering

ChatGPT Enterprise remains the fastest way to give thousands of staff a genuinely capable assistant. Business data is not used for training by default, SOC 2 attestation is in place, admin controls cover SSO, SCIM and retention, and a compliance API supports export into your archiving stack. Data residency options exist for several regions, which matters if you are subject to EU or UK localisation expectations.

Two AI governance cautions. First, ChatGPT Enterprise sits outside your Microsoft 365 permission model, so grounding on internal content means connectors and a second access-control surface to review. Second, if you consume OpenAI models through the API, negotiate zero data retention explicitly rather than assuming it. For many banks, running the same OpenAI models through Azure OpenAI in Foundry is the easier AI governance path because the contractual and network posture is already covered by an existing Microsoft agreement.

Anthropic Claude: Sonnet and Opus

Claude has become the default choice among the banks we work with for long-document work: credit memos, ISDA and facility agreements, policy gap analysis, regulatory change mapping, and internal audit sampling. Claude Sonnet handles high-volume production workloads at a sensible price point. Claude Opus is the one you reach for when a task needs multi-step reasoning over hundreds of pages and you would otherwise assign a senior analyst for two days.

From an AI governance perspective, three things stand out. The long context window reduces the amount of retrieval engineering you need, which removes a class of silent failure where the right clause never reached the model. Claude is comparatively good at citing the specific passage behind an assertion, which materially helps explainability. And Anthropic does not train on enterprise customer data by default.

Deployment flexibility is the practical differentiator. Claude is available directly from Anthropic and through Amazon Bedrock, Google Vertex AI and Microsoft's model catalogue, so you can usually place it inside a cloud you have already risk-assessed. Confirm current regional availability with your account team before you write it into an architecture document.

Open-source models and on-premises deployment

Llama, Mistral and Qwen models have closed much of the capability gap for narrow, well-defined tasks. Classification, extraction, summarisation of standard formats, PII redaction and internal search re-ranking all work well on a 70B-class open-weight model. If your institution operates in a jurisdiction where data cannot leave the premises, or you have a regulator who is uncomfortable with any external inference, this is your route.

Be honest about the cost. Self-hosting shifts AI governance work from the vendor to you. You own model evaluation, red teaming, safety filtering, prompt injection defence, version pinning, patching and the GPU estate. A single eight-GPU inference node is a six-figure capital item, and you need at least two for resilience. The break-even against hosted APIs usually arrives at very high, very predictable token volumes, not at pilot scale.

Where open source consistently wins: embedding models and small classifiers running inside your data centre, feeding retrieval to whichever frontier model handles the reasoning step. That hybrid pattern satisfies most residency concerns without forcing you to self-host a frontier model.

Cost at scale, compared honestly

• Microsoft 365 Copilot: per-user, per-month licensing on top of existing M365 E3 or E5. Predictable, but you pay for seats whether or not people use it. Measure adoption at 60 and 120 days.
• ChatGPT Enterprise: per-seat with minimum commitments, typically higher than Copilot per user, with no marginal cost per query.
• API consumption (Azure OpenAI, Anthropic, Bedrock): per-token, cheap to pilot, non-linear once an agent starts making dozens of calls per task. Set budget alerts on day one.
• Self-hosted open source: capital plus a specialist team. Two to three engineers is a realistic floor for a production platform.

The cost line most banks miss is human review. If your AI governance policy requires a second pair of eyes on every AI-assisted credit decision, the reviewer time is your dominant cost, and it should shape which use cases you fund.

An AI governance decision matrix banks can use

Score each candidate use case against the six criteria above, then apply these defaults.

• Drafting, meeting recall, email triage, internal search: Microsoft 365 Copilot. Best AI governance fit because Purview already covers it. Precondition: run a SharePoint oversharing remediation first.
• Long contract, policy and regulatory document review: Claude Sonnet for volume, Claude Opus for complex reasoning. Require passage-level citation in every output.
• Customer-facing assistants and agents over core banking data: Azure AI Foundry or an equivalent controlled platform, with retrieval over a governed data layer, full request logging and mandatory human handoff paths.
• Broad staff experimentation and productivity outside M365: ChatGPT Enterprise, gated by an acceptable use policy and an approved use case register.
• PII detection, classification, redaction, high-volume extraction: open-source models on-premises or in your own VPC.
• Credit scoring, pricing, AML decisioning: no generative model as the decision-maker. Use LLMs for narrative and evidence gathering only, and keep the decision model inside your existing validated model inventory.

Make AI governance the tiebreaker, not an afterthought

When two tools score within a few points of each other on capability, pick the one whose logging, retention and residency controls your second line already understands. Multi-vendor is normal and defensible. Multi-vendor with no central AI governance register, no model inventory and no owner for evaluation is what gets flagged in your next examination.

One last practical point. Write your AI governance standard around capabilities and controls, not brand names. Models change every few months. A standard that says "any model used for customer-impacting output must support regional processing, contractual no-training terms, exportable audit logs and documented evaluation results" will still hold in two years. A standard that names one vendor will not.