Enterprise Generative AI Architecture

Problem

Teams across the bank wanted to use large language models, but there was no approved, governed way to take a Generative AI use case from experiment to production.

Approach

• Designed a modular, phased reference architecture covering data ingestion, processing, embedding, retrieval, generation and validation.
• Set an enterprise model strategy across public cloud and on-premises infrastructure, spanning frontier and open models.
• Published reusable archetypes for model deployment and inferencing, and for model development, training and validation.

Governance

• Model guardrails and AI gateways against prompt injection and data leakage.
• Security testing of every Generative AI use case before it goes live.
• A governed experimentation environment with automated lineage, detective controls and secure data access.

Outcome

• Approved through enterprise architecture governance and adopted as the reference model across business lines.
• The first use case, a retrieval-augmented assistant for internal policy guidance, went from pilot to production on this architecture.