Initiative · USAA · 2020 – 2022
Cloud Data & Analytics Platform
A cloud data platform at USAA processing 25B+ records a day, with an AWS architecture approved for identity, data protection and detective controls.
25B+records per day
800Kfiles per day
10xlower latency in the next generation
Problem
Analytics, risk and machine learning teams needed governed, reliable data in the cloud at a scale the on-premises platform could not support.
Approach
- Led architecture for modular orchestration, parallelized ingestion and dynamic workload partitioning into AWS and Snowflake.
- As part of the Cloud Data Center of Excellence, wrote and gave final approval on AWS architecture specifications for identity, data protection, financial management, infrastructure security and detective controls.
- Delivered cloud-to-cloud data movement for claims modernization.
Governance
- Automated lineage, catalog and sensitive-data protection.
- Rule-based automated reprocessing with Datadog and ELK observability.
Outcome
- 25B+ records and 800,000 files processed a day.
- Next generation: 20x scalability, 10x lower latency, 0.01% exception rate, 99.9% SLO.
Lesson
Approve the platform's controls once, carefully, and every team that builds on it inherits them.