# Cloud Data & Analytics Platform

USAA · 2020 – 2022 · Role: Staff Engineer, Data & Analytics Platform

> A cloud data platform at USAA processing 25B+ records a day, with an AWS architecture approved for identity, data protection and detective controls.

- **25B+** records per day
- **800K** files per day
- **10x** lower latency in the next generation

## Problem

Analytics, risk and machine learning teams needed governed, reliable data in the cloud at a scale the on-premises platform could not support.

## Approach

- Led architecture for modular orchestration, parallelized ingestion and dynamic workload partitioning into AWS and Snowflake.
- As part of the Cloud Data Center of Excellence, wrote and gave final approval on AWS architecture specifications for identity, data protection, financial management, infrastructure security and detective controls.
- Delivered cloud-to-cloud data movement for claims modernization.

## Governance

- Automated lineage, catalog and sensitive-data protection.
- Rule-based automated reprocessing with Datadog and ELK observability.

## Outcome

- 25B+ records and 800,000 files processed a day.
- Next generation: 20x scalability, 10x lower latency, 0.01% exception rate, 99.9% SLO.

## Lesson

Approve the platform's controls once, carefully, and every team that builds on it inherits them.

Technologies: AWS S3, SageMaker, EMR, Kinesis, Macie, Snowflake, Apache Airflow, Datadog

---
Muthukrishnan (Muthu) Bommiah · https://muthub.org · Views are my own and do not represent my employer.