Design a Budgeting System

Problem Design a personal budgeting service: users link cards/accounts, the system ingests every transaction, categorizes spend, and emails a PDF report at month end.

Functional requirements

• Link financial accounts (via an aggregator like Plaid); continuously ingest transactions.
• Auto-categorize transactions (groceries, travel, …) with user overrides; budgets per category with alerts.
• Monthly job renders a per-user PDF summary and emails it.

Non-functional requirements

• Scale to discuss: 10M users, ~30 transactions/user/month → ~10K txn/sec peak ingest; month-end fan-out of 10M reports within hours.
• Financial data: encryption at rest/in transit, idempotent ingestion (aggregators redeliver), auditability.

Key components

• Ingestion service (webhook/poll from aggregator → queue → dedupe/normalize → transaction store), categorization service (rules + ML model, override store), aggregation pipeline (per-user per-category monthly rollups — incremental on ingest or batch), report service (template → PDF render workers), email delivery via provider, scheduler (partitioned month-end job queue).

Deep dives / trade-offs

• Idempotency/dedupe of transactions (aggregator ids + hash fallback) and late-arriving/amended transactions after a report is sent.
• Pre-aggregated rollups vs compute-at-report-time: storage vs month-end compute spike.
• Month-end fan-out: shard users across workers, rate-limit email, retries with checkpointing so a crash doesn't resend half the users.