Design a real-time portfolio risk and analytics platform

Design a system that ingests live market and portfolio data and serves real-time risk and performance analytics (exposure, P&L decomposition, crowding) to institutional investors. Focus on the high-level architecture.

Functional requirements

• Ingest high-frequency market data and clients' portfolio positions.
• Continuously compute and update risk/performance metrics per portfolio.
• Serve dashboards and ad-hoc analytical queries with low latency.

Non-functional requirements

• Large fan-out updates (thousands of instruments, many portfolios) with sub-second freshness.
• Write-heavy analytical workload; high availability; horizontal scalability.

Key components

• Ingestion layer on a pub/sub backbone (e.g. Kafka) to decouple producers from consumers and fan out updates.
• Stream-processing / aggregation workers that recompute metrics incrementally.
• A write-heavy OLAP / columnar store (e.g. ClickHouse, Druid) optimized for fast analytical reads over append-heavy data.
• A serving/API layer with caching for dashboards.

Deep dives / trade-offs

• Pub/sub partitioning and ordering guarantees; backpressure handling.
• Choosing the OLAP store: ingest throughput vs. query latency vs. storage cost.
• Incremental vs. batch recomputation; consistency vs. freshness.