Design Cross-Cloud Data Sync for Backup (Azure to AWS)

Requirements: Reads and writes happen only on Azure; AWS must be kept as an asynchronous backup copy of the same data. Design the syncing mechanism.

Design:

• Baseline approach: on every write to Azure, enqueue a write event onto a durable queue; a pool of worker threads/consumers drains the queue and replicates each write to AWS asynchronously, decoupling AWS availability/latency from the primary write path.
• Overload handling: if the write rate exceeds what the queue/workers can drain in real time (backlog growing unbounded), fall back to batch synchronization — a scheduled job (e.g., hourly cron-triggered Lambda) reconciles AWS with Azure in bulk rather than relying purely on the real-time queue.
• Additional considerations: idempotent writes/dedup keys so retries don't double-apply, ordering guarantees per record (e.g., partition by record id in the queue), monitoring replication lag, and a reconciliation/checksum job to catch drift between Azure and AWS over time.