Design an Elasticsearch Cluster

Requirements: Given the number of active shards, number of instances (nodes), and number of replicas needed, design an Elasticsearch cluster topology.

Design considerations:

• Primary shards: decided at index creation time and fixed afterward; determines the maximum write parallelism and total data distribution.
• Replica shards: copies of primary shards for fault tolerance and read scaling; each replica must live on a different node than its primary.
• Node count vs shard count: total shard copies (primary + replica) should be distributed evenly across available nodes; over-sharding (too many small shards per node) hurts performance due to per-shard overhead (file handles, memory, cluster state size).
• Data storage internals: Elasticsearch stores data in inverted indices per shard (each shard is a self-contained Lucene index); documents are tokenized/analyzed at index time and stored in segments that are periodically merged.
• Failure handling: if a node holding a primary shard goes down, a replica on another node is promoted to primary automatically, provided replicas exist.
• Follow-up: how does ES internally store and look up data (inverted index, segments, refresh/flush cycle) as compared to a traditional row-store DB.