Design a Flexible Multi-Class Classification Model

Problem Design a classification model that serves 10 classes today but must tolerate classes being added or removed later — without changing the model architecture or retraining from scratch each time.

Functional requirements

• Predict over a variable, open-ended set of classes.
• Add a new class (or retire one) without re-architecting the network.
• Keep accuracy on existing classes stable when the class set changes.
• Support few-shot addition: a new class may arrive with only a handful of labelled examples.

Non-functional requirements

• ~1-5k predictions/sec at peak; single-digit-millisecond p99 model latency.
• Class set churns weekly; adding a class should be a minutes-to-hours operation, not a multi-hour full retrain.
• Training corpus in the low millions of labelled examples; prototype/embedding index must fit comfortably in memory.

Key components

• Shared feature extractor/encoder trained once to produce a general-purpose embedding.
• Metric-learning objective (triplet/contrastive/ArcFace-style) so same-class examples cluster and different-class examples separate.
• Class prototypes or class embeddings stored outside the network — adding a class means adding a prototype vector, not an output unit.
• Inference by nearest-prototype/similarity lookup (ANN index if the class count grows large), with a confidence threshold for "unknown".
• Periodic re-training of the encoder on accumulated data, decoupled from day-to-day class additions.

Deep dives / trade-offs

• Embedding + prototypes vs. a fixed-size softmax head: softmax is simpler and usually a bit more accurate on a frozen class set, but every class change forces re-architecting the output layer and a full retrain.
• Middle ground: frozen feature extractor with a small swappable classifier head — cheaper than full retraining, but still needs a head retrain per class change.
• Embedding drift: when the encoder is retrained, every stored prototype must be recomputed; how to version and roll that out safely.
• Class imbalance and the open-set "none-of-the-above" case: thresholding on similarity rather than forcing an argmax.