LLM Architecture and Training Fundamentals

Question: Explain the basic architecture of large language models (LLMs) and how they are trained. Key points: Modern LLMs are built on the Transformer architecture (stacked self-attention + feed-forward layers). Training typically happens in stages: large-scale unsupervised pre-training on next-token prediction over massive text corpora, followed by fine-tuning/alignment (e.g. instruction tuning, RLHF) for downstream tasks. Candidates are expected to speak to tokenization, embeddings, attention, and the pre-training/fine-tuning distinction at a basic level.