CNN and NLP: Algorithms and Underlying Math

Problem Explain Convolutional Neural Networks and NLP algorithms along with the mathematics underpinning them.

Be ready to discuss

• CNN mechanics: convolution kernels sliding over the input to extract spatial features, non-linear activations, and pooling for dimensionality reduction. State the convolution operation itself, and trace how gradients flow back through convolution and pooling layers to train the filters.
• Output-shape arithmetic and cost: deriving dimensions from kernel size, stride, padding, and dilation, plus the parameter count of a conv layer versus a dense one.
• NLP algorithm coverage: word embeddings (the Word2Vec/GloVe objectives), sequence models (RNN/LSTM recurrences and gating equations), and attention — what each computes, written out.
• The mathematical basis shared across all of them: loss functions (cross-entropy and its derivation from maximum likelihood), softmax, and gradient descent with its variants.
• Why softmax paired with cross-entropy is the standard, and what its gradient simplifies to.
• What's evaluated: depth of derivation rather than high-level intuition — expect to be pushed to the underlying math for whichever algorithm you bring up.