Machine Learning Interview Questions — What You Memorize vs What Gets Asked

Most ML interview candidates spend their final two weeks reviewing definitions. The interviewers they're facing spent those same two weeks designing questions that definitions can't answer. That mismatch explains why candidates who can recite the bias-variance tradeoff verbatim still get rejected — and why engineers who reason out loud through messy tradeoffs get offers even when they stumble on syntax.

Machine learning interview questions at strong companies are not knowledge tests. They're reasoning tests wearing the costume of knowledge tests. Understanding that distinction is the most important thing you can internalize before your next ML interview.

What Machine Learning Interviews Actually Test

The interviewer asking about gradient descent doesn't want your optimization textbook answer. They want to watch you navigate uncertainty. Do you know which assumptions break when the loss surface isn't convex? Do you know what to do when training loss diverges? Can you reason from first principles when you encounter something you haven't seen before?

Here is the exact gap between what most candidates prepare and what interviewers actually probe:

The best preparation isn't memorizing more answers. It's practicing the habit of asking "and when does this stop working?" after every concept you review.

Core ML Concepts: The Questions That Filter Candidates

These 8 questions appear across virtually every ML interview. The version below isn't the definition you'll find on a cheat sheet — it's the depth the interviewer is probing for.

1. What is the bias-variance tradeoff?

The answer is never "balance them." The answer is: diagnose which one you have, then fix that specific problem. High bias means your model is too simple — it can't fit the training data regardless of how much data you add. High variance means your model is too complex — it fits noise in the training set. In practice: if training error is high, you have bias; if training error is low but validation error is high, you have variance. The fix for bias is more model capacity or better features. The fix for variance is more data, regularization, or a simpler model.

2. Why does gradient descent work?

The update rule is easy to memorize. The intuition matters more. Gradient descent works because for smooth loss surfaces, the negative gradient is the direction of steepest local descent. At each step you move a small distance in that direction (controlled by the learning rate), which reduces the loss locally. The assumption baked in: the loss surface is smooth enough that a small step in the gradient direction actually reduces loss. When this breaks — saddle points, sharp minima, exploding gradients — standard gradient descent fails. That's why adaptive optimizers like Adam matter: they approximate the curvature to take better steps.

3. L1 vs L2 regularization — why does L1 produce sparse models?

The formula difference (absolute value vs squared) isn't the real answer. The geometry is. The L1 constraint set is a diamond (in 2D) with corners at the axis-aligned points. When gradient descent minimizes the loss subject to the L1 constraint, it tends to hit those corners — which correspond to coefficients being exactly zero. L2's constraint set is a sphere with no corners, so the minimum tends to land somewhere on the curved surface, shrinking coefficients toward zero but rarely reaching it. This is why L1 (Lasso) is used for feature selection and L2 (Ridge) is used when you want to keep all features but reduce their magnitude.

4. How does a decision tree decide splits?

A decision tree finds splits that maximize information purity in the child nodes. The two dominant metrics are information gain (based on entropy reduction) and Gini impurity. CART — the algorithm behind scikit-learn's DecisionTreeClassifier — uses Gini impurity by default, not entropy. The practical reason: Gini impurity is computationally cheaper (no logarithm) and produces nearly identical splits in most real datasets. The Gini impurity of a node is 1 minus the sum of squared class probabilities. A pure node (all one class) has Gini = 0. A completely mixed node has Gini near 0.5.

5. Why do we need train/val/test split?

The real answer isn't "to evaluate generalization." It's data leakage prevention. If you tune your model on the test set — even indirectly, by trying 50 architectures and picking the one with best test performance — your test set becomes a part of training. Its error estimate is now optimistic. The validation set exists to absorb all the decisions you make (hyperparameter search, architecture selection, threshold tuning) so the test set stays untouched. A dataset touched for any decision other than final reporting is no longer an unbiased estimator of production performance.

6. What is overfitting, and how do you know you have it?

This is a diagnostic question, not a definition question. You know you have overfitting when training loss is substantially lower than validation loss — and the gap grows as you train longer. The practical follow-up matters more: what do you do about it? Regularization (L1/L2), dropout, early stopping, data augmentation, reducing model capacity, adding more training data. The right lever depends on how severe the gap is and what's computationally feasible.

7. Explain cross-entropy loss.

Cross-entropy is the negative log likelihood under a Bernoulli distribution (for binary classification) or a categorical distribution (for multiclass). When the model outputs probability p for the correct class, the loss is -log(p). Minimizing cross-entropy is exactly equivalent to maximizing the likelihood of your training labels under the model's probability distribution. Geometrically: cross-entropy penalizes confident wrong predictions enormously (log(0.01) ≈ -4.6) and penalizes uncertain correct predictions mildly (log(0.6) ≈ -0.5). This asymmetry is what makes it work so well for classification — it rewards calibrated confidence.

8. What is the vanishing gradient problem, and how did deep learning solve it?

In early deep networks with sigmoid activations, gradients shrank exponentially as they backpropagated through layers. Sigmoid's maximum derivative is 0.25 — stack 10 layers and the gradient arriving at layer 1 is at most 0.25^10 ≈ 0.000001. Training stalled.

Three innovations fixed this: First, ReLU activations have a derivative of 1 for positive activations — gradients don't shrink passing through a ReLU. Second, batch normalization keeps activations in a sensible range across layers, preventing the distribution shift that causes gradients to explode or vanish. Third, residual connections (ResNets) create direct gradient highways — even if gradients shrink passing through residual blocks, they flow unattenuated through the skip connections. All three are still standard in modern architectures because all three address a real failure mode.

The 8 Machine Learning Interview Questions on Model Evaluation

Model evaluation is the area candidates most consistently underestimate. Memorizing metric definitions isn't enough — you need to know which metric fits which problem and why the others are misleading.

Precision vs recall — and when each matters

Precision is the fraction of your positive predictions that are actually positive. Recall is the fraction of actual positives you correctly predicted. In fraud detection, recall is what matters: a missed fraud (false negative) is expensive. Missing 10% of fraud costs real money. In search ranking, precision matters: showing irrelevant results (false positives) degrades user experience immediately. The F1 score is the harmonic mean of both — useful when neither extreme is acceptable, but it obscures which direction you're underperforming in.

AUC-ROC vs AUC-PR

AUC-ROC measures the model's discrimination ability across all classification thresholds. It answers: across the whole operating range, how well does my model separate positives from negatives? AUC-PR measures the precision-recall trade-off across thresholds. For imbalanced datasets — fraud (0.1% positive rate), rare disease detection, spam in a mostly-clean inbox — AUC-PR is more informative.

Here's why: AUC-ROC is dominated by the true negative rate because negatives are so common. A model that correctly labels 99% of negatives looks great on ROC even if it misses most positives. AUC-PR ignores true negatives entirely — it only measures what you got right among positive predictions and positive ground truth. If your positive class is less than 5% of the dataset, report AUC-PR, not AUC-ROC.

What's wrong with accuracy on a 99% negative class problem?

A model that predicts "negative" for every sample achieves 99% accuracy. It has zero utility. Accuracy hides performance on the minority class entirely. This is why precision, recall, F1, and AUC-PR exist — they force the model to perform on the cases that matter.

Evaluating a ranking model

Ranking models require ranking metrics, not classification metrics. NDCG (Normalized Discounted Cumulative Gain) measures whether the most relevant items appear at the top of the list, with logarithmic discounting for lower positions. MAP (Mean Average Precision) averages precision at each rank where a relevant result appears. Both metrics penalize models that bury relevant results. In practice, NDCG is more common for search and recommendation; MAP is more common for information retrieval benchmarks.

A/B testing a model in production

The naive approach — "run the experiment for a week and check" — fails in two ways. First, you need to calculate the minimum detectable effect before you start: how large a change in your metric do you actually care about detecting? Second, you need to calculate the required sample size from that MDE, your baseline rate, desired statistical power (80% is common), and significance level (0.05 is common). Runtime follows from sample size and traffic volume. Running the experiment until you see p < 0.05 is p-hacking — you inflate the false positive rate by checking repeatedly. Commit to a fixed runtime before you launch.

Deep Learning Questions That Appear at AI-First Companies

If you're targeting MLE roles at companies building AI products — Swiggy, Razorpay, Flipkart's recommendations team, Coupang's personalization team — these questions appear in virtually every loop.

Self-attention: the Q/K/V computation

Self-attention computes: Attention(Q, K, V) = softmax(QK^T / √d_k) V

Q (query), K (key), and V (value) are linear projections of the same input sequence. For each position, the query is compared against all keys to produce attention scores — a measure of "how much should this position attend to every other position?" The softmax normalizes those scores into a probability distribution. The output is the weighted sum of values.

The √d_k scaling is not cosmetic. For large d_k, the dot products QK^T grow in magnitude, pushing softmax into saturation regions where gradients are near zero. Dividing by √d_k keeps the softmax in a numerically stable operating range.

Why did transformers replace RNNs?

The performance difference is real, but the fundamental reason is parallelization. RNNs process sequences step-by-step: token n cannot be computed until token n-1 is done. That sequential dependency makes training slow and limits how long sequences you can practically handle. Transformers compute attention over the entire sequence simultaneously — every position attends to every other position in parallel. On modern hardware with thousands of CUDA cores, this means transformers scale with hardware in a way RNNs fundamentally cannot.

Batch normalization — when NOT to use it

Batch normalization normalizes activations across the batch dimension at each layer, then applies learnable scale and shift parameters. It stabilizes training, allows higher learning rates, and reduces sensitivity to initialization.

Two situations where batch norm breaks: recurrent networks and small batch sizes. In RNNs, batch norm applied across the time dimension causes the normalization statistics to differ during training (where many time steps are available) vs inference (where you may have a single step). Layer normalization — which normalizes across the feature dimension instead — solves this and is standard in transformers. For small batch sizes (< 8), the batch statistics are too noisy for normalization to be stable, which is why group normalization or instance normalization is preferred in image segmentation with large models.

How does dropout prevent overfitting?

Dropout randomly zeros out neurons during training with probability p. The ensemble interpretation is more useful than the "noise injection" framing: during training, you're effectively training 2^n different network architectures (one per dropout mask). At inference, you use the full network with weights scaled by (1 - p), which approximates the geometric mean of all those subnetworks. This prevents any individual neuron from becoming too specialized (co-adaptation) because it can't rely on specific other neurons always being present.

Transfer learning — why it works for NLP

Large language models trained on internet-scale text learn layered representations. Lower transformer layers capture syntax, grammar, and local token relationships — patterns that generalize across virtually all language tasks. Upper layers capture task-specific semantics. When you fine-tune BERT or a similar model on a downstream task with limited labeled data, you're inheriting the linguistic universals learned from billions of tokens. The alternative — training from scratch — requires orders of magnitude more data to learn those same representations. For most NLP tasks, the first 80% of what makes a strong model is already captured in the pretrained weights.

The Questions That Actually Separate Strong Candidates

These are the questions that junior candidates answer with textbook responses and senior candidates answer with scar tissue.

"How would you debug a model that performs well offline but poorly in production?"

The naive answer is "retrain with more data." The real answer starts with diagnosis. The three most common causes are train-serving skew (the features available at training time don't match the features available at serving time — a very common bug with time-based features), distribution shift (the production data distribution has drifted from the training distribution — common after seasonal events, product changes, or user behavior shifts), and feature staleness (features in the training set were computed at a different point in time than at serving — "user's last 30 days of activity" means different things at training time vs real-time serving).

The debugging sequence: first, check if the features your model receives in production match your training distribution. Log a sample of production feature vectors and compare distributions. Second, check if the label distribution has shifted. Third, compute online metrics on a held-out slice of recent production data. Only after you've ruled out data problems should you consider model changes.

"Your model's accuracy dropped after deployment. What do you do first?"

Do not retrain immediately. Retraining on drifted data produces a model adapted to the drift rather than a model that's robust to it. First step: investigate whether the input data distribution has changed (data drift) or whether the relationship between features and labels has changed (concept drift). Look at feature statistics over time. Check if any upstream data pipelines changed. Verify your feature engineering code is deterministic. Only after you understand why performance dropped can you decide whether the fix is retraining, fixing a data pipeline, or adding a fallback rule.

"How do you handle class imbalance?"

Don't just say SMOTE. The right approach depends on how severe the imbalance is and what you're optimizing.

For mild imbalance (1:10 to 1:30), class-weighted loss functions — available in virtually every scikit-learn and PyTorch classifier — are the lowest-friction approach. They adjust the loss contribution of each class so the minority class gets more gradient signal.

For severe imbalance (1:100 to 1:10000), oversampling the minority class with SMOTE or undersampling the majority class helps. But SMOTE generates synthetic samples in feature space, which can be misleading for non-linear boundaries — it should be applied only to the training set, never the validation or test set.

Cost-sensitive learning is appropriate when you have a precise cost model: in fraud detection, you know that a false negative (missed fraud) costs 100x more than a false positive (blocked legitimate transaction). That cost ratio should be reflected in your loss function or decision threshold.

"What features would you engineer for a fraud detection model?"

This question tests domain intuition as much as feature engineering technique. Strong answers typically include: velocity features (how many transactions in the last 1 hour / 24 hours / 7 days from the same card, device, or IP), deviation features (how much does this transaction deviate from the user's historical average transaction amount and merchant category), graph-based features (has this device been associated with a previously flagged account?), temporal features (time of day, day of week, time since last transaction), and behavioral biometrics (typing speed, swipe pattern — if available). The signal in fraud detection is almost entirely in the deviations from normal behavior, so features that capture recency and anomaly are more valuable than absolute values.

How to Structure Your Answers: The Framework Interviewers Reward

Technical correctness is necessary but not sufficient. Interviewers reward a specific structure that most candidates skip.

The four-part answer structure that consistently lands well:

1. Restate what you're optimizing. One sentence naming the objective. "You want a model that's precise enough to avoid annoying good customers but sensitive enough to catch fraud."

2. Give the concept. Explain the mechanism clearly. Not the definition — the mechanism. Why does it work?

3. Name the failure mode. Where does this break? What assumption has to hold? This is where most candidates stop too early.

4. Give a concrete example. Not hypothetical — either something from your experience or a specific scenario. "At my previous company, we had exactly this problem when..."

Most candidates give parts 1 and 2. Senior candidates give all four. The failure mode (part 3) is the most important differentiator because it demonstrates that you've actually used the technique, not just read about it.

One more pattern worth knowing: interviewers at strong companies will follow up until you hit your limit. They're not trying to trick you — they're calibrating your depth. "I don't know, but here's how I'd think through it" is a better answer than a confident wrong one. The reasoning process is what they're buying.

For a deeper look at how this preparation strategy applies to the broader AI engineering landscape in 2026, see the AI Engineering Interview Guide — it covers the same reasoning-over-recall philosophy across system design, inference serving, and agentic system questions.

What Separates the Candidates Who Pass

The engineers who consistently pass ML interviews share one characteristic: they've moved from knowledge to judgment. They don't just know that L1 produces sparse models — they know when that property is useful and when it makes things worse (when you have correlated features and need to keep all of them). They don't just know what batch normalization does — they know the two scenarios where it fails and what to use instead.

That transition from knowledge to judgment doesn't happen from reading more documentation. It happens from deliberate practice on adversarial questions — questions designed to find the edge of what you understand.