ML System Design Interview Questions — The 5-Component Framework
ML system design is a distinct interview format — not software design with a model bolted on. This covers the 5-component answer structure, the 5 problems that appear most (recommendation, fraud detection, demand forecasting, search ranking, content moderation), and company-specific patterns at Amazon, Flipkart, Coupang, and Uber.
After reading: You'll have the 5-component ML system design framework, model answers for the 5 most common problems, and company-specific patterns for your target company.
Seventy percent of ML engineers who fail a system design round at Amazon or Flipkart don't fail because they can't design a model — they fail because they treat ML system design like software system design with a neural network bolted on at the end. The interviewer asked "design a recommendation system," and the candidate drew boxes for a database, an API layer, and then said "and here we'd add a model." That's not an ML system design answer. That's a system design answer with the word "model" in it. The distinction is what this post is about.
ML system design interviews test a specific 5-component reasoning structure. Candidates who know it pass. Candidates who don't, regardless of how much ML they know, don't. Here's every ML system design interview question pattern you'll encounter, the canonical answers, and the filter questions interviewers use to separate the top 10% from everyone else.
ML System Design vs Software System Design: The Structural Difference
Software system design interviews test your ability to reason about capacity, data models, consistency, and availability. ML system design interviews test all of that — plus a five-component structure that has no equivalent in pure software design. Every strong ML system design answer hits all five components. Missing even one is a red flag.
The five components are: (1) problem framing and metrics, (2) data layer, (3) feature engineering, (4) model architecture and training, (5) serving and monitoring. This is not a loose framework — it's the structural backbone interviewers use when scoring your answer. At Amazon, the MLE loop rubric maps almost directly to these components.
Key point
The most consistent failure pattern in ML system design interviews is skipping the metrics component entirely. Candidates jump straight to architecture. Interviewers at Amazon and Flipkart specifically probe: "How do you know if this system is working?" If you can't name both an offline metric and an online metric and explain the gap between them, you've already lost the round.
The offline-online metric distinction is a known filter question. Offline metrics (AUC-ROC for classification, NDCG@10 for ranking, precision@K for recommendations) tell you how the model performs on historical data. Online metrics (click-through rate, add-to-cart rate, revenue per user, 7-day retention) tell you what actually happens when real users interact with the system. A model with 0.94 AUC can produce flat or negative online lift if you trained on biased data or if the feature distribution at inference time differs from training. Interviewers know this. They're checking if you do too.
The 5 ML System Design Problems That Appear Most Often
These five problems account for roughly 80% of ML system design interview questions at product companies. Each has a canonical architecture — and each has a hard follow-up question that separates the candidates who've thought through production from those who've only read papers.
1. Recommendation System
Business context: an e-commerce or content platform wants to show each user the items most likely to drive engagement or purchase. At Amazon, this is the "Customers Also Bought" widget. At Coupang, it's the product feed on the homepage.
Canonical architecture: two-stage pipeline. Stage 1 — candidate generation using a two-tower model (one tower encodes user history and context, the other encodes item features; trained with in-batch negatives; inference uses Faiss or ScaNN for approximate nearest neighbor over 300M+ items). Stage 2 — ranking using a gradient boosted tree (LightGBM or XGBoost) that takes the top 500 candidates and scores them on cross features (user × item interactions that the two-tower can't capture in separate towers).
The hard part interviewers probe: "How do you handle new items with no interaction history?" Item cold-start. You need to fall back to content-based features (item category, price point, textual description embedding) until the item accumulates enough interactions for collaborative filtering signals. Bonus: mention that you'd track cold-start items separately in your online metrics to ensure they don't get permanently buried.
2. Fraud Detection
Business context: a payments company needs to approve or decline transactions in under 100ms at 10,000 transactions per second. A 0.1% false positive rate means legitimate customers get blocked — which has direct revenue and NPS impact.
Canonical architecture: synchronous track and asynchronous track. The synchronous track runs inside the transaction approval window: extract velocity features from a Redis cache (transactions in the last 1 minute, 1 hour, 24 hours on this card/device/IP), run a GBT model (not a neural network — GBT serves in under 5ms on CPU, is interpretable for compliance, and doesn't require GPU infrastructure), and return approve/decline/review. The asynchronous track runs post-approval: graph analysis (is this account connected to known fraud networks in the past 90 days?), sequence models on full transaction history, and feedback from manual review back into the training set.
The hard part interviewers probe: "Why GBT instead of a neural net for the sync track?" GBT serves in 3–5ms on commodity CPU. A two-layer neural network on a GPU with batching can get to 10–20ms, but you've added GPU infrastructure dependency and non-interpretability. When a transaction is declined and the customer calls to dispute it, you need feature attributions (SHAP values on a GBT) to explain the decision. Regulatory requirements in India and the EU often mandate explainability for adverse decisions.
3. Demand Forecasting
Business context: a logistics company needs to predict demand for each SKU in each warehouse for the next 7–30 days to optimize inventory placement. At Coupang, this feeds directly into their Rocket Delivery promise — if the demand forecast is wrong, items aren't in the right warehouse and delivery SLAs break.
Canonical architecture: hierarchical time-series forecasting. At the bottom level: per-SKU-per-warehouse models (LightGBM with lag features, rolling statistics, calendar features — day of week, holiday flags, promotion schedules). At higher levels: aggregate across warehouses or product categories using reconciliation (bottom-up or optimal reconciliation). For SKUs with long histories: temporal fusion transformer (TFT) or N-HiTS outperforms classical methods. For new SKUs (cold-start): fall back to category-level forecasts plus sales velocity in the first 2 weeks.
The hard part interviewers probe: "How do you evaluate this in production?" Offline: MAPE (mean absolute percentage error) and wMAPE stratified by SKU sales volume. Online: stockout rate, overstock rate, and the actual P&L impact of forecast error (understocking a high-margin item is worse than understocking a low-margin one — interviewers notice if you treat all SKUs equally in your loss function).
4. Search Ranking
Business context: a user types a query and expects the most relevant results in the top 5 positions. At Flipkart, a 1% improvement in search ranking quality directly translates to hundreds of millions in GMV annually because search drives 70%+ of product discovery.
Canonical architecture: Learning-to-Rank (LTR) with a multi-stage pipeline. Stage 1 — retrieval: hybrid search combining BM25 (term matching via Elasticsearch) with dense retrieval (query embedding matched against pre-computed product embeddings using Faiss). Stage 2 — reranking: a pointwise or pairwise LTR model (LambdaMART or a small transformer reranker) that takes the top 200 candidates and produces a final ranked list using cross features (query-product interaction signals, user session context, historical CTR by position). Offline metric: NDCG@10. Online metric: conversion rate on page 1, revenue per search session.
The hard part interviewers probe: "How do you handle query-document mismatch when users don't use the product's exact terminology?" Semantic expansion — map the user query through the dense retrieval model so "running shoes for flat feet" retrieves items tagged "overpronation support" even without keyword overlap. Then: how do you update item embeddings when product descriptions change? You need a re-embedding pipeline triggered by product catalog updates, with a staleness threshold on embeddings (don't serve embeddings older than 7 days for high-velocity catalog items).
5. Content Moderation
Business context: a social platform needs to prevent harmful content (hate speech, nudity, spam) from being visible to users, at a scale of millions of posts per day, without creating a content review backlog that slows legitimate posts.
Canonical architecture: multi-stage cascade. Stage 1 — fast filter (< 10ms): hash-matching against known bad content (PhotoDNA for images, hash database for known spam text). Stage 2 — ML classifier (< 50ms): a lightweight text or image classifier (DistilBERT fine-tuned for text, MobileNet for images) that produces a confidence score. High-confidence harmful content is auto-removed. High-confidence safe content is auto-approved. The middle band (e.g., confidence 0.4–0.8) routes to human review. Stage 3 — periodic deep analysis: full transformer models (fine-tuned LLaMA or GPT-4o) on a sampled queue for model improvement and policy calibration.
The hard part interviewers probe: "How do you set the auto-remove threshold?" This is a business decision, not just an ML decision. False positives (removing legitimate content) have user trust and regulatory implications. False negatives (missing harmful content) have reputational and legal implications. The threshold should be set based on the cost ratio of each error type, reviewed with policy and legal, and tracked as a separate online metric (escalation rate, appeal success rate) distinct from raw accuracy.
How to Answer an ML System Design Question: The 45-Minute Framework
Most candidates improvise structure. The ones who pass have a time-boxed framework they execute consistently. Here's the one that works across every ML system design problem.
Minutes 0–5: Problem framing and clarification
Don't start designing until you've aligned on scope. Ask: What are we optimizing for? What's the latency budget? What scale are we operating at? What counts as success in production? The output of this phase is a crisp statement: "We're designing a product recommendation system that maximizes add-to-cart rate, serves 50ms p99 recommendations, handles 100M users and 20M products, and considers a deployment successful when we see a 3%+ lift in add-to-cart rate in A/B testing."
Minutes 5–10: Metrics (both offline and online)
Name your offline metric and explain its limitation. Name your online metric and explain how you'd measure it. Then explicitly state the gap: "NDCG@10 will tell me if the ranking model is learning the right relevance signal, but it won't tell me if the relevance signal actually correlates with purchase intent. That's why I also track add-to-cart rate per session." Interviewers will nod visibly at this.
Tip
Before you touch architecture, write your metrics on the whiteboard and leave them visible for the entire interview. It signals that all your design decisions will be anchored to these metrics — which is exactly the reasoning style senior ICs use in production.
Minutes 10–20: Data layer and feature engineering
Cover data sources, labeling strategy (explicit vs implicit feedback), class imbalance handling. Then cover the feature store pattern: online feature store (Redis or DynamoDB, sub-10ms lookups, stores precomputed user features updated every few minutes) vs offline feature store (S3 or BigQuery, used during training, full history). Interviewers at Uber and Flipkart specifically probe the feature store — they want to know if you understand the training-serving skew problem (the feature values at training time must match the feature values at serving time, or your model learns on data it won't see in production).
Minutes 20–35: Model architecture and training
Propose a specific architecture and justify it with constraints. Don't say "I'd use a neural network." Say "I'd use a two-tower model for candidate generation because it allows pre-computation of item embeddings offline, so we only need to compute the user tower at inference time and then do an ANN lookup — that gets us under 30ms end-to-end." Cover training pipeline: data pipeline (Spark or Beam for feature computation), experiment tracking (MLflow or Weights & Biases), and the retraining schedule (daily batch retraining vs online learning, based on how fast the data distribution shifts).
Minutes 35–45: Serving, A/B testing, and monitoring
Serving architecture: model registry, shadow deployment (run new model in parallel without affecting production traffic to catch latency regressions), canary rollout. A/B testing: traffic split by user ID hash, minimum detectable effect, runtime, primary and guardrail metrics. Monitoring: feature drift (PSI on input features), prediction drift (distribution shift in model output scores), and business metric monitoring (if CTR drops 5% with no product changes, the model has likely degraded).
Watch out
The most common omission at this stage is not discussing monitoring at all, or mentioning it in one sentence. Interviewers know that ML systems degrade silently — unlike software bugs that throw exceptions. If you don't describe a concrete drift detection and alerting strategy, you've signaled that you've never shipped a model to production.
The 3 Things Interviewers Are Actually Testing
ML system design interviews aren't testing whether you know the names of algorithms. Every MLE candidate knows what a transformer is. What they're testing is harder to fake.
1. Can you reason about trade-offs, not just describe architectures?
There are three valid candidate generation architectures for a recommendation system: matrix factorization, two-tower neural model, and item-based CF. The question isn't which one you pick. The question is whether you can say "matrix factorization gives us strong offline NDCG but struggles with cold-start and doesn't incorporate context signals; two-tower handles cold-start via content features in the item tower and lets us inject session context into the user tower, at the cost of more complex training infrastructure." That's a trade-off argument. "I'd use a two-tower" with no justification is architecture trivia.
2. Do you connect model decisions to business impact?
At Flipkart's scale, a 1% improvement in search ranking precision generates hundreds of millions in incremental GMV annually. When you choose a model architecture, training frequency, or serving latency budget, you should be able to say what the business consequence of getting it wrong is. "I'd retrain daily instead of weekly because product prices and inventory change daily — a week-stale model at Coupang would recommend out-of-stock items, which is a direct Rocket Delivery SLA failure." Business grounding is what separates an MLE answer from a data scientist answer.
Key point
The thing interviewers are most testing, but rarely name, is production failure mode literacy. Can you describe the three most likely ways this system fails in production — and what your design does to prevent each one? Most candidates describe the happy path. Senior engineers design for failure.
3. Do you know the production failure modes?
For every ML system, there are canonical failure modes. Recommendation systems: popularity bias (the model recommends only already-popular items, creating a rich-get-richer feedback loop — fix with diversity injection or exploration/exploitation mechanisms). Fraud detection: the adversarial adaptation problem (fraudsters observe your model's behavior and adjust — you need periodic behavioral analysis of flagged accounts to detect evolving patterns). Demand forecasting: promotions that weren't in the training data (a flash sale that's 5x normal demand will be catastrophically underforecast if the model has no promotion feature — fix with an explicit promotion calendar feature). Naming these failure modes signals production experience.
Company-Specific Patterns: Amazon, Flipkart, Coupang, Uber
Each company has a predictable question bias based on their core technical problems. If you're interviewing at one of these companies specifically, knowing their ML system design bias is a concrete preparation advantage. For a deeper look at how Amazon structures their ML systems end-to-end, read our analysis of how Amazon uses AI across its product stack.
Amazon
Amazon asks recommendation and supply chain forecasting overwhelmingly often. For SDE-2+ and MLE roles, expect "design the Customers Also Bought recommendation system" or "design the inventory placement system for Prime delivery SLA." Amazon's ML interviews lean hard on the data flywheel question: "How does the model improve over time as more users interact with the system?" They also probe on explainability — Amazon's ML systems operate under internal compliance requirements, so they want to know if you'd use SHAP or integrated gradients for feature attribution. A strong Amazon answer connects every technical decision to a customer metric (conversion rate, delivery success rate, return rate).
Flipkart
Flipkart's ML design interviews are dominated by search ranking and fraud detection. The search question is almost always scoped to catalog scale: 150M+ products, multilingual queries (Hindi, Tamil, Telugu, English), and the challenge that many sellers don't write good product descriptions (weak text signals). Expect a deep probe on: "How do you handle the language mismatch between a Tamil-language query and an English product description?" Answer: multilingual embeddings (Cohere's multilingual model or LaBSE) at both query and product encoding. Fraud at Flipkart maps directly to their cash-on-delivery model — COD fraud is structurally different from card fraud and requires different features (delivery address novelty, order-to-COD ratio for that PIN code, seller reputation score).
Coupang
Coupang's canonical ML design question is demand forecasting under a hard constraint: the Rocket Delivery promise. Coupang guarantees same-day or next-day delivery for millions of items. That guarantee is only possible if inventory is pre-positioned in the right fulfillment centers before demand occurs. A demand forecast that's off by 20% for a high-velocity item breaks the SLA. Expect deep probing on: how do you handle forecast uncertainty (not just point estimates — prediction intervals and safety stock calculations), and how do you handle new product categories where you have no history (transfer learning from similar categories, or conservative fallback to category-level forecasts). For more on the cross-company pattern differences, read our breakdown of AI systems at Amazon, Coupang, and Walmart.
Uber
Uber's ML system design interviews split between ETA prediction and surge pricing (dynamic pricing). ETA prediction is one of the harder ML system design problems because it's a live system that must update its predictions as the trip progresses — the 8-minute estimate at pickup should update to 6 minutes when the driver takes a faster route, and to 11 minutes when they hit unexpected traffic. This requires an online learning component (updating features from real-time GPS telemetry) alongside the batch-trained base model. Uber also probes deeply on the feature store — they were one of the earliest companies to build a production feature store (Michelangelo), and interviewers specifically ask how you'd design the online vs offline feature pipelines for real-time ETA. For surge pricing, the question is a demand forecasting question with a pricing optimization layer on top — expect probes on "how do you prevent the surge multiplier from wildly oscillating in a 5-minute window?" (temporal smoothing, upper-bound caps, gradual step-up logic).
The meta-skill in ML system design interviews is not knowing more algorithms — it's being able to trace a line from a business problem through a data architecture, a model choice, and a serving design to a set of metrics that prove the system is working. Engineers who can do this clearly, in 45 minutes, with specific numbers and named tools, consistently pass. The architecture component is the smallest part of the answer. The reasoning around it is everything.
Deep-dive: System Design
Company-specific ML system design: recommendation systems, fraud detection, demand forecasting
KnowEaseAI
Practice ML System Design With Real Interview Questions
KnowEase.AI has ML system design questions modeled on actual Amazon, Flipkart, Coupang, and Uber interview patterns — with structured answer frameworks, AI feedback on your design decisions, and peer comparisons.
Start ML System Design Practice →Ready to practice?
Turn what you just read into skills you own
KnowEaseAI gives you AI-graded practice, structured learning checks, and a personalised roadmap. Start free — no credit card needed.
More guides
The Complete System Design Interview Guide for Indian Product Companies
Everything you need to crack the HLD round at Swiggy, Razorpay, PhonePe, CRED, Meesho, and Flipkart — from the 45-minute framework to numbers every SDE must memorise.
The 45-Minute HLD Framework: How to Ace Any System Design Interview
A battle-tested, minute-by-minute framework for structuring your system design answer — from requirements gathering to trade-off discussion — used by engineers who've cracked FAANG and unicorn interviews.
Recommended reads
Guides engineers at similar stages found useful
How to Crack Swiggy & Zomato SDE Interviews (2026 Guide)
A no-fluff breakdown of the Swiggy and Zomato engineering interview process — round structure, DSA patterns they favour, system design topics they probe, and what differentiates candidates who get offers.
SQL Interview Questions for Data Engineering — Joins, GROUP BY, Window Functions, CTEs
The four SQL primitives that show up in every data engineering interview: LEFT JOIN anti-joins for orphan detection, GROUP BY with HAVING for duplicates, ROW_NUMBER vs RANK vs DENSE_RANK for top-N ranking, and CTE composition with recursive CTEs — with worked examples and full traces.
Agentic AI Engineer Interview Preparation 2026 — The Complete Guide
The role didn't exist 2 years ago. Now every AI lab and enterprise team is hiring for it. This guide covers what agentic AI engineers actually build, how interviews are structured, 10 real questions with model answers, and a 3-month prep plan.