MLOps Interview Questions and Roadmap — What's Actually Asked in 2026

MLOps interview questions have changed. Three years ago, "MLOps" meant "can you write a Dockerfile and deploy a Flask app?" Today, companies expect you to know feature stores, CI/CD for ML pipelines, production drift detection, and distributed training orchestration.

The bar went up because ML in production went mainstream. This guide covers what MLOps interviewers actually test in 2026, domain by domain, with the questions that come up most often and the answers that pass.

What MLOps Actually Means in 2026

MLOps is the discipline of getting ML models into production reliably and keeping them working there. That sounds simple. The reality is a set of genuinely hard engineering problems that don't exist in traditional software:

• Training/serving skew — the data distribution during training looks different from what the model sees at inference time
• Model decay — a model that's accurate today will degrade as the world changes, usually without any error you can catch with traditional monitoring
• Reproducibility — ML experiments are notoriously hard to reproduce. Same code, same data, different random seed: different model.
• Data dependencies — your model is a function of your data pipeline. The data pipeline is usually messier than the model.
• Non-deterministic failures — unlike a web service where a 500 error is obvious, a degraded ML model continues returning 200s with quietly wrong predictions

MLOps engineers build the systems that manage all of this: training pipelines, feature stores, model registries, serving infrastructure, and monitoring.

Interview Structure at MLOps Roles

The scenario-based round is unique to MLOps and trips most candidates. It's testing your mental model of production ML systems — how you think through degradation, what you check first, how you isolate root cause. We'll cover this in the monitoring section.

Domain 1: CI/CD for ML

Software CI/CD: code changes → build → test → deploy. ML CI/CD: code changes + data changes + model changes → retrain → evaluate → deploy.

The extra complexity: the artifact you're testing is a trained model, not compiled code. A model that passes unit tests can still fail on production data distribution. So CI/CD for ML needs an extra validation gate: does the new model actually perform better than the current production model?

A mature ML CI/CD pipeline looks like this:

Code push → 
  Unit tests (data validation, preprocessing functions) →
  Training pipeline run (on a subset of data for speed) →
  Model evaluation (compare new model vs champion on holdout set) →
  Integration tests (does the serving endpoint work?) →
  Shadow deploy (run new model in parallel, compare outputs) →
  Canary deploy (5% traffic) →
  Full rollout

Key tools: GitHub Actions or GitLab CI for orchestration, MLflow or W&B for experiment tracking and model registry, Great Expectations or dbt tests for data validation, Evidently for model comparison reports.

Interview question: "How do you handle a CI/CD pipeline where training takes 6 hours?"

Answer: Don't run full training on every commit. Trigger full training only on merges to main, or on a schedule, or when data drift is detected. For PRs, run a fast smoke test — train on 10% of data, check that loss decreases and evaluation metrics are reasonable.

Domain 2: Feature Stores

A feature store is a centralized system for computing, storing, and serving ML features. It solves two specific problems:

Point-in-time correctness — when training, you must use only features that were available at the time of each label. If you accidentally use future information (e.g., a feature computed after the label was generated), you get training/serving skew and your model will underperform in production. Feature stores handle this automatically.

Online/offline consistency — the same feature logic must produce the same output whether you're training (reading from a batch warehouse) or serving (reading from a low-latency cache). Without a feature store, teams often accidentally have two implementations — one in Python for training, one in Java for serving — that diverge.

The most common interview question on feature stores: "Explain point-in-time correctness and why it matters."

Walk through a concrete example: predicting whether a customer will churn. The label is "churned within 30 days of date D." A feature is "number of logins in the past 7 days." If you compute that feature using data from after date D, you're leaking future information. Point-in-time correct queries filter features to only include data available as of the label timestamp.

Domain 3: Model Serving

Getting a model into production means choosing: batch vs real-time, and at what scale.

Batch inference — run predictions on a dataset overnight, store results. Best for: churn prediction, recommendation pre-computation, risk scoring. Latency doesn't matter; throughput does.

Real-time inference — serve predictions synchronously with low latency. Best for: fraud detection (must happen before transaction completes), personalization (user is waiting). Latency matters; p99 < 100ms is typical.

Key latency levers:

• Model optimization: quantization (FP32 → INT8 reduces model size 4×, inference 2–3× faster), ONNX export (platform-independent format, usually faster than native Python)
• Batching: even in real-time serving, batch multiple requests together if they arrive within a 5–10ms window — GPU utilization goes from 20% to 80%
• Caching: if the same input repeats (common in recommendation systems), cache the output. Redis with a 1-hour TTL handles most cases.
• Hardware: GPU for large deep learning models, CPU with optimized libraries (OpenBLAS, MKL) for classical ML

Key tools: BentoML (general ML serving), Triton Inference Server (NVIDIA, optimized for GPU), TorchServe (PyTorch native), Ray Serve (distributed), vLLM (LLMs specifically).

Domain 4: Monitoring and Drift Detection

This is the domain that separates junior MLOps from senior. A deployed model is not a deployed web service. The model can silently degrade while returning HTTP 200 every time.

Two types of drift you must know:

Data drift — the distribution of input features changes. Example: a fraud model trained on 2024 transaction data starts seeing 2026 transaction patterns (new payment methods, different amounts, different merchant categories). The model's input distribution has shifted.

Detection: compare the distribution of live inference features against the training distribution using statistical tests. Population Stability Index (PSI) is the most common: PSI < 0.1 means stable, 0.1–0.25 means moderate drift, >0.25 means significant drift. Kolmogorov-Smirnov test for continuous features. Chi-squared test for categorical.

Concept drift — the relationship between features and labels changes. Example: a spam classifier trained in 2023 starts failing because spammers changed their patterns. The inputs look fine, but the ground truth has shifted.

Detection is harder because you need ground truth labels from production, which are delayed or expensive. Common approaches: shadow labeling (label a sample of production predictions), upstream proxy metrics (click-through rate, conversion rate as a proxy for model quality).

Tools: Evidently (open source, excellent dashboards), WhyLabs, Arize AI, Fiddler AI (enterprise), Prometheus + Grafana (if you roll your own).

LLM-specific monitoring (increasingly tested in 2026 interviews): If you're at a company using LLMs in production, classical drift metrics don't fully apply. For LLM pipelines you need:

• Hallucination rate: track what percentage of responses contain claims not grounded in context. Use an LLM judge sampling 5–10% of production outputs.
• Prompt regression monitoring: every prompt change is a production deployment. Run a comparison on sampled historical queries before rolling out — prompt edits silently break edge cases.
• Token cost attribution: attribute LLM API costs to specific features. A single poorly-optimized prompt can double your inference bill without appearing in any accuracy metric.
• Latency components: track TTFT (time to first token) and TPOT (time per output token) separately — they have different optimization levers.

Tools for LLM observability: LangSmith (LangChain-native tracing), Langfuse (open source, self-hostable), Helicone (proxy-based, zero code changes), Braintrust (eval + monitoring combined).

Domain 5: Infrastructure and Orchestration

ML workloads have different infrastructure requirements than web services:

• Training jobs are long-running (hours to days) and stateful — they checkpoint progress
• Training jobs need GPU access — regular Kubernetes scheduling doesn't handle GPU affinity well
• Data pipelines are DAG-shaped — step B can't start until step A finishes and its output is validated

Key tools to know:

Orchestration: Apache Airflow (most common, Python DAGs), Prefect (cleaner API, better error handling), Metaflow (ML-focused, built by Netflix), Kubeflow Pipelines (Kubernetes-native).

Training infrastructure: Ray Train (distributed training on any cluster), Horovod (distributed deep learning), SageMaker Training Jobs (AWS-managed), Vertex AI Training (GCP-managed).

Experiment tracking: MLflow (open source, most widely used), Weights & Biases (better UI, strong community), Neptune AI (enterprise).

Model registry: MLflow Model Registry or W&B Artifact Registry — you register model versions, transition them through staging → production → archived.

12 MLOps Interview Questions with Answers

Q1: What is training/serving skew and how do you prevent it?

Training/serving skew happens when the feature engineering logic is different between training and inference. Prevention: use the same feature computation code in both environments — ideally through a feature store that enforces it. If you can't, write exhaustive integration tests that run both code paths on the same input and compare outputs.

Q2: How do you handle a model that degrades over time?

Build automated retraining triggers based on drift metrics (PSI > 0.25) or performance proxy metrics (click-through rate drops by 5%). The retrained model must beat the champion model on a holdout set before it gets deployed. Without the holdout validation gate, you can retrain yourself into a worse model.

Q3: Explain point-in-time correctness.

When training a model, each training example has a timestamp — the time the label was generated. All features used for that example must be computed using only data available before that timestamp. A feature store enforces this with temporal joins. Without it, you'll see inflated training metrics and disappointing production performance.

Q4: Design a CI/CD pipeline for an ML model that retrains weekly.

Schedule: weekly cron job triggers training pipeline. Data validation gate: check that new training data doesn't have null rates > 5% or distribution shift. Training: run on full dataset, log to MLflow. Evaluation: compare new model against champion on a held-out test set — new model must improve by at least 0.5% on the primary metric. If it passes: shadow deploy for 24 hours, then canary to 10% traffic, then full rollout.

Q5: How would you detect if your model is making poor predictions in production without ground truth labels?

Monitor the model's output distribution — if the distribution of predicted probabilities shifts (e.g., scores cluster near 0 or 1 when they didn't before), something is wrong. Monitor input feature distributions for drift. Use proxy metrics where available (for a recommendation model, watch click-through rate). Set up a human review queue for a small % of predictions.

Q6: What is the difference between an online and offline feature store?

Offline store: historical feature values stored in a data warehouse (BigQuery, Redshift). Used for training — you read a point-in-time snapshot of features. Slow (seconds to minutes). Online store: current feature values in a low-latency database (Redis, DynamoDB). Used for inference — you look up features for the current user in real time. Fast (< 10ms). A feature store manages both layers and keeps them in sync.

Q7: How do you version ML models?

A model version includes: the model artifact (weights), the training code commit hash, the dataset version (DVC hash or data snapshot timestamp), and the hyperparameters. All four must be logged to reproduce the model. Store in a model registry (MLflow). Tag versions by stage: dev → staging → production → archived.

Q8: Your training job runs for 8 hours and fails at hour 7 due to OOM. What do you do?

Short term: add checkpointing so training can resume from the last checkpoint. Investigate the OOM: gradient accumulation instead of large batches, mixed precision training (FP16 halves memory), model parallelism if the model itself is too large. Profile memory usage per batch with PyTorch's memory profiler to find the exact step where memory spikes.

Q9: What is shadow deployment and when would you use it?

Shadow deployment: the new model runs in parallel with the production model on real traffic, but its predictions are logged and not served to users. You compare the two models' predictions and metrics before committing to the new model. Use it when you want to validate a major model change (new architecture, new features) on real traffic with zero user impact.

Q10: How do you handle class imbalance in production?

First: make sure your evaluation metric accounts for it (use F1 or AUC-PR, not accuracy). During training: oversample the minority class (SMOTE), undersample the majority class, or use class weights. In serving: monitor the model's calibration — a model trained on imbalanced data often returns overconfident probabilities. Use isotonic regression or Platt scaling to recalibrate.

Q11: Describe the key components of an ML platform.

Data ingestion and validation layer → Feature store (online + offline) → Training orchestration → Experiment tracking + model registry → Model serving layer → Monitoring and alerting. The platform exists to make ML engineers productive — self-service training, one-click deployment, and automatic monitoring without infra expertise.

Q12: A senior ML engineer says "the model worked in the notebook but fails in production." How do you debug this?

Classic training/serving skew. Check: (1) are the feature preprocessing steps identical? (2) are categorical encodings consistent (same mapping of category → integer)? (3) is the feature order the same? (4) are there null values in production that didn't exist in training? (5) is the model artifact correctly serialized (pickle, ONNX, or TorchScript) and loaded?

4-Month MLOps Roadmap

Practice on KnowEaseAI

The scenario and system design rounds are where MLOps candidates struggle most — they require integrating knowledge across all 5 domains simultaneously. The best way to prepare is structured practice with feedback.