Inference Fundamentals in AI Inference & ServingInterview Readiness Check
Prefill vs decode phase, autoregressive generation, memory bandwidth vs compute bound regimes, roofline model for LLMs, FLOP counts, and why inference is harder than training..
Learning path
Try a sample question
During LLM inference, a 7B parameter model processes a 512-token prompt and generates 256 tokens. Which phase is most likely to be compute-bound rather than memory-bandwidth-bound?
Sign up free to take the full check and see the answer with an explanation.
What this module tests
- Conceptual understanding of inference fundamentals fundamentals
- Trade-offs and design decisions in real system scenarios
- prefill and decode — applied knowledge
- Interview-style reasoning and explanation quality
Free to start
Take the Inference Fundamentals check →
10 questions · Dimension scoring · AI coach feedback · Shareable result
Test your knowledge — free →No credit card · Takes 10–15 min
Frequently asked questions
- What does the Inference Fundamentals check test?
- The check covers prefill vs decode phase, autoregressive generation, memory bandwidth vs compute bound regimes, roofline model for llms, flop counts, and why inference is harder than training.. Questions span beginner to advanced difficulty and test both conceptual understanding and practical trade-off reasoning.
- What keywords should I know for Inference Fundamentals?
- Key concepts include: prefill, decode, autoregressive, memory bandwidth, roofline model, FLOP, inference bottleneck. These frequently appear in ai inference & serving interviews at product and infrastructure companies.
- How many questions are in the check?
- Each check has 10 questions — a mix of MCQ and short-answer. You get scored by dimension (e.g., scalability, consistency) so you know exactly where to focus.
- How long does it take to study Inference Fundamentals?
- Most engineers spend around 2 hours on this module. The check itself takes about 10–15 minutes. You can retake it up to 3 times per day.
- What prerequisite knowledge do I need before Inference Fundamentals?
- Inference Fundamentals is a foundational module — no prerequisites needed.
- Is prefill, decode, autoregressive covered?
- Yes. The Inference Fundamentals module covers prefill, decode, autoregressive among other key concepts. The check includes questions that reflect real ai inference & serving interview patterns.
What this unlocks
Transformer Memory Math
KV cache sizing formulas, activation memory, parameter memor…
Model Parallelism for Inference
Tensor parallelism (column/row linear splits), pipeline para…
GPU Architecture for Inference Engineers
A100 vs H100 vs H200 vs B100 specs, NVLink vs PCIe bandwidth…
Inference Observability & Tracing
OpenTelemetry gen_ai.* semantic conventions, tracing prefill…