Intermediate~2h
Small Language Models & On-Device Inference in LLM CoreInterview Readiness Check
Sub-8B models in 2026: Phi-4, Gemma 3, Llama 3.2 1B/3B, Qwen 2.5, distilled reasoning. Quantization (GGUF, AWQ, MLX), runtimes (llama.cpp, Ollama, WebLLM, MediaPipe, ExecuTorch), mobile and browser deployment, hybrid small-local + big-cloud routing..
small language modelSLMon-device LLMPhi-4Gemma 3Llama 3.2QwenGGUFAWQMLXOllamallama.cppWebLLMMediaPipeExecuTorchquantization
Learning path
Cost & Token EconomicsOpen-source Models & Self-hosting→Small Language Models & On-Device Inference ← you are here
Try a sample question
A mobile app wants to run an LLM for grammar correction entirely on-device (no internet required). Which model family is most appropriate?
A.Small language models (SLMs) like Phi-3-mini (3.8B), Gemma 2B, or SmolLM (1.7B) — these can run on mobile devices with 4-8GB RAM using quantisation, enabling offline inference
B.GPT-4o accessed via API — mobile apps can cache responses for offline use
C.BERT-large (340M parameters) — the smallest model that can handle grammar tasks
D.LLaMA-3-70B with 4-bit quantisation — large models always produce better grammar corrections
Sign up free to take the full check and see the answer with an explanation.
What this module tests
- Conceptual understanding of small language models & on-device inference fundamentals
- Trade-offs and design decisions in real system scenarios
- small language model and SLM — applied knowledge
- Interview-style reasoning and explanation quality
Free to start
Take the Small Language Models & On-Device Inference check →
10 questions · Dimension scoring · AI coach feedback · Shareable result
Test your knowledge — free →No credit card · Takes 10–15 min
Frequently asked questions
- What does the Small Language Models & On-Device Inference check test?
- The check covers sub-8b models in 2026: phi-4, gemma 3, llama 3.2 1b/3b, qwen 2.5, distilled reasoning. quantization (gguf, awq, mlx), runtimes (llama.cpp, ollama, webllm, mediapipe, executorch), mobile and browser deployment, hybrid small-local + big-cloud routing.. Questions span beginner to advanced difficulty and test both conceptual understanding and practical trade-off reasoning.
- What keywords should I know for Small Language Models & On-Device Inference?
- Key concepts include: small language model, SLM, on-device LLM, Phi-4, Gemma 3, Llama 3.2, Qwen, GGUF, AWQ, MLX, Ollama, llama.cpp, WebLLM, MediaPipe, ExecuTorch, quantization. These frequently appear in llm core interviews at product and infrastructure companies.
- How many questions are in the check?
- Each check has 10 questions — a mix of MCQ and short-answer. You get scored by dimension (e.g., scalability, consistency) so you know exactly where to focus.
- How long does it take to study Small Language Models & On-Device Inference?
- Most engineers spend around 2 hours on this module. The check itself takes about 10–15 minutes. You can retake it up to 3 times per day.
- What prerequisite knowledge do I need before Small Language Models & On-Device Inference?
- Small Language Models & On-Device Inference builds on Cost & Token Economics and Open-source Models & Self-hosting. Complete those first for the best results.
- Is small language model, SLM, on-device LLM covered?
- Yes. The Small Language Models & On-Device Inference module covers small language model, SLM, on-device LLM among other key concepts. The check includes questions that reflect real llm core interview patterns.