Skip to main content
🪔✨10% off ProKNOWEASEAI10Claim offer →✨🎆
Intermediate~3h

cuDNN, cuBLAS, CUTLASS & NCCL in GPU Compute & ML KernelsInterview Readiness Check

When to use library kernels vs custom kernels. cuBLAS: the workhorse GEMM library — batched GEMM, cublasGemmEx for mixed precision, when cuBLAS underperforms and why. cuDNN: the conv/pooling/normalization library for deep learning — workspace management, algorithm selection, determinism tradeoffs. CUTLASS: NVIDIA's composable GEMM kernel template library — why it matters for writing kernels that exceed cuBLAS on custom matrix shapes. NCCL: the communication library for multi-GPU — AllReduce, AllGather, ReduceScatter semantics and performance. How PyTorch and JAX call these libraries under the hood..

cuBLAScuDNNCUTLASSNCCLGEMMbatched GEMMcublasGemmExAllReduceAllGatherReduceScatterworkspacealgorithm selectionPyTorch backendlibrary kernelcustom kernel

What this module tests

  • Conceptual understanding of cudnn, cublas, cutlass & nccl fundamentals
  • Trade-offs and design decisions in real system scenarios
  • cuBLAS and cuDNN — applied knowledge
  • Interview-style reasoning and explanation quality

Free to start

Take the cuDNN, cuBLAS, CUTLASS & NCCL check →

10 questions · Dimension scoring · AI coach feedback · Shareable result

Test your knowledge — free →

No credit card · Takes 10–15 min

Frequently asked questions

What does the cuDNN, cuBLAS, CUTLASS & NCCL check test?
The check covers when to use library kernels vs custom kernels. cublas: the workhorse gemm library — batched gemm, cublasgemmex for mixed precision, when cublas underperforms and why. cudnn: the conv/pooling/normalization library for deep learning — workspace management, algorithm selection, determinism tradeoffs. cutlass: nvidia's composable gemm kernel template library — why it matters for writing kernels that exceed cublas on custom matrix shapes. nccl: the communication library for multi-gpu — allreduce, allgather, reducescatter semantics and performance. how pytorch and jax call these libraries under the hood.. Questions span beginner to advanced difficulty and test both conceptual understanding and practical trade-off reasoning.
What keywords should I know for cuDNN, cuBLAS, CUTLASS & NCCL?
Key concepts include: cuBLAS, cuDNN, CUTLASS, NCCL, GEMM, batched GEMM, cublasGemmEx, AllReduce, AllGather, ReduceScatter, workspace, algorithm selection, PyTorch backend, library kernel, custom kernel. These frequently appear in gpu compute & ml kernels interviews at product and infrastructure companies.
How many questions are in the check?
Each check has 10 questions — a mix of MCQ and short-answer. You get scored by dimension (e.g., scalability, consistency) so you know exactly where to focus.
How long does it take to study cuDNN, cuBLAS, CUTLASS & NCCL?
Most engineers spend around 3 hours on this module. The check itself takes about 10–15 minutes. You can retake it up to 3 times per day.
What prerequisite knowledge do I need before cuDNN, cuBLAS, CUTLASS & NCCL?
cuDNN, cuBLAS, CUTLASS & NCCL is a foundational module — no prerequisites needed.
Is cuBLAS, cuDNN, CUTLASS covered?
Yes. The cuDNN, cuBLAS, CUTLASS & NCCL module covers cuBLAS, cuDNN, CUTLASS among other key concepts. The check includes questions that reflect real gpu compute & ml kernels interview patterns.