Learn more about the program in this spotlight article!
Certificate Program in GPU-Accelerated Computing & Visualization (ONLINE)
The Graphics Processing Unit (GPU) has become the computational engine of modern artificial intelligence. Every frontier model — large language models, diffusion-based image and video generators, and the neural 3D reconstruction methods reshaping computer vision — is trained and served on massively parallel GPU hardware. The defining challenge is no longer whether GPUs can accelerate a workload, but how to program them well: how to map an algorithm onto the memory hierarchy, saturate Tensor Cores, keep thousands of cores busy, and scale a computation across dozens or thousands of interconnected devices. This certificate program teaches exactly those skills.
In this hands-on, project-based program, you’ll learn and apply the theory, practical engineering, and the undocumented “black art” techniques required to develop optimized, high-performance parallel GPU software — the same techniques used to build the kernels and systems behind today’s production AI. Across three complementary online courses, you’ll progress from GPU hardware and the CUDA programming model, through multi-GPU and multi-node distributed training and inference, to the Python-centric ML systems ecosystem (Triton, CuTe DSL, and PyTorch) that practitioners use in the field. Throughout, abstractions are grounded in current, real-world case studies — FlashAttention, state-space models (Mamba), 3D Gaussian Splatting, Megatron-Core and vLLM, quantized inference, diffusion-model generation, and neural radiance fields — so you learn each parallel pattern by seeing it inside the systems that made it famous. We use NVIDIA CUDA on modern Hopper- and Blackwell-class GPUs, together with the current professional toolchain (NVIDIA Nsight profilers, GPU kernel debuggers, Triton, and PyTorch), and you’ll finish able to read, write, profile, and scale production-grade GPU code.
Admissions
This certificate program is designed for working professionals in science, engineering, and software industries interested in developing their skills in optimized parallel GPU software design and programming, with applications in machine learning, scientific/engineering and numerical computing, biomedical, computer graphics, and scientific visualization.
See Admissions Requirements
What You’ll Learn
- GPU hardware architecture and the CUDA execution model: streaming multiprocessors, the SIMT model, warps and thread-block clusters, and the memory hierarchy from registers and shared memory to HBM
- CUDA kernel design, debugging, profiling, and performance analysis — including roofline and arithmetic-intensity reasoning, coalesced memory access, occupancy and other key HW metrics, and Nsight-based kernel optimization
- The canonical parallel patterns behind modern ML kernels: tiled matrix multiplication (GEMM) and Tensor Core (WGMMA/UMMA) acceleration, reductions and online softmax, parallel scan, sort, and generalized convolution
- Multi-GPU and multi-node scaling: NCCL collective communication, NVLink/NVSwitch and InfiniBand interconnects, and the full parallelism toolkit (data, tensor, pipeline, sequence, context, and expert parallelism) used to train and serve large models
- The Python-centric GPU programming stack: writing high-performance kernels in Triton and the CuTe DSL, and integrating custom operators with PyTorch and torch.compile
- Applied case studies across the ML landscape: FlashAttention, Mamba selective-scan, 3D Gaussian Splatting, distributed training (Megatron-Core) and inference serving (vLLM), quantization (FP8/FP4, GPTQ/AWQ), diffusion-model generation, and neural radiance fields (Instant-NGP)
Curriculum
Autumn: Applied High-Performance GPU Computing
Winter: GPU-Accelerated Methods for Multi-GPU and Multi-Node — Scaling Machine Learning
Spring: GPU-Accelerated Machine Learning — In Practice and Under-the-Hood
To successfully complete the certificate, students must complete three classes related to the certificate, with total credits equaling 12. A minimum of 2.7 is required in each course that is counted toward the certificate.
Earn Graduate Credit
By completing this certificate program, you can earn up to 12 credits toward degree requirements should you later be accepted into the Professional Master’s Program. To use these credits, you must obtain graduate non-matriculated status before you register for the first course of the certificate program.
See Tuition & Fees Information
Get Hands-on Experience
The certificate’s project-based courses utilize the latest highly-parallelized graphics and compute APIs such as Khronos OpenGL and NVIDIA CUDA standards.
Technology Requirements
Students in ECE certificate programs are expected to adhere to the College of Engineering’s technology expectations. Students in this certificate will not need to provide their own GPU hardware, and will have access to AWS Appstream virtual desktop development environments.
Certificate Instructor
Courses in the certificate are taught by Colin Reinhardt, an affiliate assistant professor in the Department of Electrical & Computer Engineering at the University of Washington, and an engineer at SPAWAR. His research areas include physically-based rendering, scientific visualization, GPU computing, CUDA and OpenCL, OpenGL, modeling & simulation, predictive rendering, atmospheric propagation, machine learning and neural networks, electro-optics, turbulence theory, scattering theory, radiative transfer, wave propagation in random media, communications, imaging, and stochastic process theory.