CUDA Official Site
(https://www.nvidia.com/object/cuda_home_new.html)
What is CUDA?
Background
EcoSystem
NVIDIA CUDA Getting Started Guide for Microsoft Windows – CUDA Toolkit Documentation – NVIDIA Developer Zone
(http://docs.nvidia.com/cuda/cuda-getting-started-guide-for-microsoft-windows/#axzz45QciDIk9)
CUDA® is a parallel computing platform and programming model invented by NVIDIA. It enables dramatic increases in computing performance by harnessing the power of the graphics processing unit (GPU).
CUDA was developed with several design goals in mind:
- Provide a small set of extensions to standard programming languages, like C, that enable a straightforward implementation of parallel algorithms. With CUDA C/C++, programmers can focus on the task of parallelization of the algorithms rather than spending time on their implementation.
- Support heterogeneous computation where applications use both the CPU and GPU. Serial portions of applications are run on the CPU, and parallel portions are offloaded to the GPU. As such, CUDA can be incrementally applied to existing applications. The CPU and GPU are treated as separate devices that have their own memory spaces. This configuration also allows simultaneous computation on the CPU and GPU without contention for memory resources.
CUDA-capable GPUs have hundreds of cores that can collectively run thousands of computing threads. These cores have shared resources including a register file and a shared memory. The on-chip shared memory allows parallel tasks running on these cores to share data without sending it over the system memory bus.
This guide will show you how to install and check the correct operation of the CUDA development tools.
Sanders, Jason and Kandrot, Edward (2011) CUDA by Example – An Introduction to General-Purpose GPU Programming, NVIDIA Corporation, Addison-Wesley
Download PDF (PDF 2,554KB)
Download Source Code (ZIP 337KB)
CUDA by Example, written by two senior members of the CUDA software platform team, shows programmers how to employ this new technology. The authors introduce each area of CUDA development through working examples. After a concise introduction to the CUDA platform and architecture, as well as a quick-start guide to CUDA C, the book details the techniques and trade-offs associated with each key CUDA feature. You’ll discover when to use each CUDA C extension and how to write CUDA software that delivers truly outstanding performance.
Table of Contents
- Why CUDA? Why Now?
- Getting Started
- Introduction to CUDA C
- Parallel Programming in CUDA C
- Thread Cooperation
- Constant Memory and Events
- Texture Memory
- Graphics Interoperability
- Atomics
- Streams
- CUDA C on Multiple GPUs
- The Final Countdown
All the CUDA software tools you’ll need are freely available for download from NVIDIA.
Palmer, Oliver (2015) Newbies Guide to CUDA, Second Edition, CreateSpace Independent Publishing Platform
(https://www.udacity.com/course/intro-to-parallel-programming–cs344)
Learn the fundamentals of parallel computing with the GPU and the CUDA programming environment! In this class, you’ll learn about parallel programming by coding a series of image processing algorithms, such as you might find in Photoshop or Instagram. You’ll be able to program and run your assignments on high-end GPUs, even if you don’t own one yourself.
Why It’s Important to Think Parallel
Third Pillar of Science
Learn how scientific discovery can be accelerated by combining theory and experimentation with computing to fight cancer, prevent heart attacks, and spur new advances in robotic surgery.
Lesson 1: GPU Programming Model
Project 1: Greyscale Conversion (for that classy touch!)
Lesson 2: GPU Hardware and Parallel Communication
Project 2: Smart Blurring (miracle product for removing wrinkles!)
Lesson 3: Fundamental Parallel Algorithms
Project 3: HDR Tonemapping (when 1000:1 contrast is not enough!)
Lesson 4: Using Sort and Scan
Project 4: Red Eye Removal (soothing relief for bright red eyes)
Lesson 5: Optimizing GPU Programs
Project 5: Accelerating Histograms (when fast isn’t fast enough)
Lesson 6: Parallel Computing Patterns
Project 6: Seamless Image Compositing (polar bear in the swimming pool)
Lesson 7: The Frontiers and Future of GPU Computing
Introduction to Accelerated Computing
(https://nvidia.qwiklab.com/searches/lab?keywords=Introduction%20to%20Accelerated%20Computing)
Learn about the three techniques for accelerating code on a GPU; Libraries, Directives like OpenACC, and writing code directly in CUDA-enabled langauges. In 45 minutes, you will work through a few different exercises demonstrating the potential speed-ups and ease of use of porting to the GPU.
CUDA University Courses
(https://developer.nvidia.com/educators/existing-courses)
ECE 408 Applied Parallel Programming
University of Illinois
(https://wiki.cites.illinois.edu/wiki/display/ece408fa15/ECE+408+Applied+Parallel+Programming+Home)
CS 193G: Programming Massively Parallel Processors with CUDA
Stanford University
(https://github.com/jaredhoberock/stanford-cs193g-sp2010)
EEC 171, Parallel Computer Architecture
UC Davis
(https://www.nvidia.com/object/cudau_ucdavis)
ME964/ME759: High-Performance Computing for Applications in Engineering
University of Wisconsin, Madison:
(http://sbel.wisc.edu/Courses/ME964/2015/)
Course on CUDA Programming on NVIDIA GPUs
University of Oxford
(https://people.maths.ox.ac.uk/~gilesm/cuda/
Many resources available
CUDA, Supercomputing for the Masses: Part 1
(http://www.drdobbs.com/parallel/cuda-supercomputing-for-the-masses-part/207200659)
Part 1 of 20 Parts