Fifteen Eighty Four

My new book “Programming in Parallel with CUDA – A Practical Guide” was born out of the excitement I feel about computing with GPUs.  I have always had passion for science and computer programming.  I wrote my first program in 1964 for the Cambridge EDSAC II computer using a Fortran like programming language. Since then, I used various dialects of Fortran in the 1970s and 80s for applications in experimental high energy physics at CERN and elsewhere. More recently I have been using C and C++ to develop applications in medical imaging and other areas of physics. Watching the Moore’s Law doubling of computing power every 18 months or so over more than 5 decades has have been a truly amazing journey. The figure dramatically illustrates Moore’s Law in terms of transistor counts per card.

NVIDIA’s launch of CUDA for GPU programming in 2007 broke Moore’s Law in the sense that suddenly computing power suddenly jumped by a factors of 100 not 2. Moreover, the C++ programming language was used to write the GPU code. Thus, there was really is very little new stuff to learn!

My book is designed for people who want to get more processing power out of their PC, workstation or large-scale computing system by using the power of modern GPUs. It will teach you how to use NDVIDA’s C/C++ CUDA programming language to get the most out your GPUs.

My book is by no means the first book on CUDA, but it is different – it has a much richer set of real-world examples than previous books. Also, I care deeply about style in computer code. Good code should be compact and clear avoiding “clever tricks” to optimise fragments of code.  One revelation in developing the examples was just how good the CUDA NVCC compiler is at optimising code.  We think our examples are elegant, efficient and uncluttered. For example, we use modern C++ container classes to hold arrays rather than relying of the older C style malloc based allocation still seen in most current tutorial CUDA code.  We also use small set of utilities provided in “cx” header files.

Many of our examples are fully working programs and cover topics including

• Iterative solution of partial differential equations,
• Image manipulation including Richardson-Lucy image deblurring,
• Medical image registration,
• Simulation of a medical PET scanner,
• Reconstruction of PET images from listmode data using the MLEM method,
• The Ising model in solid state Physics, with a simple interactive visualisation,
• Use of CUDA textures, including our cx container class which simplifies their creation,
• Use of Tensor Core hardware.

While this code may be directly useful in some cases its main purpose is to provide models which readers can rapidly modify to solve their own problems.

We also include many other short examples to illustrate various features of CUDA  and other parallel programming tools such as openMP and MPI.

I do not have space here to tell you everything about the book but you will be able to find all the examples at https://github.com/RichardAns/CUDA-Programs.  I hope you enjoy my book.