Data Parallel C++ brief book review and recommendation (Part 1)

Bjarne Stroustrup has written in one of his books that “Our civilization runs on software”. Hence writing on anything related to software is itself part of learning for me. One such topic is parallel programming which is making inroads in super computing (SC) domain. The SC conference series 2020 has been once again full of tutorials, workshops and novel research papers. As a researcher involved in accelerating algorithms I’m interested in reading different technical books.

The recent addition is this interesting book namely Data Parallel C++.

The book comes straight from the horse’s mouth i.e. the authors have an extensive experience in parallel programming.

Preface

The book is for people interested in programming for data parallelism using C++. Hence you should be at least familiar with C based languages (C/C++) .

If you are a novice to parallel programming, no problem. Even if you have not heard of SYCL or the compiler namely DPC++, no problem.

Considering programmers who are novice to parallel programming, its important to give a brief background about the group involved behind the language and compiler.

Khronos Group

The Khronos Group, Inc. is an American non-profit member-funded industry consortium based in Beaverton, Oregon, focused on the creation of open standard, royalty-free application programming interfaces (APIs) for authoring and accelerated playback of dynamic media on a wide variety of platforms and devices. Khronos members may contribute to the development of Khronos API specifications, vote at various stages before public deployment, and accelerate delivery of their platforms and applications through early access to specification drafts and conformance tests.

SYCL:

SYCL is a higher-level programming model for OpenCL as a single-source domain specific embedded language (DSEL) based on pure C++11 for SYCL 1.2.1 to improve programming productivity. This is a standard developed by Khronos Group, announced in March 2014.

DPC++:

DPC++ is an open source compiler project that is based on SYCL, a few extensions, and broad heterogeneous support that includes GPU, CPU, and FPGA support.

Chapters:

There are 19 chapters:

Chapter 1: Introduction Chapter 2: Where code executes Chapter 3: Data Management Chapter 4: Expressing Parallelism Chapter 5 :Error Handling Chapter 6: Unified Shared Memory Chapter 7: Buffers Chapter 8: Scheduling Kernels and Data Movement Chapter 9: Communication and Synchronization Chapter 10: Defining kernels Chapter 11: Vectors Chapter 12: Device Information Chapter 13: Practical tips Chapter 14: Common Parallel Patterns Chapter 15: Programming for GPUs Chapter 16: Programming for CPUs Chapter 17: Programming for FPGAs Chapter 18: Libraries Chapter 19: Memory Model Atomics

Hold your horses!

There is a reason why I have mentioned all these chapters above. The book mentions “Read the Book, Not the Spec”. I agree. In fact I would add more to it i.e. no one learnt programming only through reading a book. In next parts of the blog I will be going through code examples from each chapter and sharing my experience of running these codes on DPC++ compiler.