Preface Parallel hardware has been ubiquitous for some time now.It's difficult to find a lap- top,desktop,or server that doesn't use a multicore processor.Beowulf clusters are nearly as common today as high-powered workstations were during the 1990s,and cloud computing could make distributed-memory systems as accessible as desktops. In spite of this,most computer science majors graduate with little or no experience in parallel programming.Many colleges and universities offer upper-division elective courses in parallel computing,but since most computer science majors have to take numerous required courses,many graduate without ever writing a multithreaded or multiprocess program. It seems clear that this state of affairs needs to change.Although many programs can obtain satisfactory performance on a single core,computer scientists should be made aware of the potentially vast performance improvements that can be obtained with parallelism,and they should be able to exploit this potential when the need arises. An Introduction to Parallel Programming was written to partially address this problem.It provides an introduction to writing parallel programs using MPI, Pthreads,and OpenMP-three of the most widely used application programming interfaces (APIs)for parallel programming.The intended audience is students and professionals who need to write parallel programs.The prerequisites are mini- mal:a college-level course in mathematics and the ability to write serial programs in C.They are minimal because we believe that students should be able to start programming parallel systems as early as possible. At the University of San Francisco,computer science students can fulfill a requirement for the major by taking the course,on which this text is based,immedi- ately after taking the"Introduction to Computer Science I"course that most majors take in the first semester of their freshman year.We've been offering this course in parallel computing for six years now,and it has been our experience that there really is no reason for students to defer writing parallel programs until their junior or senior year.To the contrary,the course is popular,and students have found that using concurrency in other courses is much easier after having taken the Introduction course. If second-semester freshmen can learn to write parallel programs by taking a class,then motivated computing professionals should be able to learn to write paral- lel programs through self-study.We hope this book will prove to be a useful resource for them. About This Book As we noted earlier,the main purpose of the book is to teach parallel programming in MPI,Pthreads,and OpenMP to an audience with a limited background in computer science and no previous experience with parallelism.We also wanted to make it as XVPreface Parallel hardware has been ubiquitous for some time now. It’s difficult to find a lap￾top, desktop, or server that doesn’t use a multicore processor. Beowulf clusters are nearly as common today as high-powered workstations were during the 1990s, and cloud computing could make distributed-memory systems as accessible as desktops. In spite of this, most computer science majors graduate with little or no experience in parallel programming. Many colleges and universities offer upper-division elective courses in parallel computing, but since most computer science majors have to take numerous required courses, many graduate without ever writing a multithreaded or multiprocess program. It seems clear that this state of affairs needs to change. Although many programs can obtain satisfactory performance on a single core, computer scientists should be made aware of the potentially vast performance improvements that can be obtained with parallelism, and they should be able to exploit this potential when the need arises. An Introduction to Parallel Programming was written to partially address this problem. It provides an introduction to writing parallel programs using MPI, Pthreads, and OpenMP—three of the most widely used application programming interfaces (APIs) for parallel programming. The intended audience is students and professionals who need to write parallel programs. The prerequisites are mini￾mal: a college-level course in mathematics and the ability to write serial programs in C. They are minimal because we believe that students should be able to start programming parallel systems as early as possible. At the University of San Francisco, computer science students can fulfill a requirement for the major by taking the course, on which this text is based, immedi￾ately after taking the “Introduction to Computer Science I” course that most majors take in the first semester of their freshman year. We’ve been offering this course in parallel computing for six years now, and it has been our experience that there really is no reason for students to defer writing parallel programs until their junior or senior year. To the contrary, the course is popular, and students have found that using concurrency in other courses is much easier after having taken the Introduction course. If second-semester freshmen can learn to write parallel programs by taking a class, then motivated computing professionals should be able to learn to write paral￾lel programs through self-study. We hope this book will prove to be a useful resource for them. About This Book As we noted earlier, the main purpose of the book is to teach parallel programming in MPI, Pthreads, and OpenMP to an audience with a limited background in computer science and no previous experience with parallelism. We also wanted to make it as xv