Contents Preface ................................................................................ XV Acknowledgments.… xviii About the Author…… xix CHAPTER 1 Why Parallel Computing?................................... 1 1.1 Why We Need Ever-Increasing Performance.................. 2 1.2 Why We're Building Parallel Systems ....................... 3 1.3 Why We Need to Write Parallel Programs..................... 3 1.4 How Do We Write Parallel Programs?......................... 6 1.5 What We'll Be Doing.................................... 8 1.6 Concurrent,Parallel,Distributed............................... 9 1.7 The Rest of the Book................................. 10 1.8 A Word of Warning............................................... 10 1.9 Typographical Conventions..... 11 1.10 Summary............................................ 12 1.11 Exercises.… 12 CHAPTER 2 Parallel Hardware and Parallel Software.................... 15 2.1 Some Background.... 15 2.1.1 The von Neumann architecture........................ 15 2.1.2 Processes,multitasking,and threads.................. 17 2.2 Modifications to the von Neumann Model..................... 18 2.2.1 The basics of caching................................ 19 2.2.2 Cache mappings................ 20 2.2.3 Caches and programs:an example 22 2.2.4 Virtual memory ....................................... 23 2.2.5 Instruction-level parallelism........................... 25 2.2.6 Hardware multithreading.............................. 28 2.3 Parallel Hardware.… 29 2.3.1 SIMD systems.… 29 2.3.2 MIMD systems.........… 32 2.3.3 Interconnection networks.............................. 35 2.3.4 Cache coherence................... 43 2.3.5 Shared-memory versus distributed-memory.......... 46 2.4 Parallel Software..................................... 47 2.4.1 Caveats.......... 47 2.4.2 Coordinating the processes/threads.................... 48 2.4.3 Shared-memory.............................. 49 i送Contents Preface .............................................................................. xv Acknowledgments .................................................................. xviii About the Author ................................................................... xix CHAPTER 1 Why Parallel Computing?................................... 1 1.1 Why We Need Ever-Increasing Performance .................. 2 1.2 Why We’re Building Parallel Systems ......................... 3 1.3 Why We Need to Write Parallel Programs ..................... 3 1.4 How Do We Write Parallel Programs? ......................... 6 1.5 What We’ll Be Doing ............................................. 8 1.6 Concurrent, Parallel, Distributed ................................ 9 1.7 The Rest of the Book ............................................. 10 1.8 A Word of Warning ............................................... 10 1.9 Typographical Conventions ...................................... 11 1.10 Summary ........................................................... 12 1.11 Exercises ........................................................... 12 CHAPTER 2 Parallel Hardware and Parallel Software .................... 15 2.1 Some Background................................................. 15 2.1.1 The von Neumann architecture ........................ 15 2.1.2 Processes, multitasking, and threads .................. 17 2.2 Modifications to the von Neumann Model ..................... 18 2.2.1 The basics of caching................................... 19 2.2.2 Cache mappings ........................................ 20 2.2.3 Caches and programs: an example .................... 22 2.2.4 Virtual memory ......................................... 23 2.2.5 Instruction-level parallelism ........................... 25 2.2.6 Hardware multithreading............................... 28 2.3 Parallel Hardware ................................................. 29 2.3.1 SIMD systems .......................................... 29 2.3.2 MIMD systems ......................................... 32 2.3.3 Interconnection networks .............................. 35 2.3.4 Cache coherence ........................................ 43 2.3.5 Shared-memory versus distributed-memory .......... 46 2.4 Parallel Software .................................................. 47 2.4.1 Caveats .................................................. 47 2.4.2 Coordinating the processes/threads.................... 48 2.4.3 Shared-memory ......................................... 49 ix