6.9.3 Signed characters and reductions 22g 6.9.4 MINLOC and MAXLOC 6.9.5 User-Defined Reduction Operation le of User-Defined Reduc 60e All-Redu 807 Pro s-Local Reduction 40 6 10 Redu 2 6.10.1 MPI REDUCE SCATTER BLOCK 610.2 MPI REDUCE SCATTER 244 6.11 24E sive Sca 240 6.112 247 6.11.3 g MPI_SCAN 248 6.12 50 25 6.12.2 Broad MPI_IBCAST ng Sca o1 ng( -t catter/Gather ing Re ng All-Reduce Scatter with E Equal Blocks 6.12.1 Nonbloc ocking Inclusive Scan 6.13 Pe tent Collectiv 6.13. Persistent Barrier Synchronization 6.13.2 Persistent Broadcast.······. 6.13. Persistent Gather 6.13.4 77 Persistent Scatter 6.13. Persistent Gather-to 6.13.0 Persistent All-to-All Scatter/Gather 29 6.13. Persistent Reduce 29 6.13. Persistent All-Reduce 295 6.13.9 Persistent Reduce-Scatter with Equal Blocks 296 6.13.10 Persistent Reduce-Scatter 298 6.13.11 Persistent Inclusive Scan 299 6.13.12 Persistent Exclusive Scan 6.14 Correctness............ 301 oduc upport Librarie MPI's Support for Libraries..... 7.2 7.2.1 Concepts Groups viii6.9.3 Signed Characters and Reductions . . . . . . . . . . . . . . . . . . 229 6.9.4 MINLOC and MAXLOC . . . . . . . . . . . . . . . . . . . . . . . 229 6.9.5 User-Defined Reduction Operations . . . . . . . . . . . . . . . . . 233 Example of User-Defined Reduce . . . . . . . . . . . . . . . . . . . 237 6.9.6 All-Reduce . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238 6.9.7 Process-Local Reduction . . . . . . . . . . . . . . . . . . . . . . . 240 6.10 Reduce-Scatter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242 6.10.1 MPI_REDUCE_SCATTER_BLOCK . . . . . . . . . . . . . . . . . . 242 6.10.2 MPI_REDUCE_SCATTER . . . . . . . . . . . . . . . . . . . . . . . 244 6.11 Scan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245 6.11.1 Inclusive Scan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246 6.11.2 Exclusive Scan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247 6.11.3 Example using MPI_SCAN . . . . . . . . . . . . . . . . . . . . . . 248 6.12 Nonblocking Collective Operations . . . . . . . . . . . . . . . . . . . . . . . 250 6.12.1 Nonblocking Barrier Synchronization . . . . . . . . . . . . . . . . . 252 6.12.2 Nonblocking Broadcast . . . . . . . . . . . . . . . . . . . . . . . . 253 Example using MPI_IBCAST . . . . . . . . . . . . . . . . . . . . . 253 6.12.3 Nonblocking Gather . . . . . . . . . . . . . . . . . . . . . . . . . . 254 6.12.4 Nonblocking Scatter . . . . . . . . . . . . . . . . . . . . . . . . . . 257 6.12.5 Nonblocking Gather-to-all . . . . . . . . . . . . . . . . . . . . . . . 260 6.12.6 Nonblocking All-to-All Scatter/Gather . . . . . . . . . . . . . . . . 263 6.12.7 Nonblocking Reduce . . . . . . . . . . . . . . . . . . . . . . . . . . 269 6.12.8 Nonblocking All-Reduce . . . . . . . . . . . . . . . . . . . . . . . . 270 6.12.9 Nonblocking Reduce-Scatter with Equal Blocks . . . . . . . . . . . 271 6.12.10 Nonblocking Reduce-Scatter . . . . . . . . . . . . . . . . . . . . . 273 6.12.11 Nonblocking Inclusive Scan . . . . . . . . . . . . . . . . . . . . . . 274 6.12.12 Nonblocking Exclusive Scan . . . . . . . . . . . . . . . . . . . . . . 275 6.13 Persistent Collective Operations . . . . . . . . . . . . . . . . . . . . . . . . 276 6.13.1 Persistent Barrier Synchronization . . . . . . . . . . . . . . . . . . 277 6.13.2 Persistent Broadcast . . . . . . . . . . . . . . . . . . . . . . . . . . 278 6.13.3 Persistent Gather . . . . . . . . . . . . . . . . . . . . . . . . . . . 279 6.13.4 Persistent Scatter . . . . . . . . . . . . . . . . . . . . . . . . . . . 283 6.13.5 Persistent Gather-to-all . . . . . . . . . . . . . . . . . . . . . . . . 286 6.13.6 Persistent All-to-All Scatter/Gather . . . . . . . . . . . . . . . . . 289 6.13.7 Persistent Reduce . . . . . . . . . . . . . . . . . . . . . . . . . . . 294 6.13.8 Persistent All-Reduce . . . . . . . . . . . . . . . . . . . . . . . . . 295 6.13.9 Persistent Reduce-Scatter with Equal Blocks . . . . . . . . . . . . 296 6.13.10 Persistent Reduce-Scatter . . . . . . . . . . . . . . . . . . . . . . . 298 6.13.11 Persistent Inclusive Scan . . . . . . . . . . . . . . . . . . . . . . . 299 6.13.12 Persistent Exclusive Scan . . . . . . . . . . . . . . . . . . . . . . . 300 6.14 Correctness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301 7 Groups, Contexts, Communicators, and Caching 311 7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 311 7.1.1 Features Needed to Support Libraries . . . . . . . . . . . . . . . . 311 7.1.2 MPI’s Support for Libraries . . . . . . . . . . . . . . . . . . . . . . 312 7.2 Basic Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 314 7.2.1 Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 314 viii