6.3.2 Group Constructors······················· 193 6.3.3 Group Destructors······················· 198 6.4 Communicator Management·············· 199 6.4.1 Communicator Accessors 199 6.4.2 Communicator Constructors.... 200 6.4.3 Communicator Destructors 208 6.5 Motivating Examples......... 209 6.5.1 Current Practice #1 209 6.5.2 Current Practice #2... 210 6.5.3 (Approximate)Current Practice #3 210 6.5.4 Example#4.... 211 6.5.5 Library Example #1 212 6.5.6 Library Example #2 213 6.6 Inter-Communication...... 216 6.6.1 Inter-communicator Accessors... 217 6.6.2 Inter-communicator Operations 219 6.6.3 Inter-Communication Examples .. 221 Example 1:Three-Group“Pipeline'” 221 Example2:Three-Group“Ring”... 222 6.7 Caching. 224 6.7.1 Functionalitv 225 6.7.2 Communicators..····· 226 6.7.3 Windows 230 6.7.4 Datatypes 233 6.7.5 Error Class for Invalid Keyval 236 6.7.6 Attributes Example 236 6.8 Naming Objects 238 6.9 Formalizing the Loosely Synchronous Model.. 242 6.9.1 Basic Statements . 242 6.9.2 Models of Execution· 242 Static communicator allocation . 243 Dynamic communicator allocation.........····. 243 The General case..·························· 243 7 Process Topologies 245 7.1 Introduction..... 245 7.2 Virtual Topologies 246 7.3 Embedding in MPI 246 7.4 Overview of the Functions 246 7.5 Topology Constructors·..··.··········· 248 7.5.1 Cartesian Constructor ..... 248 7.5.2 Cartesian Convenience Function:MPI_DIMS_CREATE 248 7.5.3 General (Graph)Constructor ..................... 250 7.5.4 Distributed (Graph)Constructor ........... 252 7.5.5 Topology Inquiry Functions·············· 257 7.5.6 Cartesian Shift Coordinates .. 265 7.5.7 Partitioning of Cartesian structures.·...·.··.··. 266 7.5.8 Low-Level Topology Functions 267 viii6.3.2 Group Constructors . . . . . . . . . . . . . . . . . . . . . . . . . . . 193 6.3.3 Group Destructors . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198 6.4 Communicator Management . . . . . . . . . . . . . . . . . . . . . . . . . . . 199 6.4.1 Communicator Accessors . . . . . . . . . . . . . . . . . . . . . . . . 199 6.4.2 Communicator Constructors . . . . . . . . . . . . . . . . . . . . . . . 200 6.4.3 Communicator Destructors . . . . . . . . . . . . . . . . . . . . . . . 208 6.5 Motivating Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209 6.5.1 Current Practice #1 . . . . . . . . . . . . . . . . . . . . . . . . . . . 209 6.5.2 Current Practice #2 . . . . . . . . . . . . . . . . . . . . . . . . . . . 210 6.5.3 (Approximate) Current Practice #3 . . . . . . . . . . . . . . . . . . 210 6.5.4 Example #4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211 6.5.5 Library Example #1 . . . . . . . . . . . . . . . . . . . . . . . . . . . 212 6.5.6 Library Example #2 . . . . . . . . . . . . . . . . . . . . . . . . . . . 213 6.6 Inter-Communication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216 6.6.1 Inter-communicator Accessors . . . . . . . . . . . . . . . . . . . . . . 217 6.6.2 Inter-communicator Operations . . . . . . . . . . . . . . . . . . . . . 219 6.6.3 Inter-Communication Examples . . . . . . . . . . . . . . . . . . . . . 221 Example 1: Three-Group “Pipeline” . . . . . . . . . . . . . . . . . . 221 Example 2: Three-Group “Ring” . . . . . . . . . . . . . . . . . . . . 222 6.7 Caching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224 6.7.1 Functionality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225 6.7.2 Communicators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226 6.7.3 Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230 6.7.4 Datatypes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233 6.7.5 Error Class for Invalid Keyval . . . . . . . . . . . . . . . . . . . . . . 236 6.7.6 Attributes Example . . . . . . . . . . . . . . . . . . . . . . . . . . . 236 6.8 Naming Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238 6.9 Formalizing the Loosely Synchronous Model . . . . . . . . . . . . . . . . . . 242 6.9.1 Basic Statements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242 6.9.2 Models of Execution . . . . . . . . . . . . . . . . . . . . . . . . . . . 242 Static communicator allocation . . . . . . . . . . . . . . . . . . . . . 243 Dynamic communicator allocation . . . . . . . . . . . . . . . . . . . 243 The General case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243 7 Process Topologies 245 7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245 7.2 Virtual Topologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246 7.3 Embedding in MPI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246 7.4 Overview of the Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246 7.5 Topology Constructors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 248 7.5.1 Cartesian Constructor . . . . . . . . . . . . . . . . . . . . . . . . . . 248 7.5.2 Cartesian Convenience Function: MPI_DIMS_CREATE . . . . . . . . 248 7.5.3 General (Graph) Constructor . . . . . . . . . . . . . . . . . . . . . . 250 7.5.4 Distributed (Graph) Constructor . . . . . . . . . . . . . . . . . . . . 252 7.5.5 Topology Inquiry Functions . . . . . . . . . . . . . . . . . . . . . . . 257 7.5.6 Cartesian Shift Coordinates . . . . . . . . . . . . . . . . . . . . . . . 265 7.5.7 Partitioning of Cartesian structures . . . . . . . . . . . . . . . . . . 266 7.5.8 Low-Level Topology Functions . . . . . . . . . . . . . . . . . . . . . 267 viii