MPI:A Message-Passing Interface Standard Version 2.2 Message Passing Interface Forum September 4,2009
MPI: A Message-Passing Interface Standard Version 2.2 Message Passing Interface Forum September 4, 2009
Contents Acknowledgments viii 1 Introduction to MPl 1 1.1 Overview and Goals 1 1.2 Background of MPI-1.0........... 2 1.3 Background of MP-1.1,MP1.2,and MPI--2.0.·······.·······- 3 1.4 Background of MP-1.3 and MP-2.1....·················- 3 1.5 Background of MPl-2.2............................. 4 1.6 Who Should Use This Standard?.......... 4 1.7 What Platforms Are Targets For Implementation? 4 1.8 What Is Included In The Standard? 5 1.9 What Is Not Included In The Standard? 6 l.l0 Organization of this Document···.··.················· 6 2 MPI Terms and Conventions 9 2.1 Document Notation 9 2.2 Naming Conventions.·· 9 2.3 Procedure Specification 10 2.4 Semantic Terms.··. 11 2.5 Data Types 12 2.5.1 Opaque Objects 12 2.5.2 Array Arguments 14 2.5.3 State.......。·············“ 14 2.5.4 Named Constants 14 2.5.5 Choice 15 2.5.6 Addresses 。 15 2.5.7 File Offsets 。。。 16 2.6 Language Binding 16 2.6.1 Deprecated Names and Functions.... 16 2.6.2 Fortran Binding Issues..·..··. 17 2.6.3 C Binding Issues 18 2.6.4 C++Binding Issues 18 2.6.5 Functions and Macros 2.7 Processes 22 2.8 Error Handling..·············· 2.9 Implementation Issues... 223 2.9.1 Independence of Basic Runtime Routines............... 23
Contents Acknowledgments viii 1 Introduction to MPI 1 1.1 Overview and Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 Background of MPI-1.0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.3 Background of MPI-1.1, MPI-1.2, and MPI-2.0 . . . . . . . . . . . . . . . . . 3 1.4 Background of MPI-1.3 and MPI-2.1 . . . . . . . . . . . . . . . . . . . . . . 3 1.5 Background of MPI-2.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.6 Who Should Use This Standard? . . . . . . . . . . . . . . . . . . . . . . . . 4 1.7 What Platforms Are Targets For Implementation? . . . . . . . . . . . . . . 4 1.8 What Is Included In The Standard? . . . . . . . . . . . . . . . . . . . . . . 5 1.9 What Is Not Included In The Standard? . . . . . . . . . . . . . . . . . . . . 6 1.10 Organization of this Document . . . . . . . . . . . . . . . . . . . . . . . . . 6 2 MPI Terms and Conventions 9 2.1 Document Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2.2 Naming Conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2.3 Procedure Specification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 2.4 Semantic Terms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.5 Data Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 2.5.1 Opaque Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 2.5.2 Array Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 2.5.3 State . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 2.5.4 Named Constants . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 2.5.5 Choice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 2.5.6 Addresses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 2.5.7 File Offsets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 2.6 Language Binding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 2.6.1 Deprecated Names and Functions . . . . . . . . . . . . . . . . . . . . 16 2.6.2 Fortran Binding Issues . . . . . . . . . . . . . . . . . . . . . . . . . . 17 2.6.3 C Binding Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 2.6.4 C++ Binding Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 2.6.5 Functions and Macros . . . . . . . . . . . . . . . . . . . . . . . . . . 21 2.7 Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 2.8 Error Handling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 2.9 Implementation Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 2.9.1 Independence of Basic Runtime Routines . . . . . . . . . . . . . . . 23 v
2.9.2 nteraction with Signals.························ 24 2.l0 Examples··································· 24 3 Point-to-Point Communication 25 3.1 Introduction..·.........:......·..······· 25 3.2 Blocking Send and Receive Operations.............. 26 3.2.1 Blocking Send 26 3.2.2 Message Data.·.················ 27 3.2.3 Message Envelope 29 3.2.4 Blocking Receive 3 3.2.5 Return Status..·. 31 3.2.6 Passing MPI_STATUS_IGNORE for Status 3.3 Data Type Matching and Data Conversion 34 3.3.1 Type Matching Rules 34 Type MPI_CHARACTER........ 。 36 3.3.2 Data Conversion...·.···· 37 3.4 Communication Modes.. 38 3.5 Semantics of Point-to-Point Communication 42 3.6 Buffer Allocation and Usage... 45 3.6.1 Model Implementation of Buffered Mode... 47 3.7 Nonblocking Communication 48 3.7.1 Communication Request Objects ...... 西 3.7.2 Communication Initiation ......... 50 3.7.3 Communication Completion..·..··..···· 53 3.7.4 Semantics of Nonblocking Communications 56 3.7.5 Multiple Completions.................. 3.7.6 Non-destructive Test of status.....·.······, 。。。 64 3.8 Probe and Cancel... 3.9 Persistent Communication Requests 69 3.10 Send-Receive 73 3.11 Null Processes 75 4 Datatypes 77 4.1 Derived Datatypes ... 4.l.1 Type Constructors with Explicit Addresses·..··.··...... 79 4.l.2 Datatype Constructors.···················,····· 79 4.l.3 Subarray Datatype Constructor..····· 87 4.1.4 Distributed Array Datatype Constructor. 89 4.1.5 Address and Size Functions 94 4.1.6 Lower-Bound and Upper-Bound Markers 96 4.l.7 Extent and Bounds of Datatypes·..·.. 9 4.1.8 True Extent of Datatypes.......... 98 4.l.9 Commit and Free.···········- 99 4.1.10 Duplicating a Datatype.... 100 4.1.11 Use of General Datatypes in Communication 101 4.1.12 Correct Use of Addresses 104 4.l.l3 Decoding a Datatype...··················… 104 4.1.14 Examples......········ 112
2.9.2 Interaction with Signals . . . . . . . . . . . . . . . . . . . . . . . . . 24 2.10 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 3 Point-to-Point Communication 25 3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 3.2 Blocking Send and Receive Operations . . . . . . . . . . . . . . . . . . . . . 26 3.2.1 Blocking Send . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 3.2.2 Message Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 3.2.3 Message Envelope . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 3.2.4 Blocking Receive . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 3.2.5 Return Status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 3.2.6 Passing MPI_STATUS_IGNORE for Status . . . . . . . . . . . . . . . . 33 3.3 Data Type Matching and Data Conversion . . . . . . . . . . . . . . . . . . 34 3.3.1 Type Matching Rules . . . . . . . . . . . . . . . . . . . . . . . . . . 34 Type MPI_CHARACTER . . . . . . . . . . . . . . . . . . . . . . . . . . 36 3.3.2 Data Conversion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 3.4 Communication Modes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 3.5 Semantics of Point-to-Point Communication . . . . . . . . . . . . . . . . . . 42 3.6 Buffer Allocation and Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 3.6.1 Model Implementation of Buffered Mode . . . . . . . . . . . . . . . . 47 3.7 Nonblocking Communication . . . . . . . . . . . . . . . . . . . . . . . . . . 48 3.7.1 Communication Request Objects . . . . . . . . . . . . . . . . . . . . 49 3.7.2 Communication Initiation . . . . . . . . . . . . . . . . . . . . . . . . 50 3.7.3 Communication Completion . . . . . . . . . . . . . . . . . . . . . . . 53 3.7.4 Semantics of Nonblocking Communications . . . . . . . . . . . . . . 56 3.7.5 Multiple Completions . . . . . . . . . . . . . . . . . . . . . . . . . . 57 3.7.6 Non-destructive Test of status . . . . . . . . . . . . . . . . . . . . . . 64 3.8 Probe and Cancel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 3.9 Persistent Communication Requests . . . . . . . . . . . . . . . . . . . . . . 69 3.10 Send-Receive . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 3.11 Null Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 4 Datatypes 77 4.1 Derived Datatypes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 4.1.1 Type Constructors with Explicit Addresses . . . . . . . . . . . . . . 79 4.1.2 Datatype Constructors . . . . . . . . . . . . . . . . . . . . . . . . . . 79 4.1.3 Subarray Datatype Constructor . . . . . . . . . . . . . . . . . . . . . 87 4.1.4 Distributed Array Datatype Constructor . . . . . . . . . . . . . . . . 89 4.1.5 Address and Size Functions . . . . . . . . . . . . . . . . . . . . . . . 94 4.1.6 Lower-Bound and Upper-Bound Markers . . . . . . . . . . . . . . . 96 4.1.7 Extent and Bounds of Datatypes . . . . . . . . . . . . . . . . . . . . 97 4.1.8 True Extent of Datatypes . . . . . . . . . . . . . . . . . . . . . . . . 98 4.1.9 Commit and Free . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 4.1.10 Duplicating a Datatype . . . . . . . . . . . . . . . . . . . . . . . . . 100 4.1.11 Use of General Datatypes in Communication . . . . . . . . . . . . . 101 4.1.12 Correct Use of Addresses . . . . . . . . . . . . . . . . . . . . . . . . 104 4.1.13 Decoding a Datatype . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 4.1.14 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112 vi
4.2 Pack and Unpack.. 121 4.3 Canonical MPI_PACK and MPI_UNPACK 127 5 Collective Communication 131 5.1 Introduction and Overview 。。。。 131 5.2 Communicator Argument.....·...·.··.······· 134 5.2.1 Specifics for Intracommunicator Collective Operations . 134 5.2.2 Applying Collective Operations to Intercommunicators... 134 5.2.3 Specifics for Intercommunicator Collective Operations 137 5.3 Barrier Synchronization..····················· 137 5.4 Broadcast·. 138 5.4.1 Example using MPI_BCAST.... 138 5.5 Gather.. 4 139 5.5.1 Examples using MPI_GATHER,MPI_GATHERV 142 5.6 Scatter..·..·· 149 5.6.1 Examples using MPI_SCATTER,MPI_SCATTERV 152 5.7 Gather-to-all 154 5.7.1 Exampleusing MPI_ALLGATHER 156 5.8 All-to-All Scatter/Gather...... 157 5.9 Global Reduction Operations ..... 162 5.9.1 Reduce·············· 163 5.9.2 Predefined Reduction Operations 164 5.9.3 Signed Characters and Reductions 167 5.9.4 MINLOC and MAXLOC..···· 167 5.9.5 User-Defined Reduction Operations 171 Example of User-defined Reduce 174 5.9.6 All-Reduce 175 5.9.7 Process-local reduction.······ 176 5.10 Reduce-Scatter .. 177 5.10.1 MPI_REDUCE_SCATTER_BLOCK 178 5.10.2 MPI_REDUCE_SCATTER.. 179 5.11Scan.. 180 5.11.1 Inclusive Scan........ 180 5.11.2 Exclusive Scan 181 5.11.3 Example using MPI_SCAN... 182 5.12 Correctness................. 183 6 Groups,Contexts,Communicators,and Caching 187 6.1 Introduction..··.·················· 187 6.1.1 Features Needed to Support Libraries 187 6.1.2 MPI's Support for Libraries 188 6.2 Basic Concepts 190 6.2.1 Groups 190 6.2.2 Contexts 190 6.2.3 Intra-Communicators.. 191 6.2.4 Predefined Intra-Communicators·.··········· 191 6.3 Group Management.························ 192 6.3.1 Group Accessors.·········· 192 i
4.2 Pack and Unpack . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121 4.3 Canonical MPI_PACK and MPI_UNPACK . . . . . . . . . . . . . . . . . . . 127 5 Collective Communication 131 5.1 Introduction and Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . 131 5.2 Communicator Argument . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134 5.2.1 Specifics for Intracommunicator Collective Operations . . . . . . . . 134 5.2.2 Applying Collective Operations to Intercommunicators . . . . . . . . 134 5.2.3 Specifics for Intercommunicator Collective Operations . . . . . . . . 137 5.3 Barrier Synchronization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137 5.4 Broadcast . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138 5.4.1 Example using MPI_BCAST . . . . . . . . . . . . . . . . . . . . . . . 138 5.5 Gather . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139 5.5.1 Examples using MPI_GATHER, MPI_GATHERV . . . . . . . . . . . . 142 5.6 Scatter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149 5.6.1 Examples using MPI_SCATTER, MPI_SCATTERV . . . . . . . . . . 152 5.7 Gather-to-all . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154 5.7.1 Exampleusing MPI_ALLGATHER . . . . . . . . . . . . . . . . . . . . 156 5.8 All-to-All Scatter/Gather . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157 5.9 Global Reduction Operations . . . . . . . . . . . . . . . . . . . . . . . . . . 162 5.9.1 Reduce . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163 5.9.2 Predefined Reduction Operations . . . . . . . . . . . . . . . . . . . . 164 5.9.3 Signed Characters and Reductions . . . . . . . . . . . . . . . . . . . 167 5.9.4 MINLOC and MAXLOC . . . . . . . . . . . . . . . . . . . . . . . . 167 5.9.5 User-Defined Reduction Operations . . . . . . . . . . . . . . . . . . 171 Example of User-defined Reduce . . . . . . . . . . . . . . . . . . . . 174 5.9.6 All-Reduce . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175 5.9.7 Process-local reduction . . . . . . . . . . . . . . . . . . . . . . . . . . 176 5.10 Reduce-Scatter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177 5.10.1 MPI_REDUCE_SCATTER_BLOCK . . . . . . . . . . . . . . . . . . . 178 5.10.2 MPI_REDUCE_SCATTER . . . . . . . . . . . . . . . . . . . . . . . . 179 5.11 Scan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180 5.11.1 Inclusive Scan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180 5.11.2 Exclusive Scan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181 5.11.3 Example using MPI_SCAN . . . . . . . . . . . . . . . . . . . . . . . . 182 5.12 Correctness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183 6 Groups, Contexts, Communicators, and Caching 187 6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187 6.1.1 Features Needed to Support Libraries . . . . . . . . . . . . . . . . . 187 6.1.2 MPI’s Support for Libraries . . . . . . . . . . . . . . . . . . . . . . . 188 6.2 Basic Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190 6.2.1 Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190 6.2.2 Contexts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190 6.2.3 Intra-Communicators . . . . . . . . . . . . . . . . . . . . . . . . . . 191 6.2.4 Predefined Intra-Communicators . . . . . . . . . . . . . . . . . . . . 191 6.3 Group Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192 6.3.1 Group Accessors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192 vii
6.3.2 Group Constructors······················· 193 6.3.3 Group Destructors······················· 198 6.4 Communicator Management·············· 199 6.4.1 Communicator Accessors 199 6.4.2 Communicator Constructors.... 200 6.4.3 Communicator Destructors 208 6.5 Motivating Examples......... 209 6.5.1 Current Practice #1 209 6.5.2 Current Practice #2... 210 6.5.3 (Approximate)Current Practice #3 210 6.5.4 Example#4.... 211 6.5.5 Library Example #1 212 6.5.6 Library Example #2 213 6.6 Inter-Communication...... 216 6.6.1 Inter-communicator Accessors... 217 6.6.2 Inter-communicator Operations 219 6.6.3 Inter-Communication Examples .. 221 Example 1:Three-Group“Pipeline'” 221 Example2:Three-Group“Ring”... 222 6.7 Caching. 224 6.7.1 Functionalitv 225 6.7.2 Communicators..····· 226 6.7.3 Windows 230 6.7.4 Datatypes 233 6.7.5 Error Class for Invalid Keyval 236 6.7.6 Attributes Example 236 6.8 Naming Objects 238 6.9 Formalizing the Loosely Synchronous Model.. 242 6.9.1 Basic Statements . 242 6.9.2 Models of Execution· 242 Static communicator allocation . 243 Dynamic communicator allocation.........····. 243 The General case..·························· 243 7 Process Topologies 245 7.1 Introduction..... 245 7.2 Virtual Topologies 246 7.3 Embedding in MPI 246 7.4 Overview of the Functions 246 7.5 Topology Constructors·..··.··········· 248 7.5.1 Cartesian Constructor ..... 248 7.5.2 Cartesian Convenience Function:MPI_DIMS_CREATE 248 7.5.3 General (Graph)Constructor ..................... 250 7.5.4 Distributed (Graph)Constructor ........... 252 7.5.5 Topology Inquiry Functions·············· 257 7.5.6 Cartesian Shift Coordinates .. 265 7.5.7 Partitioning of Cartesian structures.·...·.··.··. 266 7.5.8 Low-Level Topology Functions 267 viii
6.3.2 Group Constructors . . . . . . . . . . . . . . . . . . . . . . . . . . . 193 6.3.3 Group Destructors . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198 6.4 Communicator Management . . . . . . . . . . . . . . . . . . . . . . . . . . . 199 6.4.1 Communicator Accessors . . . . . . . . . . . . . . . . . . . . . . . . 199 6.4.2 Communicator Constructors . . . . . . . . . . . . . . . . . . . . . . . 200 6.4.3 Communicator Destructors . . . . . . . . . . . . . . . . . . . . . . . 208 6.5 Motivating Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209 6.5.1 Current Practice #1 . . . . . . . . . . . . . . . . . . . . . . . . . . . 209 6.5.2 Current Practice #2 . . . . . . . . . . . . . . . . . . . . . . . . . . . 210 6.5.3 (Approximate) Current Practice #3 . . . . . . . . . . . . . . . . . . 210 6.5.4 Example #4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211 6.5.5 Library Example #1 . . . . . . . . . . . . . . . . . . . . . . . . . . . 212 6.5.6 Library Example #2 . . . . . . . . . . . . . . . . . . . . . . . . . . . 213 6.6 Inter-Communication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216 6.6.1 Inter-communicator Accessors . . . . . . . . . . . . . . . . . . . . . . 217 6.6.2 Inter-communicator Operations . . . . . . . . . . . . . . . . . . . . . 219 6.6.3 Inter-Communication Examples . . . . . . . . . . . . . . . . . . . . . 221 Example 1: Three-Group “Pipeline” . . . . . . . . . . . . . . . . . . 221 Example 2: Three-Group “Ring” . . . . . . . . . . . . . . . . . . . . 222 6.7 Caching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224 6.7.1 Functionality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225 6.7.2 Communicators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226 6.7.3 Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230 6.7.4 Datatypes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233 6.7.5 Error Class for Invalid Keyval . . . . . . . . . . . . . . . . . . . . . . 236 6.7.6 Attributes Example . . . . . . . . . . . . . . . . . . . . . . . . . . . 236 6.8 Naming Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238 6.9 Formalizing the Loosely Synchronous Model . . . . . . . . . . . . . . . . . . 242 6.9.1 Basic Statements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242 6.9.2 Models of Execution . . . . . . . . . . . . . . . . . . . . . . . . . . . 242 Static communicator allocation . . . . . . . . . . . . . . . . . . . . . 243 Dynamic communicator allocation . . . . . . . . . . . . . . . . . . . 243 The General case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243 7 Process Topologies 245 7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245 7.2 Virtual Topologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246 7.3 Embedding in MPI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246 7.4 Overview of the Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246 7.5 Topology Constructors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 248 7.5.1 Cartesian Constructor . . . . . . . . . . . . . . . . . . . . . . . . . . 248 7.5.2 Cartesian Convenience Function: MPI_DIMS_CREATE . . . . . . . . 248 7.5.3 General (Graph) Constructor . . . . . . . . . . . . . . . . . . . . . . 250 7.5.4 Distributed (Graph) Constructor . . . . . . . . . . . . . . . . . . . . 252 7.5.5 Topology Inquiry Functions . . . . . . . . . . . . . . . . . . . . . . . 257 7.5.6 Cartesian Shift Coordinates . . . . . . . . . . . . . . . . . . . . . . . 265 7.5.7 Partitioning of Cartesian structures . . . . . . . . . . . . . . . . . . 266 7.5.8 Low-Level Topology Functions . . . . . . . . . . . . . . . . . . . . . 267 viii
7.6 An Application Example..····· 268 8 MPI Environmental Management 271 8.1 Implementation Information.... 271 8.1.1 Version Inquiries 271 8.l.2 Environmental Inquiries..·.·.·.. 272 Tag Values 272 Host Rank 272 IO Rank..... 273 Clock Synchronization.·················· 273 8.2 Memory Allocation.·············· 274 8.3 Error Handling 276 8.3.1 Error Handlers for Communicators... 278 8.3.2 Error Handlers for Windows.......... 279 8.3.3 Error Handlers for Files.......... 281 8.3.4 Freeing Errorhandlers and Retrieving Error Strings... 282 8.4 Error Codes and Classes... 283 8.5 Error Classes,Error Codes,and Error Handlers..... 285 8.6 Timers and Synchronization...··.··..·. 289 8.7 290 8.7.1 Allowing User Functions at Process Termination..... 295 8.7.2 Determining Whether MPI Has Finished.......... 296 8.8 Portable MPI Process Startup......·..··.······. 296 9 The Info Object 299 10 Process Creation and Management 305 l0.1 Introduction.·············· 305 10.2 The Dynamic Process Model 306 10.2.1 Starting Processes 306 10.2.2 The Runtime Environment 306 10.3 Process Manager Interface....... 308 10.3.1 Processes in MPI... 308 10.3.2 Starting Processes and Establishing Communication 308 10.3.3 Starting Multiple Executables and Establishing Communication . 313 10.3.4 Reserved Keys.........··.······.·····.···· 315 10.3.5 Spawn Example...... 316 Manager-worker Example,Using MPI_COMM_SPAWN..······· 316 10.4 Establishing Communication .... 318 l0.4.1 Names,Addresses,.Ports,.and All That.··..··..·. 318 10.4.2 Server Routines..····················· 320 10.4.3 Client Routines... 322 10.4.4 Name Publishing . 323 l0.4.5 Reserved Key Values.··········· 325 l0.4.6 Client/Server Examples·······.·· 325 Simplest Example-Completely Portable. 325 Ocean/Atmosphere-Relies on Name Publishing 326 Simple Client-Server Example.,·.··,······· 326 ix
7.6 An Application Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268 8 MPI Environmental Management 271 8.1 Implementation Information . . . . . . . . . . . . . . . . . . . . . . . . . . . 271 8.1.1 Version Inquiries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271 8.1.2 Environmental Inquiries . . . . . . . . . . . . . . . . . . . . . . . . . 272 Tag Values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272 Host Rank . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272 IO Rank . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273 Clock Synchronization . . . . . . . . . . . . . . . . . . . . . . . . . . 273 8.2 Memory Allocation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274 8.3 Error Handling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276 8.3.1 Error Handlers for Communicators . . . . . . . . . . . . . . . . . . . 278 8.3.2 Error Handlers for Windows . . . . . . . . . . . . . . . . . . . . . . . 279 8.3.3 Error Handlers for Files . . . . . . . . . . . . . . . . . . . . . . . . . 281 8.3.4 Freeing Errorhandlers and Retrieving Error Strings . . . . . . . . . . 282 8.4 Error Codes and Classes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283 8.5 Error Classes, Error Codes, and Error Handlers . . . . . . . . . . . . . . . . 285 8.6 Timers and Synchronization . . . . . . . . . . . . . . . . . . . . . . . . . . . 289 8.7 Startup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 290 8.7.1 Allowing User Functions at Process Termination . . . . . . . . . . . 295 8.7.2 Determining Whether MPI Has Finished . . . . . . . . . . . . . . . . 296 8.8 Portable MPI Process Startup . . . . . . . . . . . . . . . . . . . . . . . . . . 296 9 The Info Object 299 10 Process Creation and Management 305 10.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305 10.2 The Dynamic Process Model . . . . . . . . . . . . . . . . . . . . . . . . . . 306 10.2.1 Starting Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 306 10.2.2 The Runtime Environment . . . . . . . . . . . . . . . . . . . . . . . 306 10.3 Process Manager Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . 308 10.3.1 Processes in MPI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 308 10.3.2 Starting Processes and Establishing Communication . . . . . . . . . 308 10.3.3 Starting Multiple Executables and Establishing Communication . . 313 10.3.4 Reserved Keys . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315 10.3.5 Spawn Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 316 Manager-worker Example, Using MPI_COMM_SPAWN. . . . . . . . 316 10.4 Establishing Communication . . . . . . . . . . . . . . . . . . . . . . . . . . 318 10.4.1 Names, Addresses, Ports, and All That . . . . . . . . . . . . . . . . 318 10.4.2 Server Routines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 320 10.4.3 Client Routines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 322 10.4.4 Name Publishing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323 10.4.5 Reserved Key Values . . . . . . . . . . . . . . . . . . . . . . . . . . . 325 10.4.6 Client/Server Examples . . . . . . . . . . . . . . . . . . . . . . . . . 325 Simplest Example — Completely Portable. . . . . . . . . . . . . . . 325 Ocean/Atmosphere - Relies on Name Publishing . . . . . . . . . . . 326 Simple Client-Server Example. . . . . . . . . . . . . . . . . . . . . . 326 ix
10.5 Other Functionality 328 10.5.1 Universe Size .. 328 10.5.2 Singleton MPI_INIT 329 10.5.3MPI_APPNUM.·..··. 329 10.5.4 Releasing Connections 330 10.5.5 Another Way to Establish MPI Communication.···.······. 331 11 One-Sided Communications 335 11.1 ntroduction..··..·.··· 335 1l.2 Initialization....··.····· 336 11.2.1 Window Creation. 336 1l.2.2 Window Attributes.·.....··... 338 11.3 Communication Calls 339 11.3.1Put 340 11.3.2Get 342 11.3.3 Examples ... 342 11.3.4 Accumulate Functions.. 345 11.4 Synchronization Calls 347 11.4.1 Fence... 352 11.4.2 General Active Target Synchronization.. 353 11.4.3L0ck·················…··· 357 11.4.4 Assertions 359 1l.4.5 Miscellaneous Clarifications..·......·..· 360 11.5 Examples 360 l1.6 Error Handling···· 363 11.6.1 Error Handlers 363 11.6.2 Error Classes ... 363 ll.7 Semantics and Correctness.···················· 363 11.7.1 Atomicity 369 11.7.2 Progress 369 11.7.3 Registers and Compiler Optimizations 371 12 External Interfaces 373 12.1 Introduction.... 373 12.2 Generalized Requests....... 373 12.2.1 Examples 377 12.3 Associating Information with Status 379 12.4 MPI and Threads 381 12.4.1 General. 381 12.4.2 Clarifications 382 12.4.3 Initialization 。 384 131/0 389 13.1 ntroduction.···. 389 13.1.1 Definitions 389 13.2 File Manipulation 391 13.2.1 Opening a File 391 13.2.2 Closing a File.. 393 X
10.5 Other Functionality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 328 10.5.1 Universe Size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 328 10.5.2 Singleton MPI_INIT . . . . . . . . . . . . . . . . . . . . . . . . . . . 329 10.5.3 MPI_APPNUM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 329 10.5.4 Releasing Connections . . . . . . . . . . . . . . . . . . . . . . . . . . 330 10.5.5 Another Way to Establish MPI Communication . . . . . . . . . . . . 331 11 One-Sided Communications 335 11.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335 11.2 Initialization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 336 11.2.1 Window Creation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 336 11.2.2 Window Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 338 11.3 Communication Calls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 339 11.3.1 Put . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 340 11.3.2 Get . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 342 11.3.3 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 342 11.3.4 Accumulate Functions . . . . . . . . . . . . . . . . . . . . . . . . . . 345 11.4 Synchronization Calls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 347 11.4.1 Fence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 352 11.4.2 General Active Target Synchronization . . . . . . . . . . . . . . . . . 353 11.4.3 Lock . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 357 11.4.4 Assertions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 359 11.4.5 Miscellaneous Clarifications . . . . . . . . . . . . . . . . . . . . . . . 360 11.5 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 360 11.6 Error Handling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 363 11.6.1 Error Handlers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 363 11.6.2 Error Classes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 363 11.7 Semantics and Correctness . . . . . . . . . . . . . . . . . . . . . . . . . . . . 363 11.7.1 Atomicity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 369 11.7.2 Progress . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 369 11.7.3 Registers and Compiler Optimizations . . . . . . . . . . . . . . . . . 371 12 External Interfaces 373 12.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373 12.2 Generalized Requests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373 12.2.1 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 377 12.3 Associating Information with Status . . . . . . . . . . . . . . . . . . . . . . 379 12.4 MPI and Threads . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 381 12.4.1 General . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 381 12.4.2 Clarifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 382 12.4.3 Initialization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384 13 I/O 389 13.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 389 13.1.1 Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 389 13.2 File Manipulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 391 13.2.1 Opening a File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 391 13.2.2 Closing a File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 393 x
l3.2.3 Deleting a File.········.············· 394 l3.2.4 Resizing a File.······· 395 13.2.5 Preallocating Space for a File. 395 13.2.6 Querying the Size of a File 396 13.2.7 Querying File Parameters 397 13.2.8 File Info 398 Reserved File Hints 399 l3.3 File Views......。·.·.·. 401 l3.4 Data Access···: 404 13.4.1 Data Access Routines 。。 404 Positioning 405 Synchronism········· 405 Coordination..·.····· 405 Data Access Conventions 406 13.4.2 Data Access with Explicit Offsets... 407 13.4.3 Data Access with Individual File Pointers 410 l3.4.4 Data Access with Shared File Pointers.·.·····.. 416 Noncollective Operations 417 Collective Operations.....·..·. 419 Seek..· 420 13.4.5 Split Collective Data Access Routines 421 l3.5 File Interoperability....·..········ 428 13.5.1 Datatypes for File Interoperability 430 13.5.2 External Data Representation:"external32' 431 13.5.3 User-Defined Data Representations..... 432 Extent Callback 434 Datarep Conversion Functions 435 13.5.4 Matching Data Representations.. 437 13.6 Consistency and Semantics 437 13.6.1 File Consistency···· 437 13.6.2 Random Access vs.Sequential Files 440 13.6.3Pr0gss.··。·.····················· 441 l3.6.4 Collective File Operations..··..··..········ 441 l3.6.5 Type Matching...................·... 441 13.6.6 Miscellaneous Clarifications................ 441 13.6.7 MPl_Offset Type......... 。 442 l3.6.8 Logical vs.Physical File Layout.············ 442 13.6.9 File Size................··.·· 442 13.6.10 Examples .. 443 Asynchronous I/O 445 l3.7I/0 Error Handling..·..······ 447 l3.8I/O Error Classes......·..·· 447 13.9 Examples.... 448 13.9.1 Double Buffering with Split Collective I/O 448 l3.9.2 Subarray Filetype Constructor··..·.. 450 xi
13.2.3 Deleting a File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 394 13.2.4 Resizing a File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 395 13.2.5 Preallocating Space for a File . . . . . . . . . . . . . . . . . . . . . . 395 13.2.6 Querying the Size of a File . . . . . . . . . . . . . . . . . . . . . . . 396 13.2.7 Querying File Parameters . . . . . . . . . . . . . . . . . . . . . . . . 397 13.2.8 File Info . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 398 Reserved File Hints . . . . . . . . . . . . . . . . . . . . . . . . . . . 399 13.3 File Views . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 401 13.4 Data Access . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 404 13.4.1 Data Access Routines . . . . . . . . . . . . . . . . . . . . . . . . . . 404 Positioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 405 Synchronism . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 405 Coordination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 405 Data Access Conventions . . . . . . . . . . . . . . . . . . . . . . . . 406 13.4.2 Data Access with Explicit Offsets . . . . . . . . . . . . . . . . . . . . 407 13.4.3 Data Access with Individual File Pointers . . . . . . . . . . . . . . . 410 13.4.4 Data Access with Shared File Pointers . . . . . . . . . . . . . . . . . 416 Noncollective Operations . . . . . . . . . . . . . . . . . . . . . . . . 417 Collective Operations . . . . . . . . . . . . . . . . . . . . . . . . . . 419 Seek . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 420 13.4.5 Split Collective Data Access Routines . . . . . . . . . . . . . . . . . 421 13.5 File Interoperability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 428 13.5.1 Datatypes for File Interoperability . . . . . . . . . . . . . . . . . . . 430 13.5.2 External Data Representation: “external32” . . . . . . . . . . . . . . 431 13.5.3 User-Defined Data Representations . . . . . . . . . . . . . . . . . . . 432 Extent Callback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 434 Datarep Conversion Functions . . . . . . . . . . . . . . . . . . . . . 435 13.5.4 Matching Data Representations . . . . . . . . . . . . . . . . . . . . . 437 13.6 Consistency and Semantics . . . . . . . . . . . . . . . . . . . . . . . . . . . 437 13.6.1 File Consistency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 437 13.6.2 Random Access vs. Sequential Files . . . . . . . . . . . . . . . . . . 440 13.6.3 Progress . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 441 13.6.4 Collective File Operations . . . . . . . . . . . . . . . . . . . . . . . . 441 13.6.5 Type Matching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 441 13.6.6 Miscellaneous Clarifications . . . . . . . . . . . . . . . . . . . . . . . 441 13.6.7 MPI_Offset Type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 442 13.6.8 Logical vs. Physical File Layout . . . . . . . . . . . . . . . . . . . . . 442 13.6.9 File Size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 442 13.6.10 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 443 Asynchronous I/O . . . . . . . . . . . . . . . . . . . . . . . . . . . . 445 13.7 I/O Error Handling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 447 13.8 I/O Error Classes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 447 13.9 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 448 13.9.1 Double Buffering with Split Collective I/O . . . . . . . . . . . . . . 448 13.9.2 Subarray Filetype Constructor . . . . . . . . . . . . . . . . . . . . . 450 xi
14 Profiling Interface 453 14.1 Requirements··············· 453 14.2 Discussion....·.······················ 453 14.3 Logic of the Design..·.···. 454 14.3.1 Miscellaneous Control of Profiling 454 l4.4 Examples.·.·.······················ 455 14.4.1 Profiler Implementation...... 455 14.4.2 MPI Library Implementation . 。 456 Systems with Weak Symbols . 456 Systems Without Weak Symbols 456 14.4.3 Complications......····· 457 Multiple Counting········· 457 Linker Oddities..···· 457 14.5 Multiple Levels of Interception .. 458 15 Deprecated Functions 459 15.1 Deprecated since MPl-2.0 459 15.2 Deprecated since MPI-2.2 465 16 Language Bindings 467 16.1C++......·.。.. 467 16.1.1 Overview 467 16.1.2 Design.···· 467 16.1.3 C++Classes for MPI 。。 468 16.1.4 Class Member Functions for MPI.. 468 16.l.5 Semantics.········· 469 16.1.6 C++Datatypes 471 16.1.7 Communicators........ 474 16.1.8 Exceptions.·········· 476 16.1.9 Mixed-Language Operability 477 16.1.10 Profiling........·.··· 477 16.2 Fortran Support 480 16.2.1 Overview 480 l6.2.2 Problems With Fortran Bindings for MPI.·...·.·· 481 Problems Due to Strong Typing 482 Problems Due to Data Copying and Sequence Association,····· 482 Special Constants···························· 484 Fortran 90 Derived Types... 484 A Problem with Register Optimization...······ 485 16.2.3 Basic Fortran Support.·················· 487 l6.2.4 Extended Fortran Support.·..·..·..··.······ 488 The mpi Module.......,··..··.····: 488 No Type Mismatch Problems for Subroutines with Choice Arguments 489 16.2.5 Additional Support for Fortran Numeric Intrinsic Types .. 489 Parameterized Datatypes with Specified Precision and Exponent Range490 Support for Size-specific MPI Datatypes.··············· 494 Communication With Size-specific Types 496 l6.3 Language Interoperability.......,·.·。.···.···. 497 xii
14 Profiling Interface 453 14.1 Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 453 14.2 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 453 14.3 Logic of the Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 454 14.3.1 Miscellaneous Control of Profiling . . . . . . . . . . . . . . . . . . . 454 14.4 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 455 14.4.1 Profiler Implementation . . . . . . . . . . . . . . . . . . . . . . . . . 455 14.4.2 MPI Library Implementation . . . . . . . . . . . . . . . . . . . . . . 456 Systems with Weak Symbols . . . . . . . . . . . . . . . . . . . . . . 456 Systems Without Weak Symbols . . . . . . . . . . . . . . . . . . . . 456 14.4.3 Complications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 457 Multiple Counting . . . . . . . . . . . . . . . . . . . . . . . . . . . . 457 Linker Oddities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 457 14.5 Multiple Levels of Interception . . . . . . . . . . . . . . . . . . . . . . . . . 458 15 Deprecated Functions 459 15.1 Deprecated since MPI-2.0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 459 15.2 Deprecated since MPI-2.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 465 16 Language Bindings 467 16.1 C++ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 467 16.1.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 467 16.1.2 Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 467 16.1.3 C++ Classes for MPI . . . . . . . . . . . . . . . . . . . . . . . . . . 468 16.1.4 Class Member Functions for MPI . . . . . . . . . . . . . . . . . . . . 468 16.1.5 Semantics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 469 16.1.6 C++ Datatypes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 471 16.1.7 Communicators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 474 16.1.8 Exceptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 476 16.1.9 Mixed-Language Operability . . . . . . . . . . . . . . . . . . . . . . 477 16.1.10 Profiling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 477 16.2 Fortran Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 480 16.2.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 480 16.2.2 Problems With Fortran Bindings for MPI . . . . . . . . . . . . . . . 481 Problems Due to Strong Typing . . . . . . . . . . . . . . . . . . . . 482 Problems Due to Data Copying and Sequence Association . . . . . . 482 Special Constants . . . . . . . . . . . . . . . . . . . . . . . . . . . . 484 Fortran 90 Derived Types . . . . . . . . . . . . . . . . . . . . . . . . 484 A Problem with Register Optimization . . . . . . . . . . . . . . . . . 485 16.2.3 Basic Fortran Support . . . . . . . . . . . . . . . . . . . . . . . . . . 487 16.2.4 Extended Fortran Support . . . . . . . . . . . . . . . . . . . . . . . . 488 The mpi Module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 488 No Type Mismatch Problems for Subroutines with Choice Arguments 489 16.2.5 Additional Support for Fortran Numeric Intrinsic Types . . . . . . . 489 Parameterized Datatypes with Specified Precision and Exponent Range490 Support for Size-specific MPI Datatypes . . . . . . . . . . . . . . . . 494 Communication With Size-specific Types . . . . . . . . . . . . . . . 496 16.3 Language Interoperability . . . . . . . . . . . . . . . . . . . . . . . . . . . . 497 xii
16.3.1 Introduction 497 16.3.2 Assumptions 498 16.3.3 Initialization 498 16.3.4 Transfer of Handles 498 16.3.5 Status 502 16.3.6 MPI Opaque Objects 502 Datatypes······· 503 Callback Functions ... 504 Error Handlers ... 504 Reduce Operations.. 504 Addresses 504 16.3.7 Attributes.······ 505 16.3.8 Extra State 509 16.3.9 Constants 509 l6.3.l0 nterlanguage Communication..·..·..······. 510 A Language Bindings Summary 513 A.1 Defined Values and Handles .. 513 A.l.1 Defined Constants....·· 513 A1.2 Types.············· 524 A.l.3 Prototype definitions..····· 525 A.1.4 Deprecated prototype definitions 528 A.1.5 nfo Keys.........····· 529 A.1.6 Info Values 529 A.2 C Bindings·.····· 531 A.2.1 Point-to-Point Communication C Bindings 531 A.2.2 Datatypes C Bindings.................... 532 A.2.3 Collective Communication C Bindings 534 A.2.4 Groups,Contexts,Communicators,and Caching C Bindings.... 535 A.2.5 Process Topologies C Bindings 538 A.2.6 MPI Environmenta Management C Bindings 539 A.2.7 The Info Object C Bindings..... 540 A.2.8 Process Creation and Management C Bindings 540 A.2.9 One-Sided Communications C Bindings 。 541 A.2.10 External Interfaces C Bindings 541 A.2.11/0 C Bindings..........·..··.··.···· 542 A.2.12 Language Bindings C Bindings 544 A.2.l3 Profiling Interface C Bindings.····· 545 A.2.14 Deprecated C Bindings 545 A.3 Fortran Bindings.........··.···.····· 547 A.3.1 Point-to-Point Communication Fortran Bindings··········· 547 A.3.2 Datatypes Fortran Bindings······················· 549 A.3.3 Collective Communication Fortran Bindings.............. 552 A.3.4 Groups,Contexts,Communicators,and Caching Fortran Bindings 554 A.3.5 Process Topologies Fortran Bindings.················· 558 A.3.6 MPI Environmenta Management Fortran Bindings·..··.···· 559 A.3.7 The Info Object Fortran Bindings ..... 561 A.3.8 Process Creation and Management Fortran Bindings····· 562 xiii
16.3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 497 16.3.2 Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 498 16.3.3 Initialization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 498 16.3.4 Transfer of Handles . . . . . . . . . . . . . . . . . . . . . . . . . . . 498 16.3.5 Status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 502 16.3.6 MPI Opaque Objects . . . . . . . . . . . . . . . . . . . . . . . . . . 502 Datatypes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 503 Callback Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 504 Error Handlers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 504 Reduce Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 504 Addresses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 504 16.3.7 Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 505 16.3.8 Extra State . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 509 16.3.9 Constants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 509 16.3.10 Interlanguage Communication . . . . . . . . . . . . . . . . . . . . . . 510 A Language Bindings Summary 513 A.1 Defined Values and Handles . . . . . . . . . . . . . . . . . . . . . . . . . . . 513 A.1.1 Defined Constants . . . . . . . . . . . . . . . . . . . . . . . . . . . . 513 A.1.2 Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 524 A.1.3 Prototype definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . 525 A.1.4 Deprecated prototype definitions . . . . . . . . . . . . . . . . . . . . 528 A.1.5 Info Keys . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 529 A.1.6 Info Values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 529 A.2 C Bindings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 531 A.2.1 Point-to-Point Communication C Bindings . . . . . . . . . . . . . . 531 A.2.2 Datatypes C Bindings . . . . . . . . . . . . . . . . . . . . . . . . . . 532 A.2.3 Collective Communication C Bindings . . . . . . . . . . . . . . . . . 534 A.2.4 Groups, Contexts, Communicators, and Caching C Bindings . . . . 535 A.2.5 Process Topologies C Bindings . . . . . . . . . . . . . . . . . . . . . 538 A.2.6 MPI Environmenta Management C Bindings . . . . . . . . . . . . . 539 A.2.7 The Info Object C Bindings . . . . . . . . . . . . . . . . . . . . . . . 540 A.2.8 Process Creation and Management C Bindings . . . . . . . . . . . . 540 A.2.9 One-Sided Communications C Bindings . . . . . . . . . . . . . . . . 541 A.2.10 External Interfaces C Bindings . . . . . . . . . . . . . . . . . . . . . 541 A.2.11 I/O C Bindings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 542 A.2.12 Language Bindings C Bindings . . . . . . . . . . . . . . . . . . . . . 544 A.2.13 Profiling Interface C Bindings . . . . . . . . . . . . . . . . . . . . . . 545 A.2.14 Deprecated C Bindings . . . . . . . . . . . . . . . . . . . . . . . . . 545 A.3 Fortran Bindings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 547 A.3.1 Point-to-Point Communication Fortran Bindings . . . . . . . . . . . 547 A.3.2 Datatypes Fortran Bindings . . . . . . . . . . . . . . . . . . . . . . . 549 A.3.3 Collective Communication Fortran Bindings . . . . . . . . . . . . . . 552 A.3.4 Groups, Contexts, Communicators, and Caching Fortran Bindings . 554 A.3.5 Process Topologies Fortran Bindings . . . . . . . . . . . . . . . . . . 558 A.3.6 MPI Environmenta Management Fortran Bindings . . . . . . . . . . 559 A.3.7 The Info Object Fortran Bindings . . . . . . . . . . . . . . . . . . . 561 A.3.8 Process Creation and Management Fortran Bindings . . . . . . . . . 562 xiii
