2. Regular MPI_PROBE Checks to see if a matching message has arrived Tag: 3 Source: 14 Comm. ID: 32 Incoming message queue Time Tag: 9 Source: 94 Comm. ID: 17 Tag: 9 Source: 67 Comm. ID: 17
3. Regular MPI_PROBE Checks to see if a matching message has arrived MPI_PROBE looking for: Tag 9, ANY_SOURCE, COMM ID 17 Tag: 3 Source: 14 Comm. ID: 32 Incoming message queue Time Tag: 9 Source: 94 Comm. ID: 17 Tag: 9 Source: 67 Comm. ID: 17
4. Regular MPI_PROBE Checks to see if a matching message has arrived MPI_PROBE looking for: Tag 9, ANY_SOURCE, COMM ID 17 Tag: 3 Source: 14 Comm. ID: 32 Incoming message queue Time Tag: 9 Source: 94 Comm. ID: 17 Match Tag: 9 Source: 67 Comm. ID: 17
5. MPI_PROBE Succeeded Now issue a receive to actually get the message MPI_RECV(…, tag=9, src=94,comm=17, …) Tag: 3 Source: 14 Comm. ID: 32 Incoming message queue Time Tag: 9 Source: 94 Comm. ID: 17 Message is removed from incoming queue Tag: 9 Source: 67 Comm. ID: 17
6. Race Condition …but what if another MPI thread issues the receive first? Tag: 3 Source: 14 Comm. ID: 32 Incoming message queue Time Tag: 9 Source: 94 Comm. ID: 17 MPI_RECV(…, tag=9, src=ANY_SOURCE,comm=17) Tag: 9 Source: 67 Comm. ID: 17
7. Race Condition In this case, your receive will end up unexpectedly blocking (!) MPI_RECV(…, tag=9, src=94,comm=17, …) Tag: 3 Source: 14 Comm. ID: 32 Incoming message queue Time Blocked waiting for a matching message Tag: 9 Source: 67 Comm. ID: 17
8. Race Condition If / when the receive finally completes, it’s not the message you probed MPI_RECV(…, tag=9, src=94,comm=17, …) Tag: 3 Source: 14 Comm. ID: 32 Incoming message queue Time Tag: 9 Source: 67 Comm. ID: 17 Tag: 9 Source: 94 Comm. ID: 17
9. MPI_MPROBE MPROBE = Match + probe Fixes this race condition When a message is successfully probed, it is removed from the matching queue
10. MPI_MPROBE MPI_MPROBE looking for: Tag 9, ANY_SOURCE, COMM ID 17 Tag: 3 Source: 14 Comm. ID: 32 Incoming message queue Time Tag: 9 Source: 94 Comm. ID: 17 Tag: 9 Source: 67 Comm. ID: 17
11. MPI_MPROBE When the match occurs, message is removed from the incoming queue MPI_MPROBE looking for: Tag 9, ANY_SOURCE, COMM ID 17 Tag: 3 Source: 14 Comm. ID: 32 Incoming message queue Time Tag: 9 Source: 94 Comm. ID: 17 Tag: 9 Source: 67 Comm. ID: 17
12. MPI_MPROBE Other probes / receives will no longer match this message Tag: 3 Source: 14 Comm. ID: 32 Incoming message queue Time Tag: 9 Source: 94 Comm. ID: 17 MPI_RECV(…, tag=9, src=ANY_SOURCE,comm=17) Tag: 9 Source: 67 Comm. ID: 17
13. MPI_MRECV “Matched” receive is used to receive a message that was mprobed MPI_MRECV(…, match_handle, …) Tag: 3 Source: 14 Comm. ID: 32 Incoming message queue Time Tag: 9 Source: 94 Comm. ID: 17 Guarantees that you get exactly the message you mprobed
14. Another Useful Case Probe to find the size of an incoming message MPI_PROBE(…);
15. Another Useful Case But malloc takes some time to complete MPI_PROBE(…); buf = malloc(incoming_size); Malloc takes some time
16. Another Useful Case But malloc takes some time to complete MPI_PROBE(…); buf = malloc(incoming_size); Malloc takes some time Vulnerable race condition window
17. Another Useful Case But malloc takes some time to complete MPI_PROBE(…); buf = malloc(incoming_size); MPI_RECV(…) Malloc takes some time Vulnerable race condition window Message could be stolen!
18. Another Useful Case Delays between MPROBE and MPRECV do not matter MPI_MPROBE(…); buf = malloc(incoming_size); MPI_MRECV(…) Malloc takes some time
19. Another Useful Case Delays between MPROBE and MPRECV do not matter MPI_MPROBE(…); buf = malloc(incoming_size); MPI_MRECV(…) Malloc takes some time Message cannot be stolen
20. Summary MPI_MPROBE eliminates race condition between probe and corresponding receive Good for: Event-based applications Mutli-threaded MPI applications When message lengths are unknown Strings, serialized objects, etc. E.g., bindings for Perl, Python, Boost.mpi