MPI Presentation

Parallel Programming
using
Message Passing
Interface
(MPI)
metu-ceng
ts@TayfunSen.com
11/05/08 Parallel Programming Using MPI 1
25 April 2008

Outline
• What is MPI?
• MPI Implementations
• OpenMPI
• MPI
• References
• Q&A

11/05/08 Parallel Programming Using MPI 2/26

What is MPI?
• A standard with many implementations
(lam-mpi and mpich, evolving into
OpenMPI and MVAPICH).
• message passing API
• Library for programming clusters
• Needs to be high performing, scalable,
portable ...


MPI Implementations
• Is it up for the challenge?
MPI does not have many alternatives
(what about OpenMP, map-reduce etc?).
• Many implementations out there.
• The programming interface is all the same. But
underlying implementations and what they
support in terms of connectivity, fault tolerance
etc. differ.
• On ceng-hpc, both MVAPICH and OpenMPI is
installed.


OpenMPI
• We'll use OpenMPI for this presentation
• It's open source, MPI2 compliant,
portable, has fault tolerance, combines
best practices of number of other MPI
implementations.
• To install it, for example on
Debian/Ubuntu type:
# apt-get install openmpi-bin libopenmpi-dev
openmpi-doc

MPI – General Information
• Functions start with MPI_* to differ
from application
• MPI has defined its own data types to
abstract machine dependent
implementations (MPI_CHAR,
MPI_INT, MPI_BYTE etc.)


MPI - API and other stuff
• Housekeeping (initialization,
termination, header file)
• Two types of communication: Point-
to-point and collective communication
• Communicators


Housekeeping
• You include the header mpi.h
• Initialize using MPI_Init(&argc, &argv)
and end MPI using MPI_Finalize()
• Demo time, “hello world!” using MPI


Point-to-point
communication
• Related definitions – source,
destination, communicator, tag,
buffer, data type, count
• man MPI_Send, MPI_Recv
int MPI_Send(void *buf, int count, MPI_Datatype
datatype, int dest,int tag, MPI_Comm comm)

• Blocking send, that is the processor
doesn't do anything until the message
is sent

P2P Communication (cont.)
• int MPI_Recv(void *buf, int count, MPI_Datatype
datatype, int source, int tag, MPI_Comm comm,
MPI_Status *status)

• Source, tag, communicator has to be
correct for the message to be received
• Demo time – simple send
• One last thing, you can use wildcards in
place of source and tag.
MPI_ANY_SOURCE and MPI_ANY_TAG


• The receiver actually does not know
how much data it received. He takes
a guess and tries to get the most.
• To be sure of how much received, one
can use:
• int MPI_Get_count(MPI_Status *status, MPI_Datatype
dtype, int *count);

• Demo time – change simple send to
check the received message size.

• For a receive operation, communication ends when
the message is copied to the local variables.
• For a send operation, communication is completed
when the message is transferred to MPI for
sending. (so that the buffer can be recycled)
• Blocked operations continue when the
communication has been completed
• Beware – There are some intricacies
Check [2] for more information.


• For blocking communications, deadlock is a
possibility:
if( myrank == 0 ) {
/* Receive, then send a message */
MPI_Recv( b, 100, MPI_DOUBLE, 1, 19, MPI_COMM_WORLD, &status );
MPI_Send( a, 100, MPI_DOUBLE, 1, 17, MPI_COMM_WORLD );
}
else if( myrank == 1 ) {
/* Receive, then send a message */
MPI_Recv( b, 100, MPI_DOUBLE, 0, 17, MPI_COMM_WORLD, &status );
MPI_Send( a, 100, MPI_DOUBLE, 0, 19, MPI_COMM_WORLD );
}

• How to remove the deadlock?

• When non-blocking communication is used,
program continues its execution
• A program can send a blocking send and
the receiver may use non-blocking receive
or vice versa.
• Very similar function calls
int MPI_Isend(void *buf, int count, MPI_Datatype dtype, int dest,
int tag, MPI_Comm comm, MPI_Request *request);

• Request handle can be used later
eg. MPI_Wait, MPI_Test ...

• Demo time – non_blocking
• There are other modes of sending
(but not receiving!) check out the
documentation for synchronous,
buffered and ready mode send in
addition to standard one we have
seen here.


• Keep in mind that each send/receive is costly
– try to piggyback
• You can send different data types at the same
time – eg. Integers, floats, characters,
doubles... using MPI_Pack. This function gives
you an intermediate buffer which you will
send.
• int MPI_Pack(void *inbuf, int incount, MPI_Datatype
datatype, void *outbuf, int outsize, int *position,
MPI_Comm comm)
• MPI_Send(buffer, count, MPI_PACKED, dest, tag,
MPI_COMM_WORLD);


• You can also send your own structs
(user defined types). See the
documentation


Collective Communication
• Works like point to point except you
send to all other processors
• MPI_Barrier(comm), blocks until each
processor calls this. Synchronizes
everyone.
• Broadcast operation MPI_Bcast copies
the data value in one processor to
others.
• Demo time - bcast_example

Collective Communication
• MPI_Reduce collects data from other
processors, operates on them and
returns a single value
• reduction operation is performed
• Demo time – reduce_op example
• There are MPI defined reduce
operations but you can define your
own

Collective Communication -
MPI_Gather
• gather and scatter operations
• Like what their name implies
• Gather – like every process sending
their send buffer and root process
receiving
• Demo time - gather_example


Collective Communication -
MPI_Scatter
• Similar to MPI_Gather but here data
is sent from root to other processors
• Like gather, you can accomplish it by
having root calling MPI_Send
repeatedly and others calling
MPI_Recv
• Demo time – scatter_example


Collective Communication –
More functionality
• Many more functions to lift hard work
from you.
• MPI_Allreduce, MPI_Gatherv,
MPI_Scan, MPI_Reduce_Scatter ...
• Check out the API documentation
• Manual files are your best friend.


Communicators
• Communicators group processors
• Basic communicator
MPI_COMM_WORLD defined for all
processors
• You can create your own
communicators to group processors.
Thus you can send messages to only
a subset of all processors.

More Advanced Stuff
• Parallel I/O – when one node is used
for reading from disk it is slow. You
can have each node use its local disk.
• One sided communications – Remote
memory access
• Both are MPI-2 capabilities. Check
your MPI implementation to see how
much it is implemented.

References
[1] Wikipedia articles in general, including but not limited to:
http://en.wikipedia.org/wiki/Message_Passing_Interface
[2] An excellent guide at NCSA (National Center for
Supercomputing Applications):
http://webct.ncsa.uiuc.edu:8900/public/MPI/
[3] OpenMPI Official Web site:
http://www.open-mpi.org/


The End

Thanks For Your Time.
Any Questions
?

MPI Presentation

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Andere mochten auch

Andere mochten auch (20)

Ähnlich wie MPI Presentation

Ähnlich wie MPI Presentation (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

MPI Presentation