SlideShare ist ein Scribd-Unternehmen logo
1 von 74
MPI Rohit Banga Prakher Anand K Swagat Manoj Gupta Advanced Computer Architecture Spring, 2010
ORGANIZATION ,[object Object],[object Object],[object Object],[object Object]
GOALS ,[object Object],[object Object],[object Object]
MESSAGE PASSING INTERFACE ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
GOALS OF MPI SPECIFICATION ,[object Object],[object Object],[object Object],[object Object]
REASONS FOR USING MPI ,[object Object],[object Object],[object Object],[object Object],[object Object]
BASIC MODEL ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
BASIC MODEL (CONTD.) ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
COMMUNICATORS ,[object Object],[object Object]
COMMUNICATOR AND GROUPS ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
VIRTUAL TOPOLOGIES ,[object Object],[object Object],[object Object],[object Object],[object Object]
SEMANTICS ,[object Object],[object Object],[object Object],[object Object],Format:  rc = MPI_Xxxxx(parameter, ... )  Example:  rc = MPI_Bsend(&buf,count,type,dest,tag,comm)  Error code:  Returned as "rc". MPI_SUCCESS if successful
MPI PROGRAM STRUCTURE
MPI FUNCTIONS – MINIMAL SUBSET ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
CLASSIFICATION OF MPI ROUTINES ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
MPI_INIT ,[object Object],[object Object],[object Object],[object Object],[object Object],int main(int argc, char **argv) { MPI_Init(&argc, &argv); … }
MPI_COMM_SIZE ,[object Object],[object Object],[object Object],int main(int argc, char **argv) { MPI_Init(&argc, &argv); int p; MPI_Comm_size(MPI_COMM_WORLD, &p); … }
MPI_COMM_RANK ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],int main(int argc, char **argv) { MPI_Init(&argc, &argv); int p; MPI_Comm_size(MPI_COMM_WORLD, &p); int rank; MPI_Comm_rank(MPI_COMM_WORLD, &rank); … }
MPI_FINALIZE ,[object Object],[object Object],[object Object],int main(int argc, char **argv) { MPI_Init(&argc, &argv); int p; MPI_Comm_size(MPI_COMM_WORLD, &p); int rank; MPI_Comm_rank(MPI_COMM_WORLD, &rank); printf(“no. of processors: %d rank: %d”, p, rank); MPI_Finalize(); }
 
HOW TO COMPILE THIS ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
HOW TO RUN THIS ,[object Object],[object Object],[object Object]
MPIRUN ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
A NOTE ON IMPLEMENTATION ,[object Object],[object Object],MPI_Init MPI Thread MPI_Init MPI Thread
SOME MORE FUNCTIONS ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
POINT TO POINT COMMUNICATION
POINT-TO-POINT COMMUNICATION ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
POINT-TO-POINT COMMUNICATION ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
POINT-TO-POINT COMMUNICATION ,[object Object],[object Object],[object Object],[object Object]
BLOCKING SEND/RECEIVE ,[object Object],[object Object],[object Object],[object Object]
BLOCKING SEND/RECEIVE ,[object Object],[object Object],[object Object],[object Object]
BLOCKING SEND/RECEIVE ,[object Object],[object Object],[object Object],[object Object]
 
BLOCKING SEND/RECEIVE ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
BLOCKING SEND/RECEIVE ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
BLOCKING SEND/RECEIVE ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
BLOCKING SEND/RECEIVE (CONTD.) Process 1 Process 2 Data Processor 1 Application Send System Buffer Processor 2 Application Send System Buffer
A WORD ABOUT SPECIFICATION ,[object Object],[object Object],[object Object],[object Object]
BLOCKING SEND/RECEIVE (CONTD.) ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
NON-BLOCKING SEND/RECEIVE ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
NON-BLOCKING SEND/RECEIVE (CONTD.) Process 1 Process 2 Data Processor 1 Application Send System Buffer Processor 2 Application Send System Buffer
[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],NON-BLOCKING SEND/RECEIVE (CONTD.)
[object Object],[object Object],NON-BLOCKING SEND/RECEIVE (CONTD.)
STANDARD MODE ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
SYNCHRONOUS MODE ,[object Object],[object Object],[object Object],[object Object],[object Object]
BUFFERED MODE ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
BUFFER MANAGEMENT ,[object Object],[object Object],[object Object],[object Object],MPI_Buffer_attach( malloc(BUFFSIZE), BUFFSIZE);  /* a buffer of BUFFSIZE bytes can now be used by MPI_Bsend */  MPI_Buffer_detach( &buff, &size);  /* Buffer size reduced to zero */  MPI_Buffer_attach( buff, size);  /* Buffer of BUFFSIZE bytes available again */
READY MODE ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
ORDER AND FAIRNESS ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
EXAMPLE OF NON-OVERTAKING MESSAGES. CALL MPI_COMM_RANK(comm, rank, ierr)  IF (rank.EQ.0) THEN  CALL MPI_BSEND(buf1, count, MPI_REAL, 1, tag, comm, ierr)  CALL MPI_BSEND(buf2, count, MPI_REAL, 1, tag, comm, ierr)  ELSE ! rank.EQ.1  CALL MPI_RECV(buf1, count, MPI_REAL, 0, MPI_ANY_TAG, comm,  status, ierr)  CALL MPI_RECV(buf2, count, MPI_REAL, 0, tag, comm, status, ierr)  END IF
EXAMPLE OF INTERTWINGLED MESSAGES. CALL MPI_COMM_RANK(comm, rank, ierr)  IF (rank.EQ.0) THEN  CALL MPI_BSEND(buf1, count, MPI_REAL, 1, tag1, comm, ierr)  CALL MPI_SSEND(buf2, count, MPI_REAL, 1, tag2, comm, ierr)  ELSE ! rank.EQ.1  CALL MPI_RECV(buf1, count, MPI_REAL, 0, tag2, comm,  status, ierr)  CALL MPI_RECV(buf2, count, MPI_REAL, 0, tag1, comm, status, ierr)  END IF
DEADLOCK EXAMPLE CALL MPI_COMM_RANK(comm, rank, ierr)  IF (rank.EQ.0) THEN  CALL MPI_RECV(recvbuf, count, MPI_REAL, 1, tag, comm, status, ierr) CALL MPI_SEND(sendbuf, count, MPI_REAL, 1, tag, comm, ierr)  ELSE ! rank.EQ.1  CALL MPI_RECV(recvbuf, count, MPI_REAL, 0, tag, comm, status, ierr) CALL MPI_SEND(sendbuf, count, MPI_REAL, 0, tag, comm, ierr)  END IF
EXAMPLE OF BUFFERING CALL MPI_COMM_RANK(comm, rank, ierr)  IF (rank.EQ.0) THEN  CALL MPI_SEND(buf1, count, MPI_REAL, 1, tag, comm, ierr)  CALL MPI_RECV (recvbuf, count, MPI_REAL, 1, tag, comm, status, ierr) ELSE ! rank.EQ.1  CALL MPI_SEND(sendbuf, count, MPI_REAL, 0, tag, comm, ierr) CALL MPI_RECV(buf2, count, MPI_REAL, 0, tag, comm, status, ierr)  END IF
COLLECTIVE COMMUNICATIONS
COLLECTIVE ROUTINES ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
COLLECTIVE ROUTINES (CONTD.) ,[object Object],[object Object],[object Object]
COLLECTIVE ROUTINES (CONTD.) ,[object Object],[object Object],[object Object]
COLLECTIVE ROUTINES (CONTD.) ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
COLLECTIVE ROUTINES (CONTD.) ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
MPI OPERATIONS MPI_OP operator MPI_MIN Minimum MPI_SUM Sum MPI_PROD product MPI_MAX maximum MPI_LAND Logical and MPI_BAND Bitwise and MPI_LOR Logical or MPI_BOR Bitwise or MPI_LXOR Logical xor MPI_BXOR Bit-wise xor MPI_MAXLOC Max value and location MPI_MINLOC Min value and location
COLLECTIVE ROUTINES (CONTD.)
Learn by Examples
Parallel Trapezoidal Rule Output:  Estimate of the integral from a to b of f(x) using the trapezoidal rule and n trapezoids. Algorithm: 1.  Each process calculates "its" interval of integration. 2.  Each process estimates the integral of f(x) over its interval using the trapezoidal rule. 3a. Each process != 0 sends its integral to 0. 3b. Process 0 sums the calculations received from the individual processes and prints the result. Notes:  1.  f(x), a, b, and n are all hardwired. 2.  The number of processes (p) should evenly divide the number of trapezoids (n = 1024)
Parallelizing the Trapezoidal Rule #include <stdio.h> #include &quot;mpi.h&quot; main(int argc, char** argv) { int  my_rank;  /* My process rank  */ int  p;  /* The number of processes  */ double  a = 0.0;  /* Left endpoint  */ double  b = 1.0;  /* Right endpoint  */ int  n = 1024;  /* Number of trapezoids  */ double  h;  /* Trapezoid base length  */ double  local_a;  /* Left endpoint my process  */ double  local_b;  /* Right endpoint my process */ int  local_n;  /* Number of trapezoids for  */ /* my calculation  */ double  integral;  /* Integral over my interval */ double  total;  /* Total integral  */ int  source;  /* Process sending integral  */ int  dest = 0;  /* All messages go to 0  */ int  tag = 0; MPI_Status  status;
Continued… double Trap(double local_a, double local_b, int local_n,double h);  /* Calculate local integral  */ MPI_Init (&argc, &argv); MPI_Barrier(MPI_COMM_WORLD); double elapsed_time = -MPI_Wtime(); MPI_Comm_rank(MPI_COMM_WORLD, &my_rank); MPI_Comm_size(MPI_COMM_WORLD, &p); h = (b-a)/n;  /* h is the same for all processes */ local_n = n/p;  /* So is the number of trapezoids */ /* Length of each process' interval of integration = local_n*h.  So my interval starts at: */ local_a = a + my_rank*local_n*h; local_b = local_a + local_n*h; integral = Trap(local_a, local_b, local_n, h);
Continued… /* Add up the integrals calculated by each process */ if (my_rank == 0) { total = integral; for (source = 1; source < p; source++) { MPI_Recv(&integral, 1, MPI_DOUBLE, source, tag,  MPI_COMM_WORLD,  &status); total = total + integral; }//End for } else  MPI_Send(&integral, 1, MPI_DOUBLE, dest, tag, MPI_COMM_WORLD); MPI_Barrier(MPI_COMM_WORLD); elapsed_time += MPI_Wtime(); /* Print the result */ if (my_rank == 0) { printf(&quot;With n = %d trapezoids, our estimate&quot;,n); printf(&quot;of the integral from %lf to %lf = %lf&quot;,a, b, total); printf(&quot;time taken: %lf&quot;, elapsed_time); }
Continued… /* Shut down MPI */ MPI_Finalize(); } /*  main  */  double Trap(  double  local_a , double  local_b, int local_n, double  h) { double integral;  /* Store result in integral  */ double x; int i; double f(double x); /* function we're integrating */ integral = (f(local_a) + f(local_b))/2.0; x = local_a; for (i = 1; i <= local_n-1; i++) { x = x + h; integral = integral + f(x); } integral = integral*h; return integral; } /*  Trap  */
Continued… double f(double x) { double return_val; /* Calculate f(x). */ /* Store calculation in return_val. */ return_val = 4 / (1+x*x); return return_val; } /* f */
Program 2 Process other than root generates the random value less than 1 and sends to root. Root sums up and displays sum.
#include <stdio.h> #include <mpi.h> #include<stdlib.h> #include <string.h> #include<time.h> int main(int argc, char **argv) { int myrank,  p; int tag =0, dest=0; int i; double randIn,randOut; int source; MPI_Status status; MPI_Init(&argc,&argv); MPI_Comm_rank(MPI_COMM_WORLD,&myrank);
MPI_Comm_size(MPI_COMM_WORLD, &p); if(myrank==0)//I am the root { double total=0,average=0; for(source=1;source<p;source++) { MPI_Recv(&randIn,1, MPI_DOUBLE, source, MPI_ANY_TAG,  MPI_COMM_WORLD, &status); printf(&quot;Message from root: From %d received number %f&quot;,source  ,randIn); total+=randIn; }//End for average=total/(p-1); }//End if
else//I am other than root { srand48((long int) myrank); randOut=drand48(); printf(&quot;randout=%f, myrank=%d&quot;,randOut,myrank); MPI_Send(&randOut,1,MPI_DOUBLE,dest,tag,MPI_COMM_WORLD); }//End If-Else MPI_Finalize(); return 0; }
MPI References ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
THANK YOU

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Lecture 1 introduction to parallel and distributed computing
Lecture 1   introduction to parallel and distributed computingLecture 1   introduction to parallel and distributed computing
Lecture 1 introduction to parallel and distributed computing
 
Communication costs in parallel machines
Communication costs in parallel machinesCommunication costs in parallel machines
Communication costs in parallel machines
 
Design issues of dos
Design issues of dosDesign issues of dos
Design issues of dos
 
Message and Stream Oriented Communication
Message and Stream Oriented CommunicationMessage and Stream Oriented Communication
Message and Stream Oriented Communication
 
Lecture 2 more about parallel computing
Lecture 2   more about parallel computingLecture 2   more about parallel computing
Lecture 2 more about parallel computing
 
Ch 20 UNICAST ROUTING SECTION 2
Ch 20   UNICAST ROUTING  SECTION  2Ch 20   UNICAST ROUTING  SECTION  2
Ch 20 UNICAST ROUTING SECTION 2
 
Distributed Operating System_1
Distributed Operating System_1Distributed Operating System_1
Distributed Operating System_1
 
Parallel processing
Parallel processingParallel processing
Parallel processing
 
Multiprocessor structures
Multiprocessor structuresMultiprocessor structures
Multiprocessor structures
 
Course outline of parallel and distributed computing
Course outline of parallel and distributed computingCourse outline of parallel and distributed computing
Course outline of parallel and distributed computing
 
Feng’s classification
Feng’s classificationFeng’s classification
Feng’s classification
 
message passing vs shared memory
message passing vs shared memorymessage passing vs shared memory
message passing vs shared memory
 
Parallel processing
Parallel processingParallel processing
Parallel processing
 
advanced computer architesture-conditions of parallelism
advanced computer architesture-conditions of parallelismadvanced computer architesture-conditions of parallelism
advanced computer architesture-conditions of parallelism
 
Computer architecture multi processor
Computer architecture multi processorComputer architecture multi processor
Computer architecture multi processor
 
Transport Layer
Transport LayerTransport Layer
Transport Layer
 
Multi processor scheduling
Multi  processor schedulingMulti  processor scheduling
Multi processor scheduling
 
Operating System-Ch8 memory management
Operating System-Ch8 memory managementOperating System-Ch8 memory management
Operating System-Ch8 memory management
 
Pram model
Pram modelPram model
Pram model
 
Message passing interface
Message passing interfaceMessage passing interface
Message passing interface
 

Ähnlich wie MPI Introduction

Tutorial on Parallel Computing and Message Passing Model - C2
Tutorial on Parallel Computing and Message Passing Model - C2Tutorial on Parallel Computing and Message Passing Model - C2
Tutorial on Parallel Computing and Message Passing Model - C2
Marcirio Chaves
 

Ähnlich wie MPI Introduction (20)

Parallel and Distributed Computing Chapter 10
Parallel and Distributed Computing Chapter 10Parallel and Distributed Computing Chapter 10
Parallel and Distributed Computing Chapter 10
 
My ppt hpc u4
My ppt hpc u4My ppt hpc u4
My ppt hpc u4
 
High Performance Computing using MPI
High Performance Computing using MPIHigh Performance Computing using MPI
High Performance Computing using MPI
 
Introduction to MPI
Introduction to MPIIntroduction to MPI
Introduction to MPI
 
Parallel computing(2)
Parallel computing(2)Parallel computing(2)
Parallel computing(2)
 
Mpi
Mpi Mpi
Mpi
 
Parallel programming using MPI
Parallel programming using MPIParallel programming using MPI
Parallel programming using MPI
 
25-MPI-OpenMP.pptx
25-MPI-OpenMP.pptx25-MPI-OpenMP.pptx
25-MPI-OpenMP.pptx
 
Tutorial on Parallel Computing and Message Passing Model - C2
Tutorial on Parallel Computing and Message Passing Model - C2Tutorial on Parallel Computing and Message Passing Model - C2
Tutorial on Parallel Computing and Message Passing Model - C2
 
Intro to MPI
Intro to MPIIntro to MPI
Intro to MPI
 
cs556-2nd-tutorial.pdf
cs556-2nd-tutorial.pdfcs556-2nd-tutorial.pdf
cs556-2nd-tutorial.pdf
 
MPI
MPIMPI
MPI
 
More mpi4py
More mpi4pyMore mpi4py
More mpi4py
 
Lecture9
Lecture9Lecture9
Lecture9
 
Introduction to MPI
Introduction to MPIIntroduction to MPI
Introduction to MPI
 
Open MPI
Open MPIOpen MPI
Open MPI
 
BASIC_MPI.ppt
BASIC_MPI.pptBASIC_MPI.ppt
BASIC_MPI.ppt
 
Message Passing Interface (MPI)-A means of machine communication
Message Passing Interface (MPI)-A means of machine communicationMessage Passing Interface (MPI)-A means of machine communication
Message Passing Interface (MPI)-A means of machine communication
 
Programming using MPI and OpenMP
Programming using MPI and OpenMPProgramming using MPI and OpenMP
Programming using MPI and OpenMP
 
Rgk cluster computing project
Rgk cluster computing projectRgk cluster computing project
Rgk cluster computing project
 

Kürzlich hochgeladen

Jual Obat Aborsi Hongkong ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
Jual Obat Aborsi Hongkong ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...Jual Obat Aborsi Hongkong ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
Jual Obat Aborsi Hongkong ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
ZurliaSoop
 

Kürzlich hochgeladen (20)

This PowerPoint helps students to consider the concept of infinity.
This PowerPoint helps students to consider the concept of infinity.This PowerPoint helps students to consider the concept of infinity.
This PowerPoint helps students to consider the concept of infinity.
 
Jual Obat Aborsi Hongkong ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
Jual Obat Aborsi Hongkong ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...Jual Obat Aborsi Hongkong ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
Jual Obat Aborsi Hongkong ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
 
REMIFENTANIL: An Ultra short acting opioid.pptx
REMIFENTANIL: An Ultra short acting opioid.pptxREMIFENTANIL: An Ultra short acting opioid.pptx
REMIFENTANIL: An Ultra short acting opioid.pptx
 
Python Notes for mca i year students osmania university.docx
Python Notes for mca i year students osmania university.docxPython Notes for mca i year students osmania university.docx
Python Notes for mca i year students osmania university.docx
 
General Principles of Intellectual Property: Concepts of Intellectual Proper...
General Principles of Intellectual Property: Concepts of Intellectual  Proper...General Principles of Intellectual Property: Concepts of Intellectual  Proper...
General Principles of Intellectual Property: Concepts of Intellectual Proper...
 
FSB Advising Checklist - Orientation 2024
FSB Advising Checklist - Orientation 2024FSB Advising Checklist - Orientation 2024
FSB Advising Checklist - Orientation 2024
 
Unit 3 Emotional Intelligence and Spiritual Intelligence.pdf
Unit 3 Emotional Intelligence and Spiritual Intelligence.pdfUnit 3 Emotional Intelligence and Spiritual Intelligence.pdf
Unit 3 Emotional Intelligence and Spiritual Intelligence.pdf
 
How to setup Pycharm environment for Odoo 17.pptx
How to setup Pycharm environment for Odoo 17.pptxHow to setup Pycharm environment for Odoo 17.pptx
How to setup Pycharm environment for Odoo 17.pptx
 
Google Gemini An AI Revolution in Education.pptx
Google Gemini An AI Revolution in Education.pptxGoogle Gemini An AI Revolution in Education.pptx
Google Gemini An AI Revolution in Education.pptx
 
Jamworks pilot and AI at Jisc (20/03/2024)
Jamworks pilot and AI at Jisc (20/03/2024)Jamworks pilot and AI at Jisc (20/03/2024)
Jamworks pilot and AI at Jisc (20/03/2024)
 
COMMUNICATING NEGATIVE NEWS - APPROACHES .pptx
COMMUNICATING NEGATIVE NEWS - APPROACHES .pptxCOMMUNICATING NEGATIVE NEWS - APPROACHES .pptx
COMMUNICATING NEGATIVE NEWS - APPROACHES .pptx
 
Application orientated numerical on hev.ppt
Application orientated numerical on hev.pptApplication orientated numerical on hev.ppt
Application orientated numerical on hev.ppt
 
2024-NATIONAL-LEARNING-CAMP-AND-OTHER.pptx
2024-NATIONAL-LEARNING-CAMP-AND-OTHER.pptx2024-NATIONAL-LEARNING-CAMP-AND-OTHER.pptx
2024-NATIONAL-LEARNING-CAMP-AND-OTHER.pptx
 
How to Add New Custom Addons Path in Odoo 17
How to Add New Custom Addons Path in Odoo 17How to Add New Custom Addons Path in Odoo 17
How to Add New Custom Addons Path in Odoo 17
 
Sensory_Experience_and_Emotional_Resonance_in_Gabriel_Okaras_The_Piano_and_Th...
Sensory_Experience_and_Emotional_Resonance_in_Gabriel_Okaras_The_Piano_and_Th...Sensory_Experience_and_Emotional_Resonance_in_Gabriel_Okaras_The_Piano_and_Th...
Sensory_Experience_and_Emotional_Resonance_in_Gabriel_Okaras_The_Piano_and_Th...
 
Holdier Curriculum Vitae (April 2024).pdf
Holdier Curriculum Vitae (April 2024).pdfHoldier Curriculum Vitae (April 2024).pdf
Holdier Curriculum Vitae (April 2024).pdf
 
Mehran University Newsletter Vol-X, Issue-I, 2024
Mehran University Newsletter Vol-X, Issue-I, 2024Mehran University Newsletter Vol-X, Issue-I, 2024
Mehran University Newsletter Vol-X, Issue-I, 2024
 
Food safety_Challenges food safety laboratories_.pdf
Food safety_Challenges food safety laboratories_.pdfFood safety_Challenges food safety laboratories_.pdf
Food safety_Challenges food safety laboratories_.pdf
 
Key note speaker Neum_Admir Softic_ENG.pdf
Key note speaker Neum_Admir Softic_ENG.pdfKey note speaker Neum_Admir Softic_ENG.pdf
Key note speaker Neum_Admir Softic_ENG.pdf
 
Fostering Friendships - Enhancing Social Bonds in the Classroom
Fostering Friendships - Enhancing Social Bonds  in the ClassroomFostering Friendships - Enhancing Social Bonds  in the Classroom
Fostering Friendships - Enhancing Social Bonds in the Classroom
 

MPI Introduction

  • 1. MPI Rohit Banga Prakher Anand K Swagat Manoj Gupta Advanced Computer Architecture Spring, 2010
  • 2.
  • 3.
  • 4.
  • 5.
  • 6.
  • 7.
  • 8.
  • 9.
  • 10.
  • 11.
  • 12.
  • 14.
  • 15.
  • 16.
  • 17.
  • 18.
  • 19.
  • 20.  
  • 21.
  • 22.
  • 23.
  • 24.
  • 25.
  • 26. POINT TO POINT COMMUNICATION
  • 27.
  • 28.
  • 29.
  • 30.
  • 31.
  • 32.
  • 33.  
  • 34.
  • 35.
  • 36.
  • 37. BLOCKING SEND/RECEIVE (CONTD.) Process 1 Process 2 Data Processor 1 Application Send System Buffer Processor 2 Application Send System Buffer
  • 38.
  • 39.
  • 40.
  • 41. NON-BLOCKING SEND/RECEIVE (CONTD.) Process 1 Process 2 Data Processor 1 Application Send System Buffer Processor 2 Application Send System Buffer
  • 42.
  • 43.
  • 44.
  • 45.
  • 46.
  • 47.
  • 48.
  • 49.
  • 50. EXAMPLE OF NON-OVERTAKING MESSAGES. CALL MPI_COMM_RANK(comm, rank, ierr) IF (rank.EQ.0) THEN CALL MPI_BSEND(buf1, count, MPI_REAL, 1, tag, comm, ierr) CALL MPI_BSEND(buf2, count, MPI_REAL, 1, tag, comm, ierr) ELSE ! rank.EQ.1 CALL MPI_RECV(buf1, count, MPI_REAL, 0, MPI_ANY_TAG, comm, status, ierr) CALL MPI_RECV(buf2, count, MPI_REAL, 0, tag, comm, status, ierr) END IF
  • 51. EXAMPLE OF INTERTWINGLED MESSAGES. CALL MPI_COMM_RANK(comm, rank, ierr) IF (rank.EQ.0) THEN CALL MPI_BSEND(buf1, count, MPI_REAL, 1, tag1, comm, ierr) CALL MPI_SSEND(buf2, count, MPI_REAL, 1, tag2, comm, ierr) ELSE ! rank.EQ.1 CALL MPI_RECV(buf1, count, MPI_REAL, 0, tag2, comm, status, ierr) CALL MPI_RECV(buf2, count, MPI_REAL, 0, tag1, comm, status, ierr) END IF
  • 52. DEADLOCK EXAMPLE CALL MPI_COMM_RANK(comm, rank, ierr) IF (rank.EQ.0) THEN CALL MPI_RECV(recvbuf, count, MPI_REAL, 1, tag, comm, status, ierr) CALL MPI_SEND(sendbuf, count, MPI_REAL, 1, tag, comm, ierr) ELSE ! rank.EQ.1 CALL MPI_RECV(recvbuf, count, MPI_REAL, 0, tag, comm, status, ierr) CALL MPI_SEND(sendbuf, count, MPI_REAL, 0, tag, comm, ierr) END IF
  • 53. EXAMPLE OF BUFFERING CALL MPI_COMM_RANK(comm, rank, ierr) IF (rank.EQ.0) THEN CALL MPI_SEND(buf1, count, MPI_REAL, 1, tag, comm, ierr) CALL MPI_RECV (recvbuf, count, MPI_REAL, 1, tag, comm, status, ierr) ELSE ! rank.EQ.1 CALL MPI_SEND(sendbuf, count, MPI_REAL, 0, tag, comm, ierr) CALL MPI_RECV(buf2, count, MPI_REAL, 0, tag, comm, status, ierr) END IF
  • 55.
  • 56.
  • 57.
  • 58.
  • 59.
  • 60. MPI OPERATIONS MPI_OP operator MPI_MIN Minimum MPI_SUM Sum MPI_PROD product MPI_MAX maximum MPI_LAND Logical and MPI_BAND Bitwise and MPI_LOR Logical or MPI_BOR Bitwise or MPI_LXOR Logical xor MPI_BXOR Bit-wise xor MPI_MAXLOC Max value and location MPI_MINLOC Min value and location
  • 63. Parallel Trapezoidal Rule Output: Estimate of the integral from a to b of f(x) using the trapezoidal rule and n trapezoids. Algorithm: 1. Each process calculates &quot;its&quot; interval of integration. 2. Each process estimates the integral of f(x) over its interval using the trapezoidal rule. 3a. Each process != 0 sends its integral to 0. 3b. Process 0 sums the calculations received from the individual processes and prints the result. Notes: 1. f(x), a, b, and n are all hardwired. 2. The number of processes (p) should evenly divide the number of trapezoids (n = 1024)
  • 64. Parallelizing the Trapezoidal Rule #include <stdio.h> #include &quot;mpi.h&quot; main(int argc, char** argv) { int my_rank; /* My process rank */ int p; /* The number of processes */ double a = 0.0; /* Left endpoint */ double b = 1.0; /* Right endpoint */ int n = 1024; /* Number of trapezoids */ double h; /* Trapezoid base length */ double local_a; /* Left endpoint my process */ double local_b; /* Right endpoint my process */ int local_n; /* Number of trapezoids for */ /* my calculation */ double integral; /* Integral over my interval */ double total; /* Total integral */ int source; /* Process sending integral */ int dest = 0; /* All messages go to 0 */ int tag = 0; MPI_Status status;
  • 65. Continued… double Trap(double local_a, double local_b, int local_n,double h); /* Calculate local integral */ MPI_Init (&argc, &argv); MPI_Barrier(MPI_COMM_WORLD); double elapsed_time = -MPI_Wtime(); MPI_Comm_rank(MPI_COMM_WORLD, &my_rank); MPI_Comm_size(MPI_COMM_WORLD, &p); h = (b-a)/n; /* h is the same for all processes */ local_n = n/p; /* So is the number of trapezoids */ /* Length of each process' interval of integration = local_n*h. So my interval starts at: */ local_a = a + my_rank*local_n*h; local_b = local_a + local_n*h; integral = Trap(local_a, local_b, local_n, h);
  • 66. Continued… /* Add up the integrals calculated by each process */ if (my_rank == 0) { total = integral; for (source = 1; source < p; source++) { MPI_Recv(&integral, 1, MPI_DOUBLE, source, tag, MPI_COMM_WORLD, &status); total = total + integral; }//End for } else MPI_Send(&integral, 1, MPI_DOUBLE, dest, tag, MPI_COMM_WORLD); MPI_Barrier(MPI_COMM_WORLD); elapsed_time += MPI_Wtime(); /* Print the result */ if (my_rank == 0) { printf(&quot;With n = %d trapezoids, our estimate&quot;,n); printf(&quot;of the integral from %lf to %lf = %lf&quot;,a, b, total); printf(&quot;time taken: %lf&quot;, elapsed_time); }
  • 67. Continued… /* Shut down MPI */ MPI_Finalize(); } /* main */ double Trap( double local_a , double local_b, int local_n, double h) { double integral; /* Store result in integral */ double x; int i; double f(double x); /* function we're integrating */ integral = (f(local_a) + f(local_b))/2.0; x = local_a; for (i = 1; i <= local_n-1; i++) { x = x + h; integral = integral + f(x); } integral = integral*h; return integral; } /* Trap */
  • 68. Continued… double f(double x) { double return_val; /* Calculate f(x). */ /* Store calculation in return_val. */ return_val = 4 / (1+x*x); return return_val; } /* f */
  • 69. Program 2 Process other than root generates the random value less than 1 and sends to root. Root sums up and displays sum.
  • 70. #include <stdio.h> #include <mpi.h> #include<stdlib.h> #include <string.h> #include<time.h> int main(int argc, char **argv) { int myrank, p; int tag =0, dest=0; int i; double randIn,randOut; int source; MPI_Status status; MPI_Init(&argc,&argv); MPI_Comm_rank(MPI_COMM_WORLD,&myrank);
  • 71. MPI_Comm_size(MPI_COMM_WORLD, &p); if(myrank==0)//I am the root { double total=0,average=0; for(source=1;source<p;source++) { MPI_Recv(&randIn,1, MPI_DOUBLE, source, MPI_ANY_TAG, MPI_COMM_WORLD, &status); printf(&quot;Message from root: From %d received number %f&quot;,source ,randIn); total+=randIn; }//End for average=total/(p-1); }//End if
  • 72. else//I am other than root { srand48((long int) myrank); randOut=drand48(); printf(&quot;randout=%f, myrank=%d&quot;,randOut,myrank); MPI_Send(&randOut,1,MPI_DOUBLE,dest,tag,MPI_COMM_WORLD); }//End If-Else MPI_Finalize(); return 0; }
  • 73.

Hinweis der Redaktion

  1. Can work with shared memory architectures also
  2. Why MPI is still being used
  3. Many vendors can compete for providing better implementation
  4. Boring topic. But fundamental for understanding the basics
  5. Safe – different libraries can work together
  6. Different return codes for different functions
  7. To start coding we need to use these functions.
  8. Mention the case where the buffer space might not be available
  9. Buffer used only during Buffered mode communication.
  10. Ready call indicates the system that a receive has already been posted.
  11. Built in collective operations. Reduce, Bcast, Datatypes