Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.
Python for High Performance and Scientific
Computing

EuroPython 2011 (23.06.2011, Florence, Italy)
Andreas Schreiber <And...
Abstract



“Python is an accepted high-level scripting language with a growing community in
academia and industry. It is ...
Outline


!   "HPC! and Python
!   "Tools and Libraries
!   "Debugging and Profiling
!   "Example Applications
!   "Python...
Related Talks at EuroPython


!   "Debugging and Profiling Techniques (Giovanni Bajo)
!   "Exploit your GPU Power with PyC...
Folie 5
EuroPython 2011 > Andreas Schreiber > Python for High Performance and Scientific Computing > 23.06.2011
DLR
German Aerospace Center




!   "Research Institution
!    "Space Agency
!     "Project Management Agency




        ...
Locations and employees


6900 employees across                                    Stade ! ! Hamburg
33 institutes and fac...
Software Development at DLR
More than 1000 employees develop!

           Mission-critical software
           (e.g., real...
HPC! and Python




                                                                                                      ...
Overview of Applications
Meteorology, Astrophysics, Particle physics, Biology,
Genetics, Quantum chemistry, Fluid dynamics...
HPC Systems


!   "Current top systems have > 500,000 cores and > 8,000 TFlops
!   "See TOP500 list of supercomputers
    ...
#1: K computer, SPARC64 VIIIfx 2.0GHz, Tofu interc. (RIKEN
Advanced Institute for Computational Science (AICS), Japan)



...
#2: Tianhe-1A - NUDT TH MPP, X5670 2.93Ghz 6C, NVIDIA GPU,
FT-1000 8C (National Supercomputing Center in Tianjin, China)

...
#3: Jaguar - Cray XT5-HE Opteron 6-core 2.6 GHz (DOE/SC/Oak
Ridge National Laboratory, United States)




                ...
Applications
Python is used in!

!   "Computational Fluid Dynamics (CFD)
!   "Plasma simulation
!   "Bio molecular simulat...
Python for Scientists and Engineers
Reasons for Python in Research and Industry

Observations:                            ...
“
!"#$#%&##'&%()%*#%(+)%&)$(&%
),%-#)-.#%+")%.)/#%01(")23%
(")&#%+")%"4(#%*$456#(&7%
428%&59#2:&(&;%%

                   ...
“
<,%9(=&%>))8%#2)?>"%,)$%%
@))>.#%428%ABCB7%9(=&%%
>))8%#2)?>"%,)$%'#7%*4*1;%%


                                        ...
“
01(")2%"4&%("#%5.#42#&(7%%
')&(D&59#2:&(D%)$%#2>92##$%%
,$9#28.1%&12(4E%428%%
&#'42:5&;%            Paul F. Dubois!



 ...
Tools and Libraries for HPC and SC
with Python




                      EuroPython 2011 > Andreas Schreiber > Python for ...
Scientific Tools and Libraries
General Tools

Very general scientific computing   High Performance Computing
! "NumPy     ...
Scientific Tools and Libraries
Domain Specific Tools
AI               Molecular & Atomic                       Electromagn...
Scientific Tools and Libraries
Special Topics

Wrapping other languages
! "weave (C/C++)
! "f2py (Fortran)
!  Cython
!  Ct...
NumPy


!   "Website: http://numpy.scipy.org/

!   "Offers capabilities similar to MATLAB within Python


!   "N-dimension...
NumPy
Arrays

!   "A NumPy array is an N-dimensional homogeneous collection of “items”
     of the same “kind”. The kind c...
SciPy
Scientific Tools for Python

!   "Website: http://www.scipy.org
!   "Large library of scientific algorithms
!   "Ext...
SciPy
Packages

!   "Special Functions
!   "Signal Processing
!   "Fourier Transforms
!   "Optimization
!   "Numerical Int...
Parallel Programming


Threading
! "Useful for certain concurrency issues, not usable for parallel computing
   due to Glo...
mpi4py


!   "Website: http://mpi4py.scipy.org/
!   "Wraps native MPI implementations
      !   "Prefers MPI2, but can wor...
How mpi4py works...


!   "mpi4py jobs must be launched with mpirun
!   "Each rank launches its own independent python int...
from mpi4py import MPI
Example:        import numpy as np
Calculating "   import random
with mpi4py     comm = MPI.COMM_WO...
GPGPU
General-purpose computing on graphics processing units

Architecture
! "Many cores per node
! "Good for stream proce...
PyCUDA


!   "Website: http://mathema.tician.de/software/pycuda
!   "PyCUDA lets you access NVIDIA‘s CUDA parallel computa...
PyOpenCL


!   "Website: http://mathema.tician.de/software/pyopencl
!   "Convenient access to full OpenCL API from Python
...
OpenCL Computing Model


Buffers
! "used for holding data, shipping data between devices


Kernels
! "functions which will...
" with PyOpenCL
import pyopencl as cl
import pyopencl.clrandom
import numpy as np

nsamples = int(12e6)

# set up context ...
Copperhead: Data-Parallelism Embedded in Python
Slides by Michael Garland (NVIDIA Research)




    http://bit.ly/jK7nsM

...
Debugging and Profiling




                                                                                              ...
Debugging Python and C


!   "Python comes with gdb-like
    command line debugger pdb
!   "Helps in debugging serial
    ...
Profiling Mixed Python-C code


!   "Finding performance bottlenecks is critical to scalability on HPC
    platforms
!   "...
TAU Performance System
Slides by Sameer Shende (University of Oregon)




http://slidesha.re/lsDZSj



                   ...
Example Applications

                                                                       Stand #




                 ...
Example:
GIST Maxwell’s Equation Solver (GMES)

!   "Object-Oriented Implementation of the Finite-Difference Time-Domain
 ...
Example: FlowSimulator – A Python-controlled
framework to unify massive parallel CFD workflows




                       ...
What does CFD mean?


!   "CFD = Computational fluid dynamics means to
       ! " solve the compressible flow equations to...
Evolution of CFD
Classical chain
                                                                                         ...
Evolution of CFD
Solver-adaptation-loop
                                                                                  ...
Evolution of CFD
Multi-disciplinary simulations

!   "Multi-disciplinary chain
                                           ...
Structure and Flight Mechanics




                                                                                       ...
FlowSimulator Environment
  Single Python script controls the whole workflow


                               FSControl
  ...
FSControl
!   "Pure Python library

!   "Set of simple Python classes for different modules/tools:
      !   "simple inter...
Python Framework for Coupled Fusion Simulations
Slides by Samantha Foley et al. (ORNL – Oak Ridge
National Laboratory, Uni...
Python & HPC Communities




               EuroPython 2011 > Andreas Schreiber > Python for High Performance and Scientif...
Where the Python and HPC Communities Meet!


!   "SciPy
     !   "Python for Scientific Computing
         Conference (sin...
The SC Conference Series (SCxx)
Tutorials (est. 2009)

!   "Python for Scientific and High Performance Computing
!   "Give...
The SC Conference Series (SCxx)
Birds-of-a-Feather Sessions, est. 2009




                                               ...
Credits
          Credits
          Achim Basermann (DLR)
          Kyungwon Chun (GIST)
          Samantha Foley (ORNL)
 ...
Questions?

Summary
•" Python used in HPC applications
•" Good tools available (SciPy etc.)
•" Wrapping of C/C++/Fortran i...
Contact


  python@dlr.de
     @onyame

          Andreas Schreiber
          A n d r e a s . S c h r e i b e r @ d l r. d...
Nächste SlideShare
Wird geladen in …5
×

von

Python for High Performance and Scientific Computing Slide 1 Python for High Performance and Scientific Computing Slide 2 Python for High Performance and Scientific Computing Slide 3 Python for High Performance and Scientific Computing Slide 4 Python for High Performance and Scientific Computing Slide 5 Python for High Performance and Scientific Computing Slide 6 Python for High Performance and Scientific Computing Slide 7 Python for High Performance and Scientific Computing Slide 8 Python for High Performance and Scientific Computing Slide 9 Python for High Performance and Scientific Computing Slide 10 Python for High Performance and Scientific Computing Slide 11 Python for High Performance and Scientific Computing Slide 12 Python for High Performance and Scientific Computing Slide 13 Python for High Performance and Scientific Computing Slide 14 Python for High Performance and Scientific Computing Slide 15 Python for High Performance and Scientific Computing Slide 16 Python for High Performance and Scientific Computing Slide 17 Python for High Performance and Scientific Computing Slide 18 Python for High Performance and Scientific Computing Slide 19 Python for High Performance and Scientific Computing Slide 20 Python for High Performance and Scientific Computing Slide 21 Python for High Performance and Scientific Computing Slide 22 Python for High Performance and Scientific Computing Slide 23 Python for High Performance and Scientific Computing Slide 24 Python for High Performance and Scientific Computing Slide 25 Python for High Performance and Scientific Computing Slide 26 Python for High Performance and Scientific Computing Slide 27 Python for High Performance and Scientific Computing Slide 28 Python for High Performance and Scientific Computing Slide 29 Python for High Performance and Scientific Computing Slide 30 Python for High Performance and Scientific Computing Slide 31 Python for High Performance and Scientific Computing Slide 32 Python for High Performance and Scientific Computing Slide 33 Python for High Performance and Scientific Computing Slide 34 Python for High Performance and Scientific Computing Slide 35 Python for High Performance and Scientific Computing Slide 36 Python for High Performance and Scientific Computing Slide 37 Python for High Performance and Scientific Computing Slide 38 Python for High Performance and Scientific Computing Slide 39 Python for High Performance and Scientific Computing Slide 40 Python for High Performance and Scientific Computing Slide 41 Python for High Performance and Scientific Computing Slide 42 Python for High Performance and Scientific Computing Slide 43 Python for High Performance and Scientific Computing Slide 44 Python for High Performance and Scientific Computing Slide 45 Python for High Performance and Scientific Computing Slide 46 Python for High Performance and Scientific Computing Slide 47 Python for High Performance and Scientific Computing Slide 48 Python for High Performance and Scientific Computing Slide 49 Python for High Performance and Scientific Computing Slide 50 Python for High Performance and Scientific Computing Slide 51 Python for High Performance and Scientific Computing Slide 52 Python for High Performance and Scientific Computing Slide 53 Python for High Performance and Scientific Computing Slide 54 Python for High Performance and Scientific Computing Slide 55 Python for High Performance and Scientific Computing Slide 56 Python for High Performance and Scientific Computing Slide 57 Python for High Performance and Scientific Computing Slide 58 Python for High Performance and Scientific Computing Slide 59
Nächste SlideShare
8.clustering algorithm.k means.em algorithm
Weiter

51 Gefällt mir

Teilen

Python for High Performance and Scientific Computing

Andreas Schreiber, EuroPython 2011 (June 23, 2011, Florence, Italy)
http://bit.ly/ep11hpc

Ähnliche Bücher

Kostenlos mit einer 30-tägigen Testversion von Scribd

Alle anzeigen

Ähnliche Hörbücher

Kostenlos mit einer 30-tägigen Testversion von Scribd

Alle anzeigen

Python for High Performance and Scientific Computing

  1. 1. Python for High Performance and Scientific Computing EuroPython 2011 (23.06.2011, Florence, Italy) Andreas Schreiber <Andreas.Schreiber@dlr.de> German Aerospace Center (DLR), Cologne http://www.dlr.de/sc EuroPython 2011 > Andreas Schreiber > Python for High Performance and Scientific Computing > 23.06.2011
  2. 2. Abstract “Python is an accepted high-level scripting language with a growing community in academia and industry. It is used in a lot of scientific applications in many different scientific fields and in more and more industries, for example, in engineering or life science). In all fields, the use of Python for high-performance and parallel computing is increasing. Several organizations and companies are providing tools or support for Python development. This includes libraries for scientific computing, parallel computing, and MPI. Python is also used on many core architectures and GPUs, for which specific Python interpreters are being developed. A related topic is the performance of the various interpreter and compiler implementations for Python. The talk gives an overview of Python’s use in HPC and Scientific Computing and gives information on many topics, such as Python on massively parallel systems, GPU programming with Python, scientific libraries in Python, and Python interpreter performance issues. The talk will include examples for scientific codes and applications from many domains..“ EuroPython 2011: http://ep2011.europython.eu/conference/talks/python-for-high-performance-and- scientific-computing Folie 2 EuroPython 2011 > Andreas Schreiber > Python for High Performance and Scientific Computing > 23.06.2011
  3. 3. Outline ! "HPC! and Python ! "Tools and Libraries ! "Debugging and Profiling ! "Example Applications ! "Python and HPC communities Folie 3 EuroPython 2011 > Andreas Schreiber > Python for High Performance and Scientific Computing > 23.06.2011
  4. 4. Related Talks at EuroPython ! "Debugging and Profiling Techniques (Giovanni Bajo) ! "Exploit your GPU Power with PyCUDA (and friends) (Stefano Brilli) ! "Experiences making CPU-bound tasks run much faster (Ian Ozsvald) ! "High-Performance Computing on Gamer PCs (Yann Le Du) ! "Python(x,y): Diving into Scientific Python (Vincent Noel) ! "and more! Folie 4 EuroPython 2011 > Andreas Schreiber > Python for High Performance and Scientific Computing > 23.06.2011
  5. 5. Folie 5 EuroPython 2011 > Andreas Schreiber > Python for High Performance and Scientific Computing > 23.06.2011
  6. 6. DLR German Aerospace Center ! "Research Institution ! "Space Agency ! "Project Management Agency Folie 6 EuroPython 2011 > Andreas Schreiber > Python for High Performance and Scientific Computing > 23.06.2011
  7. 7. Locations and employees 6900 employees across Stade ! ! Hamburg 33 institutes and facilities at ! Neustrelitz Bremen ! ! Trauen !" 15 sites. Berlin ! Braunschweig ! Offices in Brussels, ! Goettingen Paris and Washington. ! Cologne ! Bonn Lampoldshausen ! Stuttgart ! Augsburg ! ! Oberpfaffenhofen Weilheim ! Folie 7 EuroPython 2011 > Andreas Schreiber > Python for High Performance and Scientific Computing > 23.06.2011
  8. 8. Software Development at DLR More than 1000 employees develop! Mission-critical software (e.g., real-time software) Web-based software for internet and intranet Supporting software (e.g., data management) High-Performance and Scientific Computing Software Folie 8 EuroPython 2011 > Andreas Schreiber > Python for High Performance and Scientific Computing > 23.06.2011
  9. 9. HPC! and Python Folie 9 EuroPython 2011 > Andreas Schreiber > Python for High Performance and Scientific Computing > 23.06.2011
  10. 10. Overview of Applications Meteorology, Astrophysics, Particle physics, Biology, Genetics, Quantum chemistry, Fluid dynamics, Finance, Oil exploration, ! Images from: http://www.isgtw.org Folie 10 EuroPython 2011 > Andreas Schreiber > Python for High Performance and Scientific Computing > 23.06.2011
  11. 11. HPC Systems ! "Current top systems have > 500,000 cores and > 8,000 TFlops ! "See TOP500 list of supercomputers (http://www.top500.org) ! "Top 3 systems (June 2011) are: 1." K computer, Japan 2." Tianhe-1A, China 3." Jaguar, Unites States Folie 11 EuroPython 2011 > Andreas Schreiber > Python for High Performance and Scientific Computing > 23.06.2011
  12. 12. #1: K computer, SPARC64 VIIIfx 2.0GHz, Tofu interc. (RIKEN Advanced Institute for Computational Science (AICS), Japan) Folie 12 EuroPython 2011 > Andreas Schreiber > Python for High Performance and Scientific Computing > 23.06.2011
  13. 13. #2: Tianhe-1A - NUDT TH MPP, X5670 2.93Ghz 6C, NVIDIA GPU, FT-1000 8C (National Supercomputing Center in Tianjin, China) Folie 13 EuroPython 2011 > Andreas Schreiber > Python for High Performance and Scientific Computing > 23.06.2011
  14. 14. #3: Jaguar - Cray XT5-HE Opteron 6-core 2.6 GHz (DOE/SC/Oak Ridge National Laboratory, United States) Folie 14 EuroPython 2011 > Andreas Schreiber > Python for High Performance and Scientific Computing > 23.06.2011
  15. 15. Applications Python is used in! ! "Computational Fluid Dynamics (CFD) ! "Plasma simulation ! "Bio molecular simulation ! "Artificial intelligence ! "Natural language processing ! "Data mining ! "Scientific visualization ! "Robotics ! "Computer games ! "System administration ! "Education ! "! Folie 15 EuroPython 2011 > Andreas Schreiber > Python for High Performance and Scientific Computing > 23.06.2011
  16. 16. Python for Scientists and Engineers Reasons for Python in Research and Industry Observations: “I want to design ! "Scientists and engineers don’t want to write software planes, not software!” but just solve their problems ! "If they have to write code, it must be as easy as possible Why Python is perfect? ! "Very easy to learn and easy to use ( = steep learning curve) ! "Allows rapid development ( = short development time) ! "Inherent great maintainability Folie 16 EuroPython 2011 > Andreas Schreiber > Python for High Performance and Scientific Computing > 23.06.2011
  17. 17. “ !"#$#%&##'&%()%*#%(+)%&)$(&% ),%-#)-.#%+")%.)/#%01(")23% (")&#%+")%"4(#%*$456#(&7% 428%&59#2:&(&;%% Folie 17 EuroPython 2011 > Andreas Schreiber > Python for High Performance and Scientific Computing > 10.07.2008 23.06.2011
  18. 18. “ <,%9(=&%>))8%#2)?>"%,)$%% @))>.#%428%ABCB7%9(=&%% >))8%#2)?>"%,)$%'#7%*4*1;%% Folie 18 EuroPython 2011 > Andreas Schreiber > Python for High Performance and Scientific Computing > 10.07.2008 23.06.2011
  19. 19. “ 01(")2%"4&%("#%5.#42#&(7%% ')&(D&59#2:&(D%)$%#2>92##$%% ,$9#28.1%&12(4E%428%% &#'42:5&;% Paul F. Dubois! Folie 19 EuroPython 2011 > Andreas Schreiber > Python for High Performance and Scientific Computing > 10.07.2008 23.06.2011
  20. 20. Tools and Libraries for HPC and SC with Python EuroPython 2011 > Andreas Schreiber > Python for High Performance and Scientific Computing > 23.06.2011
  21. 21. Scientific Tools and Libraries General Tools Very general scientific computing High Performance Computing ! "NumPy Parallel Computing ! "SciPy ! PETSc ! PyMPI Visualization ! Pypar ! Matplotlib ! "mpi4py ! VisIt GPGPU Computing ! MayaVi ! "PyCUDA ! "Chaco ! "PyOpenCL ! "VTK Folie 21 EuroPython 2011 > Andreas Schreiber > Python for High Performance and Scientific Computing > 23.06.2011
  22. 22. Scientific Tools and Libraries Domain Specific Tools AI Molecular & Atomic Electromagnetics ! "pyem Modeling ! PyFemax ! "ffnet ! PyMOL ! "pymorph ! Biskit Astronomy ! "Monte ! "GPAW ! AstroLib ! hcluster ! PySolar Geo sciences Biology ! "GIS Python Dynamic Systems ! "Brian ! PyClimate ! Simpy ! "SloppyCell ! ClimPy ! PyDSTool ! "NIPY ! "CDAT ! "PySAT Finite Elements ! SfePy Folie 22 EuroPython 2011 > Andreas Schreiber > Python for High Performance and Scientific Computing > 23.06.2011
  23. 23. Scientific Tools and Libraries Special Topics Wrapping other languages ! "weave (C/C++) ! "f2py (Fortran) ! Cython ! Ctypes (C) ! "SWIG (C/C++) ! RPy / RSPython (R) ! MatPy (Matlab) ! Jython (Java) ! IronPython (.NET) Folie 23 EuroPython 2011 > Andreas Schreiber > Python for High Performance and Scientific Computing > 23.06.2011
  24. 24. NumPy ! "Website: http://numpy.scipy.org/ ! "Offers capabilities similar to MATLAB within Python ! "N-dimensional homogeneous arrays (ndarray) ! "Universal functions (ufunc) ! "basic math, linear algebra, FFT, PRNGs ! "Simple data file I/O ! "text, raw binary, native binary ! "Tools for integrating with C/C++/Fortran ! "Heavy lifting done by optimized C/Fortran libraries ! "ATLAS or MKL, UMFPACK, FFTW, etc... Folie 24 EuroPython 2011 > Andreas Schreiber > Python for High Performance and Scientific Computing > 23.06.2011
  25. 25. NumPy Arrays ! "A NumPy array is an N-dimensional homogeneous collection of “items” of the same “kind”. The kind can be any arbitrary structure and is specified using the data-type. ! "Example: Multidimensional NumPy array >>> a = array([[ 0, 1, 2, 3], [10,11,12,13]]) >>> a array([[ 0, 1, 2, 3], [10,11,12,13]]) >>> a[1,3] 13 >>> a[1,3] = -1 >>> a array([[ 0, 1, 2, 3], [10,11,12,-1]]) Folie 25 EuroPython 2011 > Andreas Schreiber > Python for High Performance and Scientific Computing > 23.06.2011
  26. 26. SciPy Scientific Tools for Python ! "Website: http://www.scipy.org ! "Large library of scientific algorithms ! "Extends NumPy with many tools for science and engineering ! "Computationally intensive routines implemented in C and Fortran Folie 26 EuroPython 2011 > Andreas Schreiber > Python for High Performance and Scientific Computing > 23.06.2011
  27. 27. SciPy Packages ! "Special Functions ! "Signal Processing ! "Fourier Transforms ! "Optimization ! "Numerical Integration ! "Linear Algebra ! Input/Output ! "Statistics ! "Fast Execution ! "Clustering Algorithms ! "Sparse Matrices Folie 27 EuroPython 2011 > Andreas Schreiber > Python for High Performance and Scientific Computing > 23.06.2011
  28. 28. Parallel Programming Threading ! "Useful for certain concurrency issues, not usable for parallel computing due to Global Interpreter Lock (GIL) subprocess ! "Relatively low level control for spawning and managing processes ! "multiprocessing - multiple Python instances (processes) ! "basic, clean multiple process parallelism MPI ! "mpi4py exposes your full local MPI API within Python ! "As scalable as your local MPI GPU (OpenCL & CUDA) ! "PyOpenCL and PyCUDA provide low and high level abstraction for highly parallel computations on GPUs Folie 28 EuroPython 2011 > Andreas Schreiber > Python for High Performance and Scientific Computing > 23.06.2011
  29. 29. mpi4py ! "Website: http://mpi4py.scipy.org/ ! "Wraps native MPI implementations ! "Prefers MPI2, but can work with MPI1 ! "Works best with NumPy data types, but can pass around any serializable object ! "Provides all MPI2 features ! "Well maintained ! "Distributed with Enthought Python Distribution (EPD) ! "Requires NumPy ! "Portable and scalable Folie 29 EuroPython 2011 > Andreas Schreiber > Python for High Performance and Scientific Computing > 23.06.2011
  30. 30. How mpi4py works... ! "mpi4py jobs must be launched with mpirun ! "Each rank launches its own independent python interpreter ! "Each interpreter only has access to files and libraries available locally to it, unless distributed to the ranks ! "Communication is handled by MPI layer ! "Any function outside of an if block specifying a rank is assumed to be global ! "Any limitations of your local MPI are present in mpi4py Folie 30 EuroPython 2011 > Andreas Schreiber > Python for High Performance and Scientific Computing > 23.06.2011
  31. 31. from mpi4py import MPI Example: import numpy as np Calculating " import random with mpi4py comm = MPI.COMM_WORLD rank = comm.Get_rank() mpisize = comm.Get_size() nsamples = int(12e6/mpisize) inside = 0 random.seed(rank) for i in range(nsamples): x = random.random() y = random.random() if (x*x)+(y*y)<1: inside += 1 mypi = (4.0 * inside)/nsamples pi = comm.reduce(mypi, op=MPI.SUM, root=0) if rank==0: print (1.0 / mpisize)*pi Folie 31 EuroPython 2011 > Andreas Schreiber > Python for High Performance and Scientific Computing > 23.06.2011
  32. 32. GPGPU General-purpose computing on graphics processing units Architecture ! "Many cores per node ! "Good for stream processing (independent vertices, many of them in parallel) CUDA ! "NVIDIA’s platform for C programming on GPGPUs ! "Python binding: PyCUDA OpenCL ! "Framework for writing programs that execute across heterogeneous platforms consisting of CPUs, GPUs, and other processors ! "Open standard ! "Python binding: PyOpenCL Folie 32 EuroPython 2011 > Andreas Schreiber > Python for High Performance and Scientific Computing > 23.06.2011
  33. 33. PyCUDA ! "Website: http://mathema.tician.de/software/pycuda ! "PyCUDA lets you access NVIDIA‘s CUDA parallel computation API from Python ! "Integration with NumPy ! "All CUDA errors are automatically translated into Python exceptions Folie 33 EuroPython 2011 > Andreas Schreiber > Python for High Performance and Scientific Computing > 23.06.2011
  34. 34. PyOpenCL ! "Website: http://mathema.tician.de/software/pyopencl ! "Convenient access to full OpenCL API from Python ! "OpenCL errors translated into Python exceptions ! "Object cleanup tied to object lifetime (follows Resource Acquisition Is Initialization) ! "NumPy ndarrays interact easily with OpenCL buffers Folie 34 EuroPython 2011 > Andreas Schreiber > Python for High Performance and Scientific Computing > 23.06.2011
  35. 35. OpenCL Computing Model Buffers ! "used for holding data, shipping data between devices Kernels ! "functions which will operate on buffers in parallel Queue ! "control sequence of operations: transferring buffers, running kernels, waiting on events, etc. Folie 35 EuroPython 2011 > Andreas Schreiber > Python for High Performance and Scientific Computing > 23.06.2011
  36. 36. " with PyOpenCL import pyopencl as cl import pyopencl.clrandom import numpy as np nsamples = int(12e6) # set up context and queue ctx = cl.create_some_context() queue = cl.CommandQueue(ctx) # create array of random values in OpenCL xy = pyopencl.clrandom.rand(ctx,queue,(nsamples,2),np.float32) # square values in OpenCL xy = xy**2 # 'get' method on xy is used to get array from OpenCL into ndarray print 4.0*np.sum(np.sum(xy.get(),1)<1)/nsamples Folie 36 EuroPython 2011 > Andreas Schreiber > Python for High Performance and Scientific Computing > 23.06.2011
  37. 37. Copperhead: Data-Parallelism Embedded in Python Slides by Michael Garland (NVIDIA Research) http://bit.ly/jK7nsM Folie 37 EuroPython 2011 > Andreas Schreiber > Python for High Performance and Scientific Computing > 23.06.2011
  38. 38. Debugging and Profiling Folie 38 EuroPython 2011 > Andreas Schreiber > Python for High Performance and Scientific Computing > 23.06.2011
  39. 39. Debugging Python and C ! "Python comes with gdb-like command line debugger pdb ! "Helps in debugging serial Python applications ! "C-extensions can be debugged with standard debuggers ! "Parallel debugging ! Totalview can be used for debugging C-functions ! "Custom interpreter provides easy initial breakpoint Folie 39 EuroPython 2011 > Andreas Schreiber > Python for High Performance and Scientific Computing > 23.06.2011
  40. 40. Profiling Mixed Python-C code ! "Finding performance bottlenecks is critical to scalability on HPC platforms ! "Number of profiling tools available: ! gprof, CrayPAT, scalasca - C, Fortran ! "can be used for C-extensions ! "import profile – Python ! "TAU Performance System, http://www.cs.uoregon.edu/research/tau/home.php ! "Excurse: Slides by Sameer Shende (University of Oregon) Folie 40 EuroPython 2011 > Andreas Schreiber > Python for High Performance and Scientific Computing > 23.06.2011
  41. 41. TAU Performance System Slides by Sameer Shende (University of Oregon) http://slidesha.re/lsDZSj Folie 41 EuroPython 2011 > Andreas Schreiber > Python for High Performance and Scientific Computing > 23.06.2011
  42. 42. Example Applications Stand # Folie 42 EuroPython 2011 > Andreas Schreiber > Python for High Performance and Scientific Computing > 23.06.2011
  43. 43. Example: GIST Maxwell’s Equation Solver (GMES) ! "Object-Oriented Implementation of the Finite-Difference Time-Domain Method in Parallel Computing Environment ! "Slides by Kyungwon Chun et al. (GIST - Gwagnju Institute of Science and Technology, South Korea)! http://bit.ly/mCs8W9 Folie 43 EuroPython 2011 > Andreas Schreiber > Python for High Performance and Scientific Computing > 23.06.2011
  44. 44. Example: FlowSimulator – A Python-controlled framework to unify massive parallel CFD workflows Folie 44 EuroPython 2011 > Andreas Schreiber > Python for High Performance and Scientific Computing > 23.06.2011
  45. 45. What does CFD mean? ! "CFD = Computational fluid dynamics means to ! " solve the compressible flow equations to ! " predict an aircrafts behavior and to ! " optimize its features. ! "Numerical approach: ! " Spatial discretization ! " Simplified models for turbulent effects ! " Temporal discretization (to support unsteady flows) Folie 45 EuroPython 2011 > Andreas Schreiber > Python for High Performance and Scientific Computing > 23.06.2011
  46. 46. Evolution of CFD Classical chain Z ! "Simple chain Y X ! " Geometry ! " Mesh generator ! " Pre-processor ! " Flow solver ! " Post-processor ! " Visualization ! "Tools ! " stand-alone mesher ! " stand-alone solver (with pre-/postprocessor) ! " shell scripts (e.g. different solver parameters) Folie 46 EuroPython 2011 > Andreas Schreiber > Python for High Performance and Scientific Computing > 23.06.2011
  47. 47. Evolution of CFD Solver-adaptation-loop Z ! "Chain with solver-adaptation-loop Y X ! " several consecutive solver steps ! " adaptation of mesh in between ! "Tools: ! " stand-alone mesher ! " stand-alone solver ! " stand-alone adaptation ! " shell script control ! "Solver/adaptation as Python module Folie 47 EuroPython 2011 > Andreas Schreiber > Python for High Performance and Scientific Computing > 23.06.2011
  48. 48. Evolution of CFD Multi-disciplinary simulations ! "Multi-disciplinary chain Z Y X ! " output of solver is input for ! " structural mechanics (deformation) ! " flight mechanics (control surface movement) ! " output of structural/flight mechanics modifies mesh ! "Tools: ! " solver as Python module ! " stand-alone CSM ! " Python-based flight mechanics Folie 48 EuroPython 2011 > Andreas Schreiber > Python for High Performance and Scientific Computing > 23.06.2011
  49. 49. Structure and Flight Mechanics Folie 49 EuroPython 2011 > Andreas Schreiber > Python for High Performance and Scientific Computing > 23.06.2011
  50. 50. FlowSimulator Environment Single Python script controls the whole workflow FSControl Out of scope Flight CFD CSM CAA mechanics Import Export FSDataManager Folie 50 EuroPython 2011 > Andreas Schreiber > Python for High Performance and Scientific Computing > 23.06.2011
  51. 51. FSControl ! "Pure Python library ! "Set of simple Python classes for different modules/tools: ! "simple interface: Import, Run, Export ! "data stored in FSDataManager FSDataManager (FSDM) ! "C++ library, wrapped to Python ! "FSDataManager is a real Python module! ! "Parallelization context ! "all data structures “distributed” ! "facilitated transfers (Gather, Distribute, Send, !) Folie 51 EuroPython 2011 > Andreas Schreiber > Python for High Performance and Scientific Computing > 23.06.2011
  52. 52. Python Framework for Coupled Fusion Simulations Slides by Samantha Foley et al. (ORNL – Oak Ridge National Laboratory, United States) http://slidesha.re/iLuegW Folie 52 EuroPython 2011 > Andreas Schreiber > Python for High Performance and Scientific Computing > 23.06.2011
  53. 53. Python & HPC Communities EuroPython 2011 > Andreas Schreiber > Python for High Performance and Scientific Computing > 23.06.2011
  54. 54. Where the Python and HPC Communities Meet! ! "SciPy ! "Python for Scientific Computing Conference (since 2008) ! "EuroSciPy ! "Annual European Conference for Scientists using Python (since 2008) ! "SCxx ! "The International Conference for High Performance Computing, Networking, Storage, and Analysis (annual conference since 1988) ! "Tutorial and BoF session on Python (since 2009) Folie 54 EuroPython 2011 > Andreas Schreiber > Python for High Performance and Scientific Computing > 23.06.2011
  55. 55. The SC Conference Series (SCxx) Tutorials (est. 2009) ! "Python for Scientific and High Performance Computing ! "Given by William R. Scullin (ANL – Argonne National Lab, United States) et al. ! "Content ! "Python language and interpreter basics ! "Popular modules and packages for scientific applications ! "How to improve performance in Python programs ! "How to visualize and share data using Python ! "Where to find documentation and resources Folie 55 EuroPython 2011 > Andreas Schreiber > Python for High Performance and Scientific Computing > 23.06.2011
  56. 56. The SC Conference Series (SCxx) Birds-of-a-Feather Sessions, est. 2009 Folie 56 EuroPython 2011 > Andreas Schreiber > Python for High Performance and Scientific Computing > 23.06.2011
  57. 57. Credits Credits Achim Basermann (DLR) Kyungwon Chun (GIST) Samantha Foley (ORNL) Michael Garland (NVIDIA) Michael Meinel (DLR) Tr a v i s O l i p h a n t ( E n t h o u g h t ) William R. Scullin (ANL) Sameer Shende (U Oregon) Folie 57 EuroPython 2011 > Andreas Schreiber > Python for High Performance and Scientific Computing > 23.06.2011
  58. 58. Questions? Summary •" Python used in HPC applications •" Good tools available (SciPy etc.) •" Wrapping of C/C++/Fortran is common Andreas Schreiber A n d r e a s . S c h r e i b e r @ d l r. d e http://www.dlr.de/sc Folie 58 EuroPython 2011 > Andreas Schreiber > Python for High Performance and Scientific Computing > 23.06.2011
  59. 59. Contact python@dlr.de @onyame Andreas Schreiber A n d r e a s . S c h r e i b e r @ d l r. d e http://www.dlr.de/sc Folie 59 EuroPython 2011 > Andreas Schreiber > Python for High Performance and Scientific Computing > 23.06.2011
  • BrunoSchulze2

    Jan. 29, 2020
  • CrisleydeSouzaPeixot

    Apr. 24, 2017
  • YusukeTsutsui

    Dec. 13, 2016
  • xiaolabstudent

    Aug. 24, 2016
  • AlarmChang

    Jun. 14, 2016
  • RajeshLakkakula2

    May. 13, 2016
  • tahirzdemir

    Jan. 6, 2016
  • naoki4277

    Sep. 12, 2015
  • TomLu4

    Sep. 7, 2015
  • cyberz24

    Jun. 1, 2015
  • Gonnector

    Nov. 25, 2014
  • SunnyboysSon

    Nov. 25, 2014
  • jayuloy

    Nov. 25, 2014
  • kiheumcho7

    Nov. 25, 2014
  • jogeun

    Nov. 25, 2014
  • talantyr

    Nov. 25, 2014
  • ssuser593481

    Nov. 25, 2014
  • yich

    Nov. 25, 2014
  • pedrovaroherrero

    Nov. 6, 2014
  • ChaitanyaKulkarni11

    Oct. 6, 2014

Andreas Schreiber, EuroPython 2011 (June 23, 2011, Florence, Italy) http://bit.ly/ep11hpc

Aufrufe

Aufrufe insgesamt

14.761

Auf Slideshare

0

Aus Einbettungen

0

Anzahl der Einbettungen

1.528

Befehle

Downloads

0

Geteilt

0

Kommentare

0

Likes

51

×