Developing Performance-Oriented Code: Moore's Law Over 50

Speaker – the speaker notes are important for this
presentation. Be sure to read them.
Developing
Performance
Oriented Code:
Moore’s Law Over 50

Copyright © 2015, Intel Corporation. All rights reserved. *Other
names and brands may be claimed as the property of others.
Optimization Notice
Objectives
What does Moore’s Law mean to developers
 A bit of history and perspective
– Impact
 “The Free Lunch Is Over” Parallel or Perish
– Parallelism
– Vectorization
Resources: Training, Tools
2

Optimization Notice
Moore’s Law
“The complexity for minimum
component costs has increased at
a rate of roughly a factor of two
per year. Certainly over the short
term this rate can be expected to
continue, if not to increase. Over
the longer term, the rate of
increase is a bit more uncertain,
although there is no reason to
believe it will not remain nearly
constant for at least 10 years.
That means by 1975, the number
of components per integrated
circuit for minimum cost will be
65,000.”
Photos: Intel Crop.
3
Software Developers get
“Free Lunch”

Optimization Notice
The Continuing Evolution of Moore's Law
• Moore’s observation in 1965 was about integrated circuits
for memory
• Packing more functions into an integrated circuit, the cost per function
went down and trending to double of functions every year.
• This is about economics and has remained constant
• Moore did not make an observation about
performance. For that we turn to a pair of observations
that are not as well-known as Moore.
• Robert Dennard observed in 1974 that as a transistor became smaller, if
you scaled features and voltage at the right rate, size, frequency, and power
all improved together.
• Fred Pollack from Intel observed that doubling the complexity of a
microprocessor resulted in a square root of two increase in performance.
• With these two rules and Moore we can construct a user
value triangle relating price, integration, and performance
4

Optimization Notice
NUC’s in 1965?
5
In this cartoon published in the original 1965 article, the editors
of Electronics correctly adapted Moore’s thesis to imagine a
future Intel employees would later help realize.

Optimization Notice
6

Optimization Notice
Impact of “Moore’s Law”
Economic Impact
As more transistors fit into smaller spaces, processing power increased
and energy efficiency improved, all at a lower cost for the end user.
Enhancing existing industries and spawning whole new industries.
8

Optimization Notice
9
Economic Impact of “Moore’s Law”

Optimization Notice
Economic Impact
Technological Impact
Transformed computing from a rare and expensive venture into a pervasive
and affordable necessity.
10

Optimization Notice
11
Technological Impact of “Moore’s Law”

Optimization Notice
Economic Impact
Technological Impact
Transformed computing from a rare and expensive venture into a pervasive
and affordable necessity.
Societal Impact
The foundational force of Moore’s Law has driven breakthroughs in modern
cities, transportation, healthcare, education, and energy production.
12

Optimization Notice
Societal Impact of “Moore’s Law”
Performance is a Proven Game Changer
It is driving disruptive change in multiple industries
13
Protecting buildings from extreme events
Sophisticated mechanics simulations are performed to identify
innovative ways to protect infrastructure from extreme events, such as
natural disasters.
Click on a picture for details

Optimization Notice
14
Solving Austin, Texas’s traffic problem
Running advanced traffic simulations to improve the models used to
plan infrastructure and traffic control changes
natural disasters.
Click on a picture for details

Optimization Notice
15
Solving Austin, Texas’s traffic problem
Running advanced traffic simulations to improve the models used to
plan infrastructure and traffic control changes
natural disasters.
New possible treatments for Parkinson’s
Extensive calculations performed at supercomputer helped
researchers to learn more about the protein structure’s evolution
Click on a picture for detailsImagine what our modern world might be like without Moore’s Law.

Optimization Notice
16
Software releases Hardware Performance

© 2017 Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks of Intel Corporation or its subsidiaries in the U.S. and/or other countries. *Other names and
brands may be claimed as the property of others.
For more complete information about compiler optimizations, see our Optimization Notice.
The “Free Lunch” Has Been Over for a Long Time
Original data up to the year 2010 collected and plotted by M. Horowitz, F. Labonate, O. Shacham, K. Olukotun, L.
Hammond, and C. Batten. New plot and data collected for 2010-2015 by K. Rupp.
40 Years of Microprocessor Trend Data
1980 1990 2000 2010 20201970
100
101
102
103
104
105
106
107 Transistors
(thousands)
Single-Thread
Performance
Frequency (MHz)
Typical Power
(Watts)
Number of
Logical Cores
Herb Sutter, “The Free Lunch Is
Over: A Fundamental Turn Toward
Concurrency in Software” Dr.
Dobb’s Journal, 30(3), March 2005.
Hardware performance
continues to grow.
But changes are no longer
transparent to software.

Optimization Notice
Where are most these new transistors going?
Answer: Parallel HW!
Why should you care?
Answer: It’s where (dramatically)
more performance is!

Optimization Notice
2008… the developer to developer
conversation begins
ISTEP 2009
19Intel Confidential
James Reinders
Chief Software Evangelist and Director
Intel Software Development Products

Optimization Notice
Don’t use a single Vector lane!
Un-vectorized and un-threaded software will under perform
20

Changing Hardware Impacts Software
Intel® Xeon® Processor
64-bit
5100
series
5500
series
5600
series
E5-2600
E5-2600
V2
E5-2600
V3
E5-2600
V4
Platinum
8180
Core(s) 1 2 4 6 8 12 18 22 28
Threads 2 2 8 12 16 24 36 44 56
SIMD
Width
128 128 128 128 256 256 256 256 512
Intel® Xeon Phi™
Knights Landing
72
288
512
More Cores More Threads Wider Vectors
>100x
Vectorized & Parallelized
Scalar & Parallelized
Vectorized & Single-Threaded
Scalar & Single-Threaded

Code Modernization – Using the Whole Die
1 Core, No SIMD 12 Cores, No
SIMD
12 Cores, SIMD

Optimization Notice
Intel® Xeon Running Serial Code
Intel® Xeon Parallelized Code
145X
FASTER
67.097
SECONDS
0.46
SECONDS
Leaving Performance on the Table?
Intel® Xeon Phi™ Parallelized Code
340X
FASTER
0.197
SECONDS

Optimization Notice
Permission to Design for All Lanes
Threading and Vectorization needed to fully utilize modern hardware
24

Optimization Notice
Art of doing more than one thing at a time
More programming work
More “bookkeeping”
Greater performance potential
Greater resource utilization
25
So What is Parallelism?

Optimization Notice
Example workflow
 Input array A with 64 elements
 Three functions: h(x)=f(x)+g(x)
Data Parallelism is working with multiple pieces of data simultaneously
 Task flow is still serial
Task Parallelism is performing multiple tasks simultaneously
 Can look like data parallelism
Data Parallelism is really a subset of Task Parallelism
 But the two can be treated differently
26
Data Parallelism vs. Task Parallelism
do i=1,64
a1=f(A(i))
a2=g(A(i))
h(i)=a1+a2
end do

Optimization Notice
Task and Data Flows (Serial)
27
F(x) G(x) H(x)A[64]
F(A) G(A)
H(A)

Optimization Notice
Task and Data Flows (Data Parallel)
28
F(x) G(x) H(x)A[64]
F(A) G(A)
H(A)

Optimization Notice
Task and Data Flows (Task Parallelism)
29
F(x)
G(x)
H(x)A[64]
F(A)
G(A)
H(A)

Optimization Notice
SIMD – Single Instruction Multiple Data
 Data-parallel only
Performed at CPU Instruction level
As name implies, a single instruction is performed on multiple pieces of data
 Data is loaded from memory into cache line
 Operation is performed simultaneously on all (unless masked) data in SIMD register
 Results returned to memory (or reused later)
30
Vectorization Overview

31
What is it?
 A comprehensive tool suite for building high
performance, scalable parallel code from
enterprise to cloud, and HPC to AI applications.
 Includes C++, Fortran, & Python performance
tools: industry-leading compilers, numerical
libraries, performance profilers, & code
analyzers.
 Supports Windows*, Linux* and macOS*.
 Who needs this
product?
OEMs/ISVs
C++, Fortran, & Python*
developers
Developers, domain specialists of
enterprise, data center/
cloud, HPC & AI applications
Intel® Parallel Studio XE - Overview
Build Fast, Scalable Parallel Applications from Enterprise to Cloud, & HPC to AI
Certain technical specifications and select processors/skus apply. See product site for details.
Why important ?
 Accelerate performance on Intel® Xeon® &
Core™ processors
 Deliver fast, scalable, reliable parallel code with
less effort
 Modernize code efficiently - optimize for today's
and future Intel® platforms
 Stay up-to-date with the latest standards
Download: software.intel.com/intel-parallel-studio-xe

© 2017 Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks of Intel Corporation or its subsidiaries in the U.S. and/or other countries.
*Other names and brands may be claimed as the property of others.
32
Better Tools
for Parallel
Programming
Better
Parallel
Models
Wildly more
Hardware
Parallelism
Better
Educated
Programmers
 Performance
 Intel tools are key to utilizing processor performance.
 Scale forward
 Your application investment extends to tomorrow’s
platforms.
 Code Reliability
 More Dependable and Secure
Key value propositions for intel software
development products

© 2017 Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks of Intel Corporation or its subsidiaries in the U.S. and/or other countries.
What’s Inside
Intel® Parallel Studio XE 2019
Additional configurations including, floating and academic, are available at: http://intel.ly/perf-tools
33
Composer
Edition
Professional
Edition
Cluster
Edition
Build
Intel® C++ Compiler – industry leading performance
Intel® Fortran Compiler– industry leading performance
Intel® Distribution for Python* - high performance Python distribution
Intel® Math Kernel Library – fast math library
Intel® Integrated Performance Primitives – image, signal & data processing
Intel® Threading Building Blocks – threading library
Intel® Data Analytics Acceleration Library – machine learning & analytics
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
Anal
yze
Intel® VTune™ Amplifier XE – performance profiler
Intel® Advisor – vectorization optimization and thread prototyping
Intel® Inspector – memory and thread debugging
√
√
√
√
√
√
SCA
LE
Intel® MPI Library – message passing interface library
Intel® Trace Analyzer and Collector – MPI Tuning and Analysis
Intel® Cluster Checker – cluster diagnostic expert system
√
√
√
Download a
Free Trial
Intel® Software Development Tools Floating License Change – As of Sept. 12, 2018, floating licenses for Intel® Software Development Tools 2017, 2018and 2019 versions require the latest version (Intel 2.5/lmgrd 11.14.1.1) of the Intel®
Software License Manager for successful installation. To obtain this, visit Intel Registration Center. For more details: Installation Errors Related to Intel Software License Manager Upgrade.
Rogue Wave IMSL* Library is no longer available directly from Intel. It can be obtained directly from Rogue Wave or Rogue Wave resellers. More details on IMSL is available at: www.roguewave.com

Optimization Notice
Use Compiler reports to determine why vectorization failed
 Compiler reports show which loops were vectorized and which weren’t
– Along with reasons for not vectorizing:
– ….data dependencies, flow dependencies, and inefficient vectorization, some
loops simply aren’t worth the overhead.
Help compiler determine safe vectorizations – multiple options:
 Directives/Pragmas
 SIMD features of OpenMP*
 Intel® Cilk™ Plus SIMD features
34
Vectorization Tips

Copyright © 2018, Intel Corporation. All rights reserved.
Optimization Notice
Fast, Scalable Code with Intel® Math Kernel Library
(Intel® MKL)
35
 Speeds computations for scientific, engineering, financial and
machine learning applications by providing highly optimized,
threaded, and vectorized math functions
 Provides key functionality for dense and sparse linear algebra
(BLAS, LAPACK, PARDISO), FFTs, vector math, summary
statistics, deep learning, splines and more
 Dispatches optimized code for each processor automatically
without the need to branch code
 Optimized for single core vectorization and cache utilization
 Automatic parallelism for multi-core and many-core
 Scales from core to clusters
 Available at no cost and royalty free
 Great performance with minimal effort!
1 Available only in Intel® Parallel Studio Composer Edition.
Dense & SPARSE Linear Algebra
Fast Fourier Transforms
Vector Math
Vector RNGs
Fast Poisson Solver
& More!
Intel® Math Kernel Library
Offers…

Optimization Notice
Get the Benefits of Advanced Threading
with Threading Building Blocks
36
Use Threading to Leverage Multicore Performance
& Heterogeneous Computing
 Parallelize computationally intensive work across CPUs,
GPUs & FPGAs,—deliver higher-level & simpler
solutions using C++
 Most feature-rich & comprehensive solution for parallel
programming
 Highly portable, composable, affordable, approachable,
future-proof scalability
Learn More: software.intel.com/intel-tbb

Optimization Notice
37
Faster Vectorization Optimization
 Vectorize where it will pay off most
 Quickly ID what is blocking vectorization
 Tips for effective vectorization
 Safely force compiler vectorization
 Optimize memory stride
Data & Guidance You Need
 Compiler diagnostics +
Performance Data + SIMD efficiency
 Detect problems & recommend fixes
 Loop-Carried Dependency Analysis
 Memory Access Patterns Analysis
‘Automatic’ Vectorization is Often Not Enough
A good compiler can still benefit greatly from vectorization optimization
Intel® Advisor—Vectorization Advisor
Optimize for Intel® Advanced Vector Extensions 512 (Intel® AVX-512) with or without access to Intel AVX-512 hardware
20
19

Optimization Notice
38
Find Effective Optimization Strategies
Intel® Advisor—Cache-aware Roofline Analysis
Roofline Performance Insights
 Highlights poor performing loops
 Shows performance ‘headroom’ for
each loop
– Which can be improved
– Which are worth improving
 Shows likely causes of bottlenecks
 Suggests next optimization steps “I am enthusiastic about the new "integrated roofline" in Intel®
Advisor. It is now possible to proceed with a step-by-step
approach with the difficult question of memory transfers
optimization & vectorization which is of major importance.”Nicolas Alferez, Software Architect
Onera – The French Aerospace
Lab

Optimization Notice
39
Design It, Tune, Debug, Then Implement
Intel® Advisor Thread Prototyping—Design with Disrupting Development
Have You
 Threaded an app, but seen little benefit?
 Hit a “scalability barrier?”
 Delayed release due to synchronization errors?
Data Driven Threading Design
 Quickly prototype multiple options
 Project scaling on larger systems
 Find synchronization errors before implementing threading
 Design without disrupting development
Add Parallelism with Less Effort, Less Risk & More Impact
“Intel® Advisor allowed us to quickly
prototype ideas for parallelism, saving
developer time & effort”
Simon Hammond
Senior Technical Staff
Sandia National Laboratories

Optimization Notice
Visualize Parallelism—Interactively Build, Validate & Analyze
Algorithms
Intel® Advisor—Flow Graph Analyzer (FGA)
 Visually generate code stubs
 Generate parallel C++ programs
 Click & zoom through your algorithm’s nodes &
edges to understand parallel data & program flow
 Analyze load balancing, concurrency, & other
parallel attributes to fine tune your program
Use Intel® TBB or OpenMP* 5 (draft) OMPT APIs
40

Optimization Notice
Books
42
Intel Xeon Phi
Coprocessor High-
Performance
Programming
by James Jeffers and
James Reinders
High Performance
Parallelism Pearls:
Multicore and Many-
core Programming
Approaches
by James Reinders and
James Jeffers
Structured Parallel
Programming: Patterns
for Efficient Computation
by Michael McCool and
James Reinders
High Performance
Parallelism Pearls
Volume Two: Multicore
and Many-core
Programming
Approaches
by Jim Jeffers and James
Reinders

Optimization Notice
43
Intel® Parallel Studio XE
 Overview, features, support, code samples
 Training materials, Tech.Decoded webinars,
how-to videos & articles
 Reviews & Case Studies
 More Intel® Software Development Products
Intel Code Modernization Program
 Overview
 Live training
 Remote Access
Optimize Efficiently with Valuable Resources
Sign up
now
https://intel.ly/2PdkNhN
Shortcut Optimization
Attend TEC Webinars!

44
Get the Most from Your Code Today with Intel® Tech.Decoded
Discover Intel’s vision for
key development areas.
Big Picture Videos
Essential Webinars
Gain strategies, practices and
tools to optimize application
and solution performance.
Learn how to do specific
programming tasks using
Intel® tools.
Quick Hit How-tos
Visual Computing
Code Modernization
Systems & IoT
Data Science
Data Center & Cloud
Computing
TOPICS:
Visit TechDecoded.intel.io to learn how to
put key optimization strategies into practice
with Intel development tools.

Optimization Notice
45
Summary
Moore’s Law – 50 years and still going
 Impact: economic, technological, societal
 No more Free Lunch - Parallel (vectorize) or Perish
Intel Support
 Training
 Content
 Tools

Optimization Notice
Give your
customers the
performance
they want for
their code

Optimization Notice
47

Optimization Notice
INFORMATION IN THIS DOCUMENT IS PROVIDED “AS IS”. NO LICENSE, EXPRESS OR IMPLIED, BY ESTOPPEL OR OTHERWISE, TO ANY
INTELLECTUAL PROPERTY RIGHTS IS GRANTED BY THIS DOCUMENT. INTEL ASSUMES NO LIABILITY WHATSOEVER AND INTEL DISCLAIMS ANY
EXPRESS OR IMPLIED WARRANTY, RELATING TO THIS INFORMATION INCLUDING LIABILITY OR WARRANTIES RELATING TO FITNESS FOR A
PARTICULAR PURPOSE, MERCHANTABILITY, OR INFRINGEMENT OF ANY PATENT, COPYRIGHT OR OTHER INTELLECTUAL PROPERTY RIGHT.
Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as
SYSmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of those
factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated
purchases, including the performance of that product when combined with other products.
Copyright © 2015, Intel Corporation. All rights reserved. Intel, Pentium, Xeon, Xeon Phi, Core, VTune, Cilk, and the Intel logo are trademarks of Intel
Corporation in the U.S. and other countries.
Materials distributed for lab sessions may redistribute source codes obtained under various Open Source licenses with additional materials supporting the lab
distributed under the Intel Sample Source Code License. A copy of this latter license should be an included file within the distribution and covers this
presentation along with lab source samples. The passed-through Open Source projects might not perform as well as the originals, since lab preparation
may include the de-optimization of certain sections to provide lab examples for analysis tool exercises.
48
Legal Disclaimer & Optimization Notice
Optimization Notice
Intel’s compilers may or may not optimize to the same degree for non-Intel microprocessors for optimizations that are not unique to Intel microprocessors.
These optimizations include SSE2, SSE3, and SSSE3 instruction sets and other optimizations. Intel does not guarantee the availability, functionality, or
effectiveness of any optimization on microprocessors not manufactured by Intel. Microprocessor-dependent optimizations in this product are intended for use
with Intel microprocessors. Certain optimizations not specific to Intel microarchitecture are reserved for Intel microprocessors. Please refer to the applicable
product User and Reference Guides for more information regarding the specific instruction sets covered by this notice.
Notice revision #20110804

Developing Performance-Oriented Code: Moore's Law Over 50

Developing Performance-Oriented Code: Moore's Law Over 50

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Ähnlich wie Developing Performance-Oriented Code: Moore's Law Over 50

Ähnlich wie Developing Performance-Oriented Code: Moore's Law Over 50 (20)

Mehr von SmartBear

Mehr von SmartBear (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Developing Performance-Oriented Code: Moore's Law Over 50

Hinweis der Redaktion