1. BMW Group
Dr. Norbert Grün
International
Supercomputer
Conference
June 26-29, 2007
Dresden, Germany
Page 1
HPC in Car and Motorcycle Aerodynamics at BMW.
Outline
Aerodynamic
Process
Simulation
Process
Validation
Examples
Application
Examples
Conclusion
Efficient
Dynamics
HPC in Car and Motorcycle
Aerodynamics at BMW.
HPC in Car and Motorcycle
Aerodynamics at BMW.
International
Supercomputer
Conference
June 26-29, 2007
Dresden, Germany
Dr. Norbert Grün
Aerodynamics
Simulation

2. BMW Group
Dr. Norbert Grün
International
Supercomputer
Conference
June 26-29, 2007
Dresden, Germany
Page 2
HPC in Car and Motorcycle Aerodynamics at BMW.
Outline.
Aerodynamic Development.
Simulation Process.
Hardware Resources.
Various Applications.
Conclusion.
Outline
Aerodynamic
Development
Simulation
Process
Hardware
Resources
Various
Application
Conclusion

3. BMW Group
Dr. Norbert Grün
International
Supercomputer
Conference
June 26-29, 2007
Dresden, Germany
Page 3
HPC in Car and Motorcycle Aerodynamics at BMW.
Questions & Tools in Aerodynamic Development.
(Pre) Initial Phase Concept Phase Serial Development
Level of Detail
Tool
Question
Model
Simplified
Underhood
Simplified
Underhood
If necessary, details
from predecessor
If necessary, details
from predecessor
Fully detailedFully detailed
Simulation (CFD = Computational Fluid Dynamics)Simulation (CFD = Computational Fluid Dynamics)
Wind TunnelWind Tunnel
Road TestRoad Test
Integral Forces and MomentsIntegral Forces and Moments
Thermal ManagementThermal Management
Soiling, Snow DepositionSoiling, Snow Deposition
Dynamic Properties (Unsteady Aerodynamics)Dynamic Properties (Unsteady Aerodynamics)
VirtualVirtual
1:2.51:2.5 1:11:1
Outline
Aerodynamic
Development
Simulation
Process
Hardware
Resources
Various
Application
Conclusion

4. BMW Group
Dr. Norbert Grün
International
Supercomputer
Conference
June 26-29, 2007
Dresden, Germany
Page 4
HPC in Car and Motorcycle Aerodynamics at BMW.
CFD (Computational Fluid Dynamics) Method.
PowerFLOW™ Key Features
Lattice-Boltzmann Method
(Release 3.x with 34 States, 4.x using 19 states)
Transient simulation.
Low Re-#: Direct simulation without model assumptions.
High Re-#: VLES (Very Large Eddy Simulation) in the fluid.
Boundary Layer modeled by an extended log-law
(accounting for local pressure gradients).
No manual meshing required: Automatic volume discretization
using cubic cells (lattice) of different size (variable resolution).
Automatic decomposition for parallel processing.
Stable solutions.
Outline
Aerodynamic
Development
Simulation
Process
Hardware
Resources
Various
Application
Conclusion

5. BMW Group
Dr. Norbert Grün
International
Supercomputer
Conference
June 26-29, 2007
Dresden, Germany
Page 5
HPC in Car and Motorcycle Aerodynamics at BMW.
Simulation Process (PowerFLOW).
Geometry Group
Aerodynamics Group
Simulation
PowerFLOW
≈ 1 Day
Simulation
PowerFLOW
≈ 1 Day
Postprocessing
PowerVIZ
Postprocessing
PowerVIZ
ResultResult
Shape Modification
of CAD/CAS Data
Shape Modification
of CAD/CAS Data
Morphing of the
Surface Mesh
(PowerCLAY)
Morphing of the
Surface Mesh
(PowerCLAY)
Turnaround
2-14 Days
Turnaround
2-14 Days
CAD Model
CATIA/PRISMA
CAD Model
CATIA/PRISMA
U-Hood/U-BodyU-Hood/U-Body
CAS Model
ALIAS
CAS Model
ALIAS
Clay Model
POLYWORKS
Clay Model
POLYWORKS
Styling ExteriorStyling Exterior
Simulation Model
(Surface Facetization)
ANSA, PolyWorks, PowerWRAP, ...
1- 10 Days
Simulation Model
(Surface Facetization)
ANSA, PolyWorks, PowerWRAP, ...
1- 10 Days
Outline
Aerodynamic
Development
Simulation
Process
Hardware
Resources
Various
Application
Conclusion

6. BMW Group
Dr. Norbert Grün
International
Supercomputer
Conference
June 26-29, 2007
Dresden, Germany
Page 6
HPC in Car and Motorcycle Aerodynamics at BMW.
Geometry Input (Facetized Components).
Outline
Aerodynamic
Development
Simulation
Process
Hardware
Resources
Various
Application
Conclusion

7. BMW Group
Dr. Norbert Grün
International
Supercomputer
Conference
June 26-29, 2007
Dresden, Germany
Page 7
HPC in Car and Motorcycle Aerodynamics at BMW.
Geometry Modification by „Morphing“.
Outline
Aerodynamic
Development
Simulation
Process
Hardware
Resources
Various
Application
Conclusion

8. BMW Group
Dr. Norbert Grün
International
Supercomputer
Conference
June 26-29, 2007
Dresden, Germany
Page 8
HPC in Car and Motorcycle Aerodynamics at BMW.
Automatic Discretization.
Typical cell counts for external aerodynamic
cases range from 20-80 milion cells.
Outline
Aerodynamic
Development
Simulation
Process
Hardware
Resources
Various
Application
Conclusion
Geometry
representation
embedded in a
lattice of cubic cells
(with different
levels of resolution).

9. BMW Group
Dr. Norbert Grün
International
Supercomputer
Conference
June 26-29, 2007
Dresden, Germany
Page 9
HPC in Car and Motorcycle Aerodynamics at BMW.
Transient Simulation.
Time
DragandLiftCoefficients[-]
Drag (Sampling Rate 13 Hz)
Lift (Sampling Rate 13 Hz)
Drag (Sampling Rate 100 Hz)
Lift (Sampling Rate 100 Hz)1 second
0.100
0.010
Drag Time Average
Lift Time Average
Simulation time steps are in the order of 10-5
to 10-4
seconds
Outline
Aerodynamic
Development
Simulation
Process
Hardware
Resources
Various
Application
Conclusion

10. BMW Group
Dr. Norbert Grün
International
Supercomputer
Conference
June 26-29, 2007
Dresden, Germany
Page 10
HPC in Car and Motorcycle Aerodynamics at BMW.
History of Computer Resources.
288 288
416
253
224
8
24 24
48
96
224
0
50
100
150
200
250
300
350
400
450
1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007
Number of Cores
1200 full car simulations accomplished by 30 users
Data production rate ≈ 20 TB/year
Outline
Aerodynamic
Development
Simulation
Process
Hardware
Resources
Various
Application
Conclusion

11. BMW Group
Dr. Norbert Grün
International
Supercomputer
Conference
June 26-29, 2007
Dresden, Germany
Page 11
HPC in Car and Motorcycle Aerodynamics at BMW.
History of Computer Resources.
Dedicated PowerFLOW Server
Outline
Aerodynamic
Development
Simulation
Process
Hardware
Resources
Various
Application
Conclusion

12. BMW Group
Dr. Norbert Grün
International
Supercomputer
Conference
June 26-29, 2007
Dresden, Germany
Page 12
HPC in Car and Motorcycle Aerodynamics at BMW.
0,5
0,6
0,7
0,8
0,9
1,0
1,1
1,2
1,3
1,4
1,5
1,6
1,7
1,8
1,9
2,0
32 48 64 80 96 112 128
Number of Cores
Speedup Linear
PowerFLOW 3.6a
PowerFLOW 4.0a
Parallel Efficiency.
Benchmark „medium“ on HP Superdome (Montecito, 1.6GHz)
Outline
Aerodynamic
Development
Simulation
Process
Hardware
Resources
Various
Application
Conclusion

13. BMW Group
Dr. Norbert Grün
International
Supercomputer
Conference
June 26-29, 2007
Dresden, Germany
Page 13
HPC in Car and Motorcycle Aerodynamics at BMW.
Case Size.
0
500
1.000
1.500
2.000
2.500
3.000
3.500
2001 2002 2003 2004 2005 2006 2007
CaseComplexity[GVoTS]
The size of a case (determining the
computational effort) is expressed as
the product of cells (voxels) and timesteps
GVoTS = Giga Voxel Time Steps
The size of a case (determining the
computational effort) is expressed as
the product of cells (voxels) and timesteps
GVoTS = Giga Voxel Time Steps
Outline
Aerodynamic
Development
Simulation
Process
Hardware
Resources
Various
Application
Conclusion
10·106 cells x 50·103 TS
Simple Models, isothermal
10·106 cells x 50·103 TS
Simple Models, isothermal
20·106 cells x 50·103 TS
Detailed Uhood & U-body, isothermal
20·106 cells x 50·103 TS
Detailed Uhood & U-body, isothermal
30·106 cells x 100·103 TS
Fully detailed incl. heat transfer
30·106 cells x 100·103 TS
Fully detailed incl. heat transfer

14. BMW Group
Dr. Norbert Grün
International
Supercomputer
Conference
June 26-29, 2007
Dresden, Germany
Page 14
HPC in Car and Motorcycle Aerodynamics at BMW.
Performance Development.
The common performance measure of FLOPS
does not help us to predict expected runtimes.
Instead we use the ratio of GVoTS / CPU-h
0,45
0,30
0,60
0,80
0,20
0,30
0,40
0,50
0,60
0,70
0,80
0,90
Jan 02 Jan 03 Jan 04 Jan 05 Jan 06 Jan 07
CorePerformanceinGVoTS/CPU-h
SGI Origin
R14000 / 0.6GHz
HP Superdome
Madison / 1.5GHz
SGI Altix
Itanium 2 / 1.6GHz
HP Superdome
Montecito / 1.6GHz
Benchmark „medium“ (21 Mio. Voxels)
Outline
Aerodynamic
Development
Simulation
Process
Hardware
Resources
Various
Application
Conclusion

15. BMW Group
Dr. Norbert Grün
International
Supercomputer
Conference
June 26-29, 2007
Dresden, Germany
Page 15
HPC in Car and Motorcycle Aerodynamics at BMW.
Comparison of Simulation and Experiment.
-0,100
-0,050
0,000
0,050
0,100
0,150
0,200
0,250
0,300
0,350
0,400
0,450
0,500
DragandLiftCoefficients
Cx PowerFLOW
Cx Wind Tunnel
Cz-rear PowerFLOW
Cz-rear Windtunnel
Array of different Vehicles
Outline
Aerodynamic
Development
Simulation
Process
Hardware
Resources
Various
Application
Conclusion

16. BMW Group
Dr. Norbert Grün
International
Supercomputer
Conference
June 26-29, 2007
Dresden, Germany
Page 16
HPC in Car and Motorcycle Aerodynamics at BMW.
Lift Analysis.
-0,03
-0,02
-0,01
0,00
0,01
0,02
0,03
0,0 0,1 0,3 0,4 0,5 0,6 0,7 0,9 1,0
-0,40
-0,30
-0,20
-0,10
0,00
0,10
0,20
0,0
Cz(x) Verteilung
Cz(x) Integral
CZ1
CZ2
0.011 0.013
0.143 0.123
CFD (PowerFLOW)
BMW Wind Tunnel
Outline
Aerodynamic
Development
Simulation
Process
Hardware
Resources
Various
Application
Conclusion

17. BMW Group
Dr. Norbert Grün
International
Supercomputer
Conference
June 26-29, 2007
Dresden, Germany
Page 17
HPC in Car and Motorcycle Aerodynamics at BMW.
Flow Field Visualization.
Outline
Aerodynamic
Development
Simulation
Process
Hardware
Resources
Various
Application
Conclusion

18. BMW Group
Dr. Norbert Grün
International
Supercomputer
Conference
June 26-29, 2007
Dresden, Germany
Page 18
HPC in Car and Motorcycle Aerodynamics at BMW.
Passenger Comfort.
Simulation with Screen Simulation without ScreenDraft Test Dummies in the Wind Tunnel
Outline
Aerodynamic
Development
Simulation
Process
Hardware
Resources
Various
Application
Conclusion
Transient Isosurface
Vx=0 (Reverse Flow)

19. BMW Group
Dr. Norbert Grün
International
Supercomputer
Conference
June 26-29, 2007
Dresden, Germany
Page 19
HPC in Car and Motorcycle Aerodynamics at BMW.
Temperature Loads (from Oil Cooler and Exhaust).
Outline
Aerodynamic
Development
Simulation
Process
Hardware
Resources
Various
Application
Conclusion

20. BMW Group
Dr. Norbert Grün
International
Supercomputer
Conference
June 26-29, 2007
Dresden, Germany
Page 20
HPC in Car and Motorcycle Aerodynamics at BMW.
Exhaust Gas Propagation.
X5
oldOutline
Aerodynamic
Development
Simulation
Process
Hardware
Resources
Various
Application
Conclusion
Isosurfaces of Exhaust Gas with T=50°C Time averaged Flow Field
X5
new

21. BMW Group
Dr. Norbert Grün
International
Supercomputer
Conference
June 26-29, 2007
Dresden, Germany
Page 21
HPC in Car and Motorcycle Aerodynamics at BMW.
Detail Optimization.
Wing MirrorSub-Simulation Volume
Outline
Aerodynamic
Development
Simulation
Process
Hardware
Resources
Various
Application
Conclusion
Reduced Effort due to
Sub-Simulations around
Details like Wing Mirrors,
Wheelhouse, etc.

22. BMW Group
Dr. Norbert Grün
International
Supercomputer
Conference
June 26-29, 2007
Dresden, Germany
Page 22
HPC in Car and Motorcycle Aerodynamics at BMW.
Aerodynamic Forces on Parts.
Magnitude, Direction and Point of Incidence
Outline
Aerodynamic
Development
Simulation
Process
Hardware
Resources
Various
Application
Conclusion

23. BMW Group
Dr. Norbert Grün
International
Supercomputer
Conference
June 26-29, 2007
Dresden, Germany
Page 23
HPC in Car and Motorcycle Aerodynamics at BMW.
Aerodynamic Forces on Parts.
TouringTouring StandardStandard
SportSport
Outline
Aerodynamic
Development
Simulation
Process
Hardware
Resources
Various
Application
Conclusion StandardStandard
Forces on the Driver‘s Helmet
with different Windshields.
Helmkraft
SportSportTouringTouring

24. BMW Group
Dr. Norbert Grün
International
Supercomputer
Conference
June 26-29, 2007
Dresden, Germany
Page 24
HPC in Car and Motorcycle Aerodynamics at BMW.
Aerodynamics and Stability.
Simulation of Gusty Environments.
Gust simulated via time dependent cross flow velocity
travelling downstream over the car.
Outline
Aerodynamic
Development
Simulation
Process
Hardware
Resources
Various
Application
Conclusion

25. BMW Group
Dr. Norbert Grün
International
Supercomputer
Conference
June 26-29, 2007
Dresden, Germany
Page 25
HPC in Car and Motorcycle Aerodynamics at BMW.
Aerodynamics and Stability.
Gust Response of different Vehicles.
-0,04
-0,02
0,00
0,02
0,04
0,06
0,08
0,10
1,0 1,5 2,0 2,5 3,0 3,5 4,0 4,5
Time [s]
YawingMomentCoefficient
Car A
Car B
CMZ >0
CMZ <0
Outline
Aerodynamic
Development
Simulation
Process
Hardware
Resources
Various
Application
Conclusion

26. BMW Group
Dr. Norbert Grün
International
Supercomputer
Conference
June 26-29, 2007
Dresden, Germany
Page 26
HPC in Car and Motorcycle Aerodynamics at BMW.
Wind Tunnel Design.
Outline
Aerodynamic
Development
Simulation
Process
Hardware
Resources
Various
Application
Conclusion

27. BMW Group
Dr. Norbert Grün
International
Supercomputer
Conference
June 26-29, 2007
Dresden, Germany
Page 27
HPC in Car and Motorcycle Aerodynamics at BMW.
Conclusion.
ADVANTAGES
+ Significant progress has been made in the past five years
and the field of application has broadened.
+ The level of detail that can be handled and the achievable accuracy
permit productive usage as a complementary tool to the wind tunnel.
+ Usage does not require a numerics expert,
CFD can be employed by the aerodynamicist.
SHORTCOMINGS
- Detail optimization loops with CFD still slower than the wind tunnel.
Outline
Aerodynamic
Development
Simulation
Process
Hardware
Resources
Various
Application
Conclusion
- Computer hardware requirements are very high for competitive process
times, although the per-processor performance has more than doubled in
the past five years.

28. HPC in Car and Motorcycle
Aerodynamics at BMW.
Thank You for Your Attention.
Efficient
Dynamics
International
Supercomputer
Conference
June 26-29, 2007
Dresden, Germany
Dr. Norbert Grün
Aerodynamics
Simulation