Cluster installation, management, and use made easy with Bright Computing and Intel Xeon Phi coprocessors

1. Martijn de Vries
CTO
Bright Cluster Manager
Managing Clusters with Intel Xeon Phi Coprocessors

2. Bright Cluster
Architecture
CMDaemon
SOAP+SSL
SOAP+SSL
Cluster
Management
GUI
Cluster
Management
Shell
Web-Based
User Portal
Third-Party
Applications
head node
node001
node003
node002

3. Cluster Management GUI Cluster Management ShellUser Portal
SSL / SOAP / X509 / IPtables
Cluster Management Daemon
SLES / RHEL / CentOS / SL
Bright Cluster Manager — Elements
SLES / RHEL / CentOS / SL
ScaleMP vSMP
Provisioning
Monitoring
Automation
Health Checks
Management
SLURM
Torque/Maui
Torque/MOAB
PBS Pro
Grid Engine
LSF
Compilers
Libraries
Debuggers
Profilers
CPU
MIC
Memory
PDU
IPMI/iLO
Interconnect
Ethernet
Disk

4. Management Interface
Graphical User Interface (GUI)
 Offers administrator full cluster control
 Standalone desktop application
 Manages multiple clusters simultaneously
 Runs natively on Linux, Windows and MacOS
Cluster Management Shell (CMSH)
 All GUI functionality also available through
Cluster Management Shell
 Interactive and scriptable in batch mode
Cluster
Management
GUI
Cluster
Management
Shell

6. Intel Phi Integration
 Everything needed to enable MIC on a cluster is packaged
as easy-to-install Bright packages:
• MIC driver
• MIC runtime
• MIC SDK
• MIC OFED
• MIC flash utilities
 Environment modules ensure that user environment is set
up perfectly (PATH, LD_LIBRARY_PATH, ...)
 MIC driver recompiled automatically against running kernel
at boot-time

7. Intel Phi Integration
 Set-up wizard takes care of initial MIC configuration
(e.g. creating bridge interfaces, assigning IP
addresses)
 MIC appears as a first-class device type in cluster
management infrastructure
 MIC can be configured, controlled and monitored
through CMSH and CMGUI
 MIC is automatically added to the workload
management system as a consumable resource
 Compute jobs may request MIC resource in job script

8. 8

13. Intel Phi Workload Management
 Three ways to run MIC jobs:
• Offload (i.e. MIC is used as accelerator from host)
• Native (i.e. job executes entirely on MIC)
• Symmetric (i.e. communicating processes on both host and
MIC)
 Offload: MIC represented as consumable resource in
workload management system
 Native: Ported Slurm to MIC
 Symmetric: work in progress, will require some
changes to workload managers
 Additional work in progress: make sure MIC is not
used in multiple modes simultaneously

14. Bright Cluster
Architecture — Monitoring
CMDaemon
metrics
data
Cluster
Management
GUI
Cluster
Management
Shell
Web-Based
User Portal
Third-Party
Applications
head node
node001
node003
node002
metrics
metrics
metrics
metrics
raw data consolidated
data
BMC
BMC
BMC

16. Cluster Health Management
 Goal: provide problem free environment for running jobs
 Regular health checks
• Actions that return PASS, FAIL or UNKNOWN
• Can be associated with a settable severity and a message
• Can launch an action based on any response value
 Pre-job health checks
 16 MIC health checks included by default
 Jobs will only be scheduled to nodes where MIC is working
properly (as determined by MIC health checks)
 Intel Cluster Checker included to verify that cluster is set
up properly

17. Bright Cluster Manager makes it easy
to install, manage and use clusters
with Intel Xeon Phi Coprocessors.

18. Questions?