HVAC for Data Centers

HVAC for Data Centers
Sponsored by:
Join the discussion about this Webcast on Twitter at #CSEdatacenterHVAC

Learning Objectives:
1.The audience will learn about codes and guidelines, such as
ASHRAE 90.1: Energy Standard for Buildings Except Low-
Rise Residential Buildings, and U.S. Green Building Council
LEED v4
2.Attendees will learn the relationships between HVAC
efficiency and power usage effectiveness (PUE)
3.Viewers will understand the advantages and drawbacks of
using an elevated IT equipment inlet temperature
4.Viewers will learn how running IT equipment at partial load
affects data center energy efficiency.

Bill Kosik, PE, CEM, BEMP, LEED AP BD+C,
HP Enterprise Business, Technology Services,
Chicago, Ill.
Tom R. Squillo, PE, LEED AP,
Environmental Systems Design Inc.,
Chicago, Ill.
Moderator: Jack Smith,
Consulting-Specifying Engineer and Pure Power,
CFE Media, LLC
Presenters:

Energy Code Requirements
for Data Centers
Chicago, Ill.

Energy Code Requirements for Data Centers
International Energy Conservation
Code: IECC
• Adopted by eight states and many local
jurisdictions
• Written in enforceable code language
ASHRAE Energy Standard for
Buildings: ASHRAE 90.1
• Standard instead of code. Now written
in enforceable language so it can be
adopted locally
• Has more language specific to data
centers
California Building Energy Efficiency
Standards: Title 24

Where Do They Apply?
• Check local jurisdiction for specific
requirements or city energy codes
• Many local jurisdictions refer to
state codes
• IECC allows compliance with
ASHRAE 90.1 instead. This may
be an advantage in some instances
• Title 24 compliance required in
California

Current Energy Code Adoption Status (U.S. DOE)

Projected Energy Code Adoption by end of 2015
(U.S. DOE)

IECC – 2012:
• IECC delineates between simple
and complex systems. Stand
alone DX ac units may fall under
the simple category. Only very
small units under 33,000 Btu/h
capacity do not require
economizers
• All cooling systems with some
form of common piping distribution
fall under the complex category
• All complex systems require
economizers

IECC – 2012:
C403.4.1 Economizers. Economizers shall comply with Sections
C403.4.1.1 through C403.4.1.4
• This section requires either an air or water economizer
C403.4.1.1 Design capacity. Water economizer systems shall be
capable of cooling supply air by indirect evaporation and
providing up to 100% of the expected system cooling load at
outdoor air temperatures of 50 F dry bulb/45 F wet bulb and
below.
• Exception for small systems below 33,000 Btu/h
• Unlike ASHRAE 90.1, IECC has no specific exceptions for
data centers that allow lower dry-bulb/wet-bulb temperatures.
Dry-coolers are not allowed.

ASHRAE 90.1-2010
• Data centers, considered “process
cooling,” were excluded from the
requirements of ASHRAE 90.1-2007
• The 2010 version eliminates the
“process cooling” exemption, and
adds specific language for computer
rooms
• To comply with the IECC-2012 code,
ASHRAE 90.1-2010 may be used
instead.

ASHRAE 90.1 – 2010:
6.5.1 Economizers. Each Cooling System that has a fan shall include
either an air or water economizer meeting the requirements of Sections
6.5.1.1 through 6.5.1.4.
For data centers, economizers are not required for:
• Small fan-cooling units less than 135,000 Btu/hr or 65,000 Btu/hr,
depending on climate zone
• Extremely hot and humid climate zones
• Buildings with no central CHW plant, in which the total computer room
cooling capacity is less than 250 tons
• Buildings with a central CHW plant, and the computer room cooling load
is less than 50 tons
• Where cooling towers are not allowed
• Addition of less than 50-ton computer room capacity to existing building
• Various essential facilities (national defense, emergency response, etc.)

ASHRAE 90.1 – 2010:
6.5.1.2 Water Economizers
6.5.1.2.1 Design capacity. Water economizer systems shall be
capable of cooling supply air by indirect evaporation and by
providing up to 100% of the expected system cooling load at
outdoor air temperatures of 50 F dry bulb/45 F wet bulb and below.
• For data centers, the requirement is relaxed slightly to allow
100% economizer cooling at 40 F dry bulb/35 F wet bulb and
below
• The code also allows dry-coolers for data centers, but they must
provide 100% economizer cooling at 35 F dry bulb

Important Changes to ASHRAE 90.1-2013:
6.5.1.2 Water Economizers
• For data centers, the outdoor temperature limits for 100% water
side economization are not a single condition, but are based on
the individual climate zones
6.5.1.6 Economizer Humidification Impact. Systems with
hydronic cooling and humidification systems designed to maintain
inside humidity at a dew-point temperature greater than 35 F shall
use a water economizer if an economizer is required by Section
6.5.1.
• This essentially bans air side economizer systems for most
data center systems if using a prescriptive approach.

Important Changes to ASHRAE 90.1-2013:
6.6 Alternative Compliance Path
• For data centers, the HVAC systems can comply by meeting
minimum PUE requirements instead of Section 6.5-Prescriptive
Path.
• The minimum PUE values are based on the climate zone and
range from 1.30 to 1.61.
• PUE calculation is based on Green Grid Recommendation
document dated May, 2011.
• Option 1: Use peak PUE calculation (at 50% and 100% IT load)
• Option 2: Use annual PUE, calculated with an approved hourly
energy analysis program (DOE, BLAST, EnergyPlus, etc.)

Title 24 – 2013: Highlights
Specific to Data Centers
• Data centers are exempt from normal
economizer requirements
• Air or water economizer required
• Air economizer must provide 100%
economization at 55 F dry bulb
• Water economizer must provide 100%
economization at 40 F dry bulb/35 F
wet bulb
• Economizer exceptions exist for small
systems
• Nonadiabatic humidification (steam,
infrared) is prohibited

Title 24 – 2013: Highlights
Specific to Data Centers
• Variable speed supply fans required
for DX systems over 5 tons and all
CHW systems
• Supply fans shall vary airflow rate as a
function of actual load
• Containment required for data centers
with a design load exceeding 175
W/sq ft
• Containment exception for expansions
and racks below 1 W/sq ft
• Chilled water plants can have no more
than 300 tons of air-cooled chillers

Relationships Between HVAC
Efficiency and Power Usage
Effectiveness (PUE)
Chicago, Ill.

• Extreme regional variations in CO2 from electricity generation
• Determine appropriate balance of water and electricity usage
• Climate WILL impact HVAC energy use – select sites carefully
• Use evaporative cooling where appropriate
• Economizer strategy will be driven from climate characteristics
• Design power and cooling modularity to match IT growth
• Plan for power-aware computing equipment
• Use aisle containment or direct-cooled cabinets
• Design in ability to monitor and optimize PUE in real time
• Push for highest supply temperatures and lowest moisture levels
• Identify tipping point of server fan energy/inlet temperature
• Minimize data center footprint by using high-density architecture
DataCenterClimateSynergiesConvergence
Levels of Optimization

Air Cooled
Chiller
coupled to chilled
water coil in air
handling unit
(AHU)
Direct
Expansion
packaged in ahu
or separate DX
coil & remote
condenser
Water Cooled
Chiller
coupled with chw coil
in ahu. typical with
open CT & flat plate
HX
Air Cooled
Chiller
coupled to chw
coil in ahu.
typical with
closed CT
Interior AHU
direct outside
air with direct
evaporative
cooling
Exterior AHU
indirect outside
air and indirect
evaporative
cooling
CRAH
Unit
perimeter
air deliver
with chilled
water coil
In-Row Unit
close-coupled in
rack containment
system with
module fans and
chw coil
Rear Door
HX
Individual rack
door chw HX.
Passive
system with
no fan
Overhead
Coil
Module chw
coils. Passive
system with
no fan
Typical Data Center Cooling Strategies
Air Side Economizers Water Side Economizers
System 1 – DEC
Direct Outside Air
Economizer with
Direct Evaporative
Cooling
System 2 – IEC
Recirculating
(Closed) Air System
with Indirect
Evaporative Cooling
System 3 – IDEC
Recirculating (Closed) and
Direct Outside Air System 2
Stage Indirect-Direct
Evaporative Cooling
System 4 – IOA+EC
Indirect Air to Air HX
with Direct
Evaporative Cooling
in Secondary Air
System 5 –
OCT+FP HX
Direct (Open)
Evaporative Cooling
Tower with Flat Plate
System 6 – CCT
w/Spray
Indirect (Closed)
Cooling Tower with
Spray
Mechanical Cooling Options Mechanical Cooling Options
Cooling Configuration Options
Cooling Configuration Options

0.35 difference in PUE based
on climate and cooling system
type
PUE Varies with Climate

Impacts of Climate on Economization Strategy
This analysis shows the percent of total ton-hours that require mechanical cooling.
The graphs depict two systems with two water temperatures, 12°C and 18°C:
1. Air-cooled chiller with dry-cooled economization
2. Air-cooled chiller with evaporative-cooler economization

The indirect evaporative and indirect air cooling systems have the lowest
compressor energy used to cool the facility. The air-cooled DX and air-cooled
chiller systems have the highest compressor energy. The air-cooled chiller with
economizer is in the middle of the other options.
Impacts of Climate on Economization Strategy
Santiago, CHL

HVAC System and PUE
Five HVAC options are shown. Each option was analyzed using the same input
parameters such as climate attributes, air and water temperatures, etc. Each
system performs differently based on the inherent strategies used to cool the
data center and provide the proper airflow. For each option, the annual HVAC
consumption and annual PUE us shown.

HVAC System and PUE
Two options are shown.
The only difference
between the two options
is the location of the data
centers. Everything else,
including power, cooling,
and ancillary systems are
modelled identically.
Month by month PUE
values are shown as well
as monthly HVAC,
electrical losses, lighting
and other electrical
energy. The energy
consumption of the data
center located in
Singapore is markedly
higher based on the hot
and humid climate.

Elevated IT Equipment Temperatures
Chicago, Ill.

Elevated IT Equipment Inlet Temperatures
Legacy Data Center Design
• Data center supply air set to
50 F to 55 F
• DX systems cycled on/off and
fought each other, with little
capacity control or
communication
• Chilled water temperatures of
40 F to 45 F
• No containment
• Wide variation of
temperatures entering IT
equipment

Why Were Low Supply Temperatures Needed?
• Design needed to take into account massive mixing of hot air with supply air
• Temperature of air entering IT equipment at tops of racks still acceptable
• Cold room allowed some ride-through if cooling failed
Why Was This Bad?
• Wastes Energy
– Too much airflow (low delta T)
– Inefficient chiller operation
– Limited economizer use
– Unnecessary dehumidification
• Hot spots
• Inconsistent temperature control
• Inconsistent humidity control Source: 42u.com

ASHRAE Thermal Guidelines
• Recommended range for IT inlet conditions
– Temperature: 64 F to 80.6 F
– Humidity: 41.9 F to 59 F dew point or 60% RH
• Extended range for other classes of IT equipment
Source: 42u.com

Advantages of Elevated Temperatures
• Increased equipment efficiency
– 1.5% to 2.5% increase in chiller efficiency per degree increase in chilled water
temperature
– Increasing CHW supply temperature from 50 F to 60 F decreases chiller energy by up to
25%
– Actual increase depends on chiller type and selection
• Decreased unwanted dehumidification at air handling units
– Coil temperature never gets below dew point if CHW temperature is raised
– Eliminates condensate removal issues
• Additional economizer hours
– Actual advantage highly dependent on climate and system type
– Longer equipment life

Air Side Economizer System: Phoenix, 60 F SA Temperature
• Economization
available for 5,038
hr/yr
• Chillers off when
outside air
temperature is below
60 F (1,910 hr)

• Economization
available for 7,396
hr/yr
• Huge gain in
economizer hours
due to dry climate
outside air
75 F (4,294 hr)
Air Side Economizer System: Phoenix, 75 F SA Temperature

Air Side Economizer System: Charlotte, 60 F SA Temperature
• Economization
available for 5,300
hr/yr
• More hours of full
economization
(3,778 hr) than
Phoenix

Air Side Economizer System: Charlotte, 75 F SA Temperature
• Economization
available for 5,630
hr/yr
• Due to humid climate,
increase in
economizer hours is
minimal
outside air
75 F (5,300 hr)

Water Side Economizer System: Phoenix, 60 F SA Temperature
• Economization
available for 3,829
hr/yr
outside air wet bulb
33 F (55 hr)

Water Side Economizer System: Phoenix, 75 F SA Temperature
• Economization
available for 8,629
hr/yr
• Huge gain in
economizer hours
due to dry climate
outside wet bulb
53 F (4,420 hr)

Water Side Economizer System: Charlotte, 60 F SA Temperature
• Economization
available for 4,174
hr/yr
outside wet bulb
33 F (1,009 hr)

Water Side Economizer System: Charlotte, 75 F SA Temperature
• Economization
available for 8,334 hr/yr
• water side economizer
has huge increase in
economizer hours
because less hours are
locked out due to OA
humidity
outside wet bulb
53 F (4,513 hr)

Bin Data Energy Use Analysis
• Design Criteria
– Typical enterprise/co-location data center load
• 10,000 sq ft
• 200 W/sq ft
• 2 MW of total IT load
– Within ASHRAE recommended conditions
• Supply temperature = 75 F
• Return temperature = 95 F
• Space dew point temperature between 42 F and 59 F
• Efficient adiabatic humidification used for analysis

Two Systems and two climates analyzed and compared:
System Options:
1. Direct Outside Air Economizer
• Multiple 500 kW capacity rooftop supply/exhaust AHUs
• OA Economizer control
• Air-cooled chiller system for supplemental cooling
2. Water-cooled Chillers with Cooling Towers
• High efficiency variable speed chillers (0.4 kW/ton)
• Induced draft cooling towers
• Plate and frame heat exchangers in series w/chillers
• CRAH units with high efficiency EC fans on raised floor
Climates:
1. Phoenix – Hot and dry
2. Charlotte – Warm and humid

MPUE
1.20
1.30
1.17
1.26
1.15
1.22
1.13
-
1,000,000
2,000,000
3,000,000
4,000,000
5,000,000
6,000,000
Direct
Outdoor Air
WC
Chillers/CT
Direct
Outdoor Air
WC
Chillers/CT
Direct
Outdoor Air
WC
Chillers/CT
Direct
Outdoor Air
WC
Chillers/CT
60°F Supply Air 65°F Supply Air 70°F Supply Air 75°F Supply Air
Phoenix Energy Consumption (kWh)
Chiller CHW Pump
CW Pump AHU Fan
Tower Fan Supply Fan
Exhaust Fan Humidification
1.34

MPUE
1.24
1.19
1.22
1.17
1.20
1.15
1.19
1.13
-
1,000,000
2,000,000
3,000,000
4,000,000
5,000,000
6,000,000
Direct
Outdoor Air
WC
Chillers/CT
Direct
Outdoor Air
WC
Chillers/CT
Direct
Outdoor Air
WC
Chillers/CT
Direct
Outdoor Air
WC
Chillers/CT
60°F Supply Air 65°F Supply Air 70°F Supply Air 75°F Supply Air
Charlotte Energy Consumption (kWh)
Chiller CHW Pump
CW Pump AHU Fan
Tower Fan Supply Fan
Exhaust Fan Humidification

Disadvantages of elevated temperatures
• Working conditions in hot aisle
– Hot aisle temperatures may rise above 100 F in some cases
– OSHA requirements may come into effect
– Think about temporary spot cooling for technology workers
• Temperature ratings of cables and sprinkler heads in hot aisle
– Some cabling rated for 40 C (104 F)
• Reduced ride-through time during cooling failures
– Critical server temperatures can be reached in minutes or seconds in some
cases
– Good containment will help reduce hot air recirculation, though may starve
servers if system airflow is interrupted

Conclusions:
• Increasing IT inlet temperatures can help reduce overall
energy use substantially by:
– Increasing chiller efficiency (10 degree rise can increase
efficiency up to 25%)
– Reduce humidification requirements
– Huge increases in economizer hours
• Be careful of very high temperature conditions in the hot
aisles affecting worker comfort and equipment ratings
• Advantages highly dependent on climate and system type
– Look at the psych chart for economizer and lock-out hours
– air side and water side economizer systems will be affected
differently

Partial Loads
Chicago, Ill.

PUE Sensitivity to Low IT Loads
How running
IT equipment
at partial load
affects data
center energy
efficiency.

• Multiple systems allow for
growth without over-
provisioning
• Modularity lowers fan
energy and increases
compressor effectiveness
• Modularity is not the same
as spreading the load
across multiple pieces of
equipment
Efficiency Through Modularity

Electrical losses will increase as the IT load decreases.
This increase must be included in cooling load at
different loading points.
Efficiency Through Modularity

Electrical System Topology and System Efficiency

As the cooling load is distributed over an increasing number of chillers, the
overall power (and energy) grows. To maintain the highest efficiency, the
chillers should be run as close as possible to their peak efficiency point.

Servers Are More Efficient but Use More Power
Trends in Server Turn-Down Ratio

Server Modularity
45 hot-plug cartridges
Compute, Storage, or
Combination
x86 , ARM, or
Accelerator
• Single-server = 45 servers
per chassis
• Quad-server =180 servers per
chassis (future capability) –
that is 1800 servers per
cabinet or 45 kW
Dual low-latency
switches
• Switch Module (45 x
1 GB downlinks)
Compute, Storage, or
combination x86 , ARM, or
Accelerator

The PUE values are predicted using data center energy use simulation techniques. Many assumptions are
made which affect the predicted energy use and PUE. These ranges are meant to be indicators of the PUE
envelope that might be expected based on sub-system efficiency levels, geographic location and methods
of operations. Detailed energy use simulation is required to develop more granular and accurate analyses.
Input datafor "high"PUEcase
Singapore, SGP
UseWater Economizer NO
UseAdiabatic Cooling NO
Lighting (w/SF) 1.50
Misc Power (%of IT) 6.0%
Electrical System Loss (%) 10.0%
Air-cooled evap temp (°F) 65.0
Fan Pressure 2.0
Input datafor "low"PUEcase
Helsinki, FIN
UseWater Economizer YES
UseAdiabatic Cooling YES
Lighting (w/SF) 1.00
Misc Power (%of IT) 4.0%
Electrical System Loss (%) 8.5%
Air-cooled evap temp (°F) 65.0
Fan Pressure 2.0

Codes and Standards References from Today’s Webcast
• HVAC: ASHRAE 90.1: Energy Standard for Buildings
Except Low-Rise Residential Buildings
• HVAC: ASHRAE 62.1, 62.2, and Air Movement
• International Energy Conservation Code
• U.S. Green Building Council LEED v4
• California Building Energy Efficiency Standards: Title 24

Thanks to Today’s Webcast Sponsors:

Webcasts and Research
• Modular data center design
• HVAC: ASHRAE 62.1, 62.2, and Air
Movement
• 2013 HVAC, BAS state of the industry
report

HVAC for Data Centers

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie HVAC for Data Centers

Ähnlich wie HVAC for Data Centers (20)

Mehr von ConsultingSpecifyingEngineer

Mehr von ConsultingSpecifyingEngineer (15)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

HVAC for Data Centers