Data Center Physical Infrastructure – Overview

Source: Energy University: Schneider Electric

Data Center Physical Infrastructure consists of the seven elements

  1. ⚡Power
  2. ❄️Cooling
  3. 🗄️Racks and Physical Structure
  4. 🛡️Security and 🔥Fire Protection
  5. 🔌Cabling
  6. 👨🏻‍💼Management
  7. 🛎️Services

There are 3 keys components to Data Center physical infrastructure

  1. Information Technology which consist of data processing, data storage and communications, both hardware and software
  2. Process which need to be defined and documented and standardized.
  3. People who have the right levels of skills and training with adequate manpower.

Some of the challenges face by Data Center today includes

  • Rapid changes in IT Technology
  • High density blade servers power consumption and head dissipation.
  • High operating and service cost
  • Regulatory requirements from Infocomm Media Development Authority (IMDA), Singapore Civil Defence Force (SCDF), Building and Construction Authority(BCA),etc
  • Server consolidation
  • Increasing availability expectations.
  1. Power

️Challenges:

  • Lifecycle Costs
  • Adaptability/ Scalability
  • Availability
  • Manageability
  • Maintenance / Serviceability

💡 Solutions:

Integration of Power subsystem components towards pre-engineered, pre-manufactured solutions. Such solutions should be

  • Modular
  • Standardized
  • Expandable

2.❄️Cooling

Cooling systems that are required to remove heat from data center are

  • Computer Room Air Conditioners (CRAC)
  • Associated subsystems components linked to CRAC
    • Chillers
    • Cooling Towers
    • Condensers
    • Duct works
    • Pumps
    • Pipe works
    • Rack-level distribution devices

Cooling Challenges

  • Lifecycle Costs
  • Adaptability/ Scalability
  • Availability
  • Manageability
  • Maintenance / Serviceability

Cooling Solutions

Move away from unique system design towards more open system design

Direct water cool system for high density racks above 10kw or large group of racks above 6kW

3. 🗄️Racks and Physical Structure

    The most critical physical elements are

  • IT racks, which contains the IT equipment such as servers, switches, routers, computer, etc
  • Physical room elements such as False ceiling and Raised Floor Systems

Electronic Industries Alliance EIA 310 is a common standard used for 19 inch rack mounting telecommunications and IT equipment.

Rack and Physical Structure Challenges

  • Lifecycle Costs
  • Adaptability/ Scalability
  • Availability
  • Manageability
  • Maintenance / Serviceability

Rack and Physical Structure Solutons

  • Be adaptable to changing requirements
  • Improve availability and cost effectiveness.

4. 🛡️Security and 🔥Fire Protection Systems

Security and Fire protection systems are required to protect the integrity, safety and availability of the Data Center.

Physical Security DevicesFire Detection and Suppression
Biometric devicesSmoke Detector
KeysClean Agent(Gas) Fire Suppression systems eg. FM200, Novec 1230
CodesHeat Detector
Cards 
CCTV 

National Fire Protection Association (NFPA) NAFA 75- standard for the protection of electronic computer and data processing equipment. In the 2003 edition of NFPA 75 (8.4.2.1) allows data centers to continue to power the electronic equipment upon activation of gaseous agent total flooding system,provided the data center has the following risk considerations:

  • economic loss from loss of function or loss of records
  • economic loss from value of equipment
  • life safety aspects of  function
  • fire threat of the installation to occupants of exposed property

5.🔌Cabling

 Cabling make up a significant portion of the IT budget, thus in order to avoid unnecessary wastage is through Proper Design and selection of Core Components.

Proper usage of Cable Tray and management system devices will help to reduce downtime due to human errors and overheating.

Data Cable best practice are

  1. Overhead deployment
  2. Underfloor deployment  (Raised Floor System)
  3. Rack Installation
  4. Cable Testing ( IT Manager can request for data cable test report)

Electrical Cable Installation (Singapore Standard SS 638 : 2018 or CP5)

  1. Follow Code of Practice for Electrical Installation (Singapore Standard SS 638 : 2018 or previously known as CP5)
  2. Do not exceed the manufacturers bending radius recommendation. Usually more than 6D (cable outside diameter, D) to 8D.
  3. Continuous Load- any load which is  left on more than 3hours
  4. De-rate electric current amperages (A) and wire size by 20%
  5. De-rating approach helps avoid overheated wires, short circuit and fire
  6. If the copper in the wire is insufficient for the electric current amperages requires, the insulation will melt.

6.👨🏻‍💼Management

Management is an element that encompass all other elements and having visibility to all the components of the physical infrastructure.

Management include the following

  1. Building Management System (BMS)
  2. Network Management Systems (NMS)
  3. Element Managers
  4. Other monitoring hardware and software.

Management include

  1. Maintain System Availability
  2. Managing problems
  3. Managing changes

Essentials Category of Data Center Managements include

  1. Incident Management
  2. Change Management
  3. Capacity Management
  4. Availability Management

A holistic Data Center Management solution should be integrated with open IT systems in order

  1. Manage rapid change
  2. Achieve demanded levels of availability
  3. Control Total Cost Of Ownership (TCO)

7.🛎️Services

There is a broad range of services required to support the Data Center Systems

  1. Consulting and Design Services
  2. Installation Services
  3. Maintenance and Repair Services
  4. Monitoring Services
  5. Decommissioning Services

Data Center availability success is highly dependent on the supporting services.

UPS Room – Related Fire Code

 IFC 2015, Section 608

Section 608 applies to stationary storage battery systems having an electrolyte capacity of more than 50 gal for flooded lead-acid, nickel-cadmium (Ni-Cd), and VRLA or more than 1,000 lb for Li-ion and lithium-metal-polymer used for facility standby power, emergency power, or UPS.

As defined by IFC 608.6.1, room ventilation:

Ventilation shall be provided in accordance with the International Mechanical Code and the following:

  1. For flooded lead-acid, flooded Ni-Cd, and VRLA batteries, the ventilation system shall be designed to limit the maximum concentration of hydrogen to 1% of the total volume of the room.
  2. Continuous ventilation shall be provided at a rate of not less than 1 cfm/sq ft of floor area of the room.

Exception: Li-ion and lithium-metal-polymer batteries shall not require additional ventilation beyond that which would normally be required for human occupancy of the space in accordance with the International Mechanical Code.

The two ventilation requirements are not an “either/or” permissive option. This is contrary to the requirements of NFPA 1.

Other generic provisions of IFC Section 608 include the following:

  • Must prevent access to unauthorized personnel. This can be accomplished by either locating in separate room or in noncombustible cabinets. They may be located in the same room with the equipment they support.
  • Must provide spill control and neutralization for batteries with free-flowing electrolyte (i.e., flooded cell batteries). No specific threshold is given, but it is assumed to apply where greater than 50 gal. Not required for VRLA or lithium.
  • Must have proper supervision of ventilation system.
  • Must have signage on door.
  • Must have smoke detection.
  • Requires thermal runaway protection for VRLA batteries.
  • Li-ion and lithium-metal batteries don’t require ventilation.

NFPA 1-2015, Chapter 52

NFPA 1 is not as frequently adopted by municipalities as the IFC. While the basic requirements of NFPA 1 generally parallel those of the IFC, the technical provisions within NFPA 1 do have significant difference that can impacted the design of related battery ventilation systems. These requirements are as follows:

Chapter 52 applies to stationary storage battery systems having an electrolyte capacity of more than 100 gal in sprinklered buildings or 50 gal in nonsprinklered buildings for flooded lead-acid, Ni-Cd, and VRLA batteries or 1,000 lbs for Li-ion and lithium-metal-polymer batteries used for facility standby power, emergency power, or UPS. This is a significantly lower threshold than that in IFC.

NFPA 1, 52.3.6 Ventilation indicates:

For flooded lead-acid, flooded Ni-Cd, and VRLA batteries, ventilation shall be provided for rooms and cabinets in accordance with the International Mechanical Code and one of the following:

  1. The ventilation system shall be designed to limit the maximum concentration of hydrogen to 1% of the total volume of the room during the worst-case event of simultaneous “boost” charging of all the batteries in accordance with nationally recognized standards.
  2. Continuous ventilation shall be provided at a rate of not less than 1 cfm/sq ft of floor area of the room or cabinet.

This language allows for significantly more flexibility than IFC. Other provisions of Chapter 52 include the following, which are not addressed in the IFC:

  • In assembly, educational, detention, health care, day care, etc., battery systems shall be located in a room separate from other portions of the building and be 2-hour fire-rated.
  • Thermal runaway protection is required for lithium batteries.
  • Spill control is required where there are more than 55 gal in individual vessels or an aggregate capacity of greater than 1,000 gal.
  • The battery environment shall be controlled or analyzed to maintain temperatures in a safe operating range for the specific battery technology used. In the case of VRLA batteries, they’re typically rated for an ambient of 77˚F. Although it is not specifically stated, this effectively requires that air conditioning be provided for most battery rooms.

Lithium Ion Battery Fire Video