4340

Accredited to ISO/IEC 17025:2005

UKAS Accredited Company Registered in England & Wales No. 527550

Thermography testing on Data Centers

Introduction to Infrared Thermography on Data Centers
Important buildings such as Data centers and other mission critical facilities with uninterruptible power supply (UPS) systems can experience extremely high ROI (return on investment) with a properly integrated infrared thermography program.

A successful infrared program includes all of the facility equipment from construction and operation to server efficiency. Commissioning of data center initial construction as well as the addition of new equipment is accepted as standard operating procedure for information technology (IT) facilities.

Downtime at Data Centers
Downtime in these facilities is not an option, and should be avoided at all costs. Infrared thermography is being utilised for regular electrical switchgear surveys, optimising of cooling systems and servers, and commissioning of all electrical equipment, including UPS modules , PDU (power distribution unit) equipment and computer servers. Many construction project specifications have infrared surveys as a requirement before the building is turned over to the owner.

Data center infrared thermography must have total accountability for all infrared data in the commissioning process, regardless of whether or not there are problems. This accountability can be achieved by documenting all equipment inspected with time, date, location and equipment condition. The infrared thermographer must create a data log and record the infrared video onto a digital storage device of some type. New technologies in data acquisition and report preparation will make historical data (images previously taken) available for comparison. This will enable the infrared thermographer to more closely compare circuit boards and other UPS equipment with previously acquired images. If something fails or causes downtime in the system, an IR image of that component may be referenced to document that the equipment was operational, at thermal steady-state and in acceptable condition when the survey was made.


Infrared surveys and Commissioning of Data Center Equipment
The Thermography commissioning/surveying process should include these types of equipment and considerations. The following infrastructure support equipment should be tested:

• All distribution electrical panels.
• All associated switchgear.
• Emergency diesel generator systems.
• Battery banks, breakers and charging systems.
• Cooling systems, including chillers and all HVAC equipment.
• CRAC (computer room air conditioning) units.
• ATS (automatic transfer switch) equipment.
• UPS modules.
• Resistive load banks and associated cables/connectors.
• Static transfer switches.
• Rotary UPS system, if applicable.
• Transformers (utility and site).
• PDU (power distribution unit) equipment.

Loading Considerations:
• All normal switchgear and electrical panels need to be checked under load.
• Test generator leads and emergency source for the automatic transfer switches under load.
• Resistive load banks must be attached to the PDUs and tested with increasing load percentages.
• Each UPS module must be tested independently including a full load battery test.
• UPS battery connections and individual battery cells should be checked during and after the discharge.
• Rotary UPS systems must be checked during operation. (Rotary systems utilize the same rectifier technology as static topologies on the front end to create DC current from AC, but use spanning motor-generators to re-create the sine wave on the output.)
• Each PDU must be tested on both the preferred and alternate sources as well as in each respective bypass.
• All normal transfers should be verified operable.
• PDU distribution breakers must be checked after they are put into service on the panel boards.

Figure 1: Resistive load bank is shown with a bad cable connection.

Causes for Electrical Failure and Downtime in Data Centers
The critical power distribution system takes conditioned power from the UPS and distributes it throughout the facility to individual loads. Most site failures occur in areas where hot electrical work is required and physical maintenance is difficult to perform.

Typical causes for failures include:

• cover slipped while accessing load panel,
• overheated breakers tripped unexpectedly,
• wires were not physically secured under screws,
• screws were not torqued adequately
• wires or circuit breaker handles were dislodged while adjacent work was being performed,
• screws were stripped,
• insulation was skinned causing faulted wires,
• rotations were reversed.

Infrared Thermography surveys for Servers and Server Racks
Ten percent of all server racks currently in service are too hot to meet industry standards for maximum IT reliability and performance.

“Institute research into computer room cooling indicates 1/3 all perforated tiles are incorrectly located and 60% of all available cooling capacity is being wasted by bypass airflow. Increasing under-floor static pressure to get air where it needs to go requires permanently blocking all unnecessary air escape routes. This includes sealing cable cutouts behind and underneath products or racks (this unmanaged airflow is what is really cooling most computer rooms) as well as the penetrations in the floor or walls or ceiling and any other openings in the raised floor. Perforated floor tiles with 25% openings can be replaced with 40% and 60% grates to permit a much higher airflow. For sites with unused raised floor space deliberately spreading equipment out to create white space and reduce the averaged gross watts per square foot power consumption will be a viable option.”
(4. Brill, Kenneth 2006)

Figure 2: Server cooling fans are shown. The top fan is operating normally, while the bottom fan has failed.