White paper: CFD for data centers
Extract from the EOLIOS white paper on Data Centers
- A passionate team
- Exclusive estates
- All sectors
- Data Center
- A passionate team
Continue navigation :
Our latest news :
Our projects :
Our areas of expertise :
Technical files :
White paper
The thermal load in data centers has in recent years. And this trend is continuing, as there is a process of reducing the size of electronic equipment with a simultaneous increase in computing power , leading to the release of a large amount of heat per unit (a unit of server rack height). While a few years ago it seemed that the cooling capacity of 5 kW per server rack was quite sufficient to cover all existing and anticipated customer needs for the foreseeable future, today there is equipment on the market which, even if it’s not even the whole server rack, is filled, generates more than 10 kW.
Almost all server manufacturers offer such equipment. The major groups offer a unified IT system, which achieves a 6U power consumption of 2 kW or more, depending on the operating mode. It’s not uncommon for more than 10 kW of heat to be generated per rack, and we’ve seen racks up to 45 kW.
Server virtualization leads to increased heat dissipation from a single processor
The use of server virtualization technology “worsens” the situation.
The virtualization of servers can increase their load considerably, and if the old processors were idle for 75 to 85% of the time, then via the use of virtualization, the processor load in the servers increases considerably and, consequently, more heat is generated on a server.
ASHRAE’s Best Practice for Energy Efficiency in Data Center Facilities confirms this data.
Heat production per rack is constantly increasing.
Therefore, when designing a modern data center, it is necessary to focus on thermal loads of 10 kW or more per server rack.
Or, as a last resort, it is necessary to allocate specific zones in the data center’s computer room, which will provide cooling from 10 kW per server rack.
To obtain a sufficiently precise answer to this question, the designer’s use of a heat balance equation alone, with the addition of a cooling capacity margin of 10% to 20% and an Excel program, is no longer sufficient.
what is the impact of maintenance?
A number of problems in data center operations
A number of problems also arise when operating an existing data center, even with low thermal loads.
For example, after installing additional equipment, dead zones may appear in the data center.
Local areas of equipment overheating (known as hot spots) or, on the contrary, low-temperature zones can occur in data centers (for natural cooling systems).
It’s clear that overheating servers, data storage systems, network and telecommunications equipment is bad; sooner or later, high temperatures will lead to breakdowns, and therefore potential data loss.
Hygrometry also has a negative impact on the operation of servers and data storage systems.
Low temperatures increase humidity, which can lead to condensation.
According to ASHRAE, relative humidity must not exceed 80%.
Some manufacturers incorporate temperature and relative humidity sensors in servers and storage systems, and software controls can shut down hardware when humidity and temperature limits are exceeded.
The temperature in server rooms, where IT equipment is installed and operated, is limited by standards not only by the upper limit, but also by the lower limit.
According to the latest requirements of ASHRAE Technical Committee TC 9.9 published in 2016, the temperature in the server room must not be lower than 18 degrees Celsius.
In addition, the lower temperature leads to inefficient use of electricity, resulting in higher data center operating costs .
To combat local hot spots, the customer may be obliged to install floor fans close to the server racks, or mount additional air or liquid cooling units (if, of course, there is room for them).
However, the use of such “radical” methods is not necessarily a necessity. But sometimes it turns out that only one thing should have been replaced: it may be a matter of removing or adding raised floor grilles, and the problem can be solved.
However, it is extremely difficult to identify bottlenecks without special software tools and precise knowledge of thermo-aerodynamic effects.
Often, the customer uses the data center space extremely inefficiently, without fully loading the cabinets and distributing the equipment evenly (if this can be done at all).
But all this could have been avoided by creating a CFD thermodynamic model of the data center and performing optimization calculations according to this model.
How high is the raised floor in a data center?
When designing a new data center, the architect in charge of the project always has a question: how high should a raised floor be in a data center?
Clearly, the greater the height of the raised access floor, the greater the resistance to air flow, the more diverse the networks (water networks, cable trays and cables) that can be stored, and the more additional structures and equipment that can be placed beneath the raised access floor, such as power distribution networks or consolidation points for a structured cabling system…
However, as the height of the raised floor increases, the cost of the building structure will increase and, by the way, the space between the raised floor and the ceiling will decrease, which can complicate the creation of a duct system to supply warm air to the cooling units (CRAH).
A few years ago, a recommendation was published on the height of the raised floor in relation to the surface area of the data center’s machine room.
Depending on the size of the machine room
up to 70 m², the height of the raised floor must be at least
400 – 500 mm, if the area of the machine room
over 100 m², the height of the raised floor must be at least 500 – 700 mm, if the machine room is over 300 m², the height of the raised floor must beat least 700 mm. This rule of thumb worked when the load per rack did not exceed 5 kW, and when hot and cold air insulation technology was not used (separation between hot and cold aisles).
In this context, in order to obtain a precise answer to the question of the height of the raised floor, it is advisable to carry out a CFD simulation of the air flows, calculate several options and choose the most appropriate one.
What is CFD simulation?
CFD is an acronym for Computational Fluid Dynamics.
Using specialized software, the user creates a three-dimensional model of an object, imposes certain boundary conditions, selects models representing physical phenomena occurring in gaseous and liquid media (heat transfer, media flow, thermal conductivity, radiation, convection, etc.), selects a calculation method and performs calculations… Based on the calculation results obtained, the user evaluates, if necessary modifies the computer model, and performs the calculations again. ), selects a calculation method and performs calculations… On the basis of the calculation results obtained, the user evaluates and, if necessary, modifies the computer model, and performs the calculations again.
The aim of modeling is to write down physical phenomena as accurately as possible, and then to find a suitable and satisfactory solution to any design problems you may encounter.
Simulation results are used when making design decisions, to further improve the created plant model, identify bottlenecks in the operating plant and optimize the operating system.
Using CFD simulation in data centers
Unfortunately, higher operating temperatures can reduce reaction time in the event of a rapid rise in temperature due to a cooling unit failure.
A data center containing servers operating at higher temperatures is at risk of experiencing instantly of hardware failures simultaneously.
Recent ASHRAE regulations underline the importance of proactive monitoring of environmental temperatures inside server rooms. Data centers are ideal objects for IT modeling, because it’s impossible to create a prototype or physical model of a data center. And without creating a data center model,
it’s impossible to predict with sufficient accuracy how the air-conditioning system will perform in a real operating facility, how the air-conditioning system will behave when the load changes, how the temperature will change in a row of server racks and along the height of each rack.
When designing an air-conditioning system for a data center, a large number of parameters need to be taken into account.
Here are just a few of them:
- The size and the volume of the room;
- Placement of telecommunications cabinets and racks in the server room;
- Floor height raised; direction, volume and speed of movement of cold air currents;
- Location of air-conditioning equipment;
- The types of fans used and the direction of air flow,
- Consideration of obstacles to the flow of cold air;
- The type of raised access floor slabs used and the geometry of the outlets.
When designing an air-conditioning system without using CFD analysis, most of these parameters are not properly taken into account, or are over-dimensioned.
In fact, the real influence on temperature and humidity distribution in the data center room depends on the parameter studied cannot be reliably estimated without precise computer simulation.
CFD software for data centers
There are a large number of programs on the market for solving various problems associated with the simulation of liquid and gas flows.
These include the following programs: ANSYS, Autodesk CFD, Xflow, Open Foam, Phoenics, Flow Vent,STAR-CD , FASTEST-3, Flow Vision, Tile Flow, Sigma6room, Gas Dynamics Tool… However, not all thermodynamic flow simulation programs have ready-to-use modules and libraries of integrated elements, taking into account the specificities of data centers.
Software such as Tile Flow and Sigma 6 have built-in modules, programs and libraries for simulating air flows in the data center.
For engineers not accustomed to working with CFD modeling programs, it makes sense to consider purchasing this type of software, which already contains ready-to-use models for calculating air flows in the data center, as well as equipment libraries (e.g. fans, pumps, air-conditioning units). In all cases, the quality of the study depends on the level of experience of the engineer in charge of the simulation.
CFD engineering must be carried out by specialists.
Data center CFD modeling design steps
Recovering study hypotheses
Before carrying out the process of modeling an existing data center, it is necessary to carry out a complete and precise study of the object: measure air flow velocities, measure pressure, take temperature measurements, determine air circulation channels and detect obstacles and possible locations of air leaks.
In other words, the task of examining an existing object itself is quite laborious but, nevertheless, extremely useful.
Because in the process of collecting data, bottlenecks are identified.
To solve the problem of creating a model for a new data center, it is necessary to collect initial data from the space and validate assumptions about the technologies and devices used.
3D data center modeling
The next step is to create a geometric model of the data center (or digital twin) and its component parts.
A 3D model of an object is created using CAD programs, then the data is exported to the CFD simulation module.
Mesh creation
The next step is to create the resolution model. This step is performed in programs using integrated mesh generation software modules, or using separate software products.
Accuracy, convergence and calculation speed depend on the mesh. The quality of the results obtained depends directly on the quality of the mesh (fineness, mesh adaptation, etc.).
After the mesh construction stage, the user needs to check the quality of the mesh constructed using various parameters (element asymmetry, height/width ratio).
Boundary conditions & simulations
Boundary conditions are entered into the program and models are selected according to the assumptions, then a calculation is performed, which may converge or diverge (i.e. not have a correct solution) according to the various parameters above.
After convergence, the results of the calculations can be processed by special programs and displayed in graphical, tabular or evenanimated form, clearly demonstrating the changes in the physical parameters.
For data centers, a visual representation of the calculated data is generally used in the form of a temperature distribution over the computer room area and the height of the server racks.
The engineer then analyzes the calculated results and, if necessary, modifies the object models and performs the calculations again.
What impact does this have on design?
Modern design tools enable CFD engineers to communicate with the various trades, so as to simply explain the phenomena at the root of problems, and then propose propose solutions which can then be validated collectively.
Summary
CFD programs can be used to simulate the flow of liquids and gases, as well as other physical phenomena associated with this process, such as heat transfer.
Thermodynamic modeling offers great opportunities for analyzing liquid and gas flows, enabling new systems and equipment to be designed at a high professional level, or existing systems to beoptimized .
Without the use of CFD modeling, it is impossible to obtain precise answers to such fundamental questions as the distribution of temperatures and humidity along the cold aisles and the height of the server rack room, as a function of :
- of the thermal load;
- where air-conditioning units are installed;
- heat transfer fluid and refrigerant temperatures ;
- the height of the raised floor;
- the distribution of broadcast schedules;
- fan type and other parameters.
Data centers: on the same subject
Thermal study of technical premises
Find out more
Data Center Fire Simulation
Find out more
External CFD simulation for data centers
Find out more
Understanding how a Data Center works
Find out more
