APC White Paper 144: Data Center Projects, Establishing a Floor Plan
by Neil Rasmussen and Wendy Torell
A floor plan strongly affects the power density capability and electrical efficiency of a data center. Despite this critical role in data center design, many floor plans are established through incremental deployment without a central plan. Once a poor floor plan has been deployed, it is often difficult or impossible to recover the resulting loss of performance. This paper provides structured floor plan guidelines for defining room layouts and for establishing IT equipment layouts within existing rooms.
A data center floor plan includes the layout of the boundaries of the room (or rooms) and the layout of IT equipment within the room. Most users do not understand how critical the floor layout is to the performance of a data center, or they only understand its importance after a poor layout has compromised the deployment. The floor plan either determines or strongly affects the following characteristics of a data center:
- The number of rack locations that are possible in the room
- The achievable power density
- The complexity of the power and cooling distribution systems
- The predictability of temperature distribution in the room
- The electrical power consumption of the data center
Many users do not appreciate these effects during data center planning, and do not establish the floor layout early enough. As a result, many data centers unnecessarily provide suboptimal performance.
The purpose of this paper is to explain how floor plans affect these characteristics, and to prescribe an effective method for developing a floor layout specification.
Role of the floor plan in the system planning sequence
Floor plans must be considered and developed at the appropriate point in the data center design process. Considering floor plans during the detailed design phase is typical, but simply too late in the process. Floor plans should instead be considered to be part of the preliminary specification and determined BEFORE detailed design begins. White Paper 142, Data Center Projects: System Planning explains this planning sequence in greater detail.
It is not necessary for a floor layout to comprehend the exact location of specific IT devices. This paper will show that effective floor plans only need to consider the location of equipment racks or other cabinets, and to target power densities. These preliminary floor layouts do not require knowledge of specific IT equipment. For most users it is futile to attempt to specify particular IT equipment locations in advance – in fact, racks may ultimately house equipment that is not even available on the market at the time the data center is designed.
The reasons that floor plans must be considered early, as part of the preliminary specification, and not left until the later detailed designs include:
- Density is best specified at the row level, so rows must be identified before a density specification can be created (for more information on specifying densities, see White Paper 120, Guidelines for Specification of Data Center Power Density).
- Phasing plans are best specified using rows or groups of rows, so rows must be identified before an effective phasing plan can be created.
- The floor grid for a raised floor and the ceiling grid for a suspended ceiling should be aligned to the rack enclosures, so rows must be identified before those grids can be located.
- Criticality or availability can (optionally) be specified differently for different zones of the data center – rows must be identified before a multi-tier criticality plan can be created.
Therefore, a floor plan must be established early in the specification phase of a project, after SYSTEM CONCEPT but well before DETAILED DESIGN
Floor planning concepts
The effects of floor plans on data center performance
Number of rack locations
The basic principle of floor planning is to maximize the number of rack locations possible. Small variations in the location of walls, existing IT devices, air conditioners, and power distribution units can have a surprisingly large impact on the number of possible rack locations. This effect is magnified when high power densities are required. For this reason, a careful and systematic approach to floor planning is essential.
Achievable power density
The floor plan can have a major impact on the achievable power density. With certain cooling architectures, a poor layout can decrease the permissible power for a given rack by over 50%. This is a huge performance compromise in a modern data center, where new technologies have power densities that are already stressing the capabilities of data center design.
In many data centers, users may want to establish zones of different power density. These density zones will be defined by the equipment layout. The floor plan is therefore a critical tool to describe and specify density for data centers.
Complexity of distribution systems
The floor plan can have a dramatic affect on the complexity of the power and cooling distribution systems. In general, longer rows, and rows arranged in the patterns described in this paper, simplify power and cooling distribution problems, reduce their costs, and increase their reliability.
Predictability of cooling performance
In addition to impacting the density capability of a data center, the floor plan can also significantly affect the ability to predict density capability. It is a best practice to know in advance what density capability is available at a given rack location and not to simply deploy equipment and “hope for the best,” as is a common current practice. An effective floor plan in combination with row-oriented cooling technologies allows simple and reliable prediction of cooling capacity. Design tools such as Schneider Electric’s InfraStruXure Designer can automate the process during the design cycle, and when layouts follow standard methods, off-the-shelf operating software such as InfraStruXure Manager can allow users to monitor power and cooling capacities in real time.
Most users are surprised to learn that the electrical power consumption of a data center is greatly affected by the equipment layout. This is because the layout has a large impact on the effectiveness of the cooling distribution system. This is especially true for traditional perimeter cooling techniques. For a given IT load, the equipment layout can reduce the electrical power consumption of the data center significantly by affecting the efficiency of the air conditioning system.
- The layout affects the return temperature to the CRAC units, with a poor layout yielding a lower return air temperature. A lower return temperature reduces the efficiency of the CRAC units.
- The layout affects the required air delivery temperature of the CRAC units, with a poor layout requiring a colder supply for the same IT load. A lower CRAC supply temperature reduces the efficiency of the CRAC units and causes them to dehumidify the air, which in turn increases the need for energy-consuming humidification.
- The layout affects the amount of CRAC airflow that must be used in “mixing” the data center air to equalize the temperature throughout the room. A poor layout requires additional mixing fan power, which decreases efficiency and may require additional CRAC units, which draw even more electrical power.
A conservative estimate is that billions of kilowatt hours of electricity have been wasted due to poor floor plans in data centers. This loss is almost completely avoidable.
Basic principles of equipment layout
The existence of the rack as the primary building block for equipment layouts permits a standardized floor planning approach. The basic principles are summarized as follows:
- Control the airflow using a hot-aisle/cold-aisle rack layout
- Provide access ways that are safe and convenient
- Align the floor or ceiling tile systems with the equipment
- Minimize isolated IT devices and maximize row lengths
- Plan the complete equipment layout in advance, even if future plans are not defined
Once these principles are understood, an effective floor planning method becomes clear.
Control of airflow using hot-aisle/cold-aisle rack layout
The use of the hot-aisle/cold-aisle rack layout method is well known and the principles are described in other documents, such as ASHRAE TC9.9 Mission Critical Facilities, “Thermal Guidelines for Data Processing Environments” 2004, and a white paper from the Uptime Institute titled “Alternating Cold and Hot Aisles Provides More Reliable Cooling for Server Farms.” The basic principle is to maximize the separation between IT equipment exhaust air and intake air by establishing cold aisles where only equipment intakes are present and establishing hot aisles where only equipment hot exhaust air is present. The goal is to reduce the amount of hot exhaust air that is drawn into the equipment air intakes. The basic hot-aisle/cold-aisle concept is shown in Figure 2.
In the figure, the rows represent the IT equipment enclosures (racks). The racks are arranged such that the adjacent rows face back to back, forming the hot aisles.
The benefits of the hot-aisle/cold-aisle arrangement become dramatic as the power density increases. When compared to random arrangements or arrangements where racks are all lined up in the same direction, the hot-aisle/cold-aisle approach allows for a power density increase up to 100% or more, without hot spots, if the appropriate arrangement of CRAC units is used. Because all cooling architectures (except for fully enclosed rack-based cooling) benefit dramatically from hot-aisle/cold-aisle layout, this method is a principal design strategy for any floor layout.
Provide access ways that are safe and convenient
It is a legal requirement, and common sense, to provide appropriate access ways around equipment. The hot-aisle/cold-aisle system creates natural hallways or aisles that are well defined. Since an effective data center is based on row layouts with aisles that serve as access ways, it is important to identify and understand the impact of column locations. A column could consume up to three rack locations if it falls within a row of racks – or even worse, could result in the elimination of a complete row of racks if it obstructs an aisle. Figures 3 and 4 illustrate the possible impact of a column on the number of usable rack. The room in Figure 3A fits 40 rack locations when no columns are present. When a column exists, but aligns with a row of equipment racks, as Figure 3B shows, only one rack may be impacted.
When a column aligns with an aisle, a more significant impact can occur. Figure 4A shows the column in the aisle. Occasionally, this may be determined to be an acceptable obstruction, if the local authority having jurisdiction (AHJ) believes that shifting the equipment rows (and possibly eliminating available rack positions) is an unreasonable accommodation according to local disabilities acts . Figure 4B illustrates an attempt at recreating sufficient aisles, but represents a bad alternative since the hot-aisle/cold-aisle layout is severely impacted, and cooling becomes less predictable. Figure 4C shows what happens to the rack count when the rows must be shifted so that the column is no longer in the aisle. Note that the rack count has drastically dropped from 40 to 29, a decrease of over 25%. The last scenario (Figure 4D) shows an alternate configuration of racks, aligned “north-south” rather than “east-west” (this is called the “axis” of the layout). This configuration accommodates 35 rack locations, which is more favorable than shifting rows. Later sections of this paper further explain the considerations in rotating the layout axis by 90o to obtain optimal positioning.
It is a common practice to adjust the equipment layout to place columns within rows, where they consume potential equipment locations. Keeping support columns out of aisle-ways is a severe constraint on maximizing equipment locations, because it dictates areas that cannot be used as aisle-ways. As Figures 3 and 4 illustrate, the shifting of equipment rows to accommodate column locations can cause the loss of an entire row of equipment when that row becomes trapped against a wall or other obstacle. Therefore, the careful location of equipment rows relative to the columns is of primary concern in the floor layout.
Align the floor and/or ceiling tiles with the equipment
In many data centers the floor and ceiling tile systems are used as part of the air distribution system. In a raised floor data center, it is essential that the floor grille align with racks. If the racks and the floor grid do not align, airflow can be significantly compromised. It is also beneficial to align any ceiling tile grid with the floor grid. This means the floor grid should not be designed or installed until after the equipment layout is established, and the grid should be aligned or indexed to the equipment layout according to the row layout options.
Unfortunately, specifiers and designers often miss this simple and no-cost optimization opportunity. The result is that either (1) the grid is misaligned with the racks, with a corresponding reduction in efficiency and density capability, or (2) the racks are aligned to the grid but a suboptimal layout results, limiting the number of racks that can be accommodated.
Pitch – the measurement of row spacing
The row length in a hot-aisle/cold aisle layout is adjustable in increments of rack width, which provides significant flexibility. However, the spacing between aisles has much less flexibility and is a controlling constraint in the equipment layout. The measurement of row-to-row spacing is called pitch, the same term that is used to describe the repeating center-to-center spacing of such things as screw threads, sound waves, or studs in a wall. The pitch of a data center row layout is the distance from one mid-cold-aisle to the next mid-cold-aisle (Figure 5).
Figure 6 shows the four standard pitches used in data center floor layouts. Each pitch is defined by a number of tiles, where a tile is 2 ft (600 mm) wide. Note that in all four pitches, the equipment racks are aligned to tiles in the cold aisles. This is because, in a raised-floor environment with perimeter-based cooling, full perforated tiles are needed in the cold-aisles for air delivery.
Pitch A shows the most compact geometry for a row pair, with a 7-tile (14 foot) overall width, the most commonly used “building block” for data center row layouts. However, in some situations, wider pitches are necessary. Once a reference concept design has been selected for a data center, Figure 7 provides guidelines for when to use each of the four pitches. For instance, the flowchart illustrates that pitch B provides 50% more raised floor cooling capacity in the cold aisle for higher density applications with raised floor air distribution. Pitch C or D, on the other hand, may be needed when using racks with specialized cooling plenums fixed to the back.
Special spacing for hard floor environments
Nonstandard row pitches, which are incrementally smaller than the four standard ones, can be advantageous for installations with a hard (non-raised) floor. The four pitches in Figure 6 align cold-aisle racks with floor tiles to facilitate air delivery through the raised floor; however, when a raised floor is not used, that cooling constraint goes away. For example, consider a high-density scenario where back-of-the-rack cooling accessories increase rack depth from 42 inches to 50 inches. Using the minimum row spacing of a 3-foot hot aisle and 4-foot cold aisle, the smallest allowable pitch would be 15 feet 4 inches (8 inch increase x 2 rows of racks). When a layout on a hard floor using the four standard pitches is a problem – i.e. you lose an additional row by a couple of feet or less – the pitch can be compressed, as long as aisle widths remain the minimum of 3 feet for the hot aisle and 4 feet for the cold aisle.
An effective data center floor plan should be deployed in row pairs, using the basic spacing pitches described above. However, we will show that various barriers and constraints may interfere with the optimal layout.
Minimize isolated IT devices and maximize row lengths
The control of airflow by separating hot and cold air, as described above, is compromised at the end of a row where hot air can go around the side of the end rack and return to IT equipment air intakes on the back. Therefore, the theoretical ideal design of a data center is to have no row ends – i.e. rows of infinite length. Conversely, the worst case situation would be rows of one-rack length – i.e., isolated single racks.
In addition, the ability to effectively implement redundancy is improved with longer rows. The goal of row layout is to maximize row length consistent with the goals of providing safe and convenient access ways. In general, a layout that provides longer row lengths is preferred, and a row layout that generates short rows of 1-3 racks should be avoided.
Special considerations for wide racks
Standard-width racks (2 ft or 600 mm) conveniently align with the width of raised-floor tiles. When under-floor cables must be distributed to such a rack, a hole is typically created in the tile directly below the rack to run the cables; if that particular rack is then re-located or removed, the tile is simply replaced with a new one.
Wide racks that do not align with the standard raised floor tile width are creating a new challenge, because a rack may occupy two or even three tiles. If such a rack is removed, no longer can the tile simply be replaced with a new one, since the tile is partially underneath the neighboring rack as well. These issues can be avoided altogether by overhead power and data cable distribution.
Plan the complete floor layout in advance
The first phase of equipment deployment often constrains later deployments. For this reason it is essential to plan the complete floor layout in advance.
Basic principles of structural room layout
Many data centers are fit-outs of existing space. In these cases, the structural layout of the room is fixed and cannot be specified. For some data centers, wall relocation or other modifications may be possible. For data centers in new construction, considerable options are available regarding wall locations. It is a basic and generally unappreciated principle of data center design that the ability to locate walls can greatly improve the performance of the data center. Therefore, when it is possible, the layout of the room boundaries should be chosen based on the standard data center design principles described in this section (readers who have no room layout flexibility may choose to skip this section).
Standardized room dimensions
There are preferred room dimensions for data centers, based on the pitch chosen. Given an area or room that is rectangular in shape, free of the constraints imposed by support columns (described earlier), the preferred length and width are established as follows:
- One dimension of the room should be a multiple of the hot-aisle/cold-aisle pitch, plus a peripheral access-way spacing of approximately 2-4 tiles
- The other dimension of the room is flexible and will impact the length of the rows of racks
When one of the dimensions of the room is not optimal, the performance of the room can be dramatically reduced, particularly if the room is smaller. The most obvious problem is that the number of equipment racks may be lower than expected because some space cannot be used. The second, and less obvious, problem is that when the ideal layout cannot be achieved, the power density and electrical efficiency of the system is reduced.
To understand the effect of room dimension on the number of racks, consider a room with a fixed length of 28 feet and a variable width. In such a room, the length of a row would be 10 racks, allowing for 2 tiles (4 feet) at each row-end for access clearance. The number of racks that could fit in this room will vary as a function of the width of the room as shown in
Figure 8 shows that the number of installable racks jumps at certain dimensions as new rows fit into the room. Furthermore, the chart shows that certain numbers of racks are preferred because the even row number permits a complete additional hot-aisle/cold-aisle pair to be installed. The preferred width dimensions are indicated by the arrows, for the pitch (the most compact pitch A in this case) and perimeter clearances (2 tiles) defined.
Location of support columns in room boundary layout
The location of support columns in the room can dramatically affect the equipment layout, as previously illustrated. Therefore, when an option exists to locate room boundaries, the following guidelines apply:
- For smaller rooms, arrange the room boundaries, if possible, so that no support columns are in the equipment area.
- Rooms should be rectangular, where possible. Unusual shapes, niches, and angles often cannot be effectively utilized and/or create a reduction in power density or electrical efficiency.
- For situations where columns are unavoidable but boundaries are flexible, the floor plan should be laid out as if no columns existed, based on the standardized dimensions of the room, and the pitch(es) required. Columns should then be located directly over any one particular rack location, preferably at a row end.
- For very large rooms, the location of the walls in relation to the columns is typically inflexible.
- Creating area partitions
- Advance layout of future rows
- Ability to re-purpose areas in the future
- Ability to perform radical infrastructure modifications in one area without interfering wit the operation of another area
- Ability to defer the installation of basic infrastructure (such as piping or wiring) to a future date
Note that the location of the wall in the bottom scenario has caused row 5 of the equipment layout to be lost, representing 10 racks out of the 80 rack layout, or 12% of the total – a significant loss of rack footprint space. Although the wall was only offset by a small amount, this loss occurs because the wall-to-wall spacing does not permit appropriate access ways if row 5 is included. Furthermore, the access way between row 6 and the wall has become a hot aisle. This reduces the confining effect of the hot-aisle/cold-aisle design and will result in a reduced power capacity for row 6. Furthermore, because the primary access path between row 6 and the wall is now a hot aisle, this creates an uncomfortable zone for personnel. These factors taken together demonstrate how serious a small change in a wall location can be when partitioning a data center.
Floor planning sequence
Using the rack as the basic building block for floor layout, and the row-pair “pitch” as the spacing template, a standardized floor layout approach is achievable. Starting with a floor plan diagram for the room, the basic principles are summarized as follows:
Identify and locate the room constraints
First, identify and locate all physical room constraints:
- Columns – verify the exact as-built dimensions
- Existing fixed equipment – breaker panels, pipe connections, fire suppression equipment, cooling equipment
Establish key room-level options
Next, identify what additional equipment will be placed in the room and the options available for delivering/installing that equipment within existing room constraints:
- Identify additional equipment (besides the IT equipment or in-row power and cooling equipment) that will be placed in the room, including any additional cooling equipment, fire suppression equipment, power equipment, or user workstations
- If the room uses a raised floor, determine the length(s) of the access ramp(s) and identify all possible options for locating the ramps
Establish the primary IT equipment layout axis
- Columns are not located in main access ways
- (If no raised floor) Rows are aligned to the ceiling grid so that the cold aisles contain complete tiles
- There is sufficient clearance at row-ends and between rows and walls
- There is sufficient clearance/access around any fixed equipment in the room
- Access ramps, if required, are present and have been optimally located
- Any open areas or areas for another purpose face a cold aisle, not a hot aisle
- Locations have been found for any additional equipment identified in the room level options above
- Rows that are separated by an access way should not reverse the direction they face
- All rows align with the same axis (i.e., all a rows are parallel, with no perpendicular rows)
- The entire room is laid out in the floor plan, even if no immediate plans are in place to deploy some sections of the room
- Which axis most effectively keeps support columns out of the main access ways?
- Which axis allows for the most racks?
- Which axis works best with the preferred hot-aisle/cold-aisle pitch?
- Which axis has hot-aisle/cold-aisle row pairs and does not end up with an odd number of rows?
- Which axis has the fewest short rows or isolated racks?
- Which layout provides the desired aesthetic layout of the data center for purposes of viewing or tours, if that is a consideration
Low the row boundaries
The process of selecting the primary layout axis typically establishes the row locations accurately. With the row locations established, it is critical to establish and validate the row boundaries. This includes setting the row end boundaries, and verifying the boundaries between fronts and/or backs of rows with respect to other equipment, columns, or walls.
Access must be provided between row-ends and other obstructions using the following guidelines:
- For plain walls, a minimum of 2 tiles is an acceptable spacing for a row-end; larger data centers often prefer 3 tiles to provide better accessibility.
- For some layouts, it may be desired to end a row at a wall. However, this creates a dead-end alleyway which may limit the length of the row based on code requirements.
- For long rows of over 10 racks, local regulations may require that breaks be placed in rows to allow personnel to pass through. This may also be of practical concern for technicians who need access to both sides of a rack without having to walk a long distance.
- The spacing between the row front (cold aisle) or the row back (hot aisle) and other equipment must be carefully checked to ensure that access ways are sufficient, and that any access required to those other devices for service or by regulation is sufficient and meets code.
- It must be verified that any other equipment that has been located as part of the floor plan is not constrained by piping, conduits, or access restrictions.
The above restrictions and boundaries must be marked on the room layout before the axis selection and row layout are confirmed.For small data centers – i.e., up to 2 rows of racks – this floor planning process can occur as a paper study. As the room size grows, computer-aided tools that ensure consistent scale become necessary in order to accurately plan the floor layout. Ideally, the row layout and boundary areas should also be marked out using colored masking tape in the actual facility. This step is quite feasible for many smaller fit-out designs and for retrofits, and often identifies surprise constraints that were not realized during the conceptual plans.
Specify row/cabinet density
Many users attempt rudimentary floor layout planning, yet still have downstream problems. Here are some of the most common problems observed in Schneider Electric’s experience:
Failure to plan the entire layout in advance
Most data centers begin to deploy equipment without a complete equipment deployment plan. As the deployment expands, severe constraints on the layout may emerge, including:
- Equipment groups grow toward each other and end up facing hot-to-cold instead of hot-to-hot, with resultant hot spots and loss of power density capability
- Equipment deployments grow toward a wall, and it is subsequently determined that the last row will not fit – but would have fit if the layout had been planned appropriately
- The rows have a certain axis orientation, but it is later determined that much more equipment could have been deployed if the rows had been oriented 90 degrees the other way – and it is too late to change it
- Equipment deployments grow toward a support column, and it is subsequently determined that the column lands in an access way, limiting equipment deployment – but much more equipment could have been placed if the layout had been planned in advance
- Equipment deployments drift off the standard floor tile spacing and later high-density deployments are trapped, not having full tiles in the cold aisles, with a resultant loss of power density capability
Most existing data centers have one or more of the problems listed above, with the attendant losses of performance. In typical data centers routinely observed, the loss of available rack locations due to these problems is on the order of 10-20% of total rack locations, and the loss of power density capability is commonly 20% or more. These unnecessary losses in performance represent substantial financial losses to data center operators, but can be avoided by simple planning.
Risk of ignoring support columns when planning
The analysis above shows how critical the location of the support columns is to the equipment layout, and the consequences of not comprehending the effect of columns. The columns should be exactly located on any floor layout plans to avoid surprises.
To compound this problem, many building drawings do not show the correct dimensions for support columns. Actual columns often are built out larger than the original column dimensions during or after construction to accommodate wire or piping chases. Therefore it is essential to verify the actual column dimensions by direct measurement and not rely on the architectural drawings for the building.
Adding partitions without studying the effect on the equipment layout
Many data centers are partitioned for various reasons, including deployment phasing. Often these partitions are added late in the design process and without much forethought. Yet, locating partitions to maximize the performance of the data center is a science that requires considerable thought and planning. Columns are such a problem that it is frequently the best strategy to place a partition in line with as many columns as possible. In any case, using the guidance provided in earlier section “Phased Deployment” can help avoid the problem where a partition causes the loss of an entire row of equipment.
Floor layout is a critical step in the design process for large and small data center projects. When floor plans are not considered early in the planning process, the result can be irreversible compromises to the ultimate performance of the data center – including reduction in IT equipment capacity, reduction in power density capability, and increased electrical bills.
Many users assume they can’t create a floor plan early in the process because they don’t know exactly what IT devices are going to be deployed. This paper shows that it is not necessary to identify specific IT devices in advance for most designs, because most of the benefit of floor planning is independent of the specific IT devices deployed.A proper floor plan is the necessary foundation of an effective density and phasing plan. In fact, density and phasing plans without a floor plan are technically ambiguous and incomplete. By incorporating floor planning into a standardized design methodology, the design of data centers can become automated and predictable.