More than any other industry, electric supply is the most important attribute necessary for siting and operating a data center or mission-critical facility. Here we take a look at how utilities can affect the location and operation of data centers by understanding specific power requirements and costs, and review incentives offered by utilities in each state for data centers.
Data Center Power Requirements
Simply stated, data centers require a lot of power. That said, there is no “typical load.” Rather, a small data center, especially one housed within a standard office operation, could require less than 1MW while some of the larger data centers easily exceed 100MW. The greatest amount of market activity is generally in the 5-20MW range.
From a utility perspective, data centers have desirable load factors. Many data center operators will plan for a 90 percent load factor, although utility professionals frequently report the actual number is more typically around 80-85 percent.
The voltage characteristics will be dependent on the utility’s policy and, in most cases, are determined by the requested capacity and available service in the area. Most large facilities (over 10MW) will be fed via the utility’s transmission or sub-transmission systems. Service at this level will then require the data center operator to design, construct and maintain a substation to provide distribution voltage (13.2kV) to the facility. Due to the cost of a substation ($3-$7 million), users will seek opportunities to convert this cost from a capital cost to an operation cost, with the utility or a third party providing the substation.
Reliability and Uptime
Power is the lifeline of the data center. Without power, the data center ceases to function. Employees and customers can’t access information. Transactions stop. Financial trades aren’t possible. Planes stop flying. Health records can’t be accessed.
System interruptions are measured by the cost per second of downtime. In financial service operations, these costs can be millions of dollars per second, with downtime having a tremendous impact on customer satisfaction as well as creating embarrassing media coverage, as has been seen recently with the New York Stock Exchange and United Airlines. This is why many data centers are considered “mission critical.” Keeping the data center(s) up and running is job No. 1 for the CIO and site manager. Careers end when data centers go down.
For data center operations, the key measure is “uptime,” with success measured in “9s" (the percentage of time the data center is available is indicated by the number of nines; for example, 99.998 percent of the time would be considered “four 9s”). New data centers being built today are generally targeting three 9s or better, which works out to be less than nine hours of total downtime per year.
Adequate and reliable power is necessary to avoid such downtime. For an electric utility, data centers can be challenging customers. Data center operators are very demanding, requiring information on power quality factors, outage frequencies, causes and what has been or will be done to avoid future events.
Electric utilities measure uptime or reliability using three indices:
1. Customer Average Interruption Duration Index (CAIDI), measured in hours and minutes, tracks the average time a customer on the system is out of service.
2. System Average Interruption Duration Index (SAIDI), measured in hours and minutes, tracks the sum of all customer outage duration divided by the total customers on the system, typically over a year.
3. System Average Interruption Frequency Index (SAIFI) measures number of interruptions per customer.
Cost of Power
Power represents 60-70 percent of the total operational cost (TOC) of a data center. If able, a user will chase lower-cost power to reduce the overall operating cost of the facility. This, coupled with optimizing the data center design for energy conservation, reduces the TOC, as well as the facility’s carbon footprint. As a result, data center operators are beginning to consider alternative sources of energy to reduce cost. These include renewables, fuel cells and cogeneration.
Enterprise users, which typically have fewer business limitations on where they can locate, will make the cost one of the most heavily weighted selection factors. Many non-financial services enterprise users will decline to consider a location where the average cost will be above $0.07/kWh. Often, a rate less than $0.05/kWh is the target, and multiple data center operations have been able to secure prices below $0.04, particularly in areas utilizing hydro generation.
Colocation providers, on the other hand, tend to locate in close proximity to their customers and along robust fiber routes. Thus, we see a lot of data center activity in large metropolitan markets such as New York/New Jersey, Chicago and Dallas. In the last few years, secondary markets (non-NFL cities) have seen strong growth in data center development, including Columbus, Ohio, and Indianapolis, Indiana, in the Midwest. The colocation providers are less sensitive to national price differentials, but they work hard to ensure the lowest cost possible within their target geography.
However, the price per kWh is not the only material cost to be considered. Due to the large infrastructure requirements of a data center, another critical factor is the total cost required to provide electric service, and whether any of those costs are offset by credits or refunds via the utilities tariffs/service policies. It is necessary to analyze tariffs and rates for any hidden costs that may make a region less competitive, such as duplicate service cost, demand ratchets, revenue credit, security deposits and startup costs that negatively impact data center operations.
These costs can have a significant impact on the financial analysis. The greater the cost borne by the customer, the greater the chance the customer will choose an alternate location.
Redundant Service/Alternate Feeds
In addition to considering the general reliability of the utility’s system, and particularly its record at the site and within the surrounding area, the data center site selection teams look for utilities that understand and accommodate their need for redundancy.
Redundancy to a data center can be defined as an equally reliable power source to serve as a backup to the primary or main, source. The backup power source, in almost all cases, will be provided by the utility.
The alternate can be supplied via the same substation as long as a failure of the primary would not have a significant impact on the secondary supply feed. Smaller data center users can be accommodated on lower distribution voltages, without redundancy, but it is critical that the site have adequate capacity and proximity to the source to reduce the potential for outages.
Timing and Construction
Timing is also important in selecting a data center location. Capacity that “can be added” is not as favorable as existing available capacity due to the time it will take to make the needed upgrades. For this reason, it is imperative for the utility to communicate with the data center operator to assure adequate capacity can be available within the time frame requested.
The issue of timing also relates to the provision of information. The longer it takes to perform rate estimates, provide cost of service estimates or explore other policy questions, the more likely the customer will be to find another site that works, allowing the operation to be up and running faster.
Once a site for a data center has been defined, development will typically be fast-tracked with an 18-month construction completion schedule as the goal. When construction is complete, facilities and all equipment are commissioned for a period of time during which all critical systems are tested to their design limits. This commissioning process will require cooperation from the local utility to assure adequate capacity for testing is available and that the user’s demand, capacity, transmission and ratchet charges in the tariff are waived during the commissioning period. Once commissioning is complete, the site will go into commercial operation. Depending on the end use, loads will gradually grow over time, with an enterprise user growing more quickly and colocation growth dependent on lease activity and tenant equipment deployment. Enterprise users will typically achieve design limits over a several-year period. Meanwhile, co-location may possibly never achieve full design capacity.
Utility Incentives
States offer incentives to attract new businesses or expand existing facilities within their borders. Utility companies offer incentives for the same reasons states do: they have millions of dollars of investments in their service territories and many have capacity available along existing infrastructure. Most utility incentives are offered by electric companies, though some are offered by gas utilities. However, determining the available incentives is often difficult since utility service territories don’t typically fall along established state, city or county lines. Also, many states have more than one utility company and each will have different incentive options available for companies expanding or relocating into a particular service territory.
Utilities are particularly fond of data center projects, since a data center’s high electric load factor allows them to maximize their assets. Recently, a few utilities have incorporated load factor into their incentive rates.
The incentives offered by utility companies typically range from two to five years, with a few lasting longer. All rate discounts depreciate over time during their term and most have minimum kW demand or kWh threshold requirements, either determined as a new load or incremental load increase. The kW demand requirements ranges as low as 10- 25 kW, and while most utilities require a total demand of 200 kW or more to qualify, some require as much as 1,000 kW.
In the case of facility expansions, an increase over prior use, known as incremental load, must meet a threshold over the prior year. The incremental load forms the basis of the credit. In some special cases, some utilities are permitted to negotiate ‘off tariff’ rates.
Many utility economic development tariff riders are linked with the creation of new full-time employees or the approval of state or local economic development incentives in order to receive the discounted rate.
Each utility’s threshold to qualify for an economic development tariff, how the discount is applied, the term of the discount and the value of the discount varies widely. The characteristics of each utility’s incentive program must be considered along with state and local economic development incentive programs.
About the Author
