Request–Response Distributed Power Management in Cloud Data Centers

1 Introduction

Due to the advantages in deployment, management, reliability, and cost, cloud computing services are growing rapidly in recent years, resulting in the establishment of large-scale cloud data centers containing thousands of server nodes (SNs) and consuming enormous amount of electrical energy.

Power provision is increasingly becoming the most important constraint to its development. In 2006, US data centers consumed an estimated 61 billion kilowatt-hours of energy for the total electricity cost of about 4.5 billion, enough to power 5.8 million average US households [3].

Meanwhile, most of the times, the resource utilization in the data center is very low. Data collected from more than 5000 production servers over a 6-month period have shown that although servers usually are not idle, the utilization rarely approaches 100%. This is because servers operate at 10–50% of their full capacity [2].

Improving resource utilization and optimizing power consumption are the major concerns for cloud data centers today. Most mechanisms are introduced to optimize the power consumption of computing devices according to the loads at any particular moment, such as dynamic voltage/frequency scaling (DVFS) [6], power-on–power-off [9], application migration, and consolidation. However, an effective asynchronous power management method, which synthetically makes use of these mechanisms, is still lacking.

The asynchronies and abruptness in workload fluctuation in the cloud data center make power management policymaking using periodic resource scheduling method invalid. To address this problem, in this article, a request–response distributed power management scheme (RRDPM) for the cloud data center has been suggested. The implementation details are also furnished.

The work presented here has two key contributions: first, the integration of RRDPM and DVFS, power-on–power-off, and virtual machine (VM) migration mechanisms were investigated to dynamically optimize the power consumption; second, the design and implementation of algorithms pertaining to RRDPM, which include the local decision algorithm of SNs and the global response decision algorithm on the power management server. Simulation results show that RRDPM can effectively decrease the power consumption in a data center with tiny reduction in performance. The rest of this article is organized as follows. Section 2 discusses related works. In Section 3, the RRDPM model is described. All the algorithms involved in the system are designed in Section 4. In Section 5, the effectiveness of RRDPM as evaluated through simulation experiments is briefed. The article concludes in Section 6.

2 Related Work

Much effort has been made to address power efficiency both in the computing device and in the data center. Albers [1] has reviewed the mechanisms that minimize energy consumption in a single computing device, which include power-down mechanisms, dynamic speed scaling, and power-efficient scheduling algorithms.

Pinheiro etal. [9] first put forward the idea of using a power-on–power-off mechanism to save power under a lighter load in cluster systems. Because predicting application performance is indispensable and difficult for satisfying the SLA requirements when improving power efficiency, it is proposed to predict the performance by keeping track of the demand for (not the utilization of) resources on all cluster nodes. This method is used as a reference to predict the resource demands of every VM and to ensure the performance requirements of applications packaged in VMs.

According to Chen etal. [5], the DVFS implementation for a server cluster can be broadly classified based on whether (i) the control is completely decentralized, where each node independently makes its scaling choice purely on local information or (ii) there is a coordinated (perhaps centralized) control mechanism that regulates the operation of each node. A decentralized DVS control is attractive from the implementation viewpoint, and a previous research [6] has shown that a coordinated voltage scaling approach can provide substantially higher savings.

For power management in the data center, Chen etal. [5] used a coordinated DVFS strategy where a controller was assigned for each application, and it periodically assigned the operating frequency/voltage for the servers running that application. The author assumed that each server continues to be entirely devoted to a single application until the time of server reallocation, but this is not true in the cloud computing environment. Because the load for every application hosted in every SN and the corresponding SLA requirements can be quite different, it is hard to deploy this power management strategy at the application granularity in the cloud computing environment.

Beloglazov and Buyya [2] used a centralized algorithm to periodically select VMs needed to migrate from the overloaded and underloaded SNs and compute the optimal migration destination SNs all over the data center for these VMs. The perfection in simulation results depended on the constant resource utilization of every VM in every period and the whole synchronism of all the VM’s workload fluctuation periods. In fact, a VM’s workload fluctuation is asynchronous in the data center.

In addition to the decrease in power consumption in a data center, recent works [7, 13] aimed to reduce the electric utility bill basing on the electricity price difference in different regions and different periods.

3 System Model

The cloud data center is composed of massive physical SNs, and every SN has many VMs running on it; VMs are the basic executive entities and the providers of services specified in the SLAs.

For more power consumption efficiency, SN periodically determines its optimal operating frequency, using DVFS according to the workload fluctuation. However, from the overview of the whole data center, we expect that a data center can adapt to power the fewest SNs operating at the highest frequency with the optimal resource utilization combination by means of VM migration and power-on–power-off to decrease the ratio of idle power wastage. However, this is an NP hard problem because loading SNs to a desired utilization level for each resource can be modeled as a multidimensional bin-packing problem where SNs are bins with each resource being one dimension of the bin [12].

Different from other’s solution, we put forward the RRDPM scheme. First, SNs periodically make their local optimal power decision every time T_L according to current resource demands. There are five kinds of decision conclusion for an SN:

• To increase or decrease an SN’s operating frequency owing to workload fluctuation within a narrow range;

• To emigrate one VM from SN owing to overload or to emigrate all VMs from SN and turn off owing to underload;

• To keep the SN’s current state when workload fluctuation is negligible.

Although SN decides to change its power state, it sends the power state transition request (PSTR) to the power management server node (PMN) to make the global optimal decision. If an SN requires the outside migration of its VM, the PMN will select an optimal destination SN to accommodate it. These SNs that require to decrease their CPU frequency or to emigrate all their VMs may be the potential receiver for VM migration; thus, the SN’s decision to decrease the operating frequency should be agreed by the PMN. On the contrary, when an SN decides to increase its CPU frequency, it can immediately carry out without asking for the PMN’s acknowledgment, which can assure a quick response to workload increase. The objective of the PMN making the global decision is to minimize the total power consumption of the integral data center. At last, according to responsive information, SNs execute their state transition.

An RRDPM has the following characteristics:

1. Asynchronism: Whenever an SN detects workload fluctuation, it can immediately make a response. Unlike the methods used by Beloglazov and Buyya [2], RRDPM has no synchronization limitation of periodic migration scheduling. Typically, the interval T_L between demand computations is 1 s, which means an RRDPM can grasp a tiny change in workloads.

2. Low overhead: The PMN obtains an SN’s operating state information only through the SN’s PSTRs, without proactively getting SN information.

3. Distributed computation: Distributed computation in SNs and PMN can alleviate PMN computation pressure unlike centralized control algorithms.

4 Implement-Related Algorithm

4.1 SN-Side Local Optimal Decision

4.1.1 Predict Resource Demands of VM Satisfying SLA

In the cloud computing model, every application is encapsulated in a VM of SN; the VM monitor (VMM) of SN supervises the resource utilization of every VM and is responsible for resource distribution.

To satisfy every application’s performance requirement, the VMM has to distribute enough resource share to every VM. Due to the dynamic change in workload, the VMM needs to periodically predict the resource demands of every VM based on their recent resource utilization information and performance measurement. According to Pinheiro etal. [9], the CPU demand is computed by reading the information from the /proc directory, and the network and disk demands are computed based on server internal information. To smooth out short bursts of activity, each of these demands is exponentially amortized over time using

We assume every application’s SLA depends on r kinds of computing resources (such as CPU, memory, etc.), and the set of VMs on SN, SN_j, is denoted as Ω_j. For VM_i ∈ Ω_j, the predicted demand for the kth resource is d_i,k (k = 1, 2,…, r). Obviously, the total demand for the kth resource on SN_j is

4.1.2 SN-Side Local Power Optimal Decision

The higher the resource utilization of SN, the lower the idle power wastage will be; however, it also induces lower application performance, and thus, there is a power optimal resource utilization of SN where the energy per transaction is the minimum [12]. We assume that the power optimal resource utilization combination of SN_j is

The objective of SN-side local power management is to operate the SN running in the vicinity of the optimal resource utilization combination point by means of DVFS or VM migration mechanism. Using DVFS, SN dynamically manipulates its operating frequency to control the CPU resource provision, whereas using VM migration to change set Ω_j, we can load SN_j to operate at a different utilization level.

We used S_j,k and U_j,k to denote the total provision and utilization, respectively, of the kth resource in SN_j, F_j and f_j, which in turn denote the alternative CPU frequency set and the current CPU operating frequency in SN_j, respectively. For convenience, the frequency f_j is normalized to the ratio to maximum frequency, i.e., 0 ≤ f_j ≤ 1, and the resource index k = 1 denotes the CPU. Thus, the CPU utilization of SN_j is

and the other resource utilizations are

We denote the combination of current resource utilization of SN_j as

Periodically, according to the predicted resource utilizations, SN has to first check if it is overloaded or underloaded.

Assume

If δ is smaller than the minimum utilization threshold beyond a certain time threshold, then it means that SN_j is underloaded. To reduce the idle power consumption, SN_j should ask for the PMN to migrate all its VMs to other servers and wait to be shut down.

SN_j is overloaded if there exists k satisfying D_j,k ≥ S_j,k. To not violate the applications’ SLAs homed in SN_j, SN_j has to migrate some VMs to other servers. Because the cost of migrating the VM between SNs is high, we assume that every overloaded SN only migrate one VM once.

The closer the distance between U_j and optU_j, the more efficient the power consumption will be. Thus, the optimal selection objective of VM to migration is to minimize the distance dis(U_j, optU_j) between U_j and optU_j.

We can model VM selection from overloaded SN as the following optimization problem:

where we define

and ω_k(k = 1, 2,…, r) denote the weight coefficient of the kth resource utilization’s effect on power consumption. The weight coefficients, ω_k, can be derived from experiments as in Ref. [12]. This optimization problem can be solved in O(|Ω_j|) time through a simple comparison of different function values of selected |Ω_j| VMs.

If an SN_j neither overloaded nor underloaded, SN_j should adjust its frequency to optimize its resource utilization. Accordingly, we model the optimal CPU frequency selection as the following optimization problem:

Similarly, the frequency selection optimization problem can be solved in O(|F_j|) time.

5 Experimental Evaluation

In this section, we demonstrate the effectiveness of the RRDPM through a simulation experiment.

5.2 Simulation Results and Analysis

Using the workload data described in Table 1, we have simulated three methods (RRDPM, IQR_MC_1.5, and DVFS), and showed the results in Table 6. Without VM migration in the DVFS method, every SN undertakes the lighter workloads; thus, the SLA violation of the DVFS method for every day trace is zero (omitted in Table 2) and the energy consumption is maximal, which is a benchmark evaluating an energy-efficient algorithm. Compared with the DVFS method, RRDPM can greatly save approximately 75% energy consumption with the cost of approximately 3% SLA violation time. Relative to IRQ_MC_1.5, RRDPM decreases 55% SLA violation time and 90% VM migrations with the cost of increasing 50% energy consumption. There are several reasons why this comparison is conservative. First, the 5-min scheduling period of the workload trace, instead of the several seconds of the scheduling period, makes RRDPM spend more time in reaching the optimal resource utilizations, whereas for IRQ_MC_1.5 to reach the optimal state consumes only several seconds at the beginning of every scheduling period. Second, the RRDPM scheme can also use the adaptive utilization threshold to improve its performance. The relationship of energy consumption with simulation time further shows the difference between the distributed RRDPM and the centralized method IRQ_MC_1.5 (Figure 1). In Figure 1, the curve of RRDPM has a greater gradient slope at the beginning of the simulation and gradually becomes linear, whereas the cure of IRQ_MC_1.5 is always a straight line throughout the simulation. This is because the workload is very light, with an average CPU utilization of 12.31%, which causes the RRDPM to waste more idle power in the period from the beginning of the simulation to the time when the optimal resource utilization was reached.

Table 2.

Simulation Results.

Date	RRDPM			IQR_MC_1.5			DVFS
	Energy consumption (kW-h)	SLA violation (%)	Number of VM migration	Energy consumption (kW-h)	SLA violation (%)	Number of VM migrations	Energy consumption (kW-h)
03/03/2011	209.29	2.25	2454	144.76	5.59	25,774	786.58
06/03/2011	149.85	2.93	2392	108.37	5.62	20,379	620.68
09/03/2011	188.92	2.51	3110	124.57	6.07	26,077	697.74
22/03/2011	242.75	2.55	3820	149.36	6.05	33,169	999.38
25/03/2011	197.54	2.92	2624	129.51	5.67	24,963	766.65
03/04/2011	277.78	2.72	3287	199.68	6.31	39,709	1074.28
09/04/2011	243.95	2.16	3266	155.95	5.97	31,485	921.22
11/04/2011	223.99	2.54	2953	151.67	5.93	30,427	880.12
12/04/2011	194.10	2.72	2693	133.54	5.92	26,129	756.05
20/04/2011	176.93	3.47	2589	110.58	6.56	29,122	688.48

Figure 1.

Energy Consumption of the Data Center.

To further demonstrate the RRDPM scheme’s effectiveness, Figure 2 depicts the CPU utilization distribution of all active SNs for the March 3, 2011, workload. The figure shows that most of the active SNs’ CPU utilizations are below 20% at the beginning of the simulation; after 25,000 s, the utilization of most active SNs increased to approximately 50%. The average CPU utilization and its standard deviation of active SNs are shown in Figure 3.

Figure 2.

CPU Utilization Distribution of all Active SNs.

Figure 3.

Statistical Property of Active SNs CPU Utilization.

6 Conclusions

Power provision is becoming the most important constraint to cloud computing development. Improving the resource utilization and reducing idle power consumption are the fundamental issues in power optimization and management. The purposed RRDPM elucidated in this article synthesizes DVFS, power-on–power-off, and VM migration mechanisms to dynamically optimize the power consumption in a cloud data center. An excessive resource utilization induces poor service performance, whereas a very low resource utilization results in too much idle power consumption, and RRDPM uses resource demand prediction to make the request decision of the power state transition. Resource demand prediction every other second can easily detect workload abruptness and make the response to workload fluctuation more prompt. Simulation results show that RRDPM can effectively decrease the power consumption of a data center with a tiny reduction in performance. Compared with other methods, the significant advantage of RRDPM is that it is a distributed asynchronous method with a low computation overhead. Although the initial average resource utilization of SNs is very low, RRDPM requires more time to reach the optimal state. If convergence is shortened by implanting a centralized control method at the initial phase of RRDPM, the idle power consumption can be further decreased, and this the research plan set forth for future studies.