1. Motivation
The continued progression in service and the demand for short response delays play a vital role in sha** the performance of large-scale, service-based environments. Cloud–fog computing is an example of such an environment developed to deliver fast executions and scalable services [
1,
2,
3,
4]. Fog computing in particular presents a decentralized computing infrastructure, in which services and resources could be connected to a cloud [
5,
6]. It is based on bringing the intelligence and processing power of the cloud to places close to where data are generated and acted upon, which are called edge devices or fog gateways. The goal is to process as much data as possible locally on fog computing nodes co-located with fog devices, so as to mitigate latency and bandwidth requirements of processing such data entirely on a remote cloud [
7,
8,
9].
Fog nodes are typically connected to distributed smart sensors and IoT devices that collect data from an operating environment [
10,
11,
12]. Each fog node by itself operates independently, as it decides on which data to process locally and which data to send remotely to the cloud for further analysis [
13,
14,
15]. Short-term jobs delivered from such sensor-based and IoT devices are processed locally in their corresponding fog nodes, whereas resource-intensive jobs are sent by fog nodes (in the form of fog jobs) to the cloud computing environment. Nevertheless, a huge volume of fog jobs is still transmitted by fog nodes to the cloud service provider for further processing and analysis [
16,
17]. Such jobs have various operational characteristics [
18,
19], time-sensitivities [
20,
21], energy constraints [
22,
23,
24], and service costs [
25,
26]. They tend to arrive in a random manner to the cloud service provider, as well as entail different QoS requirements and service demands that are to be fulfilled. Thus, reliability and network bandwidth are challenged in satisfying the obligations of such data in the cloud.
A cloud provider in turn experiences heterogeneous fog workloads of several timing variations and service difficulties [
27,
28,
29]. Such workloads increase not only in volume but also in complexity, which strictly forces execution latencies and energy complications on the cloud service provider [
30,
31]. The latter utilizes a pool of cloud resources in its data center to accommodate incoming fog workloads and thus allocates sufficient resources to achieve reliability and economies of scale. To execute such workloads, the service provider employs job scheduling and balancing schemes so that cloud resources are effectively utilized.
However, a client of a fog node typically tends to request a service that is both cost-efficient and high in performance [
32,
33]. To effectively meet such fog requests, a cloud service provider strives to maintain an efficient cost of service and energy consumption. Existing scheduling strategies often account for optimizing system performance based on response time and resource utilization metrics. Recent scheduling strategies start to incorporate factors of energy consumption when scheduling decisions are triggered.
A major limitation in such service schedulers is that they are not developed to manage mutual performance impacts between the fog service environment and the cloud computing environment. Such schedulers do not predict resource workloads, which is due to the lack of the load-management frameworks required to constantly measure system bottlenecks in fog environments along with cloud-resource queues.
Furthermore, such schedulers adopt allocation methods that focus on improving performance by only deciding on an optimal allocation of each individual fog job based on available cloud resources, so that particular performance metrics and QoS penalties are enhanced, in which, however, the energy constraints of fog jobs due to such backlog bottlenecks and SLA violations are not employed. As such, existing strategies do not optimize the energy efficiency performance based on the QoS obligations of fog jobs, resulting in the metric of measuring client satisfaction formulating schedules that lack a joint energy- and QoS-optimized performance.
3. Problem Statement
Recent advances in critical cloud–fog computing infrastructures as service-based dynamic environments have accelerated the deployment of intelligent devices to accomplish specific tasks and achieve market goals. Consider the example of applying IoT and sensory-based fog systems in vehicular networks. Some time-critical data, such as accident data, are of high importance to ambulance/police departments and must be sent to control systems equipped with powerful computing resources to take countermeasures. Delays in the processing of such data result in monetary and catastrophic effects, in which QoS requirements and agreements produce penalties reflective of such effects. Hence, the integration of such devices into cloud–fog computing environments constructs reliable networks that can further monitor, collect, analyze, and process data efficiently.
Constraints on energy consumption, response time, and bandwidth requirements are inevitable challenges that have further attracted the attention of researchers. A fog computing environment increasingly produces a large volume of jobs waiting to receive cloud services, each of which consumes resources and energy to strictly meet SLA obligations. The growing volume of fog data potentially produces backlog bottlenecks that cause execution difficulties on cloud computing resources, which are structured with computational processing power to tackle complex fog workloads.
Operation costs on fog clients and cloud service providers increase by increasing not only the cost of servicing fog jobs and SLA violation penalties but also the cost of the energy required to communicate and execute such fog demands. For instance, consider a fog node that requests a task to be serviced on a remote cloud within a specific tardiness limit. The longer the waiting and service times of the fog job in the cloud–fog environment, the higher the service cost and energy consumption required to meet the job’s demand. Similarly, the performance metrics of the response and execution times incur high performance costs when their time values increase in the service environment.
The research question arises when a scheduler formulates fog jobs for execution using the computing power of cloud resources so that QoS requirements are met while energy is concurrently saved. In this paper, the performance enhancement is focused on the side of the cloud computing environment, and thus the problem addressed is stated as follows:
Consider the case of fog nodes that deliver job workloads of various QoS expectations and energy demands to a cloud computing environment that comprises identical computing resources to service fog jobs. Each fog job is subject to SLA obligations that define constraints of service cost and execution energy. It is required to deploy and service fog jobs in the cloud computing environment such that energy is preserved and the cost of service is mitigated.
4. Background and Related Work
Cloud computing, as a distributed paradigm, hosts heterogeneous resources pooled in data centers to serve the demands of applications and IoT jobs [
1,
34,
35]. Such demands present challenges to the cloud computing environment in meeting the QoS obligations of clients, in which a cloud service provider strives to provision sufficient resources so as to meet the jobs’ execution demands and satisfy their SLA requirements [
36,
37]. However, the huge number of cloud resources allocated brings energy challenges to cloud data centers, in which energy consumption greatly increases and so does the cost of operation [
38,
39,
40]. Together with QoS, energy consumption in cloud computing has consequently attracted the attention of researchers from academia and industry [
41,
42,
43]. It becomes of paramount importance for a cloud provider to satisfy the QoS obligations of clients while simultaneously achieving energy efficiency based on QoS during the scheduling process.
The existing work in the literature presents various execution models and techniques to tackle such service challenges in the cloud–fog environment. A model for assigning tasks to servers is formulated in Dong et al. [
44] with the goal of minimizing the energy consumption of servers in the cloud data center. They propose a scheduling scheme that allocates tasks to a minimum number of servers while the response time of jobs is kept within an acceptable range, in which the scheme has proven its effectiveness against the random-based scheduling. Li et al. [
25] propose a load balancing model that ensures user satisfaction by efficiently executing tasks at a reduced cost. However, energy efficiency is not effectively incorporated with respect to the QoS penalties of schedules in the execution procedure.
Tadakamalla et al. [
45] present a model that controls the fraction of data processed remotely on cloud servers against the fraction of data processed locally on fog servers, where a utility function is proposed to optimize the performance metrics of average response time and cost. The task scheduling process has been optimized on a cloud–fog computing environment by Dang et al. [
46], in which efficient scheduling algorithms assign jobs among fog regions and clouds. Tsai et al. [
47] adopt an optimal task scheduling procedure that considers the operating cost and execution time of a task in a cloud–fog computing environment. The procedure particularly formulates globally optimal schedules that are computed based on task requirements and usage costs of resources. Nevertheless, jobs are modeled without considering energy factors when service decisions are triggered in the algorithm designs.
Furthermore, Guo et al. [
48] decided on the optimal scheduling of virtual machines on queues of the cloud system with heterogeneous workloads, but with considerations of energy in the scheduling process. In contrast, Anjos et al. [
49] present an algorithm that selects a suitable cloud or mobile-edge computing server to schedule IoT workloads, with the goal of achieving a better service time with a low cost. The energy required to perform such tasks are employed in the scheduling process, however, without correlating energy consumption with a QoS penalty of schedules formulated on resources.
In addition, an optimization framework to meet the deadlines of cloud applications is proposed in Alamro et al. [
50], in which they utilize a Probability of Completion before Deadlines (PoCD) metric to quantify the probability of a job to meet its deadline. Perret et al. [
51] present a deadline-based scheduler that orders jobs for execution in the cloud according to their laxity and locality, in which the algorithm demonstrates its efficacy against time-shared and space-shared scheduling algorithms. In both deadline schedulers, penalties for the energy consumption incurred due to executing a job and for violating deadlines of jobs are missing factors.
A delay-aware Earliest Deadline First (EDF) algorithm is proposed by Sharma et al. [
52] that allocates tasks for execution on a four-tier architecture. The algorithm demonstrates its effectiveness in improving the performance of energy consumption during the execution and scheduling processes of tasks. Wu et al. [
53] also present an energy-efficient scheduling algorithm that minimizes the energy consumption for IoT workflows. Xue et al. [
54] propose a scheduling algorithm to minimize the energy consumption in the cloud computing environment. They present a QoS model with respect to response time and throughput of jobs, as well as present an energy model for physical machines of the cloud environment. However, proposed algorithms do not measure mutual performance impacts between the energy consumption of machines and the QoS requirements of jobs.
In addition, the genetic algorithm, as a metaheuristic approach, has been extensively applied to mitigate the complexity of scheduling problems. Nguyen et al. [
55] tackle the scheduling process in cloud–fog computing systems by formulating a model that accounts for different performance constraints and applying metaheuristic approaches to solve a multi-objective optimization scheduling problem. Similarly, such approaches are applied by Ben-Alla et al. [
56] to propose a job scheduling method for cloud environments based on dynamic dispatch queues.
Arora et al. [
57] analyze popular first-come first-served, shortest job first, round robin, Min-Min, Max-Min, genetic, and ant colony optimization scheduling algorithms by comparing them in terms of response time and makespan. Thus, the genetic approach has proven its effectiveness in achieving the best performance on the metric of response time, whereas the ant colony optimization algorithm outperforms other scheduling algorithms in terms of makespan. In addition, the genetic approach has been utilized in Salido et al. [
58] to solve the job-shop scheduling problem and produce a good-quality, energy-efficient scheduling solution in a reasonable time. Their approach adopts machine resources that are modeled to consume different energy rates for processed jobs. Zhang et al. [
59] also minimize the energy consumption in a job-shop scheduling problem by utilizing a multi-objective, genetic-based approach.
In contrast, Lin et al. [
60] propose a framework in which they employ modern artificial intelligence techniques to overcome limitations of traditional heuristic-based scheduling algorithms and cope with dynamic changes in cloud environments. The framework utilizes the power of the deep learning approach to propose a model for scheduling jobs in cloud data centers. In addition, the framework adopts a deep Q-network model for resource allocation to deploy virtual machines to physical servers to execute jobs. Moreover, a scheduling scheme is proposed by Cui et al. [
61] to minimize average waiting times and makespans under different deadline constraints, in which a reinforcement-learning-based approach is utilized in a grid and in Infrastructure-as-a-Service (IaaS) cloud computing environments.
Furthermore, Zhang et al. [
62] propose a resource management framework for virtualized two-tier cloud data center environments, in which the framework demonstrates better performance in improving resource utilization and obtains an energy saving of 13.8%. The enhancement in energy consumption is also achieved in Zhao et al. [
63] by a multi-objective scheduling algorithm, as well as in Paul et al. [
64], in which a commonly used approach in control theory called model predictive control is utilized to address the scheduling problem for deferrable jobs in a tiered architecture data center.
Overall, the proposed scheduling methods in existing frameworks and models do not optimize the performance of energy efficiency based on the QoS obligations of fog jobs. It is required that client satisfaction is to be measured based on a metric that accounts for joint energy andQoS performance optimization, and hence, a pragmatic satisfaction between the fog jobs of the cloud environment and service providers is met. Such satisfactions are to be assessed by deriving a performance metric that measures the QoS of fog jobs so as to penalize the amounts of energy consumption and violations of service and subsequently formulare schedules across the cloud–fog computing environment.
5. Contributions
A service management framework is designed to incorporate the QoS penalty and energy consumption of fog jobs waiting for execution in resource queues of the cloud environment such that cost of service and energy is mitigated. The framework employs an analytical model for communication and computational performance metrics derived to calculate the service cost. In this perspective, the communication bandwidth is the performance metric that affects the QoS delivered to fog clients. A high bandwidth allocated to fog nodes can, for instance, mitigate the latency incurred from the transmission time of data jobs. Moreover, the resource demands of fog jobs proportionally influence transmission and computational energy, and as a result, energy constraints affect job latencies in that fog jobs with high resource demands demand high communication and computational energy consumption.
The management framework employs the following: (i) the allocation cost of the communication bandwidth assigned for a job delivered from a fog node; (ii) the waiting cost of a fog job in the resource queues of the cloud computing environment; (iii) the execution cost of a fog job in the cloud resource allocated for it; and (iv) the SLA violation cost if the service of a fog job does not meet its QoS deadline and tardiness constraints. The contributions of this paper are summarized as follows:
Designing a cost model based on a performance metric derived by utilizing QoS obligations and energy demands of fog jobs transmitted for execution in the cloud computing environment, in which the performance of energy efficiency is optimized based on the QoS of fog jobs;
Employing information of resource usage required by fog workloads to decide on their optimal allocation to cloud resources, so as to serve the demands of fog nodes such that the cost of service is mitigated;
Considering mutual performance impacts between quality metrics of fog jobs allocated for execution and factors of energy consumption required to service such jobs, so as to achieve pragmatic client satisfaction and mitigate the gross energy cost of job workloads queued for execution across the cloud–fog environment;
Mitigating the management complexity of the scheduling model so that schedules of minimum cost of QoS and energy penalties are evolved in a reasonable time.
Overall, schedules are formulated based on a joint energy- and QoS-optimized performance wherein the performance is predicted and evaluated using the cost of service and energy. Optimal schedules are thus formulated on queues of cloud resources such that the cost of service and energy are minimized, so as to maximize the probability of satisfied clients. The system performance is assessed on modeled fog jobs generated with heterogeneous service characteristics, energy demands, and SLA penalties.
6. Service Management Framework
The scheduling and allocation framework is developed to manage the execution of fog jobs in the cloud–fog computing environment. The framework is analyzed on a service-based environment modeled by employing a queuing system that represents a fog layer with IoT devices and a cloud layer with a job dispatcher associated with cloud resources.
Table 1 shows an alphabetical summary of the notations and concepts used in the paper.
6.1. System Architecture and Queuing System Model
The architecture of the cloud–fog computing environment consists of a fog tier and a cloud tier, as shown in
Figure 1. The fog tier comprises a set of fog devices interconnected together that deliver jobs of different service characteristics to fog nodes
as follows:
Jobs received by a fog node are atomic and independent of each other; they hold no information to be exchanged with other jobs from other fog nodes. Data that cannot be processed locally on fog nodes are transmitted to a remote cloud for further analysis and processing. In this paper, the performance enhancement is modeled to tackle the execution of those jobs sent to the cloud, in that it models the portion of the data analyzed and processed in the cloud. The cloud tier is structured with a number of homogeneous servers, each of which entails a queue with infinite capacity to buffer incoming fog jobs for execution in its computing resource. Factors incurred due to server failures and server-to-server communications are not considered.
The
queuing model is adopted to design the cloud computing system, where
N represents the number of cloud resources that exist in the system, as shown in
Figure 2. A system-queue, called a cloud dispatcher
, receives fog environment jobs and allocates them for execution on cloud resources for further processing. The arrival behavior of fog jobs to the dispatcher
of the cloud tier is modeled as a Poisson process. The time between consecutive arrivals of such fog jobs follows an exponential distribution with a particular arrival rate. The service demand of each fog job in a cloud resource is assumed to be known in advance based on prediction methods applied on incoming workload history to estimate a job’s execution time, and thus it is modeled from an exponential distribution with a service rate
[
65,
66].
Fog jobs allocated by the cloud dispatcher to resource queues are allowed to be both reordered in the same queue and migrated from one queue to another. A fog job can be executed by only one cloud resource at a time. A cloud resource can execute only one fog job at a time. Cloud resources are available to provide services at any time. The service discipline of such fog jobs in cloud resources is non-preemptive; a fog job cannot be interrupted once it starts data execution in a cloud resource.
6.2. Cost Analytical Model
The scheduling optimality problem is formalized for a given bag of fog jobs waiting to receive services from the cloud computing environment. The design of the cloud system employs a set of identical computing resources
, namely, virtual machines, available to service fog demands:
Each resource
in the cloud environment entails a queue
that holds incoming fog jobs waiting to receive service, which formulates a queuing system
that reflects resources
in the cloud environment as follows:
A set of
ℓ atomic, independent fog jobs
are delivered from fog nodes and received by the dispatcher
of the cloud environment as follows:
Fog jobs
arrive in a random manner to the cloud dispatcher
. The index
i of each fog job
indicates and signifies its arrival ordering to the dispatcher
. For instance,
is the first job to arrive,
is the second job, and so on. Jobs allocated by the dispatcher
are queued in cloud resources
for execution based on a scheduling order
described as follows:
where
represents indices of fog jobs in the resource-queue
. For instance,
signifies that fog jobs
,
,
, and
are queued in
such that fog job
precedes
, which in turn precedes
, and so on. It is assumed that
represents an allocation of a fog job
to either a queue
or a cloud resource
associated with that queue
as follows:
Since fog jobs are submitted by different fog nodes, they come to the cloud dispatcher with diverse computational demands and QoS obligations. Each fog job is thus stamped with a prescribed execution time and an arrival time , where denotes the service time required by a cloud resource to execute the demand of the fog job , whereas denotes the arrival time of fog job to the cloud dispatcher .
Each fog job
waits in the cloud tier to receive service from a cloud resource
. The time spent by a fog job
in the dispatcher’s queue is ignored, modeled by
:
However, the time spent by a fog job
in resource queues
is modeled by
, which is formalized according to a scheduling order
in the cloud tier. Once a fog job
receives a service and leaves the cloud tier, the time of departure is modeled by
, which in turn models a response time
that is a function of the execution time
of fog job
in a cloud resource
, and the total waiting time
of a fog job
governed by a scheduling order
in the cloud–fog environment so far, as follows:
where
models the waiting time of a fog job
in the fog environment, and
models the waiting time of fog job
in the cloud computing environment. Fog jobs
are governed by various SLAs, each of which entails a job’s service deadline
that in turn stipulates a target completion time
for the fog job
in the cloud environment. The
represents an explicit QoS obligation on the cloud service provider to complete the servicing of the fog job
, which incurs a waiting time allowance
that represents a service deadline
at the level of resource queuing
as follows:
For a fog job
that starts the service at its allocated cloud resource
, an SLA violation
occurs when its response time
is higher than its pre-defined service deadline
, which accordingly incurs a QoS penalty described as follows:
Utilizing such execution and service factors, a penalty cost and an energy cost are accordingly formalized to model and evaluate the system performance across the cloud–fog computing environment. The penalty cost and energy cost are formulated based on the communication and queue waiting across the cloud–fog environment, the SLA violation in the cloud environment, the service in the cloud environment, and the bandwidth allocation in the fog environment. Such QoS attributes are selected to measure system performance because they can be easily captured and predicted in the queuing system adopted to model the system design
6.2.1. Communication Penalty Cost for Bandwidth Allocation in Fog Environment
Each fog job
demands a pre-defined bandwidth requirement denoted by
allocated to communicate data between fog nodes and the cloud tier. A cost
of bandwidth usage is incurred per data unit of a fog job
, modeled by an exponential distribution of bandwidth penalty mean
as follows:
The communication bandwidth
allocated for a fog job
in the fog tier is subject to an SLA that stipulates an exponential bandwidth penalty cost curve modeled by
, formulating the total penalty cost of bandwidth usage per time unit of data as follows:
where
is a monetary cost factor for the bandwidth allocation penalty and
is an arbitrary scaling factor.
6.2.2. QoS Penalty Cost for Queue Waiting across the Cloud–Fog Environment
For each fog job
waiting in resource queues
of the cloud tier to receive service, there exists a waiting cost denoted by
for each time unit of waiting
, modeled by an exponential distribution with a waiting penalty mean
as follows:
As explained in (
8), the waiting
of a fog job
in the fog environment and its waiting
in the cloud environment compose a total waiting
. Thus, the waiting
of a fog job
reaching the cloud tier is subject to an SLA that stipulates an exponential waiting penalty cost curve modeled by
, formulating the penalty cost for each time unit of waiting
as follows:
where
is a monetary cost factor for the waiting penalty.
6.2.3. QoS Penalty Cost for Cloud Service in the Cloud Environment
After waiting for
in the cloud tier and
in total, a fog job
starts the execution
in a cloud resource
with a service cost denoted by
per time unit of execution, which is modeled by an exponential distribution with a penalty execution mean
as follows:
The service execution
of a fog job
in a cloud tier is subject to an SLA that stipulates an exponential service penalty cost curve modeled by
, forming the cost of servicing a fog job
in a cloud resource
as follows:
where
is a monetary cost factor for execution penalty.
6.2.4. QoS Penalty Cost for Cloud SLA Violation in the Cloud Environment
A violation in the SLA agreed upon with the cloud service provider is caused if a fog job
waits for a time longer than the waiting time allowance
prescribed in the SLA. A time unit of SLA violation
incurs an SLA cost
modeled by an exponential distribution with an SLA penalty mean
as follows:
The service-level violation
of a fog job
in a cloud tier is subject to an SLA that stipulates an exponential penalty cost curve modeled by
as follows:
where
is a monetary cost factor for the SLA violation penalty.
6.3. Problem Formulation: Minimum Cost of QoS and Energy Penalty
The problem is modeled by analyzing the performance penalty cost of QoS and energy for allocating and serving a fog job across the cloud–fog computing environment, represented by and , respectively.
6.3.1. Penalty Cost of QoS across the Cloud–Fog Environment
The total penalty cost of scheduling the stream ℓ across the cloud–fog computing environment is given by , which formulates the performance of:
The communication penalty cost of bandwidth allocated to transmit a fog job ;
The service penalty cost to execute a time unit of for a fog job in a cloud resource ;
The waiting penalty cost for each time unit of waiting to queue a fog job in resource queues of the cloud tier;
The violation penalty cost of not fulfilling SLA of a fog job .
Thus, the schedule penalty cost
across the cloud–fog computing environment is modeled by:
where
,
,
, and
are scaling factors for communication, service, waiting, and SLA-violation penalty costs, respectively.
The objective is to formalize the performance penalty cost by allocating a stream
ℓ of fog jobs
in the cloud tier with a scheduling order
, such that the QoS penalty cost
is minimized at the level of the cloud–fog computing environment, and thus the schedule performance is optimized, as follows:
Each cloud resource can only execute one fog job at a time. The service execution discipline of a fog job is non-preemptive; a fog job cannot be interrupted once it starts the execution on a cloud resource . Cloud resources are homogeneous, and hence the cost of servicing any fog job on any cloud resource is similar.
6.3.2. Penalty Cost of Energy across the Cloud–Fog Environment
Scheduling the stream ℓ of fog jobs across the cloud–fog computing environment incurs an energy cost that formulates the energy performance of bandwidth allocation , waiting in a resource queue in the cloud tier, service time in a cloud resource , and SLA violation with the cloud service provider.
As such, a fog job
is delivered by a fog device for execution in cloud resources
. Allocating a bandwidth
for a fog job
in the fog tier incurs an energy cost
per time unit of bandwidth allocation, that is, a function of an energy consumption per bit
u modeled by an exponential distribution with a rate
and a communication bit-rate
q modeled by a uniform distribution as follows:
which, as a result, incurs a total bandwidth energy cost
modeled by:
Once a fog job
arrives to the cloud computing environment, the cloud dispatcher
allocates a resource
that fulfills the job’s QoS waiting requirements with the least energy cost. There exists an energy cost
per time unit of waiting
in a resource queue
in the cloud tier modeled by an exponential distribution with a rate
, which accordingly incurs a total waiting energy cost
modeled by:
When a fog job
is delivered from a resource queue
to start the service in a cloud resource
, an energy cost
is incurred for each time unit of service
in the cloud resource
modeled by an exponential distribution
, which thus incurs a total service energy cost
modeled by:
If, however, an SLA violation occurs, an energy cost
is developed per each time unit of SLA violation
with the cloud service provider modeled by
, which therefore incurs a total SLA violation energy cost
modeled by:
As such, the entire cost of energy
for a stream
ℓ of fog jobs
across the cloud–fog computing environment is modeled by:
The objective is to formulate a schedule for a stream
ℓ of fog jobs
in the cloud tier with a scheduling order
such that the entire cost of energy
is minimized at the level of the cloud–fog computing environment, as follows:
7. Evaluation
The case study conducted in this paper evaluates the efficacy of the cost-aware scheduling framework in serving heterogeneous job workloads. The cloud–fog computing environment is built in Java, in which queues are utilized to implement the cloud layer. Service demands, QoS requirements, and service cost of energy for each fog job are generated using the mathematical model proposed in this paper. The implementation of the framework is coded using a Workstation with 8 GB main memory in a Core i7-8550U CPU @ 1.80 GHz 1.99 GHz.
7.1. Workload Characterizations and Design of the Cloud–Fog Computing Environment
The IoT layer consists of devices distributed throughout the fog environment, that are with various platforms and architectures. Such devices are modeled in this paper by sensors that detect, collect, and transmit data for processing in fog nodes and the cloud computing environment. Jobs delivered by such device sensors are thus heterogenous in their service demands, costs, and QoS requirements. The cloud layer consists of computing servers resided in data centers that provide on-demand services with high processing performance. A one-tier cloud layer is adopted with three servers , each of which respectively entails a queue to buffer fog jobs for execution and each server follows the queuing system.
Fog nodes
deliver a set of
ℓ atomic, independent fog jobs
to the dispatcher
of the cloud layer. The dispatcher
utilizes a scheduling strategy to allocates each fog job
on a particular cloud queue
for execution. Since the service demand
for each fog job
can be estimated beforehand using workload prediction models, the
is thus assumed to be known in advance and modeled by an exponential distribution with a service mean
as follows:
7.2. Modeling for Penalty Cost of QoS
The bandwidth penalty mean at the fog layer is set for
, which makes the cost
of bandwidth usage per data unit of a fog job
to be
, according to Equation (
13). At the cloud layer, the waiting penalty mean is to be
, and the execution penalty mean is to be
, which respectively produce a waiting cost
per time unit of waiting using Equation (
15) and a service cost
per time unit of execution according to Equation (
17). An SLA penalty mean is similarly set for
, which, as a result, incurs an SLA cost
per time unit of service violation as in Equation (
19).
7.3. Modeling for Penalty Cost of Energy
The energy model determines the consumption at the transmission stage in the fog layer and the waiting/execution stage in the cloud layer. The model presents that the energy consumed to execute a fog job in a cloud resource is higher than the energy required to hold a fog job in a cloud queue waiting for execution in the cloud resource . Yet, the bandwidth energy consumed at the fog layer required for transmitting a fog job to be executed at the cloud layer is higher than the energy consumed to execute such a job in a cloud resource .
At the fog layer, the energy consumption per bit
g (measured in Joule per bit) is modeled by the rate
, and hence,
, according to Equation (
25). The communication bit-rate (measured in bits per second) according to Equation (
26) is modeled by
. As a result, the energy cost per time unit of bandwidth allocation
is modeled by
, as in Equation (
27).
At the cloud layer, the energy cost per time unit of waiting
in a resource queue
is modeled by Equation (
29) to become
with the rate
. For a fog job
being serviced in a cloud resource
, the energy cost per time unit of service
in the cloud resource
is modeled using Equation (
31) by
with the rate
. Similarly, the energy cost developed per time unit of SLA violation
with the cloud service provider is modeled by
using Equation (
33) with the rate
. It is shown that rates of execution
and SLA violation
are modeled to be higher than the rate of waiting
, but to be lower than the rate of data communication
.
7.4. The Genetic Approach
The process of scheduling fog jobs
in the cloud layer such that the QoS penalty cost
and the energy penalty cost
are mitigated is an NP problem. The huge number of fog jobs
received at the cloud layer makes it difficult to formulate cost-optimal schedules in a timely manner. However, the permutation genetic algorithm, as a meta-heuristic search strategy, demonstrates its effectiveness in such cases [
67,
68,
69]. The genetic algorithm and virtualized-queue design scheme proposed in [
41,
65,
66] demonstrate their effectiveness in efficiently exploring and exploiting the scheduling space such that a near-optimal schedule of jobs is formed in a reasonable time, which are adopted in this paper to find a cost near-optimal schedule of fog jobs
at the cloud layer.
As such, a fitness function is formed to evaluate the quality of each virtualized queue (chromosome). The fitness value of a chromosome r in a generation G represents the penalty cost of the QoS and the penalty cost of energy, each of which computes a normalized fitness value for each schedule candidate. Accordingly, Russian Roulette is used to select a set of schedule candidates to produce the population of the next generation using crossover and mutation operators. Two fitness values are presented: to represent the fitness of the cost of QoS penalty and to represent the fitness of the cost of energy penalty. The Single-Point crossover and Insert mutation genetic operators are utilized to evolve the schedule of fog jobs at the cloud layer. The rates of such operators are both set to be of the population size in each generation. The population size is set to 10, and the maximum number of tours is set to 3000.
7.5. Discussions on Obtained Results
Findings of applying the cost-aware scheduling framework across the cloud–fog computing environment validates the performance and demonstrates the framework’s effectiveness in mitigating cost of the QoS penalty and cost of energy for formulated schedules at the cloud layer.
7.6. Cost of QoS Penalty of Schedules
The schedules of fog jobs
at the cloud layer are formulated on cloud resources to mitigate the cost
of the QoS penalty. The performance optimality is measured by evaluating the quality of formulated schedules on cloud queues
.
Table 2 presents the assessment of the QoS penalty cost
by utilizing the genetic approach, where a system state of a virtualized-queue for 30 fog jobs is evaluated.
The cost
of schedule in the initial state is
, which results in a
QoS penalty. When the genetic algorithm is employed, the cost
of schedule is accordingly enhanced to a near-optimal value of
with a
QoS penalty. Thus, the cost
and QoS penalty of schedule are improved by
and
, respectively.
Figure 3 corroborates such findings and shows the mitigation of the QoS penalty cost
for fog jobs
at the level of cloud layer, in which the genetic algorithm utilizes only 3000 iterations to reach a near-optimal penalty cost.
Furthermore, the QoS penalty cost
is calculated at the level of each resource queue at the cloud layer. For that, queue
in
Table 2 entails 16 fog jobs organized in a virtualized-queue. The cost
of QoS in the initial system state is
, which produces a
penalty. The cost
is enhanced to become
with a
penalty. The cost
and penalty are improved by
and
, respectively.
Figure 4a affirms such improvements and assures the effectiveness of the genetic algorithm along with the virtualized-queue design scheme in enhancing the cost performance
of the QoS penalty for fog jobs of queue
in only 600 iterations.
In addition, the cost
and penalty of queue
in
Table 2 are improved by
and
, respectively. Similarly for the virtualized queue of
with five fog jobs, the improvements are
on the schedule cost
and
on the penalty. The cost of the QoS penalty in
Figure 4b for the virtualized queue of
and
Figure 4c for the virtualized queue of
foster such findings in a reasonable time, which overall emphasizes the performance of reaching a near-optimal QoS penalty cost
within 200 and 100 iterations, respectively.
7.7. Cost of the Energy Penalty of Schedules
The schedule optimality is evaluated by computing the cost
of the energy penalty at the cloud layer. Fog jobs
are allocated on resource queues
of the cloud by utilizing the virtualized queue design scheme.
Table 3 presents the cost
of the energy penalty for job schedules at the cloud layer where an allocation of 30 fog jobs on a virtualized queue is assessed.
The energy cost
at the cloud layer is initially
, which carries a
penalty. The genetic approach is applied along with the virtualized queue design scheme, and hence, the energy cost
is enhanced to become
with a
penalty. It is shown that improvements on the cost
and penalty reach
and
, respectively.
Figure 5 demonstrates such conclusions and shows the efficacy of the genetic algorithm in formulating a near-optimal energy cost schedule in only 3000 iterations for a virtualized queue of 30 fog jobs.
In addition, the energy cost
and penalty are measured at the level of each cloud queue. For instance, the allocation of nine fog jobs of
on a virtualized queue produces an initial energy cost
of
with a
penalty, which is enhanced to
with a
penalty, as shown in
Table 3. The improvements achieved on the energy cost
and penalty of queue
are
and
, respectively.
Similarly, the improvements are proven on queues
and
. For queue
, the energy cost
is mitigated from
to
with a
enhancement, whereas the penalty is mitigated from
to
with a
reduction. For queue
, reductions in the energy cost
and penalty reach
and
, respectively.
Figure 6a–c illustrate the mitigation of the cost
of energy penalty in a reasonable time for queues
,
, and
, respectively, wherein only 200 iterations are utilized to reach a near-optimal cost
of energy penalty.
8. Conclusions
The cost-aware framework demonstrates its efficacy in managing the allocation and execution of fog jobs in a cloud–fog computing environment. Scheduling and load balancing decisions are frequently triggered at run-time such that quality and service obligations of fog jobs are fulfilled. It is shown that the framework emphasizes the notion of energy-efficient scheduling based on the QoS penalty of fog jobs, in which the formulations of scheduling decisions in the framework tolerate risks of delays and energy on the cost performance.
The scheduling mechanisms employed in the framework demonstrate the effectiveness of decisions in incorporating the impacts of SLA obligations and energy incurred due to communication, service, and waiting performance metrics on cost reduction. Such decisions mitigate the cost of energy and cost of QoS penalty required to execute fog workloads, as well as cope with heterogeneity and variations in IoT workloads experienced in the cloud computing environment with considerations to SLA obligations for each fog job. The improvement in energy cost at the level of the cloud tier has reached around , which results in around a enhancement in the QoS penalty. At the queuing level of the tier, the improvements in cost and penalty reach around and , respectively.
The genetic-based approach utilized in the framework shows a great enhancement in forming near-optimal schedules in a reasonable time, and the approach improves the cost performance of energy and QoS penalties as well. It is shown that an optimal schedule with a reduced cost of energy penalty is formulated at the queuing level of the cloud tier by utilizing only 200 genetic iterations. In addition, only 3000 genetic iterations are employed to mitigate the cost of the energy penalty at the tier level of the cloud environment. Future directions include proposing a resource allocation framework, in which the goal is to decide on an optimal set of resource configurations and setups such that QoS requirements are met. It involves proposing an SLA penalty and profit models based on workloads’ heterogeneity and client demands for resources, that are to be utilized by the framework so that client satisfactions are maximized.