K. Borodulin, G. Radchenko: Principles of Computing Resources Planning in Cloud-Based Problem Solving Environment

1. Principles of planning
the computing resources
in cloud-based problem
solving environment
K. Borodulin and G. Radchenko
South Ural State University
The reported paper was supported by RFBR, research project No. 15-29-07959.

2. Problem definition
• Modern problems of computational science are
characterised by high demands on provided computing
resources and complex computing tasks structure, which
can be defined as a workflow.
• Also, problems of this type are characterised by usage of
multivariant calculations computing task is run hundreds
or thousands of times with different variations of the
input parameters.
2

3. 3
Example workflow for mixer simulation
Design
Modeler
CFX-Mesh
CFX-Pre
CFX-Solver
CFX-Post
Creation or
correction of a
geometric model
Creation or
correction of a mesh
Creation or
correction of problem
definition
Problem Simulation
in CFX-Solver
Visualization in CFX-
Post
Values ​​of the
optimization
criteria are not
satisfactory
The accuracy of
the calculation is
not satisfactory

4. Problem-solving
environment
• Problem-solving environment is a program system that
warps and provides a problem-oriented access to
computational resources to solve a specific class e-
Science problems
• This limitation would allow using a domain-specific
information of task to predict a computational
characteristics of the task in planning and scheduling
workflow’s execution, inflating the efficiency of available
computing resources’ consumption in cloud computing
system.
03.10.16 4

5. 5
Scheduler
Cloud Platform
DiVTB Web
Interface
A Driver
Simulation Results
Engineer
Distributed Virtual Test Bed
(DiVTB ) includes
 an interface for a problem
statement;
 a driver (a set of software tools
enabling the use of cloud resources
for virtual experiment);
 a set of services (a set of images of
virtual machines)
 a set of computing resources (a
cloud computing environment)
Distributed Virtual Test Bed

6. Goal and tasks
The aim of the paper is to describe the principles of
computing resources planning in Cloud-based Problem
Solving Environment.
To gain the aim of paper:
• Analyze related solutions for the planning of execution of
problem-solving workflow’s.
• Define a structure of cloud system for problem-solving
environment’s deployment.
• Describe a scheme of an approach for the computing
resources planning in Cloud-based problem-solving
environment.
03.10.16 6

7. Scheduling methods in
workflow systems
• Pandey, S., Wu, L., Guru, S.M., Buyya, R.: A particle swarm optimization-
based heuristic for scheduling workflow applications in cloud computing
environments. In: Proceedings - International Conference on Advanced
Information Networking and Applications, AINA. pp. 400407 (2010).
do not use information about previous executions.
• Sokolinsky, L.B., Shamakina, A. V.: Methods of resource management in
problem-oriented computing environment. Program. Comput. Softw. 42, 17–
26 (2016).
• Nepovinnykh, E.A., Radchenko, G.I.: Problem-Oriented Scheduling of Cloud
Applications : PO-HEFT Algorithm Case Study. 2016 39th Int. Conv. Inf.
Commun. Technol. Electron. Microelectron. MIPRO 2016 - Proc. 196–201
(2016).
scheduling the workflow actions’ execution - when the task has to run to
provide a minimal makespan, for example.
03.10.16 7

8. Model of cloud-based problem-
solving environment
• Components of Cloud-based problem-
solving environment
• Input data for planning the computing
resources
• Levels of planning the computing
resources in cloud-based problem solving
environment
03.10.16 8

9. Components of cloud
system
Base image
• OS
• Middleware
• Initialization
service
• agent of
remote task
execution
• Agent of
monitoring
system
Software
• Application
or software
for task’s
execution
• Service of
software
execution
Service
03.10.16 9

10. Virtual machine
Mem
CPU
Storag
e
Net
VMx
Service
1
Node
CPU Mem Net
Local
Storage
VM1
VM2
10
Network
Storage
03.10.16

11. Input data
• Executable workflow
• A set of the domain-specific
arguments’ values
• QoS
o Makespan
o A number of the computing resources
o Maximum of the cost
• Path of Input files
• Path of result files
03.10.16 11

12. VM
1
VM
2
VM3
VM4
Vm
6
VM
5
VM7
T1
T2
Node Node
VM
1
Vm
6
VM3
VM
2
VM4
VM7
VM
5T1
T2
Workflow level
Virtual machine Level
Computing nodes level
Servic
e
T1
T2
Service level
1203.10.16

13. Workflow level
The workflow layer implements the
transformation of the abstract workflow
into the executable job.
• The data sources of the input
parameters are being connected
with the certain tasks and sub-flows
of the workflow during the
transformation.
• The abstract workflow is executable
if input arguments of any task are
independent of the result of another
task’s execution.
03.10.16 13
Creation or
correction of a
geometric model
Creation or
correction of a mesh
Creation or
correction of problem
definition
Problem Simulation
in CFX-Solver
Visualization in CFX-
Post
Values ​​of the
optimization
criteria are not
satisfactory
The accuracy of
the calculation is
not satisfactory

14. Service level
The Service layer provides assignment of particular services to
the required computing resources for any task in the workflow.
The Workflow predictor sends to the Workflow planner the
following prediction information:
• time of the task execution (on the 1 computing core);
• the amount of main memory, needed for the task execution;
• maximum task scaling (how much cores can be provided to
the task);
• the amount of the result data;
• prediction accuracy for each value.
03.10.16 14

15. Virtual machine level
This layer performs the instances selection, using the
prediction of computing resources required for the certain
task execution, but
• Workflow executor can allocate another set of resources
for QoS satisfaction,
• If the prediction accuracy is low, i.e. most likely
prediction if false, then executor choose the type of
virtual machine which is default for the certain service.
03.10.16 15

16. Computing node level
The Computing node planning layer maps virtual machines’
onto computing nodes on the basis of a virtual machine
computing resources and a volume of node’s local storage.
At this layer, planner tends to place related virtual machine
from the workflow (which on this layer are presented as
Task-to-VM list on the same node to reduce the amount of
data are transferred between the computing nodes.
03.10.16 16

17. Tasks that need to be
addressed
Future work:
• Development of workflow applications planning algorithm
with effective virtual machines allocation and possibility
of dynamically adjustment of the execution plan of the
application.
• Development of a database to support an estimation of
execution characteristics of calculation tasks.
• Development of a model of workflow execution in cloud
computing environment.
• Development of an experimental “Problem-oriented
Scheduler” system
03.10.16 17

18. Thanks for your attention
03.10.16 18