Suche senden
Hochladen
Automatic Self-Tuning Architecture for Batch Scheduler on Large Scale Computing System
•
Als ODP, PDF herunterladen
•
2 gefällt mir
•
707 views
Sugree Phatanapherom
Folgen
Presentation of academic research with 140-char limit
Weniger lesen
Mehr lesen
Technologie
Business
Melden
Teilen
Melden
Teilen
1 von 83
Jetzt herunterladen
Empfohlen
Event Scheduling
Event Scheduling
Ayesha Kanwal
Â
Example Solutions for Scheduling and Work Planning
Example Solutions for Scheduling and Work Planning
SIS Group International
Â
Why Average Response Time is not a right measure of your web application's pe...
Why Average Response Time is not a right measure of your web application's pe...
vodQA
Â
Performance testing basics
Performance testing basics
Charu Anand
Â
Mining model for hotel recommendations (Kaggle Challenge)
Mining model for hotel recommendations (Kaggle Challenge)
Arjun Varma
Â
Time advance mehcanism
Time advance mehcanism
Nikhil Sharma
Â
Predictive control 1 introduction
Predictive control 1 introduction
jamestpp
Â
UC4 - One Automation
UC4 - One Automation
k1k2sdad
Â
Empfohlen
Event Scheduling
Event Scheduling
Ayesha Kanwal
Â
Example Solutions for Scheduling and Work Planning
Example Solutions for Scheduling and Work Planning
SIS Group International
Â
Why Average Response Time is not a right measure of your web application's pe...
Why Average Response Time is not a right measure of your web application's pe...
vodQA
Â
Performance testing basics
Performance testing basics
Charu Anand
Â
Mining model for hotel recommendations (Kaggle Challenge)
Mining model for hotel recommendations (Kaggle Challenge)
Arjun Varma
Â
Time advance mehcanism
Time advance mehcanism
Nikhil Sharma
Â
Predictive control 1 introduction
Predictive control 1 introduction
jamestpp
Â
UC4 - One Automation
UC4 - One Automation
k1k2sdad
Â
Data Mining and Analytics
Data Mining and Analytics
Nathaniel Palmer
Â
A New Approach for Job Scheduling Using Hybrid GA-ST Optimization-Crimson Pub...
A New Approach for Job Scheduling Using Hybrid GA-ST Optimization-Crimson Pub...
CrimsonPublishersRDMS
Â
Presenting an Algorithm for Tasks Scheduling in Grid Environment along with I...
Presenting an Algorithm for Tasks Scheduling in Grid Environment along with I...
Editor IJCATR
Â
Workforce Management & BPM Integration
Workforce Management & BPM Integration
Nathaniel Palmer
Â
Workforce Management & BPM Integration
Workforce Management & BPM Integration
Nathaniel Palmer
Â
genetic paper
genetic paper
Swathi Rampur
Â
Design and Implementation of a Multi-Agent System for the Job Shop Scheduling...
Design and Implementation of a Multi-Agent System for the Job Shop Scheduling...
CSCJournals
Â
MULTIPROCESSOR AND REAL TIME SCHEDULING.ppt
MULTIPROCESSOR AND REAL TIME SCHEDULING.ppt
CUO VEERANAN VEERANAN
Â
CS 23 Operating System Design Principles_MULTIPROCESSOR AND REAL TIME SCHEDULING
CS 23 Operating System Design Principles_MULTIPROCESSOR AND REAL TIME SCHEDULING
CUO VEERANAN VEERANAN
Â
International Journal of Computational Engineering Research(IJCER)
International Journal of Computational Engineering Research(IJCER)
ijceronline
Â
@SIMUL8 Virtual User Group, September: Brian Harrington, Less is More
@SIMUL8 Virtual User Group, September: Brian Harrington, Less is More
SIMUL8 Corporation
Â
Methods of Optimization in Machine Learning
Methods of Optimization in Machine Learning
Knoldus Inc.
Â
Carasik BPM ECM
Carasik BPM ECM
Bob Carasik
Â
Analytics for Process Excellence
Analytics for Process Excellence
Denis Gagné
Â
IRJET- Advance Approach for Load Balancing in Cloud Computing using (HMSO) Hy...
IRJET- Advance Approach for Load Balancing in Cloud Computing using (HMSO) Hy...
IRJET Journal
Â
performance
performance
manogallery
Â
G017314249
G017314249
IOSR Journals
Â
Improved Max-Min Scheduling Algorithm
Improved Max-Min Scheduling Algorithm
iosrjce
Â
STATISTICAL APPROACH TO DETERMINE MOST EFFICIENT VALUE FOR TIME QUANTUM IN RO...
STATISTICAL APPROACH TO DETERMINE MOST EFFICIENT VALUE FOR TIME QUANTUM IN RO...
ijcsit
Â
LEARNING SCHEDULER PARAMETERS FOR ADAPTIVE PREEMPTION
LEARNING SCHEDULER PARAMETERS FOR ADAPTIVE PREEMPTION
cscpconf
Â
Twitter, Facebook and etc: Quick Startup Guide for Marketing
Twitter, Facebook and etc: Quick Startup Guide for Marketing
Sugree Phatanapherom
Â
@sugree and Twitter
@sugree and Twitter
Sugree Phatanapherom
Â
Weitere ähnliche Inhalte
Ă„hnlich wie Automatic Self-Tuning Architecture for Batch Scheduler on Large Scale Computing System
Data Mining and Analytics
Data Mining and Analytics
Nathaniel Palmer
Â
A New Approach for Job Scheduling Using Hybrid GA-ST Optimization-Crimson Pub...
A New Approach for Job Scheduling Using Hybrid GA-ST Optimization-Crimson Pub...
CrimsonPublishersRDMS
Â
Presenting an Algorithm for Tasks Scheduling in Grid Environment along with I...
Presenting an Algorithm for Tasks Scheduling in Grid Environment along with I...
Editor IJCATR
Â
Workforce Management & BPM Integration
Workforce Management & BPM Integration
Nathaniel Palmer
Â
Workforce Management & BPM Integration
Workforce Management & BPM Integration
Nathaniel Palmer
Â
genetic paper
genetic paper
Swathi Rampur
Â
Design and Implementation of a Multi-Agent System for the Job Shop Scheduling...
Design and Implementation of a Multi-Agent System for the Job Shop Scheduling...
CSCJournals
Â
MULTIPROCESSOR AND REAL TIME SCHEDULING.ppt
MULTIPROCESSOR AND REAL TIME SCHEDULING.ppt
CUO VEERANAN VEERANAN
Â
CS 23 Operating System Design Principles_MULTIPROCESSOR AND REAL TIME SCHEDULING
CS 23 Operating System Design Principles_MULTIPROCESSOR AND REAL TIME SCHEDULING
CUO VEERANAN VEERANAN
Â
International Journal of Computational Engineering Research(IJCER)
International Journal of Computational Engineering Research(IJCER)
ijceronline
Â
@SIMUL8 Virtual User Group, September: Brian Harrington, Less is More
@SIMUL8 Virtual User Group, September: Brian Harrington, Less is More
SIMUL8 Corporation
Â
Methods of Optimization in Machine Learning
Methods of Optimization in Machine Learning
Knoldus Inc.
Â
Carasik BPM ECM
Carasik BPM ECM
Bob Carasik
Â
Analytics for Process Excellence
Analytics for Process Excellence
Denis Gagné
Â
IRJET- Advance Approach for Load Balancing in Cloud Computing using (HMSO) Hy...
IRJET- Advance Approach for Load Balancing in Cloud Computing using (HMSO) Hy...
IRJET Journal
Â
performance
performance
manogallery
Â
G017314249
G017314249
IOSR Journals
Â
Improved Max-Min Scheduling Algorithm
Improved Max-Min Scheduling Algorithm
iosrjce
Â
STATISTICAL APPROACH TO DETERMINE MOST EFFICIENT VALUE FOR TIME QUANTUM IN RO...
STATISTICAL APPROACH TO DETERMINE MOST EFFICIENT VALUE FOR TIME QUANTUM IN RO...
ijcsit
Â
LEARNING SCHEDULER PARAMETERS FOR ADAPTIVE PREEMPTION
LEARNING SCHEDULER PARAMETERS FOR ADAPTIVE PREEMPTION
cscpconf
Â
Ă„hnlich wie Automatic Self-Tuning Architecture for Batch Scheduler on Large Scale Computing System
(20)
Data Mining and Analytics
Data Mining and Analytics
Â
A New Approach for Job Scheduling Using Hybrid GA-ST Optimization-Crimson Pub...
A New Approach for Job Scheduling Using Hybrid GA-ST Optimization-Crimson Pub...
Â
Presenting an Algorithm for Tasks Scheduling in Grid Environment along with I...
Presenting an Algorithm for Tasks Scheduling in Grid Environment along with I...
Â
Workforce Management & BPM Integration
Workforce Management & BPM Integration
Â
Workforce Management & BPM Integration
Workforce Management & BPM Integration
Â
genetic paper
genetic paper
Â
Design and Implementation of a Multi-Agent System for the Job Shop Scheduling...
Design and Implementation of a Multi-Agent System for the Job Shop Scheduling...
Â
MULTIPROCESSOR AND REAL TIME SCHEDULING.ppt
MULTIPROCESSOR AND REAL TIME SCHEDULING.ppt
Â
CS 23 Operating System Design Principles_MULTIPROCESSOR AND REAL TIME SCHEDULING
CS 23 Operating System Design Principles_MULTIPROCESSOR AND REAL TIME SCHEDULING
Â
International Journal of Computational Engineering Research(IJCER)
International Journal of Computational Engineering Research(IJCER)
Â
@SIMUL8 Virtual User Group, September: Brian Harrington, Less is More
@SIMUL8 Virtual User Group, September: Brian Harrington, Less is More
Â
Methods of Optimization in Machine Learning
Methods of Optimization in Machine Learning
Â
Carasik BPM ECM
Carasik BPM ECM
Â
Analytics for Process Excellence
Analytics for Process Excellence
Â
IRJET- Advance Approach for Load Balancing in Cloud Computing using (HMSO) Hy...
IRJET- Advance Approach for Load Balancing in Cloud Computing using (HMSO) Hy...
Â
performance
performance
Â
G017314249
G017314249
Â
Improved Max-Min Scheduling Algorithm
Improved Max-Min Scheduling Algorithm
Â
STATISTICAL APPROACH TO DETERMINE MOST EFFICIENT VALUE FOR TIME QUANTUM IN RO...
STATISTICAL APPROACH TO DETERMINE MOST EFFICIENT VALUE FOR TIME QUANTUM IN RO...
Â
LEARNING SCHEDULER PARAMETERS FOR ADAPTIVE PREEMPTION
LEARNING SCHEDULER PARAMETERS FOR ADAPTIVE PREEMPTION
Â
Mehr von Sugree Phatanapherom
Twitter, Facebook and etc: Quick Startup Guide for Marketing
Twitter, Facebook and etc: Quick Startup Guide for Marketing
Sugree Phatanapherom
Â
@sugree and Twitter
@sugree and Twitter
Sugree Phatanapherom
Â
Behind the madness
Behind the madness
Sugree Phatanapherom
Â
drupal.in.th
drupal.in.th
Sugree Phatanapherom
Â
Twitter API and Startup Ideas
Twitter API and Startup Ideas
Sugree Phatanapherom
Â
Readme Read Sugree
Readme Read Sugree
Sugree Phatanapherom
Â
SCMSWeb and Condor-G Demonstration
SCMSWeb and Condor-G Demonstration
Sugree Phatanapherom
Â
Hand-on Resources II: Extending SCMSWeb
Hand-on Resources II: Extending SCMSWeb
Sugree Phatanapherom
Â
Drupal: blog and beyond
Drupal: blog and beyond
Sugree Phatanapherom
Â
The Spirit of Open Source
The Spirit of Open Source
Sugree Phatanapherom
Â
mbpurple - the replacement twitter im
mbpurple - the replacement twitter im
Sugree Phatanapherom
Â
jibjib - ultimate twitter client for your phone
jibjib - ultimate twitter client for your phone
Sugree Phatanapherom
Â
Next Web Application - Brainstorm
Next Web Application - Brainstorm
Sugree Phatanapherom
Â
Optimizing Drupal for Mobile Devices
Optimizing Drupal for Mobile Devices
Sugree Phatanapherom
Â
Call for Students: Google Summer of Code 2008
Call for Students: Google Summer of Code 2008
Sugree Phatanapherom
Â
Twitter Rules
Twitter Rules
Sugree Phatanapherom
Â
Mehr von Sugree Phatanapherom
(16)
Twitter, Facebook and etc: Quick Startup Guide for Marketing
Twitter, Facebook and etc: Quick Startup Guide for Marketing
Â
@sugree and Twitter
@sugree and Twitter
Â
Behind the madness
Behind the madness
Â
drupal.in.th
drupal.in.th
Â
Twitter API and Startup Ideas
Twitter API and Startup Ideas
Â
Readme Read Sugree
Readme Read Sugree
Â
SCMSWeb and Condor-G Demonstration
SCMSWeb and Condor-G Demonstration
Â
Hand-on Resources II: Extending SCMSWeb
Hand-on Resources II: Extending SCMSWeb
Â
Drupal: blog and beyond
Drupal: blog and beyond
Â
The Spirit of Open Source
The Spirit of Open Source
Â
mbpurple - the replacement twitter im
mbpurple - the replacement twitter im
Â
jibjib - ultimate twitter client for your phone
jibjib - ultimate twitter client for your phone
Â
Next Web Application - Brainstorm
Next Web Application - Brainstorm
Â
Optimizing Drupal for Mobile Devices
Optimizing Drupal for Mobile Devices
Â
Call for Students: Google Summer of Code 2008
Call for Students: Google Summer of Code 2008
Â
Twitter Rules
Twitter Rules
Â
KĂĽrzlich hochgeladen
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhi
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhi
soniya singh
Â
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
Gabriella Davis
Â
Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)
Allon Mureinik
Â
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
Rafal Los
Â
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreter
naman860154
Â
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
shyamraj55
Â
How to Remove Document Management Hurdles with X-Docs?
How to Remove Document Management Hurdles with X-Docs?
XfilesPro
Â
Understanding the Laravel MVC Architecture
Understanding the Laravel MVC Architecture
Pixlogix Infotech
Â
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf
hans926745
Â
How to convert PDF to text with Nanonets
How to convert PDF to text with Nanonets
naman860154
Â
AI as an Interface for Commercial Buildings
AI as an Interface for Commercial Buildings
Memoori
Â
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
carlostorres15106
Â
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day Presentation
Michael W. Hawkins
Â
Pigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food Manufacturing
Pigging Solutions
Â
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI Solutions
Enterprise Knowledge
Â
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR Systems
Mark Billinghurst
Â
Azure Monitor & Application Insight to monitor Infrastructure & Application
Azure Monitor & Application Insight to monitor Infrastructure & Application
AndikSusilo4
Â
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
ThousandEyes
Â
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptx
Malak Abu Hammad
Â
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
Safe Software
Â
KĂĽrzlich hochgeladen
(20)
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhi
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhi
Â
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
Â
Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)
Â
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
Â
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreter
Â
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Â
How to Remove Document Management Hurdles with X-Docs?
How to Remove Document Management Hurdles with X-Docs?
Â
Understanding the Laravel MVC Architecture
Understanding the Laravel MVC Architecture
Â
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf
Â
How to convert PDF to text with Nanonets
How to convert PDF to text with Nanonets
Â
AI as an Interface for Commercial Buildings
AI as an Interface for Commercial Buildings
Â
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Â
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day Presentation
Â
Pigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food Manufacturing
Â
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI Solutions
Â
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR Systems
Â
Azure Monitor & Application Insight to monitor Infrastructure & Application
Azure Monitor & Application Insight to monitor Infrastructure & Application
Â
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
Â
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptx
Â
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
Â
Automatic Self-Tuning Architecture for Batch Scheduler on Large Scale Computing System
1.
Automatic Self-Tuning Architecture
for Batch Scheduler on Large Scale Computing System
2.
I am Sugree
Phatanapherom from Kasetsart University.
3.
This research is
a co-work with Asst. Prof. Putchong Uthayopas.
4.
Ready, steady, go.
5.
What is batch
scheduler?
6.
Batch scheduler is
responsible to schedule jobs to execute on resources at the right time.
7.
Why do we
need batch scheduler?
8.
To utilize resources
efficiently.
9.
To finish all
jobs as fast as possible.
10.
To minimize power
consumption.
11.
In general, it
is so called "resource scheduling problem".
12.
Jobs, Resources and
Time time resources
13.
In this research,
main criteria is to minimize cost to run the resources.
14.
Back to the
past, most works focused on improving algorithms.
15.
To simplify the
problem, this research limits scope job characteristics to independent sequential jobs.
16.
In short, a
job contains the one and only one task.
17.
In other words,
job = task.
18.
Scheduling Algorithms Scheduling
On-line Batch RR OLB MET MCT MinMin MaxMin Sufferage XSufferage CMinMin CMaxMin CSufferage
19.
There are on-line
and batch scheduling.
20.
The most simple
algorithm is "Round Robin".
21.
"Opportunistic Load Balancing"
assigns job to the next available machine.
22.
"Minimum Execution Time"
assigns job to the fastest machine.
23.
"Minimum Completion Time"
assigns job to the machine with minimum completion time for that job.
24.
Next are batch
scheduling algorithms.
25.
"MinMin" assigns shortest
job to the fastest machine.
26.
"MaxMin" assign longest
job to the fastest machine.
27.
"Sufferage" is reassignable
MaxMin.
28.
"XSufferage" is Sufferage
with data locality.
29.
CMinMin, CMaxMin and
CSufferage are derivative with costing.
30.
How to verify?
How to evaluate?
31.
The answer is
simulation. Why?
32.
Closed. Controllable. Reproducible.
33.
Simulation is assumption
and modeling.
34.
Grid is a
meta-scheduler and underlying cluster schedulers managing hosts.
35.
Grid Grid Scheduler
Cluster Scheduler Host Cluster Scheduler Cluster Scheduler jobs Host
36.
Interconnection between scheduler
and processors are dedicated.
37.
Network Scheduler Processor
Storage Processor Processor Processor
38.
Job consists of
inputs, outputs and executable.
39.
Job Executable Input
Output Machine
40.
Operations are 2
steps; mapping and scheduling.
41.
Mapping "job" to
"machine".
42.
Schedule "job" to
the exact time.
43.
In short, the
result is generic priority index.
44.
Â
45.
Time ready time
execution time deadline period before deadline time
46.
Cost cumulative cost
cost cost
47.
Experimented based on
GAMESS job log in ThaiGrid to assume a small and a big system and named them, KUGrid and ThaiGrid, respectively.
48.
Makespan and cost
are observed.
49.
Makespan is the
period of time from when the first job submitted to the last job finished.
50.
Price-Performance
51.
Cost
52.
Makespan
53.
Looks great! Any
problems? Yes!
54.
Priority index contains
5 factors. What are the right values?
55.
What are the
factors of those factors?
56.
There are so
many dependencies. Job characteristics. Resource characteristics. User characteristics.
57.
This problem is
so called "Multi-variate Optimization".
58.
Plus, a bit
more complex with evaluation in simulator.
59.
How to solve?
60.
Optimization Architecture Optimizer
Simulator Simulator Simulator Simulator Batch Scheduler Monitoring System Accounting System
61.
Optimization Algorithm?
62.
Particle Swarm Optimization
is selected as the first one to try.
63.
The position of
each particle in n-dimension plane represents solution.
64.
PSO is social
influence in various scopes.
65.
Local, neighbor and
global.
66.
Usually, one trust
oneself, friends and the world, respectively. The level of trust.
67.
PSO
68.
How to fully
automate self-tuning process?
69.
Historical data are
the key.
70.
The quality of
solution depends on optimizer.
71.
Running optimizer longer
may return better solution.
72.
Precision of using
historical data depends on data period and amount of data.
73.
How to use
historical data? Log replay or estimation.
74.
How to maximize
solution quality to near optimal?
75.
Just run more
simulations using the whole grid system to optimize itself at night!
76.
Results? Please accept
my apologize. They are not published yet.
77.
Conclusion.
78.
Flexible algorithms introduce
more adjustable factors.
79.
The factors are
vary from time to time.
80.
In other view,
these algorithms are improved by external optimization periodically.
81.
Particle swarm optimization
is selected to solve multi-variate optimization.
82.
Improve scheduler by
scheduler itself.
83.
Any questions?
Jetzt herunterladen