a GUI based grid computing framework that provides tools, daemons and libraries that facilitate users to submit jobs, monitor jobs, Dynamic participation of grid nodes, resource discovery. Best suited for high speed web crawling, Image processing and any job with sufficient degree of parallelism; Developed in C,C++
2. Grid Computing
▪ Grid computing is the combination of several computing resources from
several administrative domains or organizations applied to a common
task, usually scientific research areas.
▪ Grid computing is distributed, large scale clustered computing as well
as network distributed parallel processing.
▪ The main strategy of grid computing is using software to divide a job into
pieces and assign to different computing resources.
▪ Grid computing facilitate virtual organizations to access remote
computing resources to achieve common goal.
▪ Virtual organization refers to set of individuals defined around the
resource sharing policies, protocols.
2
3. Grid Computing Principle
3 Node
1) Task Request.
2
2) Task splitting into jobs (or
sub tasks) and job Task
submission
Control 2
3) Sub-results collection Node
Server
4) Final result aggregation
3
Result
2
3
Node
Node: Grid node with 2 : Sub task 3 : Sub results
Computing resources
3
5. DRMS (Distriuted Resource Management System)
• Batch processing
• User interface and a single point control for defining and monitoring
• In current scenario there are two popular DRMS available : Load Sharing Facility (LSF) by
Platform Computing Inc. and Sun Grid Engine (SGE) by Sun Microsystems
• Our Aim- UI for SGE & LSF or Develop a new DRMS ?
5
6. Types of Grid
Participating Nodes Nature of Nodes
• Cluster Grid • Compute Grid
• Departmental Grid • Data Grid
• Global Grid • Utility Grid
6
7. Pebble Grid Framework
• Grid Computing middleware that provides daemons, tools and shared
libraries to implement and extend the grid computing functionality
Features
• GUI
• Grid computing framework for Linux OS written in C/C++
• Daemon based implementation
• Job submission
• Job monitoring
• Dynamic load balancing & Rescheduling
• Dynamic participating nodes
• Highly scalable
• Middleware
• Framework: Shared Libraries*
• Implement Desktop Grid, Cluster Grid, Department Grid & Global
Grid + Data, Utility Grid
*framework is in progress
7
12. PGM-Scheduling & Monitoring
Master-Slave Architecture PGC-Execution Host
-Cluster Grid Job
PGM
Result
PGM-PGC connection
Execution
PGC PGC PGC PGC PGC
**New jobs are allocated as per the rate of execution of execution hosts
12
13. PGM-Scheduling &
Tree Architecture PGM Monitoring
PGC-Execution Host
-Department Grid
Job
-Global Grid
Result
PGM-PGC connection
PGM-PGM connection
-PGM acting as slave
PGM -Not a execution host
-Receive Jobs
Execution
PGC PGC PGC
**New jobs are allocated as per the rate
of execution of execution hosts
PGC PGC PGC
13
14. Peer-to-Peer(PGM) Architecture PGM-Scheduling & Monitoring
PGC-Execution Host
-Department Grid Job
-Global Grid
Result
PGM-PGC connection
PGM PGM PGM-PGM connection
-PGM acting as peer
-Not a execution host
-Send & Receive Jobs
Execution
PGC PGC PGC PGC PGC
**New jobs are allocated as per the rate of execution of execution hosts
14
15. pebshd: Scheduler Daemon
Dynamic Scheduling
• Priority Scheduling
• FIFO Scheduling
pebshd
Load Balancing
• Real time load
• Status
Queues
• TaskQ- Idle jobs
• StatQ- Running jobs
• FinQ- Finished jobs
Rescheduling TaskQ StatQ FinQ
• Timeout for each job
Resource Reservation*
• Scheduling as per nature of job
* Work in progress
15
16. libPebble*
• Shared Library
Functionality includes:
• Job class
• Job scheduler + Rescheduling policy
• Execution Host
• Support for Divide & Conquer jobs
• API for developer to extend the framework
Resources used
• CentOS 5.3
• C,C++,GCC
• Glib/GTK+
* Work in progress
16
17. Pebble Divide & Conquer Framework 100,000
numbers
• Framework to support distributed
computing on large data sets on clusters
• Inspired by Google’s MapReduce
framework
• Basis: Divide & Conquer To PGC Divider
20,000
• GUI / API in libPebble numbers
20,000
Applications
numbers
• Quick sorting of 100,000 no. 20,000 20,000
• Crawling the WWW numbers 20,000
numbers
• Image Processing numbers
& many more
Combiner
Final Output
17
18. Sy
st e
m
Property
Architecture Hierarchical, Peer Centralised Hierarchical, Peer
Implementation C, C++, POSIX C++, Win32, POSIX C#, Web
Technology services,.NET
Framework
Extendable Yes No Yes
GUI Yes Yes Yes
CLI Yes Yes No
Projects CSIRO Land &
Soon… SETI@home,
Water ,
Einstein@home,etc
FMI Biomedical
Research Inst, etc
User/Developer
Soon… Massive Good
Community
18