a GUI based grid computing framework that provides tools, daemons and libraries that facilitate users to submit jobs, monitor jobs, Dynamic participation of grid nodes, resource discovery. Best suited for high speed web crawling, Image processing and any job with sufficient degree of parallelism; Developed in C,C++

1. Kartik Kumar Perisetla
kartik.peri@gmail.com

2. Grid Computing
▪ Grid computing is the combination of several computing resources from
several administrative domains or organizations applied to a common
task, usually scientific research areas.
▪ Grid computing is distributed, large scale clustered computing as well
as network distributed parallel processing.
▪ The main strategy of grid computing is using software to divide a job into
pieces and assign to different computing resources.
▪ Grid computing facilitate virtual organizations to access remote
computing resources to achieve common goal.
▪ Virtual organization refers to set of individuals defined around the
resource sharing policies, protocols.
2

3. Grid Computing Principle
3 Node
1) Task Request.
2
2) Task splitting into jobs (or
sub tasks) and job Task
submission
Control 2
3) Sub-results collection Node
Server
4) Final result aggregation
3
Result
2
3
Node
Node: Grid node with 2 : Sub task 3 : Sub results
Computing resources
3

4. Grid Computing Architecture
Model OSI Model
Application
Transport
Network
Link
4

5. DRMS (Distriuted Resource Management System)
• Batch processing
• User interface and a single point control for defining and monitoring
• In current scenario there are two popular DRMS available : Load Sharing Facility (LSF) by
Platform Computing Inc. and Sun Grid Engine (SGE) by Sun Microsystems
• Our Aim- UI for SGE & LSF or Develop a new DRMS ?
5

6. Types of Grid
Participating Nodes Nature of Nodes
• Cluster Grid • Compute Grid
• Departmental Grid • Data Grid
• Global Grid • Utility Grid
6

7. Pebble Grid Framework
• Grid Computing middleware that provides daemons, tools and shared
libraries to implement and extend the grid computing functionality
Features
• GUI
• Grid computing framework for Linux OS written in C/C++
• Daemon based implementation
• Job submission
• Job monitoring
• Dynamic load balancing & Rescheduling
• Dynamic participating nodes
• Highly scalable
• Middleware
• Framework: Shared Libraries*
• Implement Desktop Grid, Cluster Grid, Department Grid & Global
Grid + Data, Utility Grid
*framework is in progress
7

8. Components: Daemons
Pebble Grid Manager (PGM) Pebble Grid Client (PGC)
Administation & Submission Host Execution Host
• Job submission • Execution of Jobs
• Dynamic load balancing • Cluster-Dept.-Global grid
& Rescheduling • Participation in several grids
• Job Monitoring
• Cluster-Dept.-Global grid
8

9. Pebble Grid Manager (PGM): Host Configuration
Adding a Host : At PGM
• GUI Host configuration
• Dynamic cluster-grid nodes
• Cluster-Grid host configuration
Auto-Register : By PGC
• Memory available
• Imports list of installed packages
9

10. Pebble Grid Manager (PGM): Job Management
Submitting a Job
• GUI Job Management
• Job Submission
• Jobs : C, C++, Fortran, Pascal
Python scripts
Nature of Job
• Independent batch
• Divide & Conquer
10

11. Pebble Grid Manager (PGM): Job Management
Monitoring a Job
• GUI Job Management
• Job Monitoring
Queues
• TaskQ
• StatQ
• FinQ
11

12. PGM-Scheduling & Monitoring
Master-Slave Architecture PGC-Execution Host
-Cluster Grid Job
PGM
Result
PGM-PGC connection
Execution
PGC PGC PGC PGC PGC
**New jobs are allocated as per the rate of execution of execution hosts
12

13. PGM-Scheduling &
Tree Architecture PGM Monitoring
PGC-Execution Host
-Department Grid
Job
-Global Grid
Result
PGM-PGC connection
PGM-PGM connection
-PGM acting as slave
PGM -Not a execution host
-Receive Jobs
Execution
PGC PGC PGC
**New jobs are allocated as per the rate
of execution of execution hosts
PGC PGC PGC
13

14. Peer-to-Peer(PGM) Architecture PGM-Scheduling & Monitoring
PGC-Execution Host
-Department Grid Job
-Global Grid
Result
PGM-PGC connection
PGM PGM PGM-PGM connection
-PGM acting as peer
-Not a execution host
-Send & Receive Jobs
Execution
PGC PGC PGC PGC PGC
**New jobs are allocated as per the rate of execution of execution hosts
14

15. pebshd: Scheduler Daemon
Dynamic Scheduling
• Priority Scheduling
• FIFO Scheduling
pebshd
Load Balancing
• Real time load
• Status
Queues
• TaskQ- Idle jobs
• StatQ- Running jobs
• FinQ- Finished jobs
Rescheduling TaskQ StatQ FinQ
• Timeout for each job
Resource Reservation*
• Scheduling as per nature of job
* Work in progress
15

16. libPebble*
• Shared Library
Functionality includes:
• Job class
• Job scheduler + Rescheduling policy
• Execution Host
• Support for Divide & Conquer jobs
• API for developer to extend the framework
Resources used
• CentOS 5.3
• C,C++,GCC
• Glib/GTK+
* Work in progress
16

17. Pebble Divide & Conquer Framework 100,000
numbers
• Framework to support distributed
computing on large data sets on clusters
• Inspired by Google’s MapReduce
framework
• Basis: Divide & Conquer To PGC Divider
20,000
• GUI / API in libPebble numbers
20,000
Applications
numbers
• Quick sorting of 100,000 no. 20,000 20,000
• Crawling the WWW numbers 20,000
numbers
• Image Processing numbers
& many more
Combiner
Final Output
17

18. Sy
st e
m
Property
Architecture Hierarchical, Peer Centralised Hierarchical, Peer
Implementation C, C++, POSIX C++, Win32, POSIX C#, Web
Technology services,.NET
Framework
Extendable Yes No Yes
GUI Yes Yes Yes
CLI Yes Yes No
Projects CSIRO Land &
Soon… SETI@home,
Water ,
Einstein@home,etc
FMI Biomedical
Research Inst, etc
User/Developer
Soon… Massive Good
Community
18

19. References
RedHat https://www.redhatrenewals.com/.../selinux
Fedora docs.fedoraproject.org/selinux-faq-fc5
Wikipedia en.wikipedia.org/wiki/Grid_computing
Grid Computing www.gridcomputing.com
IBM www.ibm.com/grid/
Oracle www.oracle.com/us/technologies/grid/index.htm
Google Books books.google.co.in/books?isbn=1558609334
selinux docs.fedoraproject.org/selinux-faq-fc5
Gridcomputing en.wikipedia.org/wiki/Grid_computing
Gridcomputing www.gridcomputing.com
Gridcomputing www.ibm.com/grid/
Grid www.oracle.com/us/technologies/grid/index.htm
iptables iptables-tutorial.frozentux.net/iptables-tutorial.html
19

20. Thank You !
20