SlideShare ist ein Scribd-Unternehmen logo
1 von 20
Kartik Kumar Perisetla
kartik.peri@gmail.com
Grid Computing

▪ Grid computing is the combination of several computing resources from
  several administrative domains or organizations applied to a common
  task, usually scientific research areas.

▪ Grid computing is distributed, large scale clustered computing as well
  as network distributed parallel processing.

▪ The main strategy of grid computing is using software to divide a job into
  pieces and assign to different computing resources.

▪ Grid computing facilitate virtual organizations to access remote
  computing resources to achieve common goal.

▪ Virtual organization refers to set of individuals defined around the
  resource sharing policies, protocols.




                                                                               2
Grid Computing Principle


                                                     3                     Node

1) Task Request.
                                                         2
2) Task splitting into jobs (or
   sub tasks) and job             Task
   submission
                                           Control               2
3) Sub-results collection                                                         Node
                                           Server
4) Final result aggregation
                                                                       3
                                  Result
                                                             2

                                                     3
                                                                        Node


                 Node: Grid node with        2 : Sub task            3 : Sub results
                 Computing resources

                                                                                         3
Grid Computing Architecture
         Model                OSI Model




                                   Application




                                   Transport
                                   Network

                                   Link




                                                 4
DRMS (Distriuted Resource Management System)

• Batch processing

• User interface and a single point control for defining and monitoring

• In current scenario there are two popular DRMS available : Load Sharing Facility (LSF) by
  Platform Computing Inc. and Sun Grid Engine (SGE) by Sun Microsystems

• Our Aim- UI for SGE & LSF or Develop a new DRMS ?




                                                                                      5
Types of Grid



Participating Nodes    Nature of Nodes

• Cluster Grid        • Compute Grid
• Departmental Grid   • Data Grid
• Global Grid         • Utility Grid




                                         6
Pebble Grid Framework
• Grid Computing middleware that provides daemons, tools and shared
  libraries to implement and extend the grid computing functionality



Features
• GUI
• Grid computing framework for Linux OS written in C/C++
• Daemon based implementation
• Job submission
• Job monitoring
• Dynamic load balancing & Rescheduling
• Dynamic participating nodes
• Highly scalable
• Middleware
• Framework: Shared Libraries*
• Implement Desktop Grid, Cluster Grid, Department Grid & Global
  Grid + Data, Utility Grid




                                                              *framework is in progress
                                                                                          7
Components: Daemons

    Pebble Grid Manager (PGM)     Pebble Grid Client (PGC)




Administation & Submission Host   Execution Host
• Job submission                  • Execution of Jobs
• Dynamic load balancing          • Cluster-Dept.-Global grid
  & Rescheduling                  • Participation in several grids
• Job Monitoring
• Cluster-Dept.-Global grid



                                                                     8
Pebble Grid Manager (PGM): Host Configuration

Adding a Host : At PGM
• GUI Host configuration
• Dynamic cluster-grid nodes
• Cluster-Grid host configuration

Auto-Register : By PGC
• Memory available
• Imports list of installed packages




                                                9
Pebble Grid Manager (PGM): Job Management

Submitting a Job
• GUI Job Management
• Job Submission
• Jobs : C, C++, Fortran, Pascal
  Python scripts

Nature of Job
• Independent batch
• Divide & Conquer




                                            10
Pebble Grid Manager (PGM): Job Management

Monitoring a Job
• GUI Job Management
• Job Monitoring

Queues
• TaskQ
• StatQ
• FinQ




                                            11
PGM-Scheduling & Monitoring
Master-Slave Architecture                                PGC-Execution Host
-Cluster Grid                                               Job
                                      PGM
                                                            Result

                                                            PGM-PGC connection
                                                            Execution




  PGC           PGC             PGC             PGC               PGC


  **New jobs are allocated as per the rate of execution of execution hosts



                                                                              12
PGM-Scheduling &
Tree Architecture                                PGM                   Monitoring
                                                                   PGC-Execution Host
-Department Grid
                                                                       Job
-Global Grid
                                                                       Result

                                                                       PGM-PGC connection

                                                                       PGM-PGM connection

                                                                       -PGM acting as slave
                                                       PGM             -Not a execution host
                                                                       -Receive Jobs

                                                                        Execution




      PGC              PGC                 PGC



**New jobs are allocated as per the rate
  of execution of execution hosts



                                                 PGC         PGC             PGC

                                                                                     13
Peer-to-Peer(PGM) Architecture                              PGM-Scheduling & Monitoring
                                                            PGC-Execution Host
-Department Grid                                                Job
-Global Grid
                                                                Result

                                                                PGM-PGC connection
                       PGM                      PGM             PGM-PGM connection

                                                                 -PGM acting as peer
                                                                 -Not a execution host
                                                                 -Send & Receive Jobs

                                                                 Execution




  PGC         PGC                   PGC           PGC          PGC


     **New jobs are allocated as per the rate of execution of execution hosts

                                                                                  14
pebshd: Scheduler Daemon
 Dynamic Scheduling
 • Priority Scheduling
 • FIFO Scheduling
                                                     pebshd
 Load Balancing
 • Real time load
 • Status

 Queues
 • TaskQ- Idle jobs
 • StatQ- Running jobs
 • FinQ- Finished jobs

 Rescheduling                        TaskQ   StatQ    FinQ
 • Timeout for each job

 Resource Reservation*
 • Scheduling as per nature of job
                                                         * Work in progress

                                                                          15
libPebble*

• Shared Library

 Functionality includes:
• Job class
• Job scheduler + Rescheduling policy
• Execution Host
• Support for Divide & Conquer jobs
• API for developer to extend the framework
Resources used

 • CentOS 5.3
 • C,C++,GCC
 • Glib/GTK+




                                              * Work in progress

                                                               16
Pebble Divide & Conquer Framework                           100,000
                                                            numbers
• Framework to support distributed
  computing on large data sets on clusters
• Inspired by Google’s MapReduce
  framework
• Basis: Divide & Conquer                        To PGC       Divider
                                        20,000
• GUI / API in libPebble               numbers

                                            20,000
Applications
                                           numbers
• Quick sorting of 100,000 no.                     20,000              20,000
• Crawling the WWW                                numbers    20,000
                                                                      numbers
• Image Processing                                          numbers

  & many more


                             Combiner

  Final Output


                                                                                17
Sy
         st e
             m
 Property


 Architecture    Hierarchical, Peer     Centralised       Hierarchical, Peer


Implementation   C, C++, POSIX        C++, Win32, POSIX   C#, Web
Technology                                                services,.NET
                                                          Framework

Extendable            Yes                   No                 Yes

GUI                   Yes                  Yes                 Yes


CLI                   Yes                  Yes                 No

Projects                                                  CSIRO Land &
                     Soon…            SETI@home,
                                                          Water ,
                                      Einstein@home,etc
                                                          FMI Biomedical
                                                          Research Inst, etc
User/Developer
                     Soon…                 Massive             Good
Community
                                                                               18
References

RedHat           https://www.redhatrenewals.com/.../selinux
Fedora           docs.fedoraproject.org/selinux-faq-fc5
Wikipedia        en.wikipedia.org/wiki/Grid_computing
Grid Computing   www.gridcomputing.com
IBM              www.ibm.com/grid/
Oracle           www.oracle.com/us/technologies/grid/index.htm
Google Books     books.google.co.in/books?isbn=1558609334
selinux          docs.fedoraproject.org/selinux-faq-fc5
Gridcomputing    en.wikipedia.org/wiki/Grid_computing
Gridcomputing    www.gridcomputing.com
Gridcomputing    www.ibm.com/grid/
Grid             www.oracle.com/us/technologies/grid/index.htm
iptables         iptables-tutorial.frozentux.net/iptables-tutorial.html
                                                                          19
Thank You !




              20

Weitere ähnliche Inhalte

Was ist angesagt?

Simple asynchronous remote invocations for distributed real-time Java
Simple asynchronous remote invocations for distributed real-time JavaSimple asynchronous remote invocations for distributed real-time Java
Simple asynchronous remote invocations for distributed real-time JavaUniversidad Carlos III de Madrid
 
Hanborq Optimizations on Hadoop MapReduce
Hanborq Optimizations on Hadoop MapReduceHanborq Optimizations on Hadoop MapReduce
Hanborq Optimizations on Hadoop MapReduceHanborq Inc.
 
Resource Management for Computer Operating Systems
Resource Management for Computer Operating SystemsResource Management for Computer Operating Systems
Resource Management for Computer Operating Systemsinside-BigData.com
 
Memory Bandwidth QoS
Memory Bandwidth QoSMemory Bandwidth QoS
Memory Bandwidth QoSRohit Jnagal
 
下午1 intel yang, elton_mee_go-arch-update-final
下午1 intel yang, elton_mee_go-arch-update-final下午1 intel yang, elton_mee_go-arch-update-final
下午1 intel yang, elton_mee_go-arch-update-finalcsdnmobile
 
The IRMOS Real-Time Scheduler
The IRMOS Real-Time SchedulerThe IRMOS Real-Time Scheduler
The IRMOS Real-Time Schedulertcucinotta
 
Ultra-scalable Architectures for Telecommunications and Web 2.0 Services
Ultra-scalable Architectures for Telecommunications and Web 2.0 ServicesUltra-scalable Architectures for Telecommunications and Web 2.0 Services
Ultra-scalable Architectures for Telecommunications and Web 2.0 ServicesMauricio Arango
 
Scheduler performance in manycore architecture
Scheduler performance in manycore architectureScheduler performance in manycore architecture
Scheduler performance in manycore architecturechiportal
 
customization of a deep learning accelerator, based on NVDLA
customization of a deep learning accelerator, based on NVDLAcustomization of a deep learning accelerator, based on NVDLA
customization of a deep learning accelerator, based on NVDLAShien-Chun Luo
 
Task Scheduling Algorithm for Multicore Processor Systems with Turbo Boost an...
Task Scheduling Algorithm for Multicore Processor Systems with Turbo Boost an...Task Scheduling Algorithm for Multicore Processor Systems with Turbo Boost an...
Task Scheduling Algorithm for Multicore Processor Systems with Turbo Boost an...Naoki Shibata
 
Tungsten University: Setup and Operate Tungsten Replicators
Tungsten University: Setup and Operate Tungsten ReplicatorsTungsten University: Setup and Operate Tungsten Replicators
Tungsten University: Setup and Operate Tungsten ReplicatorsContinuent
 

Was ist angesagt? (17)

Simple asynchronous remote invocations for distributed real-time Java
Simple asynchronous remote invocations for distributed real-time JavaSimple asynchronous remote invocations for distributed real-time Java
Simple asynchronous remote invocations for distributed real-time Java
 
Hanborq Optimizations on Hadoop MapReduce
Hanborq Optimizations on Hadoop MapReduceHanborq Optimizations on Hadoop MapReduce
Hanborq Optimizations on Hadoop MapReduce
 
Resource Management for Computer Operating Systems
Resource Management for Computer Operating SystemsResource Management for Computer Operating Systems
Resource Management for Computer Operating Systems
 
Enhancing the region model of RTSJ
Enhancing the region model of RTSJEnhancing the region model of RTSJ
Enhancing the region model of RTSJ
 
Memory Bandwidth QoS
Memory Bandwidth QoSMemory Bandwidth QoS
Memory Bandwidth QoS
 
下午1 intel yang, elton_mee_go-arch-update-final
下午1 intel yang, elton_mee_go-arch-update-final下午1 intel yang, elton_mee_go-arch-update-final
下午1 intel yang, elton_mee_go-arch-update-final
 
Cat @ scale
Cat @ scaleCat @ scale
Cat @ scale
 
The IRMOS Real-Time Scheduler
The IRMOS Real-Time SchedulerThe IRMOS Real-Time Scheduler
The IRMOS Real-Time Scheduler
 
2020 icldla-updated
2020 icldla-updated2020 icldla-updated
2020 icldla-updated
 
Ultra-scalable Architectures for Telecommunications and Web 2.0 Services
Ultra-scalable Architectures for Telecommunications and Web 2.0 ServicesUltra-scalable Architectures for Telecommunications and Web 2.0 Services
Ultra-scalable Architectures for Telecommunications and Web 2.0 Services
 
Scheduler performance in manycore architecture
Scheduler performance in manycore architectureScheduler performance in manycore architecture
Scheduler performance in manycore architecture
 
customization of a deep learning accelerator, based on NVDLA
customization of a deep learning accelerator, based on NVDLAcustomization of a deep learning accelerator, based on NVDLA
customization of a deep learning accelerator, based on NVDLA
 
2011.jtr.pbasanta.
2011.jtr.pbasanta.2011.jtr.pbasanta.
2011.jtr.pbasanta.
 
Task Scheduling Algorithm for Multicore Processor Systems with Turbo Boost an...
Task Scheduling Algorithm for Multicore Processor Systems with Turbo Boost an...Task Scheduling Algorithm for Multicore Processor Systems with Turbo Boost an...
Task Scheduling Algorithm for Multicore Processor Systems with Turbo Boost an...
 
No Heap Remote Objects for Distributed real-time Java
No Heap Remote Objects for Distributed real-time JavaNo Heap Remote Objects for Distributed real-time Java
No Heap Remote Objects for Distributed real-time Java
 
Tungsten University: Setup and Operate Tungsten Replicators
Tungsten University: Setup and Operate Tungsten ReplicatorsTungsten University: Setup and Operate Tungsten Replicators
Tungsten University: Setup and Operate Tungsten Replicators
 
Management 5 g
Management 5 gManagement 5 g
Management 5 g
 

Andere mochten auch

Andere mochten auch (8)

Introduction to Grid computing and e-infrastructures
Introduction to Grid computing and e-infrastructuresIntroduction to Grid computing and e-infrastructures
Introduction to Grid computing and e-infrastructures
 
GCF
GCFGCF
GCF
 
Globus toolkit in grid
Globus toolkit in gridGlobus toolkit in grid
Globus toolkit in grid
 
Grid computing
Grid computingGrid computing
Grid computing
 
Grid computing
Grid computingGrid computing
Grid computing
 
Grid computing by vaishali sahare [katkar]
Grid computing by vaishali sahare [katkar]Grid computing by vaishali sahare [katkar]
Grid computing by vaishali sahare [katkar]
 
Grid computing
Grid computingGrid computing
Grid computing
 
Grid computing Seminar PPT
Grid computing Seminar PPTGrid computing Seminar PPT
Grid computing Seminar PPT
 

Ähnlich wie Pebble Grid Computing Framework

Jug gridgain java_grid_computing_made_simple
Jug gridgain java_grid_computing_made_simpleJug gridgain java_grid_computing_made_simple
Jug gridgain java_grid_computing_made_simpleSubhashiniSukumar
 
A Grid Proxy Architecture for Network Resources
A Grid Proxy Architecture for Network ResourcesA Grid Proxy Architecture for Network Resources
A Grid Proxy Architecture for Network ResourcesTal Lavian Ph.D.
 
Graphical packet generator
Graphical packet generatorGraphical packet generator
Graphical packet generatortusharjadhav2611
 
GPU Support In Spark And GPU/CPU Mixed Resource Scheduling At Production Scale
GPU Support In Spark And GPU/CPU Mixed Resource Scheduling At Production ScaleGPU Support In Spark And GPU/CPU Mixed Resource Scheduling At Production Scale
GPU Support In Spark And GPU/CPU Mixed Resource Scheduling At Production ScaleSpark Summit
 
Introduction to Python Celery
Introduction to Python CeleryIntroduction to Python Celery
Introduction to Python CeleryMahendra M
 
MEW22 22nd Machine Evaluation Workshop Microsoft
MEW22 22nd Machine Evaluation Workshop MicrosoftMEW22 22nd Machine Evaluation Workshop Microsoft
MEW22 22nd Machine Evaluation Workshop MicrosoftLee Stott
 
(ATS6-GS04) Performance Analysis of Accelrys Enterprise Platform 9.0 on IBM’s...
(ATS6-GS04) Performance Analysis of Accelrys Enterprise Platform 9.0 on IBM’s...(ATS6-GS04) Performance Analysis of Accelrys Enterprise Platform 9.0 on IBM’s...
(ATS6-GS04) Performance Analysis of Accelrys Enterprise Platform 9.0 on IBM’s...BIOVIA
 
How our Cloudy Mindsets Approached Physical Routers
How our Cloudy Mindsets Approached Physical RoutersHow our Cloudy Mindsets Approached Physical Routers
How our Cloudy Mindsets Approached Physical RoutersSteffen Gebert
 
Boyang gao gpu k-means_gmm_final_v1
Boyang gao gpu k-means_gmm_final_v1Boyang gao gpu k-means_gmm_final_v1
Boyang gao gpu k-means_gmm_final_v1Gao Boyang
 
2009.08 grid peer-slides
2009.08 grid peer-slides2009.08 grid peer-slides
2009.08 grid peer-slidesYehia El-khatib
 
Troubleshooting Apache® Ignite™
Troubleshooting Apache® Ignite™Troubleshooting Apache® Ignite™
Troubleshooting Apache® Ignite™Tom Diederich
 
App Engine Dev Days DC 20091026
App Engine Dev Days DC 20091026App Engine Dev Days DC 20091026
App Engine Dev Days DC 20091026jblocksom
 
Continuous delivery of Windows micro services in the cloud
Continuous delivery of Windows micro services in the cloud Continuous delivery of Windows micro services in the cloud
Continuous delivery of Windows micro services in the cloud Owain Perry
 
GPU Support in Spark and GPU/CPU Mixed Resource Scheduling at Production Scale
GPU Support in Spark and GPU/CPU Mixed Resource Scheduling at Production ScaleGPU Support in Spark and GPU/CPU Mixed Resource Scheduling at Production Scale
GPU Support in Spark and GPU/CPU Mixed Resource Scheduling at Production Scalesparktc
 
GPGPU Accelerates PostgreSQL ~Unlock the power of multi-thousand cores~
GPGPU Accelerates PostgreSQL ~Unlock the power of multi-thousand cores~GPGPU Accelerates PostgreSQL ~Unlock the power of multi-thousand cores~
GPGPU Accelerates PostgreSQL ~Unlock the power of multi-thousand cores~Kohei KaiGai
 
INSIDE M2M products & references
INSIDE M2M products & referencesINSIDE M2M products & references
INSIDE M2M products & referencesDaniel Stanke
 
Classification of EEG P300 ERPs using Riemannian Geometry and Quantum Computing
Classification of EEG P300 ERPs using Riemannian Geometry and Quantum ComputingClassification of EEG P300 ERPs using Riemannian Geometry and Quantum Computing
Classification of EEG P300 ERPs using Riemannian Geometry and Quantum ComputingAntonAndreev13
 
Performance challenges in software networking
Performance challenges in software networkingPerformance challenges in software networking
Performance challenges in software networkingStephen Hemminger
 
High Performance Computing - Cloud Point of View
High Performance Computing - Cloud Point of ViewHigh Performance Computing - Cloud Point of View
High Performance Computing - Cloud Point of Viewaragozin
 
Grid Job Management
Grid Job ManagementGrid Job Management
Grid Job Managementpycontw
 

Ähnlich wie Pebble Grid Computing Framework (20)

Jug gridgain java_grid_computing_made_simple
Jug gridgain java_grid_computing_made_simpleJug gridgain java_grid_computing_made_simple
Jug gridgain java_grid_computing_made_simple
 
A Grid Proxy Architecture for Network Resources
A Grid Proxy Architecture for Network ResourcesA Grid Proxy Architecture for Network Resources
A Grid Proxy Architecture for Network Resources
 
Graphical packet generator
Graphical packet generatorGraphical packet generator
Graphical packet generator
 
GPU Support In Spark And GPU/CPU Mixed Resource Scheduling At Production Scale
GPU Support In Spark And GPU/CPU Mixed Resource Scheduling At Production ScaleGPU Support In Spark And GPU/CPU Mixed Resource Scheduling At Production Scale
GPU Support In Spark And GPU/CPU Mixed Resource Scheduling At Production Scale
 
Introduction to Python Celery
Introduction to Python CeleryIntroduction to Python Celery
Introduction to Python Celery
 
MEW22 22nd Machine Evaluation Workshop Microsoft
MEW22 22nd Machine Evaluation Workshop MicrosoftMEW22 22nd Machine Evaluation Workshop Microsoft
MEW22 22nd Machine Evaluation Workshop Microsoft
 
(ATS6-GS04) Performance Analysis of Accelrys Enterprise Platform 9.0 on IBM’s...
(ATS6-GS04) Performance Analysis of Accelrys Enterprise Platform 9.0 on IBM’s...(ATS6-GS04) Performance Analysis of Accelrys Enterprise Platform 9.0 on IBM’s...
(ATS6-GS04) Performance Analysis of Accelrys Enterprise Platform 9.0 on IBM’s...
 
How our Cloudy Mindsets Approached Physical Routers
How our Cloudy Mindsets Approached Physical RoutersHow our Cloudy Mindsets Approached Physical Routers
How our Cloudy Mindsets Approached Physical Routers
 
Boyang gao gpu k-means_gmm_final_v1
Boyang gao gpu k-means_gmm_final_v1Boyang gao gpu k-means_gmm_final_v1
Boyang gao gpu k-means_gmm_final_v1
 
2009.08 grid peer-slides
2009.08 grid peer-slides2009.08 grid peer-slides
2009.08 grid peer-slides
 
Troubleshooting Apache® Ignite™
Troubleshooting Apache® Ignite™Troubleshooting Apache® Ignite™
Troubleshooting Apache® Ignite™
 
App Engine Dev Days DC 20091026
App Engine Dev Days DC 20091026App Engine Dev Days DC 20091026
App Engine Dev Days DC 20091026
 
Continuous delivery of Windows micro services in the cloud
Continuous delivery of Windows micro services in the cloud Continuous delivery of Windows micro services in the cloud
Continuous delivery of Windows micro services in the cloud
 
GPU Support in Spark and GPU/CPU Mixed Resource Scheduling at Production Scale
GPU Support in Spark and GPU/CPU Mixed Resource Scheduling at Production ScaleGPU Support in Spark and GPU/CPU Mixed Resource Scheduling at Production Scale
GPU Support in Spark and GPU/CPU Mixed Resource Scheduling at Production Scale
 
GPGPU Accelerates PostgreSQL ~Unlock the power of multi-thousand cores~
GPGPU Accelerates PostgreSQL ~Unlock the power of multi-thousand cores~GPGPU Accelerates PostgreSQL ~Unlock the power of multi-thousand cores~
GPGPU Accelerates PostgreSQL ~Unlock the power of multi-thousand cores~
 
INSIDE M2M products & references
INSIDE M2M products & referencesINSIDE M2M products & references
INSIDE M2M products & references
 
Classification of EEG P300 ERPs using Riemannian Geometry and Quantum Computing
Classification of EEG P300 ERPs using Riemannian Geometry and Quantum ComputingClassification of EEG P300 ERPs using Riemannian Geometry and Quantum Computing
Classification of EEG P300 ERPs using Riemannian Geometry and Quantum Computing
 
Performance challenges in software networking
Performance challenges in software networkingPerformance challenges in software networking
Performance challenges in software networking
 
High Performance Computing - Cloud Point of View
High Performance Computing - Cloud Point of ViewHigh Performance Computing - Cloud Point of View
High Performance Computing - Cloud Point of View
 
Grid Job Management
Grid Job ManagementGrid Job Management
Grid Job Management
 

Kürzlich hochgeladen

Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxMatsuo Lab
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...Aggregage
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPathCommunity
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDELiveplex
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1DianaGray10
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsSafe Software
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024SkyPlanner
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesDavid Newbury
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfAijun Zhang
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.YounusS2
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdfPedro Manuel
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsSeth Reyes
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureEric D. Schabell
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Will Schroeder
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024D Cloud Solutions
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarPrecisely
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintMahmoud Rabie
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAshyamraj55
 

Kürzlich hochgeladen (20)

Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptx
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation Developers
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond Ontologies
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdf
 
20150722 - AGV
20150722 - AGV20150722 - AGV
20150722 - AGV
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdf
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and Hazards
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability Adventure
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity Webinar
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership Blueprint
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
 

Pebble Grid Computing Framework

  • 2. Grid Computing ▪ Grid computing is the combination of several computing resources from several administrative domains or organizations applied to a common task, usually scientific research areas. ▪ Grid computing is distributed, large scale clustered computing as well as network distributed parallel processing. ▪ The main strategy of grid computing is using software to divide a job into pieces and assign to different computing resources. ▪ Grid computing facilitate virtual organizations to access remote computing resources to achieve common goal. ▪ Virtual organization refers to set of individuals defined around the resource sharing policies, protocols. 2
  • 3. Grid Computing Principle 3 Node 1) Task Request. 2 2) Task splitting into jobs (or sub tasks) and job Task submission Control 2 3) Sub-results collection Node Server 4) Final result aggregation 3 Result 2 3 Node Node: Grid node with 2 : Sub task 3 : Sub results Computing resources 3
  • 4. Grid Computing Architecture Model OSI Model Application Transport Network Link 4
  • 5. DRMS (Distriuted Resource Management System) • Batch processing • User interface and a single point control for defining and monitoring • In current scenario there are two popular DRMS available : Load Sharing Facility (LSF) by Platform Computing Inc. and Sun Grid Engine (SGE) by Sun Microsystems • Our Aim- UI for SGE & LSF or Develop a new DRMS ? 5
  • 6. Types of Grid Participating Nodes Nature of Nodes • Cluster Grid • Compute Grid • Departmental Grid • Data Grid • Global Grid • Utility Grid 6
  • 7. Pebble Grid Framework • Grid Computing middleware that provides daemons, tools and shared libraries to implement and extend the grid computing functionality Features • GUI • Grid computing framework for Linux OS written in C/C++ • Daemon based implementation • Job submission • Job monitoring • Dynamic load balancing & Rescheduling • Dynamic participating nodes • Highly scalable • Middleware • Framework: Shared Libraries* • Implement Desktop Grid, Cluster Grid, Department Grid & Global Grid + Data, Utility Grid *framework is in progress 7
  • 8. Components: Daemons Pebble Grid Manager (PGM) Pebble Grid Client (PGC) Administation & Submission Host Execution Host • Job submission • Execution of Jobs • Dynamic load balancing • Cluster-Dept.-Global grid & Rescheduling • Participation in several grids • Job Monitoring • Cluster-Dept.-Global grid 8
  • 9. Pebble Grid Manager (PGM): Host Configuration Adding a Host : At PGM • GUI Host configuration • Dynamic cluster-grid nodes • Cluster-Grid host configuration Auto-Register : By PGC • Memory available • Imports list of installed packages 9
  • 10. Pebble Grid Manager (PGM): Job Management Submitting a Job • GUI Job Management • Job Submission • Jobs : C, C++, Fortran, Pascal Python scripts Nature of Job • Independent batch • Divide & Conquer 10
  • 11. Pebble Grid Manager (PGM): Job Management Monitoring a Job • GUI Job Management • Job Monitoring Queues • TaskQ • StatQ • FinQ 11
  • 12. PGM-Scheduling & Monitoring Master-Slave Architecture PGC-Execution Host -Cluster Grid Job PGM Result PGM-PGC connection Execution PGC PGC PGC PGC PGC **New jobs are allocated as per the rate of execution of execution hosts 12
  • 13. PGM-Scheduling & Tree Architecture PGM Monitoring PGC-Execution Host -Department Grid Job -Global Grid Result PGM-PGC connection PGM-PGM connection -PGM acting as slave PGM -Not a execution host -Receive Jobs Execution PGC PGC PGC **New jobs are allocated as per the rate of execution of execution hosts PGC PGC PGC 13
  • 14. Peer-to-Peer(PGM) Architecture PGM-Scheduling & Monitoring PGC-Execution Host -Department Grid Job -Global Grid Result PGM-PGC connection PGM PGM PGM-PGM connection -PGM acting as peer -Not a execution host -Send & Receive Jobs Execution PGC PGC PGC PGC PGC **New jobs are allocated as per the rate of execution of execution hosts 14
  • 15. pebshd: Scheduler Daemon Dynamic Scheduling • Priority Scheduling • FIFO Scheduling pebshd Load Balancing • Real time load • Status Queues • TaskQ- Idle jobs • StatQ- Running jobs • FinQ- Finished jobs Rescheduling TaskQ StatQ FinQ • Timeout for each job Resource Reservation* • Scheduling as per nature of job * Work in progress 15
  • 16. libPebble* • Shared Library Functionality includes: • Job class • Job scheduler + Rescheduling policy • Execution Host • Support for Divide & Conquer jobs • API for developer to extend the framework Resources used • CentOS 5.3 • C,C++,GCC • Glib/GTK+ * Work in progress 16
  • 17. Pebble Divide & Conquer Framework 100,000 numbers • Framework to support distributed computing on large data sets on clusters • Inspired by Google’s MapReduce framework • Basis: Divide & Conquer To PGC Divider 20,000 • GUI / API in libPebble numbers 20,000 Applications numbers • Quick sorting of 100,000 no. 20,000 20,000 • Crawling the WWW numbers 20,000 numbers • Image Processing numbers & many more Combiner Final Output 17
  • 18. Sy st e m Property Architecture Hierarchical, Peer Centralised Hierarchical, Peer Implementation C, C++, POSIX C++, Win32, POSIX C#, Web Technology services,.NET Framework Extendable Yes No Yes GUI Yes Yes Yes CLI Yes Yes No Projects CSIRO Land & Soon… SETI@home, Water , Einstein@home,etc FMI Biomedical Research Inst, etc User/Developer Soon… Massive Good Community 18
  • 19. References RedHat https://www.redhatrenewals.com/.../selinux Fedora docs.fedoraproject.org/selinux-faq-fc5 Wikipedia en.wikipedia.org/wiki/Grid_computing Grid Computing www.gridcomputing.com IBM www.ibm.com/grid/ Oracle www.oracle.com/us/technologies/grid/index.htm Google Books books.google.co.in/books?isbn=1558609334 selinux docs.fedoraproject.org/selinux-faq-fc5 Gridcomputing en.wikipedia.org/wiki/Grid_computing Gridcomputing www.gridcomputing.com Gridcomputing www.ibm.com/grid/ Grid www.oracle.com/us/technologies/grid/index.htm iptables iptables-tutorial.frozentux.net/iptables-tutorial.html 19