Grass cloud
•
1 gefällt mir•2,273 views
The presentation of g.cloud module for GRASS GIS at FOSS4G 2011
Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Ähnlich wie Grass cloud
Ähnlich wie Grass cloud (20)
Mehr von luca delucchi
Mehr von luca delucchi (14)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
Grass cloud
- 1. Introduction Idea Implementation Conclusions GRASS GIS in the Cloud Luca Delucchi, Markus Neteler Fondazione Edmund Mach – GIS & Remote Sensing Platform http://gis.cri.fmach.it FOSS4G 2011 14th September 2011, Denver (USA) Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
- 2. Introduction Idea GRASS GIS Implementation Our cluster Conclusions GRASS GIS The first Open Source GIS Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
- 3. Introduction Idea GRASS GIS Implementation Our cluster Conclusions GRASS GIS The first Open Source GIS More than 400 modules for every kind of analysis Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
- 4. Introduction Idea GRASS GIS Implementation Our cluster Conclusions GRASS GIS The first Open Source GIS More than 400 modules for every kind of analysis Interoperability with other software Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
- 5. Introduction Idea GRASS GIS Implementation Our cluster Conclusions Our cluster: infrastructure FEM GIS cluster consists of 300 nodes 610 GB RAM 10 TB disk storage + 15 TB tape backup 10 Gb/s internal bus Scientific Linux 5.6 Grid Engine Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
- 6. Introduction Idea GRASS GIS Implementation Our cluster Conclusions Our cluster: massive spatial data analysis MODIS Land Surface Temperature (LST) map gap-filling (13,000 maps) MODIS data analysis (LST, NDVI, Snow products) LiDAR data analysis Solar energy calculations Neteler et al., 2011: Tiger mosquito, IJHG, doi:10.1186/1476-072X-10-49 Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
- 7. Introduction Idea Implementation Conclusions Idea to facilitate GRASS GIS usage on a cluster Simplify the work Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
- 8. Introduction Idea Implementation Conclusions Idea to facilitate GRASS GIS usage on a cluster Simplify the work Offer the possibility to use Grid Engine by all our colleagues Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
- 9. Introduction Idea Implementation Conclusions Idea to facilitate GRASS GIS usage on a cluster Simplify the work Offer the possibility to use Grid Engine by all our colleagues Launch jobs remotely without connecting every time to the cluster Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
- 10. Introduction Idea Implementation Conclusions Idea to facilitate GRASS GIS usage on a cluster Simplify the work Offer the possibility to use Grid Engine by all our colleagues Launch jobs remotely without connecting every time to the cluster Offer the possibility to external people to use the cluster without learning Grid Engine Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
- 11. Introduction Requirements Idea How does it work? Implementation Future Conclusions Implementation: requirements GRASS GIS 7 (current development version) Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
- 12. Introduction Requirements Idea How does it work? Implementation Future Conclusions Implementation: requirements GRASS GIS 7 (current development version) Grid Engine Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
- 13. Introduction Requirements Idea How does it work? Implementation Future Conclusions Implementation: requirements GRASS GIS 7 (current development version) Grid Engine Python > 2.4 Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
- 14. Introduction Requirements Idea How does it work? Implementation Future Conclusions Implementation: how does it work? Required parameters conf = file with username and password or stdin to connect to the cluster Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
- 15. Introduction Requirements Idea How does it work? Implementation Future Conclusions Implementation: how does it work? Required parameters conf = file with username and password or stdin to connect to the cluster server = hostname of the cluster Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
- 16. Introduction Requirements Idea How does it work? Implementation Future Conclusions Implementation: how does it work? Required parameters conf = file with username and password or stdin to connect to the cluster server = hostname of the cluster qsub script = file containing qsub script (template) Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
- 17. Introduction Requirements Idea How does it work? Implementation Future Conclusions Implementation: how does it work? Required parameters conf = file with username and password or stdin to connect to the cluster server = hostname of the cluster qsub script = file containing qsub script (template) grass script = own GRASS GIS script Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
- 18. Introduction Requirements Idea How does it work? Implementation Future Conclusions Implementation: how does it work? Required parameters conf = file with username and password or stdin to connect to the cluster server = hostname of the cluster qsub script = file containing qsub script (template) grass script = own GRASS GIS script g.cloud conf=/tmp/passwd server=giscluster qsub script=test launch SGE grassjob.sh grass script=test novariables.sh Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
- 19. Introduction Requirements Idea How does it work? Implementation Future Conclusions Implementation: how does it work? Optional parameters raster = name of raster(s) Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
- 20. Introduction Requirements Idea How does it work? Implementation Future Conclusions Implementation: how does it work? Optional parameters raster = name of raster(s) vector = name of vector(s) Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
- 21. Introduction Requirements Idea How does it work? Implementation Future Conclusions Implementation: how does it work? Optional parameters raster = name of raster(s) vector = name of vector(s) variables = string with name and values of variables stated as Python dictionary “’key’:[’value0’,’value1’]” Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
- 22. Introduction Requirements Idea How does it work? Implementation Future Conclusions Implementation: how does it work? Optional parameters raster = name of raster(s) vector = name of vector(s) variables = string with name and values of variables stated as Python dictionary “’key’:[’value0’,’value1’]” email = user’s email address Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
- 23. Introduction Requirements Idea How does it work? Implementation Future Conclusions Implementation: how does it work? Optional parameters raster = name of raster(s) vector = name of vector(s) variables = string with name and values of variables stated as Python dictionary “’key’:[’value0’,’value1’]” email = user’s email address path = path to the folder which must be accessible from cluster frontend and blades Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
- 24. Introduction Requirements Idea How does it work? Implementation Future Conclusions Implementation: more examples North Carolina sample data r.texture example g.cloud conf=/tmp/passwd server=giscluster qsub script=test launch SGE grassjob.sh grass script=test onevariables raster.sh variables=“’TEXT’ : [’asm’,’contrast’,’corr,var’,’idm’,’sa’,’se’,’sv’,’entr’,’dv’,’de’,’moc1’,’moc2’]” raster=lsat7 2002 40 Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
- 25. Introduction Requirements Idea How does it work? Implementation Future Conclusions Implementation: more examples North Carolina sample data r.texture example g.cloud conf=/tmp/passwd server=giscluster qsub script=test launch SGE grassjob.sh grass script=test onevariables raster.sh variables=“’TEXT’ : [’asm’,’contrast’,’corr,var’,’idm’,’sa’,’se’,’sv’,’entr’,’dv’,’de’,’moc1’,’moc2’]” raster=lsat7 2002 40 r.sun example g.cloud conf=/tmp/passwd server=giscluster qsub script=test launch SGE grassjob.sh grass script=test onevariables sun.sh variables=“’DOY’ : [1,2,3,4,5,6,7,8,9,10]” raster=elevation Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
- 26. Introduction Requirements Idea How does it work? Implementation Future Conclusions Implementation: future Support for more clustering system (for example Eucalyptus) Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
- 27. Introduction Requirements Idea How does it work? Implementation Future Conclusions Implementation: future Support for more clustering system (for example Eucalyptus) Auto installation of GRASS and its dependencies on the cluster if not present (GRASS 7 release as stable version) Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
- 28. Introduction Requirements Idea How does it work? Implementation Future Conclusions Implementation: future Support for more clustering system (for example Eucalyptus) Auto installation of GRASS and its dependencies on the cluster if not present (GRASS 7 release as stable version) Show the status of jobs Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
- 29. Introduction Requirements Idea How does it work? Implementation Future Conclusions Implementation: future Support for more clustering system (for example Eucalyptus) Auto installation of GRASS and its dependencies on the cluster if not present (GRASS 7 release as stable version) Show the status of jobs Return outputs to the client Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
- 30. Introduction Idea Implementation Conclusions Conclusions First implementation of a module to run GRASS on a cluster Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
- 31. Introduction Idea Implementation Conclusions Conclusions First implementation of a module to run GRASS on a cluster Experimental version Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
- 32. Introduction Idea Implementation Conclusions Conclusions First implementation of a module to run GRASS on a cluster Experimental version Please test it and report bugs or improvements Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
- 33. Introduction Idea Implementation Conclusions Conclusions First implementation of a module to run GRASS on a cluster Experimental version Please test it and report bugs or improvements Contact us for more info markus.neteler@iasma.it - luca.delucchi@iasma.it Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
- 34. Introduction Idea Implementation Conclusions Conclusions Thanks for your attention gis.cri.fmach.it Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver