PyModESt: A Python Framework for Staging of Geo-referenced Data on the Collaborative Climate Community Grid (C3-Grid)

PyModESt : A Python Framework for Staging of Geo-referenced Data on the Collaborative Climate Community Grid (C3-Grid) Henning Bergmeyer German Aerospace Center Simulation- and Software Technology Henning.Bergmeyer [at] dlr.de

Talk Overview ,[object Object],[object Object],[object Object],[object Object],[object Object]

C3-Grid Layered Architecture (simplified) WDC Climate WDC Mare WDC RSAT DWD DKRZ, PIK, GKSS, AWI, MPI-M IFM-Geomar FU Berlin Uni Cologne dCache OGSA-DAI Flat Files … heterogeneous! Grid-Service Portal Workflow Management Data Management Data Information System Archive Users View Grid Middleware Resources

C3-Grid : A D-Grid 1 Project ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

DLR Use Cases on C3-Grid ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Data Providers I Portal-Based Discovery ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Data Providers II Uniform Data Access ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],log_surface_pressure mole_fraction_of_ozone_in_air …

Data Providers III Delivering Data ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Becoming a C3-Grid Data Provider ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Data Providers know their tools and their data!

General Implementation of File Stagers for C3-Grid ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

PyModESt Staging Process Skeleton initialize Environment read Stage Request select DataProcessor choose Request Mode handle Cancel Request handle Stage Request handle Estimation Request handle Exceptions tidy Work Space write Responses ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Necessary Implementation effort for DP when using PyModESt

What the DP does and What is done for her ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

PyModESt Modular Extensibility ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

PyModESt Metadata Update md.removeQuicklook() md.filterContentInfo(self.thesaurus.translateToC3( self.requested_vars, self.object_id)) md.setHorizontalBounds(self.lon_min, self.lon_max, self.lat_min, self.lat_max) md.setVerticalExtent(self.alt_min, self.alt_max, self.alt_verticalcrs) md.updateTimePeriod(self.timeperiod_begin_date, self.timeperiod_stop_date) md.setObjectId(self.object_id + "." + datetime.utcnow().isoformat().replace(":","-")) md.addLineageProcessStep( PROCESS_DESCRIPTION, datetime.datetime.utcnow(), self.stage_request.object_ids[0], RESPONSIBLE_PERSON, "http://wis.wmo.int/2006/catalogues/gmxCodelists.xml#CI_RoleCode_distributor", INSTITUTE_IDENTIFIER)

Estimation for Offer Contracts ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Examples: DLR Use Cases ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Benefits of Using PyModESt ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Open Issues ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Summary ,[object Object],[object Object],[object Object],[object Object]

PyModESt: A Python Framework for Staging of Geo-referenced Data on the Collaborative Climate Community Grid (C3-Grid)

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Andere mochten auch

Andere mochten auch (6)

Ähnlich wie PyModESt: A Python Framework for Staging of Geo-referenced Data on the Collaborative Climate Community Grid (C3-Grid)

Ähnlich wie PyModESt: A Python Framework for Staging of Geo-referenced Data on the Collaborative Climate Community Grid (C3-Grid) (20)

Mehr von Andreas Schreiber

Mehr von Andreas Schreiber (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

PyModESt: A Python Framework for Staging of Geo-referenced Data on the Collaborative Climate Community Grid (C3-Grid)