20161028 strahlendorff fmi experience in openness

Openness in research
Experiences at Finnish Meteorological
Institute
28.10.2016
Finnish Meteorological Institute
Mikko Strahlendorff
Roope Tervo, Mikko Visa, Tero Koivunen, Jukka A. Pakarinen, Tarja Riihisaari

Background for my
experience
Mostly employed by Finnish Meteorological Institute
 Weatherforecasting 1995-1997
 Public service development 1998--99
 Commercial service development and head of it
2000-2003
 Head of Informations system services 2004-
2007
 Seconded Expert to European Commission
Earth Observation program 2008-2010
 Ministry of Transport and Communications
Ministerial adviser 2011-2012
 EU research policy and international
organisation 2012-
 Helping to build European Research
Infrastructure Consortias
 ICOS, EuroARGO, SIOS, COPAL
 Executive commiittee member in Group on Earth
Observation
28.10.2016 Openness experience at FMI – Mikko Strahlendorff 2

FOSS revolution and
the internet
• Researcher Linus Torvalds at Helsinki
university releases Linux as Free and
Open-Source Software
• 25 years anniversary now and
• 96% of all web-servers run Linux
• 69% of all mobile devices run Android
(developed from Linux)
• Sharing ones work with a huge
network of like-minded ensures high
quality results faster than anything
• Motivation comes from the common
utility of the result
28.10.2016
Openness experience at FMI – Mikko Strahlendorff
3

• Openness requires extra efforts
• Free and open doesn’t generate revenue
• How to motivate financing? How to share expenses
fairly?
• Sharing efforts is making the list of contributors long
and single persons are less visible
• How to divide the fame?
Solutions:
• Application of the common to many challenges in
many places
• Analysis, understanding and education for sale
• Fame to the whole system:
• Data production is publishing!
• Each use of data should be rewarded!
• FOSS tools and internet technologies reduce costs
Challenges with
Openness –
rewarding effort
28.10.2016 4Openness experience at FMI – Mikko Strahlendorff

Data is a great
common motivation
o Analysis needs data and it gets better with
more data
o Selling scientific data is rarely lucrative
and data lying in databases unused is only
expenses - no gain
o Data is good to be collected in similar
ways for similar applications – common
interest naturally networks collectors
o Quality control and technical solutions are
perfect tasks for sharing
o Data dissemination has costs that are
usually not planned in research
o Tracking usage is good for development
and for finance motivation
28.10.2016 5

Finnish Meteorological Institute opened its data in 2013 after
government decision for all the Finnish public sector.
Basically everything that FMI has property rights on was opened.
Data is provided freely in machine readable format over the internet.
28.10.2016 Meteorological and Aviation Weather Open Data implementation OGC standards
6
FMI Open Data
https://en.ilmatieteenlaitos.fi/open-data

Data set Description Time
Interval
Estimated
publish date
Weather
Observations
Temperature, Wind,
Humidity, Ground
Temperature…
10 min Open,
older data to be
added
Sun Radiation UV, Short and Long
Term Radiation…
1 min Open
Marine
Observations
Waves, Sea
Temperature, Sea
Level…
1 h Open
Weather Radars Precipitation Rate,
Precipitation Amount…
5 min Open,
older data to be
added
Lightning Thunder Strikes in
Finland
5 min Open
Example Data Sets
28.10.2016 7Meteorological and Aviation Weather Open Data implementation OGC standards

Example of Data Sets
Data set Description Time Interval Estimated
publish date
Real Time
Observations
Real Time Observations from
specific location(s)
AWS 2010 –
Soundings 1959 –
Flashes 1998 –
Sea Level 1971 –
Waves 2005 –
Open
older data will
be added
Climatological
Observations
Dayly and monthly
temperature mean and
extreme values from weather
stations
1959 - Open
Climatological
Observations
Monthly temperature and
precipitation rate mean
values interpolated to grid
1961 - Open
Climatological
Reference
Climatological Reference.
Temperature, humidity,
pressure, precipitation
amount and snow depth.
Reference seasons:
1971-2000 1981-
2010
Open

Example Data Sets
publish date
Weather forecast
model HIRLAM RCR
Point forecasts and grid
data
Latest model
run
(4 times a day)
0…54 h
Open
Sea forecast models Sea level point
forecasts, Wave (WAM)
and current (HBM) as
grid data
Latest model
run
(4 times a day)
0...54 h
Open
Environmental
Monitoring Facilities
Weather observation
stations, radars…
2015
Aviation
Observations
METAR 30 min open
Ground & mast
observations
Special observations
from ground and masts
2016 /Open

Example Data Sets
publish date
Air Quality
Observations
Air Quality Observations 1h open
Silam Model Dispersion Model for Air
Quality, Forest Fire and
Pollen
Latest model
run (once a day)
0…96h
2015
HELMI Ice Model Ice forecast model Latest model
run
(4 times a day)
0...54 h
open
Soundings Temperature, Humidity,
Pressure, Wind from
ground to 25 km height
2 times a day open
Road Weather
Observations (LIVI)
Road Weather
Observations
10 min open

FMI Open Data Portal follows INSPIRE requirements.
FMI Open Data Portal
Meta data
Data
Models
Services
The very same data portal works as Open Data and
INSPIRE portal.

Registration
o Registration is required to use View and Download
Services
o Working email address is the only mandatory
information
o After registration the user gets an API key which have to
be added into all requests
o GET parameter fmi-apikey=…&
o Header fmi-apikey; …
o Part of url http://wms.fmi.fi/fmi-apikey/…/wms?
o One can create several API keys with one email

Usage Limits
With one API key it’s allowed to
o do at most 20 000 requests per day to Download Service
o do at most 10 000 requests per day to View Service
o do at most 600 requests per 5 minutes to both services
o If all observations from one time step is calculated to as one,
little over 17 000 new data sets are published daily
o So, with one API key it’s allowed to load everything once
o View service can be used for testing but can not be used as a
back end for popular clients

And a little over
450 000 data
downloads
per day
(5.2 req/s)
At the moment
about 11 700
registered users
3 year experience

Industry is
happy to use
standardized
services
Amateur and
freelancer coders
would prefer simple
JSON API
Some Experiences

MetoLib
o Open source JavaScript library produced by Finnish
Meteorological Institute
o Helps users to load and use the data
o Supports multi point coverage data format
o Python version is on the list
Easy
requests Cache
Parse the data
to as JSON

SmartMet Server
• High capacity & availability Data server for MetOcean data
• Data is extracted and products generated always on-demand
• INSPIRE Compliant
• Operational use since 2008
• FMI client services (since 2008)
• Finnish Meteorological Institute (FMI) Open Data Portal (since 2013)
• Several input sources
• GRIB-, NetCDF-, etc. files via internal data format querydata
• PostGIS database (vectors)
• SQL database (point observations)
• Several output interfaces and formats
• WMS, WFS 2.0
• JSON, XML, ASCII, HTML, SERIAL
• GRIB1, GRIB2, NetCDF
10/28/2016
Open Meteorological Data with OGC and INSPIRE /
Roope Tervo
17

SmartMet Server goes
Open Source
• Going to be published as Open Source in Q4/2016
• MIT license
• https://github.com/fmidev
10/28/2016
Open Meteorological Data with OGC and INSPIRE /
Roope Tervo
18

Open Science vision
Science WIKIPEDIA
o Data is collected in standard ways and
published immediately
o Papers are written collaboratively
o Science is mostly about writing
scripts/models to analyze data for answers
to questions
o Analysis scripts are published with the
answers so readers can reproduce results
and evaluate conclusions themselves
o Donors get to ask questions and use the
knowledge
o Sharing the fame is visible thru linking all the
scientists, from every data set and every
model/script

• Governments: What data do we need for public interests?
• How do private entities support data production?
• Science interest
• Commercial interest
• Research funding should be two-fold:
• longer time plans and grants for data production and modeling
• short term grants for answers to societal questions for scripts and analysis
• How does a new Science approach conquer the world like Linux?
• Thanks and please comment!
What do we need to do for
this vision?

www.fmi.fi
http://www.slideshare.net/MikkoStrahlendorff
https://en.ilmatieteenlaitos.fi/open-data

20161028 strahlendorff fmi experience in openness

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (10)

Andere mochten auch

Andere mochten auch (17)

Ähnlich wie 20161028 strahlendorff fmi experience in openness

Ähnlich wie 20161028 strahlendorff fmi experience in openness (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

20161028 strahlendorff fmi experience in openness