FOSS4G 2011 presentation on the status of the implementation of the WPS protocol in the GeoServer open source project

1. The status of GeoServer WPS
Ing. Andrea Aime, GeoSolutions
Ing. Simone Giannecchini, GeoSolutions
FOSS4G 2011, Denver
12th-16th September 2011

2. Overview
 Who we are
 WPS introduction
 Vector processes
 Raster processes
 Conversion processes
 Accessing local data sources
 Chaining
 Examples
 Rendering transformations
 Limitations and wish list
 Question time
FOSS4G 2011, Denver
12th-16th September 2011

3. GeoSolutions
 Founded in Italy in 2006
 Expertise
• Image Processing, GeoSpatial Data Fusion
• Java, Java Enterprise, C++, Python
• JPEG2000, JPIP, Advanced 2D visualization
 Supporting/Developing FOSS4G projects
 GeoTools, GeoServer
 GeoBatch, GeoNetwork
 Clients
 Public Agencies
 Private Companies
 http://www.geo-solutions.it
FOSS4G 2011, Denver
12th-16th September 2011

4. WPS Quick introduction
FOSS4G 2011, Denver
12th-16th September 2011

5. WPS
 Wikipedia introduces OGC WPS as:
 [A service] designed to standardize the
way that GIS calculations are made
available to the Internet.
 WPS can describe any calculation
including all of its inputs and outputs,
and trigger its execution
 The specific processes served up by a WPS
implementation are defined by the owner of that
implementation.
 Although WPS was designed to work with spatially
referenced data, it can be used with any kind of
data.
FOSS4G 2011, Denver
12th-16th September 2011

6. Capabilities document
 Some metadata
about the server
and its owner
 The list of available
processes, with a
description
 Let’s say we’re
interested in the
JTS:buffer process
FOSS4G 2011, Denver
12th-16th September 2011

7. Process description
 List of inputs
and outputs
 Descriptions
 List of accepted
formats (not
shown in the
xml)
FOSS4G 2011, Denver
12th-16th September 2011

8. Process execution
 Buffer a L shaped
geometry with
distance “2”
 Get the result back
as GML
FOSS4G 2011, Denver
12th-16th September 2011

9. GeoServer Specifics
FOSS4G 2011, Denver
12th-16th September 2011

10. GeoServer WPS history
 Started by Refractions in 2008, with limited capabilities
(only single geometry and single feature support)
 First overhaul attempt end of 2008 by the community,
added testing, support for vector collections
 New development since mid 2010, mostly new
processes and input/output formats
Refractions GS community Current activity
2008 2009 2010 2011
FOSS4G 2011, Denver
12th-16th September 2011

11. WPS: demo builder
 List processes
 Describe
 Set parameters and
execute
 All in one form
FOSS4G 2011, Denver
12th-16th September 2011

12. Inputs and outputs
FOSS4G 2011, Denver
12th-16th September 2011

13. IO setup
 Processes parameters
defined in terms of Java
objects
 Pluggable converters trade
between the java object
and the serialized
representation
 If a input is internal straight
read from the source
 Leveraging on all the
available optimizations!
FOSS4G 2011, Denver
12th-16th September 2011

14. “Primitives” IO Parameters
 Numbers: byte, short, int, long, float, double, any Number
subclass, properly mapped in XML types
 Strings and CharSequence in general
 Date, Time, Timestamp
 CoordinateReferenceSystem (EPSG:xxx and urn:… forms)
 URLs
 Range (min -> max)
 Interpolation method
FOSS4G 2011, Denver
12th-16th September 2011

15. Complexes IO Parameters
 Geometries  Vectors
 GML 2  WFS 1.0 collection
 GML 3  WFS 1.1 collection
 WKT  GeoJSON
 Rasters  Zipped shapefile
 ArcGrid  Others
 GeoTiff  SLD 1.0
 Unreferenced  OGC Filter (1.0, 1.1)
PNG/JPEG
 CQL Filter
FOSS4G 2011, Denver
12th-16th September 2011

16. Open API
 Easy to plug your own custom process parameter I/O
FOSS4G 2011, Denver
12th-16th September 2011

17. Available Vector processes
FOSS4G 2011, Denver
12th-16th September 2011

18. JTS processes
FOSS4G 2011, Denver
12th-16th September 2011

19. Example JTS process
Intersection
POLYGON ((5 10, 10 10, 10 5, 5 5, 5 10))
FOSS4G 2011, Denver
12th-16th September 2011

20. Query oriented processes
 AKA “All you wanted WFS to do for you but he never
wanted to”
 gs:Aggregate: count/avg/max/median/min/stddev/sum on
a feature collection
 gs:Count: like WFS Hits, but on whatever source
 gs:Bounds: bounds of whatever source, missing from
WFS
 gs:Query: query any source like WFS
 gs:Unique: unique values of an attribute
 gs:Nearest: find the nearest features
FOSS4G 2011, Denver
12th-16th September 2011

21. Aggregation example
 Get the min, max and
sum of the PERSONS
attribute in the
topp:states layer
FOSS4G 2011, Denver
12th-16th September 2011

22. Other Vector processes
 gs:BufferFeatureCollection: buffer all features
 gs:Clip: cookie cut featuresgs:Reproject: reproject any
vector source
 gs:Simplify: DouglasPeucker simplifier, retain attributes
 gs:Snap: snap to grid
 gs:InclusionFeatureCollection: overlay and get all
features contained
 gs:IntersectionFeatureCollection: overlay and intersect,
retain attributes from both
 gs:UnionFeatureCollection: merge two collections in one
 gs:Import: save the features as a new GeoServer layer
FOSS4G 2011, Denver
12th-16th September 2011

23. Buffer example
FOSS4G 2011, Denver
12th-16th September 2011

24. Geometry <=> Feature
 gs:collectGeometries: lump up all feature geometries into
a geometry collection
 gs:feature: turn a single geometry into a feature collection
MULTIPOINT (
(-74.01046109936333 40.707587626256554),
(-74.0108375113659 40.70754683896324),
(-74.01053023879955 40.70938711687079),
(-74.00857344353275 40.711945649065406),
(-74.0118315772888 40.708529961953786),
(-74.00153046439813 40.719885123828675))
FOSS4G 2011, Denver
12th-16th September 2011

25. Raster processes
FOSS4G 2011, Denver
12th-16th September 2011

26. Raster processes
 AKA “All you wanted WCS to do for you but it never
wanted to”
 Add/Multiply: add and multiply two rasters (waiting for full
algebra to be implemented)
 Crop: crop a coverage based on the specified cutting
geometry
 RangeLookup: classify raster image based on a set of
ranges ([min,max] -> value)
 ScaleCoverage: rescale and translate a given raster
 StyleCoverage: apply a SLD style to a raster, getting back a
styled (but still georeferenced) one
 Georectify: turn a non geo-referenced coverage into one
based on ground control points
FOSS4G 2011, Denver
12th-16th September 2011

27. Crop example

28. Conversion processes
FOSS4G 2011, Denver
12th-16th September 2011

29. Conversion processes
 Bridging the raster and the vector world
 Contour: extracts isolines given a set of levels or a interval
 RasterAsPointCollection: extracts one point for each cell,
with band contents as attributes
 PolygonExtraction: extracts uniform polygons from raster,
eventually given a set of value ranges
 RasterZonalStatistics: given a raster and a polygonal
compute min/max/sum/avg/stddev of the cells falling in
each polygon, return an augmented polygonal
 VectorToRaster: rasterizes vectors keeping a chosen
attribute (a CQL expression eventually) as the band value

30. Contour example

31. Chaining
FOSS4G 2011, Denver
12th-16th September 2011

32. WPS Chaining
 Feed the output of a process into another process
 Allows for tree-like composition
 Let’s see a typical “clip and ship” example, both raster and
vector:
 Extract the rivers into the “restricted areas” polygons
 Extract from Blumarble any pixel in the USA
FOSS4G 2011, Denver
12th-16th September 2011

33. Raster clip and ship
FOSS4G 2011, Denver
12th-16th September 2011

34. Raster clip and ship
FOSS4G 2011, Denver
12th-16th September 2011

35. Vector clip and ship
FOSS4G 2011, Denver
12th-16th September 2011

36. GeoServer Integration 1: local data access
FOSS4G 2011, Denver
12th-16th September 2011

37. Direct data integration
 WPS normally reads from remote WFS/WCS, parsing
GML/GeoJSON or GeoTiff/ArcGrid
 When the source is local we can dodge it though, read
directly from the source (shapefile, DBMS, geotiff)
 Hit the fictious http://geoserver/wfs url for local WFS
 Hit the fictious http://geoserver/wcs url for local WCS

38. Store back results (only vector)
 gs:Import: saves the
vector results into a
store of choice, and
publish as a layer
 Use right away the
new layer from
WMS/WFS/WCS/WPS
 Still missing the
equivalent for rasters
FOSS4G 2011, Denver
12th-16th September 2011

39. GeoServer Integration 2:
rendering transformations
FOSS4G 2011, Denver
12th-16th September 2011

40. Rendering transformations
 On-the-fly data transformations inside rendering
chain
 Calling WPS processes from SLD docs
 Optimized for performance
FOSS4G 2011, Denver
12th-16th September 2011

41. Rendering transformations
 Point feature extraction from two band raster data (e.g.
Wind(u,v))
 Computation of direction and module from SLD
 Full SLD
 Call gs:RasterAsPointCollection
 Magnitude and direction of the arrow are computed on
the fly by using filter functions
 <WellKnownName>shape://carrow</WellKnownName>
 Working at visual resolution
 Use overviews and
decimation
 Fast with large datasets
FOSS4G 2011, Denver
12th-16th September 2011

42. Evolution and closing remarks
FOSS4G 2011, Denver
12th-16th September 2011

43. WPS*
 Deficiencies
 No support for asynchronous requests
 Missing request limits enforcements (e.g. input/output
maximum dimensions)
 Wish list:
 Scripting (Jython, GeoScript)
 Sextante, IDL, JGrass (Grass?) integration
 Improved robustness
 Jiffle (jai-tools) based raster algebra
 New layers as dynamic WPS processes (computing
data on the fly as people do WMS/WCS/WFS requests)
FOSS4G 2011, Denver
12th-16th September 2011

44. The End
Questions?
andrea.aime@geo-solutions.it
simone.giannecchini@geo-solutions.it
FOSS4G 2011, Denver
12th-16th September 2011