Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...
Raster data in GeoServer and GeoTools: Achievements, issues and future developments
1. Raster data in GeoServer and GeoTools:
Achievements, issues and future developments
Ing. Simone Giannecchini, GeoSolutions
Ing. Daniele Romagnoli
FOSS4G 2011, Denver
12th-16th September 2011
2. Outline
Who we are
Raster Developments
JAITools
ImageIO-Ext
GeoTools
GeoServer
GeoBatch
FOSS4G 2011, Denver
12th-16th September 2011
3. GeoSolutions
Founded in Italy in 2006
Expertise
• Image Processing, GeoSpatial Data Fusion
• Java, Java Enterprise, C++, Python
• JPEG2000, JPIP, Advanced 2D visualization
Supporting/Developing FOSS4G projects
GeoTools, GeoServer
GeoBatch, GeoNetwork
Clients
Public Agencies
Private Companies
http://www.geo-solutions.it
FOSS4G 2011, Denver
12th-16th September 2011
4. JAITools
High performance raster processing
Extends, complements, replaces Oracle Java Advanced
Imaging (JAI) Library
Open Source, BSD license
Low Level Operations
e.g. RangeLookup
Higher Level Operations
Vectorize, Contour
Works mostly in pure raster space (Image Processing)
Stable release is 1.2.0
FOSS4G 2011, Denver
12th-16th September 2011
5. JAITools – Low Level Operations
Basic Extensions to JAI processing
RangeLookup
Raster (re)classification
Raster Statistics
ZonalStats
ClassifiedStats*
KernelStats
High Perf ColorMap*
ColorMap Extraction/Inversion
High Perf VectorBinarize
Vector to raster conversion
FOSS4G 2011, Denver
12th-16th September 2011
7. ImageIO-Ext
High Performance raster I/O
Extends, complements, replaces Oracle JAI ImageIO
Library
Open Source, LGPL and BSD licenses
Plugins:
BigTiff*, netCDF-CF,GriB1,MatFile 5, HDF
JP2000 (Kakadu)
GDAL integration via ImageIO-ext
JPEG2000, MrSID, ECW
BigTIFF, ERDAS Image, HDF4
Complete list here
FOSS4G 2011, Denver
12th-16th September 2011
8. ImageIO-Ext – Improving Tiff
Improved standard Tiff reader and writer
Support for r/w double and float data type
JP2 compressor/decompressor*
Improved metadata Management
More aggressive caching
Based on soft references
BigTiff support
Read/write
Reduced resolution page support
Overviews!
FOSS4G 2011, Denver
12th-16th September 2011
9. ImageIO-Ext – JPEG2000
Kakadu based reader/writer
Wraps kakadu JNI APIs as ImageIO
ImageReader/ImageWriter
Support for GeoJP2
Supporting additional JPEG2000 code params
Tiling, Corder, Tile Marker, Part Tile Marker,
Sprofile , etc…
Some of them needed for NITF support
Fine grain control over JPEG2000 codestream
creation
FOSS4G 2011, Denver
12th-16th September 2011
10. ImageIO-Ext – GDAL
Wraps GDAL behind ImageIO API
Mainly Read Capabilities
Write support experimental (JP2Kak, GeoTiff)
Geotiff, JP2Kak, ECW, MrSID, JP2MrSid, Erdas, IDRIS,
Esri Binary Grid, Envisat, NITF, DTED, DOQ1, DOQ2, etc..
Expose Generic IIOMetadata for basic Dataset info
Georeferencing, Band layout, CRS, etc..
Expose Specific IIoMetadata for Driver specific
info
E.g. MrSid metadata
1.1.0 supports GDAL 1.7.3 directly (no custom bindings!)
FOSS4G 2011, Denver
12th-16th September 2011
11. GeoTools
The Open Source Java GIS Toolkit
LGPL
Management of Geospatial data
CRUD Operations
Raster and vector
Rendering
GDAL Support
Based on best-of-breed Open Source projects
Java Advanced Imaging (JAI), ImageIO
Java Topology Suite (JTS)
FOSS4G 2011, Denver
12th-16th September 2011
12. GeoTools - JPEG2K Improvements
Leverage on ImageIO API
Read/Write Capabilities
Fine grain support for writing with code params
GeoJP2 support
Other capabilities
Re-Tiling on the fly
ColorMasking
Immediate (Java2D) VS Deferred Mode (JAI)
Multithreaded loading (per tile)
FOSS4G 2011, Denver
12th-16th September 2011
13. GeoTools - Tiff & BigTiff
Leverages on ImageIO API
Read/Write Capabilities
Internal/External Overviews support
Improved Metadata management
Less memory
Wise caching – more speed
Heavy Refactor ongoing
Re-Tiling on the fly
ColorMasking
Immediate (Java2D) VS Deferred Mode (JAI)
FOSS4G 2011, Denver
12th-16th September 2011
14. GeoTools - Raster Processes
Basic Building Blocks for GeoServer WPS
Clip and Ship (raster and vector), buffering, vector
intersection, vector union, etc…
JAITOOLs
Vectorize, Contour
Statistics
GDAL based*
gdal_translate wrapper
gdalwarp wrapper
org2ogr wrapper
FOSS4G 2011, Denver
12th-16th September 2011
15. GeoTools - ImageMosaic
Wraps multiple homogeneus raster as a single one
Improvements
Support for footprint on single tiles
Support for external catalog of tiles
Autoindexing of directories
Support for TIME, ELEVATION
Support for OGC & CQL Filtering
Multithreaded loading
Support for heterogeneous overviews layout
Supports any format with an ImageIO reader
Re-Tiling on the fly
ColorMasking on the fly
Immediate VS Deferred Mode
FOSS4G 2011, Denver
12th-16th September 2011
16. GeoTools - ImagePyramid
Multiresolution/Multiscale tile based raster
plugin
Take a large raster split in small
chunks/tiles mosaic and
downsample start over until happy
Upon request
Select right resolution level select needed
chunks/tiles assemble do some magic and return
Improvements
Automatically recognize output of gdal_retile
Automatic tile indexing
Other improvements inherited from ImageMosaic
FOSS4G 2011, Denver
12th-16th September 2011
17. GeoTools – Optimized Reprojection
Standard Approach Accurate Transformation
Each pixel is back mapped using the full transformation
Grid to world on the target
Backwards projection towards the source
World to grid on the source
Slow!, cannot be optimized in hardware
Optimization Approximate Transformation
Local Linear Approximation using Grid Warps
Use a set of known positions and their transformations
Everything else is linearly interpolated between the known
positions
Estimation through iterative process (speed vs quality)
FOSS4G 2011, Denver
12th-16th September 2011
18. GeoTools – Optimized Reprojection
Linearly interpolate on the top and bottom lines
Linearly interpolate between the two resulting points
FOSS4G 2011, Denver
12th-16th September 2011
20. GeoServer
GeoSpatial enterprise gateway
Java Enterprise
Management of raster and vector data
Standards compliant
OGC WCS 1.0, 1.1.1 (RI)
OGC WFS 1.0, 1.1 (RI), 2.0
OGC WMS 1.1.1, 1.3
OGC WPS 1.0.0
Google Earth/Maps support
KML, GeoSearch, etc..
FOSS4G 2011, Denver
12th-16th September 2011
21. GeoServer - Rendering Transformations
In Place Transformations during rendering
Specified in SLD docs
Applied right before the rest of the
directives
Can be used to:
Extract and draw contours on
the fly
Extract and draw polygons
on the fly
Extract and draw symbols (e.g. arrows) on the fly
Transform geometries on the fly
(Add your own!)
FOSS4G 2011, Denver
12th-16th September 2011
22. GeoServer - Rendering Transformations
On-the-fly
Polygonalization
On-the-fly
Contouring
Band
Raster to Combine +
point vector Symbology
FOSS4G 2011, Denver
12th-16th September 2011
23. GeoServer – Improved Raster Rendering
Hit JAVA2D Bottleneck – Scalability Issue
http://bit.ly/qJcZBi
http://bit.ly/oe0CHo
Created direct raster rendering path with JAI
On-the-fly
Polygonalization
Drop-in replacement (1 raster at time)
On-the-fly
2x speedup
Contouring
2x/3x scalability improvement under heavy load
Enabled/Disabled via Java Switch
Band
Raster to Combine +
point vector Symbology
FOSS4G 2011, Denver
12th-16th September 2011
24. GeoServer – Improved Raster Chains
Performance
Improved internal JAI Concurrency
Introduced and Improved JAI Tile Recycling
Reduced depth of JAI Processing Chains
More aggressive factory caching On-the-fly
Added a few specific high perf JAI operations
Polygonalization
e.g. crop On-the-fly
Contouring
Created Specific high perf ROI class
Improving Tile Caching
Work ongoing
Band
Transparent File vs Memory Caching when writing
Raster to Combine +
raster point vector Symbology
FOSS4G 2011, Denver
12th-16th September 2011
25. GeoServer – More Raster WPS Processes
Relies on GeoTools Processes
Vectorize
Contouring
RasterClassification On-the-fly
Polygonalization
ZonalStats On-the-fly
Contouring
GDAL Processes
gdal_translate
gdalwarp
Band
ogr2ogr Raster to Combine +
point vector Symbology
FOSS4G 2011, Denver
12th-16th September 2011
26. GeoServer – Support for TIME &
ELEVATION dimensions + Filters
WMS TIME and ELEVATION dimensions
…&TIME=20110914T00:00:000Z&ELEVATION=10
Implemented thanks to ImageMosaic
Support for the Current keyword
On-the-fly
Support for Default values Polygonalization
Support for On-the-fly
Contouring
TimeInstant
Discrete Range Tb/Te/Res, Eb/Ee/Res
Continuous Range Tb/Te, Eb/Ee
Any combination of the above
Support for OGC and CQL filters Band
Raster to
Filter Mosaic granules based on index Combine +
attributes
point vector Symbology
FOSS4G 2011, Denver
12th-16th September 2011
27. GeoBatch
Real-time produced data
Remote Sensing, In situ, Models, etc..
Recognizable, isolate data streams
Flow
Incoming series of geospatial data
Specific format
Defined rules for preprocessing
Defined rules for ingestion and/or exploitation
Automatic Ingestion and preprocessing No manual intervention
Example, GeoTiff preprocessing
GeoNetwork
Producer Tile Overviews Publish
GeoServer
FOSS4G 2011, Denver
12th-16th September 2011
28. GeoBatch – GeoTiff Preprocessing
GeoTiff serving optimizations
Internal Tiling
Compression
Internal/External Overviews
Mosaicking On-the-fly
Polygonalization
Pyramiding
On-the-fly
GeoServer publishing
Contouring
2 Options
Pure Java based on ImageIO and GeoTools
External Processes with calls to GDAL
Band
Raster to Combine +
point vector Symbology
FOSS4G 2011, Denver
12th-16th September 2011
29. GeoBatch – MetOc Data Preprocessing
Optimization of MetOc for WMS/WCS serving
Conversion from
GRiB 1
netCDF-CF
COAMPS On-the-fly
Polygonalization
BUFr*
On-the-fly
To Contouring
Multiple 2D geotiff (Internal Overviews)
Indexing with ImageMosaic
More preprocessing
Fixing No Data Band
Change UoM Raster to Combine +
point vector Symbology
Band Processing Wind(U,V) Wind ( mod, theta)
FOSS4G 2011, Denver
12th-16th September 2011
30. The End
Questions?
simone.giannecchini@geo-solutions.it
FOSS4G 2011, Denver
12th-16th September 2011
32. GeoTools – Optimized Reprojection
A raster reprojection
involves chaining
togheter the standard
reprojection with the grid
to world transformations
of source and target
FOSS4G 2011, Denver
12th-16th September 2011
33. GeoTools – Optimized Reprojection
Set of evaluation vectors used in the
Recursive algorithm
Test distance between actual position of the reprojected point
and the predicted by a linear interpolation
FOSS4G 2011, Denver
12th-16th September 2011