1.
4D Data Visualisation and Quality
Control
Peter Löwe, Jens Klump
GeoForschungsZentrum Potsdam
ploewe@gfz-potsdam.de, jklump@gfz-potsdam.de

2.
Intention of the talk
● Give a better understanding of the relevance of
„preview formats“ for quality control
of complex enironmental data.
● Demonstrate how preview formats can be generated with
FOSS geoinformatics applications.
● Real world example:
– A soil erosion topic, relying on the processing of
„complex data“
– which uses FOSS GIS for this task.

3.
•Topic, theory
•Hypothesis and approach
•Technical frameworkTechnical framework
•Modelling
•Results / Lessons learnedResults / Lessons learned
Overview: Soil Erosion Project

4.
Soil erosion is the process of
soil destruction, a natural process,
which can be initiated or amplified by
human land management. ... Soil
erosion can deminish the agricultural
yield significantly.
Soil Erosion: The Problem
In addition to human use of the
Earth's surface, climate is a key
factor. It provides a means of
transportation for soil material to be
carried away.
How can theseHow can these
processes beprocesses be
modelled ?modelled ?
Climate
Terrain
Soils
Vegetation
Humans
SOIL EROSIONWind Water
Transportation
Control Enhance
Start

5.
Water, as in ... rainfall
● We need a map of the rainfall distribution (simple task)
– Regular updates
– Sufficient resolution
– Reliable
● The changes of this map can be used to calculate the
„potential erosivity“ of the rainfall for a given area:
– Total amount: how much water comes down in total ?
– Temporal Pattern: Once only, repeated soaking ?
– Small droplets, big droplets ?

6.
Idea and approach
Hypothesis: There are small, temporally fluctuating peaks ofThere are small, temporally fluctuating peaks of
erosiviy due to the convective weather situation. How can theseerosiviy due to the convective weather situation. How can these
erositiy peaks be charted?erositiy peaks be charted?
A sufficiently high temporal (When ?) and spatial data coverage (Where ?),
is needed, and also a measure of confidence for the data values (How
much ?).
To answer „When - How much - Where“ the radar reflectivity
products must be accessed and processed.
ModellingTechnical
Framework
Results
Analysis and
encoding of the
Erosivity.
Geoinformatics,
Information-
logistics,
Remote sensing,
Radarmeteorology
•Verification
•Validation
•Results

7.
Practical Approach
We use ground-based weather radar for a test
site:
– 5 Minutes scan rate (200 km radius, 18 km vertical)
– Pixel/Voxel resolution 1 km
– continous Reflectivity Data (not rain)
400 km
18 km Lower Atmosphere

8.
X
X
X
X
X
a
b
c
d
Erosivity
maps
Reflectivity GIS-layers
for defined altitudes
Rainfall maps, Pluviogram
X
XXX
X X X
X
1
4
2
Erosivity-
Model
What type of
weather occurs
when, where?
Where does erosivity potential
occur ?
When does how much rain fall
where ?
-
Data flow
3D -> 2D
transformation
18km altitude
1km altitude

9.
Precipitation-
data stream
of „radar rain“
Init
State 1
Hibernate
State 2
Store
State 3
Pause
P: Precipitation
D: Dry
D
D
D
P
P
P
A „virtual rain gauge“ [state
machine] is simulated for each
spatial cell of the radar coverage.
Erosivity values are derived
according to the cell's individual
input data stream.
For this reason, agent technology
is used, as each „gauge-agent“
must keep its own record of
previous precipitation events.
Erosivity-Modelling
Erosivity-
data stream Maps
Index-
value
Cell-
Agent

10.
Tools of the trade
● GRASS GIS
– Raster and volume data processing
– 2D Animations (flip-books)
● NVIZ (part of GRASS)
– 3D visualisation and animation
● Paraview
– 3D visualisation and animation

11.
GRASS GIS
● GRASS GIS is a Geographic Information
System (GIS) used for geospatial data
management and analysis, image processing,
graphics/maps production, spatial modeling,
and visualization.
● Oldest and largest FOSS GIS project
● GRASS is official project of the Open Source
Geospatial Foundation.
● Scriptable
● „Backend use“ in QuantumGIS, PyWPS,
JGRASS

12.
Calculating Totals
24h total
erosive
16:18:50
Hours
16:43:30
Hours
16:59:56
Hours
Erosivity
Reflectivity
Σ
Σ
Left: Reflectivity
Centre: Rainfall
Right: Erosivity

13.
Erosivity-Totals display of local erosivity pulses
Elevation: Rainfall total
Color: Erosivity total
Conclusion:
The model implementation works !

14.
Animated Time Series
Animated displays of reflectivity, derived “radar rainfall” and the
corresponding erosivity peak pattern were created with GRASS
GIS.
Reflectivity “Radar-Rain” Erosivity
The erosivity ribbons (right) follow the rainfall fields (center): The model works

15.
The Challenge
● 2D animations and 2.5D images show that the
erosivity modelling „works well“
– [the erosivitiy peaks follow the rainfall fields]
● However, the model depends on input data:
● Can we trust the data ?
– Metadata appears correct.
– [are the rainfall fields correct ?]
● Weather Radar provides 3D data.
– [3D->2D transformation: Correctly done?]

16.
3D: Straightforward Approach
NVIZ-Animation (GRASS): „Real Clouds“
Volume [Full Information]= Reflectivity Values (30/40/50 dBz)
2.5D Surface: MaxReflecivity(Color), Radar horizon (Elevation)
Flattening

17.
NVIZ
● „GRASS in-house“ visualisation tool
● Pro: Works directly on the internal database
● Pro: Scriptable
● Con: small user base, bugs, missing documentation
● http://grass.itc.it/nviz/

18.
● Parallel Visualisation Toolkit
● Frontend to VTK + QT
● Large userbase.
● GRASS-related import issues:
– Loosely coupled via file system
– Ascii-VTK-Format – currently not effective for use. Improvements
are hoped for later this year
● http://www.paraview.org

19.
Garbage in, Garbage out
● Can we trust the rainfall information of the
weather radar ?
● Model results are based on rainfall data.
● Errors and Biases in the rainfall data will affect
all derived products.
● What about transient biases which might
vary in time or space?
Flattening
Trust
Trust
3D data
2D information

20.
Boredom in, Boredom out
● Large data archives exist and more data are
added every day.
● How can we easily identify time intervals when
„some interesting weather“ has occurred?
● We could watch it all in 4D (3D over time):
– Takes too much time, is incredibly boring
– Problem to watch the right things at the right time.

21.
Introducing Preview Formats
● A visual preview format provides a condensed
view with the relevant information for the
current question („determinant“).
● Humans are visualy oriented: preview formats
(shapes/volumes) are easier to comprehend
than numbers.

22.
Transient Phenomena Preview
● What was the weather like
for a 24 h period ?
● Try this: 2D Animation
(Flip-Book)
● Alternative: Stacking of the flip-books pages
(just the ink, really) and look at all pages at
once.
● Howto: Creation of a volume in GRASS GIS,
visualisation by Paraview.

23.
From single drawings
1 2 3
„Radar Rain“
Erosivity

24.
Flip-book Volume
time
1
2D Space: Rainfall field
2
3
Yellow: Rainfall
Red: Erosivity
Data Errors
(ground targets)
Rainfall field
Rainfall field
Erosivity Peaks
Not real
clouds !
1
2
3

25.
Ce n'est pas un nuage!
● Detailed top-view of the track of a precipitation field (yellow) and the derived erosivity pulses
(red). Note the highly localized distribution of the erosivity pulses. This information can be
used to calibrate the interaction between point-sampling rain gauge networks, weather radar
calibration and soil erosion plots.
Painting of a pipe
Rainfall tracks
of clouds
(+ „erosivity tracks“)

26.
Quality Control 1.0
A precipitation field and its resulting erosivity
pulses shown in side-view.
The image does not show real world clouds but
precipitation- and erosivity tracks.
„Soaking“ The height of a rainfall track
tells us how long it did rain at
a certain location
Rods of
eternal
soaking:
Data errors

27.
Dimensional Collapse
The 2D (xy) rainfall field was „squeezed“ out of
the 3D (xyz) weather radar data, implicitly
„collapsing“ the vertical dimension.
The stacking of the time frame „flip-book“ pages
substituted the altitude (z) dimension by the
time dimension.

28.
Collapse 2.0
This approach can be followed further:
● In the previous example we collapsed the z-
dimension
● Now we collapse the horizontal (xy)
dimension.
● The resulting diagram is a preview format
commonly used in meteorology: the
„Contoured Altitude by Frequency Diagram“
(CFAD).

29.
Contoured Frequency by Altitude
Diagrams (CFAD)
● CFAD can be created from 3D radar reflectivity data
(original airspace radar scan). The 3D data set is sliced
vertically.
● A histogram of the reflectivities (1D) is generated for each
slice/layer.
● Stacking the histograms gives us a 2D synopsis of the
current situation in the scanned airspace.
● This tells us a lot about the weather and potential
measurement errors.
● In our case, this task is done in GRASS.

30.
CFAD – An Example
● Contoured Frequency by Altitude Diagram (CFAD).
Numbers on contour lines give the number of voxels in the
observation area with a given radar reflectivity. The CFAD
gives a snapshot of weather intensity at different altitudes in
the lower atmosphere.

31.
2D Animation
Leafing through the flip-book:
Weather CFAD

32.
CFATD: One step beyond
● Contoured Frequency Altitude by Time Diagram
adds the time dimension, resulting in a volume
body.
● The shape of the CFATD makes it easy to
identify:
● periods of high radar reflectivity, i.e. intense
weather, and
● Errors in the radar or processing chain.
● Done in GRASS, displayed in Paraview

33.
Flip-book volume
● Once again we can create a volume from the
flipbook.
Time
Altitude
„Droplet Size“
Iso Surfaces
resemble levels of
droplet counts (a
few, many, lots)
Critical threshold: If the inner
layer (many droplets) of the
„loaf“ exceeds it, then there is
heavy downpour or even hail.

34.
Visual Quality Control
● CFATD gives a convenient and reliable quality measure
for observations not to use
● If the CFATD structure appears blocky, or „non-organic“:
discard the data
Faulty data
Faulty data

35.
Better data, better models
● 4D previews for „Live Quality Control“ in sensor
systems:
– Weather Radar does „now-casting“
● It looks into the distance (right now)
● but not into the future
– Real-time generation of CFATD „loaves“ could be
used for radar system calibration and maintenance.
What level of quality
do we get RIGHT NOW ?

36.
Applications in Grid/eScience
● Dimensional collapse and 3D animation of data
can be used as a preview format for very
large/complex datasets.
● The computing power needed for the
generation of these preview formats can be
sourced from the Grid.

37.
Conclusion
● Complex (4D) data are not easy to interpret.
● Preview formats enable identification of biases
drifting in time and space in complex data.
● Preview formats save time in the selection of
useful data.
● You can do it, too:
– Ukrainian Radarsystem (MRL-5 + Linux-based
Operating Software (Titan))
– GRASS + Clips + Database + Paraview

38.
Thank you for your attention !