Presentation to UK QGIS South East User Group, 2nd April 2014 at Imperial College London

1. AnalysingAnalysing
OpenStreetMap DataOpenStreetMap Data
with QGISwith QGIS
JerryJerry CloughClough
SK53 on OpenStreetMap
@SK53onOSM
SK53.osm@gmail.com

2. My BackgroundMy Background
● Biologist, Computer Scientist, Management Consultant
Naturalist
● GIS--, DB++
– OLAP platforms since late 1980s
● OSM since Dec 2008
● QGIS since Jan 2011 (1.1 => 2.0)
● Mainly analytical uses
● Interests: landuse, landcover, biotopes, local
government open data, (pubs)

3. OSM Need to KnowOSM Need to Know
● Open Vector Data
● 3 Geo-primitives
– Node (= point)
– Way (= linestring)
● Closed ways may represent
areas
– Relations
● More complex geothings
– Multipolygons
– Geo-relations
● NO layers
● Volunteer Sourced
– “Wiki map of the
world”
● Free Tagging
– aka Folksonomy
● Variable Coverage
–

4. Some 'Interesting' Stats for GBSome 'Interesting' Stats for GB
(with apologies to Ordnance Survey)
● Pylons: 58,487 (OSGB: 80,517)
● Post Boxes: 42,742 (93.728)
● Camp sites: 3,192 (8,908)
● Buildings: 1,890,835 (35,397,754)
● Bus Stops: 215,720 (354,099)
● Petrol Stations: (7,702)
● Addresses: 27,341,262 (OSGB);
532,886
● Electricity Poles: 94,199 (183, 987)
● Road length: 522,627 km
(407,532 km)
● 5 post boxes with Edward VIII
cypher
● Only 110 War Memorials
● 847 Fire Hydrants
● 1,378 Real Ale pubs
– 82 with Real Fires
● 4771 Cycle Parking
● 300 Wildlife Hides
● 5,552 Stiles
● 1,774 Canal Locks
● 2 Knitting Shops
Ordnance Survey figures: /www.ordnancesurvey.co.uk/blog/2013/04/10-fascinating-facts-from-
ordnance-survey/
OSM figures (April '13): /taginfo.openstreetmap.org.uk/

5. How I use QGISHow I use QGIS
● OSM data => PostGIS DB
● Initial analysis in QGIS
● PostGIS routines for more complex data
manipulation
● R and other tools for stats/segmentation
● Visualisation in QGIS

6. Case Study 1 : PubsCase Study 1 : Pubs

7. Pub Density in Great BritainPub Density in Great Britain

8. Cartograms based on PubsCartograms based on Pubs

9. Cartograms based on PubsCartograms based on Pubs

10. Case Study 2:Case Study 2:
Simulating Urban AtlasSimulating Urban Atlas
● 300+ EU cities population >100k
– 119 in April 2010
– 228 in Sept. 2010
● Baseline date 2006-7
● Used 2.5 m imagery
● 5-6 year refresh cycle
● Minimum Map Unit (MMU) 0.25 ha
urban / 1 ha rural
http://sia.eionet.europa.eu/Land Monitoring Core Service/Urban Atlas

11. Examples of mapping OSM TagsExamples of mapping OSM Tags
to Urban Atlas Categoriesto Urban Atlas Categories
UA
Code
UA Description OSM Tags Comments
14100 Parks, Urban Green Space amenity=graveyard
landuse=cemetery
leisure=park
leisure=village_green
14200 Sports Areas landuse=allotments
landuse=recreation_ground
leisure=golf_course
leisure=pitch
leisure=stadium
20000 Agricultural Land landuse=farm
landuse=farmland
landuse=pasture
landuse=orchard
landuse=vineyard
leisure=nature_reserve
natural=scrub,natural=heath
natural=wetland
natural=rock,natural=scree
Additional OSM tags are also valid for
this code (e.g., natural=glacier)
30000 Woods & Forest natural=wood
landuse=forest
50000 Water landuse=reservoir
waterway=riverbank
natural=water

12. Painter’s Algorithm in QGISPainter’s Algorithm in QGIS

15. Case Study 3:Case Study 3:
Retail in OSMRetail in OSM

16. Retail Geo-dataRetail Geo-data

DriversDrivers
–Personal interest
• Used to consult to large retail chains & FMCG firm
–Article in Directions about Geolytix
• Featured Nottingham, my main mapping location
– Availability of Food Hygiene Open Data

QuestionsQuestions
– How difficult was it to systematically get retail landuse and retail
sites into OSM?
– Was OSM data good enough for segmentation of landuse?
Source: Geolytix in Directions Magazine

17. FHRS 1
(local) Government Open Data
• Addresses
• Partial geolocation
– postcode
• Business Type
– Pub/Bar/Nightclub
– Supermarket
– Café/Restaurant
– Other Retail
• Covers at least 50-60% of retail
outlets
• Usually current
– Typical inspection interval 6-12
months

18. Tracking my ownTracking my own
OSM MappingOSM Mapping
●
Plot premises by postcode centroid
●
OpenLayers plugin for background
●
Track areas visited and added to
OSM in Excel Spreadsheet
●
S/s linked in as layer
●
Update to show places to map
●
Push un-surveyed postcodes out as
a GPX
●
Load GPX on Garmin

19. Conclusions Nottingham Retail 2

20. Conclusions Nottingham Retail 3

21. Classifying Retail
Areas

22. Case Study 4 : Street LightsCase Study 4 : Street Lights

23. Street Lights and OSM QualityStreet Lights and OSM Quality

24. Street Lights and OSM QualityStreet Lights and OSM Quality

25. Maps for DogsMaps for Dogs

26. Approaches to using OSM DataApproaches to using OSM Data
● Direct from OSM (API/ XML
files)
– Earlier Plugin (deprecated)
– 2.0 method
– ogr2ogr
● via Postgres DB
– osm2pgsql
– osmosis
– imposm
– osm2postgresql
– osm2pgrouting
● via Shapefiles
– Geofabrik
● Limited number of
layers
● Limited sets of
attributes
– Roll your own
http://wiki.openstreetmap.org/wiki/Osmosis
http://wiki.openstreetmap.org/wiki/Osm2postgresql
http://sourceforge.net/projects/osm2postgresql/
http://download.geofabrik.de/

27. Postgre-SQL/GIS and osm2pgsqlPostgre-SQL/GIS and osm2pgsql
● osm2pgsql converts osm
data to postgres/postgis
– Slightly lossy
● Relationship between members
of multipolygons
● Road and other network
topologies
– Can choose projection
● default 3087
– Can tweak import rules
● Style files
● LUA
– Fiddly under Windows
● osmconvert & osmfilter
– Very useful tools to preprocess
data for particular purposes
● Filter on OSM tag values
● Convert polygons to centroids
●
ALWAYS USE -k option
– Stores less widely used tags as
an hstore column
– Maximises flexibility
– Throws away coastline by
default (sometimes useful to
keep it)
http://wiki.openstreetmap.org/wiki/Osm2pgsql
http://wiki.openstreetmap.org/wiki/Osmconvert
http://wiki.openstreetmap.org/wiki/Osmfilter

28. ProblemsProblems
● Polygon Handling
● Generalisation
● Missing data
● Free-form Tagging

29. The Problem with PolygonsThe Problem with Polygons
• No Area primitive in OSM
• Overlapping polygons
• OSM
– Broken polygons
– Intersecting polygons
– osm2pgsql
• In QGIS
– Render OK
– Geometry Operations fail
• Essential tool:
cleangeometry PostGIS
function (SOGIS)
http://www.sogis1.so.ch/sogis/dl/postgis/cleanGeometry.sql

30. GeneralisationGeneralisation
• Multiple Ways
– Most objects will be formed
from many OSM ways (e.g,
Thames, M4)
• No simplified data
– Dual carriageways
– Roundabouts and flares
– Built-up areas
– Over noded for many uses
• Fine-grain tagging
• May require elaborate pre-
processing

31. Tagging IssuesTagging Issues
• Synonymy
– natural=wood
– landuse=forest
• Variable Semantics
– highway=path
– place=hamlet
– highway=trunk
(gets changed every now & then)
• Tagging for the Render
– natural=sand for Golf bunker
– landuse=grass Everywhere
• Semantic Degradation
– Tag with accepted semantics being used for
something else
– landuse=recreation_ground for Ski areas in US
• Odd names
– shop=mall Shopping Centre

32. Incomplete DataIncomplete Data

33. Other things I do in QGISOther things I do in QGIS
● Vice County maps using OSGB Open Data
– Plan to investigate Atlas module now
● Distribution Maps of Trees in N. Hemisphere
● Attempts to analyse suburban structure based
on building dates
– Used Portland Oregon data
– Huge Delauney triangulation

34. ConclusionsConclusions
● QGIS fantastic tool for a wide range of manipulations of
OpenStreetMap data
– Particularly well suited for
● Prototyping & visualisation
● Combining with other Open Data sources
● Recommend use with PostGIS
– Maximises flexibility
– Reduces complexity of potential learning curve for the OSM
toolchain
– Ability to manipulate data in PostGIS may be important
● Be aware of limitations and gotchas of OSM data

35. Supplementary SlidesSupplementary Slides
● Managing polygons for detailed analysis (Urban
Atlas)

36. PostGIS Processing
OSM
Polygons
OSM
Lines
Painter's
Algorithm
Rules
Clipped
Polygons
Clipped
Lines
Cleaned &
Clipped
Polygons
UA Shape
Polygons
Clean Geometry
Gridded UA
Classes
Filter on Tags & Grid
Gridded &
Buffered
UA Classes
Tag Filter, Grid & Buffer
Clip to Area
Clip to Area
Piecewise Union Union Step 1
Union
Union Step 2
Merge
Class Gridded
Polygons
Merge
Grid
Gridded UA
Polygons
Union
Clipping areas
by UA Class
ClippingRegion
Final
Polygons
Compare
UA/OSM
Union/Intersect/
Difference

37. Comparison 1
No OSM Data
Residential
Disagreement
Agreement
Nottingham Area

38. Comparison 2
No OSM Data
Residential
Disagreement
Agreement

41. Agreement

42. Supplementary SlidesSupplementary Slides
● Examples of OSM Mapping from Port-au-Prince
January 2010