Presented at the 46th Society of Cartographers Summer School in Manchester on September 10 2010. The abstract for the talk was as follows: "OpenStreetMap is coming of age, but as it starts to be used more in the mainstream, the age-old questions of quality and completeness are coming to the fore. A range of data sources have been used to build up the map in the UK, from GPS traces to aerial imagery, historic mapping, NaPTAN and the OS Open Data release, each with their own benefits and limitations. This talk looks at a number of studies and tools developed to quantify, compare and address accuracy and coverage of the project in the UK, in an attempt to answer the key questions - is it complete yet and just how good is it?"
2. Contents Quality & Completeness Sources of Data Scans of Imagery for Tracing Imports Studies, Tools and Methods Comparison Other Considerations OSM in the Real World
3. Quality & Completeness OpenStreetMap – a crowdsourced spatial database of the world. How good is the data compared with real life? Is everything important on the ground in the database?
4. GPS Traces In the beginning (~2004) Tracks from GPS receivers Consumer-grade handheld devices Prone to random and systematic error Particularly in urban areas (multipath) Particularly if walking Walk both ways down street, take the “average”? In theory quality improves with multiple traces Metadata doesn’t include measured quality eCourier van GPS traces for London
8. Out-of-Copyright Map Tracing Crown Copyright expires after 50 years Not so useful for urban areas Great for rural areas – less likely to change Old, low resolution map better than nothing Started with Richard Fairhurst’s collection in early 2006 Quality dependent on scan resolution and quality of rectification Paper warps, especially over 50 years
10. Yahoo Aerial Imagery Tracing Agreement in late 2006 that allowed tracing In the UK, some large urban areas covered Accuracy better than GPS but not as good as newer imagery sets (e.g. Google) Phantom roads Hidden barriers
11. Yahoo Aerial Imagery Tracing Coverage of many countries Iraq: “The whole road and rail network of the Baghdad area has been mapped using the Yahoo! imagery. As have some parks, forested areas, etc.” Australia (Sydney): “Completion of the greater metropolitan railway network… several complex road interchanges not practical to map on the bike”
13. NaPTAN Public transport nodes Once-off copy of DtT database in 2009 Most useful for “less glamorous” bus stops as railway station coverage was already complete in OpenStreetMap Rich metadata Envisaged as two-way, OSM would fix poor locations and might be fed back Imported on a county-by-county basis
14. NaPTAN NOVAM-Viewer Bus stops For reconciling NaPTAN/OSM Graphical tool Manage manual verification of the locations and metadata
15. Walking Papers Designed for large scale POI addition The source is annotations to existing OpenStreetMap data so ultimately dependent on the other sources Specially rendered to print out and add Points of Interest to Automatic georeferencing for tracing If you want it
17. Imports A common theme so far – tracing Inherently poor quality due to user error Tiger (U.S.) AND (Netherlands) OS Open Data
18. Tiger (U.S.) From US Government Imported in late 2007 (data from 2005) Data quality low but good coverage More difficult to create a community to fix rather than map Specialist tools: CloudMade has realised potential of a cleaned-up set and has created web tools to help [animation]
19. AND (Netherlands) July 2007 Relatively straightforward as there was little existing OSM mapping for the Netherlands From a digital mapping data company with navigation and LBS products First “complete” country in OSM “15-20m” offsets in places between Yahoo aerial imagery and the AND data
20. OS OpenData (U.K.) No bulk imports (c.f. Andy Robinson) Community still debating UK coverage already too comprehensive Possible licensing concerns Building outlines possibly the biggest win Some test imports – automated feature detection from Street View rasters Isle of Sheppey
23. Studies on Quality MukiHaklay (UCL CEGE) “Haklay, M., 2010, How good is volunteered geographical information? A comparative study of OpenStreetMap and Ordnance Survey datasets” Environment and Planning B: Planning and Design 37(4) 682 – 703 “on average within about 6 m of the position recorded by the OS, and with approximately 80% overlap of motorway objects between the two datasets” Analysis in 2008 (Meridian 2)
24. Studies on Quality MukiHaklay and AamerAther Beyond good enough? Spatial Data Quality and OpenStreetMap data OSM vsMasterMap I.T.N., looking at road lengths “The analysis shows that when A-roads, B-roads and a motorway from ITN are compared to OSM data, the overlap can reach values that are over 95%. … OSM is of better quality than Meridian 2.” “Positional accuracy is satisfactory for many applications. Attribute accuracy is also satisfactory.” Spring 2009
25. Studies on Quality “Just how good is it?” As good as many consumer-grade online mapping websites Depends on the care and keenness of the “local champion”
26. Studies on Completeness MukiHaklay Completeness in volunteered geographical information – the evolution of OpenStreetMap coverage (2008-2009) “OpenStreetMap already covers 65% of the area of England, although when details such as street names are taken into consideration, the coverage is closer to 25%. Significantly, this 25% of England’s area covers 45% of its population.” Affluence bias Covers to October 2009
27. Studies on Completeness MukiHaklay: OSM vs Meridian 2 (red) Left: Features only Right: With attributes
28. Studies on Completeness Peter Reed DfT highway length figures per county “Is it complete yet?” Depends on what you need Depends on where you need it Improving!
31. Getting Completeness Mapping Parties May 2006: “Mapchester” Chris Perkins, Andy Robinson January 2007: Central London Steve Chilton Richard Fairhurst May 2009: Milton Keynes Me, Andy, Steve October 2009: Atlanta
34. Getting Completeness London Mapping Marathon series One hour of mapping followed by pub 2008: Streets 2009: POIs 2010: Building outlines Build and keep the community Crises January 2010: Haiti earthquake High quality aerial imagery
35. Comparison Quality vs Google Maps POIs (Over)simplification of roads Completeness vs Google Maps POIs again Databases vscrowdsourcing Coverage… A few surprises – Pyongyang in OSM
38. It Started Life as a Street Map It’s not Ordnance Survey MasterMap Roads are line features, not areal Not yet, anyway OpenStreetMap is just a database Anything can go in it…
40. Loose Taxonomy Need to consider the metadata too – is it on the map, is it in the right place, is it tagged appropriately? Community proposes and documents on wiki but doesn’t enforce Worldwide considerations Metadata sometimes present for source, always present for input/update date & user
41. Loose Taxonomy Partial order ensured by Steve Chilton’s OSM Mapnikstylesheet If you want to see your data appear on the map that “most” people see, follow the guidelines
42. Curves ...or lack of... Ability to tag ways as curved Still straight lines in DB
44. CycleStreets “We'll be launching a page for Local Authorities v. soon about embedded versions of the CycleStreets planner on their sites” - CycleStreets
45. Skobbler Commercial Sat-Nav application based on OSM data with custom additions Motorway lanes for junction approaches Completeness crucial Display Google Maps on website for address finding (Google Local) but use OSM for routing and app 3D view Accuracy not so important
47. Skobbler Started in Germany The map is more complete Just launched in U.K. The map is probably good enough, though missing a lot of turn restrictions Users report bugs Skobbler fix remotely or highlight to OSM Both communities benefit
51. Attribution Various – Imagery of OpenStreetMap data which are CC-By-SA OpenStreetMap and contributors. Various – Imagery of Ordnance Survey OpenData Street View rasterswhich are Crown Copyright and Database Right 2010. p4 – Photograph of GPS unit by Steve Coast. p9 – Imagery of out-of-copyright map by Richard Fairhurst p16 – Photograph of Walking Papers in pub by Harry Wood. p27 – Maps produced by MukiHaklay. p28 – Map produced by Peter Reed. p42 –Screenshot of motorway junctions in OSM by Barry Crabtree. p43 – Photograph of ambulance by DBH NHS media services. p50 –Photograph of Boris Bike by Pocket Lint website. Other imagery may be copyright of the producers referred to on the same page.
Before UCLYear 1 Spatial Interaction Model of HE AccessPaperFlow mapsSubversion and TracEducation AtlasSchool Catchment contoursHE Access ProfilerMarr Map of ManchesterHEFCE Funding MapUK Scenic PubsFacebookMyNameProfilerWordpress MU: Geography BlogsYear 2CensusGIVThe Mock version vs the final oneChorogenElection VisualisationsUK Election, London council resultsTube VisualisationScottish Popular Edition for OSMOpenOrienteeringMapProduction use of ChorogenMinutely updates from OSMLonely BusesOS Open Data: Meridian vs OSMHistorical Maps of ManchesterNextCASAMaptubeContact MeBlog, Twitter