SlideShare ist ein Scribd-Unternehmen logo
1 von 22
SotM-US 2015
Osmose QA
Issue reported
at NYC on 2015-06-07
Frédéric Rodrigo <frederic@carte-libre.fr>
(c)left 2015 - CC-BY-SA v3.0
http://osmose.openstreetmap.fr
Osmose QA ?
Quality tool
Data analysis
Issues detection with rules
Missing data from OpenData
Integration suggestions
Osmose QA way
No duplicate analyses with other tools
No force mapping way
– Adapt to local mapping approach
– No choose one alternative among others
Frontend - Interface
1 ligne 2 ligne 3 ligne 4 ligne
0
2
4
6
8
10
12
1 colonne
2 colonne
3 colonne
Frontend
Issues map
Filter: severity, topic, fixable, items
– Responsive, filter for survey
Coverage map, heatmap
Connection Oauth via osm.org
Export issues: GPX, remote JOSM, Json, RSS…
Issues dumps: SQL, CSV
Frontend - Popup
1. Description
2. Suspects elements
3. Tags
4. Zone links
5. Buttons
6. Suggested fixs
7. Status change: corrected, false positive
Frontend – Tags Editor
Make change into OSM from Osmose QA
Edit only tags
Can apply « fix »
– A nice differential
tags editor
Backend - Frontend
Backend Frontend
OSM Data
API
Backend - Countries
Split : specifics analyser
– Languages, Country, Projection
→ allow statistics
Sublevels split : France, Italy, Belgium, USA, Canada...
70% of planet data
→ Need more servers
→ Aim self support from local community: mapping practice,
knowledge, OpenData
Backend - Analysers
Analysers
– Rules on one object tags
● Capitals, ref...
– SQL spatial rules or multi-objects check
● Reverse roundabout, unconnected highway...
– OpenData features checking
● Schools, post offices, bus stops...
More than 200 analysers
Sax analyser
class Name_Spaces(Plugin):
def init(self, logger):
self.errors[903] = { "item": 5010, "level": 2, "tag": ["name", "fix:chair"], "desc": T_(u"Too
many spaces") }
def node(self, data, tags):
name = tags[u"name"]
if u" " in name:
err.append({"class": 903, "subclass": 0, "text": T_("Name contains successive
spaces")})
if len(err) > 0:
name = re.sub(r' +', ' ', name.strip())
for e in err:
e["fix"] = {"name": name}
issue definition
issue detection
issue possible fix
Osmosis analyserSql10 = """
SELECT
id,
ST_AsText(way_locate(linestring))
FROM
ways
WHERE
tags ? 'junction' AND tags->'junction' = 'roundabout' AND
is_polygon AND ST_IsSimple(linestring) AND
{1} ST_OrderingEquals(ST_Makepolygon(linestring), st_forceRHR(ST_Makepolygon(linestring)))"""
class Analyser_Osmosis_Roundabout_Reverse(Analyser_Osmosis):
def __init__(self, config, logger = None):
Analyser_Osmosis.__init__(self, config, logger)
self.classs_change[1] = {"item":"1050", "level": 1, "tag": ["highway", "roundabout", "fix:chair"], "desc": T_(u"Reverse
roundabout") }
self.callback10 = lambda res: {"class":1, "data":[self.way_full, self.positionAsText]}
if self.config.options.get("driving_side") == "left":
self.driving_side = "NOT "
else:
self.driving_side = ""
def analyser_osmosis_all(self):
self.run(sql10.format("", self.driving_side), self.callback10)
issue detection
issue definition
OpenData analyser
class Analyser_Merge_Poste_FR(Analyser_Merge):
def __init__(self, config, logger = None):
self.missing_official = {"item":"8020", "class": 1, "level": 3, "tag": ["merge", "post"], "desc": T_(u"Post office not integrated") }
self.missing_osm = {"item":"7050", "class": 2, "level": 3, "tag": ["merge", "post"], "desc": T_(u"Post office without ref:FR:LaPoste") }
self.possible_merge = {"item":"8021", "class": 3, "level": 3, "tag": ["merge", "post"], "desc": T_(u"Post office, integration suggestion") }
Analyser_Merge.__init__(self, config, logger,
Source(
name = u"points de contact du réseau postal français",
file = "poste_FR.csv.bz2",
encoding = "ISO-8859-15",
csv = CSV(separator = ";")),
Load("Longitude", "Latitude", table = "poste_fr"),
Mapping(
select = Select(
types = ["nodes", "ways"],
tags = {"amenity": "post_office"}),
osmRef = "ref:FR:LaPoste",
conflationDistance = 1000,
generate = Generate(
static = {
"amenity": "post_office",
"operator": "La Poste",
"source": "data.gouv.fr:LaPoste - 01/2013"},
mapping = {
"ref:FR:LaPoste": "Identifiant",
"name": lambda res: re.sub(self.Principal, " Principal", res[u"Libellé du site"]),
"addr:postcode": "Code postal"} )))
issue definitions
OpenData source file description
OSM data pick-up
Join OpenData & OSM
Map OpenData columns to OSM tags
15 Millions of issues
USA
Germany
World, level=1
France, level=1
Corrections
Errors
New
Rules
USA, level=1
Germany, level=1
Stats
15 millions of issues
– 850 000 of level=1
3 700 000 executed analyses
– 12 000 analysis / 2 days
15 virtuals serveurs
500 rules checked
19,8 / 28 Go : 70 % of planet
352 area (countries or sub areas)
Maproulette
Bridge to Maproulette
Reports choose:
– [world] Highway intersecting building
– [world] Object tagged twice as node or way
– [france] Missing soccer pitch
– [world] Broken highway level continuity
→ OSM as a Marathon
Osmose
http://osmose.openstreetmap.fr
http://wiki.osm.org/Osmose
@osmose_qa

Weitere ähnliche Inhalte

Ähnlich wie 20150607 sotm-us-osmose-qa

Osmose, quality assurance tool
Osmose, quality assurance toolOsmose, quality assurance tool
Osmose, quality assurance toolFrédéric Rodrigo
 
Building an ML Platform with Ray and MLflow
Building an ML Platform with Ray and MLflowBuilding an ML Platform with Ray and MLflow
Building an ML Platform with Ray and MLflowDatabricks
 
Nodejs性能分析优化和分布式设计探讨
Nodejs性能分析优化和分布式设计探讨Nodejs性能分析优化和分布式设计探讨
Nodejs性能分析优化和分布式设计探讨flyinweb
 
Megamodeling of Complex, Distributed, Heterogeneous CPS Systems
Megamodeling of Complex, Distributed, Heterogeneous CPS SystemsMegamodeling of Complex, Distributed, Heterogeneous CPS Systems
Megamodeling of Complex, Distributed, Heterogeneous CPS SystemsEugenio Villar
 
Lab 2: Classification and Regression Prediction Models, training and testing ...
Lab 2: Classification and Regression Prediction Models, training and testing ...Lab 2: Classification and Regression Prediction Models, training and testing ...
Lab 2: Classification and Regression Prediction Models, training and testing ...Yao Yao
 
【論文紹介】Relay: A New IR for Machine Learning Frameworks
【論文紹介】Relay: A New IR for Machine Learning Frameworks【論文紹介】Relay: A New IR for Machine Learning Frameworks
【論文紹介】Relay: A New IR for Machine Learning FrameworksTakeo Imai
 
Intravert Server side processing for Cassandra
Intravert Server side processing for CassandraIntravert Server side processing for Cassandra
Intravert Server side processing for CassandraEdward Capriolo
 
NYC* 2013 - "Advanced Data Processing: Beyond Queries and Slices"
NYC* 2013 - "Advanced Data Processing: Beyond Queries and Slices"NYC* 2013 - "Advanced Data Processing: Beyond Queries and Slices"
NYC* 2013 - "Advanced Data Processing: Beyond Queries and Slices"DataStax Academy
 
What's Coming Next in Sencha Frameworks
What's Coming Next in Sencha FrameworksWhat's Coming Next in Sencha Frameworks
What's Coming Next in Sencha FrameworksGrgur Grisogono
 
Hierarchical free monads and software design in fp
Hierarchical free monads and software design in fpHierarchical free monads and software design in fp
Hierarchical free monads and software design in fpAlexander Granin
 
Large Scale Log Analytics with Solr: Presented by Rafał Kuć & Radu Gheorghe, ...
Large Scale Log Analytics with Solr: Presented by Rafał Kuć & Radu Gheorghe, ...Large Scale Log Analytics with Solr: Presented by Rafał Kuć & Radu Gheorghe, ...
Large Scale Log Analytics with Solr: Presented by Rafał Kuć & Radu Gheorghe, ...Lucidworks
 
Adventures in civic hacking
Adventures in civic hackingAdventures in civic hacking
Adventures in civic hackingosfameron
 
"Scala in Goozy", Alexey Zlobin
"Scala in Goozy", Alexey Zlobin "Scala in Goozy", Alexey Zlobin
"Scala in Goozy", Alexey Zlobin Vasil Remeniuk
 
More Than po: Debugging in LLDB @ CocoaConf SJ 2015
More Than po: Debugging in LLDB @ CocoaConf SJ 2015More Than po: Debugging in LLDB @ CocoaConf SJ 2015
More Than po: Debugging in LLDB @ CocoaConf SJ 2015Michele Titolo
 
Large Scale Log Analytics with Solr (from Lucene Revolution 2015)
Large Scale Log Analytics with Solr (from Lucene Revolution 2015)Large Scale Log Analytics with Solr (from Lucene Revolution 2015)
Large Scale Log Analytics with Solr (from Lucene Revolution 2015)Sematext Group, Inc.
 
SVCC 2013 D3.js Presentation (10/05/2013)
SVCC 2013 D3.js Presentation (10/05/2013)SVCC 2013 D3.js Presentation (10/05/2013)
SVCC 2013 D3.js Presentation (10/05/2013)Oswald Campesato
 
Eclipse Con 2015: Codan - a C/C++ Code Analysis Framework for CDT
Eclipse Con 2015: Codan - a C/C++ Code Analysis Framework for CDTEclipse Con 2015: Codan - a C/C++ Code Analysis Framework for CDT
Eclipse Con 2015: Codan - a C/C++ Code Analysis Framework for CDTElena Laskavaia
 

Ähnlich wie 20150607 sotm-us-osmose-qa (20)

Osmose, quality assurance tool
Osmose, quality assurance toolOsmose, quality assurance tool
Osmose, quality assurance tool
 
huhu
huhuhuhu
huhu
 
Building an ML Platform with Ray and MLflow
Building an ML Platform with Ray and MLflowBuilding an ML Platform with Ray and MLflow
Building an ML Platform with Ray and MLflow
 
Nodejs性能分析优化和分布式设计探讨
Nodejs性能分析优化和分布式设计探讨Nodejs性能分析优化和分布式设计探讨
Nodejs性能分析优化和分布式设计探讨
 
Megamodeling of Complex, Distributed, Heterogeneous CPS Systems
Megamodeling of Complex, Distributed, Heterogeneous CPS SystemsMegamodeling of Complex, Distributed, Heterogeneous CPS Systems
Megamodeling of Complex, Distributed, Heterogeneous CPS Systems
 
Lab 2: Classification and Regression Prediction Models, training and testing ...
Lab 2: Classification and Regression Prediction Models, training and testing ...Lab 2: Classification and Regression Prediction Models, training and testing ...
Lab 2: Classification and Regression Prediction Models, training and testing ...
 
【論文紹介】Relay: A New IR for Machine Learning Frameworks
【論文紹介】Relay: A New IR for Machine Learning Frameworks【論文紹介】Relay: A New IR for Machine Learning Frameworks
【論文紹介】Relay: A New IR for Machine Learning Frameworks
 
Intravert Server side processing for Cassandra
Intravert Server side processing for CassandraIntravert Server side processing for Cassandra
Intravert Server side processing for Cassandra
 
NYC* 2013 - "Advanced Data Processing: Beyond Queries and Slices"
NYC* 2013 - "Advanced Data Processing: Beyond Queries and Slices"NYC* 2013 - "Advanced Data Processing: Beyond Queries and Slices"
NYC* 2013 - "Advanced Data Processing: Beyond Queries and Slices"
 
What's Coming Next in Sencha Frameworks
What's Coming Next in Sencha FrameworksWhat's Coming Next in Sencha Frameworks
What's Coming Next in Sencha Frameworks
 
Hierarchical free monads and software design in fp
Hierarchical free monads and software design in fpHierarchical free monads and software design in fp
Hierarchical free monads and software design in fp
 
Large Scale Log Analytics with Solr: Presented by Rafał Kuć & Radu Gheorghe, ...
Large Scale Log Analytics with Solr: Presented by Rafał Kuć & Radu Gheorghe, ...Large Scale Log Analytics with Solr: Presented by Rafał Kuć & Radu Gheorghe, ...
Large Scale Log Analytics with Solr: Presented by Rafał Kuć & Radu Gheorghe, ...
 
Adventures in civic hacking
Adventures in civic hackingAdventures in civic hacking
Adventures in civic hacking
 
"Scala in Goozy", Alexey Zlobin
"Scala in Goozy", Alexey Zlobin "Scala in Goozy", Alexey Zlobin
"Scala in Goozy", Alexey Zlobin
 
More Than po: Debugging in LLDB @ CocoaConf SJ 2015
More Than po: Debugging in LLDB @ CocoaConf SJ 2015More Than po: Debugging in LLDB @ CocoaConf SJ 2015
More Than po: Debugging in LLDB @ CocoaConf SJ 2015
 
AngularJS and SPA
AngularJS and SPAAngularJS and SPA
AngularJS and SPA
 
Large Scale Log Analytics with Solr (from Lucene Revolution 2015)
Large Scale Log Analytics with Solr (from Lucene Revolution 2015)Large Scale Log Analytics with Solr (from Lucene Revolution 2015)
Large Scale Log Analytics with Solr (from Lucene Revolution 2015)
 
Svcc 2013-d3
Svcc 2013-d3Svcc 2013-d3
Svcc 2013-d3
 
SVCC 2013 D3.js Presentation (10/05/2013)
SVCC 2013 D3.js Presentation (10/05/2013)SVCC 2013 D3.js Presentation (10/05/2013)
SVCC 2013 D3.js Presentation (10/05/2013)
 
Eclipse Con 2015: Codan - a C/C++ Code Analysis Framework for CDT
Eclipse Con 2015: Codan - a C/C++ Code Analysis Framework for CDTEclipse Con 2015: Codan - a C/C++ Code Analysis Framework for CDT
Eclipse Con 2015: Codan - a C/C++ Code Analysis Framework for CDT
 

Mehr von Frédéric Rodrigo

Cartography: Vector Tiles From an Open Initiative To an Industry Standard
Cartography: Vector Tiles  From an Open Initiative To an Industry StandardCartography: Vector Tiles  From an Open Initiative To an Industry Standard
Cartography: Vector Tiles From an Open Initiative To an Industry StandardFrédéric Rodrigo
 
Osmose-QA, OpenData, Mapillary and MapCSS
Osmose-QA, OpenData, Mapillary and MapCSSOsmose-QA, OpenData, Mapillary and MapCSS
Osmose-QA, OpenData, Mapillary and MapCSSFrédéric Rodrigo
 
Automatic Enhancement of Pedestrian Route using extracted Landmarks from OSM
Automatic Enhancement of Pedestrian Route using extracted Landmarks from OSMAutomatic Enhancement of Pedestrian Route using extracted Landmarks from OSM
Automatic Enhancement of Pedestrian Route using extracted Landmarks from OSMFrédéric Rodrigo
 
Annoter automatiquement un itinéraire piéton avec des repères issus d'OSM
Annoter automatiquement un itinéraire piéton avec des repères issus d'OSMAnnoter automatiquement un itinéraire piéton avec des repères issus d'OSM
Annoter automatiquement un itinéraire piéton avec des repères issus d'OSMFrédéric Rodrigo
 
Osmose-QA, Qualité et intégration de données
Osmose-QA, Qualité et intégration de donnéesOsmose-QA, Qualité et intégration de données
Osmose-QA, Qualité et intégration de donnéesFrédéric Rodrigo
 
OSRM - Open Source Routing Machine
OSRM - Open Source Routing MachineOSRM - Open Source Routing Machine
OSRM - Open Source Routing MachineFrédéric Rodrigo
 
Addok, BAN et BANO dans un bateau
Addok, BAN et BANO dans un bateauAddok, BAN et BANO dans un bateau
Addok, BAN et BANO dans un bateauFrédéric Rodrigo
 
Osmose : la conquête du monde
Osmose : la conquête du mondeOsmose : la conquête du monde
Osmose : la conquête du mondeFrédéric Rodrigo
 
5/5 Osm 20141118-l2.3-réutilisation
5/5 Osm 20141118-l2.3-réutilisation5/5 Osm 20141118-l2.3-réutilisation
5/5 Osm 20141118-l2.3-réutilisationFrédéric Rodrigo
 
4/5 Osm 20141118-l2.2-collecte et contribution
4/5 Osm 20141118-l2.2-collecte et contribution4/5 Osm 20141118-l2.2-collecte et contribution
4/5 Osm 20141118-l2.2-collecte et contributionFrédéric Rodrigo
 
3/5 Osm 20141118-l2.1-être à l'aise avec open streetmap
3/5 Osm 20141118-l2.1-être à l'aise avec open streetmap3/5 Osm 20141118-l2.1-être à l'aise avec open streetmap
3/5 Osm 20141118-l2.1-être à l'aise avec open streetmapFrédéric Rodrigo
 
2/5 Osm 20141106-l1.2-initiation à la contribution
2/5 Osm 20141106-l1.2-initiation à la contribution2/5 Osm 20141106-l1.2-initiation à la contribution
2/5 Osm 20141106-l1.2-initiation à la contributionFrédéric Rodrigo
 
1/5 Osm 20141106-l1.1-présentation
1/5 Osm 20141106-l1.1-présentation1/5 Osm 20141106-l1.1-présentation
1/5 Osm 20141106-l1.1-présentationFrédéric Rodrigo
 

Mehr von Frédéric Rodrigo (20)

OSXP 2021 - MAKINA MAPS
OSXP 2021 - MAKINA MAPSOSXP 2021 - MAKINA MAPS
OSXP 2021 - MAKINA MAPS
 
Cartography: Vector Tiles From an Open Initiative To an Industry Standard
Cartography: Vector Tiles  From an Open Initiative To an Industry StandardCartography: Vector Tiles  From an Open Initiative To an Industry Standard
Cartography: Vector Tiles From an Open Initiative To an Industry Standard
 
Osmose-QA, OpenData, Mapillary and MapCSS
Osmose-QA, OpenData, Mapillary and MapCSSOsmose-QA, OpenData, Mapillary and MapCSS
Osmose-QA, OpenData, Mapillary and MapCSS
 
Automatic Enhancement of Pedestrian Route using extracted Landmarks from OSM
Automatic Enhancement of Pedestrian Route using extracted Landmarks from OSMAutomatic Enhancement of Pedestrian Route using extracted Landmarks from OSM
Automatic Enhancement of Pedestrian Route using extracted Landmarks from OSM
 
Annoter automatiquement un itinéraire piéton avec des repères issus d'OSM
Annoter automatiquement un itinéraire piéton avec des repères issus d'OSMAnnoter automatiquement un itinéraire piéton avec des repères issus d'OSM
Annoter automatiquement un itinéraire piéton avec des repères issus d'OSM
 
Osmose-QA, Qualité et intégration de données
Osmose-QA, Qualité et intégration de donnéesOsmose-QA, Qualité et intégration de données
Osmose-QA, Qualité et intégration de données
 
OSRM, Utilisation avancée
OSRM, Utilisation avancéeOSRM, Utilisation avancée
OSRM, Utilisation avancée
 
Open Traffic
Open TrafficOpen Traffic
Open Traffic
 
Osmose-QA
Osmose-QAOsmose-QA
Osmose-QA
 
OSRM L'état du routage
OSRM L'état du routageOSRM L'état du routage
OSRM L'état du routage
 
OSRM - Open Source Routing Machine
OSRM - Open Source Routing MachineOSRM - Open Source Routing Machine
OSRM - Open Source Routing Machine
 
Addok, BAN et BANO dans un bateau
Addok, BAN et BANO dans un bateauAddok, BAN et BANO dans un bateau
Addok, BAN et BANO dans un bateau
 
Mapotempo
MapotempoMapotempo
Mapotempo
 
Osmose : la conquête du monde
Osmose : la conquête du mondeOsmose : la conquête du monde
Osmose : la conquête du monde
 
5/5 Osm 20141118-l2.3-réutilisation
5/5 Osm 20141118-l2.3-réutilisation5/5 Osm 20141118-l2.3-réutilisation
5/5 Osm 20141118-l2.3-réutilisation
 
4/5 Osm 20141118-l2.2-collecte et contribution
4/5 Osm 20141118-l2.2-collecte et contribution4/5 Osm 20141118-l2.2-collecte et contribution
4/5 Osm 20141118-l2.2-collecte et contribution
 
3/5 Osm 20141118-l2.1-être à l'aise avec open streetmap
3/5 Osm 20141118-l2.1-être à l'aise avec open streetmap3/5 Osm 20141118-l2.1-être à l'aise avec open streetmap
3/5 Osm 20141118-l2.1-être à l'aise avec open streetmap
 
2/5 Osm 20141106-l1.2-initiation à la contribution
2/5 Osm 20141106-l1.2-initiation à la contribution2/5 Osm 20141106-l1.2-initiation à la contribution
2/5 Osm 20141106-l1.2-initiation à la contribution
 
1/5 Osm 20141106-l1.1-présentation
1/5 Osm 20141106-l1.1-présentation1/5 Osm 20141106-l1.1-présentation
1/5 Osm 20141106-l1.1-présentation
 
Osm 20141112-montesquieu
Osm 20141112-montesquieuOsm 20141112-montesquieu
Osm 20141112-montesquieu
 

Kürzlich hochgeladen

Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingEdi Saputra
 
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024The Digital Insurer
 
GenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdfGenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdflior mazor
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FMESafe Software
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfsudhanshuwaghmare1
 
Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024The Digital Insurer
 
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)wesley chun
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...Martijn de Jong
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...DianaGray10
 
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MIND CTI
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobeapidays
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationSafe Software
 
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityPrincipled Technologies
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024The Digital Insurer
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProduct Anonymous
 
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyKhushali Kathiriya
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...apidays
 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAndrey Devyatkin
 
Real Time Object Detection Using Open CV
Real Time Object Detection Using Open CVReal Time Object Detection Using Open CV
Real Time Object Detection Using Open CVKhem
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘RTylerCroy
 

Kürzlich hochgeladen (20)

Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
 
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
 
GenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdfGenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdf
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024
 
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
 
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivity
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
 
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : Uncertainty
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of Terraform
 
Real Time Object Detection Using Open CV
Real Time Object Detection Using Open CVReal Time Object Detection Using Open CV
Real Time Object Detection Using Open CV
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘
 

20150607 sotm-us-osmose-qa

  • 1. SotM-US 2015 Osmose QA Issue reported at NYC on 2015-06-07 Frédéric Rodrigo <frederic@carte-libre.fr> (c)left 2015 - CC-BY-SA v3.0 http://osmose.openstreetmap.fr
  • 2. Osmose QA ? Quality tool Data analysis Issues detection with rules Missing data from OpenData Integration suggestions
  • 3. Osmose QA way No duplicate analyses with other tools No force mapping way – Adapt to local mapping approach – No choose one alternative among others
  • 4. Frontend - Interface 1 ligne 2 ligne 3 ligne 4 ligne 0 2 4 6 8 10 12 1 colonne 2 colonne 3 colonne
  • 5. Frontend Issues map Filter: severity, topic, fixable, items – Responsive, filter for survey Coverage map, heatmap Connection Oauth via osm.org Export issues: GPX, remote JOSM, Json, RSS… Issues dumps: SQL, CSV
  • 6. Frontend - Popup 1. Description 2. Suspects elements 3. Tags 4. Zone links 5. Buttons 6. Suggested fixs 7. Status change: corrected, false positive
  • 7. Frontend – Tags Editor Make change into OSM from Osmose QA Edit only tags Can apply « fix » – A nice differential tags editor
  • 8.
  • 9. Backend - Frontend Backend Frontend OSM Data API
  • 10. Backend - Countries Split : specifics analyser – Languages, Country, Projection → allow statistics Sublevels split : France, Italy, Belgium, USA, Canada... 70% of planet data → Need more servers → Aim self support from local community: mapping practice, knowledge, OpenData
  • 11. Backend - Analysers Analysers – Rules on one object tags ● Capitals, ref... – SQL spatial rules or multi-objects check ● Reverse roundabout, unconnected highway... – OpenData features checking ● Schools, post offices, bus stops... More than 200 analysers
  • 12. Sax analyser class Name_Spaces(Plugin): def init(self, logger): self.errors[903] = { "item": 5010, "level": 2, "tag": ["name", "fix:chair"], "desc": T_(u"Too many spaces") } def node(self, data, tags): name = tags[u"name"] if u" " in name: err.append({"class": 903, "subclass": 0, "text": T_("Name contains successive spaces")}) if len(err) > 0: name = re.sub(r' +', ' ', name.strip()) for e in err: e["fix"] = {"name": name} issue definition issue detection issue possible fix
  • 13. Osmosis analyserSql10 = """ SELECT id, ST_AsText(way_locate(linestring)) FROM ways WHERE tags ? 'junction' AND tags->'junction' = 'roundabout' AND is_polygon AND ST_IsSimple(linestring) AND {1} ST_OrderingEquals(ST_Makepolygon(linestring), st_forceRHR(ST_Makepolygon(linestring)))""" class Analyser_Osmosis_Roundabout_Reverse(Analyser_Osmosis): def __init__(self, config, logger = None): Analyser_Osmosis.__init__(self, config, logger) self.classs_change[1] = {"item":"1050", "level": 1, "tag": ["highway", "roundabout", "fix:chair"], "desc": T_(u"Reverse roundabout") } self.callback10 = lambda res: {"class":1, "data":[self.way_full, self.positionAsText]} if self.config.options.get("driving_side") == "left": self.driving_side = "NOT " else: self.driving_side = "" def analyser_osmosis_all(self): self.run(sql10.format("", self.driving_side), self.callback10) issue detection issue definition
  • 14. OpenData analyser class Analyser_Merge_Poste_FR(Analyser_Merge): def __init__(self, config, logger = None): self.missing_official = {"item":"8020", "class": 1, "level": 3, "tag": ["merge", "post"], "desc": T_(u"Post office not integrated") } self.missing_osm = {"item":"7050", "class": 2, "level": 3, "tag": ["merge", "post"], "desc": T_(u"Post office without ref:FR:LaPoste") } self.possible_merge = {"item":"8021", "class": 3, "level": 3, "tag": ["merge", "post"], "desc": T_(u"Post office, integration suggestion") } Analyser_Merge.__init__(self, config, logger, Source( name = u"points de contact du réseau postal français", file = "poste_FR.csv.bz2", encoding = "ISO-8859-15", csv = CSV(separator = ";")), Load("Longitude", "Latitude", table = "poste_fr"), Mapping( select = Select( types = ["nodes", "ways"], tags = {"amenity": "post_office"}), osmRef = "ref:FR:LaPoste", conflationDistance = 1000, generate = Generate( static = { "amenity": "post_office", "operator": "La Poste", "source": "data.gouv.fr:LaPoste - 01/2013"}, mapping = { "ref:FR:LaPoste": "Identifiant", "name": lambda res: re.sub(self.Principal, " Principal", res[u"Libellé du site"]), "addr:postcode": "Code postal"} ))) issue definitions OpenData source file description OSM data pick-up Join OpenData & OSM Map OpenData columns to OSM tags
  • 15. 15 Millions of issues USA Germany
  • 20. Stats 15 millions of issues – 850 000 of level=1 3 700 000 executed analyses – 12 000 analysis / 2 days 15 virtuals serveurs 500 rules checked 19,8 / 28 Go : 70 % of planet 352 area (countries or sub areas)
  • 21. Maproulette Bridge to Maproulette Reports choose: – [world] Highway intersecting building – [world] Object tagged twice as node or way – [france] Missing soccer pitch – [world] Broken highway level continuity → OSM as a Marathon

Hinweis der Redaktion

  1. http://osmose.openstreetmap.fr/fr/errors/graph.png?item=1070