Training Series - Build A Routing Web Application With OpenStreetMap, Neo4j, and Leaflet.js
Build A Routing Web App
With Neo4j, OpenStreetMap, & Leaflet.js
William Lyon
Neo4j
@lyonwj
lyonwj.com
dev.neo4j.com/routing-workshop
Building A Routing Web Application With Neo4j, OpenStreetMap, and Leaflet.js
Learn how to work with geospatial data in the Neo4j graph database in this hands-on workshop. We’ll start by
covering the spatial types and functions available in the Cypher query language and Neo4j then move on to spatial
search operations and routing with graph algorithms. We’ll import data from OpenStreetMap and see how to use the
Neo4j JavaScript driver to build a navigation web application to find routes between points of interest and addresses.
Build A Routing Web App W/ OpenStreetMap, Neo4j, & Leaflet.js
● Address / point of interest data
● Auto-complete search
● Plot shortest route between
locations
Build A Routing Web App With Neo4j, OSM, & Leaflet.js
Agenda
Modules:
● Intro to Neo4j & Spatial Cypher
● Working With OpenStreetMap Data
● Building Web Maps With Leaflet.js
Hands-On exercises using:
● Neo4j AuraDB Free Tier
● Python notebook
○ Local Python environment
○ GitHub Codespaces
○ Google Colab
● JavaScript / HTML
○ Local web browser
○ GitHub Codespaces
Hands-On
Exercise
Slides: dev.neo4j.com/routing-workshop
GitHub repo: dev.neo4j.com/routing-workshop-github
Build A Routing Web App With Neo4j, OSM, & Leaflet.js
Pre-requisites
● Either a local Python development environment or a GitHub account to use
GitHub Codespaces (recommended) or Google Colab
● Neo4j AuraDB (Free Tier)
○ Sign in and create free account: dev.neo4j.com/aura
Slides: dev.neo4j.com/routing-workshop
GitHub repo: dev.neo4j.com/routing-workshop-github
Intro to Neo4j & Spatial Cypher - Routing With Neo4j
● Slides / demo with airports
● set up Aura Free instance
● show we're going to build
● demo bloom (?)
Points, Lines, Polygons, & Graphs
● Spatial Cypher functions
○ Point, distance, index
● Spatial search with Neo4j
● Routing
● QGIS + Neo4j
● Combining datasets
Tobler's First Law Of Geography
http://southwesterngis.blogspot.com/2014/02/toblers-first-law-of-geography.html
Tobler's First Law Of Geography
Minneapolis
San Jose
Santa Cruz
San Mateo
San Francisco
What Is A Knowledge Graph? Knowledge graphs put things in context
"Things, not strings" https://blog.google/products/search/introducing-knowledge-graph-things-not/
Graph Database
● Database management system (DBMS)
● Property Graph data model
● Cypher query language
● Graph analytics
● Data visualization
● Neo4j Aura database-as-a-service
● Native spatial types / index
What is Neo4j?
neo4j.com
News As A Knowledge Graph
Using The NYTimes API
Requirements:
● View articles
○ Most recent
● Search articles
○ Keyword, date range
● Comments
● Save articles
● Personalized recommendations
○ My interests
○ Location-based
Data from NYTimes API: developer.nytimes.com
Code: github.com/johnymontana/news-graph
Spectrum Of Graph Access Patterns
"Local" Graph Traversals
Transactional use cases
Operational workloads
"Global" Graph Traversals
Analytics use cases
Graph Data Science
Graph Algorithms
Graph Local Graph Global
Overview of Graph Algorithms
● Path finding
○ Routing
○ A*, Breadth/Depth First Search
● Centrality
○ Most important nodes in the graph
○ PageRank, Closeness,
Betweenness, Degree
● Community detection /
clustering
○ Group nodes, strength of clusters
○ Louvain, Label propagation
● Node Similarity
● Machine learning
○ Link prediction
○ Node Embeddings
○ Node Classification
https://neo4j.com/developer/graph-data-science/graph-algorithms/
The Point type in Neo4j
● Support for 2D or 3D geographic (WGS84) or cartesian coordinate
reference system (CRS)
https://neo4j.com/docs/cypher-manual/current/syntax/spatial/
RETURN point(
{latitude:49.38713, longitude:12.12711}
)
╒════════════════════════════════════════════════╕
│"point({latitude:49.38713, longitude:12.12711})"│
╞════════════════════════════════════════════════╡
│point({srid:4326, x:12.12711, y:49.38713}) │
└────────────────────────────────────────────────┘
The Point type in Neo4j
● Points can be stored as properties on nodes and relationships
https://neo4j.com/docs/cypher-manual/current/syntax/spatial/
CREATE (p:PointOfInterest)
SET p.name = "West Side Bakery",
p.location = point(
{latitude:49.38713, longitude:12.12711}
)
RETURN p
Spatial Functions In Cypher ● point.distance()
● point.withinBBox()
● point()
https://neo4j.com/docs/cypher-manual/current/functions/spatial/
MATCH (p:PointOfInterest)
WHERE point.distance(
p.location, point({latitude:49.38713, longitude:12.12711})
) < 200
RETURN p
News As A Knowledge Graph
Using The NYTimes API
Requirements:
● View articles
○ Most recent
● Search articles
○ Keyword, date range
● Comments
● Save articles
● Personalized recommendations
○ My interests
○ Location-based
Data from NYTimes API: developer.nytimes.com
Code: github.com/johnymontana/news-graph
Geocoding With APOC
CALL apoc.spatial.geocode('Union House, London') YIELD location
{
"latitude": 51.503811999999996,
"description": "Union House, 182-194, Union Street, Bankside, Southwark, London
Borough of Southwark, London, Greater London, England, SE1 8LB, United Kingdom",
"longitude": -0.10101799277699847
}
https://neo4j.com/labs/apoc/4.1/overview/apoc.spatial/apoc.spatial.geocode/
Geocoding The News Graph With APOC & flat-graph GitHub Action
https://github.com/johnymontana/news-graph/blob/main/.github/workflows/flat.yml
OSM Points Of Interest In Neo4j
https://github.com/johnymontana/daylight-earth-graph/blob/main/POI_import.ipynb
Why We Use Indexes In Neo4j
● Find a starting point for a traversal
● index free adjacency
Create Point Index For Fast Search
CREATE POINT INDEX poiIndex
FOR (p:Point) ON (p.location)
https://neo4j.com/docs/cypher-manual/current/indexes-for-search-performance/
NodeIndexSeek vs NodeByLabelScan
PROFILE MATCH (p:Point)
WHERE
point.distance(p.location, point({latitude: 37.563434, longitude:-122.322255})) < 200
RETURN p
Indexes help us efficiently
find the starting point for a
traversal
49565697 total db hits in 3649 ms
213 total db hits in 1 ms
Radius Distance Search (with index!)
PROFILE MATCH (p:Point)
WHERE
point.distance(p.location, point({latitude: 37.563434, longitude:-122.322255})) < 200
RETURN p{.*} AS point
https://neo4j.com/docs/cypher-manual/current/functions/spatial/#functions-distance
Cypher Bounding Box Search
MATCH (p:Point)
WHERE point.withinBBox(
p.location,
point({longitude:-122.325447, latitude: 37.55948 }), // Lower left point
point({longitude:-122.314675 , latitude: 37.563596})) // Upper right point
RETURN p
https://neo4j.com/docs/cypher-manual/current/functions/spatial/#functions-withinBBox
Cypher Bounding Box Search - Using Index
PROFILE
MATCH (p:Point)
WHERE point.withinBBox(
p.location,
point({longitude:-122.325447, latitude: 37.55948 }),
point({longitude:-122.314675 , latitude: 37.563596}))
RETURN p
https://neo4j.com/docs/cypher-manual/current/functions/spatial/#functions-withinBBox
Point In Polygon Search With Index
1) Convert polygon to bounding box
2) Cypher withinBBox query
3) Filter results to within polygon
https://github.com/johnymontana/geospatial-graph-demos/blob/main/src/index.html#L175-L231
Working With Line Geometries Strava Data
// Create Activity Nodes
LOAD CSV WITH HEADERS FROM "file:///activities.csv" AS row
MERGE (a:Activity {activity_id: row.`Activity ID`})
SET a.filename = row.Filename,
a += row
// Parse geojson geometries and create Geometry:Line nodes
WITH a WHERE a.filename IS NOT NULL
MERGE (n:Geometry {geom_id:a.activity_id })
MERGE (n)<-[:HAS_FEATURE]-(a)
WITH n,a
CALL
apoc.load.json('file:///' + a.filename)
YIELD value
UNWIND value.features[0].geometry.coordinates AS coord
// FIXME: handle multiple features!
WITH n,
collect(point({latitude: coord[1], longitude: coord[0]})) AS coords
SET n.coordinates = coords
SET n:Line
Working With Line Geometries Spatial Search
WITH point({latitude: $latitude, longitude: $longitude}) AS radiusCenter
MATCH (g:Geometry)<-[:HAS_FEATURE]-(a:Activity)
WHERE any(
p IN g.coordinates WHERE point.distance(p, radiusCenter) < $radius
)
RETURN *
Neo4j Aura Free Tier Setup
Let's create a Neo4j Aura Free instance that we'll use for the rest of the workshop...
Hands-On
Exercise
Once your Neo4j Aura instance is online you'll see the connection string
(neo4j+s://xxxxx.databases.neo4j.io)
Be sure to take note of the generated
password!
It will then take a few moments for your
Neo4j Aura instance to be provisioned.
Open neo4j.com/aura
and sign in
Select "Create a new
database" button.
Choose the "Free" tier.
Enter a name for your
Neo4j Aura instance
and select "Create
database"
Step 1: Step 2:
Step 3:
55
OpenStreetMap
openstreetmap.org
● Open map data of the world
● Crowd-sourced by the community -
individuals and corporate contributors
● Map tiles, geocoding, vector data, data
downloads & access via API
56
Working With OpenStreetMap Data In Neo4j
learn.opengeoedu.de
● Nodes - point features
○ POIs or part of more complex geometry
● Ways - line features (connect nodes)
○ roads, buildings, parks, etc
● Tags - attribute data describing a feature
○ used on nodes, ways, & relations
● Relations - organize multiple ways (or nodes)
into a larger entity
○ multiple road segments that make up a
bus route, multipolygon geometries
https://labs.mapbox.com/mapping/osm-data-model/
https://wiki.openstreetmap.org/wiki/Elements
58
Working With OpenStreetMap Data In Neo4j
dev.neo4j.com/sandbox
● Example OSM dataset of Central
Park in New York City
● Available in Neo4j Sandbox
● Based on neo4j-contrib/osm
● Travel guide example app:
github.com/johnymontana/central-
perk
github.com/neo4j-graph-examples/openstreetmap
Simplified OSM Routing Graph
● What if we only model intersection nodes in the graph?
● Reduces data size and simplifies our queries
OpenStreetMap
● What is OpenStreetMap?
● The OSM data model
● How to get OSM data
○ overpass, QGIS plugin, neo4j osm importer, OSMNx
OpenStreetMap Import Notebook
Let's get our Python environment up and running and start importing OpenStreetMap data!
Hands-On
Exercise
Step 1:
● Open notebooks/00-setup.ipynb
● Replace database credentials with your Neo4j AuraDB credentials
● Run all cells → you should see a DataFrame with node_count of 0
Step 2:
● Open notebooks/01-import.ipynb
● Replace database credentials with your Neo4j AuraDB credentials
● Run all cells up to "Exploring The Road Network In Neo4j" to import
OpenStreetMap data into your Neo4j AuraDB instance
Visualizing Road Networks With Neo4j Bloom
● In Neo4j AuraDB Workspace, switch to the "Explore" tab
● Load the road network into the "scene"
● Switch to coordinate layout
● Style the visualization
Hands-On
Exercise
Let's explore our road network visually!
Leaflet.js + Neo4j
Let's create a Neo4j Aura Free instance that we'll use for the rest of the workshop...
Hands-On
Exercise
● Open web/address_routing.html
● Update with your Neo4j AuraDB credentials
● Open in web browser, you may need to run a web server:
python -m http.server
● Search for routes between two addresses
● What path-finding algorithm are we using? Can we improve using the A* algorithm?