What got you hooked on geospatial? For me it was more than just maps – it was the ability to transform geographic data to see something new or shed light on some aspect of my environment. Whether you use GDAL, ArcGIS, GRASS or IDRISI, we have usually done this type of data transformation with a variety of desktop software tools. So why have these types of capabilities been relatively rare in web and mobile applications? Speed and scalability are two important factors. It has generally required too much time to calculate a viewshed, combine a stack of raster files into a weighted overlay, or generate slope and aspect from elevation data. Azavea has been working on this problem – fast, scalable geoprocessing – for several years. In 2012 we released a new open source project called GeoTrellis (http://geotrellis.io/), an open source framework for fast, distributed geoprocessing. GeoTrellis leverages the strong type system and functional programming style of the Scala language and the Spark and Akka frameworks. This talk will give an overview of GeoTrellis and how it can be integrated with web mapping tools to create online geoprocessing applications for stormwater modeling, education games, infrastructure prioritization, climate change, and transportation.

1. 21st Century Geoprocessing
with Scala and GeoTrellis
Robert Cheetham
cheetham@azavea.com
@azavea
@rcheetham

2. B Corporation
• Civic/Social impact
• Donate share of profits
Research-Driven
• 10% Research Program
• Academic Collaborations
• Open Source
• Open Data

3. Use geodata to
do stuff that matters

4. Land
Water
People

5. Ian McHarg

6. Dana Tomlin

7. Idrisi

8. GRASS

9. advanced
spatial analysis
on the web

10. advanced
spatial analysis
on the web

11. 3 Challenges

12. 1. Performance & Scalability

13. Big Data – Cities
2. Large Data Sets – Digital City

14. 2. Large Data Sets – Social Media

15. 2. Large Data Sets - Science

16. 3. User Interface

17. 3. User Interface

18. 3. User Interface

19. 3. User Interface

20. 3. User Interface

21. We can do better

23. • IO
• Geoprocessing Operations
• Distributed Processing
• Web Services

24. Real-time Processing

25. 6183 x 4992 4598 x 4867
118 MB 86 MB

26. Cluster-style Processing

27. 1770271 x 910139
5.8 TB

28. How does it work

29. On the shoulders of giants

30. LocationTech Community

32. Some changes coming

33. • Parallel operations across tiles
• Parallel execution of operations
• Basic cluster capabilities with
GeoTrellis v0.9:
+

34. • Sharding raster data across the cluster
• Caching operation results across cluster
• HDFS support
• Advanced Fault tolerance
• Advanced Scheduling
• ...
What's missing?
+

35. • Caches results in memory
• Ideal for iterative algorithms
• Significantly outperforms Hadoop
• Uses Hadoop's file system (HDFS)

36. +

37. What becomes possible?

38. Urban Forests

39. Urban Forests

40. Simulation Modeling

41. Sea Level Rise

42. Business Siting

43. Streaming Data

44. Counting Carbon

45. Digital Humanities

46. GeoTrellis Transit

47. Travelsheds

48. Crime Analysis and Forecasting

49. It’s the second Monday in October
and school is in session. There were 2
burglaries and 3 assaults yesterday.
The Maple Leafs are not playing this
evening. Six bars, three take-out
stores, and a high school are in the
neighborhood. The forecast is 9°C
with a 50% chance of rain this evening.
Where do you focus your 3 vehicles?

50. It’s the second Monday in October
and school is in session. There were 2
burglaries and 3 assaults yesterday.
The Maples Leafs are not playing this
evening. Six bars, three take-out stores,
and a high school are in the
neighborhood. The forecast is 9°C
with 50% chance of rain.
Where do you focus your 3 vehicles?

51. Data Science + Geography

52. Data Science + Geography

53. Faster is different…

54. Educational Games

55. New Devices and Displays

58. I am very excited

59. advanced
spatial analysis
on the web

60. advanced
spatial analysis
on the web

61. Land
Water
People

62. Simulation
Modeling
Forecasting

63. • Multi-band
• Temporal bands (climate)
• More operations
• Tile indexes
• GeoMesa collab.
• Simpler setup
• More integration points
What’s next?

65. GeoTrellis.io
Get Involved

66. geotrellis-user@googlegroups.com
Get Involved

67. IRC: #geotrellis on freenode
Get Involved

68. Use geodata to
do stuff that matters

70. [is hiring]
jobs.azavea.com
cheetham@azavea.com
@rcheetham