UrbanGames is a project that uses location-based games and human computation to collect and verify information about cities from citizens. The project developed Urbanopoly, a game where players collect and validate data about venues in their neighborhoods to build their property portfolio. As players contribute data through gameplay and mini-games, a weighted voting system consolidates the information to high confidence levels for publication. An evaluation found Urbanopoly was enjoyable for players and effective at gathering data, with a precision of 92%. The project aims to engage citizens in improving smart city data through entertaining and purposeful games.

1. UrbanGames
Irene Celino, Dario Cerizza, Simone Contessa,
Marta Corubolo, Daniele Dell'Aglio,
Emanuele Della Valle, Stefano Fumeo, Federico Piccinini
December 12th, 2012, Luxembourg

2. UrbanGames - Motivation
citizens as sensors,
check-in logging,
Urban Computing and mobile apps
Location-based Services
Urban
Games
Linked Data and Games with a
Semantic Web Purpose and
Crowdsourcing
open/gov data,
structured data, collecting data, cleaning
social networks, data, engaging the user,
tourism data and supporting the user while
recommendations entertaining him/her
UrbanGames in PlanetData - 12/12/12 Slide 2 of 25

3. UrbanGames - Approach
• Urban Games:
• to consume, create and assess the quality of
Smart Cities-related Linked Data
• via a Human Computation approach
• for users in mobility with smart phone devices
• Traditional Human Computation approaches are
based on users' domain knowledge…
…while Urban Games are based on and aim at
exploiting "on site" users' experience knowledge
UrbanGames in PlanetData - 12/12/12 Slide 3 of 25

4. UrbanGames - Execution
• State-of-the-art analysis
• Concept generation and
selection
• Urbanopoly prototype
implementation
• Evaluation results and outlook
UrbanGames in PlanetData - 12/12/12 Slide 4 of 25

5. UrbanGames work plan
(WP9 – WP10 – WP11)
12 14 16 18 20 22 24
Task 9.1 Concept D9.1 UrbanGames
and Design CEFRIEL Concept and design
Task 9.2 D9.2 UrbanGames
Prototype and
Implementation CEFRIEL evaluation
and Evaluation
Task 10 UrbanGames
dissemination and CEFRIEL Deliverable number
D10.1 UrbanGames
exploitation analysis and title
Dissemination,
exploitation and
Task 11 UrbanGames management report
activity management
CEFRIEL
UrbanGames in PlanetData - 12/12/12 Slide 5 of x

6. Human Computation
Game with a Purpose
Purpose
within
the game:
Purpose outside the game:
helping image search engines with manual tagging
UrbanGames in PlanetData - 12/12/12 Slide 6 of 25

7. Urbanopoly as a
Game with a Purpose
Purpose within Create your venues' portfolio and become
the game: the greatest landlord ever!
Purpose outside Collect and verify information about your city
the game: by playing with the neighborhood around you
http://bit.ly/urbanopoly
UrbanGames in PlanetData - 12/12/12 Slide 7 of 25

8. Urbanopoly – high-level view
Game purpose: check and correct geo-spatial data
from pre-existing sources + collect missing data
game to buy / sell 2
LinkedGeoData + venues with missions
players
Lombardia Open Data
data about
1 venues as
missions
bootstrap of
"venues" data
GWAP approach to
consolidate data 3
verified / improved data
+ new data
4
UrbanGames in PlanetData - 12/12/12 Slide 8 of 25

9. Urbanopoly Input Data
 OpenStreetMap (OSM)
http://www.openstreetmap.org/
via LinkedGeoData (LGD)
http://linkedgeodata.org/
data as linked data, described by an ontology
 Lombardia Open Data
https://dati.lombardia.it
data about "agriturismo" places as CSV converted to RDF
 Urbanopoly data bootstrap: venues are "instances" of
selected LGD "classes" with their OSM tags as features,
thus Urbanopoly data are RDF statements of the form:
<venue> <feature> <value>
UrbanGames in PlanetData - 12/12/12 Slide 9 of 25

10. Urbanopoly gameplay (1/2)
http://bit.ly/u-video
UrbanGames in PlanetData - 12/12/12 Slide 10 of 25

11. Urbanopoly gameplay (2/2)
the map with the the player’s venue the “wheel of the leaderboard
close-by venues portfolio fortune” when with the best
to be visited visiting an players
occupied venue
UrbanGames in PlanetData - 12/12/12 Slide 11 of 25

12. Urbanopoly mini-games for
Data Collection
data acquisition challenges as data validation challenges to check
contributions to an advertising campaign pre-existing data or other players’
– left: inserting a value, contribution – left: answering a quiz,
right: taking a picture right: rating a poster
UrbanGames in PlanetData - 12/12/12 Slide 12 of 25

13. Urbanopoly Data Consolidation (1/2)
 Each statement has a confidence score:
{ <venue> <feature> <value> . } <confidence>
which indicates the probability of the statement to be true
 Each player action is taken as an evidence of the
associated knowledge and alters the confidence score
 A weighted majority voting algorithm aggregates the
evidences:
◦ Difficulty to acquire the contribution (e.g., typing vs. check box)
◦ Player’s reputation (e.g., number of errors)
◦ Player’s distance to the venue at contribution time (as sensed by the
device geo-positioning)
UrbanGames in PlanetData - 12/12/12 Slide 13 of 25

14. Urbanopoly Data Consolidation (2/2)
 When the confidence score overcomes a threshold, the
triple <venue> <feature> <value> gets consolidated
 There are two thresholds:
◦ An upper threshold: if the confidence score
becomes greater than this threshold,
the triple can be considered true
◦ A lower threshold: if the confidence score
becomes smaller than this threshold,
the triple can be considered false
◦ The values of thresholds are set with an initial evaluation of the
collected data in order to maximize the trade-off between the
correctness of the consolidated information (accuracy) and the
system ability to collect contributions (throughput)
UrbanGames in PlanetData - 12/12/12 Slide 14 of 25

15. Urbanopoly Data Publication (1/2)
 True statements are published as linked open data
◦ If a statement's confidence overcomes the threshold, the
statement is asserted: <venue> <feature> <value>
(as in LinkedGeoData/OpenStreetMap)
 But there's more interesting information to publish!
◦ False statements, statements' confidence, provenance
information (who said what, when and where), etc.
◦ We created a Human Computation ontology extending the
W3C PROV-O ontology (cf. http://swa.cefriel.it/ontologies/hc)
◦ We published this further knowledge as annotations to the
player's evidences (technically speaking, the reification of the
<venue> <feature> <value> statements)
UrbanGames in PlanetData - 12/12/12 Slide 15 of 25

16. Urbanopoly Data Publication (2/2)
 Human Computation ontology and its relation with the
W3C PROV-O ontology
provo:Entity
contributionFrom aggregatedFrom Consolidated
Contribution
Information
provo:Agent
solutionTo
aggregatedBy
Contributor
Human enabledBy Human
Computation Computation
solvedBy
Task Algorithm
provo:Activity
 Cf. http://swa.cefriel.it/linkeddata/
UrbanGames in PlanetData - 12/12/12 Slide 16 of 25

17. Urbanopoly Evaluation (1/2)
 "Enjoyability" of the game (engagement potential):
◦ Average life play: ALP = Played Time / Active Players
◦ ~ 100 minutes  very good result 
 "Effectiveness" of the GWAP mechanism:
◦ Throughput = Solved Problems / Played Time
◦ ~ 287 collected evidences / hour  very good 
◦ ~ 5 consolidated statements / hour  improvable 
 "Precision" of the results (measured on results' subset)
◦ Accuracy = ( (P – FP) + (N – FN) ) / (P + N)
◦ ~ 92 %  very good result 
UrbanGames in PlanetData - 12/12/12 Slide 17 of 25

18. Urbanopoly Evaluation (2/2)
 "Playability" of the game
◦ Evaluation survey at http://bit.ly/u-survey, with questions about
usability, social aspects, physical presence, motivation, etc.
◦ Feedbacks very encouraging 
 "Sociability" through Facebook channel
◦ With Facebook Insights (http://www.facebook.com/insights/), tracking
of installs, demographics,
log-ins, content sharing, etc.
◦ Example of published "story" on
Facebook Timeline:
◦ Statistics about "stories" and
"impressions":
◦ Interesting results, but channel
to be further exploited 
UrbanGames in PlanetData - 12/12/12 Slide 18 of 25

19. Creating Awareness
 Spreading the word:
◦ Institutional sites: CEFRIEL website, PoliMI website
◦ Interview and dedicated post on semanticweb.com 
Urbanopoly: Gamers’ Way To Smarten Up Cities
 Letting Urbanopoly "compete" with other apps:
◦ Participation to the 10th "Semantic Web Challenge" 
selected among finalists
◦ Participation to the "OpenApp Lombardia" contest
 Setting up various communication channels:
◦ Social networks: YouTube, Facebook, Twitter
◦ App stores and dedicated websites: Google Play,
Urbanopoly website, iTunes, UrbanMatch website
UrbanGames in PlanetData - 12/12/12 Slide 19 of 25

20. Scientific Dissemination
 I. Celino, S. Contessa, M. Corubolo, D. Dell'Aglio, E. Della Valle, S. Fumeo and T. Krüger: "Linking
Smart Cities Datasets with Human Computation – the case of UrbanMatch", Proceeding of
the 11th International Semantic Web Conference, ISWC 2012, Part II, Springer LNCS 7650, pp.
34–49, 2012.
 I. Celino, D. Cerizza, S. Contessa, M. Corubolo, D. Dell'Aglio, E. Della Valle and S. Fumeo:
"Urbanopoly – a Social and Location-based Game with a Purpose to Crowdsource your
Urban Data", Proceedings of the the 4th IEEE International Conference on Social Computing,
Workshop on Social Media for Human Computation (SoHuman2012), pp. 910-913, DOI:
10.1109/SocialCom-PASSAT.2012.138, 2012.
 I. Celino, S. Contessa, M. Corubolo, D. Dell'Aglio, E. Della Valle, S. Fumeo, T. Krüger:
"UrbanMatch – linking and improving Smart Cities Data", In Proceedings of the Linked Data
on the Web Workshop (LDOW2012), CEUR Volume 937, co-located with the World Wide Web
Conference (WWW2012), April 16th, Lyon, France, 2012.
 I. Celino, D. Cerizza, S. Contessa, M. Corubolo, D. Dell'Aglio, E. Della Valle, S. Fumeo and F.
Piccinini: "Urbanopoly: Collection and Quality Assessment of Geo-spatial Linked Data via a
Human Computation Game", Proceedings of the 10th Semantic Web Challenge 2012, November
2012.
 I. Celino: "Volunteered Geographic Information Provenance: Semantic Web-based
Representation and Publishing", accepted with revision, IEEE Transactions on Geoscience and
Remote Sensing (TGRS) for the Special Issue on "Geoscience Data Provenance"
 I. Celino, D. Dell'Aglio, E. Della Valle: "Citizen Computation: effective Community
Participation to Smart Cities", under review, IEEE Communications Magazine for the Special
Issue on "Smart Cities"
UrbanGames in PlanetData - 12/12/12 Slide 20 of 25

21. Lessons learned
 Urbanopoly achieves the purpose for which it was designed
◦ Designing a GWAP means "tuning" the right mix of gaming elements
and purpose-related features
◦ Improvements and extensions are of course possible, e.g. by adding
further mini-games in the global game narration
◦ Physical presence is often (but not always) a good replacement for
location-specific knowledge
 Urbanopoly players are not OpenStreetMap editors
◦ Different motivations can exist for the same purpose
◦ The purpose of a GWAP should definitely be hidden
◦ Different users can be complementary
 UrbanGames are not the universal solution for any task
UrbanGames in PlanetData - 12/12/12 Slide 21 of 25

22. Citizen Science
Glacier NPS http://www.flickr.com/photos/glaciernps/4427416227/
Mount Rainier NPS
http://www.flickr.com/photos/mountrainiernps/6997851139/
Glacier NPS http://www.flickr.com/photos/glaciernps/4427412443/
UrbanGames in PlanetData - 12/12/12 Slide 22 of 25

23. http://www.flickr.com/photos/borisvanhoytema/685879933/ http://www.flickr.com/photos/gareth1953/6786545520/
UrbanGames in PlanetData - 12/12/12
What about those citizens?
http://www.flickr.com/photos/8482460@N06/6884509346/ http://www.flickr.com/photos/paul_houle/3301438074/
Slide 23 of 25

24. Citizen Computation
Human Computation Citizen Science
exploiting volunteers
exploiting human capabilities
to collect scientific data or
to solve computational tasks
to conduct experiments
difficult for machines
"in the world"
Citizen Computation
exploiting human capabilities
to contribute to a mixed
computational system
by living "in the world"
UrbanGames in PlanetData - 12/12/12 Slide 24 of 25

25. Thanks for your attention!
Irene Celino
CEFRIEL – ICT Institute,
Politecnico di Milano
irene.celino@cefriel.it