1. (HOW) DO PLANNING SUPPORT
SYSTEMS IMPROVE PLANNING?
A controlled experiment

2. Urban environments – climate - mobility
NWO: CESAR PROGRAM
MARCO TE BRÖMMELSTROET UNIVERSITY OF AMSTERDAM
PETER PELZER UTRECHT UNIVERSITY
RALPH KLERKX TNO
SANDER SCHAMINÉE TNO

3. Research rationale
 PSS field is maturing
 ‘Knowledge technologies’
 Many claims & hypotheses
 Not much focus on testing
 Overview of PSS books:
 414 reported claims
 59 reported tests

4. Research rationale
 Implementation gap
 Especially in strategic planning
 Constructive learning requires:
 Consolidated aims
 Consolidated theory
 Consolidated methods
(Nijstad 2009 in: Te Brömmelstroet 2013)

5. Conceptual model

6. PSS: Urban Strategy
 Urban Strategy: developed by TNO since 2006
 Started as an environmental modeling tool
 Main features:
 Calculate and display effects interactively
 Centralized data storage
 Scenarios
 Communication between models

7. PSS: Urban Strategy
3D Database3D Database
IMB Communication Framework (Publish/Subscribe)
Traffic Noise Air Safety Energy Cost
1D
Viewer
2D (GIS)
Viewer
3D
Viewer
External
Data Link
External
Models
Offline
Models
Sensor
Link
Mobile
Client
Connect
PDA, SmartphonePDA, SmartphoneWMS, WFS,
NetCDF
Tablet, Surface TableTablet, Surface Table
OPS
Lotos
EUROS

8. PSS: Urban Strategy
Perform measures
in GIS environment

9. Experiment
Video of experiment
(5 min)

10. CLASSIC USABILITY INDICATORS
6,12 6,07
5,81 5,79 5,74 5,71 5,60 5,59 5,55 5,55
5,75
5,36 5,31 5,24
5,10 5,07
4,92
0,00
1,00
2,00
3,00
4,00
5,00
6,00
7,00
LESSTIM
E
DOM
ORE
SKETCHING
LESSEFFORT
DISPLAYED
EVALUATION
UNDERSTANDABLE
CREATING
COM
M
UNICATIVE
CREDIBLE
OVERALL
USERFRIENDLY
DETAIL
BROADNESS
TRANSPARANCY
INDICATORS
SPECIFIC

11. RESULTS ON PLANNING PROCESS QUALITY

12. RESULTS ON PLANNING OUTCOME QUALITY

13. Methodological reflection
Both observation and indicators show a positive effect
on planning (process). But:
 Low ecological validity
 combine with real world observations
 High measurement validity
 ‘Quality of planning’ concept under development
 External validity
 Is Urban Strategy a typical PSS (for strategy making)

14. Methodological reflection
Both observation and indicators show a positive effect
on planning (process). But:
 Role playing is problematic
 Students from different backgrounds
 Holding treatment constant is problematic
 More focus on fundamental mechanisms
 Final lesson: Potential improvements on organizing the exchange
of knowledge between participants and PSS

15. Thank you for your kind attention
E-mail: brommelstroet@uva.nl
Recent publications:
Te Brömmelstroet, M. (2012). "Transparency, flexibility, simplicity: From buzzwords towards
strategies for real PSS improvement." Computers, Environment and Urban Systems, 36(1),
96-104
Te Brömmelstroet M. (2013). “Performance of Planning Support Systems: What is it, and
how do we report on it?”. Computers, Environment and Urban Systems, in press