In this presentation, I illustrate, and discuss initial results from a quantitative analysis of the performance of WPS servers. To do so, two test scenarios were used to measure response time, response size, throughput, and failure rate of five WPS servers including 52North, Deegree, GeoServer, PyWPS, and Zoo. I also assess each WPS server in terms of qualitative metrics such as software architecture, perceived ease of use, flexibility of deployment, and quality of documentation. A case study addressing accessibility assessment is used to evaluate the relative advantages and disadvantages of each implementation, and point to challenges experienced while working with these WPS servers.
5. INTRODUCTION!
Spatial Web Services Evaluations:!
§ Web Map Service {Horák, J. et al. (2009), Kalbere, P. (2010), FOSS4G Shootout}!
§ Sensor Observation Service {Tamayo, A. et al. (2011), Poorazizi, M.E. et al. (2012)}!
§ Web Processing Service {Scholten, M. et al. (2006), Michaelis, C. and Ames, D. (2009),
FOSS4G Shootout}!
5!
What have been done so far:!
§ Focused on influential performance issues, the WPS protocol and its
specification, and compliance and interoperability testing!
7. WEB PROCESSING SERVICE!
7!
A web service interface to standardize the way that (spatial)
algorithms are made available on the Internet!
GET
POST
SOAP
GetCapabilities
DescribeProcess
Execute
8. WPS SERVERS!
8!
!
52°North! Deegree! GeoServer! PyWPS! Zoo!
Organization! 52°North GmbH! OSGeo! OSGeo! Intevation GmbH! OSGeo!
Development Platform! Java! Java! Java! Python! C/C++!
License! GNU GPL v. 2! LPGL! GNU GPL v. 2! GNU GPL v. 2! MIT/X-11 style!
Supported Libraries! JTS!
GeoTools!
SEXTANTE!
R!
GRASS!
ArcGIS!
SEXTANTE! JTS!
GeoTools!
GRASS!
GDAL!
R!
GRASS!
GEOS!
GDAL!
Na t i v e l y S u p p o r t e d
Development Languages!
Java! Java! Java! Python! C/C++!
Fortran!
Java!
Python!
PHP!
Perl!
JavaScript!
Service ! Servlet! Servlet! Servlet! CGI! CGI!
DCP Request! GET/POST/SOAP! GET/POST/SOAP! GET/POST! GET/POST/SOAP! GET/POST!
25. METHODOLOGY – TEST SCENARIO!
SCENARIO A!
ü To assess how fast each server responds!
ü 45 Execute requests, random WPS input parameters:!
§ Walking Start Point!
§ Walking Start Time: 5 a.m. – 12 p.m. !
§ Walking Time Period (WTP): 5 min – 20 min!
§ Walking Speed: 3 km/h – 6 km/h!
§ Bus Waiting Time (BWT): 0 – WTP!
§ Bus Ride Time: 0 – (WTP – BWT)!
§ Distance Decay Function: True/False!
!
25!
26. METHODOLOGY – TEST SCENARIO!
SCENARIO B!
ü To assess the effect of increased load on each server!
§ Number of concurrent requests: 2n , n = {0,…,7}!
§ 30 replications!
§ Random WPS input parameters, like SCENARIO A!
26!
27. METHODOLOGY – TEST ENVIRONMENT!
§ Each WPS server was installed on a separate Virtual Machine (VM)!
§ The same machine used to run the VMs and the tests!
§ JMeter was used to perform the tests!
27!
Hardware! Dell OptiPlex 990 (Host)! VMware (VM)!
CPU! Intel Core i5 3.1 GHz! 4 Cores of 8!
RAM! 8 GB! 4 GB!
HDD! 500 GB! 40 GB!
OS! Windows 7 Professional (64-bit)! Ubuntu 12.04 LTS (64-bit)!
28. EXPERIMENTAL RESULTS!
SCENARIO A!
§ Response Time: time taken for a service call to return all response bytes!
§ Response Size: the quantity of data exchanged between client and server!
!
28!
WPS Server! Response Time (sec.)! Response Size (kB)!
52°North! 2.784 ± 1.269! 2.448 ± 0.267!
Deegree! 2.499 ± 1.259! 2.505 ± 0.267!
GeoServer! 2.753 ± 1.255! 2.301 ± 0.267!
PyWPS! 3.995 ± 1.661! 2.686 ± 0.269!
Zoo! 2.999 ± 1.313! 2.479 ± 0.269!
34. LESSONS LEARNED!
34!
52°North! Deegree! GeoServer! PyWPS! Zoo!
Installation*! Easy! Easy! Easy! Difficult! Difficult!
Create new processes
Easy! Medium! Easy! Difficult! Easy!
and Configuration* !
Native Development
Languages!
One ! One ! One ! One ! Many!
Quality of
Documentation**!
Great! Good! Great! Good! Good!
Community Support!
• Many
Avenues!
• Active
Support!
• Many
Avenues!
• Active
Support!
• Many
Avenues!
• Active
Support!
• Few
Avenues!
• Little
Support!
• Many
Avenues!
• Active
Support!
* Ranking ranges: Easy; Medium; Difficult!
** Ranking ranges: Weak; Good; Great!
35. DISCUSSION!
SCENARIO A!
§ Deegree returns the response package most rapidly!
§ But, based on a One Way ANOVA test, there is no significant difference in
response time between the WPS servers tested nor data volume returned!
SCENARIO B!
§ Deegree and GeoServer performed similarly; Deegree tending to perform better
under high loads!
§ 52°North had difficulty when processing more than 16 concurrent requests, but
performed more effectively under low loads!
§ From a qualitative perspective, compared to other WPS servers, 52°North and
GeoServer seem to be the best choices!
35!
36. CONCLUSIONS!
36!
§ When selecting an appropriate WPS server, it is important to consider both
quantitative and qualitative metrics!
§ The importance of each metric can be weighted based on different
application requirements!
§ A recommendation: start with an evaluation process with some basic
questions:!
ü Who is the user of the system, end-user or application developer?!
ü How complex are the geoprocessing services?!
ü How should the system function, synchronous or asynchronous?!
ü What is the architecture used to design the processing workflow?!
ü What is the expected number of users?!
37. THANK YOU!
ANY QUESTION?!
Ebrahim Poorazizi!
mepooraz@ucalgary.ca!
ebrahim.planyourplace.ca!
Andrew Hunter!
ahunter@ucalgary.ca!
gisciencegroup.ucalgary.ca!