This document summarizes Hilary Weaver-Robb's presentation on testing RESTful web services. The presentation covered:
1) What a web service and RESTful web service are, including the constraints of a RESTful design like using HTTP methods for CRUD operations.
2) Why web services should be tested, as they are often developed before user interfaces and validate critical integration points.
3) How to test RESTful web services, including exploring documentation, implementing manual tests with tools like Postman, and setting up automated tests of common scenarios.
4) Specific testing techniques like validating CRUD operations at endpoints, checking boundary conditions, and testing error conditions.
1. W15
Testing Web Services
10/18/2017 3:00:00 PM
Testing RESTful Web Services
Presented by:
Hilary Weaver-Robb
Quicken Loans
Brought to you by:
350 Corporate Way, Suite 400, Orange Park, FL 32073
888-‐268-‐8770 ·∙ 904-‐278-‐0524 - info@techwell.com - https://www.techwell.com/
2. Hilary Weaver-Robb
Quicken Loans
Hilary Weaver-Robb is a software quality architect at Detroit-based Quicken
Loans. She is a mentor to her fellow QA team members, makes friends with
developers, and helps teams level-up their quality processes, tools, and
techniques. Hilary has always been passionate about improving the relationships
between developers and testers, and evangelizes software testing as a
rewarding, viable career. She runs the Motor City Software Testers user group,
working to build a community of quality advocates. Hilary tweets (a lot) as
@g33klady, and you can find tweet-by-tweet recaps of conferences she's
attended, as well as her thoughts and experiences in the testing world, at
g33klady.com.
4. 9/6/2017
2
OBJECTIVE
What a Web service is
What makes a Web service RESTful
Why we should test Web services
How to test RESTful Web services
@G33KLADY | G33KLADY.COM | GITHUB.COM/G33KLADY
WHAT IS A WEB SERVICE?
A Web service is a method of communication between two electronic devices over
a network.
It is a software function provided at a network address over the Web with the
service always on as in the concept of utility computing.
The W3C defines a Web service generally as: a software system designed to
support interoperable machine-to-machine interaction over a network.
- Wikipedia
@G33KLADY | G33KLADY.COM | GITHUB.COM/G33KLADY
5. 9/6/2017
3
WHAT IS A WEB SERVICE?
A website provides information consumable by humans
A web service provides information consumable by software
- A genius on StackOverflow
All Web services are APIs, but not all APIs are Web services
@G33KLADY | G33KLADY.COM | GITHUB.COM/G33KLADY
WHAT MAKES A WEB SERVICE RESTFUL?
@G33KLADY | G33KLADY.COM | GITHUB.COM/G33KLADY
6. 9/6/2017
4
WHAT IS REST?
A way for systems to talk to one another (not always Web services)
REST stands for REpresentational State Transfer
Constraints:
Uniform Interface
Stateless
Cacheable
Client-Server
Layered System
@G33KLADY | G33KLADY.COM | GITHUB.COM/G33KLADY
WHAT MAKES A WEB SERVICE RESTFUL?
Client-Server architecture using HTTP protocol
Client sends Request to Server
Server sends Response back
@G33KLADY | G33KLADY.COM | GITHUB.COM/G33KLADY
7. 9/6/2017
5
WHAT MAKES A WEB SERVICE RESTFUL?
URL tells you what you’re working with and doing
Messages can be lightweight (even just the URL!)
CRUD operations using HTTP methods
Create/Update data (POST, PUT)
Read data (GET)
Delete data (DELETE)
WAY easier to test than non-RESTful services
@G33KLADY | G33KLADY.COM | GITHUB.COM/G33KLADY
COMPARISON
@G33KLADY | G33KLADY.COM | GITHUB.COM/G33KLADY
8. 9/6/2017
6
HTTP RESPONSE CODES
200s – OK!
400s – What you’re asking for, we can’t do
500s – Should only be uncontrollable circumstances
@G33KLADY | G33KLADY.COM | GITHUB.COM/G33KLADY
WHY SHOULD WE TEST WEB SERVICES?
@G33KLADY | G33KLADY.COM | GITHUB.COM/G33KLADY
9. 9/6/2017
7
@G33KLADY | G33KLADY.COM | GITHUB.COM/G33KLADY
WHY TEST WEB SERVICES?
Usually complete before UI is even started
Validate the ductwork
UI can pull in multiple Web services
Because it’s code!
Integration Tests
UI Tests
@G33KLADY | G33KLADY.COM | GITHUB.COM/G33KLADY
Unit Tests
10. 9/6/2017
8
HOW CAN I TEST RESTFUL WEB SERVICES?
@G33KLADY | G33KLADY.COM | GITHUB.COM/G33KLADY
HOW CAN I FIGURE OUT WHAT TO TEST?
Documentation
Swagger
WADL/WSDL
RAML (RESTful API Modeling Language)
@G33KLADY | G33KLADY.COM | GITHUB.COM/G33KLADY
11. 9/6/2017
9
HOW CAN I FIGURE OUT WHAT TO TEST?
Check out the code
Controllers
Models
Tests
Run web debugger and follow the calls
@G33KLADY | G33KLADY.COM | GITHUB.COM/G33KLADY
HOW CAN WE TEST WEB SERVICES?
Manual Testing Tools
Postman
Fiddler
Advanced REST Client
ReadyAPI (SoapUI)
Swagger
Automation Tools
Postman
ReadyAPI
Most unit testing frameworks
Swagger
@G33KLADY | G33KLADY.COM | GITHUB.COM/G33KLADY
12. 9/6/2017
10
WHAT TYPES OF TESTS CAN WE PERFORM?
Smoke tests
CRUD tests
Boundary
Required Fields
Field Type
Error ConditionsError Conditions
Security
Performance
@G33KLADY | G33KLADY.COM | GITHUB.COM/G33KLADY
LET’S DO SOME TESTING!
Gitter API
GitHub chat
https://gitter.im
Interact with rooms, users, and messages
@G33KLADY | G33KLADY.COM | GITHUB.COM/G33KLADY
13. 9/6/2017
11
MESSAGES RESOURCE SPECS
List messages
GET https://api.gitter.im/v1/rooms/:roomId/chatMessages
Get a message
GET https://api.gitter.im/v1/rooms/:roomId/chatMessages/:messageId
Send a message
POST https://api.gitter.im/v1/rooms/:roomId/chatMessages
Update a messagep g
PUT https://api.gitter.im/v1/rooms/:roomId/chatMessages/:messageId
? # of characters
Can be created, updated, read via API
Can be “deleted” but not via API
@G33KLADY | G33KLADY.COM | GITHUB.COM/G33KLADY
@G33KLADY | G33KLADY.COM | GITHUB.COM/G33KLADY
14. 9/6/2017
12
CRUD
@G33KLADY | G33KLADY.COM | GITHUB.COM/G33KLADY
TESTING CRUD OPERATIONS
Create, Read, Update, Delete
Test separately to validate each method works
Test the lifecycle of an object
Catch issues with caching or concurrency
Create -> Read -> Update -> Read -> Delete -> Read
@G33KLADY | G33KLADY.COM | GITHUB.COM/G33KLADY
19. 9/6/2017
17
TESTING BOUNDARIES
Above and below numerical limits
Value must be <= 19.9 and Value must be > 0
@G33KLADY | G33KLADY.COM | GITHUB.COM/G33KLADY
TESTING BOUNDARIES
How long can a message be?
2402 worked
6666 didn’t (400 Bad Request)
4001 worked, 4151 didn’t
4096 worked, 4097 didn’t
4096 ASCII characters – UI blocks from > 4096
4096 Unicode snowmen ☃
4096 Japanese characters
We’ll visit this again with automation!
@G33KLADY | G33KLADY.COM | GITHUB.COM/G33KLADY
20. 9/6/2017
18
REQUIRED FIELDS
@G33KLADY | G33KLADY.COM | GITHUB.COM/G33KLADY
TESTING REQUIRED FIELDS
If required data is omitted, it yields an accurate response
If I fail to submit required information
Will I be able to understand how to fix it?
Is it handled gracefully?
Is the request processed as if it wasn’t required?
@G33KLADY | G33KLADY.COM | GITHUB.COM/G33KLADY
23. 9/6/2017
21
FIELD TYPES
@G33KLADY | G33KLADY.COM | GITHUB.COM/G33KLADY
TESTING FIELD TYPES
If the client sends a different data type than what is expected
Expect INTEGER, send
STRING
DECIMAL
☃
Send a Message only has StringsSend a Message only has Strings
Accepts Unicode and UTF-8
BORING
@G33KLADY | G33KLADY.COM | GITHUB.COM/G33KLADY
26. 9/6/2017
24
FAILURE
@G33KLADY | G33KLADY.COM | GITHUB.COM/G33KLADY
TESTING FOR FAILURE
Testing for uncontrollable circumstances
500 Internal Server Error
Timeouts
404s
Server Down
Database not responding
How does the UI react when Web service has these issues?
Test for 3rd Party APIs having these issues
Use Service Virtualization
@G33KLADY | G33KLADY.COM | GITHUB.COM/G33KLADY
28. 9/6/2017
26
TESTING SECURITY
Authentication vs. Authorization
Authentication
(who they are)
Does the
Authorization
(what they can do)
Does the
token allow
@G33KLADY | G33KLADY.COM | GITHUB.COM/G33KLADY
request have
a valid token?
token allow
access to this
resource?
TESTING SECURITY
Encryption
Use Fiddler or Wireshark to see what the requests look like “over the wire”
SQL Injection
Fuzzing
Sending tons of malformed data and see where it breaks
@G33KLADY | G33KLADY.COM | GITHUB.COM/G33KLADY
29. 9/6/2017
27
EXPLORATORY
@G33KLADY | G33KLADY.COM | GITHUB.COM/G33KLADY
EXPLORATORY TESTING
Test design and test execution at the same time
– James Bach
Not scripted
Not pre-planned
Let the application lead you
Use your past experience as heuristicsUse your past experience as heuristics
@G33KLADY | G33KLADY.COM | GITHUB.COM/G33KLADY
30. 9/6/2017
28
EXPLORATORY TESTING WEB SERVICES
Apply the same principles as with a GUI interface
Think about other uses
What else uses the API now?
What else could use the API in the future?
@G33KLADY | G33KLADY.COM | GITHUB.COM/G33KLADY
EXPLORATORY TESTING WEB SERVICES
q
No documentation
@G33KLADY | G33KLADY.COM | GITHUB.COM/G33KLADY
32. 9/6/2017
30
AUTOMATED CHECKING
@G33KLADY | G33KLADY.COM | GITHUB.COM/G33KLADY
AUTOMATED CHECKING OF WEB SERVICES
Automate what can be checked repeatedly
Binary decisions (true or false) can be automated
Does response code match what I’m expecting for this request?
Is the response time within the SLA?
@G33KLADY | G33KLADY.COM | GITHUB.COM/G33KLADY
33. 9/6/2017
31
AUTOMATION PROCESS
Decide on tooling
Common Utility Methods
Performing the Web request
Reading response content
Performing a query of a database
Models
Classes for the requests and responses
Easier to manipulate an object for testing than a string
Write integration tests!
@G33KLADY | G33KLADY.COM | GITHUB.COM/G33KLADY
AUTOMATION PROCESS
Create utility method(s)
Create or reference models of requests and responses
App.config to hold auth keys, URIs, etc
@G33KLADY | G33KLADY.COM | GITHUB.COM/G33KLADY
35. 9/6/2017
33
AUTOMATION PROCESS
Start writing small, simple tests
Hit the service with a valid request, get a 200 back?
Hit the service, validate we get a specific field back correctly?
Hit the service, validate we get a certain number of items?
@G33KLADY | G33KLADY.COM | GITHUB.COM/G33KLADY
@G33KLADY | G33KLADY.COM | GITHUB.COM/G33KLADY
36. 9/6/2017
34
AUTOMATION PROCESS
Make it more interesting (and complicated)
POST to the service, validate it posted
Subsequent GET to the service
Read from the database
CRUD operations, in order
@G33KLADY | G33KLADY.COM | GITHUB.COM/G33KLADY
@G33KLADY | G33KLADY.COM | GITHUB.COM/G33KLADY
37. 9/6/2017
35
AUTOMATION PROCESS
Data-driven tests
Same test structure, just different data and expected results
Required fields (especially if there are a lot)
Boundary tests
0 characters – 200 OK
10 characters – 200 OK
4096 characters 200 OK 4096 characters – 200 OK
4097 characters – 400 BAD REQUEST
@G33KLADY | G33KLADY.COM | GITHUB.COM/G33KLADY
@G33KLADY | G33KLADY.COM | GITHUB.COM/G33KLADY
38. 9/6/2017
36
AUTOMATE ALL THE THINGS!
Combine Integration and GUI Automation!
Example for Messages resource
POST via Web Service
Get the message ID and timestamp from the response body
Launch the web app with Selenium
Assert that message ID, with that timestamp and message text, appears properly
@G33KLADY | G33KLADY.COM | GITHUB.COM/G33KLADY
TEST THOSE WEB SERVICES!
@G33KLADY | G33KLADY.COM | GITHUB.COM/G33KLADY
39. 9/6/2017
37
TEST THOSE WEB SERVICES
No pretty UI for end users
Bugs in Web services cause just as many headaches for users
Different skill set for testers
A lot of the same principles
Seems complicated
Just a different window into our applications
@G33KLADY | G33KLADY.COM | GITHUB.COM/G33KLADY
THAT’S IT!
Code, Postman Collection, and Resources
https://github.com/g33klady/TestingRESTServices
Questions?Questions?
@G33KLADY | G33KLADY.COM | GITHUB.COM/G33KLADY