A survey of various tools and techniques commonly used by API craftsman. API specification languages, testing, debugging and analytics are covered. As presented by Jason Harmon at Booz Allen Hamilton's "Distinguished Speaker Series" 3/2/2015

1. API TOOLS AND
TECHNIQUES
Jason Harmon
@jharmn

2. AGENDA
Design Scale
• API specification formats
API Management
Debugging & Testing
• Request/Response capture
• Testing approaches
Analytics

3. SCALE
A slightly different look

4. DESIGN SCALE
Problems
• Producing consistent
designs
• Prototyping and rapid
design iteration
• Generate code from
design
• Generate docs from
design

5. API SPECIFICATIONS
• A format to describe operations
• Produced by implementation, or design-first
• Allows for codegen of client or server, as well as
documentation
• Mock APIs with sample responses from spec
• Quick feedback from clients from design-first allows for
rapid iteration

6. HISTORY OF API SPECS

7. API SPECIFICATION
FORMATS
• WADL
• WSDL 2.0
• Google Discovery
Document
• IODocs
• Swagger
• Blueprint
• RAML

8. WADL & WSDL 2.0
• 2007: http://www.w3.org/TR/wsdl20/
• 2009: http://www.w3.org/Submission/wadl/
• Neither was ever finalized
• Considered to be largely experimental
artifacts with little adoption

9. GOOGLE DISCOVERY
DOCUMENT
• 2010?:
https://developers.google.com/discovery/v1/referen
ce/apis
• Created primarily for discovery (thus the name)
• Still in use at Google
• JSON Schema-based
• Very little adoption

10. IODOCS
• 2011: https://github.com/mashery/iodocs
• OSS project by API management vendor
Mashery
• JSON-Based
• Largely used by customers in the Mashery
ecosystem

11. SWAGGER
• 2011: http://swagger.io/
• OSS by Reverb (https://helloreverb.com/), maker of the Reverb app
• Meant to produce spec from implementation
• Recent additions allow for design-first
• Hyperactive OSS community engagement
• Integrated with many products
• Now in version 2.0

12. BLUEPRINT
• 2013: https://apiblueprint.org/
• Created by Apiary (http://Apiary.io) (an API
portal platform) as standard for their platform
• Markdown-based
• First spec format to switch from code-first to
design-first

13. RAML
• 2013: http://raml.org/
• Created by Mulesoft, a long-standing SOA/ESB/API vendor, as
standard for their platform
• YAML-based
• Design-first
• Resource type definition/traits
• Now in version 2.0

14. CURRENT STATE
• Swagger & RAML are in version 2.0
• Swagger is the only player not profiting directly from the spec,
powered through partnerships
• Swagger has an incredible following in the OSS community
• Swagger added design-first later, RAML was design-first up-
front
• RAML seems to be well received by the enterprise community
• Blueprint continues to gain adopters; recent additions of
hypermedia

15. TRENDS
Design-first + Mocking
• Start slow to go fast
• Get API client feedback on
mock APIs
• Real usability is only
measurable with tactile
feedback
• Weakness: multi-scenario
and errors are hard to mock

16. TRENDS
Interactive editors
• Non-JSON formats growing in popularity for readability
• Swagger 1.5 added YAML-based
• RAML is YAML
• Blueprint is Markdown

17. Swagger Editor

19. DEVELOPER DEBUGGING
• Figure out why calls aren’t working
• Save calls to make retry simpler

20. POSTMAN
• Loved by developers
• Easy to use locally
• Simple Chrome plugin
• Some limitations in being
from browser
• Not great for sharing within a
team

21. POSTMAN
Able to create & save requests in collections
Parameterize requests
Import/export collections

22. COMMAND-LINE
Requires no UI
Works on any machine
Save calls as text
cURL is the lingua franca
of APIs
HTTPie is gaining
popularity
http://curl.haxx.se/
HTTPie
https://github.com/jakubroztocil/httpie

23. CURL - LINGUA
FRANCA

24. CURL - LINGUA
FRANCA

25. CURL - PHP NATIVE
Also batteries included in many other languages

26. DEBUGGING
Request/response capture
• Amazing for developers + support
• Figure out why calls aren’t working
• Often a great setup for testing, from captures
• Implemented by using an HTTP proxy

27. CAPTURE
Consum
er
Proxy Server
Request
Respons
e
Capture
api.something.comhttps://api-something-com-74776xpbe6nt.runscope.net/

28. CAPTURE IS IN ONE
PLACERunscope bought up just about everything
They’re experts at capture

29. REQUESTB.IN
Yet another “Runscope Community Project”

30. HURL.IT
Yet another “Runscope Community Project”

31. RUNSCOPE
Simple to change your hostname to Runscope's
Locally hosted proxy available
Great for debugging webhooks

32. TESTING
API tests are about
consumers making calls

33. LOTS OF
DEVELOPMENTAPI testing is still rather cutting edge
Many new options, not many clear picks

34. UNIT TESTING IS NOT
ACCEPTANCE TESTING
Only an API consumer is an effective API test

35. AC IS PROBABLY NOT FROM
PRODUCT MANAGERS
They don’t usually understand APIs
http://apiux.com/2014/02/13/api-product-manager/

36. BALLMER GETS IT
APIs are the developers’ world

37. TESTING TOOLS
There’s no reason to have a buggy API
Work like a craftsman

38. CUCUMBER
Behavior Driven Development
http://www.pragmaticapi.com/blog/2013/11/10/bdd-for-apis-talk-at-
apistrat-sf-2013

39. BDD ALTERNATIVES
Frisby.js

40. CURL FOR API
TESTINGDeath by a thousand paper cuts

41. RUNSCOPE
Tests derived from proxy/captures

42. SOAPUI (BY
SMARTBEAR)It’s called…SOAP
You can host it completely locally

43. READY API (BY
SMARTBEAR)Private cloud, capture, API virtualization/mocking,
load testing, more

44. ANALYTICS
Measure what you treasure

45. ANALYTICS
Before you do anything:
KPIs for APIs by John Musser
• Formerly of ProgrammableWeb
• Now runs API Science
https://www.youtube.com/watch
?v=UFjgbsRrLHw
http://www.slideshare.net/jmuss
er/kpis-for-apis

46. CALLS…?
There’s more to it.

47. MEASUREMENT
OVERLOAD
• Performance, defect rate, stability, support
tickets
• Customer satisfaction/NPS, adoption, churn
• Channel performance, revenue, market
share

48. SLOW
DOWN.
BREATHE.
There are a massive
amount of things to
measure

49. DEFINE
YOUR
FUNNELS
Where do you gain value?

50. PLAYERS
App Users
Apps
Developers
Your APIs
Your Company

51. VOLUME IS NOT VALUE
High call volume doesn’t mean you’re doing it
right

52. MEASURE
INSIDE AND
OUT
Internal API metrics are
critical
You may learn things about
your value chain
(see Netflix and LinkedIn)

53. TTFHW
“Time to first hello world”
Helps measure Developer Experience (DX)

54. MICROSERVICES
Optimize your scale
Scale up where volume calls for it

55. DASHBOARDING
• Think about events
• Analyze funnels
• MongoDB/Couch/etc is
tempting
• Keen.io makes it easier

56. PERFORMANCE METRICS
• API Management Vendors
• All provide this core functionality
• DIY: Often fits into existing metrics
• Mongo/Couch solutions abound
• Often standard web metrics platforms will do the
trick

57. API MANAGEMENT

58. VENDORS
GALORE
3Scale
Apigee
Axway/Vordel
CA Technologies
IBM
Mashery/Intel
Mulesoft
SOA Software
WS02

59. MORE
STUFF THAN
YOU THINK
Routing/Provisioning
Packaging
Rate limiting
Event processing
Monitoring
Analytics
Security/Auth
Caching
Geodistribution

60. MORE THAN JUST A
PROXY
Consum
er
Proxy Server
LOTS

61. TIME TO
MARKET
DIY:
• Big investment
• Long timeline
• Ongoing maintenance
• NGINX leading solutions
for proxy aspect

62. THANKS!