How to test external services in a ruby application

1. Effectively
Testing Services
Neal Kemp

2. $ whoami
Iowa native
Now: Californian
Software Developer
Ruby, Rails, Javascript, etc

6. what,why&how
of testing services

7. NOT
Building testable services

8. NOT
Test-driven development
(necessarily)

9. … and because I don’t want @dhh to rage

10. what

11. What is a service?
Internal “SOA”

12. Any time you make an HTTP
request to an endpoint in
another repository

13. why

14. Why are services important?
Build faster
Makes scaling easier
Use them on virtually every application
Increasingly prevalent

15. Services are critical to
modern Rails development

16. Why is testing services important?
You (should) test everything else
Services compose crucial features
You may encounter problems…

17. Internal API
Sometimes null responses
Inconsistencies
Catastrophe

18. Okay? But what about external APIs?

20. {"id": 24}
{"code": "ANA"}

21. "goals":[
{
"per":"1",
"ta":"CGY",
"et":"14:11",
"st":"Wrist Shot"
},
{
"per":"2",
"ta":"ANA",
"et":"11:12",
"st":"Backhand"
}
]
"goals": {
"per":"1",
"ta":"CGY",
"et":"14:11",
"st":"Wrist Shot"
}

22. No versioning!

24. Snapchat Client
Haphazard documentation
What are the requests?
Bizarre obfuscation
github.com/nneal/snapcat

25. how

26. What is different about services?
External network requests
You don’t own the code

28. On an airplane…
Failure is bad!
No network requests

29. Don’t interact with services from
test environment* **

30. * Includes “dummy” APIs

31. ** Using pre-recorded responses
is okay

32. Assuming: Rails, rspec

33. Timetostub!

34. Built-in Stubbing
Typhoeus
Faraday
Excon

35. Simplify.

36. gem 'webmock'

37. ENV['RAILS_ENV'] ||= 'test'
require File.expand_path('../../config/environment', __FILE__)
require 'rspec/autorun'
require 'rspec/rails’
Dir[Rails.root.join("spec/support/**/*.rb")].each { |f| require f }
ActiveRecord::Migration.check_pending! if defined?(ActiveRecord::Migration)
RSpec.configure do |config|
config.infer_base_class_for_anonymous_controllers = false
config.order = 'random’
end
WebMock.disable_net_connect!
spec/spec_helper.rb

39. module FacebookWrapper
def self.user_id(username)
user_data(username)['id']
end
def self.user_data(username)
JSON.parse(
open("https://graph.facebook.com/#{username}").read
)
end
end
lib/facebook_wrapper.rb

40. require 'facebook_wrapper'
config/intializers/facebook_wrapper.rb

41. require 'spec_helper'
describe FacebookWrapper, '.user_link' do
it 'retrieves user link' do
stub_request(:get, 'https://graph.facebook.com/arjun').
to_return(
status: 200,
headers: {},
body: '{
"id": "7901103","first_name": "Arjun",
"locale": "en_US","username": "Arjun"
}'
)
user_id = FacebookWrapper.user_id('arjun')
expect(user_id).to eq '7901103'
end
end
spec/lib/facebook_wrapper_spec.rb

42. require 'spec_helper'
describe FacebookWrapper, '.user_link' do
it 'retrieves user link' do
stub_request(:get, 'https://graph.facebook.com/arjun').
to_return(
status: 200,
headers: {},
body: '{
"id": "7901103","first_name": "Arjun",
"locale": "en_US","username": "Arjun"
}'
)
user_id = FacebookWrapper.user_id('arjun')
expect(user_id).to eq '7901103'
end
end
spec/lib/facebook_wrapper_spec.rb

43. require 'spec_helper'
describe FacebookWrapper, '.user_link' do
it 'retrieves user link' do
stub_request(:get, 'https://graph.facebook.com/arjun').
to_return(
status: 200,
headers: {},
body: '{
"id": "7901103","first_name": "Arjun",
"locale": "en_US","username": "Arjun"
}'
)
user_id = FacebookWrapper.user_id('arjun')
expect(user_id).to eq '7901103'
end
end
spec/lib/facebook_wrapper_spec.rb

44. require 'spec_helper'
describe FacebookWrapper, '.user_link' do
it 'retrieves user link' do
stub_request(:get, 'https://graph.facebook.com/arjun').
to_return(
status: 200,
headers: {},
body: '{
"id": "7901103","first_name": "Arjun",
"locale": "en_US","username": "Arjun"
}'
)
user_id = FacebookWrapper.user_id('arjun')
expect(user_id).to eq '7901103'
end
end
spec/lib/facebook_wrapper_spec.rb

45. Even Better
No network requests
Fast!
No intermittent failure

46. Mock-Services
AWS
FB graph mock
OmniAuth
Etc…

47. gem 'fb_graph-mock'

48. ENV['RAILS_ENV'] ||= 'test'
require File.expand_path('../../config/environment', __FILE__)
require 'rspec/autorun'
require 'rspec/rails’
require 'fb_graph/mock'
Dir[Rails.root.join("spec/support/**/*.rb")].each { |f| require f }
ActiveRecord::Migration.check_pending! if defined?(ActiveRecord::Migration)
RSpec.configure do |config|
config.infer_base_class_for_anonymous_controllers = false
config.order = 'random'
config.include FbGraph::Mock
end
WebMock.disable_net_connect!
spec/spec_helper.rb

49. describe FacebookWrapper, '.user_link' do
it 'retrieves user link' do
mock_graph :get, 'arjun', 'users/arjun_public' do
user_id = FacebookWrapper.user_id('arjun')
expect(user_id).to eq '7901103'
end
end
end
spec/lib/facebook_wrapper_spec.rb

50. describe FacebookWrapper, '.user_link' do
it 'retrieves user link' do
mock_graph :get, 'arjun', 'users/arjun_public' do
user_id = FacebookWrapper.user_id('arjun')
expect(user_id).to eq '7901103'
end
end
end
spec/lib/facebook_wrapper_spec.rb

51. Even Better
Already stubbed for you
Pre-recorded responses (sometimes)
Don’t need to know API endpoints

52. gem 'sham_rack'

53. gem 'sinatra'

54. ENV['RAILS_ENV'] ||= 'test'
require File.expand_path('../../config/environment', __FILE__)
require 'rspec/autorun'
require 'rspec/rails’
Dir[Rails.root.join("spec/support/**/*.rb")].each { |f| require f }
ActiveRecord::Migration.check_pending! if defined?(ActiveRecord::Migration)
RSpec.configure do |config|
config.infer_base_class_for_anonymous_controllers = false
config.order = 'random’
end
WebMock.disable_net_connect!
spec/spec_helper.rb

55. ShamRack.at('graph.facebook.com', 443).sinatra do
get '/:username' do
%Q|{
"id": "7901103",
"name": "Arjun Banker",
"first_name": "Arjun",
"last_name": "Banker",
"link": "http://www.facebook.com/#{params[:username]}",
"location": {
"id": 114952118516947,
"name": "San Francisco, California"
},
"gender": "male"
}|
end
end
spec/support/fake_facebook.rb

56. ShamRack.at('graph.facebook.com', 443).sinatra do
get '/:username' do
%Q|{
"id": "7901103",
"name": "Arjun Banker",
"first_name": "Arjun",
"last_name": "Banker",
"link": "http://www.facebook.com/#{params[:username]}",
"location": {
"id": 114952118516947,
"name": "San Francisco, California"
},
"gender": "male"
}|
end
end
spec/support/fake_facebook.rb

57. ShamRack.at('graph.facebook.com', 443).sinatra do
get '/:username' do
%Q|{
"id": "7901103",
"name": "Arjun Banker",
"first_name": "Arjun",
"last_name": "Banker",
"link": "http://www.facebook.com/#{params[:username]}",
"location": {
"id": 114952118516947,
"name": "San Francisco, California"
},
"gender": "male"
}|
end
end
spec/support/fake_facebook.rb

58. ShamRack.at('graph.facebook.com', 443).sinatra do
get '/:username' do
%Q|{
"id": "7901103",
"name": "Arjun Banker",
"first_name": "Arjun",
"last_name": "Banker",
"link": "http://www.facebook.com/#{params[:username]}",
"location": {
"id": 114952118516947,
"name": "San Francisco, California"
},
"gender": "male"
}|
end
end
spec/support/fake_facebook.rb

59. describe FacebookWrapper, '.user_link' do
it 'retrieves user link' do
user_id = FacebookWrapper.user_id('arjun')
expect(user_id).to eq '7901103’
end
end
spec/lib/facebook_wrapper_spec.rb

60. Even Better
Dynamic
Expressive
Readable

61. gem 'vcr'

62. ENV['RAILS_ENV'] ||= 'test'
require File.expand_path('../../config/environment', __FILE__)
require 'rspec/autorun'
require 'rspec/rails’
Dir[Rails.root.join("spec/support/**/*.rb")].each { |f| require f }
ActiveRecord::Migration.check_pending! if defined?(ActiveRecord::Migration)
RSpec.configure do |config|
config.infer_base_class_for_anonymous_controllers = false
config.order = 'random’
end
WebMock.disable_net_connect!
VCR.configure do |c|
c.cassette_library_dir = 'spec/fixtures/vcr_cassettes'
c.hook_into :webmock
end
spec/spec_helper.rb

63. describe FacebookWrapper, '.user_link' do
it 'retrieves user link' do
VCR.use_cassette('fb_user_arjun') do
user_id = FacebookWrapper.user_id('arjun')
expect(user_id).to eq '7901103'
end
end
end
spec/lib/facebook_wrapper_spec.rb

64. describe FacebookWrapper, '.user_link' do
it 'retrieves user link' do
VCR.use_cassette('fb_user_arjun') do
user_id = FacebookWrapper.user_id('arjun')
expect(user_id).to eq '7901103'
end
end
end
spec/lib/facebook_wrapper_spec.rb

65. Even Better
Record API automatically
Replay responses without network
Verify responses

66. Additional Build Process
Runs outside normal test mode
Rechecks cassettes for diffs
Avoids versioning issues

67. gem 'puffing-billy'

68. Puffing-Billy
Built for in-browser requests
Allowed to record and reuse (like VCR)

69. Be brave, venture
out of ruby

70. I also like…

71. Chrome Dev Tools

72. Postman

73. HTTPie

74. Charles

75. Additional Reading
martinfowler.com/bliki/IntegrationContractTest.html
timrogers.uk/2014/07/12/discovering-private-apis-with-charles-app/
robots.thoughtbot.com/how-to-stub-external-services-in-tests
joblivious.wordpress.com/2009/02/20/handling-intermittence-how-to-
survive-test-driven-development
railscasts.com/episodes/291-testing-with-vcr

76. Bringing it all together
Testing services is crucial
If in doubt, stub it out
Determine the flexibility you want
Record responses to save time

77. Next Up
How to Consume Lots of Data
with Doug Alcorn

78. Thank you!
me@nealke.mp
neal@women.com
(I like getting email)
@neal_kemp
(I tweet)