Roll Your Own API Management Platform with nginx and Lua

1. Roll Your Own API Management Platform with nginx and Lua Jon Moore Senior Fellow, Comcast Cable @jon_moore

2. access control

3. capacity management (“rate limiting”)

4. capacity management (“rate limiting”)

5. HTTP Proxy custom logic!

6. Lua

7. --- Validates the OAuth signature! -- @return Const.HTTP_UNAUTHORIZED if either the key or signature is invalid! -- this method is internal and should not be called directly! function _M.validate_signature(self)! local headers = self.req.get_oauth_params()! local key = headers[Const.OAUTH_CONSUMER_KEY]! local keyconf = self.conf.keys[key]! if keyconf == nil then! return {! code = Const.HTTP_UNAUTHORIZED! error = Const.ERROR_INVALID_CONSUMER_KEY! }! end! ! local sig = get_hmac_signature(self.req, keyconf.secret)! if sig ~= headers[Const.OAUTH_SIGNATURE] then! return {! code = Const.HTTP_UNAUTHORIZED,! error = Const.ERROR_INVALID_SIGNATURE! }! end! end!

8. Lua < 3k LOC

9. testing

10. function TestOAuth1:test_reject_request_when_signature_invalid()! local header = Header:new()! header[Const.OAUTH_SIGNATURE] = “invalid”! local req = Req:new({oauth_params = header })! local conf = Conf:new()! local oauth = OAuth1:new(conf, req)! ! local res = oauth:authorize()! assertEquals(res.code, Const.HTTP_UNAUTHORIZED)! assertEquals(res.error, Const.ERROR_INVALID_SIGNATURE)! end! ! lu = LuaUnit.new()! Lu:setOutputType(“tap”)! os.exit(lu:runSuite())!

11. ngx.log(ngx.ERR, “oops”)!

12. ngx.log(ngx.ERR, “oops”)!

13. function get_oauth_params_from_auth_header()!

14. function get_oauth_params_from_auth_header(env)! env = env or ngx! local auth_hdrs = env.req.get_headers()[“Authorization”]! ...!

17. function TestRequest:test_retrieve_oauth_params_from_header()! local header = [[Oauth realm=“example.com”, ]]! .. [[oauth_consumer_key=“mykey”,]]! .. [[oauth_version=“1.0”]]! local ngx = StubNgx:new({ Authorization = header })! local res = get_oauth_params_from_auth_header(ngx)! assertEquals(“1.0”, res.oauth_version)! ...!

22. test harness

23. nginx-oauth1.conf! test harness

27. def spec_using_valid_oauth_credentials(self, harness)! auth = OAuth1(“mykey”, “mysecret”)! body_data = “{‘function’: ‘tick’}”! harness.reset_data()! response = requests.post(root_url, data=body_data, auth=auth,! headers={‘Content-Type’:! ‘application/json’})! assert response.status_code == 200! assert “Authorization” in harness.forwarded_headers! assert “Oauth” in harness.forwarded_headers[“Authorization”]! assert harness.forwarded_body == body_data!

34. capacity management

35. N = XR # concurrent requests transaction rate response time

36. API Mgmt client origin 1s2 req/s

37. API Mgmt client origin 1s2 req/s N = XR = 2 req/s × 1s = 2 req

42. API Mgmt client origin 1s2 req/s

49. API Mgmt client origin 10s2 req/s ✗ N = XR = 2 req/s × 10s = 20 req

50. API Mgmt client origin 10s2 req/s ✗ ✗ N = XR = 2 req/s × 10s = 20 req

51. access_by_lua ...! log_by_lua ...! +1 -1

52. deployment

53. Version Control templates vault (keys) nginx.conf!ssh!

54. architecture

55. HAProxy HAProxy VIP . . . <API>

56. DC1 DC2 DC3 VIP VIP VIP entry-vip-dc1. A 10.1.0.1! <foo> <foo> <bar> <bar> foo-dc1. CNAME entry-vip-dc1.! foo. CNAME foo-dc1.! (GSLB) entry-vip-dc2. A 10.2.0.1! entry-vip-dc3. A 10.3.0.1!

62. Roll Your Own API Management with nginx and Lua • nginx + Lua => great for HTTP middleware with a small amount of custom logic • Automated test and deployment pipeline with Vagrant, Python, and Ansible • Concurrent request limiting, not rate limiting • Network architecture with operational flexibility Presentation title (optional)67