2. Me(tosin)
Sinclair Spectrum in 1983
...
Co-founded Metosin 2012
20 developers, Tampere & Helsinki / Finland
Projects, Startups and Design
FP, mostly Clojure/Script
https://github.com/metosin
3. ClojuTRE 2019 (8th)
September 26-27th 2019, Helsinki
FP & Clojure, 300++ on last two years
Basic ïŹights & hotels for speakers
Free Student & Diversity tickets
Call For Speakers Open
https://clojutre.org
4. This talk
Routing and dispatching
Reitit, the library
Reitit, the framework
Performance
Takeaways
5. Routing in Clojure/Script
Lot's of options: Ataraxy, Bide, Bidi, Compojure,
Keechma, Pedestal, Silk, Secretary, ...
Something we need in mostly all the (web)apps
Mapping from path -> match
Mapping from name -> match (reverse routing)
HTTP (both server & browser)
Messaging
Others
6. Dispatching in Clojure/Script
Here: composing end executing functionality in
the request-response pipeline
Common patterns
Middleware (Ring, nREPL, ..)
Interceptors (Pedestal, Re-Frame, ..)
Controllers (Keechma)
Promises
Sync & Async
7. New Routing Library?
More data-driven
Leveraging clojure.spec
Built for performance
Friendly & explicit API
Reach: browser, JVM, Node, Others
Both routing and dispatching
The Yellow (Hiccup) Castle of Routing?
10. Basics
reitit.core/router to create a router
Matching by path or by name (reverse routing)
Functions to browse the route trees
(defprotocol Router
(router-name [this])
(routes [this])
(compiled-routes [this])
(options [this])
(route-names [this])
(match-by-path [this path])
(match-by-name [this name] [this name path-params]))
18. What is in the route data?
Can be anything, the Router doesn't care.
Returned on successful Match
Can be queried from a Router
Build your own interpreters for the data
A Route First Architecture: Match -> Data -> React
(r/match-by-path router "/api/admin/db")
;#Match{:template "/api/admin/db",
; :data {:interceptors [::api ::db]
; :roles #{:db-admin}},
; :result nil,
; :path-params {},
; :path "/api/admin/db"}
19. Example use cases
Authorization via :roles
Frontend components via :view
Dispatching via :middleware and :interceptors
Stateful dispatching with :controllers
Coercion via :parameters and :coercion
Run requests in a server io-pool with :server/io
Deploy endpoints separately to :server/target
21. Route Data Validation
Route Data can be anything -> data spaghetti?
Leverage clojure.spec at router creation
DeïŹne and enforce route data specs
Fail-fast, missing, extra or misspelled keys
With help of spec-tools and spell-spec
Closed specs coming to Spec2 (TBD)
25. Failing Fast
During static analysis (e.g. linter)
At compile-time (macros, defs)
At development-time (schema/spec annos)
At creation-time
At runtime
At runtime special condition
(c) https://www.artstation.com/kinixuki
28. Framework
You call the library, but the framework calls you
Reitit ships with multiple Routing Frameworks
reitit-ring Middleware-dispatch for Ring
reitit-http Async Interceptor-dispatch for http
reitit-pedestal Reitit Pedestal
reitit-frontend (Keechma-style) Controllers,
History & Fragment Router, helpers
30. Ring Routing
A separate lightweight module
Routing based on path and :request-method
Adds support for :middleware dispatch
chain is executed after a match
ring-handler to create a ring-compatible handler
Supports Both sync & async
Supports Both JVM & Node
No magic, no default middleware
32. Accessing route data
Match and Router are injected into the request
Components can read these at request time and
do what ever they want, "Ad-hoc extensions"
Pattern used in Kekkonen and in Yada
(defn wrap-roles [handler]
;; roles injected via session-middleware
(fn [{:keys [roles] :as request}]
;; read the route-data at request-time
(let [required (-> request (ring/get-match) :data :roles)]
(if (and (seq required)
(not (set/subset? required roles)))
{:status 403, :body "forbidden"}
(handler request)))))
33. Middleware as data
Ring middleware are opaque functions
Reitit adds a ïŹrst class values, Middleware records
Recursive IntoMiddleware protocol to expand to
Attach documentation, specs, requirements, ...
Can be used in place of middleware functions
Zero runtime penalty
(defn roles-middleware []
{:name ::roles-middleware
:description "Middleware to enforce roles"
:requires #{::session-middleware}
:spec (s/keys :opt-un [::roles])
:wrap wrap-roles})
34. Compiling middleware
Each middleware knows the endpoint it's mounted to
We can pass the route data in at router creation time
Big win for optimizing chains
(def roles-middleware
{:name ::roles-middleware
:description "Middleware to enforce roles"
:requires #{::session-middleware}
:spec (s/keys :opt-un [::roles])
:compile (fn [{required :roles} _]
;; unmount if there are no roles required
(if (seq required)
(fn [handler]
(fn [{:keys [roles] :as request}]
(if (not (set/subset? required roles))
{:status 403, :body "forbidden"}
(handler request))))))})
35. Partial Specs Example
"All routes under /account should require a role"
;; look ma, not part of request processing!
(def roles-defined
{:name ::roles-defined
:description "requires a ::role for the routes"
:spec (s/keys :req-un [::roles])})
["/api" {:middleware [roles-middleware]} ;; behavior
["/ping"] ;; unmounted
["/account" {:middleware [roles-defined]} ;; :roles mandatory
["/admin" {:roles #{:admin}}] ;; ok
["/user" {:roles #{:user}}] ;; ok
["/manager"]]] ;; fail!
36. Middleware chain as data
Each endpoint has it's own vector of middleware
Documents of what is in the chain
Chain can be manipulated at router creation time
Reordering
Completing
Interleaving
Interleaving a request diff console printer:
reitit.ring.middleware.dev/print-request-diffs
37.
38. Data deïŹnitions (as data)
The core coercion for all (OpenAPI) paramerer &
response types ( :query , :body , header , :path etc)
Separerate Middleware to apply request & response
coercion and format coercion errors
Separate modules, spec coercion via spec-tools
["/plus/:y"
{:get {:parameters {:query {:x int?},
:path {:y int?}}
:responses {200 {:body {:total pos-int?}}}
:handler (fn [{:keys [parameters]}]
;; parameters are coerced
(let [x (-> parameters :query :x)
y (-> parameters :path :y)]
{:status 200
:body {:total (+ x y)}}))}}]
41. Going Async
Interceptors are much better ïŹt for async
reitit-http uses the Interceptor model
Requires an Interceptor Executor
reitit-pedestal or reitit-sieppari
Sieppari supports core.async , Manifold and Promesa
Pedestal is more proven, supports core.async
Target Both JVM & Node (via Sieppari)
45. Route-driven frameworks
Routing and dispatching is separated, middleware (or
interceptors) are applied only after a match
Each endpoint has a unique dispatch chain
Each component can be compiled and optimized against
the endpoint at creation time
Components can deïŹne partial route data specs that
only effect the routes they are mounted to.
We get both Performance & Correctness
... this is Kinda Awesome.
(c) https://www.artstation.com/kinixuki
47. Performance
how can we make compojure-api faster?
we moved from Clojure to GO because of perf
How fast are the current Clojure libraries?
How fast can we go with Java/Clojure?
48. Measuring Performance
Always measure
Both micro & macro benchmarks
In the end, the order of magnitude matters
Lot's of good tools, some favourites:
clojure.core/time
criterium
com.clojure-goes-fast/clj-async-profiler
com.clojure-goes-fast/clj-java-decompiler
https://github.com/wg/wrk
49. (def defaults {:keywords? true})
(time
(dotimes [_ 10000]
(merge defaults {})))
; "Elapsed time: 4.413803 msecs"
(require '[criterium.core :as cc])
(cc/quick-bench
(merge defaults {}))
; Evaluation count : 2691372 in 6 samples of 448562 calls.
; Execution time mean : 230.346208 ns
; Execution time std-deviation : 10.355077 ns
; Execution time lower quantile : 221.101397 ns ( 2.5%)
; Execution time upper quantile : 245.331388 ns (97.5%)
; Overhead used : 1.881561 ns
50. (require '[clj-async-profiler.core :as prof])
(prof/serve-files 8080) ;; serve the svgs here
(prof/profile
(dotimes [_ 40000000] ;; ~10sec period
(merge defaults {})))
51. Reitit performance
Designed group up to be performant
Perf suite to see how performance evolves
Measured against Clojure/JavaScript/GO Routers
Performance toolbox:
Optimized routing algorithms
Separation of creation & request time
The Usual Suspects
52. Routing algorithms
When a router is created, route tree is inspected
and a best possible routing algorith is chosen
No regexps, use linear-router as a last effort
lookup-router , single-static-path-router
trie-router , linear-router
mixed-router , quarantine-router
53. trie-router
For non-conïŹicting trees with wildcards
First insert data into Trie AST, then compile it into
fast lookup functions using a TrieCompiler
On JVM, backed by a fast Java-based Radix-trie
60. Java Trie
Set of matchers deïŹned by the TrieCompiler
Order of magnitude faster than the original impl
@Override
public Match match(int i, int max, char[] path) {
boolean hasPercent = false;
boolean hasPlus = false;
if (i < max && path[i] != end) {
int stop = max;
for (int j = i; j < max; j++) {
final char c = path[j];
hasPercent = hasPercent || c == '%';
hasPlus = hasPlus || c == '+';
if (c == end) {
stop = j;
break;
}
}
final Match m = child.match(stop, max, path);
if (m != null) {
m.params = m.params.assoc(key, decode(new String(path, i, stop - i), hasPercent, hasPlus));
}
return m;
}
return null;
}
61. The Usual Suspects
Persistent Data Structures -> Records, Reify
Multimethods -> Protocols
Map Destructuring -> Manually
Unroll recursive functions ( assoc-in , ...)
Too generic functions ( walk , zip , ...)
Dynamic Binding
Manual inlining
Regexps
63. RESTful api test
50+ routes, mostly wildcards
Reitit is orders of magnitude faster
220ns vs 22000ns, actually matters for busy sites
64.
65. Looking out of the box
https://github.com/julienschmidt/httprouter
One of the fastest router in GO
(and source of many of the optimizations in reitit)
In Github api test, Reitit is ~40% slower
That's good! Still work to do.
67. Current Status
Most Clojure libraries don't even try to be fast
And that's totally ok for most apps
Compojure+ring-defaults vs reitit, with same response
headers, simple json echo
;; 10198tps
;; http :3000/api/ping
;; wrk -d ${DURATION:="30s"} http://127.0.0.1:3000/api/ping
(http/start-server defaults-app {:port 3000})
;; 48084tps
;; http :3002/api/ping
;; wrk -d ${DURATION:="30s"} http://127.0.0.1:3002/api/ping
(http/start-server reitit-app {:port 3002})
=> that's 5x more requests per sec.
71. Web Stacks
We know Clojure is a great tool for building web stuff
We can build a REALLY fast server stack for Clojure
aleph or immutant-nio as web server (nio, zero-copy)
reitit -based routing
Fast formatters like jsonista for JSON
next.jdbc (or porsas ) for database access
Good tools for async values & streams
lot's of other important components
Simple tools on top, the (coastal) castles?
72. Clojure Web & Data Next?
We have Ring, Pedestal, Yada & friends
We have nice templates & examples
Ring2 Spec? Spec for interceptors?
New performant reference architecture?
Bigger building blocks for rapid prototypes?
Making noice that Clojure is kinda awesome?
Community built Error Formatter?
Data-driven tools & inventories?
clojure.spec as data?
74. Reitit bubbin' under
Support for OpenAPI3
JSON Schema validation
Spec2 support (when it's out)
Developer UI with remote debugger
More Batteries & Guides (frontend)
(Help make next.jdbc java-fast)
Sieppari.next
(c) https://www.artstation.com/kinixuki
75. Wrap-up
Reitit is a new routing library & framework
Embrace data-driven design, all the way down
Clojure is a Dynamic Language -> fail fast
clojure.spec to validate & transform data
Understand performance, test on your libraries
We can build beautiful & fast things with Clojure
Try out https://github.com/metosin/reitit
Chatting on #reitit in Slack
(c) https://www.artstation.com/kinixuki