Cloud Native Communication:
Using API Gateway and Service Mesh with Java Apps
Daniel Bryant
Product Architect, Datawire

tl;dr
▪ Moving to cloud and containers (cloud native) brings benefits and challenges
– Ingress and service-to-service communications change
▪ API gateway handles ingress traffic: you don’t control the client
▪ Service mesh handles service-to-service comms: you influence the client
▪ You can implement new comms via two patterns
– Outside-in, using an API gateway
– Balkanization, using a service mesh on a segment of services

Product Architect at Datawire, Freelance Tech Consultant and Writer
Java Champion, avid reader, conference tourist
@danielbryantuk

Motivations: Acceleration
▪ Lead time
▪ Deployment frequency
▪ Mean time to restore (MTTR)
▪ Change fail percentage
CIOs: “We want to go faster, and not fall over
(and if it breaks we want to detect and fix it fast)”

App Modernisation
▪ Refactoring, repurposing, or consolidation of heritage software to align it
more closely with current business needs
▪ Decoupling applications from infrastructure
– Moving workloads to take advantage of cloud-based (AI) services
– Retiring old systems (saving infra/hosting costs)
– Reducing operational burden (e.g. toil and security patching)

App Modernisation
▪ Microservices!
▪ Cloud!
▪ Containers!
▪ Kubernetes!

App modernisation: Not an overnight thing

End-to-end communication

End-to-end communication

API Gateway

API Gateway: Edge proxy, ingress, ADC...
▪ Exposes internal services to end-users (often via multiple domains)
▪ Encapsulates backends: k8s, VMs, bare metal etc
▪ Focused on managing ingress (“north-south”) traffic
▪ You don’t control the client

API Gateway: Self-Serve Routing & Security
▪ Self-serve routing
– Traffic routing, splitting, and shaping (to dynamic backends)
– Release functionality (A/B, canary, dark launch etc)
▪ Security
– End-user authentication/authorization
– TLS termination, rate limiting, WAF, DDoS protection, etc

API Gateway Options

API Gateway: Decisions Points
Java-based Traditional Cloud Native
Configuration language Java Scripts Kubernetes CRDs,
YAML
Typical owners Developers Ops or API team Developers & Platform
Learning curve Low-Medium Medium-High Medium-High
Performance
(anecdotal)
Low (e.g. DIY)
High (e.g. Netty)
High High (especially in
volatile envs)
Adding functionality DIY Plugins, modules Plugins, modules
Biggest pain point Maintenance
Coupling/cohesion
Configuration New workflow (DevOps?)

Ambassador config

Service Mesh

“Service mesh”, you say?
https://twitter.com/cesarTronLozai/status/1175327326218915840
https://twitter.com/wm/status/1173350339946274816

Service Mesh: Proxy mesh, Fabric model...
▪ Exposes internal services to internal consumers
▪ Encapsulates service infra: across k8s, VMs, bare metal etc
▪ Dynamic routing for service-to-service (“east-west”) traffic
▪ You generally control the client (or at least can influence this...)

Exploring end-to-end communication

Service Mesh: Three Pillars
▪ Observability
– “Golden signals”: latency, errors, traffic, saturation (USE, RED)
– Both global and service-to-service
▪ Reliability
– Abstracting health checks, retries, circuit breakers etc.
– Providing sane default to protect system
▪ Security
– Authn/z propagation, mTLS, ACLs, network segmentation

Service Mesh: Three Pillars
▪ Observability
– “Golden signals”: latency, errors, traffic, saturation (USE, RED)
– Both global and service-to-service
▪ Reliability
– Abstracting health checks, retries, circuit breakers etc.
– Providing sane default to protect system
▪ Security
– Authn/z propagation, mTLS, ACLs, network segmentation
https://www.infoq.com/podcasts/

Service Mesh Options

Service Mesh: Decisions Points
None:
Simple Service
Discovery
Java “Service Mesh”
(in-process)
Sidecar Service Mesh
(out-of-process)
Configuration Scripts Java Kubernetes CRDs, YAML
Typical owners Dev / ops Developers Platform
Learning curve Low Medium-High Medium-High
Performance
(anecdotal)
Highest High High
Comms-based
NFR support
DIY Circuit-breaking, retries,
bulkheads, timeouts
Circuit-breaking, reties,
bulkheads, timeouts
Cross-language Yes No (or not really) Yes
Biggest
pain point
Running support services,
Configuration
Running support services,
maintaining config (coupling)
Installation and
maintainence

Consul config

Lessons learned...

Typical Problems
▪ No clear use case
▪ Not working with the ops team…
▪ Turtles all the way down
▪ NFR-handling implemented
multiple places in stack

Migration tactics
▪ Outside in
– Start with a gateway
– Identify a endpoint/service
▪ Balkanization
– Start with a service mesh
– Identify a service segment
▪ Easy install
▪ Conceptually easy to understand
▪ Less intrusive for all platforms
▪ (Potentially) higher blast radius
▪ Less new functionality
▪ Potentially high value functionality
▪ “Easy” to deploy in Kubernetes
▪ Can support multi-cluster (beta)
▪ Operationally complex
▪ (Potentially) challenging to unwind
▪ Expectation management… :-)

bit.ly/2mr58C1

Explore in browser
https://instruqt.com/hashicorp/tracks/sock-shop-tutorial
Hat tip to:
Todd Radel, Nic Jackson & Eric Veld!

Conclusion
▪ Moving to cloud and containers (cloud native) brings benefits and challenges
– Ingress and service-to-service communications change
▪ API gateway handles ingress traffic: you don’t control the client
▪ Service mesh handles service-to-service comms: you influence the client
▪ You can implement new comms via two patterns
– Outside-in, using an API gateway
– Balkanization, using a service mesh on a segment of services

References
▪ Context:
– https://www.infoq.com/articles/api-gateway-service-mesh-app-modernisation/
▪ Reference:
– https://www.getambassador.io/user-guide/consul-connect-ambassador/
– https://www.getambassador.io/user-guide/consul/
– https://www.consul.io/docs/platform/k8s/ambassador.html
– https://www.hashicorp.com/blog/hashicorp-consul-supports-microsoft-s-new-service-mesh-framework
Experiment in an Instruqt sandbox: https://instruqt.com/hashicorp/tracks/sock-shop-tutorial
Code examples: https://github.com/emojify-app

Copyright © 2019 HashiCorp
Thanks!
@danielbryantuk

Copyright © 2019 HashiCorp
Bonus

App Modernisation: It’s a balancing act
Bare Metal / DIY VMs Hybrid Cloud Only
Private Cloud Only Public Cloud OnlyHybrid

https://blog.christianposta.com/microservices/api-gateways-are-going-through-an-identity-crisis/

https://blog.envoyproxy.io/service-mesh-data-plane-vs-control-plane-2774e720f7fc
Control planes and data planes
Data plane
Control plane

Control planes: Differing use cases
▪ North-south
– Unknown / untrusted clients
– Limited exposure of services (Mapping)
– Centralised ops ingress defaults + decentralised product team cfg
▪ East-west
– Dynamic service information update required (multiple sources)
– Identity required for all services (mTLS + ACLs)
– “Sane” internal defaults + decentralised dev cfg