6. “We are happy to announce
that starting in 2018, Netflix is
also making the transition to
Spring Boot as our core Java
framework, leveraging the
community’s contributions via
Spring Cloud Netflix.”
7. Platform at
T-Mobile
SPEED
➔Time to Prod: 6 Months to weeks
➔# of deployments: 10x increase
SCALABILITY
➔Avg: 300M Transactions a Day
➔Peak: 1Billion Transactions a Day
STABILITY
➔43% reduction in app response time
➔83% fewer incidents, fixed 67% faster
13. What we’ve
learned
➔ Events are the language
bridge to the business
➔ This method of
identifying bounded
contexts is a secret to
decoupled architecture
➔ “Tell don’t ask”
15. A platform built for a new way of thinking
➔Event + Microservice first
➔Team autonomy with
platform efficiency
➔Arbitrary scaling/scope
across enterprise
➔Turnkey multi-cloud
18. ➔ Global banking brand building greenfield
core banking and payments with Spring
Streams + Kafka
➔ Focus on microservices velocity required
platform first thinking
➔ Kleppman like view of Kafka: “We count on
Kafka for consistency, strict ordering,
replay, durability and auditability.”
➔ Cross cloud replication
A new continuously
delivered banking
platform
19. ➔ Turning moving packages into streaming
data with RFID, Kafka and Spring Streams
event based microservices
➔ Kafka, Kubernetes and Spring Boot in
every shipping center
➔ Multiple business microservices teams
can layer onto streaming platform to bin
pack last mile services.
➔ Prepared for unanticipated uses cases
Revolutionize our
shipping efficiency with
streaming microservices
20. PKS Managed Clusters
Messaging Middleware
Kafka
Binder Spring Data Repository
Event Driven Microservices
LTL Quote
Service
Scan RFIC
Services
RFID Triggered
Automation
Services
21. ➔ 100,00+ container build out of Spring
Streams, Kafka, key-value store
➔ Durability and consistency are critical
for potential legal actions
➔ Multi-phase stream processing with
Spring Streams leading to real-time
microservices alerting analysts
➔ Cross-cloud replication based on Kafka
➔ Continuously delivery required for real
time apps to improve accuracy and
functionality as project expands
Help secure a European
country?
22. Receiver
App
process queue
Fault tolerant
receiver pairs
staging
and
replication
Apps
Stream
Workers
Data
Enrichment
Stream
Workers
Data
Enrichment
Stream
Workers
Data
Enrichment
process queue
Stream
Workers
Data
Classification
Stream
Workers
Data
Classification
Stream
Workers
Data
Classification
buffer queue
S3 RAW Store
Receiver
App
X.000 Channel
Streams
RDBMS Store
3 DC
KAFKA
Replication
NoSQL
Store
Index
Store
23. ➔ Mainframe and monolithic RDBMS data
teams often the last to move to
continuous delivery
➔ CDC, Event Shunting, patterns
emerging allow streaming data platform
teams to offer mainframe and legacy
RDBMS events to microservices teams
➔ “AirBnB Pattern” growing across
enterprises
➔ Each team can build appropriate
persistence and achieve multi-DC
replication with streaming platform
Let’s empower
pharmacy microservices
developers while
evolving our legacy?
26. Spring Cloud Stream
deals with the Kafka
scaffolding, so you
don’t have to
The power of Kafka streams
for developers
➔ Rapid on-ramp for Kafka Streams
consumption
➔ Simplifies construction of Event-
Driven Stateful Microservices
➔ Focus on your processing logic not
on configuration
➔ Full support of all Kafka streams
functionality
27. We are all in…
doubling down on a platform
Spring Kafka ++ PCC Kafka Integration Pivotal Function Service
(Knative ++ )
PAS Service Brokers
(Custom today)
PKS Confluent Operator
Testing and Integration
AppTX + Kafka CDC ++
28. Call to action
➔ Tim Berglund and Josh Long talk on
Spring and Kafka
➔ Contribute to GitHub features and issues
for Spring Stream
➔ Come visit us at the booth to talk
streaming microservices
Hinweis der Redaktion
Excited to be here, Kafka community is one of my favorite places to study the evolution of enterprise organization and architecture.
Ended up here in part because through customer experience came to the personal belief that Spring Boot and Kafka are the two must have components of a flexible yet efficient enterprise platform
Spoiler alert: my experience leads me to believe every enterprise needs a focus on broad potentials of Sring Boot + Kafka
New ways of thinking, organizing, and architecture all at once; generational perfect storm is hitting enterprises today.
-
The business is aware that a broad change like this will require a platform enablement to achieve at scale; we must partner with the business to communicate at all times how these org/architecture changes can unleash them.
Yes, Spring Boot has 60M downloads but it’s one of a few technologies used at scale in enterprise and cutting edge cloud native companies like Netflix.
Netflix was one of the early champions of radically changing org/architecture and using continuous delivery to outpace a whole industry.
Netflix counts on Spring to abstract distributed infra scaffolding for their teams….even though they wrote most of it
Modern software infrastructure platforms must do three things:
Radical gains in continuous delivery, team autonomy
Economies of scope across most teams
Dramatic efficiency gains which enable a small platform team to deliver for for thousands
“Mosty enterprises are taking step one” The first one should say “IBM” for App to show monolith start if possible. Got it.
Want to do a fresh view of each of these with maybe a spring boot icon for each app.
Want to show that step one was to break apart the app tier; also likely adding a cache above the DB for the microservice. Happy to discuss this is one of the critical graphics to show the progression of the story.
Winning/helping an enterprise is always evolutionary and very rarely binary to “winning the logo”.
Outcomes, outcomes outcomes---driven by customer demand to double down on the success they have had.
“I want this operating model for EVERYTHING”
The Kafka community was working on this problem from a different but vital perspective.
Convergent evolution ….
Paul Graham on programing languages…..
Stubhub
Event Storming; even a relatively recent born on the web company but pre microservices and CI/CD
“Tell don’t ask”
-Without “tell don’t ask” incidental coupling across microservices can easily creep back in...
-we see success with a blend of pane 2-3….
Cool Thanks!
--The most important lines are the human to human connections here.
--Platform + Slack
--The platform minimizes toil
--Make it FUN
--Common tools keep communication and experience meaningful across teams
Martin Kleppman like API interfaces where the API/web application immediately writes to Kafka with everything else as a kafka consumer.
Conversation about microservices enablement with platforms; two big bets, Kafka and Spring Boot.
“Be prepared for unanticipated use cases in the future (for free)”
“Be prepared for unanticipated use cases in the future (for free)
Fed Ex edge streaming ingest project (unnamed)
In the Freight business, Less-Than-Load (LTL) and Last Mile are the two most critical and hard problems to solve and FedEx Freight in one of several companies throwing their hat in the ring to provide this service (eg: Customers buying appliances and furniture online and companies that will deliver this and assemble if needed)
Initial use case is to provide a “centrally managed” platform across 300+ FedEx Freight Service Centers.
The software stack running in each service center includes vSphere on VxRail, BOSH, PKS, Kafka, K/V Store TBD (Cassandra preferred and tested), Spring Boot [more detail here on scale]
Data generated by RFID scanners which are placed in the ceiling of service centers and scan RFID codes in their cone of vision. These scanners talk to Spring Boot apps which send the data to Kafka queues
Data produced locally is stored for approx. 36 hours and 10 days for some other data
Data capacity is approx. 1TB per day
Data is used to make critical decisions like which pallets to route where using which trucks. Kafka is the “glue” getting data from and pushing data to Spring Boot apps, while also pushing data into Cassandra for search and other processing (analytics)
Spring Boot + Kafka for, price offer generation, data ingest, and real time automation applications!
European crime agency. Developed for communication surveillance to support real time surveillance as well mandated analytics work.
This solution should never lose data or parts of streams, this could invalidate the consistency and detectability
Thousands of streams needs to be buffered, classified, enriched and stored to be replicated to create a data loss free solution
Using microservices gives the ability to plug in new modules for encoding and enrichment in real time
Kafka is connecting the processes together giving the ability to asynchronous handle thousands over thousands messages delivered as streams
Most of the microservices are based on spring boot using the spring kafka consumer and producer
EKafka CDC Use Cases can apply toenterprises
Cache Invalidation
Search Indexing
Offline Processing
Signaling (triggers)
Autoconfiguration, monitoring, metrics
These slides have attempted to clarify how Spring's projects wrap and compliment Kafka, and to demonstrate the value that this brings to developers in terms of raising the value line.
This would be a build that starts with PCC.
Spring Cloud Stream Loves Developers & Developers Love Spring Cloud DataFlow
I've tried to demonstrate how Spring's projects wrap and compliment Kafka, and to demonstrate the value that this brings to developers in terms of raising the value line.
European crime agency. Developed for communication surveillance to support real time surveillance as well mandated analytics work.
This solution should never lose data or parts of streams, this could invalidate the consistency and detectability
Thousands of streams needs to be buffered, classified, enriched and stored to be replicated to create a data loss free solution
Using microservices gives the ability to plug in new modules for encoding and enrichment in real time
Kafka is connecting the processes together giving the ability to asynchronous handle thousands over thousands messages delivered as streams
Most of the microservices are based on spring boot using the spring kafka consumer and producer