Kafka Summit NYC 2017 - Simplifying Omni-Channel Retail at Scale
•
1 gefällt mir•4,755 views
Presentation by Aaron Strey, Target
Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Was ist angesagt?
Was ist angesagt? (20)
Ähnlich wie Kafka Summit NYC 2017 - Simplifying Omni-Channel Retail at Scale
Ähnlich wie Kafka Summit NYC 2017 - Simplifying Omni-Channel Retail at Scale (20)
Mehr von confluent
Mehr von confluent (20)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
Kafka Summit NYC 2017 - Simplifying Omni-Channel Retail at Scale
- 1. Simplifying Omni-Channel Retail at Scale Aaron Strey – Principal Engineer @Target Twitter - @Strey203
- 2. Kafka Use Cases @Target •Log aggregation •Threat detection •Click stream •Business event messaging
- 3. A little business context
- 4. Target has •1,806 stores in the United States •38 distribution centers in the U.S •323,000 team members worldwide •Global locations in India •Target.com is the fourth most-visited retail website in the U.S. with more than 26 million unique visitors each month on average
- 10. TGT DC Cloud TGT DC Cloud MMMM MM MM MM MM Cloud Cloud MM MM MM MM Cluster Architecture
- 11. Immutable deployments One large shared cluster. Occasional isolation. 10K – 20K fetch consumer requests per second SSL authN/authZ + replication What makes our deployment unique Up to 300+ topics per cluster and growing Compaction widely used KafkaStreams Diversity of clients Audit Capabilities Exactly once semantics
- 13. 10K – 20K fetch consumer requests per second
- 14. If you are running 0.8.x or 0.9.x, compact with caution
- 15. Enable the visibility necessary to support a diverse set of clients.
- 18. Re-inventing log aggregation and analysis Indexing terabytes of logs to ssd’s for low latency analysis is impractical
- 19. What if you could query the log stream directly from Kafka as easily as I do from Elastic today?
- 21. Thank you
- 22. ?
Hinweis der Redaktion
- I want to start with a little business context for a few reasons Critical role Apache Kafka and the ecosystem around play at Target Transformational impact a stream data platform can have on a company Awesome engineering problems we are solving at Target that you could help with!
- Target has 1,806 stores in the United States 38 distribution centers in the United States 323,000 team members worldwide 70 + billion in revenue online business at target.com global locations in India To date, Target.com is the fourth most-visited retail Website in the U.S. with more than 26 million unique visitors each month on average.
- The title of the talk is simplifying omni-channel retail at scale. Omni channel is a bit of a loaded word. I don’t want to start a debate about what it does and doesn’t mean. But I will discuss what how the word omni channel has evolved to me. Now if you appreciate the history of retail, you will understand that brick and mortar shops initially stood up ecommerce capabilities completely separate from their brick and mortar operations. Items might have had different identifiers, different promotional models, different taxonomies and on and on and on. And initially omni channel to me meant unifying those two views so that as guests engaged through stores and online systems the experience was consistent. It started as providing a consistent view into our ecosystem. Providing microservices that provided a single interface into both our ecommerce and brick and mortar systems. After we did that it was clear we still had more to do. And what omni means to me has become a much more fundamental concept that encompasses the way we think about everything we do. It goes beyond a data mapping exercise and the merging of the teams that manage promotions for stores and .com. That is an important step, but it is not the final step. The next step in becoming a true omni channel retailer is starting to leverage all of your assets. All of your physical locations, all of your people, and yes all of systems to get product to people as cheap as and fast as possible. And there is very important distinction between those two concepts. Enabling an omni channel feel when guests engage in your digital channels doesn’t matter if it takes you 5 days to ship them a pair of socks when you have that exact same pair sitting, or could have the exact same pair with the right systems, sitting a store a mile away from their house. So true omni channel retail to me, now means rethinking your infrastructure from the ground up to best serve the ever changing retail landscape.
- To put it bluntly, we are slow and we have too much inventory. This used to be a difficult thing for us to admit to ourselves. We admit it because we we are actively doing something about it. In the past, our supply chain moved big case packs of products, and the systems built to enable that movement were batch oriented. The product were shipped in batches, the systems triggered replenishment in batches, and the truck drove them to the store in batches. Now a portion of our supply chain will always move cases, but to replenish our stores and manage growing digital demand we know we need to start moving eaches.
- The concept is really pretty simple. We sell one, we replenish one. We have systems in place now that have the capability to cut a transfer order in our of our DCs within seconds of point of sale transaction. When we move with that much speed and precision out of stocks go down, safety stocks go down, and speed goes way up. and we have zero inventory in our back rooms. What do you do with all of that backroom space you have freed up? Use it to store online only items or additional inventory to ship from stores. End of day what this does for us is increases reliability and enables us to ship faster at a lower cost. By optimizing for speed and flexibilty first rather than cost. We end up optimizing our agility, the experience, and cost is optimized as a result.
- If I can reduce the amount of inventory in the system I free up space in my stores. I then have the opportunity to leverage the real estate I already own to act as mini fulfillment centers. A lot of the capital invested in disrupting the retail space is being invested to build out geographically diverse physical infrastructure close to consumers. We can significantly reduce the cost of fulfilling online orders and dramatically decrease the lead time to delivery.
- Now, to what this means tech wise and why I am here at Kafka Summit. Apache Kafka is a critical component of the technical infrastructure enabling us transform as a company. w/o a reliable streaming platform… At a recent internal update from our supply chain partners they made an off hand comment that there has not been a single issue related to Kafka since we began this reivention. There are a couple of reasons why Kafka is right. The technical merits of which are clearly highlighted on the slide There is a little more personal story about how my relationship with Kafka began 2 ½ - 3 years ago decision made to operate as a platform
- Implementation is pretty straight forward. Running across various data centers including those run by target and public cloud providers An overwhelming majority of our core systems reside in our data center Less often mirror data in from the cloud Mirror maker has gotten the job done There is a key component missing from the picture that was hard to fit in, 1800 stores and 38 distribution centers High availability use cases
- Kafka Operations Portal