2014-08-27 | The Twitter Stack (Vienna Scala User Group)

•

3 likes•1,616 views

Dominik Gruber

Introduction to various open source projects by Twitter. Talk given at Vienna Scala User Group on Aug. 27, 2014.

Technology

The Twitter Stack
Finagle, Ostrich, Zipkin,…
Dominik Gruber, @the_dom
Vienna Scala User Group – Aug. 27, 2014

Dominik Gruber • @the_domTwitter Stack
Agenda
• Introduction
• Prominent Adopters
• Finagle
• Ostrich
• Zipkin
• Iago
• Finatra

Dominik Gruber • @the_domTwitter Stack
Introduction
• Twitter started to move oﬀ Ruby on Rails in 2010
• Currently still on Scala 2.9

Dominik Gruber • @the_domTwitter Stack
Prominent Adopters
• Box
• eBay
• Etsy
• Foursquare
• Gigya
• Nest
• Pinterest
• Rdio
• SoundCloud
• Tumblr

Dominik Gruber • @the_domTwitter Stack
Finagle
“Finagle is an extensible RPC system for the
JVM, used to construct high-concurrency
servers. Finagle implements uniform client
and server APIs for several protocols, and is
designed for high performance and
concurrency. Most of Finagle’s code is
protocol agnostic, simplifying the
implementation of new protocols.”

Dominik Gruber • @the_domTwitter Stack
Finagle provides…
• Connection pools
• Failure detection
• Failover strategies
• Load balancers
• Back-pressure techniques

Dominik Gruber • @the_domTwitter Stack
Finagle…
• publishes standard statistics, logs, and exception
reports
• supports distributed tracing across protocols
• optionally uses ZooKeeper for cluster management
• supports common sharding strategies

Dominik Gruber • @the_domTwitter Stack
Modules (Examples)
• finagle-http
• finagle-memcached
• finagle-mysql
• finagle-protobuf
• finagle-redis
• finagle-thrift

Dominik Gruber • @the_domTwitter Stack
Basic abstractions
• Future
• Service
• Filter

Dominik Gruber • @the_domTwitter Stack
Ostrich

Dominik Gruber • @the_domTwitter Stack
Ostrich
• A stats collector & reporter
• Collect runtime statistics (counters, gauges, metrics,
and labels)
• Report those statistics through a simple web
interface or into log ﬁles
• Similar to Metrics 
https://github.com/dropwizard/metrics

Dominik Gruber • @the_domTwitter Stack
Zipkin
“Zipkin is a distributed tracing system that
helps us gather timing data for all the
disparate services at Twitter. It manages both
the collection and lookup of this data through
a Collector and a Query service.”

Dominik Gruber • @the_domTwitter Stack
Zipkin

Dominik Gruber • @the_domTwitter Stack
Zipkin’s Architecture

Dominik Gruber • @the_domTwitter Stack
Zipkin
• “Scribe” to transport traces to Zipkin and Hadoop
• Data stored in either Cassandra, Redis, HBase,
MySQL, PostgreSQL, SQLite, or H2

Dominik Gruber • @the_domTwitter Stack
Iago

Dominik Gruber • @the_domTwitter Stack
Iago
• A load generator that replays production or synthetic
traﬃc
• Iago supports arbitrarily high rates of traﬃc via built-
in support for creating cluster

Dominik Gruber • @the_domTwitter Stack
Protocols
• HTTP
• Thrift
• Memcache
• Kestrel (A queueing system from Twitter)
• UDP

Dominik Gruber • @the_domTwitter Stack
Finatra

Dominik Gruber • @the_domTwitter Stack
Features
• Fully Async using Futures
• Mustache Templates
• Detailed Metrics (p99,p999, etc)
• Proﬁle/Debug over HTTP
• SSL/TLS Support
• Deploys with a single jar

Dominik Gruber • @the_domTwitter Stack
There’s more…
•
• Summingbird
• https://github.com/twitter

Dominik Gruber • @the_domTwitter Stack
Q & A

Dominik Gruber • @the_domTwitter Stack
Source
• http://blog.oskarsson.nu/post/40196324612/the-
twitter-stack 
https://blog.twitter.com/2011/ﬁnagle-a-protocol-
agnostic-rpc-system
• https://www.youtube.com/watch?v=whuT_-cFb5I 
https://www.youtube.com/watch?v=QdER-Zl_YCM
• https://github.com/twitter 
https://github.com/ﬁnagle

Viewers also liked

Red Hat JBoss Fuse integration services delivers cloud-based integration based on OpenShift by Red Hat to deliver continuous delivery of tested, production-ready integration solutions. Utilizing a drag and drop, code-free UI and combining that with the integration power of Apache Camel, Fuse integration services is the next generation iPaaS. In this session, we'll walk you through why iPaaS is important, the current Fuse integration services roadmap, and the innovation happening in open source community projects to make this a reality.

IPaaS 2.0: Fuse Integration Services (Robert Davies & Keith Babo)

Red Hat Developers

Microservices architecture examples

Channy Yun

Distributed Tracing with OpenTracing, ZipKin and Kubernetes

Container Solutions

Microservices Tracing With Spring Cloud and Zipkin @CybercomDEV

Marcin Grzejszczak

Tracing Microservices with Zipkin

takezoe

Microservices Tracing with Spring Cloud and Zipkin

Marcin Grzejszczak

By Tom Wilkie, delivered at London Microservices User Group on 2/12/15 The rise of microservice-based applications has had many knock-on effects, not least on the complexity of monitoring your application. Order-of-magnitude increase in the number of moving parts and rate of change of the application require us to reassess traditional monitoring techniques. In this talk we will discuss some different approaches to monitoring, visualising and tracing containerised, microservices-based applications. We’ll present different techniques to some of the emergent problems, and try not to rant too much.

Monitoring Microservices

Weaveworks

Technical Seminar PPT

Kshitiz_Vj

Dockercon State of the Art in Microservices

Adrian Cockcroft

Advantages and Disadvantages of Technology

Pave Maris Cortez

Implementing microservices tracing with spring cloud and zipkin (spring one)

Reshmi Krishna

Technology powerpoint presentations

ismailraesha

Viewers also liked (12)

IPaaS 2.0: Fuse Integration Services (Robert Davies & Keith Babo)

Microservices architecture examples

Distributed Tracing with OpenTracing, ZipKin and Kubernetes

Microservices Tracing With Spring Cloud and Zipkin @CybercomDEV

Tracing Microservices with Zipkin

Microservices Tracing with Spring Cloud and Zipkin

Monitoring Microservices

Technical Seminar PPT

Dockercon State of the Art in Microservices

Advantages and Disadvantages of Technology

Implementing microservices tracing with spring cloud and zipkin (spring one)

Technology powerpoint presentations

Recently uploaded

Keynote 2: APIs in 2030: The Risk of Technological Sleepwalk Paolo Malinverno, Growth Advisor - The Business of Technology Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...

apidays

💉💊+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHABI}}+971581248768 +971581248768 Mtp-Kit (500MG) Prices » Dubai [(+971581248768**)] Abortion Pills For Sale In Dubai, UAE, Mifepristone and Misoprostol Tablets Available In Dubai, UAE CONTACT DR.Maya Whatsapp +971581248768 We Have Abortion Pills / Cytotec Tablets /Mifegest Kit Available in Dubai, Sharjah, Abudhabi, Ajman, Alain, Fujairah, Ras Al Khaimah, Umm Al Quwain, UAE, Buy cytotec in Dubai +971581248768''''Abortion Pills near me DUBAI | ABU DHABI|UAE. Price of Misoprostol, Cytotec” +971581248768' Dr.DEEM ''BUY ABORTION PILLS MIFEGEST KIT, MISOPROTONE, CYTOTEC PILLS IN DUBAI, ABU DHABI,UAE'' Contact me now via What's App…… abortion Pills Cytotec also available Oman Qatar Doha Saudi Arabia Bahrain Above all, Cytotec Abortion Pills are Available In Dubai / UAE, you will be very happy to do abortion in Dubai we are providing cytotec 200mg abortion pill in Dubai, UAE. Medication abortion offers an alternative to Surgical Abortion for women in the early weeks of pregnancy. We only offer abortion pills from 1 week-6 Months. We then advise you to use surgery if its beyond 6 months. Our Abu Dhabi, Ajman, Al Ain, Dubai, Fujairah, Ras Al Khaimah (RAK), Sharjah, Umm Al Quwain (UAQ) United Arab Emirates Abortion Clinic provides the safest and most advanced techniques for providing non-surgical, medical and surgical abortion methods for early through late second trimester, including the Abortion By Pill Procedure (RU 486, Mifeprex, Mifepristone, early options French Abortion Pill), Tamoxifen, Methotrexate and Cytotec (Misoprostol). The Abu Dhabi, United Arab Emirates Abortion Clinic performs Same Day Abortion Procedure using medications that are taken on the first day of the office visit and will cause the abortion to occur generally within 4 to 6 hours (as early as 30 minutes) for patients who are 3 to 12 weeks pregnant. When Mifepristone and Misoprostol are used, 50% of patients complete in 4 to 6 hours; 75% to 80% in 12 hours; and 90% in 24 hours. We use a regimen that allows for completion without the need for surgery 99% of the time. All advanced second trimester and late term pregnancies at our Tampa clinic (17 to 24 weeks or greater) can be completed within 24 hours or less 99% of the time without the need surgery. The procedure is completed with minimal to no complications. Our Women's Health Center located in Abu Dhabi, United Arab Emirates, uses the latest medications for medical abortions (RU-486, Mifeprex, Mifegyne, Mifepristone, early options French abortion pill), Methotrexate and Cytotec (Misoprostol). The safety standards of our Abu Dhabi, United Arab Emirates Abortion Doctors remain unparalleled. They consistently maintain the lowest complication rates throughout the nation. Our Physicians and staff are always available to answer questions and care for women in one of the most difficult times in their lives. The decision to have an abortion at the Abortion Cl

+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...

?#DUbAI#??##{{(☎️+971_581248768%)**%*]'#abortion pills for sale in dubai@

💥 You’re lucky! We’ve found two different (lead) developers that are willing to share their valuable lessons learned about using UiPath Document Understanding! Based on recent implementations in appealing use cases at Partou and SPIE. Don’t expect fancy videos or slide decks, but real and practical experiences that will help you with your own implementations. 📕 Topics that will be addressed: • Training the ML-model by humans: do or don't? • Rule-based versus AI extractors • Tips for finding use cases • How to start 👨‍🏫👨‍💻 Speakers: o Dion Morskieft, RPA Product Owner @Partou o Jack Klein-Schiphorst, Automation Developer @Tacstone Technology

DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam

UiPathCommunity

Architecting Cloud Native Applications

WSO2

Exploring Multimodal Embeddings with Milvus

Zilliz

Effective data discovery is crucial for maintaining compliance and mitigating risks in today's rapidly evolving privacy landscape. However, traditional manual approaches often struggle to keep pace with the growing volume and complexity of data. Join us for an insightful webinar where industry leaders from TrustArc and Privya will share their expertise on leveraging AI-powered solutions to revolutionize data discovery. You'll learn how to: - Effortlessly maintain a comprehensive, up-to-date data inventory - Harness code scanning insights to gain complete visibility into data flows leveraging the advantages of code scanning over DB scanning - Simplify compliance by leveraging Privya's integration with TrustArc - Implement proven strategies to mitigate third-party risks Our panel of experts will discuss real-world case studies and share practical strategies for overcoming common data discovery challenges. They'll also explore the latest trends and innovations in AI-driven data management, and how these technologies can help organizations stay ahead of the curve in an ever-changing privacy landscape.

TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery

TrustArc

Following the popularity of “Cloud Revolution: Exploring the New Wave of Serverless Spatial Data,” we’re thrilled to announce this much-anticipated encore webinar. In this sequel, we’ll dive deeper into the Cloud-Native realm by uncovering practical applications and FME support for these new formats, including COGs, COPC, FlatGeoBuf, GeoParquet, STAC, and ZARR. Building on the foundation laid by industry leaders Michelle Roby of Radiant Earth and Chris Holmes of Planet in the first webinar, this second part offers an in-depth look at the real-world application and behind-the-scenes dynamics of these cutting-edge formats. We will spotlight specific use-cases and workflows, showcasing their efficiency and relevance in practical scenarios. Discover the vast possibilities each format holds, highlighted through detailed discussions and demonstrations. Our expert speakers will dissect the key aspects and provide critical takeaways for effective use, ensuring attendees leave with a thorough understanding of how to apply these formats in their own projects. Elevate your understanding of how FME supports these cutting-edge technologies, enhancing your ability to manage, share, and analyze spatial data. Whether you’re building on knowledge from our initial session or are new to the serverless spatial data landscape, this webinar is your gateway to mastering cloud-native formats in your workflows.

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME

Safe Software

ICT role in 21st century education and its challenges

rafiqahmad00786416

Artificial Intelligence Chap.5 : Uncertainty

Khushali Kathiriya

AWS Community Day CPH - Three problems of Terraform

Andrey Devyatkin

"I see eyes in my soup": How Delivery Hero implemented the safety system for ...

Zilliz

[BuildWithAI] Introduction to Gemini.pdf

Sandro Moreira

Tracing the root cause of a performance issue requires a lot of patience, experience, and focus. It’s so hard that we sometimes attempt to guess by trying out tentative fixes, but that usually results in frustration, messy code, and a considerable waste of time and money. This talk explains how to correctly zoom in on a performance bottleneck using three levels of profiling: distributed tracing, metrics, and method profiling. After we learn to read the JVM profiler output as a flame graph, we explore a series of bottlenecks typical for backend systems, like connection/thread pool starvation, invisible aspects, blocking code, hot CPU methods, lock contention, and Virtual Thread pinning, and we learn to trace them even if they occur in library code you are not familiar with. Attend this talk and prepare for the performance issues that will eventually hit any successful system. About authorWith two decades of experience, Victor is a Java Champion working as a trainer for top companies in Europe. Five thousands developers in 120 companies attended his workshops, so he gets to debate every week the challenges that various projects struggle with. In return, Victor summarizes key points from these workshops in conference talks and online meetups for the European Software Crafters, the world’s largest developer community around architecture, refactoring, and testing. Discover how Victor can help you on victorrentea.ro : company training catalog, consultancy and YouTube playlists.

Finding Java's Hidden Performance Traps @ DevoxxUK 2024

Victor Rentea

The value of a flexible API Management solution for Open Banking Steve Melan, Manager for IT Innovation and Architecture - State's and Saving's Bank of Luxembourg Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - The value of a flexible API Management solution for O...

apidays

Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood

Juan lago vázquez

Angeliki Cooney has spent over twenty years at the forefront of the life sciences industry, working out of Wynantskill, NY. She is highly regarded for her dedication to advancing the development and accessibility of innovative treatments for chronic diseases, rare disorders, and cancer. Her professional journey has centered on strategic consulting for biopharmaceutical companies, facilitating digital transformation, enhancing omnichannel engagement, and refining strategic commercial practices. Angeliki's innovative contributions include pioneering several software-as-a-service (SaaS) products for the life sciences sector, earning her three patents. As the Senior Vice President of Life Sciences at Avenga, Angeliki orchestrated the firm's strategic entry into the U.S. market. Avenga, a renowned digital engineering and consulting firm, partners with significant entities in the pharmaceutical and biotechnology fields. Her leadership was instrumental in expanding Avenga's client base and establishing its presence in the competitive U.S. market.

Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...

Angeliki Cooney

Join our latest Connector Corner webinar to discover how UiPath Integration Service revolutionizes API-centric automation in a 'Quote to Cash' process—and how that automation empowers businesses to accelerate revenue generation. A comprehensive demo will explore connecting systems, GenAI, and people, through powerful pre-built connectors designed to speed process cycle times. Speakers: James Dickson, Senior Software Engineer Charlie Greenberg, Host, Product Marketing Manager

Connector Corner: Accelerate revenue generation using UiPath API-centric busi...

DianaGray10

ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke

Product Anonymous

Strategies for Landing an Oracle DBA Job as a Fresher

Remote DBA Services

Accelerating FinTech Innovation: Unleashing API Economy and GenAI Vasa Krishnan, Chief Technology Officer - FinResults Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...

apidays

Recently uploaded (20)

Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...

+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...

DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam

Architecting Cloud Native Applications

Exploring Multimodal Embeddings with Milvus

TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME

ICT role in 21st century education and its challenges

Artificial Intelligence Chap.5 : Uncertainty

AWS Community Day CPH - Three problems of Terraform

"I see eyes in my soup": How Delivery Hero implemented the safety system for ...

[BuildWithAI] Introduction to Gemini.pdf

Finding Java's Hidden Performance Traps @ DevoxxUK 2024

Apidays New York 2024 - The value of a flexible API Management solution for O...

Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood

Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...

Connector Corner: Accelerate revenue generation using UiPath API-centric busi...

ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke

Strategies for Landing an Oracle DBA Job as a Fresher

Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...

2014-08-27 | The Twitter Stack (Vienna Scala User Group)

1. The Twitter Stack Finagle, Ostrich, Zipkin,… Dominik Gruber, @the_dom Vienna Scala User Group – Aug. 27, 2014

2. Dominik Gruber • @the_domTwitter Stack Agenda • Introduction • Prominent Adopters • Finagle • Ostrich • Zipkin • Iago • Finatra

3. Dominik Gruber • @the_domTwitter Stack Introduction • Twitter started to move oﬀ Ruby on Rails in 2010 • Currently still on Scala 2.9

4. Dominik Gruber • @the_domTwitter Stack Prominent Adopters • Box • eBay • Etsy • Foursquare • Gigya • Nest • Pinterest • Rdio • SoundCloud • Tumblr

5. Dominik Gruber • @the_domTwitter Stack

6. Dominik Gruber • @the_domTwitter Stack Finagle “Finagle is an extensible RPC system for the JVM, used to construct high-concurrency servers. Finagle implements uniform client and server APIs for several protocols, and is designed for high performance and concurrency. Most of Finagle’s code is protocol agnostic, simplifying the implementation of new protocols.”

7. Dominik Gruber • @the_domTwitter Stack Finagle provides… • Connection pools • Failure detection • Failover strategies • Load balancers • Back-pressure techniques

8. Dominik Gruber • @the_domTwitter Stack Finagle… • publishes standard statistics, logs, and exception reports • supports distributed tracing across protocols • optionally uses ZooKeeper for cluster management • supports common sharding strategies

9. Dominik Gruber • @the_domTwitter Stack Modules (Examples) • finagle-http • finagle-memcached • finagle-mysql • finagle-protobuf • finagle-redis • finagle-thrift

10. Dominik Gruber • @the_domTwitter Stack Basic abstractions • Future • Service • Filter

11. Dominik Gruber • @the_domTwitter Stack Ostrich

12. Dominik Gruber • @the_domTwitter Stack Ostrich • A stats collector & reporter • Collect runtime statistics (counters, gauges, metrics, and labels) • Report those statistics through a simple web interface or into log ﬁles • Similar to Metrics  https://github.com/dropwizard/metrics

13. Dominik Gruber • @the_domTwitter Stack

14. Dominik Gruber • @the_domTwitter Stack Zipkin “Zipkin is a distributed tracing system that helps us gather timing data for all the disparate services at Twitter. It manages both the collection and lookup of this data through a Collector and a Query service.”

15. Dominik Gruber • @the_domTwitter Stack Zipkin

16. Dominik Gruber • @the_domTwitter Stack Zipkin’s Architecture

17. Dominik Gruber • @the_domTwitter Stack Zipkin • “Scribe” to transport traces to Zipkin and Hadoop • Data stored in either Cassandra, Redis, HBase, MySQL, PostgreSQL, SQLite, or H2

18. Dominik Gruber • @the_domTwitter Stack Iago

19. Dominik Gruber • @the_domTwitter Stack Iago • A load generator that replays production or synthetic traﬃc • Iago supports arbitrarily high rates of traﬃc via built- in support for creating cluster

20. Dominik Gruber • @the_domTwitter Stack Protocols • HTTP • Thrift • Memcache • Kestrel (A queueing system from Twitter) • UDP

21. Dominik Gruber • @the_domTwitter Stack Finatra

22. Dominik Gruber • @the_domTwitter Stack

23. Dominik Gruber • @the_domTwitter Stack Features • Fully Async using Futures • Mustache Templates • Detailed Metrics (p99,p999, etc) • Proﬁle/Debug over HTTP • SSL/TLS Support • Deploys with a single jar

24. Dominik Gruber • @the_domTwitter Stack There’s more… • • Summingbird • https://github.com/twitter

25. Dominik Gruber • @the_domTwitter Stack Q & A

26. Dominik Gruber • @the_domTwitter Stack Source • http://blog.oskarsson.nu/post/40196324612/the- twitter-stack  https://blog.twitter.com/2011/ﬁnagle-a-protocol- agnostic-rpc-system • https://www.youtube.com/watch?v=whuT_-cFb5I  https://www.youtube.com/watch?v=QdER-Zl_YCM • https://github.com/twitter  https://github.com/ﬁnagle

2014-08-27 | The Twitter Stack (Vienna Scala User Group)

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (12)

Recently uploaded

Recently uploaded (20)

2014-08-27 | The Twitter Stack (Vienna Scala User Group)