Making Scala Faster: 3 Expert Tips For Busy Development Teams

•

2 gefällt mir•2,217 views

This document provides information about the author and discusses ways to improve Scala compilation performance. The author has worked on Scala tooling and is the co-founder of Triplequote. They discuss how build time differs from compilation time due to pre-processing steps. They warn that type classes and macros can significantly increase code size and slow compilation. Whitebox macros are type-checked three times while blackbox macros only participate in type inference. They recommend monitoring compilation to identify bottlenecks like macro expansion or implicit resolution. Finally, they note that the Scala compiler is single-threaded, but parallelization using Scala Hydra can improve compilation speed.

Software

Who am I?
• Scala hacker since 2005

• Worked mostly on Scala tooling (MiMa, Eclipse Scala
IDE, Scala compiler)

• Co-Founder of Triplequote

• Make Scala compile as fast as Java thanks to Hydra

Build time ≠ Compilation time
• Before starting compilation:

• Dependency resolution

• Source generators

• Formatter

• Settings evaluation

Build time ≠ Compilation time
• Before starting compilation:

• What do we need to
compile?

• Hash classpath, hash
sources (a LOT of I/O)

• Load analysis data

Build time ≠ Compilation time
Finally Sbt starts compiling, but
Zinc and incremental compiler
have a 15% overhead

Type Classes, Macro 💣
• Imports take precedence over companion objects

• Cost: implicit resolution

• Cost: additional code to be generated

Type Classes, Macro 💣
Decoders/
implicits
0 1 2
Code size after
type-checking
8 KB 31 KB 54 KB

Type Classes, Macro 💣
D[ShippingAddress]
D[FullName] D[FullName] D[String]
D[String] D[String] D[String] D[String]

Avoid re-generating the
same typeclass derivation
multiple times!

What’s the different
between whitebox and
blackbox macros?

Which one takes part in
type-inference?
Shapeless macros are whitebox

Type-checking should be
around 30%
• Type-checking Scala sources takes time

• Type-checking should not go above 30% of total
compilation time!

• Use -verbose to see times for each phase (and a
lot of noise)

Type-checking should be
around 30%
• If more, you can bet your money on one (or more) of:

• Macro expansion (useful ﬂag: -Xprint:typer)

• Implicit resolution (useful ﬂag: -Xlog-implicits)

• Quasiquotes, type tags (macros in disguise)

Monitor compilation and

identify bottlenecks

Parallelize Scala compilation

Scala compiler is single-
threaded
• It’s a batch compiler

• Careful interplay of laziness and mutability

• Laziness required for recursive types

• Mutability for performance

Scala compiler is single-
threaded
• Bigger/more powerful machines won’t make a dent
in compilation times

• Unless you have a parallel Scala compiler!

• Hint: there is one that works today! (google Scala
Hydra)

How fast is scalac?
Simple code
(Spark)
FP-heavy
(Cats)
Play Framework (LiChess)
Average Views Controllers
1’200 LoC/s 480 LoC/s 1’500 LoC/s 2’000 LoC/s 500 LoC/s
(2.12.4 numbers, with a warm compiler — except LiChess: 2.11.11)

Summary
• Compilation times are unpredictable

• A single ﬁle can account for 80% of compilation time

• Compilation speed varies greatly based on coding style

• Macro expansion can quintuple your code size

https://twitter.com/berenguel/
status/1040712559362539524

Trusted by
Parallelize Scala compilation
Monitor compilation and
identify bottlenecks

https://triplequote.com/hydra/trial15-day FREE trial

Weitere ähnliche Inhalte

Was ist angesagt?

Organizations like Starbucks, HPE, and PayPal (see our customers) have selected the Akka toolkit for their enterprise scale distributed applications; and when it comes to squeezing out the best possible performance, the secret is using two particular modules in tandem: Akka Cluster and Akka Streams. In this webinar by Nolan Grace, Senior Solution Architect at Lightbend, we look at these two Akka modules and discuss the features that will push your application architecture to the next tier of performance. For the full blog post, including the video, visit: https://www.lightbend.com/blog/akka-at-enterprise-scale-performance-tuning-distributed-applications

Akka at Enterprise Scale: Performance Tuning Distributed Applications

Lightbend

We’re at another juncture in enterprise computing where there is a large push behind a big vision of the future, the push towards serverless architectures—a world where less human oversight and participation is required in operations. At this time, serverless computing is so hot right now. A promise of ‘Opsless’, cloud-native, pay-for-what-you-use functions does sound great, but the current incarnation of serverless that most people think of–Function as a Service (FaaS)–is limited to ephemeral, stateless, and short-lived functions. Amazon Lambda caps their lifespan to 15 minutes, for example. This is not what we need out of a general platform for building modern real-time, data-centric applications and systems. What we do need are scalable, stateful services that can operate on cloud infrastructure as if they are stateless. Explore how to make your next application stateful, providing a better understanding of the technology landscape, challenges and pitfalls, and successful methods with Akka Cluster Sharding

Building stateful systems with akka cluster sharding

Knoldus Inc.

Streaming Big Data with Spark, Kafka, Cassandra, Akka & Scala (from webinar)

Helena Edelson

Hands On With Spark: Creating A Fast Data Pipeline With Structured Streaming ...

Lightbend

Kafka, Apache Kafka evolved from an enterprise messaging system to a fully distributed streaming data platform (Kafka Core + Kafka Connect + Kafka Streams) for building streaming data pipelines and streaming data applications. This talk, that I gave at the Chicago Java Users Group (CJUG) on June 8th 2017, is mainly focusing on Kafka Streams, a lightweight open source Java library for building stream processing applications on top of Kafka using Kafka topics as input/output. You will learn more about the following: 1. Apache Kafka: a Streaming Data Platform 2. Overview of Kafka Streams: Before Kafka Streams? What is Kafka Streams? Why Kafka Streams? What are Kafka Streams key concepts? Kafka Streams APIs and code examples? 3. Writing, deploying and running your first Kafka Streams application 4. Code and Demo of an end-to-end Kafka-based Streaming Data Application 5. Where to go from here?

Kafka Streams for Java enthusiasts

Slim Baltagi

Audience: Architects, Data Scientists, Developers Technical level: Introductory From home intrusion detection, to self-driving cars, to keeping data center operations healthy, Machine Learning (ML) has become one of the hottest topics in software engineering today. While much of the focus has been on the actual creation of the algorithms used in ML, the less talked-about challenge is how to serve these models in production, often utilizing real-time streaming data. The traditional approach to model serving is to treat the model as code, which means that ML implementation has to be continually adapted for model serving. As the amount of machine learning tools and techniques grows, the efficiency of such an approach is becoming more questionable. Additionally, machine learning and model serving are driven by very different quality of service requirements; while machine learning is typically batch, dealing with scalability and processing power, model serving is mostly concerned with performance and stability. In this webinar with O’Reilly author and Lightbend Principal Architect, Boris Lublinsky, we will define an alternative approach to model serving, based on treating the model itself as data. Using popular frameworks like Akka Streams and Apache Flink, Boris will review how to implement this approach, explaining how it can help you: * Achieve complete decoupling between the model implementation for machine learning and model serving, enforcing better standardization of your model serving implementation. * Enable dynamic updates of the served model without having to restart the system. * Utilize Tensorflow and PMML as model representation and their usage for building “real time updatable” model serving architecture.

Machine Learning At Speed: Operationalizing ML For Real-Time Data Streams

Lightbend

Scala Security: Eliminate 200+ Code-Level Threats With Fortify SCA For Scala

Lightbend

Reactive Streams 1.0.0 is now live, and so are our implementations in Akka Streams 1.0 and Slick 3.0. Reactive Streams is an engineering collaboration between heavy hitters in the area of streaming data on the JVM. With the Reactive Streams Special Interest Group, we set out to standardize a common ground for achieving statically-typed, high-performance, low latency, asynchronous streams of data with built-in non-blocking back pressure—with the goal of creating a vibrant ecosystem of interoperating implementations, and with a vision of one day making it into a future version of Java. Akka (recent winner of “Most Innovative Open Source Tech in 2015”) is a toolkit for building message-driven applications. With Akka Streams 1.0, Akka has incorporated a graphical DSL for composing data streams, an execution model that decouples the stream’s staged computation—it’s “blueprint”—from its execution (allowing for actor-based, single-threaded and fully distributed and clustered execution), type safe stream composition, an implementation of the Reactive Streaming specification that enables back-pressure, and more than 20 predefined stream “processing stages” that provide common streaming transformations that developers can tap into (for splitting streams, transforming streams, merging streams, and more). Slick is a relational database query and access library for Scala that enables loose-coupling, minimal configuration requirements and abstraction of the complexities of connecting with relational databases. With Slick 3.0, Slick now supports the Reactive Streams API for providing asynchronous stream processing with non-blocking back-pressure. Slick 3.0 also allows elegant mapping across multiple data types, static verification and type inference for embedded SQL statements, compile-time error discovery, and JDBC support for interoperability with all existing drivers.

A Deeper Look Into Reactive Streams with Akka Streams 1.0 and Slick 3.0

Legacy Typesafe (now Lightbend)

Microservices. Streaming data. Event Sourcing and CQRS. Concurrency, routing, self-healing, persistence, clustering… You get the picture. The Akka toolkit makes all of this simple for Java and Scala developers at Amazon, LinkedIn, Starbucks, Verizon and others. So how does Akka provide all these features out of the box? Join Hugh McKee, Akka expert and Developer Advocate at Lightbend, on an illustrated journey that goes deep into how Akka works–from individual Akka actors to fully distributed clusters across multiple datacenters.

Akka A to Z: A Guide To The Industry’s Best Toolkit for Fast Data and Microse...

Lightbend

Do's and don'ts when deploying akka in production

jglobal

Modern businesses have data at their core, and this data is changing continuously. How can we harness this torrent of information in real-time? The answer is stream processing, and the technology that has since become the core platform for streaming data is Apache Kafka. Among the thousands of companies that use Kafka to transform and reshape their industries are the likes of Netflix, Uber, PayPal, and AirBnB, but also established players such as Goldman Sachs, Cisco, and Oracle. Unfortunately, today’s common architectures for real-time data processing at scale suffer from complexity: there are many technologies that need to be stitched and operated together, and each individual technology is often complex by itself. This has led to a strong discrepancy between how we, as engineers, would like to work vs. how we actually end up working in practice. In this session we talk about how Apache Kafka helps you to radically simplify your data processing architectures. We cover how you can now build normal applications to serve your real-time processing needs — rather than building clusters or similar special-purpose infrastructure — and still benefit from properties such as high scalability, distributed computing, and fault-tolerance, which are typically associated exclusively with cluster technologies. Notably, we introduce Kafka’s Streams API, its abstractions for streams and tables, and its recently introduced Interactive Queries functionality. As we will see, Kafka makes such architectures equally viable for small, medium, and large scale use cases.

Introducing Kafka's Streams API

confluent

Since its stable release in 2016, Akka Streams is quickly becoming the de facto standard integration layer between various Streaming systems and products. Enterprises like PayPal, Intel, Samsung and Norwegian Cruise Lines see this is a game changer in terms of designing Reactive streaming applications by connecting pipelines of back-pressured asynchronous processing stages. This comes from the Reactive Streams initiative in part, which has been long led by Lightbend and others, allowing multiple streaming libraries to inter-operate between each other in a performant and resilient fashion, providing back-pressure all the way. But perhaps even more so thanks to the various integration drivers that have sprung up in the community and the Akka team—including drivers for Apache Kafka, Apache Cassandra, Streaming HTTP, Websockets and much more. In this webinar for JVM Architects, Konrad Malawski explores the what and why of Reactive integrations, with examples featuring technologies like Akka Streams, Apache Kafka, and Alpakka, a new community project for building Streaming connectors that seeks to “back-pressurize” traditional Apache Camel endpoints. * An overview of Reactive Streams and what it will look like in JDK 9, and the Akka Streams API implementation for Java and Scala. * Introduction to Alpakka, a modern, Reactive version of Apache Camel, and its growing community of Streams connectors (e.g. Akka Streams Kafka, MQTT, AMQP, Streaming HTTP/TCP/FileIO and more). * How Akka Streams and Akka HTTP work with Websockets, HTTP and TCP, with examples in both in Java and Scala.

Exploring Reactive Integrations With Akka Streams, Alpakka And Apache Kafka

Lightbend

Scala in Model-Driven development for Apparel Cloud Platform

Tomoharu ASAMI

Introduction to apache kafka

Dimitris Kontokostas

Kafka Summit NYC 2017 Introduction to Kafka Streams with a Real-life Example

confluent

William Hill is one of the UK’s largest, most well-established gaming companies with a global presence across 9 countries with over 16,000 employees. In recent years the gaming industry and in particular sports betting, has been revolutionised by technology. Customers now demand a wide range of events and markets to bet on both pre-game and in-play 24/7. This has driven out a business need to process more data, provide more updates and offer more markets and prices in real time. At William Hill, we have invested in a completely new trading platform using Apache Kafka. We process vast quantities of data from a variety of feeds, this data is fed through a variety of odds compilation models, before being piped out to UI apps for use by our trading teams to provide events, markets and pricing data out to various end points across the whole of William Hill. We deal with thousands of sporting events, each with sometimes hundreds of betting markets, each market receiving hundreds of updates. This scales up to vast numbers of messages flowing through our system. We have to process, transform and route that data in real time. Using Apache Kafka, we have built a high throughput, low latency pipeline, based on Cloud hosted Microservices. When we started, we were on a steep learning curve with Kafka, Microservices and associated technologies. This led to fast learnings and fast failings. In this session, we will tell the story of what we built, what went well, what didn’t go so well and what we learnt. This is a story of how a team of developers learnt (and are still learning) how to use Kafka. We hope that you will be able to take away lessons and learnings of how to build a data processing pipeline with Apache Kafka.

Building High-Throughput, Low-Latency Pipelines in Kafka

confluent

With the introduction of connect and streams API in 2016, Apache Kafka is becoming the defacto solution for anyone looking to build a streaming platform. The community continues to add additional capabilities to make it the complete solution for streaming data. Join us as we review the latest additions in Apache Kafka 0.10.2. In addition, we’ll cover what’s new in Confluent Enterprise 3.2 that makes it possible for running Kafka at scale.

What's new in Confluent 3.2 and Apache Kafka 0.10.2

confluent

Kafka aws

Ariel Moskovich

Having a team adopt new technologies and approaches to software development is a daunting task. New paradigms and unfamiliar ontologies headline the biggest risks to having a team be productive quickly. Lightbend (formerly Typesafe) has a suite of training classes to help you adopt whatever components of the Lightbend Reactive Platform you need to be responsive to you customers by creating resilient and elastic applications. In this webinar, we will discuss the philosophies and structures of Lightbend's training materials for Scala, Akka, Play Framework, and Spark.

Lightbend Training for Scala, Akka, Play Framework and Apache Spark

Lightbend

Is your organization committed to the power of the Java platform but stuck in the Java language? Do you spend your time wishing you could use Scala at work, but don’t see how you can get it approved? There are good reasons to push for it! Scala is a popular language with Java developers because of its expressiveness and lack of boilerplate code. It allows you to focus on the problem rather than the ceremony required by Java. Yet, businesses are often reluctant to move to Scala, citing concerns about availability of developers and risks of working in a new language. Learn strategies for bringing Scala into your organization, including using it in non-production code, integrating Scala into a Java application, and easing your fellow programmers into functional programming. See how to use Scala for new development while maintaining your investment in existing Java code. Hear stories of how other organizations have done this and succeeded. You’ll leave knowing how you can move to Scala without risking your job or the success of your projects.

Sneaking Scala through the Back Door

Dianne Marsh

Was ist angesagt? (20)

Akka at Enterprise Scale: Performance Tuning Distributed Applications

Building stateful systems with akka cluster sharding

Streaming Big Data with Spark, Kafka, Cassandra, Akka & Scala (from webinar)

Hands On With Spark: Creating A Fast Data Pipeline With Structured Streaming ...

Kafka Streams for Java enthusiasts

Machine Learning At Speed: Operationalizing ML For Real-Time Data Streams

Scala Security: Eliminate 200+ Code-Level Threats With Fortify SCA For Scala

A Deeper Look Into Reactive Streams with Akka Streams 1.0 and Slick 3.0

Akka A to Z: A Guide To The Industry’s Best Toolkit for Fast Data and Microse...

Do's and don'ts when deploying akka in production

Introducing Kafka's Streams API

Exploring Reactive Integrations With Akka Streams, Alpakka And Apache Kafka

Scala in Model-Driven development for Apparel Cloud Platform

Introduction to apache kafka

Kafka Summit NYC 2017 Introduction to Kafka Streams with a Real-life Example

Building High-Throughput, Low-Latency Pipelines in Kafka

What's new in Confluent 3.2 and Apache Kafka 0.10.2

Kafka aws

Lightbend Training for Scala, Akka, Play Framework and Apache Spark

Sneaking Scala through the Back Door

Ähnlich wie Making Scala Faster: 3 Expert Tips For Busy Development Teams

5 things you need to know about the Scala compiler

Iulian Dragos

The Why and How of Scala at Twitter

Alex Payne

Introduction to Micronaut - JBCNConf 2019

graemerocher

Clojure in real life 17.10.2014

Metosin Oy

Scala at Treasure Data

Taro L. Saito

New abstractions for concurrency make writing programs easier by moving away from threads and locks, but debugging such programs becomes harder. The call-stack, an essential tool in understanding why and how control flow reached a certain point in the program, loses meaning when inspected in traditional debuggers. Futures, actors or iteratees make code easier to write and reason about, and in this talk I'll show a simple solution to make them easier to debug. The tool I present integrates well with the Eclipse plugin for Scala, and shows how a "reactive debugger" might look like.

Rethinking the debugger

Iulian Dragos

Numberly operates business-critical data pipelines and applications where failure and latency means "lost money" in the best-case scenario. Most of those data pipelines and applications are deployed on Kubernetes and rely on Kafka and ScyllaDB, where Kafka acts as the message bus and ScyllaDB as the source of some data enrichment. The availability and latency of both systems are thus very important because they mix and match data in the early stage of their pipelines to be consumed by their platforms. Most of their applications are developed using Python. But they always felt that they could benefit from a lower-level programming language to squeeze the performance of their hardware even further for some of the most demanding applications. So, when an important part of their data pipeline was to be adjusted to reflect some important changes in their platforms, they thought it was a great opportunity to rewrite it in Rust! Moving to Rust was hard, not only because of the language itself, but because being at a lower level allowed them to see, test, and demonstrate things that they could not pinpoint or explain that well using Python. They spent a lot of time analyzing the latency impacts of code patterns and client driver settings and ended up contributing to Apache Avro as they went down the rabbit hole. This session will share their experience transitioning from Python to Rust while meeting the expectations of a business-critical application mixing data from Confluent Kafka and ScyllaDB. There will be code snippets, graphs, numbers, tears, pull requests, grins, latency results, smiles, rants of frustration, and a lot of fun! To watch all of the recordings hosted during Scylla Summit 2022 visit our website here: https://www.scylladb.com/summit.

Scylla Summit 2022: Learning Rust the Hard Way for a Production Kafka+ScyllaD...

ScyllaDB

Scripting experts from Inductive Automation cover general best practices that will help you add flexibility and customization to HMI, SCADA, IIoT, and other industrial applications. Some specific tips about using scripting in the Ignition platform will be included as well. In this webinar, learn more about: • Common scripting pitfalls and how to avoid them • The best programming languages to use • Things to consider before using scripting • How scripting environments work • Scripting timesavers • And more

Design Like a Pro: Scripting Best Practices

Inductive Automation

Design Like a Pro: Scripting Best Practices

Inductive Automation

Video and slides synchronized, mp3 and slide download available at URL https://bit.ly/2COBbRN. Aarti Parikh goes over the Go language design and talks about why Go matters in the age of multicores and cloud-native computing. Filmed at qconsf.com. Aarti Parikh is a software engineering manager with a decade of experience building teams and products for the Internet, IoT and now Fintech. Currently, she works at PayPal.

Go - A Key Language in Enterprise Application Development?

C4Media

Ruby and Distributed Storage Systems

SATOSHI TAGOMORI

Go: What's Different ?

Tarun Vashisth

Preventing Complexity in Game Programming

Yaser Zhian

An Introduction to the Laravel Framework (AFUP Forum PHP 2014)

daylerees

Life as a software engineer is so exciting! Computing power continue to rise exponentially, software demands continue to rise exponentially as well, so far so good. The bad news are that in the last decade the computing power of single threaded application remains almost flat. If you decide to continue ignoring concurrency and multi-threading the gap between the problems you are able to solve and your hardware capabilities will continue to rise. In this session we will discuss different approaches for taming the concurrency beast, such as shared mutability,shared immutability and isolated mutability actors, STM, etc we will discuss the shortcomings and the dangers of each approach and we will compare different programming languages and how they choose to tackle/ignore concurrency.

Concurrency and Multithreading Demistified - Reversim Summit 2014

Haim Yadid

Performance Benchmarking: Tips, Tricks, and Lessons Learned

Tim Callaghan

Performance and Abstractions

Metosin Oy

Hot to build continuously processing for 24/7 real-time data streaming platform?

GetInData

You spend your precious time building the perfect application. You do everything right. You carefully craft every piece of code and rigorously follow the best practices and design patterns, you apply the most successful methodologies software engineering has to offer with discipline, and you pay attention to the most minuscule of details to produce the best user experience possible. It all pays off eventually, and you end up with a beautiful code base that is not only reliable but also performs well. You proudly watch your baby grow, as new users come in bringing more traffic your way and craving new features. You keep them happy and they keep coming back. One morning, you wake up to servers crashing under load, and data stores failing to keep up with all the demand. You panic. You throw in more hardware and try optimize, but the hungry crowd that was once your happy user base catches up to you. Your success is slipping through your fingers. You find yourself stuck between having to rewrite the whole application and a hard place. It's frustrating, dreadful, and painful to say the least. Don't be that guy! Save your soul before it's too late, and come to learn how to build, deploy, and maintain enterprise-grade Java applications that scale from day one. Topics covered include: parallelism, load distribution, state management, caching, big data, asynchronous processing, and static content delivery. Leveraging cloud computing, scaling teams and DevOps will also be discuss. P.S. This session is more technical than you might think. http://jaxconf.com/sessions/economies-scaling-software

The Economies of Scaling Software

Abdelmonaim Remani

Stackato v2

Jonas Brømsø

Ähnlich wie Making Scala Faster: 3 Expert Tips For Busy Development Teams (20)

5 things you need to know about the Scala compiler

The Why and How of Scala at Twitter

Introduction to Micronaut - JBCNConf 2019

Clojure in real life 17.10.2014

Scala at Treasure Data

Rethinking the debugger

Scylla Summit 2022: Learning Rust the Hard Way for a Production Kafka+ScyllaD...

Design Like a Pro: Scripting Best Practices

Go - A Key Language in Enterprise Application Development?

Ruby and Distributed Storage Systems

Go: What's Different ?

Preventing Complexity in Game Programming

An Introduction to the Laravel Framework (AFUP Forum PHP 2014)

Concurrency and Multithreading Demistified - Reversim Summit 2014

Performance Benchmarking: Tips, Tricks, and Lessons Learned

Performance and Abstractions

Hot to build continuously processing for 24/7 real-time data streaming platform?

The Economies of Scaling Software

Stackato v2

Mehr von Lightbend

********** Watch this presentation on-demand! https://info.lightbend.com/iot-megaservices-high-throughput-microservices-with-akka-register.html ********** In this interactive presentation by Hugh McKee, Developer Advocate at Lightbend, we’ll share our experiences helping our clients create a system architecture that can support high throughput microservices (aka "Megaservices"). We’ll do that using IoT demo applications designed to push cloud service providers like Amazon and Google to their limits. Using sample code that you can later run on your own machine, we’ll look at: * Modeling real-life digital twins for hundreds of thousands of IoT devices in the field, looking into how these megaservices are implemented in Akka. * Visualizing Akka Actors–which represent IoT digital twins–in a “crop circle” formation that represents a complete distributed Reactive application, and watching at messages are processed across Akka Cluster nodes using cluster sharding. * Some code behind the whole set up, which is built using OSS like Akka, Java, JavaScript, and Kubernetes. Follow us on social: TW: https://twitter.com/lightbend LI: https://www.linkedin.com/company/lightbend-inc-/ FB: https://www.facebook.com/lightbendOfficial/ For more about Lightbend: Blog: https://www.lightbend.com/blog  Newsletter: https://www.lightbend.com/newsletter 

IoT 'Megaservices' - High Throughput Microservices with Akka

Lightbend

How Akka Cluster Works: Actors Living in a Cluster

Lightbend

The Reactive Principles: Eight Tenets For Building Cloud Native Applications

Lightbend

Putting the 'I' in IoT - Building Digital Twins with Akka Microservices

Lightbend

Digital Transformation with Kubernetes, Containers, and Microservices

Lightbend

Deploying a robust streaming data pipeline can be a daunting task when your company’s financial information is at risk. For starters, how do you ensure proper provisioning of resources? How do you preserve end-to-end application and data consistency? How do you make all of this work in the cloud with Kubernetes and avoid YAML hell? Answer: Cloudflow, a new open-source toolkit for simplifying the development, deployment, and operation of streaming data pipelines.

Detecting Real-Time Financial Fraud with Cloudflow on Kubernetes

Lightbend

Cloudstate - Towards Stateful Serverless

Lightbend

Join this highly-visual presentation by Hugh McKee, Developer Advocate at Lightbend, to learn more about the ramifications and opportunities along the evolution from monolithic systems, to microservices architectures, to serverless (FaaS). See the video presentation on the Lightbend blog at: https://www.lightbend.com/blog/digital-transformation-from-monoliths-to-microservices-to-serverless-and-beyond

Digital Transformation from Monoliths to Microservices to Serverless and Beyond

Lightbend

Akka Anti-Patterns, Goodbye: Six Features of Akka 2.6

Lightbend

Lessons From HPE: From Batch To Streaming For 20 Billion Sensors With Lightbe...

Lightbend

How to build streaming data pipelines with Akka Streams, Flink, and Spark usi...

Lightbend

In this talk by David Ogren, Enterprise Architect at Lightbend, we draw from experiences helping our clients successfully create, migrate to, and manage cloud-native system architectures. We look at some of the common pitfalls and anti-patterns of modernization efforts, and some of the best practices for taking an incremental approach to transforming legacy systems. See the full post with video on the Lightbend blog: https://www.lightbend.com/blog/microservices-kubernetes-application-modernization

Microservices, Kubernetes, and Application Modernization Done Right

Lightbend

Full Stack Reactive In Practice

Lightbend

Akka and Kubernetes: A Symbiotic Love Story

Lightbend

Scala 3 Is Coming: Martin Odersky Shares What To Know

Lightbend

A lot of businesses that never before considered themselves as “technology companies” are now faced with digital modernization imperatives that force them to rethink their application and infrastructure architecture. On the path to becoming a digital, on-demand provider, development speed is the ultimate competitive advantage. This presents challenges to many organizations that have huge investments in legacy Java EE infrastructure, where technical debt and monolithic system architectures require modernization in order to confront various business risks. Usually, changes need to be made within existing frameworks to keep pace with new web-scale organizations. If your legacy monolith is no longer serving the expanding needs of your business, then join Markus Eisele, Director of Developer Advocacy at Lightbend, to learn what you can do to migrate from Java EE to cloud-native, Reactive systems—as defined by the Reactive Manifesto.

Migrating From Java EE To Cloud-Native Reactive Systems

Lightbend

In this talk by Sean Glover, Principal Engineer at Lightbend, we will review how the Strimzi Kafka Operator, a supported technology in Lightbend Platform, makes many operational tasks in Kafka easy, such as the initial deployment and updates of a Kafka and ZooKeeper cluster. See the blog post containing the YouTube video here: https://www.lightbend.com/blog/running-kafka-on-kubernetes-with-strimzi-for-real-time-streaming-applications

Running Kafka On Kubernetes With Strimzi For Real-Time Streaming Applications

Lightbend

Designing Events-First Microservices For A Cloud Native World

Lightbend

How To Build, Integrate, and Deploy Real-Time Streaming Pipelines On Kubernetes

Lightbend

A Glimpse At The Future Of Apache Spark 3.0 With Deep Learning And Kubernetes

Lightbend

Mehr von Lightbend (20)

IoT 'Megaservices' - High Throughput Microservices with Akka

How Akka Cluster Works: Actors Living in a Cluster

The Reactive Principles: Eight Tenets For Building Cloud Native Applications

Putting the 'I' in IoT - Building Digital Twins with Akka Microservices

Digital Transformation with Kubernetes, Containers, and Microservices

Detecting Real-Time Financial Fraud with Cloudflow on Kubernetes

Cloudstate - Towards Stateful Serverless

Digital Transformation from Monoliths to Microservices to Serverless and Beyond

Akka Anti-Patterns, Goodbye: Six Features of Akka 2.6

Lessons From HPE: From Batch To Streaming For 20 Billion Sensors With Lightbe...

How to build streaming data pipelines with Akka Streams, Flink, and Spark usi...

Microservices, Kubernetes, and Application Modernization Done Right

Full Stack Reactive In Practice

Akka and Kubernetes: A Symbiotic Love Story

Scala 3 Is Coming: Martin Odersky Shares What To Know

Migrating From Java EE To Cloud-Native Reactive Systems

Running Kafka On Kubernetes With Strimzi For Real-Time Streaming Applications

Designing Events-First Microservices For A Cloud Native World

How To Build, Integrate, and Deploy Real-Time Streaming Pipelines On Kubernetes

A Glimpse At The Future Of Apache Spark 3.0 With Deep Learning And Kubernetes

Kürzlich hochgeladen

WSO2CON 2024 - WSO2's Digital Transformation Journey with Choreo: A Platforml...

WSO2

Direct Style Effect Systems -The Print[A] Example- A Comprehension Aid

Philip Schwarz

ADR, or Architecture Decision Record, is a valuable tool in software development for several reasons. It provides a centralized location for documenting and tracking architectural decisions, aiding both current and future team members. ADRs enhance communication among team members by documenting the rationale behind architectural decisions, especially beneficial during onboarding of new team members or when revisiting decisions. They serve as a knowledge base, enabling teams to learn from past decisions and refine their decision-making process. Additionally, ADRs contribute to transparency by helping stakeholders understand the reasons behind specific architectural choices. As with any other tool or process, introducing them into an organization can face several obstacles, and overcoming these challenges is crucial for successful implementation. In this talk I go through some common problems and our way of solving them.

Architecture decision records - How not to get lost in the past

Papp Krisztián

Data spaces in distributed environments should be allowed to evolve in agile ways providing data space owners with large flexibility about which data they store. Agility and heterogeneity, however, jeopardize data exchanges because representations may build on varying ontologies and data consumers may not rely on the semantic correctness of their queries in the context of semantically heterogeneous, evolving data spaces. Graph data spaces are one example of a powerful model for representing and querying data whose semantics may change over time. To assert and enforce conditions on individual graph data spaces, shape languages (e.g SHACL) have been developed. We investigate the question of how querying and programming can be guarded by reasoning over SHACL constraints in a distributed setting and we sketch a picture of how a future landscape based on semantically heterogeneous data spaces might look like.

Shapes for Sharing between Graph Data Spaces - and Epistemic Querying of RDF-...

Steffen Staab

WSO2CON 2024 - Building the API First Enterprise – Running an API Program, fr...

WSO2

We specialize in Psychic Readings, Psychic Love Spells, Binding Love Spells, Obsession Spells, Voodoo Spells, Lottery Spells, Marriage Spells, Black Magic Spells, Palm Readings & much more. Are you depressed? We perform this come-to-me love spell that works instantly with the aim of bringing back the victim to the person performing the magic. Have you lost your lover? We perform this come-to-me love spell that works instantly with the aim of bringing back the victim to the person performing the magic. Have you lost your lover? Do u need to solve any relationship problem? Contact the powerful spells caster chief kule with love spells that work overnight and love spells that really work. Have you found yourself infatuated with a special someone you think could be the one? Are you looking for a spell to provide them with a nudge in the right direction? Or maybe the spell you cast didn’t achieve the results you were hoping for? Whether you’re new or versed in the ways of spell casting, we’re here to help. Today we’re going to provide you with a detailed guide on the types of love spells to cast. Not only that but there’s something for those who wish to find outside advice from more advanced spell casters. We’re also going to provide you with the top sites available to help you with your dilemma. Let’s begin our journey by educating ourselves on love magic and what a real love caster looks like. Love Magic and Love Casters Love magic made its first appearance back in Ancient Egypt and has been an active practice since. This type of magic is a branch of traditional magic and can be practiced in various ways. Typically the more common use of love magic is through the work of spells, but other methods look like Charms Rituals-LOVE Potions-Dolls and even Amulets If you are interested in becoming a love caster, be prepared for what’s to come. A genuine love caster knows that the art of love casting is no easy feat and shouldn’t be done casually. You should know that not only does it require you to be gifted spiritually, but you must be ready to serve others. Someone who is considered a real love caster has experience in all manner of spells, no matter the difficulty. Training yourself in attraction, commitment, and marriage spells is an excellent place to start. But this by no means will make you a professional. Practice your craft and expand your knowledge; understand that you will possess the ability to help others in time truly. Types of Love Spells What better way to start broadening your experiences with love spells than by learning more about them? These spells work like just about any other spell. Simply apply your intention, use a medium (sigils, mantras, candles, or charm bags), and top it off with establishing the belief that you will receive what you want. So what kind of spells are available and which ones suit your needs the best? Let’s take a look at the many options you have at your disposal.

%+27788225528 love spells in new york Psychic Readings, Attraction spells,Bri...

masabamasaba

WSO2CON2024 - It's time to go Platformless

WSO2

%+27788225528 love spells in Boston Psychic Readings, Attraction spells,Bring...

masabamasaba

%in Hazyview+277-882-255-28 abortion pills for sale in Hazyview

masabamasaba

WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation

WSO2

%in Rustenburg+277-882-255-28 abortion pills for sale in Rustenburg

masabamasaba

In today's dynamic e-commerce landscape, the payment gateway emerges as a linchpin, ensuring smooth and secure transactions between buyers and sellers. In this discourse, we delve into the meticulous process of devising test cases tailored for scrutinizing payment gateways. Crafting precise test cases for payment gateways is a quintessential responsibility for testers operating within the service industry. This article meticulously explores pivotal scenarios integral to how to test payment gateways, coupled with essential guidelines for drafting effective test cases.

Payment Gateway Testing Simplified_ A Step-by-Step Guide for Beginners.pdf

kalichargn70th171

tonesoftg

lanshi9

We specialize in Psychic Readings, Psychic Love Spells, Binding Love Spells, Obsession Spells, Voodoo Spells, Lottery Spells, Marriage Spells, Black Magic Spells, Palm Readings & much more. Are you depressed? We perform this come-to-me love spell that works instantly with the aim of Winnipeg back the victim to the person performing the magic. Have you lost your lover? We perform this come-to-me love spell that works instantly with the aim of bringing back the victim to the person performing the magic. Have you lost your lover? Do u need to solve any relationship problem? Contact the powerful spells caster chief kule with love spells that work overnight and love spells that really work. Have you found yourself infatuated with a special someone you think could be the one? Are you looking for a spell to provide them with a nudge in the right direction? Or maybe the spell you cast didn’t achieve the results you were hoping for? Whether you’re new or versed in the ways of spell casting, we’re here to help. Today we’re going to provide you with a detailed guide on the types of love spells to cast. Not only that but there’s something for those who wish to find outside advice from more advanced spell casters. We’re also going to provide you with the top sites available to help you with your dilemma. Let’s begin our journey by educating ourselves on love magic and what a real love caster looks like. Love Magic and Love Casters Love magic made its first appearance back in Ancient Egypt and has been an active practice since. This type of magic is a branch of traditional magic and can be practiced in various ways. Typically the more common use of love magic is through the work of spells, but other methods look like Charms Rituals-LOVE Potions-Dolls and even Amulets If you are interested in becoming a love caster, be prepared for what’s to come. A genuine love caster knows that the art of love casting is no easy feat and shouldn’t be done casually. You should know that not only does it require you to be gifted spiritually, but you must be ready to serve others. Someone who is considered a real love caster has experience in all manner of spells, no matter the difficulty. Training yourself in attraction, commitment, and marriage spells is an excellent place to start. But this by no means will make you a professional. Practice your craft and expand your knowledge; understand that you will possess the ability to help others in time truly. Types of Love Spells What better way to start broadening your experiences with love spells than by learning more about them? These spells work like just about any other spell. Simply apply your intention, use a medium (sigils, mantras, candles, or charm bags), and top it off with establishing the belief that you will receive what you want. So what kind of spells are available and which ones suit your needs the best? Let’s take a look at the many options you have at your disposal. Attraction Spells

%+27788225528 love spells in Huntington Beach Psychic Readings, Attraction sp...

masabamasaba

%in ivory park+277-882-255-28 abortion pills for sale in ivory park

masabamasaba

%in Harare+277-882-255-28 abortion pills for sale in Harare

masabamasaba

The title is not connected to what is inside

shinachiaurasa2

Conference: Engage2024 in Antwerp Type: Workshop Speakers: Florian Vogler, Henning Kunz, Christoph Adler Title: Navigating the Future with The Hitchhiker's Guide to Notes and Domino 14 Abstract: Embark on an exhilarating journey with industry trailblazers Florian Vogler, Henning Kunz, and Christoph Adler in this not-to-be-missed workshop at the forefront of the tech universe. Get ready for a thrilling kick-off as we navigate the current state of the HCL universe, setting the stage for an exploration of the groundbreaking Notes and Domino 14. Discover the latest enhancements and revolutionary features that will redefine your experience. In this interactive session, unlock a treasure trove of tips and tricks to elevate your utilization of version 14, both with and without the game-changing panagenda MarvelClient. Brace yourself for also diving into Nomad, Nomad Web, and VoltMX, expanding your horizons in the expansive HCL landscape. Be a part of this exclusive opportunity to stay ahead in the ever-evolving world of HCL technologies. Your journey to mastering Notes and Domino 14 begins here. And remember, in the spirit of intergalactic exploration, don't forget to bring your towel!

W01_panagenda_Navigating-the-Future-with-The-Hitchhikers-Guide-to-Notes-and-D...

panagenda

%in Soweto+277-882-255-28 abortion pills for sale in soweto

masabamasaba

WSO2CON 2024 - Does Open Source Still Matter?

WSO2

Kürzlich hochgeladen (20)

WSO2CON 2024 - WSO2's Digital Transformation Journey with Choreo: A Platforml...

Direct Style Effect Systems -The Print[A] Example- A Comprehension Aid

Architecture decision records - How not to get lost in the past

Shapes for Sharing between Graph Data Spaces - and Epistemic Querying of RDF-...

WSO2CON 2024 - Building the API First Enterprise – Running an API Program, fr...

%+27788225528 love spells in new york Psychic Readings, Attraction spells,Bri...

WSO2CON2024 - It's time to go Platformless

%+27788225528 love spells in Boston Psychic Readings, Attraction spells,Bring...

%in Hazyview+277-882-255-28 abortion pills for sale in Hazyview

WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation

%in Rustenburg+277-882-255-28 abortion pills for sale in Rustenburg

Payment Gateway Testing Simplified_ A Step-by-Step Guide for Beginners.pdf

tonesoftg

%+27788225528 love spells in Huntington Beach Psychic Readings, Attraction sp...

%in ivory park+277-882-255-28 abortion pills for sale in ivory park

%in Harare+277-882-255-28 abortion pills for sale in Harare

The title is not connected to what is inside

W01_panagenda_Navigating-the-Future-with-The-Hitchhikers-Guide-to-Notes-and-D...

%in Soweto+277-882-255-28 abortion pills for sale in soweto

WSO2CON 2024 - Does Open Source Still Matter?

Making Scala Faster: 3 Expert Tips For Busy Development Teams

2. Who am I? • Scala hacker since 2005 • Worked mostly on Scala tooling (MiMa, Eclipse Scala IDE, Scala compiler) • Co-Founder of Triplequote • Make Scala compile as fast as Java thanks to Hydra

3. #1 Build time ≠ Compilation time

4. Build time ≠ Compilation time • Before starting compilation: • Dependency resolution • Source generators • Formatter • Settings evaluation

5. Build time ≠ Compilation time • Before starting compilation: • What do we need to compile? • Hash classpath, hash sources (a LOT of I/O) • Load analysis data

6. Build time ≠ Compilation time Finally Sbt starts compiling, but Zinc and incremental compiler have a 15% overhead

7. Use an IDE to build!

8. #2 Type Classes + Macro Derivation = 💣

9. Type Classes, Macro 💣 • Imports take precedence over companion objects • Cost: implicit resolution • Cost: additional code to be generated

10. Type Classes, Macro 💣 Decoders/ implicits 0 1 2 Code size after type-checking 8 KB 31 KB 54 KB

11. Type Classes, Macro 💣 D[ShippingAddress] D[FullName] D[FullName] D[String] D[String] D[String] D[String] D[String]

12. Avoid re-generating the same typeclass derivation multiple times!

13. #3 Whitebox macros are type-checked 3x!

14. What’s the different between whitebox and blackbox macros?

15. Which one takes part in type-inference? Shapeless macros are whitebox

16. #4 Type-checking should be around 30%

17. Type-checking should be around 30% • Type-checking Scala sources takes time • Type-checking should not go above 30% of total compilation time! • Use -verbose to see times for each phase (and a lot of noise)

18. Type-checking should be around 30% • If more, you can bet your money on one (or more) of: • Macro expansion (useful ﬂag: -Xprint:typer) • Implicit resolution (useful ﬂag: -Xlog-implicits) • Quasiquotes, type tags (macros in disguise)

19. Monitor compilation and identify bottlenecks Parallelize Scala compilation

20. #5 Scala compiler is single-threaded

21. Scala compiler is single- threaded • It’s a batch compiler • Careful interplay of laziness and mutability • Laziness required for recursive types • Mutability for performance

22. Scala compiler is single- threaded • Bigger/more powerful machines won’t make a dent in compilation times • Unless you have a parallel Scala compiler! • Hint: there is one that works today! (google Scala Hydra)

23. How fast is scalac? Simple code (Spark) FP-heavy (Cats) Play Framework (LiChess) Average Views Controllers 1’200 LoC/s 480 LoC/s 1’500 LoC/s 2’000 LoC/s 500 LoC/s (2.12.4 numbers, with a warm compiler — except LiChess: 2.11.11)

24. Summary • Compilation times are unpredictable • A single ﬁle can account for 80% of compilation time • Compilation speed varies greatly based on coding style • Macro expansion can quintuple your code size

25. https://twitter.com/berenguel/ status/1040712559362539524

26. Trusted by Parallelize Scala compilation Monitor compilation and identify bottlenecks https://triplequote.com/hydra/trial15-day FREE trial

Making Scala Faster: 3 Expert Tips For Busy Development Teams

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie Making Scala Faster: 3 Expert Tips For Busy Development Teams

Ähnlich wie Making Scala Faster: 3 Expert Tips For Busy Development Teams (20)

Mehr von Lightbend

Mehr von Lightbend (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Making Scala Faster: 3 Expert Tips For Busy Development Teams