Big Data Ecosystem - 1000 Simulated Drones

•

1 gefällt mir•874 views

A description of a complete Big Data ecosystem that can be used for operations on huge collections of data - even up to gigabytes of data per second, and a few hundred thousand customers connected in the same moment. The ecosystem can be upgraded with additional Apache tools: Apache Flume, Ambari, Mesos, Yarn.

Technologie

Presentation about Espeo's Big
Data ecosystem based on 1000
simulated drones ﬂying around
Poznan city

Drones produce and collect real
time sample data like:
latitude,
longitude,
height(m),
temp(C),
wind(m/s),
humidity,
air-polution

Drone soft written in Scala language
In real time it streams data to the server

Drone soft written in Scala language
In real time it streams data to the server
Using Kafka

On a server, data is read by Spark Streaming.
It allows us to:

It allows us to:
save data to Cassandra
send calculated data to browser through websocket
send it to another Kafka consumer
save the whole log to Hadoop cluster
On a server, data is read by Spark Streaming.

By saving logs to Hadoop cluster, we can later
access those logs, if we didn't save something
in Cassandra

By sending data to the browser through
websocket, we can see where our drones are in
realtime, monitor sensors and much more

By using Cassandra and Apache Spark data
scientists can analyze given data later,
by using:
1. Apache Zeppelin
- Apache Spark(df, RDD) + Scala
- Apache Spark MLLib
2. Azure Machine Learning

We prefer to use Azure Machine Learning
instead Spark MLLib because it is much easier
to understand - and design new predictions
Read our blog post about Azure ML:
http://espeo.eu/blog/azure-machine-learning-predictions/

CompleteComplete
ecosystem diagramecosystem diagram

Weitere ähnliche Inhalte

Was ist angesagt?

R is a hugely popular platform for Data Scientists to create analytic models in many different domains. But when these applications should move from the science lab to the production environment of large enterprises a new set of challenges arises. Independently of R, Spark has been very successful as a powerful general-purpose computing platform. With the introduction of SparkR an exciting new option to productionize Data Science applications has been made available. This talk will give insight into two real-life projects at major enterprises where Data Science applications in R have been migrated to SparkR. • Dealing with platform challenges: R was not installed on the cluster. We show how to execute SparkR on a Yarn cluster with a dynamic deployment of R. • Integrating Data Engineering and Data Science: we highlight the technical and cultural challenges that arise from closely integrating these two different areas. • Separation of concerns: we describe how to disentangle ETL and data preparation from analytic computing and statistical methods. • Scaling R with SparkR: we present what options SparkR offers to scale R applications and how we applied them to different areas such as time series forecasting and web analytics. • Performance Improvements: we will show benchmarks for an R applications that took over 20 hours on a single server/single-threaded setup. With moderate effort we have been able to reduce that number to 15 minutes with SparkR. And we will show how we plan to further reduces this to less than a minute in the future. • Mixing SparkR, SparkSQL and MLlib: we show how we combined the three different libraries to maximize efficiency. • Summary and Outlook: we describe what we have learnt so far, what the biggest gaps currently are and what challenges we expect to solve in the short- to mid-term.

Using SparkR to Scale Data Science Applications in Production. Lessons from t...

Spark Summit

Drizzle is a low latency execution engine for Apache Spark that is targeted at stream processing and iterative workloads. Currently, Spark uses a BSP computation model, and notifies the scheduler at the end of each task. Invoking the scheduler at the end of each task adds overheads and results in decreased throughput and increased latency. In Drizzle, we introduce group scheduling, where multiple batches (or a group) of computation are scheduled at once. This helps decouple the granularity of task execution from scheduling and amortize the costs of task serialization and launch. Our experiments on a 128 node EC2 cluster show that Drizzle can achieve end-to-end streaming latencies of less than 100ms and can get up to 3.5x lower latency than Spark Streaming. Compared to Apache Flink, a record-at-a-time streaming system, we show that Drizzle can recover around 4x faster from failures and that Drizzle has up to 13x lower latency during recovery.

Drizzle—Low Latency Execution for Apache Spark: Spark Summit East talk by Shi...

Spark Summit

Organizations from small startups to large enterprises are rapidly adopting Apache Spark on Amazon EMR in Amazon Web Services (AWS) to run streaming analytics, data science, machine learning, and batch processing workloads. These customers can quickly create big data architectures within minutes, and decouple compute and storage with Amazon S3 as a highly scalable, durable, and secure data lake, lower costs using Amazon EC2 Spot Instances and Auto Scaling, and utilize a wide range of encryption and access control features. In this session, we discuss how customers are using Spark on AWS and common architectures for easily running performant Spark clusters at scale and low cost with Amazon EMR.

Analytics at Scale with Apache Spark on AWS with Jonathan Fritz

Databricks

Spark Summit San Francisco 2016 - Ali Ghodsi Keynote

Databricks

Spark and Cassandra: An Amazing Apache Love Story by Patrick McFadin

Spark Summit

Big data remains a rapidly evolving field with new applications and infrastructure appearing every year. In this talk, I’ll cover new trends in 2016 / 2017 and how Apache Spark is moving to meet them. In particular, I’ll talk about work Databricks is doing to make Apache Spark interact better with native code (e.g. deep learning libraries), support heterogeneous hardware, and simplify production data pipelines in both streaming and batch settings through Structured Streaming.

Trends for Big Data and Apache Spark in 2017 by Matei Zaharia

Spark Summit

The prevailing issue when working with Operating Room (OR) scheduling within a hospital setting is that it is difficult to schedule and predict available OR block times. This leads to empty and unused operating rooms leading to longer waiting times for patients for their procedures. In this three-part session, Ayad Shammout and Denny will show: 1) How we tried to solve this problem using traditional DW techniques 2) How we took advantage of the DW capabilities in Apache Spark AND easily transition to Spark MLlib so we could more easily predict available OR block times resulting in better OR utilization and shorter wait times for patients. 3) Some of the key learnings we had when migrating from DW to Spark.

Transitioning from Traditional DW to Apache® Spark™ in Operating Room Predict...

Databricks

Integrating C* and Spark gives us a system that combines the best of both worlds. The goal of this integration is to obtain a better result than using Spark over HDFS because Cassandra´s philosophy is much closer to RDD's philosophy than what HDFS is. The goal with Cassandra is to have a system that mines all the information stored in C* in a much more efficient way than having the information stored in HDFS. Cassandra data storage and Spark data mining power: an unrivalled mix.

An efficient data mining solution by integrating Spark and Cassandra

Stratio

Migration de données structurées entre Hadoop et RDBMS par Louis Rabiet (Squid Solution) Avec l'extraction de données stockées dans une base de données relationnelle à l'aide d'un outil de BI avancé, et avec l'envoi via Kafka des données vers Tachyon, plusieurs sessions Spark peuvent travailler sur le même dataset en limitant la duplication. On obtient grâce à cela une communication à coût contrôlé entre la base de données d'origine et Spark ce qui permet de réintroduire de manière dynamique les données modifiées avec MLlib tout en travaillant sur des données à jour. Les résultats préliminaires seront partagés durant cette présentation.

HUG France Feb 2016 - Migration de données structurées entre Hadoop et RDBMS ...

Modern Data Stack France

Announcing Databricks Cloud (Spark Summit 2014)

Databricks

Learn how to deploy a managed Presto environment to interactively query log data on AWS Organizations often need to quickly analyze large amounts of data, such as logs, generated from a wide variety of sources and formats. However, traditional approaches require a lot of time and effort designing complex data transformation and loading processes; and configuring data warehouses. Using AWS, you can start querying your datasets within minutes In this webinar you will learn how you can deploy a managed Presto environment in minutes to interactively query log data using plain ANSI SQL. Presto is a popular open source SQL engine for running interactive analytic queries against data sources of all sizes. We will talk about common use cases and best practices for running Presto on Amazon EMR. Learning Objectives: • Learn how to deploy a managed Presto environment running on Amazon EMR • Understand best practices for running Presto on Amazon EMR, including use of Amazon EC2 Spot instances • Learn how other customers are using Presto to analyze large data sets

Running Fast, Interactive Queries on Petabyte Datasets using Presto - AWS Jul...

Amazon Web Services

R is the favorite language of many data scientists. In addition to a language and runtime, R is a rich ecosystem of libraries for a wide range of use cases from statistical inference to data visualization. However, handling large or distributed data with R is challenging. Hence R is used along with other frameworks and languages by most data scientist. In this mode most of the friction is at the interface of R and the other systems. For example, when data is sampled by a big data platform, results need to be transferred to and imported in R as native data structures. In this talk we show an alternative, and complimentary, approach to SparkR for integrating Spark and R. Since SparkR was released in version 1.4 of Apache Spark distributed data remains inside the JVM instead of individual R processes running on workers. This approach is more convenient when dealing with external data sources such as Cassandra, Hive, and Spark’s own distributed DataFrames. We show two specific techniques to remove the data transfer friction between R and JVM: collecting Spark DataFrames as R data frames and user space filesystems. We think this model complements and improves the day-to-day workload of many data scientists who use R. Spark’s interactive query processing, especially with in-memory datasets, closely matches the R interactive session model. When integrated together Spark and R can provide state of the art tools for the entire end-to-end data science pipeline. We will show how such a pipeline works in real world use cases in a live demo at the end of the talk.

Strata NYC 2015 - Supercharging R with Apache Spark

Databricks

How Spark Fits into Baidu's Scale-(James Peng, Baidu)

Spark Summit

Introduction to Spark R with R studio - Mr. Pragith

Sigmoid

Why spark by Stratio - v.1.0

Stratio

Hw09 Hadoop Applications At Yahoo!

Cloudera, Inc.

Spark Summit - Stratio Streaming

Stratio

While systems like Apache Spark have moved beyond a simple map-reduce model, many data scientists and scientific users still struggle with complex cluster management and configuration tools when trying to do data processing in the cloud. Recently, cloud providers have offered infrastructure such as AWS Lambda to run event-driven, stateless functions as micro-services. In this model, a function is deployed once and is invoked repeatedly whenever new inputs arrive and elastically scales with input size. In this session, the speakers claim that microservices on serverless infrastructure present a viable platform for eliminating cluster management overhead and fulfilling the promise of elasticity in cloud computing for all users. Their key insight is that they can dynamically inject code into these stateless functions and, combined with remote storage, they can build a data processing system that inherits the elasticity of the serverless model while addressing the simplicity required by end users. Using PyWren, their implementation on AWS Lambda, they show that this model is general enough to implement a number of distributed computing models, such as BSP, efficiently. Learn about a number of scientific and machine learning applications that they have built with PyWren, and how this model could be used to develop a serverless-Spark in the future.

Microservices and Teraflops: Effortlessly Scaling Data Science with PyWren wi...

Databricks

As Netflix expands their services to more countries, devices, and content, they continue to evolve their big data analytics platform to accommodate the increasing needs of product and consumer insights. This year, Netflix re-innovated their big data platform: they upgraded to Hadoop 2, transitioned to the Parquet file format, experimented with Pig on Tez for the ETL workload, and adopted Presto as their interactive querying engine. In this session, Netflix discusses their latest architecture, how they built it on the Amazon EMR infrastructure, the contributions put into the open source community, as well as some performance numbers for running a big data warehouse with Amazon S3.

(BDT403) Netflix's Next Generation Big Data Platform | AWS re:Invent 2014

Amazon Web Services

SparkR: Enabling Interactive Data Science at Scale

jeykottalam

Was ist angesagt? (20)

Using SparkR to Scale Data Science Applications in Production. Lessons from t...

Drizzle—Low Latency Execution for Apache Spark: Spark Summit East talk by Shi...

Analytics at Scale with Apache Spark on AWS with Jonathan Fritz

Spark Summit San Francisco 2016 - Ali Ghodsi Keynote

Spark and Cassandra: An Amazing Apache Love Story by Patrick McFadin

Trends for Big Data and Apache Spark in 2017 by Matei Zaharia

Transitioning from Traditional DW to Apache® Spark™ in Operating Room Predict...

An efficient data mining solution by integrating Spark and Cassandra

HUG France Feb 2016 - Migration de données structurées entre Hadoop et RDBMS ...

Announcing Databricks Cloud (Spark Summit 2014)

Running Fast, Interactive Queries on Petabyte Datasets using Presto - AWS Jul...

Strata NYC 2015 - Supercharging R with Apache Spark

How Spark Fits into Baidu's Scale-(James Peng, Baidu)

Introduction to Spark R with R studio - Mr. Pragith

Why spark by Stratio - v.1.0

Hw09 Hadoop Applications At Yahoo!

Spark Summit - Stratio Streaming

Microservices and Teraflops: Effortlessly Scaling Data Science with PyWren wi...

(BDT403) Netflix's Next Generation Big Data Platform | AWS re:Invent 2014

SparkR: Enabling Interactive Data Science at Scale

Andere mochten auch

The Big Data Ecosystem for Financial Services

DataStax

Building New Data Ecosystem for Customer Analytics, Strata + Hadoop World, 2016

Caserta

Amazon S3 is the central data hub for Netflix's big data ecosystem. We currently have over 1.5 billion objects and 60+ PB of data stored in S3. As we ingest, transform, transport, and visualize data, we find this data naturally weaving in and out of S3. Amazon S3 provides us the flexibility to use an interoperable set of big data processing tools like Spark, Presto, Hive, and Pig. It serves as the hub for transporting data to additional data stores / engines like Teradata, Redshift, and Druid, as well as exporting data to reporting tools like Microstrategy and Tableau. Over time, we have built an ecosystem of services and tools to manage our data on S3. We have a federated metadata catalog service that keeps track of all our data. We have a set of data lifecycle management tools that expire data based on business rules and compliance. We also have a portal that allows users to see the cost and size of their data footprint. In this talk, we’ll dive into these major uses of S3, as well as many smaller cases, where S3 smoothly addresses an important data infrastructure need. We will also provide solutions and methodologies on how you can build your own S3 big data hub.

AWS re:Invent 2016: Netflix: Using Amazon S3 as the fabric of our big data ec...

Amazon Web Services

Business disruptions with Robots, Drones and Algorithms for future world cong...

Sudha Jamthe

Temporal Databases: Data Models

torp42

For the full video of this presentation, please visit: http://www.embedded-vision.com/platinum-members/qualcomm/embedded-vision-training/videos/pages/may-2016-embedded-vision-summit-talluri For more information about embedded vision, please visit: http://www.embedded-vision.com Raj Talluri, Senior Vice President of Product Management at Qualcomm Technologies, presents the "Is Vision the New Wireless?" tutorial at the May 2016 Embedded Vision Summit. Over the past 20 years, digital wireless communications has become an essential technology for many industries, and a primary driver for the electronics industry. Today, computer vision is showing signs of following a similar trajectory. Once used only in low-volume applications such as manufacturing inspection, vision is now becoming an essential technology for a wide range of mass-market devices, from cars to drones to mobile phones. In this presentation, Talluri examines the motivations for incorporating vision into diverse products, presents case studies that illuminate the current state of vision technology in high-volume products, and explores critical challenges to ubiquitous deployment of visual intelligence.

"Is Vision the New Wireless?," a Presentation from Qualcomm

Edge AI and Vision Alliance

Business Disruption with Robots, Drones and Bots The IoT Tech Expo Oct 20 2016

Sudha Jamthe

JupyterHub for Interactive Data Science Collaboration

Carol Willing

Machine Vision for Intelligent Drones: An Overview

Kevin Heffner

I'm being followed by drones

DataWorks Summit/Hadoop Summit

Serene 2015 Davide Scaramuzza Abstract: With drones becoming more and more popular, safety is a big concern. A critical situation occurs when a drone temporarily loses its GPS position information, which might lead it to crash. This can happen, for instance, when flying close to buildings where GPS signal is lost. In such situations, it is desirable that the drone can rely on fall-back systems and regain stable flight as soon as possible. In this talk, I will present novel methods to automatically recover and stabilize a quadrotor from any initial condition or execute emergency landing. On the one hand, this new technology will allow quadrotors to be launched by simply tossing them in the air, like a “baseball ball”. On the other hand, it will allow them to recover back into stable flight or land on a safe area after a system failure. Since this technology does not rely on any external infrastructure, such as GPS, it enables the safe use of drones in both indoor and outdoor environments. Thus, it can become relevant for commercial use of drones, such as parcel delivery. Recent videos: Automatic failure recovery without GPS: https://youtu.be/pGU1s6Y55JI Autonomous Landing-site detection and landing: https://youtu.be/phaBKFwfcJ4

Towards Robust and Safe Autonomous Drones

SERENEWorkshop

Lawrence berkeley national laboratory sep 2015 - Jupyter Talk Scientific facilities are increasingly generating large data sets. Next-generation scientific productivity relies on user-friendly tools and efficient, effective and seamless access to resources and data. Traditional approaches to research and software development for science focus on the hardware and software of the machine and do not consider the user. In this talk, I will highlight a different approach to building software for scientific users by including user knowledge in the process. I will illustrate a few example projects where this has been used to date. GIthub repository: https://github.com/Carreau/talks/tree/master/labtech-2015

Jupyter, A Platform for Data Science at Scale

Matthias Bussonnier

BIg Data Trends in 2016

Stig-Arne Kristoffersen

1DMP: Marketing Data Platform - the future of data-driven marketing

CleverLEAF

Big data competitive landscape overview

Bisakha Praharaj

Creating R&D Centres

Drone Research

Drones - Market share , trends and Hardware

Sanchayan Sinha

Big Data Ecosystem at LinkedIn. Keynote talk at Big Data Innovators Gathering...

Mitul Tiwari

Independent of the source of data, the integration of event streams into an Enterprise Architecture gets more and more important in the world of sensors, social media streams and Internet of Things. Events have to be accepted quickly and reliably, they have to be distributed and analyzed, often with many consumers or systems interested in all or part of the events. Dependent on the size and quantity of such events, this can quickly be in the range of Big Data. How can we efficiently collect and transmit these events? How can we make sure that we can always report over historical events? How can these new events be integrated into traditional infrastructure and application landscape? Starting with a product and technology neutral reference architecture, we will then present different solutions using Open Source frameworks and the Oracle Stack both for on premises as well as the cloud.

Internet of Things and Big Data

Swiss Data Forum Swiss Data Forum

Thank Bunny - Customer Engagement Platform

Seshu Karthick

Andere mochten auch (20)

The Big Data Ecosystem for Financial Services

Building New Data Ecosystem for Customer Analytics, Strata + Hadoop World, 2016

AWS re:Invent 2016: Netflix: Using Amazon S3 as the fabric of our big data ec...

Business disruptions with Robots, Drones and Algorithms for future world cong...

Temporal Databases: Data Models

"Is Vision the New Wireless?," a Presentation from Qualcomm

Business Disruption with Robots, Drones and Bots The IoT Tech Expo Oct 20 2016

JupyterHub for Interactive Data Science Collaboration

Machine Vision for Intelligent Drones: An Overview

I'm being followed by drones

Towards Robust and Safe Autonomous Drones

Jupyter, A Platform for Data Science at Scale

BIg Data Trends in 2016

1DMP: Marketing Data Platform - the future of data-driven marketing

Big data competitive landscape overview

Creating R&D Centres

Drones - Market share , trends and Hardware

Big Data Ecosystem at LinkedIn. Keynote talk at Big Data Innovators Gathering...

Internet of Things and Big Data

Thank Bunny - Customer Engagement Platform

Ähnlich wie Big Data Ecosystem - 1000 Simulated Drones

Getting Started with Spark Structured Streaming - Current 22

Dustin Vannoy

Getting Started With Spark Structured Streaming With Dustin Vannoy | Current 2022 Many data pipelines still default to processing data nightly or hourly, but information is created all the time and should be available much sooner. While the move to stream processing adds complexity, Spark Structured Streaming makes it achievable for teams of any size to switch to streaming. This session shares techniques for data engineers who are new to building streaming pipelines with Spark Structured Streaming. It covers how to implement real-time stream processes with Apache Spark and Apache Kafka. We will discuss general concepts for Spark Structured Streaming along with introductory code examples. We will also look at important streaming concepts like triggers, windows, and state. To connect it all we will walk through a complete pipeline, including a demo using PySpark, Apache Kafka, and Delta Lake tables

Getting Started With Spark Structured Streaming With Dustin Vannoy | Current ...

HostedbyConfluent

Maria Patterson - Building a community fountain around your data stream

PyData

Fully Fault tolerant Streaming Workflows at Scale using Apache Mesos & Spark ...

Akhil Das

Learning spark ch10 - Spark Streaming

phanleson

Sparkstreaming with kafka and h base at scale (1)

Sigmoid

As the adoption of Spark Streaming in the industry is increasing, so is the community’s demand for more features. Since the beginning of this year, we have made significant improvements in performance, usability, and semantic guarantees. In particular, some of these features are: - New Kafka integration for exactly-once guarantees - Improved Kinesis integration for stronger guarantees - Addition of more sources to the Python API Significantly improved UI for greater monitoring and debuggability. In this talk, I am going to discuss these improvements as well as the plethora of features we plan to add in the near future.

Strata NYC 2015: What's new in Spark Streaming

Databricks

Building Scalable Data Pipelines - 2016 DataPalooza Seattle

Evan Chan

Apache spark installation [autosaved]

Shweta Patnaik

Spark Summit EU talk by Jim Dowling

Spark Summit

If you didn't attend, you don't want to miss a much shorter synopsis of what was covered and get some thoughts from us as to why they are important. We'll talk about the main topics of the event. 1. ACID transactions on Cassandra by Aaron Ploetz, Datastax 2. Apache Flink with Apache Cassandra at Satyajit Thadeswar, Netflix 3. Durable Execution built on Apache Cassandra by Loren Sands-Ramshaw, Temporal 4. Switching from Mongo to Cassandra with Mongoose & new Stargate JSON API, Valeri Karpov 5. Cloud Native and Realtime AI/ML with Patrick Mcfadin and Davor Boncaci, Datastax

Cassandra Lunch 130: Recap of Cassandra Forward Talks

Anant Corporation

Apache Kafka and KSQL in Action: Let's Build a Streaming Data Pipeline!

confluent

On-premise Spark as a Service with YARN

Jim Dowling

Visualizing C2_MLADS_2015

Todd Lanning

Four Things to Know About Reliable Spark Streaming with Typesafe and Databricks

Legacy Typesafe (now Lightbend)

How Sparkling Water brings Fast Scalable Machine learning via H2O to Apache Spark. By Michal Malohlava and H2O.ai Our 100th Meetup at 0xdata, September 30, 2014 Open Source meets Out Door. - Powered by the open source machine learning software H2O.ai. Contributors welcome at: https://github.com/h2oai - To view videos on H2O open source machine learning software, go to: https://www.youtube.com/user/0xdata

2014 09 30_sparkling_water_hands_on

Sri Ambati

Apache Pulsar is a cloud-native, distributed messaging and streaming platform. Apache SkyWalking is a popular application performance monitoring tool for distributed systems, specially designed for microservices, cloud-native, and container-based (Docker, K8s) architectures. When Apache SkyWalking meets Apache Pulsar, what will happen next? As we all know, message tracing is a good way that helps engineers troubleshoot problems related to messages publishing and receiving. In this talk, Sheng Wu and Penghui Li will walk through the features of Apache SkyWalking and Apache Pulsar, and run one step-by-step demo showing how to track Apache Pulsar messages by Apache SkyWalking.

Tracking Apache Pulsar Messages with Apache SkyWalking - Pulsar Virtual Summi...

StreamNative

Leverage Kafka to build a stream processing platform

confluent

Disaster Recovery in the Hadoop Ecosystem: Preparing for the Improbable

Stefan Kupstaitis-Dunkler

Alexey Orlenko ''High-performance IPC and RPC for microservices and apps''

OdessaJS Conf

Ähnlich wie Big Data Ecosystem - 1000 Simulated Drones (20)

Getting Started with Spark Structured Streaming - Current 22

Getting Started With Spark Structured Streaming With Dustin Vannoy | Current ...

Maria Patterson - Building a community fountain around your data stream

Fully Fault tolerant Streaming Workflows at Scale using Apache Mesos & Spark ...

Learning spark ch10 - Spark Streaming

Sparkstreaming with kafka and h base at scale (1)

Strata NYC 2015: What's new in Spark Streaming

Building Scalable Data Pipelines - 2016 DataPalooza Seattle

Apache spark installation [autosaved]

Spark Summit EU talk by Jim Dowling

Cassandra Lunch 130: Recap of Cassandra Forward Talks

Apache Kafka and KSQL in Action: Let's Build a Streaming Data Pipeline!

On-premise Spark as a Service with YARN

Visualizing C2_MLADS_2015

Four Things to Know About Reliable Spark Streaming with Typesafe and Databricks

2014 09 30_sparkling_water_hands_on

Tracking Apache Pulsar Messages with Apache SkyWalking - Pulsar Virtual Summi...

Leverage Kafka to build a stream processing platform

Disaster Recovery in the Hadoop Ecosystem: Preparing for the Improbable

Alexey Orlenko ''High-performance IPC and RPC for microservices and apps''

Mehr von Espeo Software

Distributed, immutable, secure...

Espeo Software

Evaluation of the legal nature of projects funded with ICOs is extremely important to define the legal and tax characteristics of tokens. The legal qualification of a token has practical implications for the legal status of its creator, token trading rules and other entities (buyers of tokens or intermediaries in their trade). Presentation will focus on legal types of tokens, specific requirements for ICO’s promoters and attitude of regulatory authorities towards token crowdsales in individual countries.

Initial Coin Offerings – legal requirements and types of tokens

Espeo Software

Blockchain technologies as we know now were not designed to sustain massive amount of computation operations: they are slow and extremely costly for such scenarios. Yet for many practical applications meaningful and sometimes massive amount of computation operations are a must. During the presentation, we will discuss and demonstrate how holding blockchain promises of trustlessness and data immutability we can still implement computation-intensive applications.

Trustless off chain computing on the blockchain

Espeo Software

How to sell your business idea to your customers & investors

Espeo Software

How to build a coin and start an ICO

Espeo Software

How to scale your tech startup for the win

Espeo Software

Before You Start Outsourcing Software Development [Checklist]

Espeo Software

What Should a Good Code Review Check?

Espeo Software

Espeo is an Agile software house focused on new, international startup projects. We're located in the heart of Poznań and we employ nearly 40 people. Our mission is helping others grow - both our customers as well as our developers. You'll have a real influence on the technology of the web and mobile applications we create. Flexible hours or remote work aren't a problem! Apply now: http://espeo.eu/open_positions/devops-engineer/

Espeo's looking for a DevOps Engineer!

Espeo Software

Software Team Efficiency: Velocity

Espeo Software

Introduction to Scrum: A How-To Guide

Espeo Software

To Hire or Not to Hire: In-house vs. Offshore Development

Espeo Software

Web Application Performance for Business Success

Espeo Software

Guide to Node.js: Basic to Advanced

Espeo Software

Docker: From Zero to Hero

Espeo Software

Azure Machine Learning

Espeo Software

Downloadable at ehealth.espeo.eu - 2016 might turn out to be a very special year for wearable health technology. This is mostly due to the fact that digital health is a rapidly growing segment, where new solutions are constantly being tested. We’ve been focusing on wearables for a while. In the beginning of February, we decided to see what an industry so keen on innovation may think. We directed our questions at over 400 people with CEO, CTO or specialist status in the healthcare industry. Here are the results, along with our professional opinions.

Report: Wearables in Healthcare

Espeo Software

Industrial Internet Solutions for Manufacturing & Logistics

Espeo Software

Big Data - Why is it important for business?

Espeo Software

A Future for Digital Health Wearables

Espeo Software

Mehr von Espeo Software (20)

Distributed, immutable, secure...

Initial Coin Offerings – legal requirements and types of tokens

Trustless off chain computing on the blockchain

How to sell your business idea to your customers & investors

How to build a coin and start an ICO

How to scale your tech startup for the win

Before You Start Outsourcing Software Development [Checklist]

What Should a Good Code Review Check?

Espeo's looking for a DevOps Engineer!

Software Team Efficiency: Velocity

Introduction to Scrum: A How-To Guide

To Hire or Not to Hire: In-house vs. Offshore Development

Web Application Performance for Business Success

Guide to Node.js: Basic to Advanced

Docker: From Zero to Hero

Azure Machine Learning

Report: Wearables in Healthcare

Industrial Internet Solutions for Manufacturing & Logistics

Big Data - Why is it important for business?

A Future for Digital Health Wearables

Kürzlich hochgeladen

AWS Community Day CPH - Three problems of Terraform

Andrey Devyatkin

Following the popularity of “Cloud Revolution: Exploring the New Wave of Serverless Spatial Data,” we’re thrilled to announce this much-anticipated encore webinar. In this sequel, we’ll dive deeper into the Cloud-Native realm by uncovering practical applications and FME support for these new formats, including COGs, COPC, FlatGeoBuf, GeoParquet, STAC, and ZARR. Building on the foundation laid by industry leaders Michelle Roby of Radiant Earth and Chris Holmes of Planet in the first webinar, this second part offers an in-depth look at the real-world application and behind-the-scenes dynamics of these cutting-edge formats. We will spotlight specific use-cases and workflows, showcasing their efficiency and relevance in practical scenarios. Discover the vast possibilities each format holds, highlighted through detailed discussions and demonstrations. Our expert speakers will dissect the key aspects and provide critical takeaways for effective use, ensuring attendees leave with a thorough understanding of how to apply these formats in their own projects. Elevate your understanding of how FME supports these cutting-edge technologies, enhancing your ability to manage, share, and analyze spatial data. Whether you’re building on knowledge from our initial session or are new to the serverless spatial data landscape, this webinar is your gateway to mastering cloud-native formats in your workflows.

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME

Safe Software

Elevate Developer Efficiency & build GenAI Application with Amazon Q

Bhuvaneswari Subramani

The value of a flexible API Management solution for Open Banking Steve Melan, Manager for IT Innovation and Architecture - State's and Saving's Bank of Luxembourg Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - The value of a flexible API Management solution for O...

apidays

Strategies for Landing an Oracle DBA Job as a Fresher

Remote DBA Services

Scaling API-first – The story of a global engineering organization Ian Reasor, Senior Computer Scientist - Adobe Radu Cotescu, Senior Computer Scientist - Adobe Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe

apidays

DBX First Quarter 2024 Investor Presentation

Dropbox

Artificial Intelligence Chap.5 : Uncertainty

Khushali Kathiriya

ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke

Product Anonymous

Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood

Juan lago vázquez

💥 You’re lucky! We’ve found two different (lead) developers that are willing to share their valuable lessons learned about using UiPath Document Understanding! Based on recent implementations in appealing use cases at Partou and SPIE. Don’t expect fancy videos or slide decks, but real and practical experiences that will help you with your own implementations. 📕 Topics that will be addressed: • Training the ML-model by humans: do or don't? • Rule-based versus AI extractors • Tips for finding use cases • How to start 👨‍🏫👨‍💻 Speakers: o Dion Morskieft, RPA Product Owner @Partou o Jack Klein-Schiphorst, Automation Developer @Tacstone Technology

DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam

UiPathCommunity

Effective data discovery is crucial for maintaining compliance and mitigating risks in today's rapidly evolving privacy landscape. However, traditional manual approaches often struggle to keep pace with the growing volume and complexity of data. Join us for an insightful webinar where industry leaders from TrustArc and Privya will share their expertise on leveraging AI-powered solutions to revolutionize data discovery. You'll learn how to: - Effortlessly maintain a comprehensive, up-to-date data inventory - Harness code scanning insights to gain complete visibility into data flows leveraging the advantages of code scanning over DB scanning - Simplify compliance by leveraging Privya's integration with TrustArc - Implement proven strategies to mitigate third-party risks Our panel of experts will discuss real-world case studies and share practical strategies for overcoming common data discovery challenges. They'll also explore the latest trends and innovations in AI-driven data management, and how these technologies can help organizations stay ahead of the curve in an ever-changing privacy landscape.

TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery

TrustArc

Keynote 2: APIs in 2030: The Risk of Technological Sleepwalk Paolo Malinverno, Growth Advisor - The Business of Technology Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...

apidays

[BuildWithAI] Introduction to Gemini.pdf

Sandro Moreira

Webinar Recording: https://www.panagenda.com/webinars/why-teams-call-analytics-is-critical-to-your-entire-business Nothing is as frustrating and noticeable as being in an important call and being unable to see or hear the other person. Not surprising then, that issues with Teams calls are among the most common problems users call their helpdesk for. Having in depth insight into everything relevant going on at the user’s device, local network, ISP and Microsoft itself during the call is crucial for good Microsoft Teams Call quality support. To ensure a quick and adequate solution and to ensure your users get the most out of their Microsoft 365. But did you know that ‘bad calls’ are also an excellent indicator of other problems arising? Precisely because it is so noticeable!? Like the canary in the mine, bad calls can be early indicators of problems. Problems that might otherwise not have been noticed for a while but can have a big impact on productivity and satisfaction. Join this session by Christoph Adler to learn how true Microsoft Teams call quality analytics helped other organizations troubleshoot bad calls and identify and fix problems that impacted Teams calls or the use of Microsoft365 in general. See what it can do to keep your users happy and productive! In this session we will cover - Why CQD data alone is not enough to troubleshoot call problems - The importance of attributing call problems to the right call participant - What call quality analytics can do to help you quickly find, fix-, and prevent problems - Why having retrospective detailed insights matters - Real life examples of how others have used Microsoft Teams call quality monitoring to problem shoot problems with their ISP, network, device health and more.

Why Teams call analytics are critical to your entire business

panagenda

The microservices honeymoon is over. When starting a new project or revamping a legacy monolith, teams started looking for alternatives to microservices. The Modular Monolith, or 'Modulith', is an architecture that reaps the benefits of (vertical) functional decoupling without the high costs associated with separate deployments. This talk will delve into the advantages and challenges of this progressive architecture, beginning with exploring the concept of a 'module', its internal structure, public API, and inter-module communication patterns. Supported by spring-modulith, the talk provides practical guidance on addressing the main challenges of a Modultith Architecture: finding and guarding module boundaries, data decoupling, and integration module-testing. You should not miss this talk if you are a software architect or tech lead seeking practical, scalable solutions. About the author With two decades of experience, Victor is a Java Champion working as a trainer for top companies in Europe. Five thousands developers in 120 companies attended his workshops, so he gets to debate every week the challenges that various projects struggle with. In return, Victor summarizes key points from these workshops in conference talks and online meetups for the European Software Crafters, the world’s largest developer community around architecture, refactoring, and testing. Discover how Victor can help you on victorrentea.ro : company training catalog, consultancy and YouTube playlists.

Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024

Victor Rentea

"I see eyes in my soup": How Delivery Hero implemented the safety system for ...

Zilliz

The Good, the Bad and the Governed - Why is governance a dirty word? David O'Neill, Chief Operating Officer - APIContext Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...

apidays

Angeliki Cooney has spent over twenty years at the forefront of the life sciences industry, working out of Wynantskill, NY. She is highly regarded for her dedication to advancing the development and accessibility of innovative treatments for chronic diseases, rare disorders, and cancer. Her professional journey has centered on strategic consulting for biopharmaceutical companies, facilitating digital transformation, enhancing omnichannel engagement, and refining strategic commercial practices. Angeliki's innovative contributions include pioneering several software-as-a-service (SaaS) products for the life sciences sector, earning her three patents. As the Senior Vice President of Life Sciences at Avenga, Angeliki orchestrated the firm's strategic entry into the U.S. market. Avenga, a renowned digital engineering and consulting firm, partners with significant entities in the pharmaceutical and biotechnology fields. Her leadership was instrumental in expanding Avenga's client base and establishing its presence in the competitive U.S. market.

Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...

Angeliki Cooney

How to Troubleshoot Apps for the Modern Connected Worker

ThousandEyes

Kürzlich hochgeladen (20)

AWS Community Day CPH - Three problems of Terraform

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME

Elevate Developer Efficiency & build GenAI Application with Amazon Q

Apidays New York 2024 - The value of a flexible API Management solution for O...

Strategies for Landing an Oracle DBA Job as a Fresher

Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe

DBX First Quarter 2024 Investor Presentation

Artificial Intelligence Chap.5 : Uncertainty

ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke

Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood

DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam

TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery

Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...

[BuildWithAI] Introduction to Gemini.pdf

Why Teams call analytics are critical to your entire business

Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024

"I see eyes in my soup": How Delivery Hero implemented the safety system for ...

Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...

Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...

How to Troubleshoot Apps for the Modern Connected Worker

Big Data Ecosystem - 1000 Simulated Drones

1. Big DataBig Data ecosystemecosystem

2. Presentation about Espeo's Big Data ecosystem based on 1000 simulated drones ﬂying around Poznan city

3. Drones produce and collect real time sample data like: latitude, longitude, height(m), temp(C), wind(m/s), humidity, air-polution

5. Drone soft written in Scala language

6. Drone soft written in Scala language In real time it streams data to the server

7. Drone soft written in Scala language In real time it streams data to the server Using Kafka

8. Drone soft written in Scala language In real time it streams data to the server Using Kafka

9. On a server, data is read by Spark Streaming. It allows us to:

10. It allows us to: save data to Cassandra send calculated data to browser through websocket send it to another Kafka consumer save the whole log to Hadoop cluster On a server, data is read by Spark Streaming.

11. By saving logs to Hadoop cluster, we can later access those logs, if we didn't save something in Cassandra

12. By sending data to the browser through websocket, we can see where our drones are in realtime, monitor sensors and much more

13.

14. By using Cassandra and Apache Spark data scientists can analyze given data later, by using: 1. Apache Zeppelin - Apache Spark(df, RDD) + Scala - Apache Spark MLLib 2. Azure Machine Learning

15. We prefer to use Azure Machine Learning instead Spark MLLib because it is much easier to understand - and design new predictions Read our blog post about Azure ML: http://espeo.eu/blog/azure-machine-learning-predictions/

16. CompleteComplete ecosystem diagramecosystem diagram

17. Drones Wiﬁ

18. Drones Wiﬁ

19. Drones Wiﬁ websocket

20. Drones Wiﬁ websocket

21. Drones Wiﬁ websocket

22. Drones Wiﬁ websocket API

23. Drones Wiﬁ websocket API