Streams and Things - Darach Ennis (Ubiquiti Networks)

•

1 like•1,341 views

Presented as a keynote at JAX London 2013 A programmer's perspective on the Internet of Things, reactive development, the hacker ethic, and how we can open up software development, making and craft to the masses. This talk features a live presentation of a home-made, modified Internet of Things. And robots. Audience participation... mandatory!

Technology Business

Streams and Things
JAX London 2013 - Darach Ennis - @darachennis

Internet of Things (noun) - a proposed
development of the Internet in which everyday
objects have network connectivity, allowing
them to send and receive data …
– “Internet of Things” - added to Oxford English Dictionary - August 2013

Internet of Things (noun) - a proposed
development of the Internet in which everyday
people can easily network everyday

objects, allowing them to send and receive
data …
– “Internet of Things” - a better deﬁnition - Darach Ennis, October 2013

Streams of Events - receiving, processing
and generating data ﬂows in near real-time
using software algorithms to analyse the event
data in

ﬂight…

AR Drone 2.0 Protocols
•

A ﬂying wireless access point with
•

9 Protocols: FTP, Auth, Video Recording,
Navigation Data, Video, Command, Raw
Capture, Printf, Control

•

Hackable: Linux based embedded system

Embedded Event
Processing
•

Simple embeddable subset of CEP

•

Flow a la Node.js streams and pipes

•

Aggregate windows

•

Extensible, modular

•

Functional, reactive, easy to port

Embedded Event
Processing
•

Windows / Aggregates
•

eep-js

•

eep-php

•

beam-js

•

eep-clj!

•

beam-erl

•

eep-java!

•

beam-java

•

eep-erl

•

Flow

ﬁlter!
up
ﬁlter!
down

In

ﬁlter!
keys

ﬂy!
up
ﬂy!
down

…

…

ﬁlter!
right

ﬂy!
right

land?

do!
land

Streams and Things
… putting it all together … demo, with you and robots!

People and Things - can we take the
simplicity of an educational tool
such as scratch and empower
ordinary people to hack and
mashup their things in ways ‘we the
creators’ never imagined?!
!
– “Internet of Things” - added to Oxford English Dictionary - August 2013
!

!

Thanks.
Questions?
!

@darachennis
http://github.com/darach/eep-java!
!
http://github.com/darach/beam-java

What's hot

The first step in any data-intensive project is understanding the available data. To this end, data scientists spend a significant part of their time carrying out data quality assessments and data exploration. In spite of this being a crucial step, it usually requires repeating a series of menial tasks before the data scientist gains an understanding of the dataset and can progress to the next steps in the project. In this talk I will present Lens (https://github.com/asidatascience/lens), a Python package which automates this drudge work, enables efficient data exploration, and kickstarts data science projects. A summary is generated for each dataset, including: - General information about the dataset, including data quality of each of the columns; - Distribution of each of the columns through statistics and plots (histogram, CDF, KDE), optionally grouped by other categorical variables; - 2D distribution between pairs of columns; - Correlation coefficient matrix for all numerical columns. Building this tool has provided a unique view into the full Python data stack, from the parallelised analysis of a dataframe within a Dask custom execution graph, to the interactive visualisation with Jupyter widgets and Plotly. During the talk, I will also introduce how Dask works, and demonstrate how to migrate data pipelines to take advantage of its scalable capabilities.

Lens: Data exploration with Dask and Jupyter widgets

Víctor Zabalza

This talk discusses integrating common data science tools like Python pandas, scikit-learn, and R with MLlib, Spark’s distributed Machine Learning (ML) library. Integration is simple; migration to distributed ML can be done lazily; and scaling to big data can significantly improve accuracy. We demonstrate integration with a simple data science workflow. Data scientists often encounter scaling bottlenecks with single-machine ML tools. Yet the overhead in migrating to a distributed workflow can seem daunting. In this talk, we demonstrate such a migration, taking advantage of Spark and MLlib’s integration with common ML libraries. We begin with a small dataset which runs on a single machine. Increasing the size, we hit bottlenecks in various parts of the workflow: hyperparameter tuning, then ETL, and eventually the core learning algorithm. As we hit each bottleneck, we parallelize that part of the workflow using Spark and MLlib. As we increase the dataset and model size, we can see significant gains in accuracy. We end with results demonstrating the impressive scalability of MLlib algorithms. With accuracy comparable to traditional ML libraries, combined with state-of-the-art distributed scalability, MLlib is a valuable new tool for the modern data scientist.

Spark Summit EU 2015: Combining the Strengths of MLlib, scikit-learn, and R

Databricks

London Spark Meetup 2014-11-11 @Skimlinks http://www.meetup.com/Spark-London/events/217362972/ To paraphrase the immortal crooner Don Ho: "Tiny Batches, in the wine, make me happy, make me feel fine." http://youtu.be/mlCiDEXuxxA Apache Spark provides support for streaming use cases, such as real-time analytics on log files, by leveraging a model called discretized streams (D-Streams). These "micro batch" computations operated on small time intervals, generally from 500 milliseconds up. One major innovation of Spark Streaming is that it leverages a unified engine. In other words, the same business logic can be used across multiple uses cases: streaming, but also interactive, iterative, machine learning, etc. This talk will compare case studies for production deployments of Spark Streaming, emerging design patterns for integration with popular complementary OSS frameworks, plus some of the more advanced features such as approximation algorithms, and take a look at what's ahead — including the new Python support for Spark Streaming that will be in the upcoming 1.2 release. Also, let's chat a bit about the new Databricks + O'Reilly developer certification for Apache Spark…

Tiny Batches, in the wine: Shiny New Bits in Spark Streaming

Paco Nathan

Building data pipelines

Jonathan Holloway

Distributed Deep Learning on Hadoop Clusters

DataWorks Summit/Hadoop Summit

Real-Time Anomoly Detection with Spark MLib, Akka and Cassandra by Natalino Busa

Spark Summit

Graph analytics has a wide range of applications, from information propagation and network flow optimization to fraud and anomaly detection. The rise of social networks and the Internet of Things has given us complex web-scale graphs with billions of vertices and edges. However, in order to extract the hidden gems within those graphs, you need tools to analyze the graphs easily and efficiently. At Spark Summit 2016, Databricks introduced GraphFrames, which implemented graph queries and pattern matching on top of Spark SQL to simplify graph analytics. In this talk, you'll learn about work that has made graph algorithms in GraphFrames faster and more scalable. For example, new implementations like connected components have received algorithm improvements based on recent research, as well as performance improvements from Spark DataFrames. Discover lessons learned from scaling the implementation from millions to billions of nodes; compare its performance with other popular graph libraries; and hear about real-world applications.

Challenging Web-Scale Graph Analytics with Apache Spark with Xiangrui Meng

Databricks

While systems like Apache Spark have moved beyond a simple map-reduce model, many data scientists and scientific users still struggle with complex cluster management and configuration tools when trying to do data processing in the cloud. Recently, cloud providers have offered infrastructure such as AWS Lambda to run event-driven, stateless functions as micro-services. In this model, a function is deployed once and is invoked repeatedly whenever new inputs arrive and elastically scales with input size. In this session, the speakers claim that microservices on serverless infrastructure present a viable platform for eliminating cluster management overhead and fulfilling the promise of elasticity in cloud computing for all users. Their key insight is that they can dynamically inject code into these stateless functions and, combined with remote storage, they can build a data processing system that inherits the elasticity of the serverless model while addressing the simplicity required by end users. Using PyWren, their implementation on AWS Lambda, they show that this model is general enough to implement a number of distributed computing models, such as BSP, efficiently. Learn about a number of scientific and machine learning applications that they have built with PyWren, and how this model could be used to develop a serverless-Spark in the future.

Microservices and Teraflops: Effortlessly Scaling Data Science with PyWren wi...

Databricks

Analyzing and comparing your energy consumption with that of other consumers provides healthy peer pressure and useful insight leading to energy conservation and impacting the bottom line. We helped GridPocket (http://www.gridpocket.com/), a smart grid company developing energy management applications for electricity water and gas utilities, implement high scale anonymized energy comparison queries with an order of magnitude lower cost and higher performance than was previously possible. IoT use cases like that of GridPocket are swamping our planet with data, and drive demand for analytics on extremely scalable and low cost storage. Enter Spark SQL over Object Storage: highly scalable and low cost storage which provides RESTful APIs to store and retrieve objects and their metadata. Key performance indicators (KPIs) of query performance and cost are the number of bytes shipped from Object Storage to Spark and the number of incurred REST requests. We propose Pluggable Spark SQL Filters, which extend the existing Spark SQL partitioning mechanism with an ability to dynamically filter irrelevant objects during query execution. Our approach handles any data format supported by Spark SQL (Parquet, JSON, csv etc.), and unlike pushdown compatible formats such as Parquet which require touching each object to determine its relevance, it avoids accessing irrelevant objects altogether. We developed a pluggable interface for developing and deploying Filters, and implemented GridPocket’s filter which screens objects according to their metadata, for example geo-spatial bounding boxes which describe the area covered by an object’s data points. This leads to drastically lower KPIs since there is no need to ship the entire dataset from Object Storage to Spark if you are only comparing yourself with your neighborhood. We demonstrate GridPocket analytics notebooks, report on our implementation and resulting 10-20x speedups, explain how to implement a Pluggable File Filter, and how we applied this to other use cases.

Using Pluggable Apache Spark SQL Filters to Help GridPocket Users Keep Up wit...

Spark Summit

Spark and Cassandra: An Amazing Apache Love Story by Patrick McFadin

Spark Summit

Spark Application Carousel: Highlights of Several Applications Built with Spark

Databricks

Re-envisioning the Lambda Architecture : Web Services & Real-time Analytics ...

Brian O'Neill

Spark Summit EU 2015: Reynold Xin Keynote

Databricks

The CERN experiments and their particle accelerator, the Large Hadron Collider (LHC), will soon have collected a total of one exabyte of data. Moreover, the next upgrade of the accelerator, the high-luminosity LHC, will dramatically increase the rate of particle collisions, thus boosting the potential for discoveries but also generating unprecedented data challenges. In order to process and analyse all those data, CERN is investigating complementary ways to the traditional approaches, which mainly rely on Grid and batch jobs for data reconstruction, calibration and skimming combined with a phase of local analysis of reduced data. The new techniques should allow for interactive analysis on much bigger datasets by transparently exploiting dynamically pluggable resources. In that sense, Spark is being used at CERN to process large physics datasets in a distributed fashion. The most widely used tool for high-energy physics analysis, ROOT, implements a layer on top of Spark in order to distribute computations across a cluster of machines. This makes it possible for physics analysis written in either C++ or Python to be parallelised on Spark clusters, while reading the input data from CERN’s mass storage system: EOS. On the other hand, another important use case of Spark at CERN has recently emerged. The LHC logging service, which collects data from the accelerator to get information on how to improve the performance of the machine, is currently migrating its architecture to leverage Spark for its analytics workflows. This talk will discuss the unique challenges of the aforementioned use cases and how SWAN, the CERN service for interactive web-based analysis, now supports them thanks to a new feature: the possibility for users to dynamically plug Spark clusters into their sessions in order to offload computations to those resources.

CERN’s Next Generation Data Analysis Platform with Apache Spark with Enric Te...

Databricks

Strata EU 2014: Spark Streaming Case Studies

Paco Nathan

Data Pipelines & Integrating Real-time Web Services w/ Storm : Improving on t...

Brian O'Neill

Implementing the Lambda Architecture efficiently with Apache Spark

DataWorks Summit

Spark 2.0 provided strong performance enhancements to the Spark core while advancing Spark ML usability to use data frames. But what happens when you run Spark 2.0 machine learning algorithms on a large cluster with a very large data set? Do you even get any benefit from using a very large data set? It depends.  How do new hardware advances affect the topology of high performance Spark clusters. In this talk we will explore Spark 2.0 Machine Learning at scale and share our findings with the community. As our test platform we will be using a new cluster design, different from typical Hadoop clusters, with more cores, more RAM and latest generation NVMe SSD’s and a 100GbE network with a goal of more performance, in a more space and energy efficient footprint.

Practical Large Scale Experiences with Spark 2.0 Machine Learning: Spark Summ...

Spark Summit

Metrics play an important role in data-driven companies like LinkedIn, where we leverage them extensively for reporting, experimentation, and in-product applications. We built an offline platform to help people define and produce metrics driven through their transformation code, mostly in Pig or Hive, and metadata-rich configurations. Many of our users would like to look at these metrics in a real-time fashion. To support this, we recently built an extension to the platform that auto-generates Samza real-time flow from existing offline transformation code with just a single command. Combining with the existing offline platform, we delivered Lambda architecture without maintaining multiple code bases. In this talk, we will describe how we use Apache Calcite to translate our offline logic, served as the single source of truth, into both Samza code and configuration for real-time execution.

Conquering the Lambda architecture in LinkedIn metrics platform with Apache C...

Khai Tran

Graph databases: Tinkerpop and Titan DB

Mohamed Taher Alrefaie

What's hot (20)

Lens: Data exploration with Dask and Jupyter widgets

Spark Summit EU 2015: Combining the Strengths of MLlib, scikit-learn, and R

Tiny Batches, in the wine: Shiny New Bits in Spark Streaming

Building data pipelines

Distributed Deep Learning on Hadoop Clusters

Real-Time Anomoly Detection with Spark MLib, Akka and Cassandra by Natalino Busa

Challenging Web-Scale Graph Analytics with Apache Spark with Xiangrui Meng

Microservices and Teraflops: Effortlessly Scaling Data Science with PyWren wi...

Using Pluggable Apache Spark SQL Filters to Help GridPocket Users Keep Up wit...

Spark and Cassandra: An Amazing Apache Love Story by Patrick McFadin

Spark Application Carousel: Highlights of Several Applications Built with Spark

Re-envisioning the Lambda Architecture : Web Services & Real-time Analytics ...

Spark Summit EU 2015: Reynold Xin Keynote

CERN’s Next Generation Data Analysis Platform with Apache Spark with Enric Te...

Strata EU 2014: Spark Streaming Case Studies

Data Pipelines & Integrating Real-time Web Services w/ Storm : Improving on t...

Implementing the Lambda Architecture efficiently with Apache Spark

Practical Large Scale Experiences with Spark 2.0 Machine Learning: Spark Summ...

Conquering the Lambda architecture in LinkedIn metrics platform with Apache C...

Graph databases: Tinkerpop and Titan DB

Similar to Streams and Things - Darach Ennis (Ubiquiti Networks)

Big Data to SMART Data : Process scenario Scenario of an implementation of a transformation process of the Data towards exploitable data and representative with treatments of the streaming, the distributed systems, the messages, the storage in an NoSQL environment, a management with an ecosystem Big Data graphic visualization of the data with the technologies: Apache Storm, Apache Zookeeper, Apache Kafka, Apache Cassandra, Apache Spark and Data-Driven Document.

Big Data to SMART Data : Process Scenario

CHAKER ALLAOUI

The Things in the INTERNET of THINGS.

Rohit Agrawal

You want to implement a Big Data or Internet of Things (IoT) solution and like to know if it should be implemented in the cloud or on-premises. You are interested in the cloud offerings of vendors and what benefits they provide and if a similar solution would not be possible on-premises. This presentation deals with this and other questions. Starting from a vendor-independent reference architecture and corresponding design patterns, different cloud solutions from various vendors are compared and rated. Additionally, it will be shown how such solution could be implemented on-premises and how a hybrid IoT solution could look like.

Internet of Things (IoT) - in the cloud or rather on-premises?

Guido Schmutz

Social Semantic (Sensor) Web

David Crowley

The evolving threat in the face of increased connectivity

APNIC

Independent of the source of data, the integration of event streams into an Enterprise Architecture gets more and more important in the world of sensors, social media streams and Internet of Things. Events have to be accepted quickly and reliably, they have to be distributed and analyzed, often with many consumers or systems interested in all or part of the events. Dependent on the size and quantity of such events, this can quickly be in the range of Big Data. How can we efficiently collect and transmit these events? How can we make sure that we can always report over historical events? How can these new events be integrated into traditional infrastructure and application landscape? Starting with a product and technology neutral reference architecture, we will then present different solutions using Open Source frameworks and the Oracle Stack both for on premises as well as the cloud.

Internet of Things and Big Data

Swiss Data Forum Swiss Data Forum

DataStax and Esri: Geotemporal IoT Search and Analytics

DataStax Academy

Unit 1.pptx

AbhishekTekavade1

Smart Cities….Smart Future

PayamBarnaghi

Methods and Instruments for the new Digital Forensics Environments

piccimario

The Future of the Internet

PayamBarnaghi

Appsec 2013-krehel-ondrej-forensic-investigations-of-web-exploitations

drewz lin

AGIT 2015 - Keynote M.Hauswirth: "Linking Everything"

jstrobl

With the advent of IOT, Every 'Thing' is getting Smart, starting from the range of smartwatches, smart refrigerators, smart bulbs to smart car, smart healthcare, smart agriculture, smart retail, smart city and what not, even smart planet. But why is every thing getting smart? People are trying to bridge the gap between Digital World & Physical World by means of ubiquitous connectivity to Internet, and when digital things become physical, digital threats also become physical threats. Security & Privacy issues are rising as never before. What if the microphone in your smart TV can be used to eavesdrop the private communications in your bed room? What if a smart driverless car deliberately crashes itself into an accident? What if you want to be Anonymous over Internet and don't want anybody to track you? This talk will focus on answering the above questions with a view on 'What are we currently doing to protect ourselves' and 'What we need to do'. What are the new security challenges that are coming up and how privacy & anonymity is taking the lead over security. The talk will also sensitive the audience about the paradigm shift that is happening in IOT DevOps, with help of Docker Containers and how they can be anonymised using TOR.

Touring the Dark Side of Internet: A Journey through IOT, TOR & Docker

Abhinav Biswas

Introduction to an Internet of Things system. This session will go through a real system: my own sprinkler system including sensors, data manipulation, consumption, BI. This will give you an overview of a full projects, from the device side to the storage, consumption, analyze and insights. Boards like Raspberry Pi running Linux, Windows as well as Arduino and Netduino are used. The server side is based on Azure using services like Azure IoT Hub, Stream Analytics, Mobile Services, SQL Azure and more!

Create The Internet of Your Things example of a real system - Laurent Ellerbach

ITCamp

Internet of Things

Aniekan Akpaffiong

IoT overview 2014

Mirko Presser

On-demand EO Processing Services in a federation of European Ground Segments Centralisation vs FedearaMake things work Limit as possible data transfert be « Bring process to data » vs « Bring data to process » when possible tween clouds Processing results should be downloaded only if needed Good option is to offer a Web service to visualize/manipulate results Interoperability layers between clouds Authentication, Catalogs search, Virtualization High speed network between clouds

2014.09.04 federated ground segments - toulouse

Gasperi Jerome

The Future of the Internet – Real-Time-Web by Synaptic

MobileMonday Norway

The Real-time Web in the Age of Agents

Joshua Shinavier

Similar to Streams and Things - Darach Ennis (Ubiquiti Networks) (20)

Big Data to SMART Data : Process Scenario

The Things in the INTERNET of THINGS.

Internet of Things (IoT) - in the cloud or rather on-premises?

Social Semantic (Sensor) Web

The evolving threat in the face of increased connectivity

Internet of Things and Big Data

DataStax and Esri: Geotemporal IoT Search and Analytics

Unit 1.pptx

Smart Cities….Smart Future

Methods and Instruments for the new Digital Forensics Environments

The Future of the Internet

Appsec 2013-krehel-ondrej-forensic-investigations-of-web-exploitations

AGIT 2015 - Keynote M.Hauswirth: "Linking Everything"

Touring the Dark Side of Internet: A Journey through IOT, TOR & Docker

Create The Internet of Your Things example of a real system - Laurent Ellerbach

Internet of Things

IoT overview 2014

2014.09.04 federated ground segments - toulouse

The Future of the Internet – Real-Time-Web by Synaptic

The Real-time Web in the Age of Agents

More from jaxLondonConference

Garbage Collection: the Useful Parts - Martijn Verburg & Dr John Oliver (jCla...

jaxLondonConference

This talk was due to be presented at JAX London 2013, but the speaker was unfortunately unable to attend. Distributed data stores give us increased availability, linear scalability, predictable latency and improved fault tolerance. The flip-side is having to deal with inconsistencies: most distributed databases will ask your application layer how to resolve such inconsistencies. Conflict-free Replicated Data Types (CRDTs) are a way for a distributed database, such as Riak, to resolve those inconsistencies logically and automatically. Unlike traditional data structures, there is always a single state on which they converge. In this talk, I’ll look at the development of CRDTs from an academic project to implementation in Riak.

Conflict Free Replicated Data-types in Eventually Consistent Systems - Joel J...

jaxLondonConference

Presented at JAX London 2013 Per-tenant resource management can help ensure that collocated tenants peacefully share computational resources based on individual quotas. This session begins with a comparison of deployment models (shared: hardware, OS, middleware, everything) to motivate the multitenant approach. The main topic is an exploration of experimental data isolation and resource management primitives in IBM’s JDK that combine to help make multitenant applications smaller and more predictable.

JVM Support for Multitenant Applications - Steve Poole (IBM)

jaxLondonConference

Presented at JAX London 2013 Worried about the future of Java? Want to see it keep moving forward? Don't be concerned. The transformation of Java is already underway. Driven by new technologies and new opportunities Java and the JVM are entering uncharted worlds and challenging old approaches. In this session learn about one such expedition in the form of an introductory talk to technology being developed by IBM. This experimental technology is exploring a new way to share data between the JVM and other runtimes.

Packed Objects: Fast Talking Java Meets Native Code - Steve Poole (IBM)

jaxLondonConference

Presented as a keynote at JAX London 2013 Lambdas are coming to the Java language in the upcoming release of Java 8! While this is generally great news, many Java developers have never experienced Lambdas before, and have not yet learned the best ways to use them for maximum productivity. In this talk, we will discuss best practices for using Lambdas in Java and other JVM-based languages, and we will investigate how we can make these constructs more usable in production.

What You Need to Know About Lambdas - Jamie Allen (Typesafe)

jaxLondonConference

Presented at JAX London 2013. Software craftsman and co-founder of the London Software Craftsmanship Community (LSCC). Sandro has been coding since a very young age but just started his professional career in 1996. He has worked for startups, software houses, product companies and international consultancy companies. Having worked as a consultant for the majority of his career, he had the opportunity to work in a good variety of projects, with different languages and technologies, and across many industries. Currently he is a director at UBS Investment Bank, where he works as a hands-on mentor, giving technical directions, looking after the quality of the systems and pair-programming with developers in the UK and abroad. His main objective is to help developers to become real software craftsmen.

Why other ppl_dont_get_it

jaxLondonConference

Databases and agile development - Dwight Merriman (MongoDB)

jaxLondonConference

Presented at JAX London 2013 Vert.x is a lightweight, high performance, application platform for the JVM that's designed for modern mobile, web, and enterprise applications. The recent Vert.x 2.0 release marks a coming of age for Vert.x, as it progresses to a fully independent project. We'll dive into the Vert.x 2.0 release and show how the powerful new module system enables a Vert.x ecosystem by allowing modules to be re-used via Maven and Bintray repositories. You'll also learn about how better build tool and IDE integration makes developing applications with Vert.x a breeze.

Introducing Vert.x 2.0 - Taking polyglot application development to the next ...

jaxLondonConference

Presented at JAX London 2013 Hypermedia or HATEOAS APIs get a lot of air cover but there aren't many of them in the wild. This session will summarize the challenges that exist when building hypermedia REST API’s and explain why it’s worth it. We’ll take a comparative look at various different approaches for using hypermedia in a typical REST API, before taking a closer look at an approach that automatically generates links from a semantically rich API definition.

Are Hypermedia APIs Just Hype? - Aaron Phethean (Temenos) & Daniel Feist (Mul...

jaxLondonConference

Presented at JAX London 2013 Whether we're talking Analytics, Big Data, Cloud, NoSQL, Continuous Integration and Deployment, Mobile computing or Minecraft, Java is at the nexus of the mass convergence of trends we're currently seeing in tech. It's still fashionable in Web developer circles to dismiss Java the language, but its not going anywhere, and the JVM as we know it goes from strength to strength. In this talk James Governor will look to put the return of Java into context, with both stories, and quantitative data.

How Java got its Mojo Back - James Governor (Redmonk)

jaxLondonConference

Real-world polyglot programming on the JVM - Ben Summers (ONEIS)

jaxLondonConference

The Spock unit testing framework is on the verge of a 1.0 release and has already proven itself to be the next generation thinking on how to test Java production code. One of the many ever present challenges to testing code is the ability to Mock classes which has simplified by Spock from a very early release. Recently added to Spock is the notion of Stubs and Spies. This sessions is designed to demonstrate proper unit testing technique showing off these new features along with a number of advanced Spock features.

Java Testing With Spock - Ken Sipe (Trexin Consulting)

jaxLondonConference

Presented at JAX London MapReduce begat Hadoop begat Big Data. NoSQL moved us away from the stricture of monolithic storage architectures to fit-for-purpose designs. But, Houston, we still have a problem. Architects are still designing systems like this is the '70s. SOA, went from buzzword to the bank with the emergence and evolution of the cloud and on-demand right-now elasticity. Yet most systems are still designed to store-then-compute rather than to observe, orient, decide and act on in-flight data.

Big Events, Mob Scale - Darach Ennis (Push Technology)

jaxLondonConference

Presented at JAX London 2013 Groovy is not a newcomer to the arena of alternative languages for the JVM. With over 1.7 million downloads a year, it's clearly ahead of the pack. But what makes it a great choice for your projects? - a flat learning curve for Java developers - its seamless Java integration where you can mix & mash Groovy & Java together - a malleable & concise syntax fit for Domain-Specific Languages - an interesting take on type safety - its rich ecosystem of projects: Grails, Gradle, GPars, Spock, Griffon, Geb...

What makes Groovy Groovy - Guillaume Laforge (Pivotal)

jaxLondonConference

Presented at JAX London 2013 Ever since Java’s inception, in 1995, people have been compiling languages that aren’t Java to bytecode and deploying them on the JVM. Lately we are seeing an explosion in JVM languages. This is partly, but not only, because of Java 7, the first JVM to ship with invokedynamic, which is a quantum leap in polyglot runtime implementation. This session explains why emerging language implementations are becoming more common and more feasible to implement on the JVM with satisfactory performance.

The Java Virtual Machine is Over - The Polyglot VM is here - Marcus Lagergren...

jaxLondonConference

Presented at JAX London 2013 The Java EE 7 platform focuses on Productivity and HTML5. JAX-RS 2 adds a new Client API to invoke the RESTful endpoints. JMS 2 is undergoing a complete overhaul to align with improvements in the Java language. The long awaited Batch Processing API and Concurrency API are also getting added to build applications using capabilities of the platform itself. Together these APIs will allow you to be more productive by simplifying enterprise development. WebSocket attempts to solve the issues and limitations of HTTP for real-time communication.

Java EE 7 Platform: Boosting Productivity and Embracing HTML5 - Arun Gupta (R...

jaxLondonConference

Exploring the Talend unified Big Data toolset for sentiment analysis - Ben Br...

jaxLondonConference

The Curious Clojurist - Neal Ford (Thoughtworks)

jaxLondonConference

Presented at JAX London 2013 Test Driven Development is a practice generally endorsed by most people. However it is also one of the most difficult to get right. I am part of a very large project where we decided to use TDD from the very start. We encountered a number of challenges and learned a lot of lessons. We are still learning and evolving our approach to TDD. We discovered that doing TDD badly is actually worse than not doing TDD at all and that it is very important to get some basics rights otherwise you'll put yourself in a world of pain.

TDD at scale - Mash Badar (UBS)

jaxLondonConference

Presented at JAX London 2013 Public, private, and hybrid; software, platform, and infrastructure. This talk will discuss the current state of the Platform-as-a-Service space, and why the keys to success lie in enabling developer productivity, and providing openness and choice. We'll do this by considering the success of Open Source in general, look at the Cloud Foundry project, and find out why Cloud Foundry-based PaaSes are the best places to host your applications written in Java and other JVM-based languages.

Run Your Java Code on Cloud Foundry - Andy Piper (Pivotal)

jaxLondonConference

More from jaxLondonConference (20)

Garbage Collection: the Useful Parts - Martijn Verburg & Dr John Oliver (jCla...

Conflict Free Replicated Data-types in Eventually Consistent Systems - Joel J...

JVM Support for Multitenant Applications - Steve Poole (IBM)

Packed Objects: Fast Talking Java Meets Native Code - Steve Poole (IBM)

What You Need to Know About Lambdas - Jamie Allen (Typesafe)

Why other ppl_dont_get_it

Databases and agile development - Dwight Merriman (MongoDB)

Introducing Vert.x 2.0 - Taking polyglot application development to the next ...

Are Hypermedia APIs Just Hype? - Aaron Phethean (Temenos) & Daniel Feist (Mul...

How Java got its Mojo Back - James Governor (Redmonk)

Real-world polyglot programming on the JVM - Ben Summers (ONEIS)

Java Testing With Spock - Ken Sipe (Trexin Consulting)

Big Events, Mob Scale - Darach Ennis (Push Technology)

What makes Groovy Groovy - Guillaume Laforge (Pivotal)

The Java Virtual Machine is Over - The Polyglot VM is here - Marcus Lagergren...

Java EE 7 Platform: Boosting Productivity and Embracing HTML5 - Arun Gupta (R...

Exploring the Talend unified Big Data toolset for sentiment analysis - Ben Br...

The Curious Clojurist - Neal Ford (Thoughtworks)

TDD at scale - Mash Badar (UBS)

Run Your Java Code on Cloud Foundry - Andy Piper (Pivotal)

Recently uploaded

As privacy and data protection regulations evolve rapidly, organizations operating in multiple jurisdictions face mounting challenges to ensure compliance and safeguard customer data. With state-specific privacy laws coming up in multiple states this year, it is essential to understand what their unique data protection regulations will require clearly. How will data privacy evolve in the US in 2024? How to stay compliant? Our panellists will guide you through the intricacies of these states' specific data privacy laws, clarifying complex legal frameworks and compliance requirements. This webinar will review: - The essential aspects of each state's privacy landscape and the latest updates - Common compliance challenges faced by organizations operating in multiple states and best practices to achieve regulatory adherence - Valuable insights into potential changes to existing regulations and prepare your organization for the evolving landscape

TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments

TrustArc

A Principled Technologies deployment guide Conclusion Deploying VMware Cloud Foundation 5.1 on next gen Dell PowerEdge servers brings together critical virtualization capabilities and high-performing hardware infrastructure. Relying on our hands-on experience, this deployment guide offers a comprehensive roadmap that can guide your organization through the seamless integration of advanced VMware cloud solutions with the performance and reliability of Dell PowerEdge servers. In addition to the deployment efficiency, the Cloud Foundation 5.1 and PowerEdge solution delivered strong performance while running a MySQL database workload. By leveraging VMware Cloud Foundation 5.1 and PowerEdge servers, you could help your organization embrace cloud computing with confidence, potentially unlocking a new level of agility, scalability, and efficiency in your data center operations.

Deploy with confidence: VMware Cloud Foundation 5.1 on next gen Dell PowerEdg...

Principled Technologies

Tata AIG General Insurance Company - Insurer Innovation Award 2024

The Digital Insurer

Automating Google Workspace (GWS) & more with Apps Script

wesley chun

Abhishek Deb(1), Mr Abdul Kalam(2) M. Des (UX) , School of Design, DIT University , Dehradun. This paper explores the future potential of AI-enabled smartphone processors, aiming to investigate the advancements, capabilities, and implications of integrating artificial intelligence (AI) into smartphone technology. The research study goals consist of evaluating the development of AI in mobile phone processors, analyzing the existing state as well as abilities of AI-enabled cpus determining future patterns as well as chances together with reviewing obstacles as well as factors to consider for more growth.

Exploring the Future Potential of AI-Enabled Smartphone Processors

debabhi2

Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving

Edi Saputra

The 7 Things I Know About Cyber Security After 25 Years | April 2024

Rafal Los

This presentations targets students or working professionals. You may know Google for search, YouTube, Android, Chrome, and Gmail, but did you know Google has many developer tools, platforms & APIs? This comprehensive yet still high-level overview outlines the most impactful tools for where to run your code, store & analyze your data. It will also inspire you as to what's possible. This talk is 50 minutes in length.

Powerful Google developer tools for immediate impact! (2023-24 C)

wesley chun

Scaling API-first – The story of a global engineering organization

Radu Cotescu

Partners Life - Insurer Innovation Award 2024

The Digital Insurer

Strategies for Landing an Oracle DBA Job as a Fresher

Remote DBA Services

Imagine a world where information flows as swiftly as thought itself, making decision-making as fluid as the data driving it. Every moment is critical, and the right tools can significantly boost your organization’s performance. The power of real-time data automation through FME can turn this vision into reality. Aimed at professionals eager to leverage real-time data for enhanced decision-making and efficiency, this webinar will cover the essentials of real-time data and its significance. We’ll explore: FME’s role in real-time event processing, from data intake and analysis to transformation and reporting An overview of leveraging streams vs. automations FME’s impact across various industries highlighted by real-life case studies Live demonstrations on setting up FME workflows for real-time data Practical advice on getting started, best practices, and tips for effective implementation Join us to enhance your skills in real-time data automation with FME, and take your operational capabilities to the next level.

From Event to Action: Accelerate Your Decision Making with Real-Time Automation

Safe Software

GenAI Risks & Security Meetup 01052024.pdf

lior mazor

Webinar Recording: https://www.panagenda.com/webinars/why-teams-call-analytics-is-critical-to-your-entire-business Nothing is as frustrating and noticeable as being in an important call and being unable to see or hear the other person. Not surprising then, that issues with Teams calls are among the most common problems users call their helpdesk for. Having in depth insight into everything relevant going on at the user’s device, local network, ISP and Microsoft itself during the call is crucial for good Microsoft Teams Call quality support. To ensure a quick and adequate solution and to ensure your users get the most out of their Microsoft 365. But did you know that ‘bad calls’ are also an excellent indicator of other problems arising? Precisely because it is so noticeable!? Like the canary in the mine, bad calls can be early indicators of problems. Problems that might otherwise not have been noticed for a while but can have a big impact on productivity and satisfaction. Join this session by Christoph Adler to learn how true Microsoft Teams call quality analytics helped other organizations troubleshoot bad calls and identify and fix problems that impacted Teams calls or the use of Microsoft365 in general. See what it can do to keep your users happy and productive! In this session we will cover - Why CQD data alone is not enough to troubleshoot call problems - The importance of attributing call problems to the right call participant - What call quality analytics can do to help you quickly find, fix-, and prevent problems - Why having retrospective detailed insights matters - Real life examples of how others have used Microsoft Teams call quality monitoring to problem shoot problems with their ISP, network, device health and more.

Why Teams call analytics are critical to your entire business

panagenda

ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke

Product Anonymous

Three things you will take away from the session: • How to run an effective tenant-to-tenant migration • Best practices for before, during, and after migration • Tips for using migration as a springboard to prepare for Copilot in Microsoft 365 Main ideas: Migration Overview: The presentation covers the current reality of cross-tenant migrations, the triggers, phases, best practices, and benefits of a successful tenant migration Considerations: When considering a migration, it is important to consider the migration scope, performance, customization, flexibility, user-friendly interface, automation, monitoring, support, training, scalability, data integrity, data security, cost, and licensing structure Next Wave: The next wave of change includes the launch of Copilot, which requires businesses to be prepared for upcoming changes related to Copilot and the cloud, and to consolidate data and tighten governance ShareGate: ShareGate can help with pre-migration analysis, configurable migration tool, and automated, end-user driven collaborative governance

Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff

sammart93

Created by Mozilla Research in 2012 and now part of Linux Foundation Europe, the Servo project is an experimental rendering engine written in Rust. It combines memory safety and concurrency to create an independent, modular, and embeddable rendering engine that adheres to web standards. Stewardship of Servo moved from Mozilla Research to the Linux Foundation in 2020, where its mission remains unchanged. After some slow years, in 2023 there has been renewed activity on the project, with a roadmap now focused on improving the engine’s CSS 2 conformance, exploring Android support, and making Servo a practical embeddable rendering engine. In this presentation, Rakhi Sharma reviews the status of the project, our recent developments in 2023, our collaboration with Tauri to make Servo an easy-to-use embeddable rendering engine, and our plans for the future to make Servo an alternative web rendering engine for the embedded devices industry. (c) Embedded Open Source Summit 2024 April 16-18, 2024 Seattle, Washington (US) https://events.linuxfoundation.org/embedded-open-source-summit/ https://ossna2024.sched.com/event/1aBNF/a-year-of-servo-reboot-where-are-we-now-rakhi-sharma-igalia

A Year of the Servo Reboot: Where Are We Now?

Igalia

presentation ICT roal in 21st century education

jfdjdjcjdnsjd

Effective data discovery is crucial for maintaining compliance and mitigating risks in today's rapidly evolving privacy landscape. However, traditional manual approaches often struggle to keep pace with the growing volume and complexity of data. Join us for an insightful webinar where industry leaders from TrustArc and Privya will share their expertise on leveraging AI-powered solutions to revolutionize data discovery. You'll learn how to: - Effortlessly maintain a comprehensive, up-to-date data inventory - Harness code scanning insights to gain complete visibility into data flows leveraging the advantages of code scanning over DB scanning - Simplify compliance by leveraging Privya's integration with TrustArc - Implement proven strategies to mitigate third-party risks Our panel of experts will discuss real-world case studies and share practical strategies for overcoming common data discovery challenges. They'll also explore the latest trends and innovations in AI-driven data management, and how these technologies can help organizations stay ahead of the curve in an ever-changing privacy landscape.

TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery

TrustArc

Discord is a free app offering voice, video, and text chat functionalities, primarily catering to the gaming community. It serves as a hub for users to create and join servers tailored to their interests. Discord’s ecosystem comprises servers, each functioning as a distinct online community with its own channels dedicated to specific topics or activities. Users can engage in text-based discussions, voice calls, or video chats within these channels. Understanding Discord Servers Discord servers are virtual spaces where users congregate to interact, share content, and build communities. Servers may revolve around gaming, hobbies, interests, or fandoms, providing a platform for like-minded individuals to connect. Communication Features Discord offers a range of communication tools, including text channels for messaging, voice channels for real-time audio conversations, and video channels for face-to-face interactions. These features facilitate seamless communication and collaboration. What Does NSFW Mean? The acronym NSFW stands for “Not Safe For Work,” indicating content that may be inappropriate for professional or public settings. NSFW Content NSFW content encompasses material that is sexually explicit, violent, or otherwise graphic in nature. It often includes nudity, profanity, or depictions of sensitive topics.

Understanding Discord NSFW Servers A Guide for Responsible Users.pdf

UK Journal

Recently uploaded (20)

TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments

Deploy with confidence: VMware Cloud Foundation 5.1 on next gen Dell PowerEdg...

Tata AIG General Insurance Company - Insurer Innovation Award 2024

Automating Google Workspace (GWS) & more with Apps Script

Exploring the Future Potential of AI-Enabled Smartphone Processors

Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving

The 7 Things I Know About Cyber Security After 25 Years | April 2024

Powerful Google developer tools for immediate impact! (2023-24 C)

Scaling API-first – The story of a global engineering organization

Partners Life - Insurer Innovation Award 2024

Strategies for Landing an Oracle DBA Job as a Fresher

From Event to Action: Accelerate Your Decision Making with Real-Time Automation

GenAI Risks & Security Meetup 01052024.pdf

Why Teams call analytics are critical to your entire business

ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke

Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff

A Year of the Servo Reboot: Where Are We Now?

presentation ICT roal in 21st century education

TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery

Understanding Discord NSFW Servers A Guide for Responsible Users.pdf

Streams and Things - Darach Ennis (Ubiquiti Networks)

1. Streams and Things JAX London 2013 - Darach Ennis - @darachennis

3. Internet of Things

4. Internet of Things (noun) - a proposed development of the Internet in which everyday objects have network connectivity, allowing them to send and receive data … – “Internet of Things” - added to Oxford English Dictionary - August 2013

5. Internet of Things (noun) - a proposed development of the Internet in which everyday people can easily network everyday objects, allowing them to send and receive data … – “Internet of Things” - a better deﬁnition - Darach Ennis, October 2013

6. Streams of Events

7. Streams of Events - receiving, processing and generating data ﬂows in near real-time using software algorithms to analyse the event data in ﬂight…

8. Things

10. #ﬁrstworldproblems

11.

12.

13.

14.

15. AR Drone 2.0 Protocols • A ﬂying wireless access point with • 9 Protocols: FTP, Auth, Video Recording, Navigation Data, Video, Command, Raw Capture, Printf, Control • Hackable: Linux based embedded system

16. Streams

17.

18. a.pipe(b).pipe(a)

19. by: @maxogden

20. Embedded Event Processing

21. Embedded Event Processing • Simple embeddable subset of CEP • Flow a la Node.js streams and pipes • Aggregate windows • Extensible, modular • Functional, reactive, easy to port

22. Embedded Event Processing • Windows / Aggregates • eep-js • eep-php • beam-js • eep-clj! • beam-erl • eep-java! • beam-java • eep-erl • Flow

23. Sources

24. Sinks

25. Operators

26. Pipelines A op B

27. Filter

28. Transform

29. Builtin Functions

30. Branch op A op op

31. Branch

32. Combine op op op B

33. Combine

34. filter! up filter! down In filter! keys fly! up fly! down … … filter! right fly! right land? do! land

35.

36. Streams and Things … putting it all together … demo, with you and robots!

37. People and Things - can we take the simplicity of an educational tool such as scratch and empower ordinary people to hack and mashup their things in ways ‘we the creators’ never imagined?! ! – “Internet of Things” - added to Oxford English Dictionary - August 2013 ! !

38. Thanks. Questions? ! @darachennis http://github.com/darach/eep-java! ! http://github.com/darach/beam-java

Streams and Things - Darach Ennis (Ubiquiti Networks)

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Streams and Things - Darach Ennis (Ubiquiti Networks)

Similar to Streams and Things - Darach Ennis (Ubiquiti Networks) (20)

More from jaxLondonConference

More from jaxLondonConference (20)

Recently uploaded

Recently uploaded (20)

Streams and Things - Darach Ennis (Ubiquiti Networks)