Using Postgres and Citus for Lightning Fast Analytics, also ft. Rollups | Live Demo | Sai Srirampur

•

0 likes•249 views

Watch Sai Srirampur, Solutions Engineer at Citus Data (now part of the Microsoft family), give a live demo of how you can use Postgres and the Citus extension to Postgres to manage real-time analytics workloads. View if you & your application need: >> A relational database that scales for customer-facing analytics dashboards, with real-time data ingest and a large volume of queries >> A way to scale out Postgres horizontally, to address the performance hiccups you’re experiencing as you run into the resource limits of single-node Postgres >> A way to roll-up and pre-aggregate data to build fast data pipelines and enable sub-second response times. >> A way to consolidate your database platforms, to avoid having separate stores for your transactional and analytics workloads Using a 4-node Citus database cluster in the cloud, Sai will show you how Citus shards Postgres to give you lightning fast performance, at scale. Also featuring rollups.

Technology

Live Demo of Using Postgres and Citus for Lightning Fast Analytics, also featuring Rollups | March 2019
WITH FROM
Thu 27 February at 10am PT
Using Postgres & Citus
for Lightning Fast
Analytics
PRESENTED BY:
SAI
SRIRAMPUR
LIVE DEMO

Sai Srirampur | Using Postgres and Citus for Lightning Fast Analytics, also featuring Rollups | March 2019
Ask private questions in
Q&A panel (mouseover at
bottom to see Q&A icon)
We plan to do 1-2 quick
polls.
Logistics
X

Sai Srirampur | PyConCA 2018
• Sai Srirampur a.k.a Sai
• Engineer at Citus Data
(now part of the Microsoft family!!)
• Joined Citus to make it so
developers never have to
worry about scaling their
database
• Recently got married!
• Follow me @saisrirampur
@citusdata

Live Demo of Using Postgres and Citus for Lightning Fast Analytics, also featuring Rollups | March 2019
Does
this
sound
like
you?
Hitting resource limits
with single-node
Postgres, but don’t want
to give up Postgres
1

Live Demo of Using Postgres and Citus for Lightning Fast Analytics, also featuring Rollups | March 2019
Want to reduce dev costs
by consolidating—&
avoid separate stores for
analytics (OLAP) &
transactional workloads
(OLTP)
Does
this
sound
like
you?
2

Live Demo of Using Postgres and Citus for Lightning Fast Analytics, also featuring Rollups | March 2019
Does
this
sound
like
you?
Pre-aggregation or rollup
pipeline lags and never
catches up
3

Live Demo of Using Postgres and Citus for Lightning Fast Analytics, also featuring Rollups | March 2019Live Demo of Using Postgres and Citus for Lightning Fast Analytics, also featuring Rollups | March 2019

Live Demo of Using Postgres and Citus for Lightning Fast Analytics, also featuring Rollups | March 201911

Live Demo of Using Postgres and Citus for Lightning Fast Analytics, also featuring Rollups | March 2019
Sai Srirampur | PyConCA 2018

Live Demo of Using Postgres and Citus for Lightning Fast Analytics, also featuring Rollups | March 2019
Sai Srirampur | PyConCA 2018
citusdata.com/customers/heap
citusdata.com/customers/heap

Live Demo of Using Postgres and Citus for Lightning Fast Analytics, also featuring Rollups | March 201922
citusdata.com/customers/algolia

Scaling Postgres for Time Series Data with Citus | Nov 15 2018 | Marco Slot | Claire Giordano
Using Postgres and Citus for Lightning Fast Analytics | February 2019
Min Wei of
Microsoft

Live Demo of Using Postgres and Citus for Lightning Fast Analytics, also featuring Rollups | March 2019

Live Demo of Using Postgres and Citus for Lightning Fast Analytics, also featuring Rollups | March 201926
Any Questions?? (And, resources)
DOWNLOAD CITUS OPEN SOURCE:
https://www.citusdata.com/download
CREATE CITUS CLOUD DEV ACCOUNT:
https://www.citusdata.com/product/cloud
JOIN OUR PUBLIC SLACK:
https://slack.citusdata.com/

Live Demo of Citus Cloud | July 2018
www.citusdata.com @citusdata
© 2019 Citus Data. All rights reserved.
Sai Srirampur | @saisrirampur
Thank you for your time

What's hot

LinkedIn's is the most advantageous social networking tool available to job seekers and business professionals today, with 610+ million members creating millions of posts, videos, and articles that generate tens of millions of shares, comments, and likes per day. LinkedIn has leveraged this activity data to build rich interactive user-facing analytics applications like “Who Viewed My Profile”, Talent Insights, Ad Analytics, and Publisher Analytics, among others. These applications are all powered by Pinot, as are internal dashboards, anomaly detection and root cause analysis platform like ThirdEye. This talk will present how Pinot has become the de-facto solution for serving analytic queries in milliseconds, ad-hoc reporting, monitoring & Anomaly Detection on multidimensional data.

Building real time analytics applications using pinot : A LinkedIn case study

Kishore Gopalakrishna

At Avast, we believe everyone has the right to be safe. We are dedicated to creating a world that provides safety and privacy for all, not matter where you are, who you are, or how you connect. With over 1.5 billion attacks stopped and 30 million new executable files monthly, big data pipelines are crucial for the security of our customers. At Avast we are leveraging Apache Spark machine learning libraries and TensorflowOnSpark for a variety of tasks ranging from marketing and advertisement, through network security to malware detection. This talk will cover our main cybersecurity usecases of Spark. After describing our cluster environment we will first demonstrate anomaly detection on time series of threats. Having thousands of types of attacks and malware, AI helps human analysts select and focus on most urgent or dire threats. We will walk through our setup for distributed training of deep neural networks with Tensorflow to deploying and monitoring of a streaming anomaly detection application with trained model. Next we will show how we use Spark for analysis and clustering of malicious files and large scale experimentation to automatically process and handle changes in malware. In the end, we will give comparison to other tools we used for solving those problems.

AI on Spark for Malware Analysis and Anomalous Threat Detection

Databricks

Unified Framework for Real Time, Near Real Time and Offline Analysis of Video...

Spark Summit

Spark and the Future of Advanced Analytics by Thomas Dinsmore

Spark Summit

Kubernetes is an open-source containerization framework that makes it easy to manage applications in isolated environments at scale. In Apache Spark 2.3, Spark introduced support for native integration with Kubernetes. Palantir has been deeply involved with the development of Spark’s Kubernetes integration from the beginning, and our largest production deployment now runs an average of ~5 million Spark pods per day, as part of tens of thousands of Spark applications. Over the course of our adventures in migrating deployments from YARN to Kubernetes, we have overcome a number of performance, cost, & reliability hurdles: differences in shuffle performance due to smaller filesystem caches in containers; Kubernetes CPU limits causing inadvertent throttling of containers that run many Java threads; and lack of support for dynamic allocation leading to resource wastage. We intend to briefly describe our story of developing & deploying Spark-on-Kubernetes, as well as lessons learned from deploying containerized Spark applications in production. We will also describe our recently open-sourced extension (https://github.com/palantir/k8s-spark-scheduler) to the Kubernetes scheduler to better support Spark workloads & facilitate Spark-aware cluster autoscaling; our limited implementation of dynamic allocation on Kubernetes; and ongoing work that is required to support dynamic resource management & stable performance at scale (i.e., our work with the community on a pluggable external shuffle service API). Our hope is that our lessons learned and ongoing work will help other community members who want to use Spark on Kubernetes for their own workloads.

Reliable Performance at Scale with Apache Spark on Kubernetes

Databricks

Spark SQL is one of the most popular components in big data warehouse for SQL queries in batch mode, and it allows user to process data from various data sources in a highly efficient way. However, Spark SQL is a general purpose SQL engine and not well designed for ad hoc queries. Intel invented an Apache Spark data source plugin called Spinach for fulfilling such requirements, by leveraging user-customized indices and fine-grained data cache mechanisms. To be more specific, Spinach defines a new Parquet-like data storage format, offering a fine-grained hierarchical cache mechanism in the unit of “Fiber” in memory. Even existing Parquet or ORC data files can be loaded using corresponding adaptors. Data can be cached in off-heap memory to boost data loading. What’s more, Spinach has extended the Spark SQL DDL, to allow users to define the customized indices based on relation. Currently, B+ tree and bloom filter are the first two types of indices supported. Last but not least, since Spinach resides in the process of Spark executor, there’s no extra effort in deployment. All you need to do is to pick Spinach from Spark packages when launching the Spark SQL. sing corresponding adaptors. Data can be cached in off-heap memory to boost data loading. What’s more, Spinach has extended the Spark SQL DDL, to allow user to define the customized indices based on relation. Currently B+ tree and bloom filter are the first 2 types of index we’ve supported. Last but not least, since Spinach resides in the process of Spark executor, there’s no extra effort in deployment, all we need to do is to pick Spinach from Spark packages when launch the Spark SQL. Spinach has been imported in Baidu’s production environment since Q4 2016. It helps several teams migrating their regular data analysis tasks from Hive or MR jobs to ad-hoc queries. In Baidu search ads system FengChao, data engineers analyze advertising effectiveness based on several TBs data of display and click logs every day. Spinach brings a 5x boost compared to original Spark SQL (version 2.1), especially in the scenario of complex search and large data volume. It optimizes the average search cost from minutes to seconds, while brings only 3% data size increase for adding a single index.

OAP: Optimized Analytics Package for Spark Platform with Daoyuan Wang and Yua...

Databricks

Slides from the Big Data Gurus meetup at Samsung R&D, August 14, 2013 This presentation covers the high level architecture of the Netflix Data Platform with a deep dive into the architecture, implementation, use cases, and future of Lipstick (https://github.com/Netflix/Lipstick) - our open source tool for graphically analyzing and monitoring the execution of Apache Pig scripts. Netflix uses Apache Pig to express many complex data manipulation and analytics workflows. While Pig provides a great level of abstraction between MapReduce and data flow logic, once scripts reach a sufficient level of complexity, it becomes very difficult to understand how data is being transformed and manipulated across MapReduce jobs. To address this problem, we created (and open sourced) a tool named Lipstick that visualizes and monitors the progress and performance of Pig scripts.

Putting Lipstick on Apache Pig at Netflix

Jeff Magnusson

This session will explain how NetApp simplifies the process of analyzing IoT data, using Apache Spark clusters across data centers and the cloud using NetApp Private Storage (NPS) for AWS/Azure, NetApp Data Fabric and NetApp Connectors for NFS and S3. IoT data originates at the edge in different geographical locations, and it can arrive at different data centers or the cloud depending on sensor location. The challenge is how to combine these different data streams across different datacenters to generate wider insights. Learn how NetApp Data Fabric helps solve this challenge. In the Data Fabric architecture, the IoT data is ingested via Kafka into an Apache Spark cluster running in AWS/Azure, but the data is stored in NPS provisioned NFS share through NFS Connector. The IoT data in NPS can then be moved to on-prem datacenters, or on-prem IoT data can be moved to NPS or ONTAP Cloud for processing in AWS/Azure using NetApp SnapMirror Flex Clone or NFS Connector. We’ll also review how NetApp StorageGRID object storage maintains IoT data for archival purposes using S3 Target. The above options allow you to analyze IoT data from AWS, StorageGRID, HDFS or NFS, providing a feasible solution for deploying Spark clusters across datacenters. Takeaways will include identifying Spark challenges that can be remedied by extending your Spark environment to take advantage of NPS; understanding how NPS and StorageGRID can provide a cost-effective alternative for dev/test, DR for Spark analytics; and understanding Spark architecture and deployment options that utilize data from multiple locations, including on-prem and cloud-based repositories.

Analyzing IOT Data in Apache Spark Across Data Centers and Cloud with NetApp ...

Databricks

Zalandos AI-driven products and distributed landscape of analytical data marts cannot wait for long-running, hard-to-recover, monolithic batch jobs taking all night to calculate already outdated data. Modern data integration pipelines need to deliver fast and easy to consume data sets in high quality. Based on Spark Streaming and Delta, the central data warehousing team was able to deliver widely-used master data as S3 or Kafka streams and snapshots at the same time. The talk will cover challenges in our fashion data platform and a detailed architectural deep dive about separation of integration from enrichment, providing streams as well as snapshots and feeding the data to distributed data marts. Finally, lessons learned and best practices about Delta’s MERGE command, Scala API vs Spark SQL and schema evolution give more insights and guidance for similar use cases.

Data Warehousing with Spark Streaming at Zalando

Databricks

Kylin and Druid Presentation

argonauts007

The Spark Listener interface provides a fast, simple and efficient route to monitoring and observing your Spark application - and you can start using it in minutes. In this talk, we'll introduce the Spark Listener interfaces available in core and streaming applications, and show a few ways in which they've changed our world for the better at SpotX. If you're looking for a "Eureka!" moment in monitoring or tracking of your Spark apps, look no further than Spark Listeners and this talk!

Apache Spark Listeners: A Crash Course in Fast, Easy Monitoring

Databricks

While the performance delivered by Spark has enabled data scientists to undertake sophisticated analyses on big and complex data in actionable timeframes, too often, the process of manually configuring the underlying Spark jobs (including the number and size of the executors) can be a significant and time consuming undertaking. Not only it does this configuration process typically rely heavily on repeated trial-and-error, it necessitates that data scientists have a low-level understanding of Spark and detailed cluster sizing information. At Alpine Data we have been working to eliminate this requirement, and develop algorithms that can be used to automatically tune Spark jobs with minimal user involvement, In this presentation, we discuss the algorithms we have developed and illustrate how they leverage information about the size of the data being analyzed, the analytical operations being used in the flow, the cluster size, configuration and real-time utilization, to automatically determine the optimal Spark job configuration for peak performance.

Spark Autotuning: Spark Summit East talk by Lawrence Spracklen

Spark Summit

Debugging big data analytics in Data-Intensive Scalable Computing (DISC) systems is a time-consuming effort. Today’s DISC systems offer very little tooling for debugging and, as a result, programmers spend countless hours analyzing log files and performing trial and error debugging. To aid this effort, UCLA developed BigDebug, an interactive debugging tool and automated fault localization service to help Apache Spark developers in debugging big data analytics. To emulate interactive step-wise debugging without reducing throughput, BigDebug provides simulated breakpoints that enable a user to inspect a program without actually pausing the entire distributed computation. It also supports on-demand watchpoints that enable a user to retrieve intermediate data using a guard predicate and transfer the selected data on demand. To understand the flow of individual records within a pipeline of RDD transformations, BigDebug provides data provenance capability, which can help understand how errors propagate through data processing steps. To support efficient trial-and-error debugging, BigDebug enables users to change program logic in response to an error at runtime through a realtime code fix feature, and selectively replay the execution from that step. Finally, BigDebug proposes an automated fault localization service that leverages all the above features together to isolate failure-inducing inputs, diagnose the root cause of an error, and resume the workflow for only affected data and code. The BigDebug system should contribute to improving Spark developerproductivity and the correctness of their Big Data applications. This big data debugging effort is led by UCLA Professors Miryung Kim and Tyson Condie, and produced several research papers in top Software Engineering and Database conferences. The current version of BigDebug is publicly available at https://sites.google.com/site/sparkbigdebug/.

Debugging Big Data Analytics in Apache Spark with BigDebug with Muhammad Gulz...

Databricks

The talk will present a MPI-based extension of the Spark platform developed in the context of light source facilities. The background and rationale of this extension are described in the attached paper “Bringing the HPC reconstruction algorithms to Big Data platforms”[1], which has been presented at New York Scientific Data Summit (NYSDS), August 14-17, 2016 (talk: https://www.bnl.gov/nysds16/files/pdf/talks/NYSDS16%20Malitsky.pdf) Specifically, the paper highlighted a gap between two modern driving forces of the scientific discovery process: HPC and Big Data technologies. As a result, it proposed to extend the Spark platform with inter-worker communication for supporting scientific-oriented parallel applications. The approach was illustrated in the context of the Spark-based deployment of the SHARP MPI/GPU ptychographic solver. Aside from its practical value, this application represents a reference use case that captures the major technical aspects of other reconstruction tasks. In the NYSDS’16 paper, the implemented approach followed the CaffeOnSpark RDMA peer-to-peer model and augmented it with the RDMA address exchange server. By the Spark Summit, we plan to further advance this direction with the Spark-MPI generic solution based on the Hydra process management framework for supporting two major MPI implementations, MPICH and MVAPICH.

Bringing HPC Algorithms to Big Data Platforms: Spark Summit East talk by Niko...

Spark Summit

GE Aviation has hundreds of data scientists and engineers developing algorithms. The majority of these people do not have the time to learn Apache Spark and continue to develop on local machines in Python or R. We also have lots of historical code that was not developed for Spark. However, the business wanted to deploy to a Spark environment for scalability, as quickly as possible. So how did we bridge the gap? A data scientist and software engineer will co-present to share how we approached the problem of building, unifying and scaling these algorithms.

Bridging the Gap Between Data Scientists and Software Engineers – Deploying L...

Databricks

How to use Parquet as a Sasis for ETL and Analytics

DataWorks Summit

Realtime Data Analysis Patterns

Mikio L. Braun

In the big data world, it's not always easy for Python users to move huge amounts of data around. Apache Arrow defines a common format for data interchange, while Arrow Flight introduced in version 0.11.0, provides a means to move that data efficiently between systems. Arrow Flight is a framework for Arrow-based messaging built with gRPC. It enables data microservices where clients can produce and consume streams of Arrow data to share it over the wire. In this session, I'll give a brief overview of Arrow Flight from a Python perspective, and show that it's easy to build high performance connections when systems can talk Arrow. I'll also cover some ongoing work in using Arrow Flight to connect PySpark with TensorFlow - two systems with great Python APIs but very different underlying internal data.

Make your PySpark Data Fly with Arrow!

Databricks

Today, users have multiple options for big data analytics in terms of open-source and proprietary systems as well as in cloud computing service providers. In order to obtain the best value for their money in a SaaS cloud environment, users need to be aware of the performance of each service as well as its associated costs, while also taking into account aspects such as usability in conjunction with monitoring, interoperability, and administration capabilities. We present an independent analysis of two mature and well-known data analytics systems, Apache Spark and Presto. Both running on the Amazon EMR platform, but in the case of Apache Spark, we also analyze the Databricks Unified Analytics Platform and its associated runtime and optimization capabilities. Our analysis is based on running the TPC-DS benchmark and thus focuses on SQL performance, which still is indispensable for data scientists and engineers. In our talk we will present quantitative results that we expect to be valuable for end users, accompanied by an in depth look into the advantages and disadvantages of each alternative. Thus, attendees will be better informed of the current big data analytics landscape and find themselves in a better position to avoid common pitfalls in deploying data analytics at a scale.

Performance Analysis of Apache Spark and Presto in Cloud Environments

Databricks

Almost all organizations now have a need for data science and, as such, the main challenge after determining the algorithm is to scale it up and make it operational. Comcast uses several tools and technologies such as Python, R, SaS, H2O and so on. In this session, they’ll show how many common use cases use the common algorithms like Logistic Regression, Random Forest, Decision Trees, Clustering, NLP, etc. Apache Spark has several machine learning algorithms built in and has excellent scalability. Hence, at Comcast, they built a platform to provide DSaaS on top of Spark with REST API as a means of controlling and submitting jobs, so as to abstract most users from the rigor of writing (repeating) code, instead focusing on the actual requirements. Learn how they solved some of the problems of establishing feature vectors, choosing algorithms and then deploying models into production. They’ll also showcase their use of Scala, R and Python to implement models using language of choice yet deploying quickly into production on 500-node Spark clusters.

Using SparkML to Power a DSaaS (Data Science as a Service) with Kiran Muglurm...

Databricks

What's hot (20)

Building real time analytics applications using pinot : A LinkedIn case study

AI on Spark for Malware Analysis and Anomalous Threat Detection

Unified Framework for Real Time, Near Real Time and Offline Analysis of Video...

Spark and the Future of Advanced Analytics by Thomas Dinsmore

Reliable Performance at Scale with Apache Spark on Kubernetes

OAP: Optimized Analytics Package for Spark Platform with Daoyuan Wang and Yua...

Putting Lipstick on Apache Pig at Netflix

Analyzing IOT Data in Apache Spark Across Data Centers and Cloud with NetApp ...

Data Warehousing with Spark Streaming at Zalando

Kylin and Druid Presentation

Apache Spark Listeners: A Crash Course in Fast, Easy Monitoring

Spark Autotuning: Spark Summit East talk by Lawrence Spracklen

Debugging Big Data Analytics in Apache Spark with BigDebug with Muhammad Gulz...

Bringing HPC Algorithms to Big Data Platforms: Spark Summit East talk by Niko...

Bridging the Gap Between Data Scientists and Software Engineers – Deploying L...

How to use Parquet as a Sasis for ETL and Analytics

Realtime Data Analysis Patterns

Make your PySpark Data Fly with Arrow!

Performance Analysis of Apache Spark and Presto in Cloud Environments

Using SparkML to Power a DSaaS (Data Science as a Service) with Kiran Muglurm...

The Art of PostgreSQL | PostgreSQL Ukraine Meetup | Dimitri Fontaine

Citus Data

How to write SQL queries | pgDay Paris 2019 | Dimitri Fontaine

Citus Data

When it all Goes Wrong |Nordic PGDay 2019 | Will Leinweber

Citus Data

I spent the early part of my career working on developer tools, operating systems, high-speed file systems, and scale-out storage. Not databases. Frankly, I always thought that databases were a bit boring. So one year in to my new job at a Postgres company, I continue to be amazed at the enthusiasm of the PostgreSQL developer community and users. I mean, people’s eyes light up when you ask them why they love Postgres. Sure, a lot of us get animated when talking about our newest iPhone, or Ronaldo’s phenomenal free-kick goal in the World Cup, or mint chip gelato from La Strega Nociola—but most platform software simply doesn’t trigger this kind of passion. So why does Postgres? Why is this open source database having such a “moment”? Why now? Well, I’ve been trying to find out, looking at this “Postgres moment” from a few different angles. In this talk I’ll share what I’ve observed to be the top 10 business, technology, and community reasons so many of you have so much affection for PostgreSQL.

Why PostgreSQL Why This Database Why Now | Nordic PGDay 2019 | Claire Giordano

Citus Data

There are a number of data architectures you could use when building a multi-tenant app. Some, such as using one database per customer or one schema per customer. These two options scale to an extent when you have say 10s of tenants. However as you start scaling to hundreds and thousands of tenants, you start running into challenges both from performance and maintenance of tenants perspective. You could solve the above problem by adding the notion of tenancy directly into the logic of your SaaS application. How to implement/automate this in Django-ORM is a challenge? We will talk about how to make the django app tenant aware and at a broader level explain how scale out applications that are built on top of Django ORM and follow a multi tenant data model. We'd take postgresql as our database of choice and the logic/implementation can be extended to any other relational databases as well.

Scaling Multi-Tenant Applications Using the Django ORM & Postgres | PyCaribbe...

Citus Data

As a developer using PostgreSQL one of the most important tasks you have to deal with is modeling the database schema for your application. In order to achieve a solid design, it’s important to understand how the schema is then going to be used as well as the trade-offs it involves. As Fred Brooks said: “Show me your flowcharts and conceal your tables, and I shall continue to be mystified. Show me your tables, and I won’t usually need your flowcharts; they’ll be obvious.” In this talk we're going to see practical normalisation examples and their benefits, and also review some anti-patterns and their typical PostgreSQL solutions, including Denormalization techniques thanks to advanced Data Types. As a developer using PostgreSQL one of the most important tasks you have to deal with is modeling the database schema for your application. In order to achieve a solid design, it’s important to understand how the schema is then going to be used as well as the trade-offs it involves. As Fred Brooks said: “Show me your flowcharts and conceal your tables, and I shall continue to be mystified. Show me your tables, and I won’t usually need your flowcharts; they’ll be obvious.” In this talk we're going to see practical normalisation examples and their benefits, and also review some anti-patterns and their typical PostgreSQL solutions, including Denormalization techniques thanks to advanced Data Types.

Data Modeling, Normalization, and Denormalisation | FOSDEM '19 | Dimitri Font...

Citus Data

Whether you’re working with a single node database, a distributed system, or an MPP database, a key factor in the flexibility you get with the system is how you shard or partition your data. Do you do it by customer, time, or some random uuid? Here we’ll walk through five different approaches to sharding your data and when you should consider each. If you’re thinking you need to scale beyond a single node this will give you the start of your roadmap for doing so. We’ll cover the basics of how you can do this directly in Postgres as well as principles that apply generically to any database.

Five data models for sharding and which is right | PGConf.ASIA 2018 | Craig K...

Citus Data

More from Citus Data (20)

Data Modeling, Normalization, and De-Normalization | PostgresOpen 2019 | Dimi...

JSONB Tricks: Operators, Indexes, and When (Not) to Use It | PostgresOpen 201...

Tutorial: Implementing your first Postgres extension | PGConf EU 2019 | Burak...

Whats wrong with postgres | PGConf EU 2019 | Craig Kerstiens

When it all goes wrong | PGConf EU 2019 | Will Leinweber

Amazing SQL your ORM can (or can't) do | PGConf EU 2019 | Louise Grandjonc

What Microsoft is doing with Postgres & the Citus Data acquisition | PGConf E...

Deep Postgres Extensions in Rust | PGCon 2019 | Jeff Davis

Why Postgres Why This Database Why Now | SF Bay Area Postgres Meetup | Claire...

A story on Postgres index types | PostgresLondon 2019 | Louise Grandjonc

Why developers need marketing now more than ever | GlueCon 2019 | Claire Gior...

The Art of PostgreSQL | PostgreSQL Ukraine | Dimitri Fontaine

When it all goes wrong (with Postgres) | RailsConf 2019 | Will Leinweber

The Art of PostgreSQL | PostgreSQL Ukraine Meetup | Dimitri Fontaine

How to write SQL queries | pgDay Paris 2019 | Dimitri Fontaine

When it all Goes Wrong |Nordic PGDay 2019 | Will Leinweber

Why PostgreSQL Why This Database Why Now | Nordic PGDay 2019 | Claire Giordano

Scaling Multi-Tenant Applications Using the Django ORM & Postgres | PyCaribbe...

Data Modeling, Normalization, and Denormalisation | FOSDEM '19 | Dimitri Font...

Five data models for sharding and which is right | PGConf.ASIA 2018 | Craig K...

Recently uploaded

In this session, we will delve into strategic approaches for optimizing knowledge management within Microsoft 365, amidst the evolving landscape of Copilot. From leveraging automatic metadata classification and permission governance with SharePoint Premium, to unlocking Viva Engage for the cultivation of knowledge and communities, you will gain actionable insights to bolster your organization's knowledge-sharing initiatives. In this session, we will also explore how to facilitate solutions to enable your employees to find answers and expertise within Microsoft 365. You will leave equipped with practical techniques and a deeper understanding of how there is more to effective knowledge management than just enabling Copilot, but building actual solutions to prepare the knowledge that Copilot and your employees can use.

Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...

Drew Madelung

Created by Mozilla Research in 2012 and now part of Linux Foundation Europe, the Servo project is an experimental rendering engine written in Rust. It combines memory safety and concurrency to create an independent, modular, and embeddable rendering engine that adheres to web standards. Stewardship of Servo moved from Mozilla Research to the Linux Foundation in 2020, where its mission remains unchanged. After some slow years, in 2023 there has been renewed activity on the project, with a roadmap now focused on improving the engine’s CSS 2 conformance, exploring Android support, and making Servo a practical embeddable rendering engine. In this presentation, Rakhi Sharma reviews the status of the project, our recent developments in 2023, our collaboration with Tauri to make Servo an easy-to-use embeddable rendering engine, and our plans for the future to make Servo an alternative web rendering engine for the embedded devices industry. (c) Embedded Open Source Summit 2024 April 16-18, 2024 Seattle, Washington (US) https://events.linuxfoundation.org/embedded-open-source-summit/ https://ossna2024.sched.com/event/1aBNF/a-year-of-servo-reboot-where-are-we-now-rakhi-sharma-igalia

A Year of the Servo Reboot: Where Are We Now?

Igalia

🐬 The future of MySQL is Postgres 🐘

RTylerCroy

Enterprise Knowledge’s Urmi Majumder, Principal Data Architecture Consultant, and Fernando Aguilar Islas, Senior Data Science Consultant, presented "Driving Behavioral Change for Information Management through Data-Driven Green Strategy" on March 27, 2024 at Enterprise Data World (EDW) in Orlando, Florida. In this presentation, Urmi and Fernando discussed a case study describing how the information management division in a large supply chain organization drove user behavior change through awareness of the carbon footprint of their duplicated and near-duplicated content, identified via advanced data analytics. Check out their presentation to gain valuable perspectives on utilizing data-driven strategies to influence positive behavioral shifts and support sustainability initiatives within your organization. In this session, participants gained answers to the following questions: - What is a Green Information Management (IM) Strategy, and why should you have one? - How can Artificial Intelligence (AI) and Machine Learning (ML) support your Green IM Strategy through content deduplication? - How can an organization use insights into their data to influence employee behavior for IM? - How can you reap additional benefits from content reduction that go beyond Green IM?

Driving Behavioral Change for Information Management through Data-Driven Gree...

Enterprise Knowledge

Boost Fertility New Invention Ups Success Rates.pdf

sudhanshuwaghmare1

Histor y of HAM Radio presentation slide

vu2urc

A Call to Action for Generative AI in 2024

Results

Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024

The Digital Insurer

How to convert PDF to text with Nanonets

naman860154

BooK Now Call us at +918448380779 to hire a gorgeous and seductive call girl for sex. Take a Delhi Escort Service. The help of our escort agency is mostly meant for men who want sexual Indian Escorts In Delhi NCR. It should be noted that any impersonator will get 100 attention from our Young Girls Escorts in Delhi. They will assume the position of reliable allies. VIP Call Girl With Original Photos Book Tonight +918448380779 Our Cheap Price 1 Hour not available 2 Hours 5000 Full Night 8000 TAG: Call Girls in Delhi, Noida, Gurgaon, Ghaziabad, Connaught Place, Greater Kailash Delhi, Lajpat Nagar Delhi, Mayur Vihar Delhi, Chanakyapuri Delhi, New Friends Colony Delhi, Majnu Ka Tilla, Karol Bagh, Malviya Nagar, Saket, Khan Market, Noida Sector 18, Noida Sector 76, Noida Sector 51, Gurgaon Mg Road, Iffco Chowk Gurgaon, Rajiv Chowk Gurgaon All Delhi Ncr Free Home Deliver

08448380779 Call Girls In Greater Kailash - I Women Seeking Men

Delhi Call girls

Heather Hedden, Senior Consultant at Enterprise Knowledge, presented “The Role of Taxonomy and Ontology in Semantic Layers” at a webinar hosted by Progress Semaphore on April 16, 2024. Taxonomies at their core enable effective tagging and retrieval of content, and combined with ontologies they extend to the management and understanding of related data. There are even greater benefits of taxonomies and ontologies to enhance your enterprise information architecture when applying them to a semantic layer. A survey by DBP-Institute found that enterprises using a semantic layer see their business outcomes improve by four times, while reducing their data and analytics costs. Extending taxonomies to a semantic layer can be a game-changing solution, allowing you to connect information silos, alleviate knowledge gaps, and derive new insights. Hedden, who specializes in taxonomy design and implementation, presented how the value of taxonomies shouldn’t reside in silos but be integrated with ontologies into a semantic layer. Learn about: - The essence and purpose of taxonomies and ontologies in information and knowledge management; - Advantages of semantic layers leveraging organizational taxonomies; and - Components and approaches to creating a semantic layer, including the integration of taxonomies and ontologies

The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf

Enterprise Knowledge

The Raspberry Pi 5 was announced on October 2023. This new version of the popular embedded device comes with a new iteration of Broadcom’s VideoCore GPU platform, and was released with a fully open source driver stack, developed by Igalia. The presentation will discuss some of the major changes required to support this new Video Core iteration, the challenges we faced in the process and the solutions we provided in order to deliver conformant OpenGL ES and Vulkan drivers. The talk will also cover the next steps for the open source Raspberry Pi 5 graphics stack. (c) Embedded Open Source Summit 2024 April 16-18, 2024 Seattle, Washington (US) https://events.linuxfoundation.org/embedded-open-source-summit/ https://eoss24.sched.com/event/1aBEx

Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...

Igalia

Real Time Object Detection Using Open CV

Khem

GenCyber Cyber Security Day Presentation

Michael W. Hawkins

Handwritten Text Recognition for manuscripts and early printed texts

Maria Levchenko

What are drone anti-jamming systems? The drone anti-jamming systems and anti-spoof technology protect against interference, jamming, and spoofing of the UAVs. To protect their security, countries are beginning to research drone anti-jamming systems, also known as drone strike weapons. The anti-jam and anti-spoof technology protects against interference, jamming and spoofing. A drone strike weapon is a drone attack weapon that can attack and destroy enemy drones. So what is so unique about this amazing system?

What Are The Drone Anti-jamming Systems Technology?

Antenna Manufacturer Coco

Microsoft's Threat Matrix for Kubernetes helps organizations understand the attack surface a Kubernetes deployment introduces to their environments. This ensures that adequate detections and mitigations are in place. By covering over 40 different attacker techniques, defenders can learn about Kubernetes-specific mitigations and controls to deploy to their environments. In this session, we will explore the MS-TA9013 Host Path Mount technique, which is commonly used by attackers to perform privilege escalation in a Kubernetes cluster. Attendees will learn how attackers and defenders can: * Escape the container's host volume mount to gain persistence on an underlying node * Move laterally from the underlying node into the customer's cloud environment * Analyze Kubernetes audit logs to detect pods deployed with a hostPath mount * Deploy an admission controller that prevents new pods from using a hostPath mount

Breaking the Kubernetes Kill Chain: Host Path Mount

Puma Security, LLC

[2024]Digital Global Overview Report 2024 Meltwater.pdf

hans926745

Abhishek Deb(1), Mr Abdul Kalam(2) M. Des (UX) , School of Design, DIT University , Dehradun. This paper explores the future potential of AI-enabled smartphone processors, aiming to investigate the advancements, capabilities, and implications of integrating artificial intelligence (AI) into smartphone technology. The research study goals consist of evaluating the development of AI in mobile phone processors, analyzing the existing state as well as abilities of AI-enabled cpus determining future patterns as well as chances together with reviewing obstacles as well as factors to consider for more growth.

Exploring the Future Potential of AI-Enabled Smartphone Processors

debabhi2

Scaling API-first – The story of a global engineering organization

Radu Cotescu

Recently uploaded (20)

Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...

A Year of the Servo Reboot: Where Are We Now?

🐬 The future of MySQL is Postgres 🐘

Driving Behavioral Change for Information Management through Data-Driven Gree...

Boost Fertility New Invention Ups Success Rates.pdf

Histor y of HAM Radio presentation slide

A Call to Action for Generative AI in 2024

Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024

How to convert PDF to text with Nanonets

08448380779 Call Girls In Greater Kailash - I Women Seeking Men

The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf

Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...

Real Time Object Detection Using Open CV

GenCyber Cyber Security Day Presentation

Handwritten Text Recognition for manuscripts and early printed texts

What Are The Drone Anti-jamming Systems Technology?

Breaking the Kubernetes Kill Chain: Host Path Mount

[2024]Digital Global Overview Report 2024 Meltwater.pdf

Exploring the Future Potential of AI-Enabled Smartphone Processors

Scaling API-first – The story of a global engineering organization

Using Postgres and Citus for Lightning Fast Analytics, also ft. Rollups | Live Demo | Sai Srirampur

1. Live Demo of Using Postgres and Citus for Lightning Fast Analytics, also featuring Rollups | March 2019 WITH FROM Thu 27 February at 10am PT Using Postgres & Citus for Lightning Fast Analytics PRESENTED BY: SAI SRIRAMPUR LIVE DEMO

2. Sai Srirampur | Using Postgres and Citus for Lightning Fast Analytics, also featuring Rollups | March 2019 Ask private questions in Q&A panel (mouseover at bottom to see Q&A icon) We plan to do 1-2 quick polls. Logistics X

3. Sai Srirampur | PyConCA 2018 • Sai Srirampur a.k.a Sai • Engineer at Citus Data (now part of the Microsoft family!!) • Joined Citus to make it so developers never have to worry about scaling their database • Recently got married! • Follow me @saisrirampur @citusdata

4. Live Demo of Using Postgres and Citus for Lightning Fast Analytics, also featuring Rollups | March 2019 Does this sound like you? Hitting resource limits with single-node Postgres, but don’t want to give up Postgres 1

5. Live Demo of Using Postgres and Citus for Lightning Fast Analytics, also featuring Rollups | March 2019 Want to reduce dev costs by consolidating—& avoid separate stores for analytics (OLAP) & transactional workloads (OLTP) Does this sound like you? 2

6. Live Demo of Using Postgres and Citus for Lightning Fast Analytics, also featuring Rollups | March 2019 Does this sound like you? Pre-aggregation or rollup pipeline lags and never catches up 3

7. Live Demo of Using Postgres and Citus for Lightning Fast Analytics, also featuring Rollups | March 2019Live Demo of Using Postgres and Citus for Lightning Fast Analytics, also featuring Rollups | March 2019

8. Live Demo of Using Postgres and Citus for Lightning Fast Analytics, also featuring Rollups | March 2019Live Demo of Using Postgres and Citus for Lightning Fast Analytics, also featuring Rollups | March 2019 Why ! Postgres? TLDR; Open source Constraints Extensions PostGIS / Geospatial HLL, TopN, Citus Foreign data wrappers Rich SQL CTEs Window functions Full text search Datatypes JSONB

9. Live Demo of Using Postgres and Citus for Lightning Fast Analytics, also featuring Rollups | March 2019Live Demo of Using Postgres and Citus for Lightning Fast Analytics, also featuring Rollups | March 2019 Q: Why Citus? A: Citus transforms Postgres into a distributed database

10. Live Demo of Using Postgres and Citus for Lightning Fast Analytics, also featuring Rollups | March 2019Live Demo of Using Postgres and Citus for Lightning Fast Analytics, also featuring Rollups | March 2019 TLDR; on scaling out Postgres w/Citus 1. Distributes data across multiple nodes 2. More memory, cpu, disk + parallelization 3. Extension to Postgres (not a fork) DATABASE AS A SERVICE ENTERPRISE SOFTWARE OPEN SOURCE

11. Live Demo of Using Postgres and Citus for Lightning Fast Analytics, also featuring Rollups | March 201911

12. Live Demo of Using Postgres and Citus for Lightning Fast Analytics, also featuring Rollups | March 2019 Live Demo of Using Postgres and Citus for Lightning Fast Analytics, also featuring Rollups | March 2019

13. Live Demo of Using Postgres and Citus for Lightning Fast Analytics, also featuring Rollups | March 2019Live Demo of Using Postgres and Citus for Lightning Fast Analytics, also featuring Rollups | March 2019 What is a rollup? Period Customer Country Site Hit Count SELECT… Pre-computed aggregates for a period and set of (group by) dimensions. Can be further filtered and aggregated to generate charts.

14. Live Demo of Using Postgres and Citus for Lightning Fast Analytics, also featuring Rollups | March 2019Live Demo of Using Postgres and Citus for Lightning Fast Analytics, also featuring Rollups | March 2019 What are the benefits of rollups? • Fast (indexed) lookups of aggregates • Avoid expensive repeated computation • Rollups can be further aggregated • Rollups are compact, can be kept over longer periods

15. Live Demo of Using Postgres and Citus for Lightning Fast Analytics, also featuring Rollups | March 2019Live Demo of Using Postgres and Citus for Lightning Fast Analytics, also featuring Rollups | March 2019 What is hll? • Approximation algorithm (“sketch” algorithm) • Estimates COUNT(DISTINCT)/cardinality of given data • Extension on postgres: hll • hll data type to store distinct values

16. Live Demo of Using Postgres and Citus for Lightning Fast Analytics, also featuring Rollups | March 2019Live Demo of Using Postgres and Citus for Lightning Fast Analytics, also featuring Rollups | March 2019 Already provisioned a 4-node Citus database cluster for today’s demo / & regular Postgres on single node

17. Live Demo of Using Postgres and Citus for Lightning Fast Analytics, also featuring Rollups | March 2019Live Demo of Using Postgres and Citus for Lightning Fast Analytics, also featuring Rollups | March 2019 Load data Create indexes (btree and GIN) Query patterns in analytics workload Rollup queries What I will show you in today’s analytics demo? Citus 4- node cluster Single node Postgres TOP BOTTOM Load data Create indexes (btree and GIN) Query patterns in analytics workload Rollup queries

18. Live Demo of Using Postgres and Citus for Lightning Fast Analytics, also featuring Rollups | March 2019 Sai Srirampur | PyConCA 2018

19. Live Demo of Using Postgres and Citus for Lightning Fast Analytics, also featuring Rollups | March 2019Live Demo of Using Postgres and Citus for Lightning Fast Analytics, also featuring Rollups | March 2019 1/ Parallelism 2/ Linear scale 3/ Freshness (real-time, concurrency) 4/ Both transactional && analytics

20. Live Demo of Using Postgres and Citus for Lightning Fast Analytics, also featuring Rollups | March 2019Live Demo of Using Postgres and Citus for Lightning Fast Analytics, also featuring Rollups | March 2019 1/ Faster rollups - parallelism - no lag 2/ Increase granularity of period 3/ Store and query longer periods of data 4/ Extensions - hll, topn, pg_cron, pg_partman Enhance your rollup pipeline with

21. Live Demo of Using Postgres and Citus for Lightning Fast Analytics, also featuring Rollups | March 2019 Sai Srirampur | PyConCA 2018 citusdata.com/customers/heap citusdata.com/customers/heap

22. Live Demo of Using Postgres and Citus for Lightning Fast Analytics, also featuring Rollups | March 201922 citusdata.com/customers/algolia

23. Scaling Postgres for Time Series Data with Citus | Nov 15 2018 | Marco Slot | Claire Giordano Using Postgres and Citus for Lightning Fast Analytics | February 2019 Min Wei of Microsoft

24. Scaling Postgres for Time Series Data with Citus | Nov 15 2018 | Marco Slot | Claire Giordano Using Postgres and Citus for Lightning Fast Analytics | February 2019

25. Live Demo of Using Postgres and Citus for Lightning Fast Analytics, also featuring Rollups | March 2019

26. Live Demo of Using Postgres and Citus for Lightning Fast Analytics, also featuring Rollups | March 201926 Any Questions?? (And, resources) DOWNLOAD CITUS OPEN SOURCE: https://www.citusdata.com/download CREATE CITUS CLOUD DEV ACCOUNT: https://www.citusdata.com/product/cloud JOIN OUR PUBLIC SLACK: https://slack.citusdata.com/

Using Postgres and Citus for Lightning Fast Analytics, also ft. Rollups | Live Demo | Sai Srirampur

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Using Postgres and Citus for Lightning Fast Analytics, also ft. Rollups | Live Demo | Sai Srirampur

Similar to Using Postgres and Citus for Lightning Fast Analytics, also ft. Rollups | Live Demo | Sai Srirampur (20)

More from Citus Data

More from Citus Data (20)

Recently uploaded

Recently uploaded (20)

Using Postgres and Citus for Lightning Fast Analytics, also ft. Rollups | Live Demo | Sai Srirampur