SlideShare ist ein Scribd-Unternehmen logo
1 von 35
Downloaden Sie, um offline zu lesen
© 2022 Altinity, Inc.
Building High Performance Apps with
Altinity Stable Builds®
for ClickHouse
Robert Hodges and Vasily Nemkov
Altinity, Inc.
7 February 2022 1
© 2022 Altinity, Inc.
Let’s make some introductions
ClickHouse support and services including Altinity.Cloud
Authors of Altinity Kubernetes Operator for ClickHouse
and other open source projects
Us
Database geeks with centuries
of experience in DBMS and
applications
You
App developers and business
leaders looking to learn about
real-time analytics*
2
© 2022 Altinity, Inc.
Understands SQL
Runs on bare metal to cloud
Shared nothing architecture
Stores data in columns
Parallel and vectorized execution
Scales to many petabytes
Is Open source (Apache 2.0)
ClickHouse is a SQL Data Warehouse
It’s the core engine for
real-time analytics
ClickHouse
Event
Streams
ELT
Object
Storage
Interactive
Graphics
Dashboards
APIs
3
© 2022 Altinity, Inc.
ClickHouse is the center of the analytic stack
4
ClickHouse
Cluster
Kafka
Table
Engine
CDN
Event
Source
Table
(Mat
View)
Mat Views
with
Aggregates
Mat Views
with
Aggregates
Kafka Broker
Kafka Broker
Managed Kafka
Next.js Backend
Load Balancer
Kafka
Table
Engine
CDN
Event
Source
Table
(Mat
View)
Mat Views
with
Aggregates
Mat Views
with
Aggregates
CDN
performance
telemetry from
Fastly
Kafka Broker
Replicated ClickHouse Tables
GraphQL API
Next.js + Recharts FrontEnd
© 2022 Altinity, Inc. 5
Key questions for any app running ClickHouse
Where can I find
production ready
ClickHouse versions?
How do I know it’s
safe to upgrade?
How long does
support last?
What if there’s a
bug or a security
problem?
© 2022 Altinity, Inc.
© 2022 Altinity, Inc.
Introducing
Altinity Stable
Builds®
for
ClickHouse
6
© 2022 Altinity, Inc.
What are Altinity Stable Builds?
7
Certified by
Altinity
Support
Extra testing
with Altinity
QA suites
Tested and
documented
upgrades
Three years of
maintenance
Reliable,
production-ready
ClickHouse builds
100% open
source
© 2022 Altinity, Inc.
How do you install Altinity Stable Builds? (Ubuntu)
sudo mkdir -p /usr/share/keyrings
curl -s https://builds.altinity.cloud/apt-repo/pubkey.gpg |
sudo gpg --dearmor >
/usr/share/keyrings/altinity-dev-archive-keyring.gpg
sudo sh -c 'echo "deb
[signed-by=/usr/share/keyrings/altinity-dev-archive-keyring.gpg]
https://builds.altinity.cloud/apt-repo stable main" >
/etc/apt/sources.list.d/altinity-dev.list'
sudo apt-get install -y clickhouse-server clickhouse-client
sudo systemctl start clickhouse-server
8
© 2022 Altinity, Inc.
How do you install Altinity Stable Builds? (Docker)
mkdir $HOME/clickhouse-data
docker run -d --name clickhouse-server 
--ulimit nofile=262144:262144 
--volume=$HOME/clickhouse-data:/var/lib/clickhouse 
-p 8123:8123 -p 9000:9000 
altinity/clickhouse-server:22.3.15.34.altinitystable
9
Altinity does not
use the Docker
latest tag
© 2022 Altinity, Inc.
Plus they are automatically available in Altinity.Cloud
10
Stable Build
© 2022 Altinity, Inc.
Where can you get information about stable builds?
11
https://docs.altinity.com/altinitystablebuilds/
© 2022 Altinity, Inc.
Why not just use ClickHouse community builds?
You can!
See instructions at https://clickhouse.com/docs/en/install/
But…
ClickHouse evolves
quickly and not all
releases are stable
Community support is
limited for new releases
© 2022 Altinity, Inc.
What does it mean to “evolve quickly?”
© 2022 Altinity, Inc.
How do ClickHouse releases work?
MAIN
22.12
22.11
22.10
22.9
22.8 LTS
22.3 LTS
Monthly
Releases
LTS
Releases
Fixes for ~1 year
Fixes for ~2
months
© 2022 Altinity, Inc.
Altinity Stable Releases extend LTS support to 3 years
MAIN
22.8 LTS
22.3 LTS
Community fixes for ~1 year Altinity fixes for 2 more years
23.3 LTS
© 2022 Altinity, Inc.
Why does Altinity focus on LTS builds?
Major ClickHouse
installations upgrade *at*
most once every 6
months
Aligns Stable
Builds with
community
support policies
© 2022 Altinity, Inc.
How do we manage changes to make LTS builds work?
ClickHouse
GitHub
Altinity
GitHub
© 2022 Altinity, Inc.
How do ClickHouse builds work?
Create Pull Request
Release Branch CI
Triggers GitHub Action
● Docker images for build
● Compile/link ClickHouse
● Function stateless tests
● Function stateful tests
● Integration tests
● Final checks and finish
Build procedure to
generate and test a
ClickHouse build
(open source)
Build in Altinity fork
of ClickHouse
© 2022 Altinity, Inc.
How does Altinity verify and release a Stable Build?
Create Pull Request
Release Branch CI Altinity QA CI/CD
Triggers GitHub Action
● Docker images for build
● Compile/link ClickHouse
● Function stateless tests
● Function stateful tests
● Integration tests
● Final checks and finish
● Run multiple suites of
automated tests
● Perform manual compatibility
testing
Production Signing
● Sign artifacts
● Post to repos
● Post to Docker Hub
(Manually
triggered)
(Manually
triggered)
© 2022 Altinity, Inc.
What kind of extra checks does Altinity QA include?
● Extra suites of automated tests:
○ Window Functions, AES Encryption Functions, ClickHouse Keeper,
○ Extended DateTime64, Disk Level Encryption, Extended Precision Data Types,
○ Kafka, Kerberos, LDAP, Lightweight Delete, Map Data Type,
○ Parquet Data Type, Part Moves Between Shards
○ S3 (MinIO, AWS, GCS), SSL Server,
○ All Tiered Storage (Local Disk, MinIO, AWS, GCS)
● Manual or semi-manual compatibility testing
○ Client drivers: Python, clickhouse-odbc, SQLAlchemy, clickhouse-jdbc,
○ clickhouse-backup
○ Kubernetes: clickhouse-operator
○ Altinity.Cloud
○ Upgrade and Downgrade (between closest Altinity Stable versions, e.g. 22.8 ⇔ 22.3)
○ BI Tools: Grafana, Tableau, Superset
© 2022 Altinity, Inc.
What additional certifications does Altinity Support add?
● Collect feedback from early users of LTS release
● Perform upgrades of live ClickHouse clusters
● Identify backwards incompatible changes
● Track backports of PRs from MAIN to fix critical bugs
● Certify that the release is stable and ready for general use
© 2022 Altinity, Inc.
When do we release Altinity Stable Builds?
Major releases:
● Release when (a) all tests pass, (b) major bugs are fixed, and (c) support
certifies release for use
Minor releases:
● In response to customer requests for bug fixes
● In response to severe bugs or security issues regardless of source
© 2022 Altinity, Inc.
© 2022 Altinity, Inc.
Best Practices for
Altinity Stable
Builds
23
© 2022 Altinity, Inc.
Using Stable and Community Builds together
ClickHouse Community Builds Altinity Stable Builds
Recommended for development or
when apps depend on cutting
edge/bleeding edge ClickHouse
features
Recommended for production systems
that require stability, tested upgrade,
and long-term support
© 2022 Altinity, Inc.
Best practices for system upgrade
Use Altinity Stable builds for production systems (not community builds!)
Check the upgrade notes carefully!
Test the upgrade before running on production systems
1. Restore cluster from backup
2. Upgrade to new version
3. Connect event streams and apps to the new cluster
4. Discard upgraded test cluster after use
Check with Altinity Support! (Let us know about major upgrades in advance.)
25
© 2022 Altinity, Inc.
© 2022 Altinity, Inc.
What’s next?
26
© 2022 Altinity, Inc.
What’s coming in the Altinity Stable 22.8 LTS
● SQL improvements
○ Experimental support for SQL DELETE (aka lightweight deletes)
○ SELECT … INTO OUTFILE and STDOUT
○ Non-constant arguments for LIKE, ILIKE, and match functions
○ Support expressions in window functions
● Security
○ Named collections extended to cover more data sources
● Replication
○ INSERT into system.zookeeper table
● Remote file systems and object storage
○ Proper support for Google Cloud Storage (GCS) in s3() table function
○ Write cache for remote file systems
○ New cache management commands, e.g., DESCRIBE CACHE, etc.
© 2022 Altinity, Inc.
Compatibility issues in 22.8 upgrade
● Ordinary database deprecated
○ System database will be upgraded to Atomic database engine on startup.
○ Can no longer create Ordinary databases (requires setting to re-enable)
● Background pools configuration changed
○ Must be configured in config.xml *and* uses.xml (it’s a bug)
● Zero-copy replication for remote file systems
○ It is turned off by default. This is different from 22.3.
● S3 multi-part uploads use more threads
○ Can consume more network bandwidth; may also saturate S3-compatible storage like MinIO
IMPORTANT: There are a number of backwards incompatible changes. See the
Altinity release notes!
© 2022 Altinity, Inc.
What’s on the roadmap for Altinity Stable Builds?
We’ll be working on 23.3 as soon as 22.8 is out
Security features -
● We are working on security enhancements to help customers meet FedRAMP
and CIS requirements
● Contact us to find out more - watch for announcements!
© 2022 Altinity, Inc.
© 2022 Altinity, Inc.
Altinity Support
Programs for
Stable Builds
30
© 2022 Altinity, Inc.
Community Support
● Altinity Stable Builds are 100% open source
○ Released under Apache 2.0 license, including any Altinity customizations
○ We strive for maximum achievable compatibility with upstream ClickHouse
● You can get help on the AltinityDB slack channel
● Or file issues against https://github.com/Altinity/ClickHouse
○ Please don’t log them against the main ClickHouse repo
● Fixes are best effort but…
● We take SEV0 and security problems very seriously
© 2022 Altinity, Inc.
Enterprise Support
● Altinity provides bug fixes to Stable Builds in response to customer cases
○ File a ticket in Zendesk or email support@altinity.com to report problems
● Response to Stable Build issues is covered by the support SLA
● We fix the following types of problems:
○ P0/P1/P2 bugs encountered by customers
○ Security issues however they arise
○ Missing features may also be backported on a case-by-case basis
Altinity also offers NRE (Non-recurring engineering) to add new features to
ClickHouse - Contact us for more information
© 2022 Altinity, Inc.
Altinity open source software for ClickHouse
● Altinity Stable Builds – 100% open source builds of ClickHouse
○ Certified for production use
○ 3 years of support
● Altinity contributions to ClickHouse (storage, connectivity, security, data types)
● Altinity Kubernetes Operator for ClickHouse
● Altinity Grafana Plugin (community plugin from Vertamedia)
● Altinity Tableau Connector for ClickHouse
● Altinity Kafka Sink Connector for ClickHouse (MySQL replication)
And many others!!
33
© 2022 Altinity, Inc.
More information about Altinity Stable Builds
See the documentation at https://docs.altinity.com
YouTube Video: Building ClickHouse and Contributing PRs
Ask on Slack or contact us
34
© 2022 Altinity, Inc.
Thank you!
Questions?
Website: https://altinity.com
Email: info@altinity.com
Slack: altinitydbworkspace.slack.com
35
Altinity.Cloud
Altinity Support
Altinity Stable
Builds
We’re hiring!

Weitere ähnliche Inhalte

Ähnlich wie Building High Performance Apps with Altinity Stable Builds for ClickHouse | Altinity Webinar.pdf

End of Support: Understanding the Impact, Timeline, and Benefits of Upgrading...
End of Support: Understanding the Impact, Timeline, and Benefits of Upgrading...End of Support: Understanding the Impact, Timeline, and Benefits of Upgrading...
End of Support: Understanding the Impact, Timeline, and Benefits of Upgrading...
DianaGray10
 
Software Update Mechanisms: Selecting the Best Solutin for Your Embedded Linu...
Software Update Mechanisms: Selecting the Best Solutin for Your Embedded Linu...Software Update Mechanisms: Selecting the Best Solutin for Your Embedded Linu...
Software Update Mechanisms: Selecting the Best Solutin for Your Embedded Linu...
ICS
 
Git Gerrit Mit Teamforge
Git Gerrit Mit TeamforgeGit Gerrit Mit Teamforge
Git Gerrit Mit Teamforge
CollabNet
 

Ähnlich wie Building High Performance Apps with Altinity Stable Builds for ClickHouse | Altinity Webinar.pdf (20)

Weave GitOps 2022.09 Release: A Fast & Reliable Path to Production with Progr...
Weave GitOps 2022.09 Release: A Fast & Reliable Path to Production with Progr...Weave GitOps 2022.09 Release: A Fast & Reliable Path to Production with Progr...
Weave GitOps 2022.09 Release: A Fast & Reliable Path to Production with Progr...
 
Room 2 - 4 - Juncheng Anthony Lin - Redhat - A Practical Approach to Traditio...
Room 2 - 4 - Juncheng Anthony Lin - Redhat - A Practical Approach to Traditio...Room 2 - 4 - Juncheng Anthony Lin - Redhat - A Practical Approach to Traditio...
Room 2 - 4 - Juncheng Anthony Lin - Redhat - A Practical Approach to Traditio...
 
Overcoming Regulatory & Compliance Hurdles with Hybrid Cloud EKS and Weave Gi...
Overcoming Regulatory & Compliance Hurdles with Hybrid Cloud EKS and Weave Gi...Overcoming Regulatory & Compliance Hurdles with Hybrid Cloud EKS and Weave Gi...
Overcoming Regulatory & Compliance Hurdles with Hybrid Cloud EKS and Weave Gi...
 
Intro to GitOps with Weave GitOps, Flagger and Linkerd
Intro to GitOps with Weave GitOps, Flagger and LinkerdIntro to GitOps with Weave GitOps, Flagger and Linkerd
Intro to GitOps with Weave GitOps, Flagger and Linkerd
 
Free GitOps Workshop
Free GitOps WorkshopFree GitOps Workshop
Free GitOps Workshop
 
How to manage Kubernetes at scale with just git
How to manage Kubernetes at scale with just git How to manage Kubernetes at scale with just git
How to manage Kubernetes at scale with just git
 
State of Steeltoe 2020
State of Steeltoe 2020State of Steeltoe 2020
State of Steeltoe 2020
 
Flux is incubating + the road ahead
Flux is incubating + the road aheadFlux is incubating + the road ahead
Flux is incubating + the road ahead
 
End of Support: Understanding the Impact, Timeline, and Benefits of Upgrading...
End of Support: Understanding the Impact, Timeline, and Benefits of Upgrading...End of Support: Understanding the Impact, Timeline, and Benefits of Upgrading...
End of Support: Understanding the Impact, Timeline, and Benefits of Upgrading...
 
Git essential training & sharing self
Git essential training & sharing selfGit essential training & sharing self
Git essential training & sharing self
 
Developing NuGet
Developing NuGetDeveloping NuGet
Developing NuGet
 
GCP Meetup #3 - Approaches to Cloud Native Architectures
GCP Meetup #3 - Approaches to Cloud Native ArchitecturesGCP Meetup #3 - Approaches to Cloud Native Architectures
GCP Meetup #3 - Approaches to Cloud Native Architectures
 
stackconf 2023 | Bringing Order to Chaos: Make Your Systems More Resilient wi...
stackconf 2023 | Bringing Order to Chaos: Make Your Systems More Resilient wi...stackconf 2023 | Bringing Order to Chaos: Make Your Systems More Resilient wi...
stackconf 2023 | Bringing Order to Chaos: Make Your Systems More Resilient wi...
 
Software Update Mechanisms: Selecting the Best Solutin for Your Embedded Linu...
Software Update Mechanisms: Selecting the Best Solutin for Your Embedded Linu...Software Update Mechanisms: Selecting the Best Solutin for Your Embedded Linu...
Software Update Mechanisms: Selecting the Best Solutin for Your Embedded Linu...
 
Continuous Lifecycle London 2018 Event Keynote
Continuous Lifecycle London 2018 Event KeynoteContinuous Lifecycle London 2018 Event Keynote
Continuous Lifecycle London 2018 Event Keynote
 
Automated Provisioning, Management & Cost Control for Kubernetes Clusters
Automated Provisioning, Management & Cost Control for Kubernetes ClustersAutomated Provisioning, Management & Cost Control for Kubernetes Clusters
Automated Provisioning, Management & Cost Control for Kubernetes Clusters
 
Git Gerrit Mit Teamforge
Git Gerrit Mit TeamforgeGit Gerrit Mit Teamforge
Git Gerrit Mit Teamforge
 
Deploying Stateful Applications Securely & Confidently with Ondat & Weave GitOps
Deploying Stateful Applications Securely & Confidently with Ondat & Weave GitOpsDeploying Stateful Applications Securely & Confidently with Ondat & Weave GitOps
Deploying Stateful Applications Securely & Confidently with Ondat & Weave GitOps
 
Api gitlab: configurazione dei progetti as a service
Api gitlab: configurazione dei progetti as a serviceApi gitlab: configurazione dei progetti as a service
Api gitlab: configurazione dei progetti as a service
 
Optimize your CI/CD with GitLab and AWS
Optimize your CI/CD with GitLab and AWSOptimize your CI/CD with GitLab and AWS
Optimize your CI/CD with GitLab and AWS
 

Mehr von Altinity Ltd

Mehr von Altinity Ltd (20)

Building an Analytic Extension to MySQL with ClickHouse and Open Source.pptx
Building an Analytic Extension to MySQL with ClickHouse and Open Source.pptxBuilding an Analytic Extension to MySQL with ClickHouse and Open Source.pptx
Building an Analytic Extension to MySQL with ClickHouse and Open Source.pptx
 
Cloud Native ClickHouse at Scale--Using the Altinity Kubernetes Operator-2022...
Cloud Native ClickHouse at Scale--Using the Altinity Kubernetes Operator-2022...Cloud Native ClickHouse at Scale--Using the Altinity Kubernetes Operator-2022...
Cloud Native ClickHouse at Scale--Using the Altinity Kubernetes Operator-2022...
 
Building an Analytic Extension to MySQL with ClickHouse and Open Source
Building an Analytic Extension to MySQL with ClickHouse and Open SourceBuilding an Analytic Extension to MySQL with ClickHouse and Open Source
Building an Analytic Extension to MySQL with ClickHouse and Open Source
 
Fun with ClickHouse Window Functions-2021-08-19.pdf
Fun with ClickHouse Window Functions-2021-08-19.pdfFun with ClickHouse Window Functions-2021-08-19.pdf
Fun with ClickHouse Window Functions-2021-08-19.pdf
 
Cloud Native Data Warehouses - Intro to ClickHouse on Kubernetes-2021-07.pdf
Cloud Native Data Warehouses - Intro to ClickHouse on Kubernetes-2021-07.pdfCloud Native Data Warehouses - Intro to ClickHouse on Kubernetes-2021-07.pdf
Cloud Native Data Warehouses - Intro to ClickHouse on Kubernetes-2021-07.pdf
 
Application Monitoring using Open Source - VictoriaMetrics & Altinity ClickHo...
Application Monitoring using Open Source - VictoriaMetrics & Altinity ClickHo...Application Monitoring using Open Source - VictoriaMetrics & Altinity ClickHo...
Application Monitoring using Open Source - VictoriaMetrics & Altinity ClickHo...
 
Own your ClickHouse data with Altinity.Cloud Anywhere-2023-01-17.pdf
Own your ClickHouse data with Altinity.Cloud Anywhere-2023-01-17.pdfOwn your ClickHouse data with Altinity.Cloud Anywhere-2023-01-17.pdf
Own your ClickHouse data with Altinity.Cloud Anywhere-2023-01-17.pdf
 
ClickHouse ReplacingMergeTree in Telecom Apps
ClickHouse ReplacingMergeTree in Telecom AppsClickHouse ReplacingMergeTree in Telecom Apps
ClickHouse ReplacingMergeTree in Telecom Apps
 
Adventures with the ClickHouse ReplacingMergeTree Engine
Adventures with the ClickHouse ReplacingMergeTree EngineAdventures with the ClickHouse ReplacingMergeTree Engine
Adventures with the ClickHouse ReplacingMergeTree Engine
 
Building a Real-Time Analytics Application with Apache Pulsar and Apache Pinot
Building a Real-Time Analytics Application with  Apache Pulsar and Apache PinotBuilding a Real-Time Analytics Application with  Apache Pulsar and Apache Pinot
Building a Real-Time Analytics Application with Apache Pulsar and Apache Pinot
 
OSA Con 2022 - What Data Engineering Can Learn from Frontend Engineering - Pe...
OSA Con 2022 - What Data Engineering Can Learn from Frontend Engineering - Pe...OSA Con 2022 - What Data Engineering Can Learn from Frontend Engineering - Pe...
OSA Con 2022 - What Data Engineering Can Learn from Frontend Engineering - Pe...
 
OSA Con 2022 - Welcome to OSA CON Version 2022 - Robert Hodges - Altinity.pdf
OSA Con 2022 - Welcome to OSA CON Version 2022 - Robert Hodges - Altinity.pdfOSA Con 2022 - Welcome to OSA CON Version 2022 - Robert Hodges - Altinity.pdf
OSA Con 2022 - Welcome to OSA CON Version 2022 - Robert Hodges - Altinity.pdf
 
OSA Con 2022 - Using ClickHouse Database to Power Analytics and Customer Enga...
OSA Con 2022 - Using ClickHouse Database to Power Analytics and Customer Enga...OSA Con 2022 - Using ClickHouse Database to Power Analytics and Customer Enga...
OSA Con 2022 - Using ClickHouse Database to Power Analytics and Customer Enga...
 
OSA Con 2022 - Tips and Tricks to Keep Your Queries under 100ms with ClickHou...
OSA Con 2022 - Tips and Tricks to Keep Your Queries under 100ms with ClickHou...OSA Con 2022 - Tips and Tricks to Keep Your Queries under 100ms with ClickHou...
OSA Con 2022 - Tips and Tricks to Keep Your Queries under 100ms with ClickHou...
 
OSA Con 2022 - The Open Source Analytic Universe, Version 2022 - Robert Hodge...
OSA Con 2022 - The Open Source Analytic Universe, Version 2022 - Robert Hodge...OSA Con 2022 - The Open Source Analytic Universe, Version 2022 - Robert Hodge...
OSA Con 2022 - The Open Source Analytic Universe, Version 2022 - Robert Hodge...
 
OSA Con 2022 - Switching Jaeger Distributed Tracing to ClickHouse to Enable A...
OSA Con 2022 - Switching Jaeger Distributed Tracing to ClickHouse to Enable A...OSA Con 2022 - Switching Jaeger Distributed Tracing to ClickHouse to Enable A...
OSA Con 2022 - Switching Jaeger Distributed Tracing to ClickHouse to Enable A...
 
OSA Con 2022 - Streaming Data Made Easy - Tim Spann & David Kjerrumgaard - St...
OSA Con 2022 - Streaming Data Made Easy - Tim Spann & David Kjerrumgaard - St...OSA Con 2022 - Streaming Data Made Easy - Tim Spann & David Kjerrumgaard - St...
OSA Con 2022 - Streaming Data Made Easy - Tim Spann & David Kjerrumgaard - St...
 
OSA Con 2022 - State of Open Source Databases - Peter Zaitsev - Percona.pdf
OSA Con 2022 - State of Open Source Databases - Peter Zaitsev - Percona.pdfOSA Con 2022 - State of Open Source Databases - Peter Zaitsev - Percona.pdf
OSA Con 2022 - State of Open Source Databases - Peter Zaitsev - Percona.pdf
 
OSA Con 2022 - Specifics of data analysis in Time Series Databases - Roman Kh...
OSA Con 2022 - Specifics of data analysis in Time Series Databases - Roman Kh...OSA Con 2022 - Specifics of data analysis in Time Series Databases - Roman Kh...
OSA Con 2022 - Specifics of data analysis in Time Series Databases - Roman Kh...
 
OSA Con 2022 - Signal Correlation, the Ho11y Grail - Michael Hausenblas - AWS...
OSA Con 2022 - Signal Correlation, the Ho11y Grail - Michael Hausenblas - AWS...OSA Con 2022 - Signal Correlation, the Ho11y Grail - Michael Hausenblas - AWS...
OSA Con 2022 - Signal Correlation, the Ho11y Grail - Michael Hausenblas - AWS...
 

Kürzlich hochgeladen

Computer science Sql cheat sheet.pdf.pdf
Computer science Sql cheat sheet.pdf.pdfComputer science Sql cheat sheet.pdf.pdf
Computer science Sql cheat sheet.pdf.pdf
SayantanBiswas37
 
Top profile Call Girls In Indore [ 7014168258 ] Call Me For Genuine Models We...
Top profile Call Girls In Indore [ 7014168258 ] Call Me For Genuine Models We...Top profile Call Girls In Indore [ 7014168258 ] Call Me For Genuine Models We...
Top profile Call Girls In Indore [ 7014168258 ] Call Me For Genuine Models We...
gajnagarg
 
Top profile Call Girls In Begusarai [ 7014168258 ] Call Me For Genuine Models...
Top profile Call Girls In Begusarai [ 7014168258 ] Call Me For Genuine Models...Top profile Call Girls In Begusarai [ 7014168258 ] Call Me For Genuine Models...
Top profile Call Girls In Begusarai [ 7014168258 ] Call Me For Genuine Models...
nirzagarg
 
Sealdah % High Class Call Girls Kolkata - 450+ Call Girl Cash Payment 8005736...
Sealdah % High Class Call Girls Kolkata - 450+ Call Girl Cash Payment 8005736...Sealdah % High Class Call Girls Kolkata - 450+ Call Girl Cash Payment 8005736...
Sealdah % High Class Call Girls Kolkata - 450+ Call Girl Cash Payment 8005736...
HyderabadDolls
 
Reconciling Conflicting Data Curation Actions: Transparency Through Argument...
Reconciling Conflicting Data Curation Actions:  Transparency Through Argument...Reconciling Conflicting Data Curation Actions:  Transparency Through Argument...
Reconciling Conflicting Data Curation Actions: Transparency Through Argument...
Bertram Ludäscher
 
+97470301568>>weed for sale in qatar ,weed for sale in dubai,weed for sale in...
+97470301568>>weed for sale in qatar ,weed for sale in dubai,weed for sale in...+97470301568>>weed for sale in qatar ,weed for sale in dubai,weed for sale in...
+97470301568>>weed for sale in qatar ,weed for sale in dubai,weed for sale in...
Health
 
Gartner's Data Analytics Maturity Model.pptx
Gartner's Data Analytics Maturity Model.pptxGartner's Data Analytics Maturity Model.pptx
Gartner's Data Analytics Maturity Model.pptx
chadhar227
 
Top profile Call Girls In bhavnagar [ 7014168258 ] Call Me For Genuine Models...
Top profile Call Girls In bhavnagar [ 7014168258 ] Call Me For Genuine Models...Top profile Call Girls In bhavnagar [ 7014168258 ] Call Me For Genuine Models...
Top profile Call Girls In bhavnagar [ 7014168258 ] Call Me For Genuine Models...
gajnagarg
 
Top profile Call Girls In Chandrapur [ 7014168258 ] Call Me For Genuine Model...
Top profile Call Girls In Chandrapur [ 7014168258 ] Call Me For Genuine Model...Top profile Call Girls In Chandrapur [ 7014168258 ] Call Me For Genuine Model...
Top profile Call Girls In Chandrapur [ 7014168258 ] Call Me For Genuine Model...
gajnagarg
 
怎样办理圣地亚哥州立大学毕业证(SDSU毕业证书)成绩单学校原版复制
怎样办理圣地亚哥州立大学毕业证(SDSU毕业证书)成绩单学校原版复制怎样办理圣地亚哥州立大学毕业证(SDSU毕业证书)成绩单学校原版复制
怎样办理圣地亚哥州立大学毕业证(SDSU毕业证书)成绩单学校原版复制
vexqp
 
Top profile Call Girls In dimapur [ 7014168258 ] Call Me For Genuine Models W...
Top profile Call Girls In dimapur [ 7014168258 ] Call Me For Genuine Models W...Top profile Call Girls In dimapur [ 7014168258 ] Call Me For Genuine Models W...
Top profile Call Girls In dimapur [ 7014168258 ] Call Me For Genuine Models W...
gajnagarg
 

Kürzlich hochgeladen (20)

Computer science Sql cheat sheet.pdf.pdf
Computer science Sql cheat sheet.pdf.pdfComputer science Sql cheat sheet.pdf.pdf
Computer science Sql cheat sheet.pdf.pdf
 
Top profile Call Girls In Indore [ 7014168258 ] Call Me For Genuine Models We...
Top profile Call Girls In Indore [ 7014168258 ] Call Me For Genuine Models We...Top profile Call Girls In Indore [ 7014168258 ] Call Me For Genuine Models We...
Top profile Call Girls In Indore [ 7014168258 ] Call Me For Genuine Models We...
 
Ranking and Scoring Exercises for Research
Ranking and Scoring Exercises for ResearchRanking and Scoring Exercises for Research
Ranking and Scoring Exercises for Research
 
Top profile Call Girls In Begusarai [ 7014168258 ] Call Me For Genuine Models...
Top profile Call Girls In Begusarai [ 7014168258 ] Call Me For Genuine Models...Top profile Call Girls In Begusarai [ 7014168258 ] Call Me For Genuine Models...
Top profile Call Girls In Begusarai [ 7014168258 ] Call Me For Genuine Models...
 
Gulbai Tekra * Cheap Call Girls In Ahmedabad Phone No 8005736733 Elite Escort...
Gulbai Tekra * Cheap Call Girls In Ahmedabad Phone No 8005736733 Elite Escort...Gulbai Tekra * Cheap Call Girls In Ahmedabad Phone No 8005736733 Elite Escort...
Gulbai Tekra * Cheap Call Girls In Ahmedabad Phone No 8005736733 Elite Escort...
 
High Profile Call Girls Service in Jalore { 9332606886 } VVIP NISHA Call Girl...
High Profile Call Girls Service in Jalore { 9332606886 } VVIP NISHA Call Girl...High Profile Call Girls Service in Jalore { 9332606886 } VVIP NISHA Call Girl...
High Profile Call Girls Service in Jalore { 9332606886 } VVIP NISHA Call Girl...
 
Sealdah % High Class Call Girls Kolkata - 450+ Call Girl Cash Payment 8005736...
Sealdah % High Class Call Girls Kolkata - 450+ Call Girl Cash Payment 8005736...Sealdah % High Class Call Girls Kolkata - 450+ Call Girl Cash Payment 8005736...
Sealdah % High Class Call Girls Kolkata - 450+ Call Girl Cash Payment 8005736...
 
Predicting HDB Resale Prices - Conducting Linear Regression Analysis With Orange
Predicting HDB Resale Prices - Conducting Linear Regression Analysis With OrangePredicting HDB Resale Prices - Conducting Linear Regression Analysis With Orange
Predicting HDB Resale Prices - Conducting Linear Regression Analysis With Orange
 
Digital Transformation Playbook by Graham Ware
Digital Transformation Playbook by Graham WareDigital Transformation Playbook by Graham Ware
Digital Transformation Playbook by Graham Ware
 
Reconciling Conflicting Data Curation Actions: Transparency Through Argument...
Reconciling Conflicting Data Curation Actions:  Transparency Through Argument...Reconciling Conflicting Data Curation Actions:  Transparency Through Argument...
Reconciling Conflicting Data Curation Actions: Transparency Through Argument...
 
Vadodara 💋 Call Girl 7737669865 Call Girls in Vadodara Escort service book now
Vadodara 💋 Call Girl 7737669865 Call Girls in Vadodara Escort service book nowVadodara 💋 Call Girl 7737669865 Call Girls in Vadodara Escort service book now
Vadodara 💋 Call Girl 7737669865 Call Girls in Vadodara Escort service book now
 
DATA SUMMIT 24 Building Real-Time Pipelines With FLaNK
DATA SUMMIT 24  Building Real-Time Pipelines With FLaNKDATA SUMMIT 24  Building Real-Time Pipelines With FLaNK
DATA SUMMIT 24 Building Real-Time Pipelines With FLaNK
 
TrafficWave Generator Will Instantly drive targeted and engaging traffic back...
TrafficWave Generator Will Instantly drive targeted and engaging traffic back...TrafficWave Generator Will Instantly drive targeted and engaging traffic back...
TrafficWave Generator Will Instantly drive targeted and engaging traffic back...
 
+97470301568>>weed for sale in qatar ,weed for sale in dubai,weed for sale in...
+97470301568>>weed for sale in qatar ,weed for sale in dubai,weed for sale in...+97470301568>>weed for sale in qatar ,weed for sale in dubai,weed for sale in...
+97470301568>>weed for sale in qatar ,weed for sale in dubai,weed for sale in...
 
SAC 25 Final National, Regional & Local Angel Group Investing Insights 2024 0...
SAC 25 Final National, Regional & Local Angel Group Investing Insights 2024 0...SAC 25 Final National, Regional & Local Angel Group Investing Insights 2024 0...
SAC 25 Final National, Regional & Local Angel Group Investing Insights 2024 0...
 
Gartner's Data Analytics Maturity Model.pptx
Gartner's Data Analytics Maturity Model.pptxGartner's Data Analytics Maturity Model.pptx
Gartner's Data Analytics Maturity Model.pptx
 
Top profile Call Girls In bhavnagar [ 7014168258 ] Call Me For Genuine Models...
Top profile Call Girls In bhavnagar [ 7014168258 ] Call Me For Genuine Models...Top profile Call Girls In bhavnagar [ 7014168258 ] Call Me For Genuine Models...
Top profile Call Girls In bhavnagar [ 7014168258 ] Call Me For Genuine Models...
 
Top profile Call Girls In Chandrapur [ 7014168258 ] Call Me For Genuine Model...
Top profile Call Girls In Chandrapur [ 7014168258 ] Call Me For Genuine Model...Top profile Call Girls In Chandrapur [ 7014168258 ] Call Me For Genuine Model...
Top profile Call Girls In Chandrapur [ 7014168258 ] Call Me For Genuine Model...
 
怎样办理圣地亚哥州立大学毕业证(SDSU毕业证书)成绩单学校原版复制
怎样办理圣地亚哥州立大学毕业证(SDSU毕业证书)成绩单学校原版复制怎样办理圣地亚哥州立大学毕业证(SDSU毕业证书)成绩单学校原版复制
怎样办理圣地亚哥州立大学毕业证(SDSU毕业证书)成绩单学校原版复制
 
Top profile Call Girls In dimapur [ 7014168258 ] Call Me For Genuine Models W...
Top profile Call Girls In dimapur [ 7014168258 ] Call Me For Genuine Models W...Top profile Call Girls In dimapur [ 7014168258 ] Call Me For Genuine Models W...
Top profile Call Girls In dimapur [ 7014168258 ] Call Me For Genuine Models W...
 

Building High Performance Apps with Altinity Stable Builds for ClickHouse | Altinity Webinar.pdf

  • 1. © 2022 Altinity, Inc. Building High Performance Apps with Altinity Stable Builds® for ClickHouse Robert Hodges and Vasily Nemkov Altinity, Inc. 7 February 2022 1
  • 2. © 2022 Altinity, Inc. Let’s make some introductions ClickHouse support and services including Altinity.Cloud Authors of Altinity Kubernetes Operator for ClickHouse and other open source projects Us Database geeks with centuries of experience in DBMS and applications You App developers and business leaders looking to learn about real-time analytics* 2
  • 3. © 2022 Altinity, Inc. Understands SQL Runs on bare metal to cloud Shared nothing architecture Stores data in columns Parallel and vectorized execution Scales to many petabytes Is Open source (Apache 2.0) ClickHouse is a SQL Data Warehouse It’s the core engine for real-time analytics ClickHouse Event Streams ELT Object Storage Interactive Graphics Dashboards APIs 3
  • 4. © 2022 Altinity, Inc. ClickHouse is the center of the analytic stack 4 ClickHouse Cluster Kafka Table Engine CDN Event Source Table (Mat View) Mat Views with Aggregates Mat Views with Aggregates Kafka Broker Kafka Broker Managed Kafka Next.js Backend Load Balancer Kafka Table Engine CDN Event Source Table (Mat View) Mat Views with Aggregates Mat Views with Aggregates CDN performance telemetry from Fastly Kafka Broker Replicated ClickHouse Tables GraphQL API Next.js + Recharts FrontEnd
  • 5. © 2022 Altinity, Inc. 5 Key questions for any app running ClickHouse Where can I find production ready ClickHouse versions? How do I know it’s safe to upgrade? How long does support last? What if there’s a bug or a security problem?
  • 6. © 2022 Altinity, Inc. © 2022 Altinity, Inc. Introducing Altinity Stable Builds® for ClickHouse 6
  • 7. © 2022 Altinity, Inc. What are Altinity Stable Builds? 7 Certified by Altinity Support Extra testing with Altinity QA suites Tested and documented upgrades Three years of maintenance Reliable, production-ready ClickHouse builds 100% open source
  • 8. © 2022 Altinity, Inc. How do you install Altinity Stable Builds? (Ubuntu) sudo mkdir -p /usr/share/keyrings curl -s https://builds.altinity.cloud/apt-repo/pubkey.gpg | sudo gpg --dearmor > /usr/share/keyrings/altinity-dev-archive-keyring.gpg sudo sh -c 'echo "deb [signed-by=/usr/share/keyrings/altinity-dev-archive-keyring.gpg] https://builds.altinity.cloud/apt-repo stable main" > /etc/apt/sources.list.d/altinity-dev.list' sudo apt-get install -y clickhouse-server clickhouse-client sudo systemctl start clickhouse-server 8
  • 9. © 2022 Altinity, Inc. How do you install Altinity Stable Builds? (Docker) mkdir $HOME/clickhouse-data docker run -d --name clickhouse-server --ulimit nofile=262144:262144 --volume=$HOME/clickhouse-data:/var/lib/clickhouse -p 8123:8123 -p 9000:9000 altinity/clickhouse-server:22.3.15.34.altinitystable 9 Altinity does not use the Docker latest tag
  • 10. © 2022 Altinity, Inc. Plus they are automatically available in Altinity.Cloud 10 Stable Build
  • 11. © 2022 Altinity, Inc. Where can you get information about stable builds? 11 https://docs.altinity.com/altinitystablebuilds/
  • 12. © 2022 Altinity, Inc. Why not just use ClickHouse community builds? You can! See instructions at https://clickhouse.com/docs/en/install/ But… ClickHouse evolves quickly and not all releases are stable Community support is limited for new releases
  • 13. © 2022 Altinity, Inc. What does it mean to “evolve quickly?”
  • 14. © 2022 Altinity, Inc. How do ClickHouse releases work? MAIN 22.12 22.11 22.10 22.9 22.8 LTS 22.3 LTS Monthly Releases LTS Releases Fixes for ~1 year Fixes for ~2 months
  • 15. © 2022 Altinity, Inc. Altinity Stable Releases extend LTS support to 3 years MAIN 22.8 LTS 22.3 LTS Community fixes for ~1 year Altinity fixes for 2 more years 23.3 LTS
  • 16. © 2022 Altinity, Inc. Why does Altinity focus on LTS builds? Major ClickHouse installations upgrade *at* most once every 6 months Aligns Stable Builds with community support policies
  • 17. © 2022 Altinity, Inc. How do we manage changes to make LTS builds work? ClickHouse GitHub Altinity GitHub
  • 18. © 2022 Altinity, Inc. How do ClickHouse builds work? Create Pull Request Release Branch CI Triggers GitHub Action ● Docker images for build ● Compile/link ClickHouse ● Function stateless tests ● Function stateful tests ● Integration tests ● Final checks and finish Build procedure to generate and test a ClickHouse build (open source) Build in Altinity fork of ClickHouse
  • 19. © 2022 Altinity, Inc. How does Altinity verify and release a Stable Build? Create Pull Request Release Branch CI Altinity QA CI/CD Triggers GitHub Action ● Docker images for build ● Compile/link ClickHouse ● Function stateless tests ● Function stateful tests ● Integration tests ● Final checks and finish ● Run multiple suites of automated tests ● Perform manual compatibility testing Production Signing ● Sign artifacts ● Post to repos ● Post to Docker Hub (Manually triggered) (Manually triggered)
  • 20. © 2022 Altinity, Inc. What kind of extra checks does Altinity QA include? ● Extra suites of automated tests: ○ Window Functions, AES Encryption Functions, ClickHouse Keeper, ○ Extended DateTime64, Disk Level Encryption, Extended Precision Data Types, ○ Kafka, Kerberos, LDAP, Lightweight Delete, Map Data Type, ○ Parquet Data Type, Part Moves Between Shards ○ S3 (MinIO, AWS, GCS), SSL Server, ○ All Tiered Storage (Local Disk, MinIO, AWS, GCS) ● Manual or semi-manual compatibility testing ○ Client drivers: Python, clickhouse-odbc, SQLAlchemy, clickhouse-jdbc, ○ clickhouse-backup ○ Kubernetes: clickhouse-operator ○ Altinity.Cloud ○ Upgrade and Downgrade (between closest Altinity Stable versions, e.g. 22.8 ⇔ 22.3) ○ BI Tools: Grafana, Tableau, Superset
  • 21. © 2022 Altinity, Inc. What additional certifications does Altinity Support add? ● Collect feedback from early users of LTS release ● Perform upgrades of live ClickHouse clusters ● Identify backwards incompatible changes ● Track backports of PRs from MAIN to fix critical bugs ● Certify that the release is stable and ready for general use
  • 22. © 2022 Altinity, Inc. When do we release Altinity Stable Builds? Major releases: ● Release when (a) all tests pass, (b) major bugs are fixed, and (c) support certifies release for use Minor releases: ● In response to customer requests for bug fixes ● In response to severe bugs or security issues regardless of source
  • 23. © 2022 Altinity, Inc. © 2022 Altinity, Inc. Best Practices for Altinity Stable Builds 23
  • 24. © 2022 Altinity, Inc. Using Stable and Community Builds together ClickHouse Community Builds Altinity Stable Builds Recommended for development or when apps depend on cutting edge/bleeding edge ClickHouse features Recommended for production systems that require stability, tested upgrade, and long-term support
  • 25. © 2022 Altinity, Inc. Best practices for system upgrade Use Altinity Stable builds for production systems (not community builds!) Check the upgrade notes carefully! Test the upgrade before running on production systems 1. Restore cluster from backup 2. Upgrade to new version 3. Connect event streams and apps to the new cluster 4. Discard upgraded test cluster after use Check with Altinity Support! (Let us know about major upgrades in advance.) 25
  • 26. © 2022 Altinity, Inc. © 2022 Altinity, Inc. What’s next? 26
  • 27. © 2022 Altinity, Inc. What’s coming in the Altinity Stable 22.8 LTS ● SQL improvements ○ Experimental support for SQL DELETE (aka lightweight deletes) ○ SELECT … INTO OUTFILE and STDOUT ○ Non-constant arguments for LIKE, ILIKE, and match functions ○ Support expressions in window functions ● Security ○ Named collections extended to cover more data sources ● Replication ○ INSERT into system.zookeeper table ● Remote file systems and object storage ○ Proper support for Google Cloud Storage (GCS) in s3() table function ○ Write cache for remote file systems ○ New cache management commands, e.g., DESCRIBE CACHE, etc.
  • 28. © 2022 Altinity, Inc. Compatibility issues in 22.8 upgrade ● Ordinary database deprecated ○ System database will be upgraded to Atomic database engine on startup. ○ Can no longer create Ordinary databases (requires setting to re-enable) ● Background pools configuration changed ○ Must be configured in config.xml *and* uses.xml (it’s a bug) ● Zero-copy replication for remote file systems ○ It is turned off by default. This is different from 22.3. ● S3 multi-part uploads use more threads ○ Can consume more network bandwidth; may also saturate S3-compatible storage like MinIO IMPORTANT: There are a number of backwards incompatible changes. See the Altinity release notes!
  • 29. © 2022 Altinity, Inc. What’s on the roadmap for Altinity Stable Builds? We’ll be working on 23.3 as soon as 22.8 is out Security features - ● We are working on security enhancements to help customers meet FedRAMP and CIS requirements ● Contact us to find out more - watch for announcements!
  • 30. © 2022 Altinity, Inc. © 2022 Altinity, Inc. Altinity Support Programs for Stable Builds 30
  • 31. © 2022 Altinity, Inc. Community Support ● Altinity Stable Builds are 100% open source ○ Released under Apache 2.0 license, including any Altinity customizations ○ We strive for maximum achievable compatibility with upstream ClickHouse ● You can get help on the AltinityDB slack channel ● Or file issues against https://github.com/Altinity/ClickHouse ○ Please don’t log them against the main ClickHouse repo ● Fixes are best effort but… ● We take SEV0 and security problems very seriously
  • 32. © 2022 Altinity, Inc. Enterprise Support ● Altinity provides bug fixes to Stable Builds in response to customer cases ○ File a ticket in Zendesk or email support@altinity.com to report problems ● Response to Stable Build issues is covered by the support SLA ● We fix the following types of problems: ○ P0/P1/P2 bugs encountered by customers ○ Security issues however they arise ○ Missing features may also be backported on a case-by-case basis Altinity also offers NRE (Non-recurring engineering) to add new features to ClickHouse - Contact us for more information
  • 33. © 2022 Altinity, Inc. Altinity open source software for ClickHouse ● Altinity Stable Builds – 100% open source builds of ClickHouse ○ Certified for production use ○ 3 years of support ● Altinity contributions to ClickHouse (storage, connectivity, security, data types) ● Altinity Kubernetes Operator for ClickHouse ● Altinity Grafana Plugin (community plugin from Vertamedia) ● Altinity Tableau Connector for ClickHouse ● Altinity Kafka Sink Connector for ClickHouse (MySQL replication) And many others!! 33
  • 34. © 2022 Altinity, Inc. More information about Altinity Stable Builds See the documentation at https://docs.altinity.com YouTube Video: Building ClickHouse and Contributing PRs Ask on Slack or contact us 34
  • 35. © 2022 Altinity, Inc. Thank you! Questions? Website: https://altinity.com Email: info@altinity.com Slack: altinitydbworkspace.slack.com 35 Altinity.Cloud Altinity Support Altinity Stable Builds We’re hiring!