VictoriaMetrics
Welcome to our Virtual Meet Up
March 30th @ 5pm GMT / 6pm CET / 9am PT

Agenda
● Welcome & Introductions - Roman
● Overview of the latest features - Roman/Alex
○ What's new in VictoriaMetrics at Q1 2023
○ Streaming Aggregation
○ VictoriaMetrics RemoteWrite protocol
○ Improvements in vmauth
● 2023 Roadmap Review/Update - Roman
● VictoriaLogs Preview - Aliaksandr
● AMA ‘Ask Me Anything’ - All
Feel free to ask questions at any time.

VictoriaMetrics: 2023 Q1 stats
● 180+ issues
● 260+ PRs
● 40 contributors
● 12 releases, from 1.86 to 1.89:
○ 114 FEATURES
○ 103 BUG FIXES

VictoriaMetrics Long-Term Support (LTS)

Latest features
● What's new in VictoriaMetrics at Q1 2023
● Stream Aggregation
● VictoriaMetrics RemoteWrite protocol
● Improvements in vmauth

What's new in VictoriaMetrics at Q1 2023

VMUI dark theme

VMUI explore mode

VMUI sticky tooltips

VMUI cardinality explorer

vmalert - GCS and S3 support for config rules
./vmalert
-eula
-rule=gcs://vmalert-test/rules
-rule=s3://vmalert-test/rules
https://docs.victoriametrics.com/vmalert.html#reading-rules-from-object-storage

Streaming Aggregation

Streaming Aggregation: Use cases
● StatsD alternative:
○ Counting input samples
○ Summing input metrics
○ Quantiles over input metrics
○ Histograms over input metrics
● More efficient recording rules alternative
● Reducing the number of stored samples
● Reducing the number of stored series

Streaming Aggregation: Benefits
● Reducing the load on the vmstorage and vmselect
● Preparing data for heavy queries with low resource consumption
● Reducing cardinality and the amounts of stored data

Streaming Aggregation: Usage
vmagent
vmagent
-remoteWrite.url=<URL>
-remoteWrite.streamAggr.keepInput=<KI>
-remoteWrite.streamAggr.dedupInterval=<DI>
-remoteWrite.streamAggr.config=<CFG>
vmsingle
vmsingle
-remoteWrite.url=<URL>
-streamAggr.keepInput=<KI>
-streamAggr.dedupInterval=<DI>
-streamAggr.config=<CFG>
# streamAggr.config example
- match: {__name__!=""}
interval: 1m
outputs: [avg, total]
by: [instance, job]
output_relabel_configs: {}
input_relabel_configs: {}

Streaming Aggregation Outputs
● total
● increase
● count_series
● count_samples
● sum_samples
● last
● min
● max
● avg
● stddev
● stdvar
● histogram_bucket
● quantiles

Streaming Aggregation Outputs
● total → sum(...)
● increase → increase(...)
● count_series → count(...)
● count_samples → count_over_time(...)
● sum_samples → sum_over_time(...)
● last → last_over_time(...)
● min → min_over_time(...)
● max → max_over_time(...)
● avg → avg_over_time(...)
● stddev → stddev_over_time(...)
● stdvar → stdvar_over_time(...)
● histogram_bucket → histogram_over_time(...)
● quantiles → quantiles_over_time(...)

vmagent
Streaming Aggregation: Example
sum:
count:
avg intermediate state last avg value
avg output

vmagent
Streaming Aggregation: Example
sum:
count:
2.0
1
avg intermediate state last avg value
2.0

vmagent
Streaming Aggregation: Example
sum:
count:
3.0
2
avg intermediate state last avg value
1.0
2.0

vmagent
Streaming Aggregation: Example
sum:
count:
7.0
3
avg intermediate state last avg value
4.0
2.0 1.0

vmagent
Streaming Aggregation: Example
sum:
count:
10.0
4
avg intermediate state last avg value
3.0
2.0 1.0 4.0

vmagent
Streaming Aggregation: Example
sum:
count:
5.0
15.0
5
2.0 1.0 4.0 3.0
avg intermediate state last avg value

vmagent
Streaming Aggregation: Example
sum:
count:
5.0
15.0
5
3.0 3.0
2.0 1.0 4.0 3.0
avg intermediate state last avg value

vmagent
Streaming Aggregation: Example
sum:
count:
5.0
15.0
5
3.0 3.0
2.0 1.0 4.0 3.0
avg intermediate state last avg value
every aggregation interval
every scrape interval

vmagent
Streaming Aggregation: Example
sum:
count:
4.0
35.0
10
3.5 3.0
8.0 0.0 6.0 2.0
avg intermediate state last avg value
every aggregation interval
every scrape interval
5.0
2.0 1.0 4.0 3.0
3.5

vmagent
Streaming Aggregation: Example
sum:
count:
9.0
60.0
15
4.0 3.0
4.0 2.0 1.0 9.0
avg intermediate state last avg value
every aggregation interval
every scrape interval
4.0
8.0 0.0 6.0 2.0
5.0
2.0 1.0 4.0 3.0
4.0
3.5

Example
- match: 'count'
interval: 1m
by: ["instance"]
outputs: ["sum_samples"]
aggregation interval = 1m
scrape/push
interval = 10s

Example
- match: 'count'
interval: 1m
outputs: ["stdvar"]

© 2023 VictoriaMetrics
Streaming Aggregation
For more details, visit:
https://docs.victoriametrics.com/stream-aggregation.html

VictoriaMetrics RemoteWrite protocol
https://victoriametrics.com/blog/victoriametrics-remote-write

VictoriaMetrics RemoteWrite protocol

VictoriaMetrics RemoteWrite protocol

vmauth improvements
Concurrency limiting as DoS protection:
● Global limit via -maxConcurrentRequests
● Global limit per user via -maxConcurrentPerUserRequests
● Individual limit per user via max_concurrent_requests param
See more at:
https://docs.victoriametrics.com/vmauth.html#concurrency-limiting

vmauth improvements
https://docs.victoriametrics.com/vmauth.html#load-balancing
Built-in load balancing
via least-loaded
round-robin fashion

vmauth improvements
● Automatically retry failed requests on other backends
● Incoming requests aren’t sent to temporarily unavailable backends
This allows performing maintenance tasks (upgrade, config update, etc.) at vmselect
and vminsert without the need to remove them from the list of backends at vmauth
config

vmauth improvements
vmauth
backend 1
backend 2
backend 3

vmauth improvements
vmauth
backend 1
backend 2
backend 3
requests:
status:
0
active
requests:
status:
0
active
requests:
status:
0
active
request

vmauth improvements
vmauth
backend 1
backend 2
backend 3
requests:
status:
1
active
requests:
status:
0
active
requests:
status:
0
active
first least loaded

vmauth improvements
vmauth
backend 1
backend 2
backend 3
requests:
status:
1
active
requests:
status:
0
active
requests:
status:
0
active

vmauth improvements
vmauth
backend 1
backend 2
backend 3
requests:
status:
1
active
requests:
status:
1
active
requests:
status:
0
active

vmauth improvements
vmauth
backend 1
backend 2
backend 3
requests:
status:
1
active
requests:
status:
0
broken
requests:
status:
0
active
error

vmauth improvements
vmauth
backend 1
backend 2
backend 3
requests:
status:
1
active
requests:
status:
0
broken
requests:
status:
1
active
retry with other backend

vmauth improvements
vmauth
backend 1
backend 2
backend 3
requests:
status:
1
active
requests:
status:
0
broken
requests:
status:
1
active

vmauth improvements
vmauth
backend 1
backend 2
backend 3
requests:
status:
2
active
requests:
status:
0
broken
requests:
status:
1
active

vmauth improvements
vmauth
backend 1
backend 2
backend 3
requests:
status:
1
active
requests:
status:
0
broken
requests:
status:
1
active

vmauth improvements
vmauth
backend 1
backend 2
backend 3
requests:
status:
1
active
requests:
status:
requests:
status:
1
active
0
active
active again after
3-second timeout

vmauth improvements
vmauth
backend 1
backend 2
backend 3
requests:
status:
1
active
requests:
status:
requests:
status:
1
active
0
active

vmauth improvements
vmauth
backend 1
backend 2
backend 3
requests:
status:
1
active
requests:
status:
requests:
status:
1
active
1
active

vmauth improvements
vmauth
backend 1
backend 2
backend 3
requests:
status:
0
active
requests:
status:
requests:
status:
0
active
1
active

vmauth improvements
vmauth
backend 1
backend 2
backend 3
requests:
status:
0
active
requests:
status:
requests:
status:
0
active
0
active

2023 Roadmap Review/Update

Grafana datasource plugin
VictoriaMetrics/grafana-datasource

Grafana datasource plugin: query trace

Grafana datasource plugin: query trace

OpenTelemetry ingestion protocol support
VictoriaMetrics/pull/2570

OpenTelemetry ingestion protocol support

Streaming aggregation

vmalert: UI for rules management

vmalert hysteresis support

vmui explore tab

VictoriaLogs

VictoriaLogs Preview

Where to find VictoriaMetrics
● 17th - 21st of April: KubeCon Europe in Amsterdam
● 15th of May: SloConf (Online)
● 22nd to 24th of May: Percona Live in Denver
● 26th to 28th of June: Monitorama in Portland
● 26th to 28th of June: GopherCon Europe in Berlin
● And more coming up!

VictoriaMetrics blog
● Save network costs with VictoriaMetrics remote write protocol
● VictoriaMetrics Long-Term Support (LTS): Commitment, Current and
Next LTS Versions
● Rules backfilling via vmalert
● Monitoring benchmark: how to generate 100 million samples/s of
production-like data
● Latest updates about backup components of VictoriaMetrics

VictoriaMetrics in Recent News
● ComputerWeekly: Is it time for time series databases?
● datanami: Open source time series database VictoriaMetrics sees significant
growth
● Data Centre & Network News: VictoriaMetrics announces 252% growth in 2022
● Information Age: Kubernetes Best Monitoring Tools
● Intelligent CIO: VictoriaMetrics leads sustainable monitoring - slashing corporate
energy usage by up to 90%
● Connected Technology Solutions: Sustainable data monitoring for high
performance
● The Stack: One to watch - The story of a startup from Ukraine

Share Your Story?
● We’d love to hear about how and why you use VictoriaMetrics
● Other users benefit from these stories
● You can help us spread the VictoriaMetrics word
○ Write a blog
○ Do a case study with us
○ Talk about VictoriaMetrics in your channels
○ Maybe a video?
● We’re happy to help you share your VictoriaMetrics story
● Contact our team if you’d like to have a chat:
○ info@victoriametrics.com
○ Community Slack

Follow us
● https://victoriametrics.com/
● https://twitter.com/VictoriaMetrics
● https://www.linkedin.com/company/victoriametrics/
● https://www.reddit.com/r/VictoriaMetrics/
● https://slack.victoriametrics.com/
● https://t.me/VictoriaMetrics_en
● https://t.me/VictoriaMetrics_ru