RedisGears is a serverless engine for Redis that supports event-driven and batch operations. It allows defining a pipe of operations like mapping, filtering, aggregation, and more. RedisGears can be used for stream processing by triggering executions on Redis stream or key events. It also serves as a multimodel engine by allowing data to be read from Redis and passed to other modules like RediSearch, RedisGraph, or external services.

1. PRESENTED BY
RedisGears
Meir Shpilraien (@Meir_Shpilraien)
Redislabs, Senior Software Engineer

2. PRESENTED BY
1 What is RedisGears
2 Stream Processing with RedisGears
3 RedisGears as a Multimodel Engine
Agenda:

3. PRESENTED BY
RedisGears is a Serverless engine for multi-model and
cluster operations in Redis, supporting both event driven
as well as batch operations
● Agnostic from redis topology (stand alone, cluster, enterprise)
● Built in coordinator for cluster support
● Built in map/reduce operations
● Support full Python and C api
● Built as a Redis module
What is RedisGears?

4. PRESENTED BY
RedisGears Architecture
Execution
Management
Cluster
Management
Map/Reduce
C API
More language Integrations to come...
Core
Base API
User API

5. PRESENTED BY
RedisGears allow defining a pipe of operations
• Returning value from one operation pass to the operation that follows it in the pipe
• Last operation returning the result to the user
• First operation is called ‘reader’ - responsible for providing data
– Keys reader - read keys from Redis
– Stream reader - read streams from Redis
– Python reader - allow to user to write his own readers in python
Using RedisGears

6. PRESENTED BY
Using RedisGears - Mapping
Reader Mapper

7. PRESENTED BY
Using RedisGears - Filtering
Reader
Filter record
with 1 doc

8. PRESENTED BY
Simple Example 1 - Delete by Type
Delete all the hashes on redis:
# create pipe builder that return only keys names
builder = KeysOnlyGB()
# filter by hash type
builder.filter(lambda x: execute('type', x) == 'hash')
# delete the key
builder.map(lambda x: execute('del', x))
# start the execution# delete the key
builder.run()

9. PRESENTED BY
Using RedisGears - Aggregate
Reader
Count
Aggregator
1
1

10. PRESENTED BY
Simple Example 2 - Count by Type
# create pipe builder that return only keys names
builder = KeysOnlyGB()
# filter by hash type
builder.filter(lambda x: execute('type', x) == 'hash')
# count the number of records
builder.count()
# start the execution# delete the key
builder.run()
Count the number of hashes in redis:

11. PRESENTED BY
RedisGears Supports the following operations:
• Map
• FlatMap
• Filter
• Groupby + Reduce
• Aggregate
• Sort
• Limit
• ForEach
• Distinct
Supported Operations

12. PRESENTED BY
Demo 1 - Imdb
• Database contains hashes, each hash represent a movie
• Each movies has a list of comma separated genres
• Task : Count for each genre how many times it appears
# create the pipe builder
builder = KeysOnlyGB()
# get from each hash the genres field
builder.map(lambda x:execute('hget', x, 'genres'))
# filter those who do not have genres
builder.filter(lambda x: x is not None)
# split genres by comma
builder.flatmap(lambda x: x.split(','))
# count for each genre the number of times it appears
builder.countby()
# start the execution
builder.run()

13. PRESENTED BY
Gears has a streaming api to allow to trigger gears execution on events.
• Redis Stream events - Trigger an execution whenever a new data enters the steam
• Redis Keys events - Trigger an operation whenever a key touched
Stream Processing with RedisGears
Stream
Data
Stream

14. PRESENTED BY
Simple Example 3 - Set of all Keys
# create the builder
builder = GB()
# filter events on key:'all_keys'
builder.filter(lambda x: x['key'] != 'all_keys')
# add the keys to 'all_keys' set
builder.map(lambda x:execute('sadd', 'all_keys', x['key']))
# register the execution on key space notification
builder.register()
Maintaining a set containing all the keys on redis:

15. PRESENTED BY
Simple Example 4 - No Write Allow
# create a builder
builder = GB()
# filter by key prefix
builder.filter(lambda x: x['key'].startswith('not_allow'))
# delete the key
builder.map(lambda x:execute('del', x['key']))
# register execution on key space notification
builder.register()
Prevent keys that starts with ‘not_allow’ to be written to redis:

16. PRESENTED BY
Because gears is so flexible (it's actually running a python code) you can achieve
internal module integration with it:
• Read from hashes and index in RediSearch/RedisGraph
• Read from stream and pass to AI
• Read json and pass to TS
• .
• .
• Many Many more combinations...
RedisGears as a Multimodel Engine

17. PRESENTED BY
Demo 2

18. Thank you!

19. PRESENTED BY