Building out Reactive systems can be a lot of work. There’s a lot of infrastructure to set up and designing a system to be resilient, responsive, and elastic requires experience and time that not every team has. We built Inngest to be an open source, cloud-native system that enables anyone to build Reactive architectures. Designed to be pluggable with your favorite messaging service like Kafka, NATS or PubSub and your favorite container orchestration like Kubernetes, Nomad, or ECS. We’ll walk through how the system was designed, how you can deploy it yourself, and the plans to make it runnable on any cloud (and even your laptop!).

1. Dan Farrelly
Founder, Inngest
Building a Pluggable,
Cloud-native Event-driven
Serverless Architecture

2. ● What is Inngest?
● What did we set out to build and why?
● Our approach
● Principles of the system design
● Where we are today and where we’re headed
Outline
What we’ll cover

3. Dan Farrelly
● Founder, Inngest.com
● Former CTO @ Buffer.com
● Interests:
○ Event-driven systems
○ Home restoration
● dan@inngest.com
● @djfarrelly
Who is speaking to you?

4. About Inngest
● Inngest is an event-driven platform that makes it easy for developers
to build, test, then deploy serverless functions triggered by events —
without worrying about infrastructure, queues, or stateful services.
● “An event-driven architecture in a box”
Features: schema registry, logging, metrics, SDKs and tooling
● Inngest: A Platform + Open Source project
● Our Mission:
“To accelerate the adoption of event-based architecture.”
The backstory

5. Why this goal?

6. ● We think event-driven can be a beautifully simple way to build
● There currently is a large learning curve
● There is lack of awareness to broad developer community
● We want to introduce event-driven systems to developers in an
approachable way.
Why this goal?
The backstory

7. High level system design ideas
The backstory
● A fundamentally simple system design
○ Make it approachable
○ Provide an abstraction layer with batteries included
○ Enable reactive applications
● Inngest combines event streams with job scheduling in an easy to
use system
● It should be easy to build applications for
● There shouldn’t be a huge learning curve

8. ● Great Developer Experience (DX)
● A cloud-native solution (Either hosted or Self-hosted)
○ The cloud solution needs to be a pluggable architecture (Open Core)
Our focuses
The backstory

9. ● Developer SDK
○ Define functions, not consumers
○ SDK generates the configuration necessary
and guide the developer while writing code
● Core user needs
○ Consumers are defined as functions a in
standard format
○ Simple to send (publish) events (just JSON)
○ Versioning, throttling, idempotency,
concurrency should all be included
Simple to build
Developer Experience

10. ● Functions should be able to run in different runtime environments
○ HTTP
○ Docker runtimes (local, Kubernetes, etc.)
○ Serverless runtimes (e.g. Lambda)
● SDK can enable write-once-ship-anywhere
● Mix and match - depending on use-case
Runtime Agnostic
Developer Experience

11. ● Automatic event schema parsing and storage
● Type generation from event schemas
● Tight integration with SDK for easy use of Types
● Tooling to share types across teams/services/repos
Event Schema Registry & Types
Developer Experience
$ inngest types ts -o __types__.ts

12. ● Principle: Run on any cloud (or your own machine)
○ Cloud: It should support services that are common and can be spun up on different
cloud providers (e.g. Postgres, Redis, Kafka)
○ Local: It should be able to run in-memory for local dev or end-to-end testing
● Drivers are the way
Platform Agnostic
Pluggable Architecture

13. High-Level Architecture
Pluggable Architecture

14. ● Event API - Receives events,
handles publishing authn & authz
● Runner - Consumes & records
events, determines functions to
execute (as job).
● Executor - Executes function
steps from job queue on external
runtimes, manages job state.
● Core API - CRUD for system data
(functions, keys, events, logs).
Services
Pluggable Architecture

15. ● Event Stream - e.g. PubSub,
Kafka, NATS.
● DataStore - System data (keys,
functions configuration, history);
e.g. Postgres
● Job Queue - Functions to be
executed; e.g. Redis, SQS
● Job State - Job context, pauses,
retry & failure info; e.g Redis
Swappable Components (Drivers)
Pluggable Architecture

16. ● Function Runtimes - System drivers
can “enable” different runtime
targets.
○ Runtime: Docker / Target: local docker
client, Kubernetes, Nomad, ECS
○ Runtime: HTTP / Target: Your web server
○ Runtime: Serverless / Target: AWS
Lambda, etc.
○ Future? WASM?
Runtimes
Pluggable Architecture

17. ● Single-dependency development environment (inngest dev)
○ Often challenging for developers to emulate production environments on their
machines (e.g. docker compose up -f everything.yml)
○ Avoid costs and awkwardness of shared cloud resources
○ A UI for visual developers
● Enables E2E testing in CI/CD
In-memory Implementations
Pluggable Architecture

18. ● Decoupling points choosing to enable:
○ Customized deployments for special needs
○ Hosted-Self/hosted hybrid setups (BYO Kafka stream or Kubernetes cluster)
● Running multi-cloud or multi-region (e.g. for data sovereignty)
Cloud-Hybrid Configurations
Pluggable Architecture

19. ● Golang
○ You probably could have guessed this.
○ Leaning into packages like Go Cloud
Development Kit (Go CDK) [gocloud.dev]
to leverage fantastic community drivers
for various cloud services.
○ A single binary
● Learning through operating a cloud
service backed by the code
How we’re building it
Pluggable Architecture
Inngest Cloud

20. ● In-memory implementations of all drivers
● Drivers MVP (Postgres, Redis)
● “Cloud stacks” via Terraform
○ AWS Hosted stack (ECS, SQS, Elasticache)
● Seeking feedback
Where we’re at today
Pluggable Architecture
$ npx inngest-cli@latest dev

21. ● Focusing on DX (SDK, local-to-prod
workflow tooling, etc.)
● Step Functions
○ Delays
○ Event coordination
○ Conditional steps/workflows
● Additional SDKs (e.g. Python)
● Additional runtimes (e.g. AWS
Lambda, GCP Cloud Functions)
Roadmap
Pluggable Architecture

22. ● Inngest’s core will be open source
○ We want to build the system that allows developers to build reactive systems that is
powerful and approachable
○ Still more work to both extract and upstream code
● First step taken this year - way more to come!
High Level Vision & Outlook
The Future

23. ● inngest.com
● Project Repo → github.com/inngest/inngest
○ SDK (New) → github.com/inngest/inngest-js
○ AWS Cloud Stack → bit.ly/inngest-aws-stack
● Join the discussion + Provide feedback
○ inngest.com/discord
○ @inngest
● dan@inngest.com
Over to you
The Slide with Links

24. Thanks!
github.com/inngest/inngest
inngest.com | @inngest | dan@inngest.com | @djfarrelly