stackconf 2022: Scaling the Grail – Cloud-Native Computing on Encrypted Data using Carbyne Stack

CARBYNE STACK
Scaling the Grail – Cloud-Native Computing on Encrypted
Data using Carbyne Stack
Berlin, Germany
Image Source: Sikov – stock.adobe.com
Sven Trieflinger, Bosch Research

Sven Trieflinger (Bosch Research) | 2022-06-23
© Robert Bosch GmbH 2022. All rights reserved, also regarding any disposal, exploitation, reproduction, editing, distribution, as well as in the event of applications for industrial property rights.
About me
Carbyne Stack – Scaling the Grail
2
Sven Trieflinger
» Research Engineer and Senior Project Manager at Bosch Research Germany
» PhD in Distributed Systems
» 15+ years experience in the design and implementation of distributed systems
» Carbyne Stack maintainer (focusing on computing services)
https://www.linkedin.com/in/sventrieflinger https://scholar.google.com/citations?user=c7ERhR8AAAAJ

About the Bosch Group
3
Mobility
Solutions
Energy & Building
Technology
Consumer
Goods
Industrial
Technology
90%
of
associates
are
scientists
top
research
facilities
around the
globe
invention
reports
highly
specialized
employees
1,740 +11 1,855
resulted in
patents
74%
PhD
students
152
Bosch Research
subsidiaries and
regional
companies in
more than 60
countries
2.0
billion euros
EBIT
from
operations
Bosch associates
worldwide at
year-end
(approx.)
billion euros
sales
revenue
71.5 395,000 440
Bosch Group

Agenda
Motivation – Why CS, OS, @Bosch?
Carbyne Stack – Overview
Summary & Outlook
4
Spotlight – Serverless MPC

WHY CARBYNE STACK,
OPEN SOURCE, AT
BOSCH?

Privacy Preserving Computing Technologies
6
Data
Algorithms
PPCTs
Privacy-Preserving Computing
Technologies (PPCTs) seal computing
environments to enforce
» Confidentiality
(Data is protected from unauthorized extraction)
» Integrity
(Data is protected from unauthorized alteration)
» Control
(Data can be processed via authorized algorithms only)
Image Source: denisik11 – stock.adobe.com

PPCTs are on the Rise
7 Icons from the Noun Project (Joana Pereira, Vector Markets)
PPCTs are Compliance-, Trust-, and Collaboration-enabling Technologies
GDPR
CCPA
LGPD
PAPA
APPI
PDPB
Increasingly strict and diverse Data
Privacy Legislation Landscape
» GDPR-like regulations became or will become
effective in many important markets
(71% of the world’s countries have enacted data protection
and privacy laws + 9% with draft legislation1)
» Globally operating organizations will have to adopt
a cross-regulatory compliance strategy
» PPCTs will play an important role in reducing
friction
(e.g., invalidated Privacy Shield, SOTA requirement for
legitimate interest as legal basis)
1 Data Protection and Privacy Legislation Worldwide. Source: https://unctad.org/page/data-protection-and-privacy-legislation-worldwide
Privacy and Security are becoming
a success factor
„There is a clear connection between consumers’
perceptions of data privacy and security practices and
commercial success“
(Source: Protecting personal data in the consumer product industry,
Deloitte Limited)
„Organizations with more mature privacy practices
are getting higher business benefits than average
[…]“
(Source: Cisco 2021 Data Privacy Benchmark Study, Cisco
Corporation)
„Issues of trust, security and fear of losing competitive
advantage prevent organizations from sharing data and
collaborating“
(Source: Maximize collaboration through secure data sharing, Accenture
PLC)
Growing Momentum in the
Industry
“Gartner believes that by 2025, half of large
organizations will implement privacy-enhancing
computation for processing data in untrusted
environments and multiparty data analytics use cases.”
(Source: Gartner Identifies the Top Strategic Technology Trends for
2021, Gartner Inc.)
“The CC market is expected to grow at a CAGR of 90-
95% in the best-case scenario and 40-45% in the
worst-case scenario through 2026. [Creates a market
between USD 10-54 billion]“
(Source: Confidential Computing – The Next Frontier in Data
Security, Everest Group for the Linux Foundation)
55 members incl. Alibaba, Bosch, Salesforce
42 members incl. Arm, Google, Intel, Meta, Microsoft

The PP(C)T Toolbelt
8
Computing on Encrypted Data (COED)
Cryptographic Software-based Methods
E.g., Secure Multiparty Computation,
Homomorphic Encryption
Keeps data encrypted in use
Confidential Computing (CC)
Secure Hardware Enclaves, Trusted
Execution Environments
E.g., Intel SGX, AMD SEV
Creates a tightly controlled on-CPU
execution environment
Statistical Disclosure Control (SDC)
Data Obfuscation Mechanisms
E.g., Differential Privacy, substitution w/
synthetic data
Impedes re-identification attacks on
data that leaves the system
Cost Efficiency
Security
Usability
Maturity
Cost Efficiency
Security
Usability
Maturity
Cost Efficiency
Security
Usability
Maturity
CARBYNE STACK

What is Secure Multiparty Computation (MPC)?
9
PROS
CONS
» Data is always encrypted
(in transit, at rest, and in use)
» Provable security
(“Math over Matter”, Unbound Security)
» No trusted third-parties
» Computational overhead
» High communication costs between parties
» No open enterprise-grade SW stacks available
» No “Lift and Shift” today
Secure Multiparty Computation (MPC) is using cryptographic
protocols to distribute a computation across multiple parties such
that no party can see the other parties’ data.
MPC
Protocols
Alice’s Infrastructure Bob’s Infrastructure
Alice Bob
Icons made by Pixel perfect, iconixar, and Freepik from FlatIcon.

Applying PPCTs @ Bosch Research
10
» Protecting PII in AI pipelines
for Advanced Driver Assistance
Systems development to lower
legal risk
» Demonstrated integration with
SOTA tools and frameworks
(Tensorflow, KubeFlow, Kserve)
» Lots of external visibility /
traction
(OC3, Intel Vision, Nvidia GTC, MS Build,
Startup Autobahn)
» Protecting model IP in cloud-
based multi-party simulations
» Trend toward moving simulation
payloads to the cloud increases
attack surface
» Goal is to be faster than real-
time
COLLABORATION
Simulation
AI Pipelines
Confidential Computing Computing on Encrypted Data
Bosch SPECS
COMPLIANCE
» Perform analytics and
benchmarking on Human
Resources data from the whole
Bosch Group
» Includes applying statistics and
machine learning on pooled
data from 400+ subsidiary and
regional companies in some 60
countries
» Improve battery health prediction
models by using OEM field
data
» Combine locally learned deltas
using Federated Learning with
COED-based Secure
Aggregation
Battery-in-the-Cloud
People Analytics
COMPLIANCE COLLABORATION
We look at a broad range of use cases for enterprise-grade PPCTs with our partners.
Image Source: 1STunningART – stock.adobe.com

What does it take to be called Enterprise-grade MPC?
11
Scalability
Capacity of the system
grows proportionally with
the added resources
Integration
Works with existing IT
infrastructure to protect
existing investments
Versatility
Deploys against various
use-cases to reduce IT
complexity
Resiliency
Tolerates failure to
minimize downtime
Elasticity
Can grow and shrink
easily with fluctuating
demand
Simplicity
Reduced to the core to
keep operations
manageable
Observability
Can be inspected to
diagnose and fix issues
Image Sources: Premium Icons, arybickii, Raul Almu, Mykyta, Artco, Инна Харламова, Fourdoty, SurfupVector, Yurii – stock.adobe.com
Security
Fits into a holistic
approach across
applications, infrastructure,
and processes
Efficiency
Gives you the most bang
for the buck (under the
constraints of the
technology)

Cloud-Native Technology to the Rescue
12
Cloud native technologies empower
organizations to build and run
scalable applications in modern,
dynamic environments such as
public, private, and hybrid clouds.
[…] These techniques enable loosely
coupled systems that are resilient,
manageable, and observable.
Cloud Native Computing Foundation
* Image Source: https://www.cncf.io/reports/cncf-annual-report-2020/, Cloud Native Computing Foundation. Carbyne Stack is not affiliated with or otherwise sponsored by the Linux Foundation.
*
OperatorFramework 5,700
Cloud-native technology addresses a major part of the enterprise needs.
Security is there! (e.g., Istio, Vault, OPA, etc.)
Cloud Native is pervasive in enterprise IT,
facilitates integration and reuse

REDUCE COST
SHARE RISK
Share the burden
of making this huge
effort with like-
minded companies,
institutions, and
individuals
IMPROVE
QUALITY
Tap the potential of
the community to
solve problems
together better
INCREASE
TRUST
Boost trust through
transparency and
materializing
provable security of
COEDs
BOOST
BOSCH BRAND
Reinforce Bosch
brand and
reputation and
attract talent
Why Open Source?
13
Our plan is to establish neutral ground by eventually
transferring Carbyne Stack into a foundation
“If you want to go fast, go alone. If you want to go far, go together.” (African proverb)
  
The Linux Foundation and the Cloud Native Computing Foundation logos are trademarks of the Linux Foundation. The Eclipse Foundation logo is a trademark of the Eclipse Foundation, Inc. The Appache Software Foundation logo is a trademark of the Apache Software Foundation in the
United States and/or other countries.

Carbyne Stack – Lifting MPC into the Cloud
15
Open-Source Cloud Native MPC
(Enterprise-grade always encrypted data processing at scale)
MPC Frameworks
(Always encrypted data)
Cloud Technologies
(Enterprise-grade processing at scale)
Carbyne Stack allows for Enterprise-grade computing on encrypted data at scale
Carbyne Stack combines 2 of 12 strategic trends identified by
Gartner that "will enable CEOs to deliver growth, digitalization and
efficiency" in 2022ff.
*
* Source: Gartner Top Strategic Technology Trends for 2022 available at https://www.gartner.com/en/information-technology/insights/top-technology-trends. Carbyne Stack frame overlay added.

System Model – The CS Design Choices
16
Client
Library
Virtual
Cloud (VC)
API
API
API
Virtual Cloud
Provider (VCP) #1
Virtual Cloud
Provider #2
Virtual Cloud
Provider #n
Topology
 2 servers, any
number of clients
» Can be instantiated as P2P or offloading system
» Operating a VCP requires expertise
» Supports resource constraint devices and dynamic (client) membership
Trust Model
Active Security,
Malicious Majority
» Passive security model turned out to be not very convincing when
talking to potential users
» Trust in yourself (P2P) or in a single VCP (offloading) required only
Execution Model
Online/Offline /
Preprocessing
» Can adapt easily to time-dependent load profiles as off- and
online phase tasks can be scheduled independently
» Preprocessing can be done with ephemeral (= cheap) cloud
resources
…
…
Icons made by iconixar and Freepik from FlatIcon.

The Carbyne Stack High-Level Architecture
17
Istio
Knative
Kubernetes
Cloud Native Foundation
(most important ones shown only)
Carbyne Stack Foundation Services Generic MPC storage and
compute services
Federated Learning
Data Analytics
Applications dealing
with sensitive data
Human Resources Healthcare Connected Industry Automotive
Higher-level services
Adds versatility /
reuse across use
cases
» Orchestrate containers across up
to 5000 hosts
(declaratively, efficiently, transparent)
» Self-healing
(via health-checks, automatic placement,
restarts, replication and scaling)
» Secure N/S & E/W communication
(TLS encryption, identity-based auth and authz)
» Load balancing
(for HTTP, gRPC, WebSocket, and TCP traffic)
» Observability
(metrics, logs, and traces for all traffic)
» Focus on code
(From container to URL with minimal
effort)
» Autoscaling
(Scale application containers up and
down based on traffic patterns)

The Carbyne Stack Foundation Services
18
Compute
Storage
Amphora
Secret Store
» Stores additively secret-shared client data
» Secrets can be tagged and filtered by them
when retrieved
» Implements secure up-/download protocols
required in client/server MPC model
Castor
Correlated Randomness Store
» Stores correlated randomness
used to accelerate the online phase
Carbyne
Stack
Clients
REST/HTTPS
CLI
Ephemeral
Serverless MPC
» Executes MPC programs using MP-SPDZ
» Scale easily up and down
» I/O via Amphora
» Correlated Randomness is fetched from Castor
Klyshko
Correlated Randomness Generation
» Generates correlated randomness across VCPs
» Klyshko Integration Interface (KII) allows for easy
integration of offline protocols
Carbyne Stack Foundation Services
Incubating
Offline
Online
Containerized
Microservices
spring®
Spring is a trademark of Pivotal Software, Inc. in the U.S. and other countries. Golang logo by Renee French is licensed under CC BY 3.0.

Carbyne Stack – Working Principle
19
The VCPs start generating input-independent
correlated randomness using heavyweight
cryptographic primitives (e.g., homomorphic
encryption, oblivious transfer) that support secure
and fast I/O operations and computations in the
online phase.
2
3 Clients register and authorize with each of the VCPs
to invoke services, i.e., upload / download data and
trigger computations.
1 Multiple Virtual Cloud Providers (VCP) providers
establish a Virtual Cloud (VC) by binding to
each other (key establishment and exchange).
4 The VCPs perform calculations on top of
previously uploaded data using lightweight
cryptographic primitives.
Amphora
Castor
Ephemeral
Store results
Read Inputs
Store Tuples
Fetch
Tuples
VCP #1
VC
Klyshko
VCP #2
API API
1
2
3
4
Icons made by iconixar and Freepik from FlatIcon.

Billionaires Problem - The Carbyne Stack Way
20
# Create secrets representing Jeff’s and Elon’s net worth
# (note that we work with billion USD here)
export JEFFS_NET_WORTH_ID=$(cs amphora create-secret 177 -t billionaire=Jeff)
export ELONS_NET_WORTH_ID=$(cs amphora create-secret 151 -t billionaire=Elon)
CLI Invocation
<service> <command> <value> Tag
1
# Let’s see what has been created
cs amphora get-secrets
ab160f93-3b7e-468f-b687-f9c46fb535f3
billionaire -> Jeff, creation-date -> 1630660117946
ef3e867f-9233-46fb-9cde-7a09c99bc32f
billionaire -> Elon, creation-date -> 1630660125951
Amphora secret identifier
2
# Write the MPC program
cat << 'EOF' > billionaires.mpc
# Prologue to read in the inputs
port=regint(10000)
listen(port)
socket_id = regint()
acceptclientconnection(socket_id, port)
v = sint.read_from_socket(socket_id, 2)
# The logic
first_billionaires_net_worth = v[0]
second_billionaires_net_worth= v[1]
result = first_billionaires_net_worth <
second_billionaires_net_worth
# Epilogue to return the outputs
resp = Array(1, sint)
resp[0] = result
sint.write_to_socket(socket_id, resp)
EOF
3
# Trigger an execution of our program
export RESULT_ID=$(cat billionaires.mpc
| cs ephemeral execute
-i $JEFFS_NET_WORTH_ID
-i $ELONS_NET_WORTH_ID
ephemeral-generic.default | tail -n +2 | sed 's/[][]//g')
IDs of Amphora secrets used as input
Knative application to execute
4
# Fetch the result of the computation
cs amphora get-secret $RESULT_ID
[0]
creation-date -> 1630661192626
gameID -> 7899b23c-4509-4ff8-a9ae-d9b59fa77fea
Our result recombined from secret shares
5

Spotlight – Serverless MPC (simplified)
22
Istio
Activator Autoscaler
Knative
Queue
Proxy
Function
Container
Application
Knative HTTP
Virtual Service
1
1 Knative application deployment using Knative Service
custom resource and Function Container with your MPC
workload. Autoscaler ensures that minimal number of
application instances are created.
Shared component Per application container component Scaled to Zero Network Path Active Network Path
Client(s)
Launch Pod(s)

Spotlight – Serverless MPC (cont’d)
23
Istio
Network
Controller
Ephemeral
Knative
Queue
Proxy
Function
Container
Application
MPC/TCP
Virtual Service
Knative HTTP
Virtual Service
Creates
2
2 Ephemeral Function Container triggers Istio MPC/TCP
Virtual Service creation (by means of Network custom
resource) for routing inter-VCP MPC engine-to-engine
traffic. We use a patched version of Knative that allows for
exposing multiple ports.
Shared component Per application container component Active Network Path
…
Other VCPs
Network
Scaled to Zero Network Path

24
Istio
Network
Controller
Ephemeral
Knative
Queue
Proxy
Function
Container
Application
MPC/TCP
Virtual Service
Knative HTTP
Virtual Service
Shared component Per application container component Active Network Path
…
Other VCPs
3 Function Container registers local MPC/TCP
Virtual Service endpoint with Discovery
Service which exchanges endpoint information with
other VCPs in VC.
Discovery HTTP
Virtual Service
Coordination
3
…
Other VCPs
Discovery
Service
Local VCP
Endpoint
Register

25
Istio
Network
Controller
Ephemeral
Knative
Queue
Proxy
Function
Container
Application
MPC/TCP
Virtual Service
Knative HTTP
Virtual Service
Shared component Per application container component
HTTP
Request
/
Response
Active Network Path
…
Other VCPs
Discovery HTTP
Virtual Service
Coordination
…
Other VCPs
Discovery
Service
4 Client invokes MPC function via Knative HTTP Virtual
Service specifying which secrets from Amphora to use
as inputs.
Client(s)
4

26
Istio
Network
Controller
Ephemeral
Knative
Queue
Proxy
Function
Container
Application
MPC/TCP
Virtual Service
Knative HTTP
Virtual Service
Ping
Protocol
HTTP
Request
/
Response
Active Network Path
…
Other VCPs
Discovery HTTP
Virtual Service
Coordination
…
Other VCPs
Discovery
Service
Client(s)
5 Function Container validates connectivity and
launches MPC engine with consolidated endpoint
information.
VCP
Endpoints
Fetch
5

27
Istio
Network
Controller
Ephemeral
Knative
Queue
Proxy
Function
Container
Application
MPC/TCP
Virtual Service
Knative HTTP
Virtual Service
MPC
Protocol
Messages
HTTP
Request
/
Response
Active Network Path
…
Other VCPs
Discovery HTTP
Virtual Service
Coordination
…
Other VCPs
Discovery
Service
Client(s)
6 Function Container executes MPC program, reads /
writes I/O secrets using Amphora, and consumes
correlated randomness from Castor.
Carbyne Stack
Castor
Amphora
Correlated
Randomness
I/O Secrets
6
Fetch / Store
Consume

Application
Application
28
Istio
Network
Controller
Discovery
Service
Ephemeral
Knative
Queue
Proxy
Function
Container Carbyne Stack
Castor
Amphora
MPC/TCP
Virtual Service
Knative HTTP
Virtual Service
Metrics
Monitor
Discovery HTTP
Virtual Service
Coordination
7 Based on metrics from Queue Proxy Knative Autoscaler
adapts number of application pods (potentially to zero using
Activator as proxy).
7
Shared component Per application container component Scaled to Zero Network Path Active Network Path
… …
Other VCPs Other VCPs
Client(s)
Launch Pod(s)

What’s next? Less cost, more value!
30
Leverage new approaches for generating Correlated
Randomness to reduce communication
bandwidth (e.g., PCGs) and use hardware
acceleration to improve computational efficiency
Reduce cost of Offline
Phase
Provide an MPC-based DBMS and
Federated Learning system on top
of Carbyne Stack
Provide value using higher-
level services?
» Reduce overall (egress bandwidth-
dominated) cost of MPC across different
public clouds* by up to 3 (!) orders of
magnitude
» Query your secret-shared database using SQL
» Protect sensitive information AI use cases by
using MPC-based secure aggregation
Cost
Efficiency
Carbyne Stack Foundation Services
Federated Learning
Data Analytics
Versatility
* Only Ingress and intra availability zone traffic is typically for free on public clouds. Other traffic costs in the order of USD 10/TB.

What’s next? More cloud-nativeness ahead!
31
+
Flexible OIDC authentication and fine-
grained policy-based authorization in the
Carbyne Stack microservices, clients and CLI
Who is allowed to do what
in a Virtual Cloud
Ensure observability of Carbyne
Stack microservices via dimensional
metrics and alerts
What goes on in my Virtual
Cloud Provider instance?
Observability
Security
» User Bob may fetch all secrets with tag t
» Output objects to execution of function f may
be fetched by every user who provided input
to that execution
» Function f has been invoked 200 times in the
last week
» GFp multiplication triples are currently
consumed at a rate of 72 billion triples / h
OPA™, Dex, and Prometheus® are trademarks or registered trademarks of the Linux Foundation.

What’s next? – More cloud-nativeness ahead (cont’d)!
32
Operator that can be used to deploy, configure, and
operate a Carbyne Stack virtual cloud provider and to
establish virtual clouds by interconnecting multiple of them
A VC(P) is a complex
thingy? How to operate it?
Gosh! How many
prerequisites?
Terraform and Helm Infrastructure-as-Code (IaC) to
deploy the “substrate” for launching a Carbyne
Stack virtual cloud provider on major public clouds
+
Simplicity
» git clone cs/ci && cd ci &&
terraform apply … K8s cluster, Docker
registry, subnets, Application Gateway,
domain names; all done!
» cat <<EOF | kubectl apply -f –
apiVersion: carbynestack.io/v1
kind: VirtualCloud
metadata:
name: alice-bob-vc
spec:
partners: [ https://cs.alice.io, https://vcp.bob.dev ]
EOF
Helm® and Operator Framework are trademarks or registered trademarks of the Linux Foundation. Terraform® is a registered trademark of Hashicorp.

Summary
34
» PPCTs are on the rise with a huge business and risk mitigation potential.
» Secure Multiparty Computation (MPC) is a PPCT with strong security guarantees that can be used to
keep data end-to-end encrypted.
» Carbyne Stack lifts MPC into the cloud enabling deployments of enterprise-grade MPC at scale by
leveraging cloud-native technology.
» Our mission is to establish Carbyne Stack as the de-facto standard for enterprise-grade MPC
https://github.com/carbynestack
https://carbynestack.io https://discord.gg/8sxNPCSBDk
JOIN US IN LIFTING COMPUTING ON ENCRYPTED DATA INTO THE CLOUD!

THANK
YOU
Visit us on https://carbnyestack.io or get involved at https://github.com/carbynestack
Data Ethics is Bosch Values in a Connected World
“Respect for privacy and the right to control one’s own data are
becoming key parameters to gain a competitive edge in today’s
business world. Companies […] which view data ethics as a social
responsibility, giving it the same importance as environmental
awareness and respect for human rights, are tomorrow’s winners.
Digital trust is paramount to digital growth and prosperity.”
Hasselbalch, Gry; Tranberg, Pernille, (2016):
Data Ethics. The New Competitive Advantage, Publishare, Copenhagen
Part of the work presented in this talk has been supported by funding
from the Federal Ministry of Education and Research under Grant
Agreement No. 16KIS1441 via the CRYPTECS publicly funded project.

stackconf 2022: Scaling the Grail – Cloud-Native Computing on Encrypted Data using Carbyne Stack

Recommended

Recommended

More Related Content

What's hot

What's hot (10)

Similar to stackconf 2022: Scaling the Grail – Cloud-Native Computing on Encrypted Data using Carbyne Stack

Similar to stackconf 2022: Scaling the Grail – Cloud-Native Computing on Encrypted Data using Carbyne Stack (20)

Recently uploaded

Recently uploaded (20)

stackconf 2022: Scaling the Grail – Cloud-Native Computing on Encrypted Data using Carbyne Stack