VPCs, Metrics Framework, Back pressure : MuleSoft Virtual Muleys Meetups

All contents © MuleSoft, LLC
19 August 2020
Online MuleSoft Meetups

Safe Harbor Statement
2
The information in this presentation is confidential and proprietary to MuleSoft and may not be disclosed without the
permission of MuleSoft. This presentation is not subject to your license agreement or any other service or subscription
agreement with MuleSoft.
MuleSoft has no obligation to pursue any course of business outlined in this document or any related presentation, or to
develop or release any functionality mentioned therein. This document, or any related presentation and MuleSoft's strategy
and possible future developments, products and or platforms directions and functionality are all subject to change and may
be changed by MuleSoft at any time for any reason without notice.
The information on this document is not a commitment, promise or legal obligation to deliver any material, code or
functionality. This document is provided without a warranty of any kind, either express or implied, including but not limited
to, the implied warranties of merchantability, fitness for a particular purpose, or non-infringement.
This document is for informational purposes and may not be incorporated into a contract. MuleSoft assumes no
responsibility for errors or omissions in this document, except if such damages were caused by MuleSoft intentionally or
grossly negligent.
All forward-looking statements are subject to various risks and uncertainties that could cause actual results to differ
materially from expectations. Readers are cautioned not to place undue reliance on these forward-looking statements,
which speak only as of their dates, and they should not be relied upon in making purchasing decisions.

Training Meetups
3
1. New concept introduced by the Training Team
2. Focused on:
a. Hands on labs
b. Technical deep dives
c. Attendees need laptops to follow through the session
3. Expected on Wednesday, 23 September⋅5:00 – 7:00pm, new to come in:
https://meetups.mulesoft.com/

Agenda
4
1. All you need to know before creating your ﬁrst VPC
2. The Metrics Framework Unwrapped
3. A performance case: Back pressure (Short session)
4. Wrap up

All You Need to Know Before
Creating Your VPC
MuleSoft Meetup
Gonzalo Marcos – Senior Technical Architect
August 2020

Agenda
1. Introductions
2. What is a Mule VPC?
3. Do I need a VPC?
4. How Many VPCs do I need?
5. How to choose your CIDR block
6. Summary
6

Creating a VPC is simple…
7
REGION
CIDR BLOCK
ENVIRONMENTS
BUSINESS GROUPS

What is a Mule VPC?

Mulesoft Shared Worker Cloud
9

What is a VPC?
10
A Virtual Private Cloud (VPC) dedicated space in CloudHub speciﬁcally for an organization to
deploy their applications
Allows you to have a dedicated range of IP addresses speciﬁcally to your Mulesoft
Organization within an AWS Region

Anatomy of a VPC
11

What is a VPC?
12

What is a VPC?
13

Do I need a VPC?

When do we need a VPC?
15
• Connectivity to your Data Center
• Isolation
• Custom Domain for your Apps
• Custom Certiﬁcates
• Mutual TLS

Connectivity to your Data Center
16
You can connect your AnyPoint VPC to
your private network using the following
methods:
• VPN - IPsec Tunnel
• VPC Peering
• CloudHub Direct Connect

Isolation
20
Traffic Segregation
Security

Isolation – Firewall Rules
21

Isolation – Traffic Segmentation
22
- It’s a good practice to isolate all the traffic between
production and non production environments
- Differentiating traffic between business units in the same
organization.

Need for a Dedicated Load Balancer
24
• Use of Custom Domain names
(Vanity Domains)
• Use of Custom Certificates
• Mutual TLS
• Traffic Segregation 🡪 Limit internal
and External traffic

How many VPCs do I need?

Where is your backend?
26
• Your VPC should be as close as possible to your backend systems
• A Mule VPC is created in an AWS region 🡪 is your backend located
in only one region or is it geographically distributed?
• Two options for distributed backend systems:
– One VPC + connections (VPN/VPC Peering/AWS DX) to your multiple data
centers
– One VPC per region + connections

Isolation – Business Groups
27
• Business Groups are a great
solution to separate and
control access to AnyPoint
resources.
• A VPC is a resource that can
be created at the master
org and also at a Business
Group level
• A VPC is only shared
vertically down, not up or
across.

Isolation – Business Groups
28
• If you don't have any requirement of isolation between the BGs of
your org the recommendation is to create your VPC at the master
Org

Isolation - Environments
29
• Environments get associated to a VPC.
• We can have multiple environments in the same VPC but each
environment can be associated to only one VPC.
• One environment cannot be part of two VPCs.
• Every app gets deployed to an environment and that environment
is associated to a VPC.

Isolation - Environments
30
• This provides us with a
mechanism of isolation for
our apps.
• The recommendation is to
have a minimum of two
VPCs. One for production
environments and one for
non production
environments (Dev, QA,
Staging, Test).
• In some other cases a VPC
per environment is required.

How to Choose Your CIDR
Block

The basics – What’s an IP address?
• It’s a numerical representation of a location within a network
• That number is a binary number and it’s 32 bits long
• For better readability we divide those 32 bits in 4 blocks and use a decimal value for each
block
• Each decimal number ranges from 0-255
Smallest IP Address: 0.0.0.0
Binary: 0000 0000 - 0000 0000 - 0000 0000 - 0000 0000
Largest IP Address: 255.255.255.255
Binary: 1111 1111 - 1111 1111 - 1111 1111 - 1111 1111

What is CIDR?
33
• Classless Inter-Domain Routing notation
• The notation identifies the network and the host in that network
• IP addresses are described as consisting of two groups of bits in the address: the most
significant bits are the network prefix, which identifies a whole network or subnet, and
the least significant set forms the host identifier, which specifies a particular interface of a
host on that network.
We use the CIDR notation to specify a range of IP addresses
• In terms of VPCs it determines how many IP addresses are available to use
• Example: 192.168.0.0/24

192.168.0.0/24 example
34
0.0.0.0 = 0000 0000 - 0000 0000 - 0000 0000 - 0000 0000
…
…
192.168.0.0 = 1100 0000 - 1010 1000 - 0000 0000 - 0000 0000
…
192.168.0.255 = 1100 0000 - 1010 1000 - 0000 0000 - 1111 1111
…
…
255.255.255.255 = 1111 1111 - 1111 1111 - 1111 1111 - 1111 1111
24 Bits we care about
8 Bits we DON’T
care about
Range of IP
Addresses
allocated to this
VPC
(256 addresses)

192.168.0.0/16 example
35
0.0.0.0 = 0000 0000 - 0000 0000 - 0000 0000 - 0000 0000
…
…
192.168.0.0 = 1100 0000 - 1010 1000 - 0000 0000 - 0000 0000
…
192.168.255.255 = 1100 0000 - 1010 1000 - 1111 1111 - 1111 1111
…
…
255.255.255.255 = 1111 1111 - 1111 1111 - 1111 1111 - 1111 1111
16 Bits we care about
16 Bits we DON’T
care about
Range of IP
Addresses
allocated to
this VPC
(65,536
addresses)

Choosing your CIDR block
36
CIDR Block Size
Number of Bits We
Care About
Number of Bits We
DON’T Care About
Exponential Notation Number of Addresses
/24 24 8 28
256
/23 23 9 29
512
/22 22 10 210
1,024
/21 21 11 211
2,048
/20 20 12 212
4,096
/19 19 13 213
8,192
/18 18 14 214
16,384
/17 17 15 215
32,768
/16 16 16 216
65,536
Note: MuleSoft only allows between /16 and /24 CIDR block sizes

Why does the CIDR block size matter?
37
• The size is very important because having a short block might cause your deployment to
run out of IPs in a short-medium term and then not able to deploy apps in the VPC.
• The moment we connect this VPC to our Data Center, using a VPN or a VPC peering, that
CIDR block will become part of our internal network and it will consume private IP
addresses from your internal addressing space 🡪 it's important not to oversize your VPC,
as it will take out more IPs than necessary from your internal network.

How do we estimate the number of IPs we need?
38
Remember: The number of IP addresses in your CIDR block isn’t the same as the number of
applications you can deploy
You need to consider:
• Number of Workers
• Horizontal Scaling and High Availability
• Fault Tolerance (Region of the VPC)
• Zero Down-Time
• Number of Environments

CIDR Block - Examples
39
EXAMPLE 1: 172.16.1.0/24 🡪 256 IP addresses
• IPs 172.16.1.0 and 172.16.1.255 are reserved --> 254 IPs
• FT required 🡪 Region of deployment: Frankfurt, 3 AZs --> 254/3 = 84
• 1 Environments: Production 🡪 84
• Zero Downtime 84/2 = 42
• Total: 42 apps

40
EXAMPLE 2: What size do we need for the non production (Dev, QA and Test) VPC for the
previous example?
- 42 apps to deploy
- HA required (x2)
- 3 environments
- Zero Downtime (x2)
- Total: 42 x 2 x 3 x2 = 504 IPs needed 🡪 /23

41
Example 3: 1000 apps to deploy in a Production environment
• 1000 apps
• HA required for all apps (x2)
• 1 environment (Prod)
• Zero Downtime (x2)
• Total: 1000 x 2 x 2 = 4,000 IPs 🡪 /20

Summary

Summary
43
REGION
CIDR BLOCK
ENVIRONMENTS
BUSINESS GROUPS

Questions?

Metrics Framework
MuleSoft Meetup
David Cisneros - Technical Architect Director
August 2020

grossly negligent.

Contents
1. Introduction and Context
2. What is the “Metrics Framework”?
3. Some deﬁnitions
4. Features
5. SDLC Metrics Extension
6. How to Extend the Framework

Introduction and Context

The Metrics Framework is an extensible Mule
application intended to collect, aggregate and
load metrics into diﬀerent visualization systems,
providing out of the box integrations and
visualization options

Definitions
● KPI: Quantifiable value that reflects a strategic business goal and how
successful is the business achieving that goal
● Metric: Quantifiable value to track an aspect of the business activity and
measure the success/failure of that activity
● Relationship of KPIs and metrics: Every single KPI is composed by one or
multiple metrics but not all metrics are KPIs
Basic Metrics:
- # of APIs
- # of vCores
allocated
- # of Users
Composed Metrics:
- # of APIs by type and BG
- vCores used vs available
(%)
- # of Active users (last 30
days)
KPIs:
- API adoption and
projection
- Platform adoption
growth

By collecting metrics over time, and drawing them
in a timeline chart, you will see the behavior of a
speciﬁc measured activity, deriving this as a KPI
that make sense for your organization

From a basic metric to a KPI
KPIs:
Platform growth - API
publication growth rate - 65%
/ quarter
Platform adoption - API
fragment reuse - 10% /
quarter

Example mapping KPI categories with metrics and
personas
53
KPI Category Metrics available Interested Personas
Platform demand /
growth
- cores/vcores allocated vs available
- API Specs published vs API
Implementations using API Specs
- API calls per environment / BG
- Active/Inactive users
- Program Leadership / Sponsors
- Platform Architect
Organization
enablement
- APIs with / without policies
- APIs with / without security policies
- API instances active/inactive
- Applications active/inactive
- Security team
- Enablement teams
- Architects
API consumption
and reuse
- API Instances with / without contracts
- API Clients
- API Fragments reuse
- API Specs reuse
- API Policies reuse
- API Specs managed in API Manager - reuse
- Lead Developers
- Architects

Metrics Framework - Features
● Compact Mule application: 1 single Mule application, easy to configure and install
● Metrics provided:
○ Anypoint Platform Metrics provided: More than 120 metrics calculated from Anypoint Platform
capabilities: Exchange, API Manager, Access Management, Runtime Manager
○ SDLC Metrics: 15 metrics collected from common tools like JIRA, Confluence, etc
○ Platform Benefits: Require manual input
● Modes:
○ Poller: To run the application as an automated process to collect and load metrics
○ API: With multiple operations to get and load metrics on demand
● Visualization options: CSV, JSON, Plain log, Splunk, ELK, Anypoint Monitoring custom metrics
(requires Titanium)
○ Splunk: 14 Dashboards with 150+ charts available to build your own dashboard
○ ELK: Basic set of Kibana dashboards

● API Response examples (JSON, CSV)

● Splunk Dashboards - grouped by Anypoint Platform features

● Splunk Dashboards - examples

Metrics Framework - Considerations
● Anypoint Platform APIs: Rate Limiting
○ For large accounts (>25 BGs with multiple environments), Anypoint Platform parallel API calls
might fail with status code 429 due to rate-limiting policies
■ Limit the BGs/environments using available features like “ignore lists” or “collectors”
■ Use different schedulers with different cron-expressions for different BGs
■ Include a retry strategy in specific collectors
● Anypoint Platform user permissions
○ Some API calls require Organization Admin access
○ Segregation of permissions and roles is possible but losing some metrics (e.g API Analytics - trx last 30 days)
● Poller frequency: poll once a day or once a week, higher frequencies (e.g minutes) don’t make any
sense. The metrics framework is not intended to be a monitoring tool
● For large scale accounts - large implementations
○ A Database/Data Warehouse loader might be needed to decouple responsibilities and to lighten
API Calls and data ingest

SDLC Metrics collectors and extension

The SDLC Metrics is an extension to the Metrics
framework intended to collect, aggregate and load
metrics from several 3rd party applications

SDLC Metrics Framework - Features
● Applications Integrated (So far!):
○ Jenkins
○ Jira
○ Conﬂuence
○ Bitbucket
○ Splunk
● Selectively pick applications using a toggle switch
● Supports both Poller and API modes
● Ability to fetch raw data and enhance aggregation logic

SDLC Metrics Framework - Features

How can I extend the framework?

4 Steps to add an external system
1. Create api-call-<application>.xml under sources.external.systems.apis (similar to
system api)
2. Create collector-<application>.xml under collectors.external.systems (similar to
process api)
3. Add new route to the new collector in aggregator-sdlc-metrics.xml and update
response dataweave mapping
4. Add new properties to
○ properties/metrics-framework-{env}.yaml
○ properties.secure/metrics-framework-{env}.yaml

Contact MuleSoft Professional Services /
Customer Success to get support on the analysis,
deﬁnition and implementation of metrics and KPIs
using the metrics-framework

Metrics Framework
● Source code and installation manual: https://github.com/mulesoft-catalyst/metrics-framework-m4

Case Study:
A Back-Pressure Mechanism
Journey
Sebastian Gonzalez- Integration Support Engineer
August 2020

69
grossly negligent.

Agenda
70
● Customer background
● The Application Issue
○ Back-Pressure Mechanism
○ Controlling Concurrency
● Architecture Layout
● Reproducing the Issue
● Test Analysis
● Final Conclusions

A little context about the Customer
71
● Multinational and leading company in the energy sector, working from oil processing up
to natural gas
● With more than 2000 employees and ten years of being in the market
● Using MuleSoft heavily across the company

The Application Issue
73
Performance of an application with multiple Anypoint MQ subscribe components
Two Main Concerns:
❑ Why no Back-pressure mechanism was showing up?
❑ Why their application wasn’t auto-regulating the number of messages that can process? –
controlling concurrency

Controlling Concurrency
74
maxConcurrency defines the level of parallelism within the flow
Customer refused using this attribute even though it brought a significant performance
improvement to the app

Back-Pressure Mechanism
75
Under heavy load, Mule does not have resources available to process a specific event. Here is
when the Back-pressure can occur.
This might occur because all threads are busy and cannot perform the handoff of the newly
arrived event, or because the current flow's maxConcurrency value has been exceeded.

Architecture Layout

Application’s Architecture Layout
77

Reproducing the issue
Reproducing the issue

Reproducing The Issue
79
❑ Call with customer to ensure same understanding
❑ As always, ﬁrst attempts were a failure - lack of dependencies
❑ Shipping a huge messaging load brought up all issues

Test Analysis
HTTP Logger Appender

Test Analysis
81
HTTP Logger Appender

Test Analysis
82

Test Analysis
83

Test Analysis
84

Test Analysis
85

Test Analysis
Removing HTTP Logger Appender

Test Analysis
87
NO HTTP LOG APPENDER

Test Analysis
88

Test Analysis
89

Test Analysis
90

Test Analysis
91

Final Conclusions

Final Conclusions
93
Key takeaways:
❑ HTTP Appender could impact performance after 1st
burst hour (vCore)
❑ As with any logging capability, do consider HTTP Appender when you are doing your Performance
Tests (e.g. Soak/Endurance Testing) where using the native Appenders may improve performance
(E.g. Splunk HTTP Appender)
❑ Back-pressure mechanism worked always as expected
❑ Back-pressure can’t prevent app failure once credits are low

Thank You!

VPCs, Metrics Framework, Back pressure : MuleSoft Virtual Muleys Meetups

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to VPCs, Metrics Framework, Back pressure : MuleSoft Virtual Muleys Meetups

Similar to VPCs, Metrics Framework, Back pressure : MuleSoft Virtual Muleys Meetups (20)

More from Angel Alberici

More from Angel Alberici (12)

Recently uploaded

Recently uploaded (20)

VPCs, Metrics Framework, Back pressure : MuleSoft Virtual Muleys Meetups