Talk from 2018 UXPA Conference. Given Tuesday, June 26, 2018

1. D E S I G N + D E V O P S : : C O N V E Y U X : : 2 0 1 8 1
Design + DevOps
what we’ve learned from our dev friends
U X D E S I G N
Esteban Pérez-Hemminger, Design Lead
Troy Bjerke, Design Researcher
@ IBM Design, Austin TX

2. who dis?
E S T E B A N P É R E Z - H E M M I N G E R
Design Lead
IBM Studios, Austin TX
@certifyDesign
T R O Y B J E R K E
Design Researcher
IBM Studios, Austin TX
@TheBeerKey
D E S I G N + D E V O P S : : C O N V E Y U X : : 2 0 1 8 2

3. D E S I G N + D E V O P S : : C O N V E Y U X : : 2 0 1 8 3
today
Send us questions
and come talk to us.
@certifyDesign
@TheBeerKey
1 why + what
2 case study
3 parallels
4 wrap

4. D E S I G N + D E V O P S : : C O N V E Y U X : : 2 0 1 8 4
Why are we here

5. D E S I G N + D E V O P S : : C O N V E Y U X : : 2 0 1 8 5
the why
Share how our design team at IBM followed
radical collaboration practices, inspired by our
end-users, to overcome project obstacles and
build a better team.

6. D E S I G N + D E V O P S : : C O N V E Y U X : : 2 0 1 8 6
takeaways
1 DevOps best practices can help break barriers
2 Design research helps kill control egos
3 Cross-discipline teams: a must, not a luxury

7. D E S I G N + D E V O P S : : C O N V E Y U X : : 2 0 1 8 7
What is DevOps

8. D E S I G N + D E V O P S : : C O N V E Y U X : : 2 0 1 8 8
DevOps is about
practice + culture speed + stability

9. D E S I G N + D E V O P S : : C O N V E Y U X : : 2 0 1 8 9
Dev Ops

10. D E S I G N + D E V O P S : : C O N V E Y U X : : 2 0 1 8 1 0
before
blame finger pointing

11. D E S I G N + D E V O P S : : C O N V E Y U X : : 2 0 1 8 1 1
DevOps

12. D E S I G N + D E V O P S : : C O N V E Y U X : : 2 0 1 8 1 2
high-per forming
• 50% higher market cap growth over 3yrs
• Better employee loyalty
• More time to improve the products
vs just maintaining them
50%
loyalty
time

13. D E S I G N + D E V O P S : : C O N V E Y U X : : 2 0 1 8 1 3
• Increased time spent on
developing new features by
700%
thanks to automation
what ’s possible

14. D E S I G N + D E V O P S : : C O N V E Y U X : : 2 0 1 8 1 4
I
don’t
like
change!

15. D E S I G N + D E V O P S : : C O N V E Y U X : : 2 0 1 8 1 5
But what do we know

16. D E S I G N + D E V O P S : : C O N V E Y U X : : 2 0 1 8 1 6
IBM Design Thinking
observe
reflect
make

17. D E S I G N + D E V O P S : : C O N V E Y U X : : 2 0 1 8 17
Focus: application performance monitoring products
Goal: to help DevOps team understand performance changes of the
code/infrastructure decisions they make
Outcome: give them the confidence empowerment to focus on
improvements instead of dealing with firestorms
Three years

18. D E S I G N + D E V O P S : : C O N V E Y U X : : 2 0 1 8 1 8
Anybody say research?
300+
Research participants
50hrs
Usability test footage
• Contextual Research
• Usability Studies
• Sponsor users
• Interviews
• Surveys
• Workshops
• Conferences

19. D E S I G N + D E V O P S : : C O N V E Y U X : : 2 0 1 8 19
Availabilit y Monitoring
as-is
Monitoring is an afterthought.
Devs needed an easier way to test
their code with little overhead
to-be
Devs can test their service/app
early on, without depending on
Ops, to ensure their code is ready
to move across the pipeline

20. D E S I G N + D E V O P S : : C O N V E Y U X : : 2 0 1 8 2 0
Homestead : Monitoring Go to: Summary
Docs
| Org NameUS South
Maureen
ComputeIBM Bluemix
Critical
CURRENT STATUS
15
Warning
35
Critical
ALERTS (PAST 24 HRS)
280
Bluemix
42
Pipeline
ACTIVITIES (PAST 24 HRS) AVAILABILITY REPORT
url_slow_response on
url_slow_response on
DESCRIPTION
1/20/16 | 9:05 AM
1/20/16 | 9:05 AM
TIMESTAMP
AwesomeContainer
Containter-Y
TRIGGERED BY
504
504
ERRORS
Breakdown
Breakdown
012
11
# STATUS
Critical
Critical
12 Warning30 Warning Major103 CriticalAlerts All Locations
100
3
30
12
0
0
Alert Frequency In the Past 24 Hours
Homestead App Running From: Dallas
Budapest
0 Critical
0 Warning
Mumbai
Test here
Alerts: Open
View Contributors
1/3 1/4 1/5 1/5 1/6 1/7 Today
.98
.98Average
7/12 transactions
Satisfaction Score
1/3 1/4 1/5 1/5 1/6 1/7 Today
98%
99%
Average
3/12 transactions
Availability (Now)
1/3 1/4 1/5 1/5 1/6 1/7 Today
.48s
30.1k
31.4k
Average
11/12 transactions
Request Volume Response
Time: 7 daysApplication Health Past 7 Days
insight
Testing is a crucial, yet forgotten,
part of the development process
outcome
A synthetic monitoring experience
to help developers proactively
test their apps across locations
throughout the release cycle
health overview global testing auto-setup

21. D E S I G N + D E V O P S : : C O N V E Y U X : : 2 0 1 8 2 1
Show me how
Looking for some more data?
Gain access to these metrics and more by instrumenting our data collector
Availability
See your app’s availability and how it has
been performing over time.
Satisfaction Score
Are your end-users happy with your app?
Capture this insight in an industry-standard
response time calculation.
Request Volume
See how many end users are making
requests to your application
Homestead : Monitoring
Docs | Org NameUS South
Maureen
ComputeIBM Bluemix
Homestead App Running From: Dallas
Alerts: OpenAlert Frequency In the Past 24 Hours
1. login
2. checkout
3. addToCart
4. purchase
5. browse
TRANSACTION
82%
61%
40%
20%
1%
PERCENT OF TOTAL
7,532
6,123
5,100
3,280
2,166
VOLUME
1. login
2. checkout
3. addToCart
4. purchase
5. browse
TESTS
82%
61%
40%
20%
1%
FAILURE RATE
3
30
45
10
5
INSTANCES
82%
61%
40%
20%
1%
DISSATISFACTION
.75
.76
.80
.85
.90
SIZE
1. login
2. checkout
3. addToCart
4. purchase
5. browse
TRANSACTION
1/3 1/4 1/5 1/5 1/6 1/7 Today
.98
.98Average
7/12 transactions
Satisfaction Score
1/3 1/4 1/5 1/5 1/6 1/7 Today
98%
99%
Average
3/12 transactions
Availability (Now)
1/3 1/4 1/5 1/5 1/6 1/7 Today
30.1k
31.4k
Average
11/12 transactions
Request Volume
Time: 7 daysApplication Health Past 7 Days
Hide Contributors
insight
Identified the need to understand
the business impact of an
application’s performance
outcome
Allowed developers to determine
if they need to invest time
trouble-shooting a problem
biz metrics end-user POV low performers

22. D E S I G N + D E V O P S : : C O N V E Y U X : : 2 0 1 8 2 2
Clear All ShareSan Jose Hong Kong Dallas London
12am 8am 4pm4am 12pm 8pm 8pm2am 10am 6pm6am 2pm
1.6s
1.4s
1.2s
1s
800ms
600ms
400ms
200ms
0ms
Alerts
Activity
Interact with the graph to surface data
Metric Feed Metric Selection
1/20/16 | 2:00 PM Now
Response Time Availability Script: CheckoutTime: 24 hr
Maureen
Maureen
Maureen
Maureen
Maureen
Maureen
Maureen
Github
Github
Bluemix
Bluemix
Bluemix
Bluemix
Bluemix
ad3jr6k
ad3jr6k
—
—
—
—
—
Merging with development
Additional multi-factor authentication functionality
Additional multi-factor authentication functionality
Merging with development
Completed code fix for duplicate checkout item error jamie’s intuition says...
Additional multi-factor authentication functionality
Merging with development
1/20/16 | 9:05 AM
1/20/16 | 9:05 AM
1/20/16 | 9:05 AM
1/20/16 | 9:05 AM
1/20/16 | 9:05 AM
1/20/16 | 9:05 AM
1/20/16 | 9:05 AM
Master
—
—
Master
—
—
—
insight
Devs tend to correlate their
code deployments to changes
in performance
outcome
We visually connected
performance trends with alerts
and deployments in a single
visual pattern
activities changes historical mapping

23. D E S I G N + D E V O P S : : C O N V E Y U X : : 2 0 1 8 2 3
1/20/16 | 9:05 am
1/20/16 | 8:35 am
1/20/16 | 8:05 am
TIMESTAMP
3
6
1
ERRORS
Dallas
San Jose
Beijing
LOCATION
2.1s
2.0s
1.8s
RESPONSESTATUS
Critical
Critical
Critical
Page Load
Search Submit
Click and Wait
Checkout
NAME
—
—
—
504
ERRORS
8MS
230MS
320MS
1.2S
TIME
Loaded
Loaded
Loaded
Failed
STATUSSEQUENCE
mobilizer.js
https://urlname.server.app.php —JS 2.8B 8MS Loaded
index.html
…me.server.app.php/index-en
—HTML 65.3KB 730MS Loaded
styles.css
.../urlname.server.app.php/css
—CSS 234KB 920MS Loaded
how-to-video.mpeg
…/urlnameurlname.server.app.php/img1
—MPG 4.8MB 1.2S Loaded
FILE PATH ERRORSTYPE SIZE TIME STATUSSEQUENCE
Test Instances
Expert Advice
Critical
5% Warning 5% Normal10% Warning Major75% Critical
Breakdown: Checkout Script In the Past 24 Hours script.checkout.html
AVG. RESPONSE TIME
1.2s
AVG. AVAILABILITY
15%
Critical Warning
Errors:
3
Response (Avg.):
1.8s (1.2s)
Script Steps
6
Total Requests (External):
7(1)
TEST INSTANCES LATEST STATUS
Expand
Expand
Expand
Collapse
Collapse
insight
Developers follow a drill-down
mentality when looking for a sore
spot in a chain of events
outcome
Reduced info overload by
grouping data to mimics how
devs inspect a problem
status ratios nested data visual evidence

24. D E S I G N + D E V O P S : : C O N V E Y U X : : 2 0 1 8 2 4
Homestead : Monitoring Go to: Summary
100 30
0
Alert Frequency In the Past 24 Hours
Homestead App Running From: Dallas
Budapest
0 Critical
0 Warning
Alerts: Open
ALERTS (50) TEAM ACTIVITY (305)
Bluemix 280
Github 25
Toolchain —
SLA Target 99.95%
Last Day 96.75%
Last Week 98.48%
Last Month 99.83%
15
Warning
35
Critical
Critical
CURRENT STATUS
App-Name Total Tests (6)
WORST PERFORMING TESTS Checkout-test my-api Onboarding-scripts1 2 3
1
2
Next: Tests Overview
Ok, Got It!Previous
Guides
1 This map displays where alerts
are coming from. Each location
will show you the number of
alerts, their severity, and if the
location is available or unavailable.
2 When you start seeing alerts, click
on a location on the map to filter
the table below. From here, you
can go to a Breakdown page to
get details about an alert.
Pinpoint Alert Source
Breakdown
Monitoring Setup
Edit Mode
Minutes5
Interval
Settings Save
Locations
AddDoes it require a username or password?
Type or paste your URL
Link:
Give it a short name
Name
Alert Notification
Edit Mode
Development is a continuous process. Here you
can modify your test’s parameters, settings and
thresholds and see your related costs.
Notification Setup
Create and modify your alerting policies so you
only get notified when and how you want to. Keep
your team in the loop, share responsibilities and
make sure each alert is documented and
addressed.
Testing frequency
Simultaneous Staggered
Run test on
Firefox Chrome Internet Explorer
Frankfurt LondonHong Kong
Dallas
San Jose
Tokyo
Queretaro
Chennai
Paris
Singapore
Washington
Amsterdam
Melbourne
San Paulo
Toronto
Documentation
Private Locations
Learn more about how Private Locations work,
why you would want to use them, and how to
trouble shoot if things go wrong.
Public Locations Private Locations
Testing 3/12 locations Testing 0 locations
insight
Users expect a learning curve
but want it to be brief
outcome
Built a guided walkthrough
library to speed up the
learning process and build
the user’s self-confidence
onboarding model in-context view microinteractions

25. D E S I G N + D E V O P S : : C O N V E Y U X : : 2 0 1 8 2 5
Yes, and…

26. D E S I G N + D E V O P S : : C O N V E Y U X : : 2 0 1 8 2 6
parallels

27. D E S I G N + D E V O P S : : C O N V E Y U X : : 2 0 1 8 2 7
lense 1
how development
teams navigate
culture change
lense 2
the actual practices
of DevOps

28. D E S I G N + D E V O P S : : C O N V E Y U X : : 2 0 1 8 2 8
transformational
leadership
• Predicts high team performance
and employee engagement
• Decentralize decision making to
unlock intrinsic motivation
• Inspire and align with mission
• Lead the change know the way
• Develop leaders

29. D E S I G N + D E V O P S : : C O N V E Y U X : : 2 0 1 8 2 9
blameless
culture
• Process and culture change
can be hard and get heated
• Discussions are about process
and technology, not people
• DevOps considers blameless
retrospectives table stakes for
culture transformation

30. D E S I G N + D E V O P S : : C O N V E Y U X : : 2 0 1 8 3 0
• Each time Devs push code they validate
that it meets the predetermined goals
• Test Driven Development writes the
tests before they code
Test assumptions. Learn cheaply.
Should designers develop the
usability tests before we design?
devOps
continuous testing
design
usability testing

31. D E S I G N + D E V O P S : : C O N V E Y U X : : 2 0 1 8 3 1
• In DevOps the person who builds the
feature doesn’t deploy it
• Ensures another set of eyes on the
code, incentives quality
For us critiques are like breathing,
we do them constantly to validate intent
What other forms of review do we have?
devOps
peer code reviews
design
critique sessions

32. D E S I G N + D E V O P S : : C O N V E Y U X : : 2 0 1 8 3 2
• Devs often swap responsibilities daily
• Developer pushing the code to be
reviewed will review someone else’s
code tomorrow
• Not saying we should be unicorns
We learned to co-own ideas and
concepts, but empower each designer
to own its manifestation, the deliverable
How do we balance having shared skills
vs specialization?
devOps
share responsibilities
design
shared ownership

33. D E S I G N + D E V O P S : : C O N V E Y U X : : 2 0 1 8 3 3
• Automation is critical to DevOps
success as it increases stability
and agility
• The key for automation is to focus on
the repetitive and boring tasks
Design systems and or redline plugins let
us spend less time on the tedious tasks
and more time on the important tasks
If design systems our form of
automation, what’s the next step?
devOps
increased automation
design
design systems

34. D E S I G N + D E V O P S : : C O N V E Y U X : : 2 0 1 8 3 4
• DevOps teams measure themselves
against biz goals, not Dev-specific goals
• Instead of measuring code updates,
they measure if it helped increase
customer satisfaction
This translates directly because we don’t
just designing for emotions, but to impact
the business
How could we use business metrics to
help prioritize our team tasks?
devOps
business outcomes
design
user outcomes

35. D E S I G N + D E V O P S : : C O N V E Y U X : : 2 0 1 8 3 5
• We share user feedback, prioritize work and track our progress
• Define why should we care about things and why the work matters
• Ensure alignment of biz goals : user outcomes
But we’re still three different teams
Des + Dev + Mgmt

36. D E S I G N + D E V O P S : : C O N V E Y U X : : 2 0 1 8 3 6
team collaboration
dev
mgmt
design

37. D E S I G N + D E V O P S : : C O N V E Y U X : : 2 0 1 8 3 7
in a world…

38. D E S I G N + D E V O P S : : C O N V E Y U X : : 2 0 1 8 3 8
outcome teams
outcome 3
outcome 2
outcome 4
outcome 1
mgmt + dev +
design + sales…

39. D E S I G N + D E V O P S : : C O N V E Y U X : : 2 0 1 8 3 9
the utopia
/ We don’t want unicorn people,
we need unicorn teams!
// Build teams based on the skill
set needed to deliver a
user outcome, not vice versa
/// Get comfy with failing.
Embrace it. Plan for it.

40. D E S I G N + D E V O P S : : C O N V E Y U X : : 2 0 1 8 4 0
outcome teams NOT product teams
build faster better

41. D E S I G N + D E V O P S : : C O N V E Y U X : : 2 0 1 8 4 1
Who would be in your
outcome team

42. D E S I G N + D E V O P S : : C O N V E Y U X : : 2 0 1 8 4 2
Thank you
questions + feedback
Icons courtesy of FlatIcon.com // Images used for presentation purposes only, following Creative Commons licenses.
come talk to us (let’s have a beer)