Empowered development teams are the backbone of the agile development idea. Ideally each team incorporates the complete knowledge needed in the development process, including front end development, quality assurance techniques and site operation know-how. At mobile.international we tried to achieve this with including specialists of the different disciplines in our teams. In 2012 we presented this topic at the TechCon in Lissabon, an eBay internal technology event. We gave an overview of the former and current team structures and the benefits and challenges we encountered so far (yes, there are challenges…), with an special emphasis on the QA discipline. Here we included the slides of the presentation for the interested readers.

1. x-functional teams and
„embedded“ QA
eBay-Dev-Blog, 2013 Christian Verwiebe, Jerome Brandt

2. Agenda
“Traditional Setup”
Agile Setup
Achievements
Challenges – Responses
What has changed in our work
Conclusions
1

3. “Traditional” Setup @ mobile.de 2007
Separated teams:
QA + content + webdesign (part of product department),
backend development & site operations
Big Bang Releases (up to 500 tickets)
PD developing backend features
WD implementing the front-end tasks
Content creating new content
QA testing + managing big releases
SiteOps rolling out the releases every 2-3 weeks
A pure “waterfall”
2

4. Remember the old days…
1. PD develops feature A in backend
2. WD designs feature A
3. “release-branch” is created every three weeks
4. QA tests feature A and finds a bug and gives it to WD
5. WD looks at the bug and decides it is a backend error
6. PD analyses feature A, fixes the error
7. QA rechecks feature A and it works now, but layout is now broken
8. WD fixes layout and returns feature A to QA
9. QA rechecks feature A and layout is fine now, QA approves the feature
10. SiteOps rolls out release with feature A but detects problem with a missing
production-property
3

5. What could be improved?
High lead time for each new product feature
Bad transparency of the transaction costs for a feature
Bad transparency of the actual costs for the delivery of a given feature,
including development, design, content, testing, operations, maintenance
High number of prod bugs after the bing bang releases
4

6. Could x-functional teams be an answer
5
?

7. Transition 2008 - 2010
2008/2009
Project-QA engineers testing features for new project teams
Core-QA testing releases for regression and projects without QA,
managing releases hand in hand with new release manager
Web design and QA move to technology department for better integration
One Pre Production System for test (PPS)
Re-Introduce the scrum process to the project teams
6

8. Transition 2008 - 2010
2010
SiteOps engineers moving from the operations-fishbowl-area to the project teams
No more separate QA department, QA Engineers now reporting directly to team lead of
project team
reduced the release-cycle from two/three weeks to one week
7

9. Current agile team setup @ mobile
Up to four backend developers and at least one QA and one web developer team member in each
agile development team
Two of the members own the roles of team lead and technical lead
Every development team is supported by an Operations Engineer
The product owner is co-located with “his/her” project team
At least one dedicated integration system for each development team
8

10. What did we achieve?
Overview
Better understanding & more transparency
More speed & reduced lead time
More motivation & team spirit
Team responsibility
9

11. Achievements
Better understanding & more transparency
The team develops a common vocabulary to describe products/problems
More knowledge about work strategies/procedures across the disciplines
different views/perspectives on the product lead to an holistic understanding
Feature costs can now be estimated more transparent and accurate
10

12. Speed & reduced lead time
When problems arise, chances are high that the team finds a quick solution
Bug fixing and retesting can now be done in minutes!
The team finds the approach/tools which suits its needs
Fast release cycles are only possible with x-functional teams
Achievements
11

13. release frequency in 2009
12
Releases: Every 2 weeks with multiple features from several teams

14. release frequency today
13
Releases: several single feature contributions per day

15. Achievements
Motivation &Team Spirit
Because the team is empowered to make decisions how it aims to achieve its goals, it
is highly motivated
Team is measured by success of features (by team goals)
project teams get more feedback about feature success and team members identifies
more with their product features
14

16. Achievements
Team responsibility
The whole team knows it is responsible for the quality of the applications and upcoming
production bugs
Because the team is aware of its responsibility, every team member helps out
at other disciplines
No more “over the fence throwing” of tickets
15

17. Challenges
Overview
S.P.O.F.
organisational blindness
Inter-team coordination & Inter-team tensions
“the voice of the many”
Knowledge transfer
16

18. Challenges
S.P.O.F:
x-functional teams have the implicit danger of SPOFs (Single point of failure) for the smaller
disciplines.
„Our QA-guy is on Bali. AGAIN! We cannot publish anything!“
What we can do:
less „What is my job?“, more „What needs to be done?
Identify the tasks that needs to be done and afterwards identify who the best available
team member(s) to do it.
Share knowledge for basic tasks of other disciples and use it
17

19. Challenges
organisational blindness / “Betriebsblindheit”
With involvement in the very beginning, the whole team is prone to the same conceptual
mistakes, because they share a common view on a feature (“complete brain outage”)
What you can do:
initiate a code/story/testcase review by a collegues of a different team
What an organization can do:
encourage Show-rooms and Test-Dojos
18

20. Challenges
Inter-team coordination
With each team focused on its project scope, teams can hinder each other with concurring
changes in the same applications/side effects
New features get broken/made obsolete by other teams
Design decisions in one team increase effort in other teams.
What you can do:
Be curious! Speak about what you do. Send mails for important things.
What an organisation can do:
Give room and encourage exchange, g.e. regularly technology open-space meetings,
Show-room events
19

21. Challenges
Inter-team tensions (“us vs. them”)
One backside of the human nature and its favor for small groups is that humans like to
blame “the others” for bad things
Quite fast you have “them” always breaking the master or “them” always causing a
stop-the-line or “them” calling for a hotfix
What you can do:
not much. Try to remember, “they” are human, too!
What an organisation can do:
As several studies have shown, the only thing that helps to counter these tendencies is
to work together on a common goal.
Find these goals and emphasize them
20

22. Challenges
“The voice of the many”
Psychological factor affecting x-functional teams if there is a majority of one discipline
(„Gewerk“)
21

23. Challenges
Before:
22
QA DepartmentPD Department
These frontend
tests are essential!
Really
necessary?
Let’s talk…

24. Challenges
Now:
23
Cross-functional team with embedded QA
These frontend
tests are
essential!
I find your lack of faith disturbing!
relax, this code is so easy,
it will never break!
really?

25. Challenges
“The voice of the many”
What you can do:
Talk to the other team members and convince them, try things out
What an organisation can do:
open space forum where topics can be addressed across teams with self-organized
participants
24

26. Challenges
Knowledge transfer
Team members become experts on their new team-features only
New technologies and concepts remain inside the project teams
What you can do:
Create easily accessible documentation, initiate test-dojos, present new
concepts/technologies outside the team
What the organisation can do:
showroom-events, open-space, have a good infrastructure for documentation
25

27. What has changed in our work
For QA:
• we test earlier
• we test more adapted to the story
• we test more accurately
• we write more tests
• more code reviews with PD
• more requirement engineering with Business
• more requirement reviews
• more meta-tasks
26