SlideShare ist ein Scribd-Unternehmen logo
1 von 12
Downloaden Sie, um offline zu lesen
Reaching Business Goals with Value Adding CMMI Assessments, by Ben Linders

Ericsson EuroLab Netherlands (ELN) is a company within Ericsson, the world's leading supplier
in telecommunications. They have facilities in the south (Rijen) and in the east (Enschede, and
Emmen). In total about 1100 employees are working for this company, almost all are directly
involved in software and hardware development. The scope of this article is about the design unit
in Rijen, and in particular about the Charging Competence Centre department. This department
employs about 160 persons, developing charging systems for telephone exchanges. The
technology is UNIX and NT based systems, using UML based Object Oriented development with
C++ and JAVA, deploying mostly Rational tools

This paper describes how to apply the CMMI in a business-focused way, by doing frequent small
assessments. Goals are input to the assessments, and a matrix is used to assure both organisation
and process area coverage. The results of assessments integrate smoothly with operational targets
of existing groups within the organisation.

Practical tools support the frequent assessments, and experience with the CMMI is used to
optimize the assessment and improvement cycle, resulting in continuously improving the
performance of the organisation based on its goals.


1    History: CMM as a rating model

CMM assessments have been done for quite some years within Ericsson. This paragraph
describes the history, a more detailed description can be found in a separate paper on this subject
[ref 1].

Ericsson was one of the early adopters of the CMM model. The process focus, and the concept of
measurement based improvement were the unique selling points why the model was chosen. The
first assessment, back in 1994, showed Ericsson Rijen was on level 1, which was no surprise. An
improvement program was initiated, which established a process support organisation that rolled
out all level 2 processes in the projects. The next assessment in 1995 showed that the Research
and Development Unit of Ericsson Rijen was on level 3, the first company within Ericsson
worldwide.

Reaching level 3 was a big reward for the organisation. As a result, the focus changed, and
operational goals such as meeting deadlines, combined with many vacancies demanded their toll.
Projects became less focused on processes and measurements. Since the process support
organisation remained, as did the culture to improve in most of the departments, the projects still
managed to live up to the expectations. But things didn’t always go that smoothly…

One department in Ericsson Rijen, the Charging Competence Centre, kept on believing in the
CMM and its usefulness. It started their own program to come to level 4, by defining
measurements and applying them in their projects [ref 2]. In June 1998, they made their first
preparations towards a new assessment. Bill Curtis, associated with the Software Engineering
Institute, stated that the Unix development department could be "among the world elite in one
year". Finally, the department was re-assessed in November 1998, where it was found to be a
professional "solid as a rock" level 3 organization, with parts of level 4 institutionalised.




Reaching Business Goals with Value Adding CMMI Assessments, Ben Linders                 Page 1 of 12
Having worked for quite some years on process improvements, and not getting the reward of
being fully at level 4, came somewhat as a disappointment to this department. And it also
confused them, as they didn’t just implement the practices of the CMM model. The approach had
been from the goals and concepts of the model, implemented it in such a way that it made sense
to do it. Though no process areas were excluded, not all practices were implemented. For some
there were alternative solutions. But most of all they were confused because in their opinion the
improvement programs had delivered results, being an organisation able to perform projects in a
controlled way, and capable of improving where business goals required it.

As a next step, the department didn’t start a new improvement program to fill the gaps found in
the assessment. Instead, the approach focussing on business goals and selecting improvements
that contribute to them was continued, regardless of CMM levels involved. This focus on CMM
matched perfectly with operational goals set by the department, in the sense that CMM would
contribute directly to the business.

Along with this focus, the assessment approach had to change also. Full assessments were no
longer useful, instead there was a need for small focused assessments which would come up with
detailed findings. These assessments should be an instrument to be used next to audits and
improvement sessions in projects.


2     Approach: Business Focused CMMI Assessments

A new assessment method has been defined: Business-focused CMMI assessments. Such an
assessment is a class C type assessment [ref 3], where the main purpose is to find strengths and
weaknesses. The assessment consists of the following steps:

•     Assign Orderer and Customer
•     Select Goals
•     Define Organizational Scope
•     Determine Process Areas
•     Organise Assessment Team
•     Plan Assessment
•     Perform Assessment
•     Support Improvements

The purpose of the first six steps is to make an assessment proposal. This document fully defines
the assessment, and after approval it is used to perform the assessment and support the
improvements. The steps will be discussed in detail in the following paragraphs.


2.1     Assign Orderer and Customer

The first step is to define the interface between the assessment and the organisation. This
interface consists of two persons: An orderer and a customer. They will play an essential role.

The orderer has to be a member of the highest management of the organisation. The orderer is
responsible for one or more goals that will selected for this assessment, for that reason (s)he is
particularly interested that a good assessment will be done. Also, this person arranges funding for
the assessment, including hours of the interviewees. It is also the person who has end



Reaching Business Goals with Value Adding CMMI Assessments, Ben Linders                 Page 2 of 12
The customer is the person who needs the result from the assessment: The findings. This person
is responsible for implementing improvement actions. It can be the same person as the orderer. It
must be clear, before the assessment is started, that the customer is able to take over the results of
the assessment, and arrange the improvement actions. If there is no room in the organisation to
improve, then there is no need to assess either.

Both the orderer and customer must be clearly stated in the assessment proposal. They will also
attend kick-off and final meeting, where they personally show their stakes in this assessment and
what they require the assessment to deliver.


2.2   Select Goals

For every assessment, there must be a business reason to do it. This reason has to do with one or
more goals of the organisation, or with strategies performed to reach the goal(s). This step results
in a list of goals and strategies that have to be assessed.

For instance, an organisation can have the goal: “All projects must be finished on the planned end
date”. To reach this goal, there are strategies such as improvement programs for project planning,
and lead-time improvement. When it is uncertain if the strategies will assure that the goal will be
met within the defined time frame (usually one or more calendar years), then an assessment is
useful to investigate it.

Goal setting is a complex, but vital activity. This assessment approach requires that the goals are
clear, but sometimes it turns out that when discussing them, additional clarification is needed.
Also priorities between goals are not always sufficiently defined. It is of the utmost importance
that there are no more uncertainties about the goals before the assessment planning is started.

An assessment focuses on a subset of the goals, in order to come with findings specifically related
to reaching those goals. Assessing against all goals of an organisation results in too many
findings, where priorities will have to be set after an assessment. In such a case, findings will be
thrown away and valuable assessment time has been wasted. More important: An expectation was
given that the problems would be solved. Not living up to this expectation can seriously hamper
future assessments and improvement programs.

The experience is that assessments at the Charging Competence Centre have been focussing on 1
to 2 main goals. Further on in this paper it will become clear how it was assured that all the goals
were covered by assessments over time.


2.3   Define Organizational scope

The result of the previous step is a list of goals and strategies. In this step the part of the
organisation involved in reaching these goals or performing the strategies must be identified. The
result will be a list of groups and roles/functions, which have to be interviewed.



Reaching Business Goals with Value Adding CMMI Assessments, Ben Linders                   Page 3 of 12
A goal is assigned to a responsible person or group in the organisation. Also, other persons and
groups are performing work that relates to the goal. The boundary of the assessment must be
defined such that it includes the right parts of the organisation, nothing more, and nothing less.

A misunderstanding is that in an assessment, every group of the organisation has to be
represented. This should not be the case. If groups are not involved in activities related to the
goals to be assessed, then there is no need to interview them. Any findings from this group would
be put aside and considered irrelevant, because they do not endanger the goal. So why waste their
time, and yours?

So the scope includes only groups giving input to the work, performing the work, and requiring
output from the work, where the work is that part of all work done in the department that is
required to reach the goal or perform the strategy.


2.4    Determine Process Areas

Now that the scope of the assessment is defined from the organisation’s perspective, let’s look at
the scope from the model’s perspective. That is, given the organisational scope there is usually
need to assess only a selected set of process areas from the CMMI model.

Looking at the same example that the organisational goal to investigate is lead-time precision,
and the groups involved would be two product development teams running projects, and a staff
group on configuration management. It was decided in the organisational scope step that the
assessment should only interview the project management staff, and not the engineers. In that
case, it would be useful to select process areas mainly from the project management cluster, and
additionally add configuration management from the support cluster. There is no need to assess
process areas from engineering or process management clusters, since no activities from these
process areas are performed within the organisational boundaries of the assessment. Again this
prevents coming up with findings which are not goal related.

Some experience from the Charging Competence Centre department: In the assessments done in
2000 and 2001, the number of process areas varied a lot. There were assessments with only 1
(Decision Analysis and Resolution) or 2 (Requirements Management/development) process areas,
but there have also been assessments with 7 areas. The latter is about the maximum, when it
becomes more then one should return to the first step and try to reduce the number of goals
involved. There has been no assessment covering all process areas on a specific level.

The result of this step is a list of process areas that will be assessed. With this step, the total scope
of the assessment is defined.


2.5    Organise Assessment Team

Until now, there has only been an Assessment Team Leader. This person has done all previous
steps, together with persons from the line organisation. Now the team can be assembled, so that
they can take the next steps together.

A team consists of a team leader, and 1 to 2 assessors. The minimum relates to the fact that it is
essential to have at least two views on the findings. At any time an assessor can discuss
preliminary findings, and do a peer review before presenting them to the interviewees. The


Reaching Business Goals with Value Adding CMMI Assessments, Ben Linders                      Page 4 of 12
maximum relates to the number of process areas and communication; adding more people (unless
they are expert in a certain area) doesn’t improve the quality nor the time needed.

In an assessment team there has to be at least 1 member from the assessed organisation. The role
of this “local” member is to assure that the right people are interviewed, and that potential
findings within the goals are derived, which are formulated using terminology known and used
within the organisation. This helps coping with the so-called “CMMI speak problem”, where an
organisation doesn’t understand the questions and cannot use the findings, because of terms used
from the CMMI model that they do not understand.

Assessment team members must have excellent communication skills, have knowledge both in
the way an assessment is conducted and in the process areas being assessed, and must have
sufficient time available within the time frame. The latter is very important, as an assessment has
a fixed time schedule from start to finish. You cannot postpone a consensus or a final meeting if
you run out of time, re-planning meetings with all the people involved will almost always mean
that an assessment slips, and then loses the momentum in the organisation.

Roughly the time needed in an assessment is 40-60 hours for a team leader, and 20-40 hours for
an assessor. This depends of course on experience, and on the scope of the assessment.


2.6     Plan Assessment

The next step now that the part of the organisation to be interviewed and the process areas are
defined, is that the assessment team makes a detailed plan of the assessment. This plan includes:

•     Select Interviewees
•     Define Timeplan


2.6.1    Select Interviewees

The interviewees have to be selected within the defined organisation scope (groups and
roles/functions) defined earlier. The purpose of selecting interviewees is assuring that every
process area is sufficiently covered. That is, the group of people interviewed should contain
sufficient persons that perform the activities of the process areas from several angles.

Not only persons doing the work should be interviewed, but also persons ordering (and being
responsible for) the work, and persons depending on the outcome of the work, should be
interviewed. This can be compared to a 360-degree appraisal, where a person’s boss, customers
and people below him/her have to give input on the performance.

Given the previous example (lead-time in project management teams and configuration
management staff, with process areas from project management and configuration management)
the following roles could be interviewed:
• Project managers                   Performing project management activities
• Team leaders                       Idem, but for the design teams
• Configuration managers             Performing configuration management activities
• Product managers                   Customers of a project
• Members from maintenance Taking over work and products from a project


Reaching Business Goals with Value Adding CMMI Assessments, Ben Linders                 Page 5 of 12
In each role there should be sufficient people interviewed, usually that is 2-4 persons. This
depends on how far the activity is performed in a uniform way by different people or groups,
variance in experience and competence levels, different customers with unique expectations, etc.
When in doubt it is usually better to include an additional person initially, it is easier to exclude
that person later on when it appears that (s)he is not needed than to add an additional person when
coverage is not enough. On the other hand, some persons might perform several roles.

Again some experience from the Charging Competence Centre: Assessments done until now have
interviewed a minimum of 7 persons, and a maximum of 19. If the number becomes too big, then
reconsider the goals, organisational groups, or number of process areas involved. It may be useful
to split up an assessment, in order to have more insight into specific goals, groups or process
areas.


2.6.2   Define Timeplan

For performing the assessment, several meetings have to be planned and deadlines have to be set.
More information about the meetings can be found in step “perform assessment”, in this step only
the people attending the meeting and the timeline is defined. Please note that all meetings are
planned in advance, and that deadlines are fixed in an assessment.

The kick off meeting is attended by everybody involved in the assessment, so the interviewees,
orderer, customer, and the assessment team.

The overview presentation is optional, and open to anybody interested in an explanation of the
process area. It can also include people from outside the assessment.

A deadline must be set for submittal of the questionnaires. On this date, all interviewees must
have returned their questionnaire, and it must be checked for completeness and any unclearness.

The consensus meeting is only for the interviewees and the assessment team, others are not
allowed due to confidentiality reasons.

The final meeting is meant for everybody in the assessment. However, if the orderer doesn’t
object, then others may join the meeting to hear the results of the assessment at first hand. Please
note that the purpose is to communicate strengths and weaknesses in order to improve, and not to
judge.

At Ericsson we have experimented during the years with the time plan for this kind of
assessment. The scheme below has appeared to be most effective, both in coping with
organisational pressure and keeping people mentally involved in the assessment.

 Week T – 4 weeks:          First proposal for the assessment
 Week T – 2 weeks:          Final proposal, and all further dates and interviewees fixed
 Week T:                    Kick off and distribution of assessment questionnaires
 Week T + 1 week:           Overview presentation of the process areas
 Week T + 2 weeks:          Deadline for having the questionnaires returned
 Week T + 3 ½ weeks:        Consensus meeting
 Week T + 4 weeks:          Final presentation



Reaching Business Goals with Value Adding CMMI Assessments, Ben Linders                    Page 6 of 12
This schedule gives two weeks for the interviewee to answer all the questions, and 1 ½ week for
the assessment team to come up with findings. If planned in advance, and sufficient commitment
from the organisation is available, this has proved to be feasible.


2.7   Perform Assessment

An assessment starts with a kick off meeting. In this meeting the assessment is presented to the
interviewees; after this meeting they should know what they could expect from the assessment
and what will be their role.

In the kick off meeting, both the orderer and the customer should be present. They should tell the
audience why they want the assessment, and what kind of results they expect. Also they should
ask the interviewees to be open and honest, and that they will value all input and accept the
results from the assessment. The assessment team leader must assure confidentiality and non
attribution, that is that the results sent in by the interviewees will only be known to the assessment
team, and that finding will be stated in such a way that they cannot be attributed to individuals or
projects.

After the kick off, questionnaires will be sent out to the interviewees to collect information. The
questionnaires are based on the goals and activities of the CMMI model. To enable them to fill in
these questionnaires, it is good to organise an overview presentation of the process areas so that
they can better understand the goals and activities of the model. This presentation is not
mandatory, but highly recommended. Note that we deliberately do not use interview sessions,
instead we use consensus meetings.

Defining findings is a creative, and very difficult and complex part of an assessment. It is done by
going through ratings and comments from the interviewee, and look for common opinions,
contradictions, patterns, etc. Drivers for the findings are of course the goals and strategies that are
being investigated. Competence in the process areas that are investigated is essential. It is also
important to know the roles that people perform, and sometimes also their background, favorite
issues, frustrations, etc. The result will be an overview of strengths, weaknesses, and open issues
(things which are not yet clear) per process area.

As a next step a consensus meeting is held. In this meeting the preliminary findings are presented,
and feedback is sought to verify if these findings are correct, complete, and accepted by the
organisation. As a rule, every interviewee has to be in the consensus meeting; for those who
cannot attend a private meeting has to be arranged to go through the findings. You cannot risk
coming up with findings, which are not accepted by any of the interviewees, and thus will be
considered questionable after the assessment. Also, since the assessment is not actively searching
for evidence, but focuses on opinions of the organisation, it is essential that the consensus
meeting results in a common view of the main strengths and weaknesses of the organisation.

At the end of an assessment, a final presentation is given. For all process areas the strengths and
weaknesses are presented. In this meeting, the orderer and the customer of the assessment must
be present to officially accept the results and reconfirm the organisations commitment to do
whatever is needed to solve the findings. In this meeting, the interviewees are thanked for their
contribution. This final meeting is attended by all the interviewees, the orderer and customer. But
it is also good to invite others for which the results can be useful information. Basically, this
meeting is used to buy in to the organisation for the needed improvements.


Reaching Business Goals with Value Adding CMMI Assessments, Ben Linders                    Page 7 of 12
2.8    Support Improvements

In most assessment methods, the assessment is finished after the final meeting, and the final
report. However in this method it is required that the results of the assessment are represented to
the customer, in order to define actions.

For that purpose, a meeting is planned with the assessment team and the customer. Others, who
can assist in defining or execute improvements, are also invited to the meeting. Together they go
through the findings, and define the needed improvement actions. Most actions should be within
their scope of control, for actions outside a member of the meeting will have to assure proper
handover to a responsible.

Please note that actions are not only defined for weaknesses, but also for strengths. An
organisation that only works on improving weaknesses will never be more then average when
compared to other organisations. In order to succeed reaching business goals, an organisation
should also have strengths in which it differentiates with others.

The resulting actions of the meeting will have to be assigned to either line or project bodies. The
assessment team will not do follow up on the actions, but they will remain available if there are
further issues regarding the findings. In some occasions, a member of the assessment team did a
follow presentation on the findings some months after the assessment, to check the effect of the
actions. This can be useful, on the one hand it is a good check if and how things are progressing,
and on the other hand it is a measurement to check whether the findings and the defined actions
are still considered useful.

When a substantial part of the actions has been done, it will be good to re-assess the organisation
in order to verify if the improvements have been successful. For that purpose a new assessment
can be defined, repeating the process as has been described in these 8 steps. Given the fact that
there are frequent assessments, a new assessment reuses results from a previous assessment, like
presentations, tools, etc. More detail on this will be given in the next chapter.


3     Implementation: Tools

No method is complete without tools to support it efficiently. For this assessment method, some
tools have been developed that supported the roll out and acceptance of findings, and have turned
it into an efficient way of improving the organisation.

The term tools is used in a broad sense. What is meant is a combination of templates, checklists,
etc. The following tools are used:
• Assessment matrix
• Questionnaires
• Integrated organisation

Each tool will be explained in more detail. It is important however that a tool never exists on its
own, it is based on a method. On one hand this implies that when an organisation has different
assessment methods, the tools may not be useful. On the other hand, there may be other, more
useful tools for an organisation even when the same method is used. Be sure that the first focus is
on the method, and then on the tools, not visa versa.


Reaching Business Goals with Value Adding CMMI Assessments, Ben Linders                  Page 8 of 12
3.1   Assessment matrix

In an organisation that is continuously improving, there will be frequent assessments focussing on
different goals, process areas, and groups within the organisation. The assessment matrix
provides an overview of all assessments done and planned. The main purpose is strategical
planning of assessments, and assuring that over time everything considered important enough is
assessed.


             CMM-I Assessment matrix ELN/V Q2/2001
                                   Product Area     XXX           YYY             ZZZ
 Process Area
 Project Management
 Project Planning
 Project Monitoring and Control                            Q4 2000                   No
 Integrated Project Management
 Risk Management
                                                           Q1 2000                   No
 Quantitative Project Management                           Q1 2001
 Support
 Configuration Management                                     Q4 2000                No
 Causal Analyses and Resolution                               Q4 2000   Q3 2001




As you can see in the matrix, the two axes represent the organisational groups and the process
areas. A combination of the two is covered in an assessment, which is based on one of more
organisational goals. Each cell states when the last assessment has been done, or when a next
assessment is planned. It is also possible to state that certain combinations will not be assessed.
So this matrix provides an overview of all executed and planned assessment.

There are some basic rules that are used at the Charging Competence Centre. Process Areas and
groups that are considered most vital for the organisation are assessed twice a year (like “project
management” and “engineering”). Other groups directly contributing to the goals will be assessed
at least once every year (like “process management”). Staff units indirectly contributing may be
assessed less frequently.

Another rule is that every assessment has a main purpose. You either want to assess mainly to
explore, i.e. to provide insight and come up with initial findings. Or the main purpose is to verify,
so to confirm that improvement actions have resulted in effectively reaching the goals. If you
already know what is wrong, and haven’t achieved sufficient improvement yet, then there is no
need to do a new assessment.

The assessment matrix has proved to be a valuable tool in assessment planning together with the
management of our organisation. One of the main benefits has been that the matrix helped us to
limit the number of assessments, and assure that an assessment will have a specific focus, while
still knowing that in the long run nothing will be overlooked.




Reaching Business Goals with Value Adding CMMI Assessments, Ben Linders                   Page 9 of 12
3.2   Questionnaires

Ericsson has a long experience with questionnaires with light and class C assessments. Initially
checklists and question forms have been used based on Word. The advantage was that they were
very flexible, on the other hand collecting the questionnaires and arranging them in a way to
come up with findings took a lot of effort.

As one of the first pilots of the CMMI model, Ericsson did an official pilot of the SEI
questionnaire tool in the 4th quarter of 2000. The tool has proved to be valuable, both for the
interviewees and for the assessment team. Major advantage was the automated generation of
questionnaires based on the model, and the collection of questionnaires into combined sheets to
be used for definition of findings. Since it uses the SEI Word file of the CMMI model, upgrading
to newer versions of the model should be fairly easy. The major drawbacks were that the tool was
somewhat inflexible towards changes during the assessment, such as interviewees who are
replaced or withdrawn from the assessment, or who cannot make the deadline.

A project was done, together with the School for Information Technology in Breda (HIO) to
develop a prototype of a new questionnaire tool. The main requirements were that it should
support interviewees when filling in questions, by giving additional information and support, and
that it should help an assessment team in organizing an assessment. The latter is done by have a
set of questionnaires defined, which can be reused in multiple assessments. Also a tracking
system is defined, which shows which assessment sheets have been returned, and who has to be
reminded of their “duties”. A pilot is planned in Q2 2001, the results are not known yet.

Conclusion: There is no uniform questionnaire yet, because different questionnaires all have their
advantages and disadvantages. A strive is towards a more uniform approach, based on previous
experiences.


3.3   Integrated organisation

Though one can argue whether an organisation and its structure can be considered a tool, it has
been very important that the right organisation was available that supported this assessment and
improvement approach.

One main concept within the Charging Competence Centre organisation is that many groups have
a double responsibility. They both have to perform operational work, which is development of
products, and they also have to assure that improvements, which are needed for reaching business
goals, are executed.

To give an example: There is a group called the project office. This groups consists of project
managers, software process engineers, and project administrators. Their first responsibility is to
run high quality and controllable projects in the organisation, and directly following is the
responsibility to improve the way projects are run.

Why is that so important? The main reason is that this same combination of roles comes back in
the assessment method. As interviewees, they must give input on how projects are performed. But
starting in the consensus meeting, the other responsibility becomes important and they start
thinking about improvements needed. At the final meeting they will have some initial thought on
what is needed. When the findings are discussed in detail after the assessment, they will come up



Reaching Business Goals with Value Adding CMMI Assessments, Ben Linders                Page 10 of 12
with improvement actions, which they mostly will do themselves. So this learning organisation
fully integrates the assessments method as one of theirs feedback loops for improvement.

 Another advantage is that these kinds of teams drive the assessments. They are the customers,
and they will be involved in selecting the goals, defining organisation scope and process areas,
and defining improvement actions. By being involved in all stages of an assessment, they greatly
improve the efficiency and the effectiveness of an assessment, including the acceptance of
findings and commitment to take any action that is needed.


4   Experiences with the assessment method

The assessment method came to exist during the assessments in Q1 of 2000. Willing to do an
assessment which was not level based, but functionally oriented, so-called sub assessments were
done for the first time. This was still using the CMM V2.0. Draft C, but taking concepts from the
CMMI V0.2. The success of these assessments led to an initial definition of the new assessment
method.

In August after the CMMI model was released, Ericsson decided to move towards the CMMI
continuous model. Based on that, pilot assessments were defined for all assessment classes within
Ericsson. The Charging Competence Centre has done an official pilot of CMMI class C
assessments. After discussing and analysing the business goals and performance, 4 assessments
were defined. Each one had its own focus, process areas, and groups within the organisation that
will be assessed. For instance, there was one assessment focussing on the project office, including
mainly project management process areas but also including organisational training and some
support process areas considered vital for project management. Another assessment includes only
one process area and group: Decision analyses and resolution in the management team of the
department.

Based on the results, an assessment plan has been made for 2001. The plan has defined 6
business-focused assessments, to be done throughout the year. At the time of the article, two
assessments were finished, and a third one was ongoing. This clearly shows the commitment of
the organisation to improve, and to use business focused CMMI assessments as the approach to
do this.

The assessment method has recently been evaluated. The advantages are:
• Very time effective (assessment time of roughly 180 hours total)
• Quick follow up after the assessment (within 1-2 weeks)
• Good follow up, most findings directly lead to actions (at least 50-70%)

Disadvantages are:
• Much overlap in the CMMI, making it sometimes difficult to focus
• Extracting findings is very difficult to learn (learning by doing)

Given these results, the decision is to continue this assessment method. Improvements which are
considered are removing some of the “sales talk” from assessments, since most people now are
familiar with the CMMI, and put more focus on overview meetings and presentations about the
contents of the process areas and thus improve understanding of the model.




Reaching Business Goals with Value Adding CMMI Assessments, Ben Linders               Page 11 of 12
5   Conclusions

The business-focused assessment method has shown to be very worthwhile. Key success factors
are that assessment are easy to conduct, have a short lead time, and that the findings lead to
actions considered vital for reaching business goals.

Currently the method is being rolled out to other departments. Further on it will be fine-tuned to
fit even better in the performance management program at Ericsson, which defines goal and
measurement based improvement of the performance of organisations. Combined with
improvements in the questionnaire, the result is a solid tool useful for continuously improving the
performance of the organisation.


About the author

Ben Linders has been working in process improvement using CMM based models from 1992
onwards. At Philips he participated in CMM improvement programs for Industrial Electronics,
and for Audio. At Ericsson he was a member of the improvement team that worked for the CMM
level 3 assessment in 1995, and the level 4 assessment in 1998. Currently he is working on the
CMMI and Performance Management programs with Ericsson EuroLab Netherlands. He is a
member of several (national and international) CMM and SPI based improvement networks. He
can be reached by email: Ben.Linders@eln.ericsson.se.


References

Ref 1: From Staged CMM to Continuous CMMI (and back).
Paper by Ben Linders, in SPIder koerier januari 2001.
http://www.st-spider.nl

Ref 2: Institutionalising change, Application of CMM as a model for organisational change at
Ericsson. Paper and presentation by Bart ter Horst, presented at the IEEE EMC’99 conference.
http://ewh.ieee.org/soc/ems/confs.html

Ref 3: ARC, Assessment Requirements for CMMI, Version 1.0.
Software Engineering Institute.
http://www.sei.cmu.edu/cmmi/products/assess.html

Ref 4: CMMI Models
Software Engineering Institute.
http://www.sei.cmu.edu/cmmi/products/models.html




Reaching Business Goals with Value Adding CMMI Assessments, Ben Linders               Page 12 of 12

Weitere ähnliche Inhalte

Was ist angesagt?

Performance management an introduction
Performance management an introductionPerformance management an introduction
Performance management an introductionGopal Verma
 
Planning, mbo, strategy & decision making by arun verma
Planning, mbo, strategy  & decision making by arun vermaPlanning, mbo, strategy  & decision making by arun verma
Planning, mbo, strategy & decision making by arun vermaArun Verma
 
Management Chapter08
Management Chapter08Management Chapter08
Management Chapter08WanBK Leo
 
Performance Management - Herman Augnis
Performance Management - Herman Augnis Performance Management - Herman Augnis
Performance Management - Herman Augnis Preeti Bhaskar
 
Learning & Development and the Performance management
Learning & Development and the Performance managementLearning & Development and the Performance management
Learning & Development and the Performance managementAhmed Shamim
 
Performance management system in malaysia
Performance management system in malaysiaPerformance management system in malaysia
Performance management system in malaysiatanzisoha
 
Chapter iii daye 3 of 18th jan 2015
Chapter iii daye 3 of 18th jan 2015Chapter iii daye 3 of 18th jan 2015
Chapter iii daye 3 of 18th jan 2015pyi kyaw lynn
 
The Top 5 Performance Management Tools
The Top 5 Performance Management ToolsThe Top 5 Performance Management Tools
The Top 5 Performance Management ToolsBernard Marr
 
Project performance management
Project performance managementProject performance management
Project performance managementdamogupto
 
Performance Measurement: Rules For Driving Results
Performance Measurement:  Rules For Driving ResultsPerformance Measurement:  Rules For Driving Results
Performance Measurement: Rules For Driving Resultshenrytk2
 
Sixsigmadtifactsheet
SixsigmadtifactsheetSixsigmadtifactsheet
Sixsigmadtifactsheetillangakoon
 
Creating Performance Based Culture trough Performance Management Systems
Creating Performance Based Culture trough Performance Management Systems Creating Performance Based Culture trough Performance Management Systems
Creating Performance Based Culture trough Performance Management Systems Kenny Ong
 
Assignment on Performance management
Assignment on Performance managementAssignment on Performance management
Assignment on Performance managementKamal Hossain
 
Corporate performance management how_to_build_a_better_organization_through_...
Corporate performance management  how_to_build_a_better_organization_through_...Corporate performance management  how_to_build_a_better_organization_through_...
Corporate performance management how_to_build_a_better_organization_through_...rifki_rizal_ahmad
 

Was ist angesagt? (20)

Chap05
Chap05Chap05
Chap05
 
Performance management an introduction
Performance management an introductionPerformance management an introduction
Performance management an introduction
 
Planning, mbo, strategy & decision making by arun verma
Planning, mbo, strategy  & decision making by arun vermaPlanning, mbo, strategy  & decision making by arun verma
Planning, mbo, strategy & decision making by arun verma
 
Management Chapter08
Management Chapter08Management Chapter08
Management Chapter08
 
Performance Management - Herman Augnis
Performance Management - Herman Augnis Performance Management - Herman Augnis
Performance Management - Herman Augnis
 
Learning & Development and the Performance management
Learning & Development and the Performance managementLearning & Development and the Performance management
Learning & Development and the Performance management
 
Case Studies on Performance Management II
Case Studies on Performance Management IICase Studies on Performance Management II
Case Studies on Performance Management II
 
Performance management system in malaysia
Performance management system in malaysiaPerformance management system in malaysia
Performance management system in malaysia
 
Unit 2.pom
Unit 2.pomUnit 2.pom
Unit 2.pom
 
Chapter iii daye 3 of 18th jan 2015
Chapter iii daye 3 of 18th jan 2015Chapter iii daye 3 of 18th jan 2015
Chapter iii daye 3 of 18th jan 2015
 
The Top 5 Performance Management Tools
The Top 5 Performance Management ToolsThe Top 5 Performance Management Tools
The Top 5 Performance Management Tools
 
Strategic Implementation
Strategic Implementation Strategic Implementation
Strategic Implementation
 
Project performance management
Project performance managementProject performance management
Project performance management
 
Performance Measurement: Rules For Driving Results
Performance Measurement:  Rules For Driving ResultsPerformance Measurement:  Rules For Driving Results
Performance Measurement: Rules For Driving Results
 
Managing Effectively
Managing EffectivelyManaging Effectively
Managing Effectively
 
Sixsigmadtifactsheet
SixsigmadtifactsheetSixsigmadtifactsheet
Sixsigmadtifactsheet
 
Make Change That Lasts
Make Change That LastsMake Change That Lasts
Make Change That Lasts
 
Creating Performance Based Culture trough Performance Management Systems
Creating Performance Based Culture trough Performance Management Systems Creating Performance Based Culture trough Performance Management Systems
Creating Performance Based Culture trough Performance Management Systems
 
Assignment on Performance management
Assignment on Performance managementAssignment on Performance management
Assignment on Performance management
 
Corporate performance management how_to_build_a_better_organization_through_...
Corporate performance management  how_to_build_a_better_organization_through_...Corporate performance management  how_to_build_a_better_organization_through_...
Corporate performance management how_to_build_a_better_organization_through_...
 

Ähnlich wie Reaching Business Goals with Value Adding CMMI Assessments

Rinat Galyautdinov: Measurement concept of operation
Rinat Galyautdinov: Measurement concept of operationRinat Galyautdinov: Measurement concept of operation
Rinat Galyautdinov: Measurement concept of operationRinat Galyautdinov
 
Steps to blueprint successful erp projects immortal
Steps to blueprint successful erp projects immortalSteps to blueprint successful erp projects immortal
Steps to blueprint successful erp projects immortalImmortal Technologies
 
How to-measure-the-return-on-your-hr-investment
How to-measure-the-return-on-your-hr-investmentHow to-measure-the-return-on-your-hr-investment
How to-measure-the-return-on-your-hr-investmentAhmed A.samra
 
Encouraging Business Excellence through Continuous Improvement
Encouraging Business Excellence through Continuous ImprovementEncouraging Business Excellence through Continuous Improvement
Encouraging Business Excellence through Continuous ImprovementGroup50 Consulting
 
Optimizing Organizational Performance by Managing Project Benefits
Optimizing Organizational Performance by Managing Project BenefitsOptimizing Organizational Performance by Managing Project Benefits
Optimizing Organizational Performance by Managing Project BenefitsUMT
 
2023 - English_CSMP Guide.pdf
2023 - English_CSMP Guide.pdf2023 - English_CSMP Guide.pdf
2023 - English_CSMP Guide.pdfGregorPetri2
 
İTÜ İşletme ve Teknoloji Yönetimi - Lojistik & Tedarik Zinciri Yönetimi
İTÜ İşletme ve Teknoloji Yönetimi - Lojistik & Tedarik Zinciri YönetimiİTÜ İşletme ve Teknoloji Yönetimi - Lojistik & Tedarik Zinciri Yönetimi
İTÜ İşletme ve Teknoloji Yönetimi - Lojistik & Tedarik Zinciri YönetimiElif Tuygan Arslançeri
 
Business Analysis in A Nutshell
Business Analysis in A Nutshell  Business Analysis in A Nutshell
Business Analysis in A Nutshell Goran Mutalib
 
Chick-fil-A Training Program DevelopmentRunning head .docx
Chick-fil-A Training Program DevelopmentRunning head .docxChick-fil-A Training Program DevelopmentRunning head .docx
Chick-fil-A Training Program DevelopmentRunning head .docxchristinemaritza
 
The Metrics of Project Management Performance and PMBOK
The Metrics of Project Management Performance and PMBOKThe Metrics of Project Management Performance and PMBOK
The Metrics of Project Management Performance and PMBOKLiana Underwood
 
How Do You Measure The KM Maturity Of Your Organization Final Ver.
How Do You Measure The KM Maturity Of Your Organization Final Ver.How Do You Measure The KM Maturity Of Your Organization Final Ver.
How Do You Measure The KM Maturity Of Your Organization Final Ver.Art Schlussel
 
Mobile Marketing Business Case
Mobile Marketing Business CaseMobile Marketing Business Case
Mobile Marketing Business CaseDemand Metric
 
Cracking The Business Case For Learning Code
Cracking The Business Case For Learning CodeCracking The Business Case For Learning Code
Cracking The Business Case For Learning CodeWorkforce Group
 
Assessment 1 – Case Study Project Overview and context You.docx
Assessment 1 – Case Study Project Overview and context You.docxAssessment 1 – Case Study Project Overview and context You.docx
Assessment 1 – Case Study Project Overview and context You.docxgalerussel59292
 
Practical Suggestions to Successfully Adopt the CMMI V2.0 Development for Bet...
Practical Suggestions to Successfully Adopt the CMMI V2.0 Development for Bet...Practical Suggestions to Successfully Adopt the CMMI V2.0 Development for Bet...
Practical Suggestions to Successfully Adopt the CMMI V2.0 Development for Bet...Dr. Mustafa Değerli
 
TOTAL QUALITY MANAGEMENT
TOTAL QUALITY MANAGEMENTTOTAL QUALITY MANAGEMENT
TOTAL QUALITY MANAGEMENTReaz Uddin
 
Session Five- DETERMINING PROJECT PROGRESS AND RESULTS .pptx
Session Five- DETERMINING PROJECT PROGRESS AND RESULTS .pptxSession Five- DETERMINING PROJECT PROGRESS AND RESULTS .pptx
Session Five- DETERMINING PROJECT PROGRESS AND RESULTS .pptxyuduf1
 

Ähnlich wie Reaching Business Goals with Value Adding CMMI Assessments (20)

Rinat Galyautdinov: Measurement concept of operation
Rinat Galyautdinov: Measurement concept of operationRinat Galyautdinov: Measurement concept of operation
Rinat Galyautdinov: Measurement concept of operation
 
Steps to blueprint successful erp projects immortal
Steps to blueprint successful erp projects immortalSteps to blueprint successful erp projects immortal
Steps to blueprint successful erp projects immortal
 
Developing quality program
Developing quality programDeveloping quality program
Developing quality program
 
How to-measure-the-return-on-your-hr-investment
How to-measure-the-return-on-your-hr-investmentHow to-measure-the-return-on-your-hr-investment
How to-measure-the-return-on-your-hr-investment
 
Encouraging Business Excellence through Continuous Improvement
Encouraging Business Excellence through Continuous ImprovementEncouraging Business Excellence through Continuous Improvement
Encouraging Business Excellence through Continuous Improvement
 
Optimizing Organizational Performance by Managing Project Benefits
Optimizing Organizational Performance by Managing Project BenefitsOptimizing Organizational Performance by Managing Project Benefits
Optimizing Organizational Performance by Managing Project Benefits
 
2023 - English_CSMP Guide.pdf
2023 - English_CSMP Guide.pdf2023 - English_CSMP Guide.pdf
2023 - English_CSMP Guide.pdf
 
İTÜ İşletme ve Teknoloji Yönetimi - Lojistik & Tedarik Zinciri Yönetimi
İTÜ İşletme ve Teknoloji Yönetimi - Lojistik & Tedarik Zinciri YönetimiİTÜ İşletme ve Teknoloji Yönetimi - Lojistik & Tedarik Zinciri Yönetimi
İTÜ İşletme ve Teknoloji Yönetimi - Lojistik & Tedarik Zinciri Yönetimi
 
Business Analysis in A Nutshell
Business Analysis in A Nutshell  Business Analysis in A Nutshell
Business Analysis in A Nutshell
 
Chick-fil-A Training Program DevelopmentRunning head .docx
Chick-fil-A Training Program DevelopmentRunning head .docxChick-fil-A Training Program DevelopmentRunning head .docx
Chick-fil-A Training Program DevelopmentRunning head .docx
 
The Metrics of Project Management Performance and PMBOK
The Metrics of Project Management Performance and PMBOKThe Metrics of Project Management Performance and PMBOK
The Metrics of Project Management Performance and PMBOK
 
Mb0049 project management
Mb0049 project managementMb0049 project management
Mb0049 project management
 
How Do You Measure The KM Maturity Of Your Organization Final Ver.
How Do You Measure The KM Maturity Of Your Organization Final Ver.How Do You Measure The KM Maturity Of Your Organization Final Ver.
How Do You Measure The KM Maturity Of Your Organization Final Ver.
 
Mobile Marketing Business Case
Mobile Marketing Business CaseMobile Marketing Business Case
Mobile Marketing Business Case
 
Cracking The Business Case For Learning Code
Cracking The Business Case For Learning CodeCracking The Business Case For Learning Code
Cracking The Business Case For Learning Code
 
Assessment 1 – Case Study Project Overview and context You.docx
Assessment 1 – Case Study Project Overview and context You.docxAssessment 1 – Case Study Project Overview and context You.docx
Assessment 1 – Case Study Project Overview and context You.docx
 
Practical Suggestions to Successfully Adopt the CMMI V2.0 Development for Bet...
Practical Suggestions to Successfully Adopt the CMMI V2.0 Development for Bet...Practical Suggestions to Successfully Adopt the CMMI V2.0 Development for Bet...
Practical Suggestions to Successfully Adopt the CMMI V2.0 Development for Bet...
 
TOTAL QUALITY MANAGEMENT
TOTAL QUALITY MANAGEMENTTOTAL QUALITY MANAGEMENT
TOTAL QUALITY MANAGEMENT
 
Session Five- DETERMINING PROJECT PROGRESS AND RESULTS .pptx
Session Five- DETERMINING PROJECT PROGRESS AND RESULTS .pptxSession Five- DETERMINING PROJECT PROGRESS AND RESULTS .pptx
Session Five- DETERMINING PROJECT PROGRESS AND RESULTS .pptx
 
Topic for Organization and Management 'Planning'
Topic for Organization and Management 'Planning'Topic for Organization and Management 'Planning'
Topic for Organization and Management 'Planning'
 

Mehr von Ben Linders

Psychological Safety in Teams - FlowCon France 2024 - Ben Linders
Psychological Safety in Teams - FlowCon France 2024 - Ben LindersPsychological Safety in Teams - FlowCon France 2024 - Ben Linders
Psychological Safety in Teams - FlowCon France 2024 - Ben LindersBen Linders
 
Why people hate working in Agile teams - QA Challenge Accepted 2023 - Ben Lin...
Why people hate working in Agile teams - QA Challenge Accepted 2023 - Ben Lin...Why people hate working in Agile teams - QA Challenge Accepted 2023 - Ben Lin...
Why people hate working in Agile teams - QA Challenge Accepted 2023 - Ben Lin...Ben Linders
 
Improving Your Testing Skills and Practices with Gamification - Testing Unite...
Improving Your Testing Skills and Practices with Gamification - Testing Unite...Improving Your Testing Skills and Practices with Gamification - Testing Unite...
Improving Your Testing Skills and Practices with Gamification - Testing Unite...Ben Linders
 
Start up distributed teams online - Mini XP days 2022 - Ben Linders
Start up distributed teams online - Mini XP days 2022 - Ben LindersStart up distributed teams online - Mini XP days 2022 - Ben Linders
Start up distributed teams online - Mini XP days 2022 - Ben LindersBen Linders
 
Increasing psychological safety in agile teams - Agile humans lean coffee 202...
Increasing psychological safety in agile teams - Agile humans lean coffee 202...Increasing psychological safety in agile teams - Agile humans lean coffee 202...
Increasing psychological safety in agile teams - Agile humans lean coffee 202...Ben Linders
 
Improving your quality and testing skills with gamification - Spring 2021 Onl...
Improving your quality and testing skills with gamification - Spring 2021 Onl...Improving your quality and testing skills with gamification - Spring 2021 Onl...
Improving your quality and testing skills with gamification - Spring 2021 Onl...Ben Linders
 
How agile are you? - Agile New England 2021 - Ben Linders
How agile are you? - Agile New England 2021 - Ben LindersHow agile are you? - Agile New England 2021 - Ben Linders
How agile are you? - Agile New England 2021 - Ben LindersBen Linders
 
Mini workshop collaborative problem solving - OOP 2021 - Ben Linders
Mini workshop collaborative problem solving - OOP 2021 - Ben LindersMini workshop collaborative problem solving - OOP 2021 - Ben Linders
Mini workshop collaborative problem solving - OOP 2021 - Ben LindersBen Linders
 
Futurespective on Software Development in 2040 - Agile Tour Brussels 2020 - B...
Futurespective on Software Development in 2040 - Agile Tour Brussels 2020 - B...Futurespective on Software Development in 2040 - Agile Tour Brussels 2020 - B...
Futurespective on Software Development in 2040 - Agile Tour Brussels 2020 - B...Ben Linders
 
How agile are you - Agile Tour London 2020 - Ben Linders
How agile are you - Agile Tour London 2020 - Ben LindersHow agile are you - Agile Tour London 2020 - Ben Linders
How agile are you - Agile Tour London 2020 - Ben LindersBen Linders
 
Mini workshop retrospecting your retrospectives - Experience Agile 2020 - Be...
Mini workshop retrospecting your retrospectives  - Experience Agile 2020 - Be...Mini workshop retrospecting your retrospectives  - Experience Agile 2020 - Be...
Mini workshop retrospecting your retrospectives - Experience Agile 2020 - Be...Ben Linders
 
Webinar enhancing quality and testing in agile teams - PractiTest - Ben Linders
Webinar enhancing quality and testing in agile teams - PractiTest - Ben LindersWebinar enhancing quality and testing in agile teams - PractiTest - Ben Linders
Webinar enhancing quality and testing in agile teams - PractiTest - Ben LindersBen Linders
 
Futurespective on software development in 2040 - Aginext - Ben Linders
Futurespective on software development in 2040 - Aginext - Ben LindersFuturespective on software development in 2040 - Aginext - Ben Linders
Futurespective on software development in 2040 - Aginext - Ben LindersBen Linders
 
Leading for Self-organization - Stretch 2020 - Ben Linders
Leading for Self-organization - Stretch 2020 - Ben LindersLeading for Self-organization - Stretch 2020 - Ben Linders
Leading for Self-organization - Stretch 2020 - Ben LindersBen Linders
 
Pecha Kucha How to screw up your agile retrospective big time - Ben Linders -...
Pecha Kucha How to screw up your agile retrospective big time - Ben Linders -...Pecha Kucha How to screw up your agile retrospective big time - Ben Linders -...
Pecha Kucha How to screw up your agile retrospective big time - Ben Linders -...Ben Linders
 
Agile Retrospectives to the Next Level - Organizational Agility - OOP 2020 - ...
Agile Retrospectives to the Next Level - Organizational Agility - OOP 2020 - ...Agile Retrospectives to the Next Level - Organizational Agility - OOP 2020 - ...
Agile Retrospectives to the Next Level - Organizational Agility - OOP 2020 - ...Ben Linders
 
Learning at Scale - FlowCon France 2019 - Ben Linders
Learning at Scale - FlowCon France 2019 - Ben LindersLearning at Scale - FlowCon France 2019 - Ben Linders
Learning at Scale - FlowCon France 2019 - Ben LindersBen Linders
 
Organizational agility: Taking retrospectives to the next level - DevOpsCon M...
Organizational agility: Taking retrospectives to the next level - DevOpsCon M...Organizational agility: Taking retrospectives to the next level - DevOpsCon M...
Organizational agility: Taking retrospectives to the next level - DevOpsCon M...Ben Linders
 
Dealing effectively with impediments - Agile Management Congress 2019 - Ben L...
Dealing effectively with impediments - Agile Management Congress 2019 - Ben L...Dealing effectively with impediments - Agile Management Congress 2019 - Ben L...
Dealing effectively with impediments - Agile Management Congress 2019 - Ben L...Ben Linders
 
Teams what is in it for me - Agile Portugal 2019 - Ben Linders
Teams what is in it for me - Agile Portugal 2019 - Ben LindersTeams what is in it for me - Agile Portugal 2019 - Ben Linders
Teams what is in it for me - Agile Portugal 2019 - Ben LindersBen Linders
 

Mehr von Ben Linders (20)

Psychological Safety in Teams - FlowCon France 2024 - Ben Linders
Psychological Safety in Teams - FlowCon France 2024 - Ben LindersPsychological Safety in Teams - FlowCon France 2024 - Ben Linders
Psychological Safety in Teams - FlowCon France 2024 - Ben Linders
 
Why people hate working in Agile teams - QA Challenge Accepted 2023 - Ben Lin...
Why people hate working in Agile teams - QA Challenge Accepted 2023 - Ben Lin...Why people hate working in Agile teams - QA Challenge Accepted 2023 - Ben Lin...
Why people hate working in Agile teams - QA Challenge Accepted 2023 - Ben Lin...
 
Improving Your Testing Skills and Practices with Gamification - Testing Unite...
Improving Your Testing Skills and Practices with Gamification - Testing Unite...Improving Your Testing Skills and Practices with Gamification - Testing Unite...
Improving Your Testing Skills and Practices with Gamification - Testing Unite...
 
Start up distributed teams online - Mini XP days 2022 - Ben Linders
Start up distributed teams online - Mini XP days 2022 - Ben LindersStart up distributed teams online - Mini XP days 2022 - Ben Linders
Start up distributed teams online - Mini XP days 2022 - Ben Linders
 
Increasing psychological safety in agile teams - Agile humans lean coffee 202...
Increasing psychological safety in agile teams - Agile humans lean coffee 202...Increasing psychological safety in agile teams - Agile humans lean coffee 202...
Increasing psychological safety in agile teams - Agile humans lean coffee 202...
 
Improving your quality and testing skills with gamification - Spring 2021 Onl...
Improving your quality and testing skills with gamification - Spring 2021 Onl...Improving your quality and testing skills with gamification - Spring 2021 Onl...
Improving your quality and testing skills with gamification - Spring 2021 Onl...
 
How agile are you? - Agile New England 2021 - Ben Linders
How agile are you? - Agile New England 2021 - Ben LindersHow agile are you? - Agile New England 2021 - Ben Linders
How agile are you? - Agile New England 2021 - Ben Linders
 
Mini workshop collaborative problem solving - OOP 2021 - Ben Linders
Mini workshop collaborative problem solving - OOP 2021 - Ben LindersMini workshop collaborative problem solving - OOP 2021 - Ben Linders
Mini workshop collaborative problem solving - OOP 2021 - Ben Linders
 
Futurespective on Software Development in 2040 - Agile Tour Brussels 2020 - B...
Futurespective on Software Development in 2040 - Agile Tour Brussels 2020 - B...Futurespective on Software Development in 2040 - Agile Tour Brussels 2020 - B...
Futurespective on Software Development in 2040 - Agile Tour Brussels 2020 - B...
 
How agile are you - Agile Tour London 2020 - Ben Linders
How agile are you - Agile Tour London 2020 - Ben LindersHow agile are you - Agile Tour London 2020 - Ben Linders
How agile are you - Agile Tour London 2020 - Ben Linders
 
Mini workshop retrospecting your retrospectives - Experience Agile 2020 - Be...
Mini workshop retrospecting your retrospectives  - Experience Agile 2020 - Be...Mini workshop retrospecting your retrospectives  - Experience Agile 2020 - Be...
Mini workshop retrospecting your retrospectives - Experience Agile 2020 - Be...
 
Webinar enhancing quality and testing in agile teams - PractiTest - Ben Linders
Webinar enhancing quality and testing in agile teams - PractiTest - Ben LindersWebinar enhancing quality and testing in agile teams - PractiTest - Ben Linders
Webinar enhancing quality and testing in agile teams - PractiTest - Ben Linders
 
Futurespective on software development in 2040 - Aginext - Ben Linders
Futurespective on software development in 2040 - Aginext - Ben LindersFuturespective on software development in 2040 - Aginext - Ben Linders
Futurespective on software development in 2040 - Aginext - Ben Linders
 
Leading for Self-organization - Stretch 2020 - Ben Linders
Leading for Self-organization - Stretch 2020 - Ben LindersLeading for Self-organization - Stretch 2020 - Ben Linders
Leading for Self-organization - Stretch 2020 - Ben Linders
 
Pecha Kucha How to screw up your agile retrospective big time - Ben Linders -...
Pecha Kucha How to screw up your agile retrospective big time - Ben Linders -...Pecha Kucha How to screw up your agile retrospective big time - Ben Linders -...
Pecha Kucha How to screw up your agile retrospective big time - Ben Linders -...
 
Agile Retrospectives to the Next Level - Organizational Agility - OOP 2020 - ...
Agile Retrospectives to the Next Level - Organizational Agility - OOP 2020 - ...Agile Retrospectives to the Next Level - Organizational Agility - OOP 2020 - ...
Agile Retrospectives to the Next Level - Organizational Agility - OOP 2020 - ...
 
Learning at Scale - FlowCon France 2019 - Ben Linders
Learning at Scale - FlowCon France 2019 - Ben LindersLearning at Scale - FlowCon France 2019 - Ben Linders
Learning at Scale - FlowCon France 2019 - Ben Linders
 
Organizational agility: Taking retrospectives to the next level - DevOpsCon M...
Organizational agility: Taking retrospectives to the next level - DevOpsCon M...Organizational agility: Taking retrospectives to the next level - DevOpsCon M...
Organizational agility: Taking retrospectives to the next level - DevOpsCon M...
 
Dealing effectively with impediments - Agile Management Congress 2019 - Ben L...
Dealing effectively with impediments - Agile Management Congress 2019 - Ben L...Dealing effectively with impediments - Agile Management Congress 2019 - Ben L...
Dealing effectively with impediments - Agile Management Congress 2019 - Ben L...
 
Teams what is in it for me - Agile Portugal 2019 - Ben Linders
Teams what is in it for me - Agile Portugal 2019 - Ben LindersTeams what is in it for me - Agile Portugal 2019 - Ben Linders
Teams what is in it for me - Agile Portugal 2019 - Ben Linders
 

Kürzlich hochgeladen

Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...Neo4j
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Miguel Araújo
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Drew Madelung
 
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024The Digital Insurer
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationSafe Software
 
Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024The Digital Insurer
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptxHampshireHUG
 
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxThe Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxMalak Abu Hammad
 
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking MenDelhi Call girls
 
Top 5 Benefits OF Using Muvi Live Paywall For Live Streams
Top 5 Benefits OF Using Muvi Live Paywall For Live StreamsTop 5 Benefits OF Using Muvi Live Paywall For Live Streams
Top 5 Benefits OF Using Muvi Live Paywall For Live StreamsRoshan Dwivedi
 
Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...
Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...
Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...gurkirankumar98700
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slidevu2urc
 
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...Neo4j
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking MenDelhi Call girls
 
Slack Application Development 101 Slides
Slack Application Development 101 SlidesSlack Application Development 101 Slides
Slack Application Development 101 Slidespraypatel2
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024The Digital Insurer
 
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfThe Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfEnterprise Knowledge
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking MenDelhi Call girls
 
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure serviceWhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure servicePooja Nehwal
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)Gabriella Davis
 

Kürzlich hochgeladen (20)

Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
 
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
 
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxThe Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptx
 
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men
 
Top 5 Benefits OF Using Muvi Live Paywall For Live Streams
Top 5 Benefits OF Using Muvi Live Paywall For Live StreamsTop 5 Benefits OF Using Muvi Live Paywall For Live Streams
Top 5 Benefits OF Using Muvi Live Paywall For Live Streams
 
Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...
Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...
Kalyanpur ) Call Girls in Lucknow Finest Escorts Service 🍸 8923113531 🎰 Avail...
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
 
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
 
Slack Application Development 101 Slides
Slack Application Development 101 SlidesSlack Application Development 101 Slides
Slack Application Development 101 Slides
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024
 
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfThe Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
 
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure serviceWhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
 

Reaching Business Goals with Value Adding CMMI Assessments

  • 1. Reaching Business Goals with Value Adding CMMI Assessments, by Ben Linders Ericsson EuroLab Netherlands (ELN) is a company within Ericsson, the world's leading supplier in telecommunications. They have facilities in the south (Rijen) and in the east (Enschede, and Emmen). In total about 1100 employees are working for this company, almost all are directly involved in software and hardware development. The scope of this article is about the design unit in Rijen, and in particular about the Charging Competence Centre department. This department employs about 160 persons, developing charging systems for telephone exchanges. The technology is UNIX and NT based systems, using UML based Object Oriented development with C++ and JAVA, deploying mostly Rational tools This paper describes how to apply the CMMI in a business-focused way, by doing frequent small assessments. Goals are input to the assessments, and a matrix is used to assure both organisation and process area coverage. The results of assessments integrate smoothly with operational targets of existing groups within the organisation. Practical tools support the frequent assessments, and experience with the CMMI is used to optimize the assessment and improvement cycle, resulting in continuously improving the performance of the organisation based on its goals. 1 History: CMM as a rating model CMM assessments have been done for quite some years within Ericsson. This paragraph describes the history, a more detailed description can be found in a separate paper on this subject [ref 1]. Ericsson was one of the early adopters of the CMM model. The process focus, and the concept of measurement based improvement were the unique selling points why the model was chosen. The first assessment, back in 1994, showed Ericsson Rijen was on level 1, which was no surprise. An improvement program was initiated, which established a process support organisation that rolled out all level 2 processes in the projects. The next assessment in 1995 showed that the Research and Development Unit of Ericsson Rijen was on level 3, the first company within Ericsson worldwide. Reaching level 3 was a big reward for the organisation. As a result, the focus changed, and operational goals such as meeting deadlines, combined with many vacancies demanded their toll. Projects became less focused on processes and measurements. Since the process support organisation remained, as did the culture to improve in most of the departments, the projects still managed to live up to the expectations. But things didn’t always go that smoothly… One department in Ericsson Rijen, the Charging Competence Centre, kept on believing in the CMM and its usefulness. It started their own program to come to level 4, by defining measurements and applying them in their projects [ref 2]. In June 1998, they made their first preparations towards a new assessment. Bill Curtis, associated with the Software Engineering Institute, stated that the Unix development department could be "among the world elite in one year". Finally, the department was re-assessed in November 1998, where it was found to be a professional "solid as a rock" level 3 organization, with parts of level 4 institutionalised. Reaching Business Goals with Value Adding CMMI Assessments, Ben Linders Page 1 of 12
  • 2. Having worked for quite some years on process improvements, and not getting the reward of being fully at level 4, came somewhat as a disappointment to this department. And it also confused them, as they didn’t just implement the practices of the CMM model. The approach had been from the goals and concepts of the model, implemented it in such a way that it made sense to do it. Though no process areas were excluded, not all practices were implemented. For some there were alternative solutions. But most of all they were confused because in their opinion the improvement programs had delivered results, being an organisation able to perform projects in a controlled way, and capable of improving where business goals required it. As a next step, the department didn’t start a new improvement program to fill the gaps found in the assessment. Instead, the approach focussing on business goals and selecting improvements that contribute to them was continued, regardless of CMM levels involved. This focus on CMM matched perfectly with operational goals set by the department, in the sense that CMM would contribute directly to the business. Along with this focus, the assessment approach had to change also. Full assessments were no longer useful, instead there was a need for small focused assessments which would come up with detailed findings. These assessments should be an instrument to be used next to audits and improvement sessions in projects. 2 Approach: Business Focused CMMI Assessments A new assessment method has been defined: Business-focused CMMI assessments. Such an assessment is a class C type assessment [ref 3], where the main purpose is to find strengths and weaknesses. The assessment consists of the following steps: • Assign Orderer and Customer • Select Goals • Define Organizational Scope • Determine Process Areas • Organise Assessment Team • Plan Assessment • Perform Assessment • Support Improvements The purpose of the first six steps is to make an assessment proposal. This document fully defines the assessment, and after approval it is used to perform the assessment and support the improvements. The steps will be discussed in detail in the following paragraphs. 2.1 Assign Orderer and Customer The first step is to define the interface between the assessment and the organisation. This interface consists of two persons: An orderer and a customer. They will play an essential role. The orderer has to be a member of the highest management of the organisation. The orderer is responsible for one or more goals that will selected for this assessment, for that reason (s)he is particularly interested that a good assessment will be done. Also, this person arranges funding for the assessment, including hours of the interviewees. It is also the person who has end Reaching Business Goals with Value Adding CMMI Assessments, Ben Linders Page 2 of 12
  • 3. The customer is the person who needs the result from the assessment: The findings. This person is responsible for implementing improvement actions. It can be the same person as the orderer. It must be clear, before the assessment is started, that the customer is able to take over the results of the assessment, and arrange the improvement actions. If there is no room in the organisation to improve, then there is no need to assess either. Both the orderer and customer must be clearly stated in the assessment proposal. They will also attend kick-off and final meeting, where they personally show their stakes in this assessment and what they require the assessment to deliver. 2.2 Select Goals For every assessment, there must be a business reason to do it. This reason has to do with one or more goals of the organisation, or with strategies performed to reach the goal(s). This step results in a list of goals and strategies that have to be assessed. For instance, an organisation can have the goal: “All projects must be finished on the planned end date”. To reach this goal, there are strategies such as improvement programs for project planning, and lead-time improvement. When it is uncertain if the strategies will assure that the goal will be met within the defined time frame (usually one or more calendar years), then an assessment is useful to investigate it. Goal setting is a complex, but vital activity. This assessment approach requires that the goals are clear, but sometimes it turns out that when discussing them, additional clarification is needed. Also priorities between goals are not always sufficiently defined. It is of the utmost importance that there are no more uncertainties about the goals before the assessment planning is started. An assessment focuses on a subset of the goals, in order to come with findings specifically related to reaching those goals. Assessing against all goals of an organisation results in too many findings, where priorities will have to be set after an assessment. In such a case, findings will be thrown away and valuable assessment time has been wasted. More important: An expectation was given that the problems would be solved. Not living up to this expectation can seriously hamper future assessments and improvement programs. The experience is that assessments at the Charging Competence Centre have been focussing on 1 to 2 main goals. Further on in this paper it will become clear how it was assured that all the goals were covered by assessments over time. 2.3 Define Organizational scope The result of the previous step is a list of goals and strategies. In this step the part of the organisation involved in reaching these goals or performing the strategies must be identified. The result will be a list of groups and roles/functions, which have to be interviewed. Reaching Business Goals with Value Adding CMMI Assessments, Ben Linders Page 3 of 12
  • 4. A goal is assigned to a responsible person or group in the organisation. Also, other persons and groups are performing work that relates to the goal. The boundary of the assessment must be defined such that it includes the right parts of the organisation, nothing more, and nothing less. A misunderstanding is that in an assessment, every group of the organisation has to be represented. This should not be the case. If groups are not involved in activities related to the goals to be assessed, then there is no need to interview them. Any findings from this group would be put aside and considered irrelevant, because they do not endanger the goal. So why waste their time, and yours? So the scope includes only groups giving input to the work, performing the work, and requiring output from the work, where the work is that part of all work done in the department that is required to reach the goal or perform the strategy. 2.4 Determine Process Areas Now that the scope of the assessment is defined from the organisation’s perspective, let’s look at the scope from the model’s perspective. That is, given the organisational scope there is usually need to assess only a selected set of process areas from the CMMI model. Looking at the same example that the organisational goal to investigate is lead-time precision, and the groups involved would be two product development teams running projects, and a staff group on configuration management. It was decided in the organisational scope step that the assessment should only interview the project management staff, and not the engineers. In that case, it would be useful to select process areas mainly from the project management cluster, and additionally add configuration management from the support cluster. There is no need to assess process areas from engineering or process management clusters, since no activities from these process areas are performed within the organisational boundaries of the assessment. Again this prevents coming up with findings which are not goal related. Some experience from the Charging Competence Centre department: In the assessments done in 2000 and 2001, the number of process areas varied a lot. There were assessments with only 1 (Decision Analysis and Resolution) or 2 (Requirements Management/development) process areas, but there have also been assessments with 7 areas. The latter is about the maximum, when it becomes more then one should return to the first step and try to reduce the number of goals involved. There has been no assessment covering all process areas on a specific level. The result of this step is a list of process areas that will be assessed. With this step, the total scope of the assessment is defined. 2.5 Organise Assessment Team Until now, there has only been an Assessment Team Leader. This person has done all previous steps, together with persons from the line organisation. Now the team can be assembled, so that they can take the next steps together. A team consists of a team leader, and 1 to 2 assessors. The minimum relates to the fact that it is essential to have at least two views on the findings. At any time an assessor can discuss preliminary findings, and do a peer review before presenting them to the interviewees. The Reaching Business Goals with Value Adding CMMI Assessments, Ben Linders Page 4 of 12
  • 5. maximum relates to the number of process areas and communication; adding more people (unless they are expert in a certain area) doesn’t improve the quality nor the time needed. In an assessment team there has to be at least 1 member from the assessed organisation. The role of this “local” member is to assure that the right people are interviewed, and that potential findings within the goals are derived, which are formulated using terminology known and used within the organisation. This helps coping with the so-called “CMMI speak problem”, where an organisation doesn’t understand the questions and cannot use the findings, because of terms used from the CMMI model that they do not understand. Assessment team members must have excellent communication skills, have knowledge both in the way an assessment is conducted and in the process areas being assessed, and must have sufficient time available within the time frame. The latter is very important, as an assessment has a fixed time schedule from start to finish. You cannot postpone a consensus or a final meeting if you run out of time, re-planning meetings with all the people involved will almost always mean that an assessment slips, and then loses the momentum in the organisation. Roughly the time needed in an assessment is 40-60 hours for a team leader, and 20-40 hours for an assessor. This depends of course on experience, and on the scope of the assessment. 2.6 Plan Assessment The next step now that the part of the organisation to be interviewed and the process areas are defined, is that the assessment team makes a detailed plan of the assessment. This plan includes: • Select Interviewees • Define Timeplan 2.6.1 Select Interviewees The interviewees have to be selected within the defined organisation scope (groups and roles/functions) defined earlier. The purpose of selecting interviewees is assuring that every process area is sufficiently covered. That is, the group of people interviewed should contain sufficient persons that perform the activities of the process areas from several angles. Not only persons doing the work should be interviewed, but also persons ordering (and being responsible for) the work, and persons depending on the outcome of the work, should be interviewed. This can be compared to a 360-degree appraisal, where a person’s boss, customers and people below him/her have to give input on the performance. Given the previous example (lead-time in project management teams and configuration management staff, with process areas from project management and configuration management) the following roles could be interviewed: • Project managers Performing project management activities • Team leaders Idem, but for the design teams • Configuration managers Performing configuration management activities • Product managers Customers of a project • Members from maintenance Taking over work and products from a project Reaching Business Goals with Value Adding CMMI Assessments, Ben Linders Page 5 of 12
  • 6. In each role there should be sufficient people interviewed, usually that is 2-4 persons. This depends on how far the activity is performed in a uniform way by different people or groups, variance in experience and competence levels, different customers with unique expectations, etc. When in doubt it is usually better to include an additional person initially, it is easier to exclude that person later on when it appears that (s)he is not needed than to add an additional person when coverage is not enough. On the other hand, some persons might perform several roles. Again some experience from the Charging Competence Centre: Assessments done until now have interviewed a minimum of 7 persons, and a maximum of 19. If the number becomes too big, then reconsider the goals, organisational groups, or number of process areas involved. It may be useful to split up an assessment, in order to have more insight into specific goals, groups or process areas. 2.6.2 Define Timeplan For performing the assessment, several meetings have to be planned and deadlines have to be set. More information about the meetings can be found in step “perform assessment”, in this step only the people attending the meeting and the timeline is defined. Please note that all meetings are planned in advance, and that deadlines are fixed in an assessment. The kick off meeting is attended by everybody involved in the assessment, so the interviewees, orderer, customer, and the assessment team. The overview presentation is optional, and open to anybody interested in an explanation of the process area. It can also include people from outside the assessment. A deadline must be set for submittal of the questionnaires. On this date, all interviewees must have returned their questionnaire, and it must be checked for completeness and any unclearness. The consensus meeting is only for the interviewees and the assessment team, others are not allowed due to confidentiality reasons. The final meeting is meant for everybody in the assessment. However, if the orderer doesn’t object, then others may join the meeting to hear the results of the assessment at first hand. Please note that the purpose is to communicate strengths and weaknesses in order to improve, and not to judge. At Ericsson we have experimented during the years with the time plan for this kind of assessment. The scheme below has appeared to be most effective, both in coping with organisational pressure and keeping people mentally involved in the assessment. Week T – 4 weeks: First proposal for the assessment Week T – 2 weeks: Final proposal, and all further dates and interviewees fixed Week T: Kick off and distribution of assessment questionnaires Week T + 1 week: Overview presentation of the process areas Week T + 2 weeks: Deadline for having the questionnaires returned Week T + 3 ½ weeks: Consensus meeting Week T + 4 weeks: Final presentation Reaching Business Goals with Value Adding CMMI Assessments, Ben Linders Page 6 of 12
  • 7. This schedule gives two weeks for the interviewee to answer all the questions, and 1 ½ week for the assessment team to come up with findings. If planned in advance, and sufficient commitment from the organisation is available, this has proved to be feasible. 2.7 Perform Assessment An assessment starts with a kick off meeting. In this meeting the assessment is presented to the interviewees; after this meeting they should know what they could expect from the assessment and what will be their role. In the kick off meeting, both the orderer and the customer should be present. They should tell the audience why they want the assessment, and what kind of results they expect. Also they should ask the interviewees to be open and honest, and that they will value all input and accept the results from the assessment. The assessment team leader must assure confidentiality and non attribution, that is that the results sent in by the interviewees will only be known to the assessment team, and that finding will be stated in such a way that they cannot be attributed to individuals or projects. After the kick off, questionnaires will be sent out to the interviewees to collect information. The questionnaires are based on the goals and activities of the CMMI model. To enable them to fill in these questionnaires, it is good to organise an overview presentation of the process areas so that they can better understand the goals and activities of the model. This presentation is not mandatory, but highly recommended. Note that we deliberately do not use interview sessions, instead we use consensus meetings. Defining findings is a creative, and very difficult and complex part of an assessment. It is done by going through ratings and comments from the interviewee, and look for common opinions, contradictions, patterns, etc. Drivers for the findings are of course the goals and strategies that are being investigated. Competence in the process areas that are investigated is essential. It is also important to know the roles that people perform, and sometimes also their background, favorite issues, frustrations, etc. The result will be an overview of strengths, weaknesses, and open issues (things which are not yet clear) per process area. As a next step a consensus meeting is held. In this meeting the preliminary findings are presented, and feedback is sought to verify if these findings are correct, complete, and accepted by the organisation. As a rule, every interviewee has to be in the consensus meeting; for those who cannot attend a private meeting has to be arranged to go through the findings. You cannot risk coming up with findings, which are not accepted by any of the interviewees, and thus will be considered questionable after the assessment. Also, since the assessment is not actively searching for evidence, but focuses on opinions of the organisation, it is essential that the consensus meeting results in a common view of the main strengths and weaknesses of the organisation. At the end of an assessment, a final presentation is given. For all process areas the strengths and weaknesses are presented. In this meeting, the orderer and the customer of the assessment must be present to officially accept the results and reconfirm the organisations commitment to do whatever is needed to solve the findings. In this meeting, the interviewees are thanked for their contribution. This final meeting is attended by all the interviewees, the orderer and customer. But it is also good to invite others for which the results can be useful information. Basically, this meeting is used to buy in to the organisation for the needed improvements. Reaching Business Goals with Value Adding CMMI Assessments, Ben Linders Page 7 of 12
  • 8. 2.8 Support Improvements In most assessment methods, the assessment is finished after the final meeting, and the final report. However in this method it is required that the results of the assessment are represented to the customer, in order to define actions. For that purpose, a meeting is planned with the assessment team and the customer. Others, who can assist in defining or execute improvements, are also invited to the meeting. Together they go through the findings, and define the needed improvement actions. Most actions should be within their scope of control, for actions outside a member of the meeting will have to assure proper handover to a responsible. Please note that actions are not only defined for weaknesses, but also for strengths. An organisation that only works on improving weaknesses will never be more then average when compared to other organisations. In order to succeed reaching business goals, an organisation should also have strengths in which it differentiates with others. The resulting actions of the meeting will have to be assigned to either line or project bodies. The assessment team will not do follow up on the actions, but they will remain available if there are further issues regarding the findings. In some occasions, a member of the assessment team did a follow presentation on the findings some months after the assessment, to check the effect of the actions. This can be useful, on the one hand it is a good check if and how things are progressing, and on the other hand it is a measurement to check whether the findings and the defined actions are still considered useful. When a substantial part of the actions has been done, it will be good to re-assess the organisation in order to verify if the improvements have been successful. For that purpose a new assessment can be defined, repeating the process as has been described in these 8 steps. Given the fact that there are frequent assessments, a new assessment reuses results from a previous assessment, like presentations, tools, etc. More detail on this will be given in the next chapter. 3 Implementation: Tools No method is complete without tools to support it efficiently. For this assessment method, some tools have been developed that supported the roll out and acceptance of findings, and have turned it into an efficient way of improving the organisation. The term tools is used in a broad sense. What is meant is a combination of templates, checklists, etc. The following tools are used: • Assessment matrix • Questionnaires • Integrated organisation Each tool will be explained in more detail. It is important however that a tool never exists on its own, it is based on a method. On one hand this implies that when an organisation has different assessment methods, the tools may not be useful. On the other hand, there may be other, more useful tools for an organisation even when the same method is used. Be sure that the first focus is on the method, and then on the tools, not visa versa. Reaching Business Goals with Value Adding CMMI Assessments, Ben Linders Page 8 of 12
  • 9. 3.1 Assessment matrix In an organisation that is continuously improving, there will be frequent assessments focussing on different goals, process areas, and groups within the organisation. The assessment matrix provides an overview of all assessments done and planned. The main purpose is strategical planning of assessments, and assuring that over time everything considered important enough is assessed. CMM-I Assessment matrix ELN/V Q2/2001 Product Area XXX YYY ZZZ Process Area Project Management Project Planning Project Monitoring and Control Q4 2000 No Integrated Project Management Risk Management Q1 2000 No Quantitative Project Management Q1 2001 Support Configuration Management Q4 2000 No Causal Analyses and Resolution Q4 2000 Q3 2001 As you can see in the matrix, the two axes represent the organisational groups and the process areas. A combination of the two is covered in an assessment, which is based on one of more organisational goals. Each cell states when the last assessment has been done, or when a next assessment is planned. It is also possible to state that certain combinations will not be assessed. So this matrix provides an overview of all executed and planned assessment. There are some basic rules that are used at the Charging Competence Centre. Process Areas and groups that are considered most vital for the organisation are assessed twice a year (like “project management” and “engineering”). Other groups directly contributing to the goals will be assessed at least once every year (like “process management”). Staff units indirectly contributing may be assessed less frequently. Another rule is that every assessment has a main purpose. You either want to assess mainly to explore, i.e. to provide insight and come up with initial findings. Or the main purpose is to verify, so to confirm that improvement actions have resulted in effectively reaching the goals. If you already know what is wrong, and haven’t achieved sufficient improvement yet, then there is no need to do a new assessment. The assessment matrix has proved to be a valuable tool in assessment planning together with the management of our organisation. One of the main benefits has been that the matrix helped us to limit the number of assessments, and assure that an assessment will have a specific focus, while still knowing that in the long run nothing will be overlooked. Reaching Business Goals with Value Adding CMMI Assessments, Ben Linders Page 9 of 12
  • 10. 3.2 Questionnaires Ericsson has a long experience with questionnaires with light and class C assessments. Initially checklists and question forms have been used based on Word. The advantage was that they were very flexible, on the other hand collecting the questionnaires and arranging them in a way to come up with findings took a lot of effort. As one of the first pilots of the CMMI model, Ericsson did an official pilot of the SEI questionnaire tool in the 4th quarter of 2000. The tool has proved to be valuable, both for the interviewees and for the assessment team. Major advantage was the automated generation of questionnaires based on the model, and the collection of questionnaires into combined sheets to be used for definition of findings. Since it uses the SEI Word file of the CMMI model, upgrading to newer versions of the model should be fairly easy. The major drawbacks were that the tool was somewhat inflexible towards changes during the assessment, such as interviewees who are replaced or withdrawn from the assessment, or who cannot make the deadline. A project was done, together with the School for Information Technology in Breda (HIO) to develop a prototype of a new questionnaire tool. The main requirements were that it should support interviewees when filling in questions, by giving additional information and support, and that it should help an assessment team in organizing an assessment. The latter is done by have a set of questionnaires defined, which can be reused in multiple assessments. Also a tracking system is defined, which shows which assessment sheets have been returned, and who has to be reminded of their “duties”. A pilot is planned in Q2 2001, the results are not known yet. Conclusion: There is no uniform questionnaire yet, because different questionnaires all have their advantages and disadvantages. A strive is towards a more uniform approach, based on previous experiences. 3.3 Integrated organisation Though one can argue whether an organisation and its structure can be considered a tool, it has been very important that the right organisation was available that supported this assessment and improvement approach. One main concept within the Charging Competence Centre organisation is that many groups have a double responsibility. They both have to perform operational work, which is development of products, and they also have to assure that improvements, which are needed for reaching business goals, are executed. To give an example: There is a group called the project office. This groups consists of project managers, software process engineers, and project administrators. Their first responsibility is to run high quality and controllable projects in the organisation, and directly following is the responsibility to improve the way projects are run. Why is that so important? The main reason is that this same combination of roles comes back in the assessment method. As interviewees, they must give input on how projects are performed. But starting in the consensus meeting, the other responsibility becomes important and they start thinking about improvements needed. At the final meeting they will have some initial thought on what is needed. When the findings are discussed in detail after the assessment, they will come up Reaching Business Goals with Value Adding CMMI Assessments, Ben Linders Page 10 of 12
  • 11. with improvement actions, which they mostly will do themselves. So this learning organisation fully integrates the assessments method as one of theirs feedback loops for improvement. Another advantage is that these kinds of teams drive the assessments. They are the customers, and they will be involved in selecting the goals, defining organisation scope and process areas, and defining improvement actions. By being involved in all stages of an assessment, they greatly improve the efficiency and the effectiveness of an assessment, including the acceptance of findings and commitment to take any action that is needed. 4 Experiences with the assessment method The assessment method came to exist during the assessments in Q1 of 2000. Willing to do an assessment which was not level based, but functionally oriented, so-called sub assessments were done for the first time. This was still using the CMM V2.0. Draft C, but taking concepts from the CMMI V0.2. The success of these assessments led to an initial definition of the new assessment method. In August after the CMMI model was released, Ericsson decided to move towards the CMMI continuous model. Based on that, pilot assessments were defined for all assessment classes within Ericsson. The Charging Competence Centre has done an official pilot of CMMI class C assessments. After discussing and analysing the business goals and performance, 4 assessments were defined. Each one had its own focus, process areas, and groups within the organisation that will be assessed. For instance, there was one assessment focussing on the project office, including mainly project management process areas but also including organisational training and some support process areas considered vital for project management. Another assessment includes only one process area and group: Decision analyses and resolution in the management team of the department. Based on the results, an assessment plan has been made for 2001. The plan has defined 6 business-focused assessments, to be done throughout the year. At the time of the article, two assessments were finished, and a third one was ongoing. This clearly shows the commitment of the organisation to improve, and to use business focused CMMI assessments as the approach to do this. The assessment method has recently been evaluated. The advantages are: • Very time effective (assessment time of roughly 180 hours total) • Quick follow up after the assessment (within 1-2 weeks) • Good follow up, most findings directly lead to actions (at least 50-70%) Disadvantages are: • Much overlap in the CMMI, making it sometimes difficult to focus • Extracting findings is very difficult to learn (learning by doing) Given these results, the decision is to continue this assessment method. Improvements which are considered are removing some of the “sales talk” from assessments, since most people now are familiar with the CMMI, and put more focus on overview meetings and presentations about the contents of the process areas and thus improve understanding of the model. Reaching Business Goals with Value Adding CMMI Assessments, Ben Linders Page 11 of 12
  • 12. 5 Conclusions The business-focused assessment method has shown to be very worthwhile. Key success factors are that assessment are easy to conduct, have a short lead time, and that the findings lead to actions considered vital for reaching business goals. Currently the method is being rolled out to other departments. Further on it will be fine-tuned to fit even better in the performance management program at Ericsson, which defines goal and measurement based improvement of the performance of organisations. Combined with improvements in the questionnaire, the result is a solid tool useful for continuously improving the performance of the organisation. About the author Ben Linders has been working in process improvement using CMM based models from 1992 onwards. At Philips he participated in CMM improvement programs for Industrial Electronics, and for Audio. At Ericsson he was a member of the improvement team that worked for the CMM level 3 assessment in 1995, and the level 4 assessment in 1998. Currently he is working on the CMMI and Performance Management programs with Ericsson EuroLab Netherlands. He is a member of several (national and international) CMM and SPI based improvement networks. He can be reached by email: Ben.Linders@eln.ericsson.se. References Ref 1: From Staged CMM to Continuous CMMI (and back). Paper by Ben Linders, in SPIder koerier januari 2001. http://www.st-spider.nl Ref 2: Institutionalising change, Application of CMM as a model for organisational change at Ericsson. Paper and presentation by Bart ter Horst, presented at the IEEE EMC’99 conference. http://ewh.ieee.org/soc/ems/confs.html Ref 3: ARC, Assessment Requirements for CMMI, Version 1.0. Software Engineering Institute. http://www.sei.cmu.edu/cmmi/products/assess.html Ref 4: CMMI Models Software Engineering Institute. http://www.sei.cmu.edu/cmmi/products/models.html Reaching Business Goals with Value Adding CMMI Assessments, Ben Linders Page 12 of 12