User survey analysis customers rate their CPM vendors, 2012 Gartner

User Survey Analysis: Customers Rate Their CPM
Vendors, 2012
4 June 2013 ID:G00247384
Analyst(s): Christopher Iervolino, John E. Van Decker
VIEW SUMMARY
Gartner's survey of 275 customer references from 13 corporate performance management vendors
between September and November 2012 has produced valuable insights and benchmarking data
for IT leaders assessing CPM solutions. Don't just focus on the megavendors when evaluating
prospective providers.
Overview
Key Findings
 Among the megavendors, SAP has improved overall satisfaction ratings in relation to
Oracle, and IBM has maintained an above-average overall rating; however, survey results
highlight room for improvement for all three providers.
 Annualized cost comparisons indicate that software-as-a-service solutions are not
significantly less expensive than many traditional on-premises offerings. For both options,
business value attained and total cost of ownership should primarily guide buying
decisions.
 Results indicate distinctive customer satisfaction differences among corporate performance
management vendors in vendor-, product- and implementation-related scores. Statistics
related to average number of users per implementation, popularity of modules, length of
time used and cost also characterize vendor solutions.
 Many specialist CPM vendors score consistently higher than the megavendors, which
shows they have credible solutions, despite their relatively small size and market share.
Recommendations
 Extend product evaluations past the assessment of individual functions, and consider the
cost and complexity of implementations and ongoing use, as well as the overall value of
the vendor relationship.
 Conduct cost comparisons of SaaS versus on-premises solutions that consider a wide
range of TCO factors, including ongoing consulting, IT support and upgrade expenditures.
 Consider specialist CPM vendors in assessments, and do not focus only on the
megavendors.
TABLE OF CONTENTS
CONTENTS
 Survey Objective
 Data Insights

o Vendor Experience Ratings
o Product Experience Ratings
o Implementation Experience Ratings
o Solution Usage Comparisons
o Solution Cost Comparisons
o Methodology
TABLES
Table 1.
Response by Vendor Customer Locations
Table 2.
Estimated Number of Years Solution Has Been in Use
FIGURES
Figure 1.
Overall Satisfaction
Figure 2.
Overall Satisfaction Versus Average Implementation Size
Figure 3.
Vendor Experience
Figure 4.
Overall Satisfaction Versus the Top-Three Differentiating Vendor Attributes
Figure 5.
Product Experience
Figure 6.
Overall Satisfaction Versus the Top-Three Differentiating Product Attributes
Figure 7.
Implementation Ratings
Figure 8.
Overall Satisfaction Versus the Top-Three Differentiating Implementation Attributes
Figure 9.
Strategic CPM Usage
Figure 10.
Module Usage
Figure 11.
Cost Characteristics: Total Cost to Date
Figure 12.
Annual Cost Characteristics
Figure 13.
Organization Size (Number of Employees)
Figure 14.
Geographic Composition
Figure 15.
Industry Composition
Survey Objective
This document was revised on 6 June 2013. For more information, see the Corrections page.

Toward the end of 2012, Gartner conducted its annual corporate performance management (CPM)
Magic Quadrant survey.1
In that survey, 275 CPM customers from 13 different vendors shared
valuable insights for companies selecting CPM vendors and for customers benchmarking suppliers.
The detailed results provide additional insights to support prospective customers in fine-tuning
their RFPs, product selection criteria and project plan approaches for implementations and
upgrades. This survey comprised a key component of the CPM Suite Magic Quadrant report (see
"Magic Quadrant for Corporate Performance Management Suites"). The analysis of these detailed
results is an important complement to that study.
Table of Contents
Data Insights
This research reveals customer satisfaction ratings as they relate to vendor, product and
implementation attributes. It uses the three scores from each category with the greatest variances
among vendors to highlight the most-differentiating factors. It then evaluates responses alongside
the average number of users per implementation, and the levels of use for each module in the
vendors' product suites. This research also sheds light on other differentiating solution cost
characteristics.
Table 1 shows the number of responses by the vendor and describes their customer locations.
Table 1. Response by Vendor Customer Locations
Vendor No. of Survey Participants Customer Locations
IBM 30 Worldwide
Prophix 29 Mainly North America, but also worldwide
Board International 26 Mainly EMEA, but also worldwide
Host Analytics 24 Mainly North America, but also worldwide
Longview 24 Mainly North America, but also worldwide
Oracle 23 Worldwide
Tagetik 21 Mainly EMEA, but also worldwide
Bitam 19 Mainly North and South America, but also worldwide
KCI Computing 17 Mainly North America, but also worldwide
SAP 17 Worldwide
SAS 17 Worldwide
Infor 15 Worldwide
prevero 13 Mainly EMEA, but also worldwide
Source: Gartner (June 2013)
Table of Contents
Overall, most survey participants were satisfied with their CPM vendors (see Figure 1). On a scale
of 1 to 7, the average rating was 6.09. This favorable rating, which is higher than those in previous
years, indicates that CPM solutions are mature and are delivering value. However, the standard
deviation was fairly high (0.94), which reflects a greater variation in scores, especially for KCI

Computing (standard deviation = 1.46) and prevero (standard deviation = 1.38). Most of the
customers chosen by the vendors were satisfied with their solutions; however, the level of
variability underscores the importance of matching the correct CPM vendor, product and
implementation capabilities to a given set of organizational needs.
Figure 1. Overall Satisfaction
The rating is an average of respondents' overall aggregate score by vendor. The figure represents
customer perceptions, not Gartner's opinion. The graph may feature vendors that in Gartner's
opinion don't deliver the benefits described (N = 275).
Table of Contents
Although it's reasonable to assume that vendors with smaller implementations and few users
would consistently score higher in questions related to vendor support, these survey results do not
wholly support this. As in 2011, vendors Tagetik and Longview score above-average satisfaction
ratings and have an above-average number of users per implementation. IBM also scored above
average in both categories; however, Tagetik and Longview's higher scores are standouts in this
area (see Figure 2).
Figure 2. Overall Satisfaction Versus Average Implementation Size

Table of Contents
The overall customer satisfaction ratings are an aggregation of vendor, product and
implementation scores. To gain insights into these overall ratings, it's necessary to drill down into
more-detailed survey results that illustrate more-specific strengths and weaknesses of each
vendor.
Table of Contents
Vendor Experience Ratings
The following information highlights results related to the vendor experience, such as sales,
support and ongoing communication. The survey asked this question regarding the vendor
experience: "Compared with experiences with other vendors, how was the experience with this
vendor on the following attributes?"
 Ongoing support and after-sales care
 Overall value for the money
 Ongoing communication
 Responsiveness of telephone support
 Experience throughout the sales process
 Additional implementation and deployment
 Ease of applying upgrades, fixes and patches
 Professional service implementation, and deployment training and handover

The attributes that resulted in the largest variations between high and low scores have the biggest
impact on the overall client satisfaction score, and contribute most to differentiating the vendors
within this category. The three attributes with the greatest variances, and their respective scores
for each vendor, are represented in Figure 3. The question asked was, "Compared with
experiences with other vendors, how was the experience with this vendor on the following
attributes?"
Figure 3. Vendor Experience
The figure represents customer perceptions, not Gartner's opinion, and may feature vendors that
in Gartner's opinion don't deliver the functional capability described (N = 275).
Table of Contents
Figure 4 shows overall customer satisfaction, compared with the top vendor-differentiating
attributes.

Figure 4. Overall Satisfaction Versus the Top-Three Differentiating Vendor Attributes
The figure represents customer perceptions, not Gartner's opinion, and may feature vendors that
in Gartner's opinion don't deliver the functional capability described (N = 275).
Table of Contents
Similar to last year, Longview and Tagetik scored well in this category, and Board International's
rating is also on a par with these vendors. Of the three, Tagetik's high rating stands out, given its
above-average revenue growth rate of 25.7% in 2012. Host Analytics, which had ranked among
these vendors in previous years, scored the highest in 2012. As a pure software as a service
(SaaS) vendor, it naturally scored the highest rating for "ease of applying
upgrades/patches/fixes"; however, it also scored highest on the other two differentiating ratings,
including the important "overall value for the money" rating. Host Analytics has achieved this score
during a period of high revenue growth (78.3%), which indicates its growing organizational
maturity. (All vendor revenue growth figures can be found in "Market Share Analysis: Business
Intelligence, Analytics and Performance Management, Worldwide, 2012.")
The largest variation of all vendor, product and implementation ratings existed for "ease of
applying upgrades/patches/fixes." This suggests that the greatest single difference between the
vendors, from the customer's perspective, relates to upgrades, which encompasses ongoing
maintenance and support cost sensitivity, as well as the desire for new product features available
in more-current software versions.
Consistent with previous years, all three megavendors (IBM, Oracle and SAP) scored below
average in this vendor-specific category. Although Oracle scored the lowest in average, it also had
the lowest "overall value for the money" rating. This low rating may have been affected by the
increased awareness of less-expensive CPM options in the marketplace. Slowed CPM market

growth figures for 2012 seem to indicate that additional cost pressures are affecting CPM vendors
differently; for example subscription-based options, such as Host Analytics, have increased their
revenue in 2012 at higher-than-average rates. On the whole, Oracle's average score was most
affected by "responsiveness of telephone support," where it received the lowest score of any other
vendor rating.
All three of the highest variations in overall vendor rating are related to cost. Furthermore, the
largest variations occurred for "ease of applying upgrades/fixes/patches" and "responsiveness of
telephone support." These ratings relate to ongoing solution effort and cost, indicating a growing
gap in customer sentiment related to ease of use and cost considerations among the different
vendors. Prospective customers should note the correlation of these vendor-specific criteria to
overall customer satisfaction. The horizontal distance from average is an indication of how these
vendor-specific factors have affected their overall customer satisfaction ratings.
Table of Contents
Product Experience Ratings
The survey asked the following question regarding product experience: "To what extent has the
organization's vendor CPM solution met the following requirements?"
 Improved the quality, accuracy and timeliness of financial and management reporting
 Added more sophistication and confidence to the budgeting, planning and forecasting
(BP&F) processes
 Reduced the effort to produce budgets, and reduced the budget cycle time
 Allowed business users to get the information they need, with reduced reliance on IT
 Provided a better understanding of current performance
 Allowed a better prediction of financial and business performance
 Provided a linkage between strategic planning, financial budgets and operational activity
 Allowed a better understanding of profitability drivers
 Improved the ability to meet compliance requirements, including external disclosure
As with responses related to vendor experience, the attributes that resulted in the largest
variations between high and low scores have the biggest impact on the overall client satisfaction
score, and contribute most to differentiating the vendors within this product experience category.
The three attributes with the greatest variances in this category and their respective scores for
each vendor are represented in Figure 5. The question asked was, "To what extent has the
organization's vendor CPM solution met the following requirements?"
Figure 5. Product Experience

Table of Contents
Compared with last year's results, Host Analytics and Board International scored notably higher.
This indicates a growing satisfaction with available product functionality in these areas and an
increased sophistication of use for these vendors' solutions (see "User Survey Analysis: Customers
Rate Their Corporate Performance Management Vendors"). These ratings indicate that the widest
range of end-user satisfaction exists for some key product functionality —compliance, giving
insight into the drivers of profitability and providing a linkage between strategic financial planning,
budgeting and operational planning. The rating for Host Analytics is somewhat surprising in that it
is newer to the market than most of the others. A reasonable assumption is that Host Analytics
does not have the breath of functionality of some of the more widely used solutions that have been
around longer; however, these ratings are a measure of satisfaction with ease of use (accessibility
of the functionality), as well as satisfaction with the functionality itself (robustness of the
functionality). Vendors that scored well in this category are likely to be able to attribute their

success to customer satisfaction with a combination of feature accessibility and capability (see
Figure 6).
Figure 6. Overall Satisfaction Versus the Top-Three Differentiating Product Attributes
Table of Contents
The horizontal distance from average is an indication of how these product-specific factors have
affected their overall customer satisfaction ratings. Of the three satisfaction categories surveyed
(vendor, product and implementation), the most narrow range of scores exists for product ratings
(between 4.63 and 5.72). That is, vendor- and implementation-related areas had the greatest
impact on overall satisfaction scores. This indicates that customers are more satisfied with the
product-specific functionality their solutions provide than with vendor- and implementation-specific
items. As a result, prospective customers should ensure that they evaluate new solutions from a
product functionality perspective, as well as in terms of information gathered from vendor
references regarding their vendor and implementation experiences.
Table of Contents
Implementation Experience Ratings
The survey requested the following information regarding each customer's implementation
experience by requesting its level of agreement with the following statements about the
organization's implementation of the vendor's CPM solution:
 The vendor's professional service staff had the skills and knowledge to make the project a
success.

 The CPM applications delivered the functionality expected.
 The anticipated business benefits from implementing the CPM applications were realized.
 It was easy to adapt the CPM applications to meet business requirements over time.
 Users found the system easy to use and manage.
 IT finds the CPM application technology straightforward to maintain and support.
 The solution gained wider acceptance than expected.
 The implementation was easier than expected.
 The implementation was less expensive than expected.
Because the attributes that resulted in the largest variations between high and low scores have the
biggest impact on the overall client satisfaction score, and contribute most to differentiating the
vendors within this product-experience-related category, the three attributes with the greatest
variances within this category and their respective scores for each vendor are examined in Figure
7. The request was, "Indicate agreement with the following statements about the organization's
implementation of the vendor's CPM solution."
Figure 7. Implementation Ratings

Table of Contents
Some of these ratings are expected — for example, it's natural that Host Analytics would score
highest for ease of IT maintenance and support, because it's a SaaS solution. Interestingly, Board,
which does not offer a SaaS solution, and Tagetik, whose customer respondents did not use its
cloud-based offering, were not far behind. This shows that CPM apps are different from ERP and
other business applications that require heavy IT support; IT complexity should not be a forgone
conclusion.
Another significant expectation-related result here is that every vendor had its lowest rating for the
item, "our implementation was cheaper than expected." Although installation costs varied among
the vendors, this indicates a consistent set of false expectations with CPM solutions. This may
occur during the sales or the implementation scoping or a combination of the two. This result also
reflects the importance of cost as a satisfaction factor. Prospective customers should ensure they
include a rigorous customer reference effort that includes an exploration of consulting and support
fees as a necessary component of any CPM purchase or project scoping exercise (see Figure 8).
Figure 8. Overall Satisfaction Versus the Top-Three Differentiating Implementation Attributes
Table of Contents
The tight diagonal arrangement indicates close relationship between implementation and
satisfaction. This is correlation is highest for these implementation ratings, underscoring the
importance of considering implementation project needs alongside the software purchase when
selecting a particular vendor's solution.

Table of Contents
Solution Usage Comparisons
Although most CPM implementations take advantage of office-of-finance CPM capability initially,
organizations can realize significant improvements by taking advantage of strategic CPM. (see
"Getting More Value From CPM: Strategic Versus Office-of-Finance CPM"). This graphic is one
indication of the degree to which different vendor's solutions are used strategically. Figure 9
illustrates the relationship between overall satisfaction and two key metrics that indicate a more-
strategic use of each vendor's CPM solution. The first is the average number of users per
implementation, and the second is module use, which is represented by circle size (more
information regarding module use can be found in Figure 10).
Figure 9. Strategic CPM Usage
Table of Contents
Figure 10. Module Usage

Table of Contents
Those vendors with a larger number of users per implementation (distance to the right) and a
larger circle in Figure 9 (more widely used set of CPM modules) indicate more strategic CPM use.
Those that have also scored high levels of satisfaction (vertical position) are able to do so with
more-complex installations that support a wider number of CPM processes.
How many CPM modules are used is an indication of the popularity of the each of the vendors'
modules. The survey asked the question, "How many users does the organization currently have
for each of the following features of the vendor's CPM solution?" The answers were converted into
percentages of the number of overall users for each vendor to provide consistent comparison
mechanism among vendors. Total scores were ranked from highest to lowest.
This information was then compared with the average number of users per implementation,
another measure of CPM maturity, as well as the average customer satisfaction rating. By
comparing how extensively the product is used (by estimating the modules used and number of
users) the average level of CPM maturity can be estimated. Furthermore, by comparing these
statistics against overall satisfaction, the ease in which the solution allows for additional CPM
maturity for implementations of different sizes can be roughly estimated.

Although the customer sample size is small, compared with the total number of customers (this is
especially true for IBM, Oracle and SAP), the survey data indicates patterns of usage among
survey respondents, such as:
 IBM's solutions were reported to be used for a broader set of CPM processes than were
SAP's (and somewhat more than Oracle's as well). IBM also had a higher overall
satisfaction rating than either Oracle or SAP.
 Longview and Tagetik were able to maintain high customer satisfaction ratings, even with
a high number of average users. Also, a lower percentage of Tagetik customer
respondents use their applications for BP&F, as compared with Longview's.
 Prophix, Board International, SAS Institute and Host Analytics respondents support a wide
array of CPM processes, although each does so for implementations of different average
size and, most likely, of varied complexity.
Table of Contents
Solution Cost Comparisons
Respondents were asked to provide the approximate total cost to date of their organizations' CPM
solutions for each of five cost categories. This included subscription fees; external services
(including consulting, system integration, managed services, hosting, and outsourced process);
hardware; software; and internal labor and head count. Because IBM and KCI Computing each had
one customer response with very high costs, median results were used, instead of averages, to
reduce the impact of these two outliers. The breakdown of costs by vendor is shown in Figure 11.
Figure 11. Cost Characteristics: Total Cost to Date

Table of Contents
These results clearly indicated that Oracle's CPM implementations are the most expensive,
especially for external services, among survey respondents. However, Oracle's CPM solutions have,
on average, been used for longer periods of time. Table 2 shows the median number of years that
respondents reported their solution has been in use (if they have multiple solutions from the
vendor, the solution that was used the longest was reported on).
Table 2. Estimated Number of Years Solution Has Been in Use
Vendor Years Used
Oracle 8
Longview 7
SAS Institute 6

Table 2. Estimated Number of Years Solution Has Been in Use
Vendor Years Used
IBM 5
KCI Computing 5
Bitam 4
Board International 4
Infor 3
prevero 3
Tagetik 3
SAP 2.5
Host Analytics 2
Prophix 2
Table of Contents
Clearly, vendors with customers that have used their solutions for longer periods of time (such as
those using Longview and SAS Institute's solutions) now show lower comparable costs, because
they have had more years over which to spread initial software purchase and implementation
costs. Others that, on average, have relatively new installations, such as those using SAP's and
Host Analytics' solutions, have higher per-year costs. Oracle remains the most expensive, both on
a total and per year basis; however, it's important to consider that Oracle's customer respondents
also estimated a relatively high number of average users. On a per-user basis Oracle's average
costs per year are lower, as are Longview's and Tagetik's. These annualized cost comparisons also
illustrate ongoing costs over time — for example, Figure 12 highlights the higher annual cost of
external services for both SAP and Oracle.
Figure 12. Annual Cost Characteristics

Table of Contents
Host Analytics, a SaaS vendor, has a relatively low initial cost; however, when these costs are
annualized, they are higher than many on-premises vendors. Also, because Host Analytics'
respondents reported a relatively low number of users, costs per year, per user are also higher
relative to many of the other solutions. Regarding SaaS versus on-premises cost comparisons, this
survey was generally completed by finance users and internal labor and head count estimates may
not always reflect accurate ongoing IT support costs. In addition, ongoing costs, such as software
upgrades, are generally built into multitenant SaaS vendor costs; such is the case with Host
Analytics. However, these results suggest that prospective customers should look past short-term
costs and perform on-premises versus SaaS cost comparisons from a total cost of ownership (TCO)
perspective over the anticipated life of the solution. Customers should consider all cost
considerations including ongoing internal and external consulting expenses and both long-term
subscription and license fees when selecting a CPM vendor.

On the whole, this more-detailed analysis of survey results illustrates the credibility of smaller CPM
vendors across a number of areas for larger, more-complex CPM solutions or more-focused ones.
A number of these vendors consistently score high marks in key satisfaction areas, such as overall
value for the money and ease in adapting the solution for changing business requirements. Despite
the consistent leader ratings of the megavendors in the Magic Quadrant, smaller vendors are
highly credible options for many CPM needs and should be considered in a wide array of product
evaluations. In addition, survey results regarding average implementation size, modules used,
length of use and cost can help determine correct product fit and guide product selection
processes.
Table of Contents
Methodology
As part of our research for the CPM Magic Quadrant, Gartner conducted an English-language Web
survey of 275 CPM suite customers (up from 233 in 2011). The individuals surveyed were
nominated by CPM vendors at Gartner's request (Magic Quadrant methodology mandates
gathering data from vendor customers). The survey was composed of 21 questions regarding
customer experiences in working with vendors. Gartner requested contacts knowledgeable about
how the vendors' CPM products were used by customer organizations. The vendors did not see the
questionnaire before the customers were nominated, and responders were made aware that their
answers would remain strictly anonymous. The survey took 20 minutes and covered the
customer's use of the nominating vendor's CPM solution.
Gartner expected that, because the survey participants were vendor reference customers, the
results would reflect a more positive experience of using and implementing each vendor's CPM
solutions. However, the relative comparison of results is valid, since all respondents were
nominated by their respective vendors. Although representative of this group, and providing useful
indications of customer experiences and deployment characteristics, the survey results may not
reflect the views of the general population using these products.
Cost characteristics were calculated by converting rates to U.S. dollars as of the last day of the
survey, 13 November 2012. Median results were then selected to reduce the effect of outliers.
Size, geography and industry characteristics are shown in Figure 13, Figure 14 and Figure 15,
respectively.
Figure 13. Organization Size (Number of Employees)

Table of Contents
Figure 14. Geographic Composition

Table of Contents
Figure 15. Industry Composition
Table of Contents
EVIDENCE
1
This annual survey was conducted from early September until the middle of November 2012. A total of 275
completed surveys were returned. Additional details regarding this survey can be found in "Magic Quadrant for
Corporate Performance Management Suites."
© 2013 Gartner, Inc. and/or its affiliates. All rights reserved. Gartner is a registered trademark of
Gartner, Inc. or its affiliates. This publication may not be reproduced or distributed in any form
without Gartner’s prior written permission. If you are authorized to access this publication, your use of
it is subject to the Usage Guidelines for Gartner Services posted on gartner.com. The information
contained in this publication has been obtained from sources believed to be reliable. Gartner disclaims
all warranties as to the accuracy, completeness or adequacy of such information and shall have no
liability for errors, omissions or inadequacies in such information. This publication consists of the
opinions of Gartner’s research organization and should not be construed as statements of fact. The
opinions expressed herein are subject to change without notice. Although Gartner research may
include a discussion of related legal issues, Gartner does not provide legal advice or services and its
research should not be construed or used as such. Gartner is a public company, and its shareholders
may include firms and funds that have financial interests in entities covered in Gartner research.

Gartner’s Board of Directors may include senior managers of these firms or funds. Gartner research is
produced independently by its research organization without input or influence from these firms, funds
or their managers. For further information on the independence and integrity of Gartner research, see
“Guiding Principles on Independence and Objectivity.”
About Gartner | Careers | Newsroom | Policies | Site Index | IT Glossary | Contact Gartner

User survey analysis customers rate their CPM vendors, 2012 Gartner

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (18)

Ähnlich wie User survey analysis customers rate their CPM vendors, 2012 Gartner

Ähnlich wie User survey analysis customers rate their CPM vendors, 2012 Gartner (20)

Mehr von Miguel Garcia

Mehr von Miguel Garcia (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

User survey analysis customers rate their CPM vendors, 2012 Gartner