The 2014 IDG Enterprise Big Data research was completed with the goal of gaining a better understanding of organizations’ big data initiatives, investments and strategies. Additional information on the research can be found here: http://bit.ly/1cW0wWR.
1. Big Data
Growing Trends and Emerging Opportunities
Conducted across the IDG Enterprise brands: CIO,
Computerworld, CSO, InfoWorld, ITworld and Network World
2. Purpose and Methodology
Survey Sample
Survey Method
Total Respondents
751
Collection
Online Questionnaire
Margin of Error
+/- 3.6%
Audience Base
CIO, Computerworld, CSO,
InfoWorld, ITworld, and
Network World
sites, email, and LinkedIn
forums.
Number of
Questions
46 (incl. demographics)
Survey Goal
To gain a better understanding of organizations'
big data initiatives, investments and strategies.
Source: IDG Enterprise Big Data Study, 2014
1
3. Big Data Definition
Large volumes of a wide variety of data
collected from various sources across the
enterprise including transactional data from
enterprise applications/databases, social media
data, mobile device data, unstructured
data/documents, machine-generated data and
more.
Big data technologies are a new generation of
technologies and architectures designed to
extract value economically from these data sets
by enabling high-velocity capture, discovery
and/or analysis.
Data in high volume, high velocity and a high
variety of information assets can deliver
enhanced insights and decision making.
Source: IDG Enterprise Big Data Study, 2014
2
4. Data Terms
UNIT
SIZE
WHAT IT MEANS
Bit (b)
1 or 0
Short for “binary digit,” after the binary code (1 or 0) computers use to store and
process data.
Byte (B)
8 bits
Enough information to create an English letter or number in computer code. It is
the basic unit of computing.
Kilobyte (KB)
1,000 or 210 bytes
From “thousand” in Greek. One page of typed text is 2KB.
Megabyte (MB)
1,000KB; 220 bytes
From “large” in Greek. The complete works of Shakespeare total 5MB. A typical
pop song is about 4MB.
Gigabyte (GB)
1,000MB; 230 bytes
From “giant” in Greek. A two-hour film can be compressed into 1-2GB.
Terabyte (TB)
1,000GB; 240 bytes
From “monster” in Greek. All the catalogued books in America’s Library of
Congress total 15TB.
Petabyte (PB)
1,000TB; 250 bytes
All letters delivered by America’s postal service this year will amount to around
5PB. Google processes around 1PB every hour.
Exabyte (EB)
1,000PB; 260 bytes
Equivalent to 10 billion copies of The Economist.
Zettabyte (ZB)
1,000EB; 270 bytes
Approximately 1,000 Exabytes. Data in 2010 cracked the zettabyte barrier.
Yottabyte (YB)
1,000ZB; 280 bytes
Currently too big to imagine.
Source: The Economist
3
The prefixes are set by an intergovernmental group, the International Bureau of weights and Measures.
Yotta and Zetta were added in 1991; terms for larger amounts have yet to be established.
5. Amount of Data Climbing Sharply
Q. Approximately, how much data is your organization currently managing? AND Q. How much data do you expect to be managing 12-18 months from now?
BASE: Plans to deploy/implement big data projects.
Source: IDG Enterprise Big Data Study, 2014
4
6. Interest in Big Data Continues to Rise
39%
15%
12%
14%
10%
10%
eady deployed/implemented big datain the process… We have plans likely to implement big data projects inhavefuture but to deployspecific timeframedata projects
We are projects considering deploying to…
We are
We are or implementing big data projectsWe the no plans have no or implement big in place
within the next 13-24 months
Q. Is your company currently implementing, planning or considering projects (i.e. devising strategies and projects to generate more value from existing data)?
Source: IDG Enterprise Big Data Study, 2014
5
7. CEO Largest Supporter of Big Data Efforts
CEO
47%
34%
Line of Business (as head)
29%
Board of Directors
27%
Marketing
24%
Finance/Accounting
21%
Strategy
20%
Engineering
19%
Sales
18%
Research & Development
14%
Customer Service
10%
None (this is an IT-only initiative)
Not sure
4%
Q. Which of the following business groups are supporting or sponsoring your big data efforts? BASE: Plans to deploy/implement big data projects.
Source: IDG Enterprise Big Data Study, 2014
6
8. Exec. IT Heavily Involved in Big Data Initiatives
36%
Capturing
23%
18%
21%
38%
Storing
24%
32%
10%
39%
Searching/
Sorting
28%
25%
21%
31%
Sharing
21%
14%
20%
48%
Analyzing
34%
34%
45%
33%
Visualizing
22%
17%
21%
IT Exec
Mid-Level IT
IT Pro
LOB
Q What role do you play personally in your organization’s big data initiatives? I am involved in… BASE: Plans to deploy/implement big data projects.
Source: IDG Enterprise Big Data Study, 2014
7
9. Laying the Groundwork for Big Data
Developing or buying
software applications
38%
Investing in additional server or
storage hardware/software
37%
34%
Hiring staff with analytics skills
Utilizing an open source software
framework (e.g., Apache Hadoop)
Migrating to cloud-based
services for storage
Increasing network bandwidth
30%
28%
27%
Migrating to cloud-based
services for analytics
26%
Contracting outside
big data expertise
26%
Q. Which of the following is your organization doing to manage big data initiatives? BASE: Plans to deploy/implement big data projects.
Source: IDG Enterprise Big Data Study, 2014
8
10. Need for Skilled Employees Ranks High
40%
41%
Limited availability of skilled employees to analyze data
35%
Limited availability of skilled employees to manage big data
Limited budget
40%
30%
Legacy issues (integration of exisiting tools)
36%
31%
33%
29%
32%
39%
31%
28%
30%
27%
29%
27%
26%
Poor data quality
Difficulty integrating or analyzing real-time data
Development time
Growing demand on storage capacity/infrastructure
Scalability issues
Ability to demonstrate ROI from investments
18%
Ownership/control issues
23%
25%
22%
24%
21%
21%
21%
Access and usability for end users
Ensuring adequate bandwidth to support data capture and analysis
Difficulty uncovering actionable insights/inadequate reporting
14%
Escalating infrastructure and maintence costs due to data growth
Lack of management support
49%
34%
37%
Security issues
Compliance/regulatory issues
41%
13%
21%
22%
19%
17%
<1,000
1,000+
Q. What are the challenges that your organization is facing or expects to experience with its big data initiatives? BASE: Plans to deploy/implement big data projects.
Source: IDG Enterprise Big Data Study, 2014
9
11. Staffing Up for Big Data Initiatives
Q. With regard to big data initiatives, what skill sets does your organization currently have? AND Q. Which skill sets is your organization planning to hire within the next
12-18 months? BASE: Plans to deploy/implement big data projects.
Source: IDG Enterprise Big Data Study, 2014
10
12. Conclusion
•
Organizations are seeing exponential growth in the amount of data managed with an expected increase
of 76% within the next 12-18 months.
•
Companies are intensifying their efforts to derive value through big data initiatives with nearly half (49%)
of respondents already implementing big data projects or in the process of doing so in the future;
however, enterprise organizations are ahead of the curve in implementation plans compared to SMB
organizations.
•
CEOs are focused on the value of big data and are partnering with IT executives who will
purchase/manage/execute on the strategies.
•
Organizations are investing in developing or buying software applications, additional sever
hardware, and hiring staff with analytics skills in preparation for big data initiatives.
•
Organizations are facing numerous challenges with big data initiatives and limited availability of skilled
employees to analyze and manage data tops the list.
•
In the next 12-18 months, organizations plan to invest in skill sets necessary for big data
deployments, including data scientists (27%), data architects (24%), data analysts (24%), data
visualizers (23%), research analysts (21%), and business analysts (21%).
Source: IDG Enterprise Big Data Study, 2014
11
13. Learn More
• To view the full results, please contact Sue Yanovitch, VP, Marketing of
IDG Enterprise at syanovitch@idgenterprise.com.
12
Hinweis der Redaktion
Processor or Virtual Storage· 1 Bit = Binary Digit· 8 Bits = 1 Byte· 1024 Bytes = 1 Kilobyte · 1024 Kilobytes = 1 Megabyte · 1024 Megabytes = 1 Gigabyte · 1024 Gigabytes = 1 Terabyte · 1024 Terabytes = 1 Petabyte · 1024 Petabytes = 1 Exabyte· 1024 Exabytes = 1 Zettabyte · 1024 Zettabytes = 1 Yottabyte · 1024 Yottabytes = 1 Brontobyte· 1024 Brontobytes = 1 Geopbyte Disk Storage· 1 Bit = Binary Digit· 8 Bits = 1 Byte· 1000 Bytes = 1 Kilobyte · 1000 Kilobytes = 1 Megabyte · 1000 Megabytes = 1 Gigabyte · 1000 Gigabytes = 1 Terabyte · 1000 Terabytes = 1 Petabyte · 1000 Petabytes = 1 Exabyte· 1000 Exabytes = 1 Zettabyte · 1000 Zettabytes = 1 Yottabyte · 1000 Yottabytes = 1 Brontobyte· 1000 Brontobytes = 1 Geopbyte This is based on the IBM Dictionary of computing method to describe disk storage - the simplest. Bit: A Bit is the smallest unit of data that a computer uses. It can be used to represent two states of information, such as Yes or No.Byte: A Byte is equal to 8 Bits. A Byte can represent 256 states of information, for example, numbers or a combination of numbers and letters. 1 Byte could be equal to one character. 10 Bytes could be equal to a word. 100 Bytes would equal an average sentence.Kilobyte: A Kilobyte is approximately 1,000 Bytes, actually 1,024 Bytes depending on which definition is used. 1 Kilobyte would be equal to this paragraph you are reading, whereas 100 Kilobytes would equal an entire page.Megabyte: A Megabyte is approximately 1,000 Kilobytes. In the early days of computing, a Megabyte was considered to be a large amount of data. These days with a 500 Gigabyte hard drive on a computer being common, a Megabyte doesn't seem like much anymore. One of those old 3-1/2 inch floppy disks can hold 1.44 Megabytes or the equivalent of a small book. 100 Megabytes might hold a couple volumes of Encyclopedias. 600 Megabytes is about the amount of data that will fit on a CD-ROM disk.Gigabyte: A Gigabyte is approximately 1,000 Megabytes. A Gigabyte is still a very common term used these days when referring to disk space or drive storage. 1 Gigabyte of data is almost twice the amount of data that a CD-ROM can hold. But it's about one thousand times the capacity of a 3-1/2 floppy disk. 1 Gigabyte could hold the contents of about 10 yards of books on a shelf. 100 Gigabytes could hold the entire library floor of academic journals.Terabyte: A Terabyte is approximately one trillion bytes, or 1,000 Gigabytes. There was a time that I never thought I would see a 1 Terabyte hard drive, now one and two terabyte drives are the normal specs for many new computers. To put it in some perspective, a Terabyte could hold about 3.6 million 300 Kilobyte images or maybe about 300 hours of good quality video. A Terabyte could hold 1,000 copies of the Encyclopedia Britannica. Ten Terabytes could hold the printed collection of the Library of Congress. That's a lot of data.Petabyte: A Petabyte is approximately 1,000 Terabytes or one million Gigabytes. It's hard to visualize what a Petabyte could hold. 1 Petabyte could hold approximately 20 million 4-door filing cabinets full of text. It could hold 500 billion pages of standard printed text. It would take about 500 million floppy disks to store the same amount of data.Exabyte: An Exabyte is approximately 1,000 Petabytes. Another way to look at it is that an Exabyte is approximately one quintillion bytes or one billion Gigabytes. There is not much to compare an Exabyte to. It has been said that 5 Exabytes would be equal to all of the words ever spoken by mankind.Zettabyte: A Zettabyte is approximately 1,000 Exabytes. There is nothing to compare a Zettabyte to but to say that it would take a whole lot of ones and zeroes to fill it up.Yottabyte: A Yottabyte is approximately 1,000 Zettabytes. It would take approximately 11 trillion years to download a Yottabyte file from the Internet using high-power broadband. You can compare it to the World Wide Web as the entire Internet almost takes up about a Yottabyte.Brontobyte: A Brontobyte is (you guessed it) approximately 1,000 Yottabytes. The only thing there is to say about a Brontobyte is that it is a 1 followed by 27 zeroes!Geopbyte: A Geopbyte is about 1000 Brontobytes! Not sure why this term was created. I'm doubting that anyone alive today will ever see a Geopbyte hard drive. One way of looking at a geopbyte is 15267 6504600 2283229 4012496 7031205 376 bytes!
Question: All_10 Given the rate of data growth at most companies, findings such as 65 percent of respondents feeling overwhelmed by incoming data makes sense. On average, respondents currently manage about 164 terabytes of data, although the number varies considerably by size--larger companies report an average of 291 terabytes, while smaller companies average about 57 terabytes. However, respondents expect that average to reach nearly 290 terabytes of data within the next year to 18 months, and 31 percent of larger companies--companies with $1 billion or more in annual revenues--expect to manage more than one petabyte of data. In 2012, respondents indicated an increase from194 terabytes managed to 297 terabytes by the end of 2013. The decrease in difference in the amount of data currently and expected to manage from 2012 to 2013 signifies overestimated expectations moving to a more realistic view.
Question: All_1 As game-changing technology trends such as mobile device usage, social media and sensor-derived data further feed the explosive growth of data, companies are intensifying their efforts to derive economic value through big data initiatives. Interest in big data continues to rise, as nearly half (49 percent) of respondents are implementing or plan to implement big data projects within their organizations in the future -- a 5 percentage point increase from 2012.
Question: CIO1Further underscoring big data’s compelling business case and the need for IT/business partnership, organizationssurveyed indicated that their big data efforts are being supported and sponsored at the CEO (47 percent) or Line of Business level (34 percent).
Question: All_20
Question: All_9Companies are concentrating on building out internal infrastructure and building up their analytics expertise. When it comes to technology, 38 percent in developing or buying software, 37 percent are investing in server hardware, and 34 percent are hiring staff with analytics skills. Not sure 11%
Question: All_6Unsurprisingly, smaller companies (49 percent) were more likely to run into fiscal constraints than enterprise-size companies (40 percent.) However, while last year’s survey respondents cited legacy issues such as integration of existing tools and security issues as top challenges, those hurdles have been supplanted in 2013 by the much-vaunted scarcity of skilled data scientists. In fact, respondents cited the skill shortage as part of the top challenges to their big data projects, an issue that remained consistent regardless of company size.With graduates of top collegiate math programs deluged with job offers, many companies find themselves scrambling to find skilled employees capable of managing and analyzing big data. Other <1,000: 2 percent1,000+: 1 percentNone <1,000: 2 percent1,000+: 1 percentNot sure <1,000: 2 percent1,000+: 4 percent
Question: All_16Organizations are planning staff increases to support Big Data initiatives. Only 18 percent of respondents currently employ a data scientist, but about a quarter more plan to hire for the position in the next 12-18 months.