The document discusses challenges in developing job-ready business analytics graduates. It reports on research finding a skills gap, with graduates lacking real-world skills due to obsolete software training and not having interdisciplinary or "T-shaped" skills. Both industry and universities want more collaboration to develop graduates prepared for rapidly changing technology and data volumes. Recommendations include curriculum changes through industry partnerships, focusing on interdisciplinary and real-world learning through internships.
3. Overview of Research
• 2013 research study into how employers and
universities can collaborate to achieve job-ready
Business Analytics (BA) graduates.
• Four research groups: business leaders, faculty,
recent graduates and supervisors of recent
graduates.
• Findings demonstrate that BA graduates are
lacking real world preparation, use obsolete and
irrelevant software training and lack
interdisciplinary training.
• The rapidly changing business and technological
world is resulting a need for T-Shaped graduates.
4. Skills Gap: T-Shaped Skills
T-Shaped
graduates possess
deep disciplinary
knowledge along
with a keen ability
to communicate
across social,
cultural and
economic
boundaries.
Source: T-Summit, 2015
5. Industry Context
• BA has become pervasive as industry
recognise its importance for their business
operations – to determine risk, detect emerging
trends and make strategic decisions.
• Data from digital activity continues to grow
exponentially – 90% of the world’s data has
been generated in just the past 2 years.
• The current shortage of BA graduates with the
required skills is expected to increase.
7. Challenges and Opportunities
Challenges
• Industry struggling to cope with
unprecedented volumes of data.
• New type of BA is now required:
multi-skilled, T-shaped graduates.
Opportunities
• Collaboration between industry and
universities to develop BA’s with
real-world capacity through
internships, mentoring, etc.
8. Research Methodology
• Purpose: explore the perceptions, attitudes
and experiences of BA stakeholders.
• Stakeholder Groups: business leaders who
employ graduates in BA positions, university
faculty in analytics, recent graduates and
immediate supervisors of recent graduates.
• Approach: in-depth interviews with 25 people.
• Results: qualitative interview results provided
critical data previously not gained through
quantitative research methods.
9. Internal Findings
• Mixed views on interdisciplinary education with a
lack of understanding of the pedagogy.
• Students lack a firm grounding of the theory behind
analytics and data modeling as well as the “soft
skills” required – curriculum deficiency.
• Learning approaches are inconsistent with real-
world collaboration.
• Lack of a clear definition of what “real world” skills
mean resulting in a lack of T-shaped graduates.
• Limited opportunities for collaboration between
industry and education providers.
10. External Findings
Disconnect between university and industry:
• Interdisciplinary education (IE) – graduates,
business leaders and supervisors expressed the
need for an IE curriculum – however not part of the
University mindset.
• Source of Knowledge – need to bring the rapidly
advancing BA innovation into universities and into
the curriculum.
• Collaboration – discouraged in the classroom,
embraced in the workplace.
• Canned problems Vs real-word disorder –
students work on pre-packaged exercises.
11. Recommendations
• Building trust between university and industry.
• Curriculum change is a concept that university
understands and where industry can offer
invaluable insight.
• Make IE teaching the norm by promoting
incentives and faculty sabbaticals.
• In the classroom, allow students to make
mistakes, encourage interns to provide feedback,
provide a forum for recent graduates and invest in
the right technology tools.
• In the workplace, provide student-centered
internships and ongoing supervisor engagement.
13. Conclusion
• BA is growing exponentially, fueled by the rapid
pace of technological change and industry
recognizing that analytics provides a crucial
competitive edge in a world flooded with data.
• Universities are recognizing that delivery of
relevant student skills and an effective BA
curriculum provides their own competitive
advantage.
• To meet the high demand for qualified BA staff,
universities and industry have a strong
incentive to work together.
Overcoming the Skills Gap in Big Data Analytics
Patricia Cotter Ed.D
About Me:
Believe in the promise and power of innovation when it comes to management, technology, business and education.
As a high technology leader and senior business executive, I have brought two companies public (Netezza in 2010 and Visual Networks in 1998).
As President of Netezza LLC , I was instrumental in the integration of Netezza into IBM following IBM’s acquisition of Netezza for $1.8 billion in 2010.
Recently completed my doctorate in work-based learning at the University of Pennsylvania.
For my dissertation, research was focused on interdisciplinary approach to analytics education that will better prepare skilled analytics professionals across all industries.
This research study provides insight about how employers and universities can and should partner, to achieving readiness of recent graduates to fill business analytics (BA) jobs.
A phenomenological approach was used to interview 4 BA groups: business leaders, faculty, recent graduates, and supervisors of recent graduates. Twenty-five individuals yielded data via semi-structured interviews about their lived experiences.
Findings show that while BA graduates are expected to be leaders in future analytics, most are initially lacking in real-world preparation, use obsolete and irrelevant software training, and lack beneficial interdisciplinary training.
The rapidly changing business world, including rapid technological change, is leaving an unfilled need for T-Shaped graduates.
While companies look to universities to increase their pools of analytical staff, business leaders consistently report that there is a yawning gap between the skills that are required and the training that most academic institutions provide.
What is a T-shaped professional?
Currently higher education is producing I-shaped graduates, or students with deep disciplinary knowledge. T-shaped professionals are characterized by their deep disciplinary knowledge in at least one area, an understanding of systems, and their ability to function as “adaptive innovators” and cross the boundaries between disciplines.
The defining characteristic of the “T-shaped professional” is the horizontal stroke, which represents their ability to collaborate across a variety of different disciplines. To contribute to a creative and innovative process, one has to fully engage in a wide range of activities within a community that acknowledges their expertise in a particular craft or discipline and share information competently with those who are not experts.
T-shaped graduates possess broad expertise coupled with in-depth discipline knowledge. Responsibility for training graduates is mistakenly seen as belonging to universities. But businesses are in an essential training role through internships, mentoring, and on-the-job training.
Interviewee data suggests ways to improve: partnerships between universities, placement offices and businesses; better internship and mentoring programs; including students in curriculum planning; introducing and enhancing T-shaped thinking.
These suggested improvements can help corporations and universities to become better partners in educating BA students.
Organizations in many industries are flooded with data from the processes, transactions and events that occur during their daily operations. They rely on the field of BA to determine what this data means and make it actionable — to determine risk, detect emerging trends, find deeply buried or unforeseen nuggets of information and make more informed decisions.
Utilizing mathematical models and the latest computer technology, BA answers questions like:
What’s the best that can happen? (Optimization)
What if we try this? (Random Testing)
What will happen next? (Forecasting/predictive models)
What are the cause and effects? (Statistical models)
What actions are needed now? (Alerts)
Where exactly is the problem (Query/drill down)
What information really matters? (Scorecards)
What can the data tell me that I didn’t ask for?
The past few decades have seen an explosion of data from all of the digital activity that has become part of daily life — from mobile networks, social media, electronic sensors, retail transactions, supply chain networks, electronic medical records and much more. The amount of data generated continues to grow exponentially — the equivalent of 2.5 quintillion bytes of data every day, so much so that 90 percent of the world's data has been generated in just the past two years.
In the U.S. and many other countries, the use of analytics has become pervasive as companies recognize its importance for their business operations (and increasingly, for their very existence). For example, a study by Bloomberg Business Week Research Services reported that 97% of American companies with revenues greater than US$100 million reported using some form of business analytics, up from 90% just a few years earlier.
However, industries consistently report a severe shortage of people with the skills needed to succeed in positions focused on analytics.
This shortage is projected to get worse, as evidenced by a recent McKinsey study whose findings show that 140,000 to 190,000 analysts will be needed in the U.S alone by 2018 in order to do these jobs.
Organizations worldwide are struggling to correlate, understand and above all monetize these unprecedented volumes of data.
Analytics has also become highly diverse, with many different genres according to the type of analysis, the industry and how the information is used. Examples include enterprise decision management, web analytics, price and promotion modeling, credit risk analysis, fraud analytics, etc.
To achieve these types of business-impacting data analysis, a company has to hire and develop the right talent.
The rise of “knowledge-based” businesses is fueling the demand for a new type of employee: one who not only understands mathematics and business, but can also communicate verbally and in writing, solve problems, manage projects and meet other requirements that are often referred to collectively as “soft skills”. For example, the website for Villanova’s analytics program suggests there are seven “must-haves” for analysts’ skills: communication skills, technical skills, analytical skills, problem-solving skills, decision-making skills, managerial skills, and negotiation and persuasion.
Companies pay university graduates quite well to join their team, and they are expecting them to arrive with all the requisite skills. But as the demand for high-aptitude, multi-skilled employees is increasing, so is the shortage of qualified candidates.
Companies constantly report that candidates lack the relevant academic training and real-world seasoning that the business analytics field requires. For example, technical knowledge is often inadequate, with graduates trained on software that is obsolete or irrelevant. Other common complaints include minimal understanding of basic business practices, poor communication skills and an inability to work as part of a team.
Universities and industry have a strong incentive to work together to provide opportunities for students interested in a BA career. Concern over having capable, competitive staffing helps explain U.S. businesses’ desire to partner with higher education institutions, while universities are anxious to provide attractive career opportunities for their students (especially with today’s high tuition costs).
However, there are substantial differences in culture and expectations between the two sides — even in the language they use to describe similar ideas. The result is frustration and misunderstanding that has an adverse impact on the education that students receive.
Analytics has now come of age, used daily by thousands of U.S. companies, hospitals, government agencies and other organizations. With so many potential opportunities, universities and corporations need to find ways to work together in order to more closely align academic BA offerings with industry requirements. These concerns, and knowing that universities and companies can and must do better, were motivators for the study.
The study explored the perceptions, attitudes and experiences of four stakeholder groups: business leaders who employ college graduates in positions requiring analytic skills, university faculty in analytics, recent graduates and immediate supervisors of recent graduates.
The approach used was phenomenological, with twenty-five individuals yielding data via in-depth interviews about their lived experiences.
Qualitative interview results provided important information that might have been missing using a quantitative research method restricted to numerical data.
It should be noted that the number of people traditionally interviewed in phenomenological studies is usually small. While studies have been conducted with much larger groups, successful studies with sample sizes of six individuals or even fewer are regarded as being within the norm.
The participants included six business leaders who were a mixture of C-Suite executives with extensive exposure to business analytics globally, from companies that are all leaders in their area of technology.
The seven faculty that participated represented leading institutions in analytics and/or were program directors that have championed BA programs, on-line masters and doctoral programs, continuing education master-level program directors, and widely-published American faculty members with skills in BA.
The six supervisor participants ranged in tenure from one to over 40 years of direct experience analyzing data in various sectors of technology, financial services and specializations in areas such as learning analytics.
The six recent-hire graduate participants have been in roles utilizing BA from a few months to five years, and graduated from universities across the United States and Canada.
Each interview lasted between 30 and 60 minutes, and consisted of a set of semi-structured, open-ended questions for each stakeholder group.
The qualitative interview method was free form — although the interviews followed a protocol, the discussion went wherever the interviewees took it. This method encouraged the participants to discuss a topic in his or her own words while addressing five salient points:
The suitability of recent college graduates to perform BA jobs
How successfully businesses and universities collaborate with each other
How relatively well recent college graduates are faring in jobs that require business analytics
The interviewee’s view regarding any need for an interdisciplinary approach to the development of analytic skills among students intending to become employed in the field of analytics
How well or how poorly instructional strategies fare in terms of successful results among students in jobs requiring analytics skills
After the interviews were completed, transcribed and checked for accuracy, QSR NVivo theme analysis software was used to organize and analyze over 1000 pages of unstructured transcript data to identify themes and patterns within and across the interviews. Validation of the findings occurred throughout the successive steps of the research.
Currently there are mixed views on interdisciplinary education (IE) with many indicating that an IE approach was regarded as inappropriate for a business school. There is a lack of clear understanding of the pedagogy of IE.
Students lack firm grounding in the theory behind analytics and data modeling as well as the mechanics of how to manipulate a database and work with tools.
Universities often exclude communication skills as part of real-world training methods.
Due to the diversity of industry needs, there is a lack of clear definition of what “real world” skills mean and a lack of T-shaped individuals.
Companies constantly report that candidates lack the relevant academic training and real-world seasoning that the business analytics field requires.
Lack of existing collaborative partnerships with education providers has established “secretive learning processes” and poor communication skills in graduates.
History of dissatisfied relationships with universities due to lack of evidence of improvement in the quality of BA graduates.
Receive inadequate preparation for real-world BA scenarios and communication requirements.
Lack of coverage of “soft skills” in BA curriculum and a de-emphasized approach to collaboration and communication.
Criticism towards pre-packaged learning where university students work on canned exercises and problems rather than learning to glean information from raw data. Reliance on these packaged learning models has been described as a surface approach, an approach that has been criticized because of its reliance on memorization of facts and getting the right answer, rather than developing problem-solving and critical thinking skills.
Comparing results across the different groups reveals communication isolation between business leaders and supervisors, faculty leaders and recent graduates, faculty leaders amongst themselves, and supervisors with everyone else.
This academic and marketplace isolation is especially pronounced when it comes to opinions about what graduates are expected to know, and how and where to provide that training.
This disconnect between the university and industry manifests itself in additional ways:
Interdisciplinary education: Benefits of an interdisciplinary education (IE) approach were discussed with each of the four groups. Graduates, business leaders and supervisors want a curriculum that produces IE-trained students who can communicate, network, solve problems, present to various levels of management and project-manage their work. However, there was no universal buy-in of IE at universities. The interdisciplinary approach is not part of the mindset of most faculty, who are paid to be deep in one subject and are not rewarded for providing interdisciplinary experiences in the traditional university system.
Where the knowledge comes from: Universities have a traditional view that they are the primary source of knowledge. The BA field, where advances are happening very rapidly, upends this traditional role. With industry delivering most of the innovations, there needs to be a way to bring that knowledge into the university and make it part of the curriculum.
Collaboration — discouraged in the classroom, embraced in the workplace: In a university setting, students are judged on their individual work. Collaboration and teamwork are often frowned upon and even considered cheating. However, in the business world, collaboration skills are essential for success. Students used to working independently can have a rude awakening when they have to engage with customers, managers, and colleagues in other departments, share ideas, and give and receive criticism constructively.
Canned problems vs. real-world disorder: University students usually work on pre-packaged exercises, with clean data and where the correct answers are known. (It may not be the most effective learning tool, but the canned, predicable approach is useful for an abbreviated academic calendar.) Unlike this sanitized academic learning, the real world is unstructured, messy and ambiguous. For example. University of Pennsylvania can be rendered as “U. Penn”, “Penn.”, “Penn” (without the period), “University of Pennsylvania” and other permutations and misspelled variations. If you don’t know that all terms mean the same thing, a great deal of relevant data can be excluded, producing an analysis that is incomplete and possibly wildly misleading.
A number of recommendations emerged from the research to improve the university/industry relationship, provide a more relevant learning experience and help graduates make a smooth transition to their first BA job.
Building Trust
End the “Blame Game”
The university/industry relationship is characterized by frustration and suspicion on both sides. Universities hear a lot of heated rhetoric from businesses leaders that they need to do substantially more to prepare students in order to justify their starting salaries. Faculty counter that companies are all demanding different things, their demands change frequently and many of the demands are vague and amorphous. (For example, many companies complain that they want greater emphasis on “soft skills”, but don’t define that they believe what the term means.).
The first step to address the academic and market isolation is to tone down the rhetoric and begin to understand each other’s thought processes and motivations. Universities need to understand that they’re going to be pressed to deliver more, while companies have to understand that they’re not going to get everything they want. Once this occurs, there are many ways that industry and universities can help each other. For example, company executives and analyst practitioners can come on campus, meet with students, review class projects and provide live mentoring. Corporations have the talent to provide adjunct faculty, the industry connections, internships and credibility that set a university BA program ahead of competing programs and help the company get the qualified graduates they need.
Curriculum is a Good Starting Point for the Discussion
Curriculum change is a concept that the university understands, and where industry can offer specific suggestions. Many enterprises welcome the opportunity to review the academic program and identify areas where the skills gap could be improved. Universities can also take the initiative by reaching out to industry to identify areas where improvement is needed, for example, by conducting company surveys, interviewing supervisors and debriefing recent grads. Reaching out in this way also helps the sides start to build an atmosphere of trust.
Make Interdisciplinary Teaching Cool
It takes a special mindset to be comfortable teaching in an interdisciplinary environment. University faculty are generally not accustomed to sharing the podium and teaching duties with colleagues in other disciplines. This would be thinking ‘outside the box’ in a very literal sense for faculty who are trained, and traditionally paid, to be deep thinkers in one discipline. Faculty should be given incentives and encouraged to participate, by reminding them of new publishing opportunities and recognition by a wider audience.
Encourage Faculty Sabbaticals in Companies
Faculty members should consider spending time in BA-oriented companies to learn firsthand what type of analytic challenges they face and how they address them. These sabbaticals could involve a deep dive into one company or shorter stints with a series of companies representing different industries and approaches. Faculty could then bring the knowledge and experiences back to the university and into their classrooms. By demonstrating interest in what happens outside their ivory tower (as industry sees it), these sabbaticals bring the two cultures closer together, helping build understanding and trust that lead to further partnerships.
In the Classroom
Give Students Space to Make Mistakes
Students are traditionally graded by the number of correct answers they give. The option of failure and then learning from the experience is not most universities’ modus operandi. But business analytics is a problem-solving discipline, where mistakes are a natural part of the process. There should be room for students to get it wrong and not be penalized unduly if their thought process was sound. A better approach might be to let students work the problem, see what they did wrong and learn from their mistakes — and along the way, develop critical thinking and problem-solving skills. (Working with dirty data, where terms can be rendered in multiple ways, is a good way to get started in acquiring these skills.)
Give Returning Interns a Voice
Students today do their internship and return without providing feedback about the experience. There should a way for students to get that information back into the university so other students can benefit. Universities should arrange for students returning from internships to do a formal debrief to reflect on the experience, sharing with peers what they learned and answering questions. (How did you deal with difficult people? How did you network? What did you do that went well? What didn’t go so well? What were the biggest surprises? etc.) Faculty can take an active role, leading the discussion to show how lessons learned from internships complement academic work. Since internships take place in many different analytics sectors (healthcare, commodity pricing, hedge funds, etc.), the variety of perspectives and experiences can further enrich the classroom experience.
Provide a Forum for Recent Grads
In most universities, once a student graduates, there’s no further contact except for fundraising. This also applies to graduates from BA programs. There should be a way for recent graduates to provide feedback to the university on what worked in the analytics curriculum and what can be improved. This could occur after a certain timeframe, for example, three months after the graduate starts his/her first job. Graduates should also be encouraged to come back and give presentations to current students, sharing their insights and experiences. By sharing their knowledge, recent graduate hires could be corporations’ and universities’ best source for improving understanding and partnerships.
Invest in (or Arrange for) the Right Software Tools
Many highly regarded institutions still attempt to teach analytics using Excel spreadsheets, which leads to a rude awakening when graduates attempt to put their skills to use on their first job. Any serious analytic program will have to provide training in one of the leading software tools such as SPSS, SAS or Tableau. Students need grounding in one of the big ones; from there the main concepts for other tools are similar.
In the Workplace
Student-centered Internships
Universities tout internships as an opportunity for students to learn in a real world setting what they can’t learn in the structured environment of a classroom. However, corporations tend to emphasize bottom line value, and they tend to see the intern as a source of free labor — federal labor laws notwithstanding. The primary purpose of the internship should be relevant learning that complements and builds on the student’s previous training and advances their educational goals.
Provide Better Matching for Internships and Jobs
Career placement officers need to be attuned with the curriculum and skill sets of graduates, and understand how that preparation aligns with what companies are looking for. For example, if a company is looking for computational people, a business school graduate might not be comfortable in that setting but a candidate from a program focused on math and computational analysis probably would be.
Supervisor Engagement is Essential
The most specific, most accurate assessment of capabilities required and skills gaps of BA hires comes from their immediate supervisors. Improving the flow of communication from supervisors to business leaders, career placement staff, faculty and students is critical to ensure that graduates can succeed in their first BA position.
Education in business analytics benefits from an interdisciplinary approach, where core knowledge is complimented by other perspectives and experiences, and where learning is not compartmentalized. (The words “business” and “analytics” are a reminder that multiple disciplines are involved.)
Interdisciplinary education broadens traditional siloed education, with students and faculty from multiple fields (for example, analytics, finance, engineering and business) in class together, sharing their knowledge and perspectives.
With an interdisciplinary approach, programs that teach computational theory and math would also include business courses to challenge students to think more broadly. The best reason for an interdisciplinary approach is because that’s what the world is like, and how problems are solved.
The most positive results for BA skills training involve a fluid, interactive process that includes each of the four elements within the triangle. This includes a university foundation with an interdisciplinary focus to acquire the techniques and soft skills; internships to acquire real-world experience followed by further professional training and mentoring after the new graduate is hired.
Each element builds on and reinforces each other. Students bring the foundational knowledge acquired in the classroom to the internship, where they experience job reality, improve their skills, network with co-workers and learn to recover from mistakes. The lessons they learn from this experience flow back to the university, enhancing the student’s learning with a real-world perspective while helping improve the curriculum for other students.
Once the student is hired, on-the-job training and mentoring provides the specific in-depth knowledge about the company and its business, and is ongoing.
From the Privileged Few to the Empowered Many
The field of Business Analytics is growing exponentially, fueled by the rapid pace of technological change and recognition by more and more companies that analytics provides a crucial competitive edge in a world flooded with data.
Just as businesses are defining their need for BA in competitive terms, many universities are recognizing that delivery of relevant student skills and an effective BA curriculum provides their own competitive advantage. The BA workforce of the future must advance as rapidly as the market itself, with universities bringing more students of higher caliber into play.
To meet the high demand for qualified BA staff, universities and industry have a strong incentive to work together. Universities are anxious to provide attractive career opportunities for their students (especially with today’s high tuition costs) while companies are clamoring for qualified graduates in order to compete and grow.
It is my hope that the ideas and recommendations from my research will help university leadership and their industry counterparts understand each other better, start to forge mutually-beneficial partnerships, and create a BA curriculum that prepares students to succeed in this exciting field.
Welcome requests for speaking engagements and collaborative opportunities in the Big Data space, particularly related to the critical role that women will play in this exciting and promising new field.
You can contact me via email, LinkedIn or Twitter.