After developing an Ad Hoc persona as the core of your engagement strategy, it's important to test your assumptions against real people and real data. Content analysis is a methodology for evaluating text-based data that can be gathered from social media tools.

1. Content Analysis
THE PSYCHOLOGY OF AUDIENCE ENGAGEMENT
Dr. Pamela Rutledge | prutledge@fielding.edu
Dr. Jerri Lynn Hogg | jhogg@fielding.edu

2. Qualitative Research
“
“
A form of social inquiry that focuses
on the way people interpret and
make sense of their experiences
and the world in which they live.

3. Main Types of Qualitative Research
Case Study
Attempts to shed light on a phenomena by studying in depth a
single case example of the phenomena. The case can be an
individual person, an event, a group, or an institution.
Grounded
Theory
Theory is developed inductively from a corpus of data acquired by a
participant-observer.
Describes the structures of experience as they present themselves to
Phenomenology consciousness, without recourse to theory, deduction, or assumptions from
other disciplines
Ethnography
Focuses on the sociology of meaning through close field observation of
sociocultural phenomena. Typically, the ethnographer focuses on a
community.
Historical
Systematic collection and objective evaluation of data related to past
occurrences in order to test hypotheses concerning causes, effects, or
trends of these events that may help to explain present events and
anticipate future events.

4. “
Main Types of Data Collection &
Analysis
Those who are not familiar with
“
qualitative methodology may be
surprised by the sheer volume of
data and the detailed level of
analysis that results even when
research is confined to a small
number of subjects
(Myers, 2002).

5. Three Main Methods Of Data
Collection
Interactive interviewing
People asked to verbally described their
experiences of phenomenon
Written descriptions by
participants
People asked to write descriptions of their
experiences of phenomenon
Observation
Descriptive observations of verbal and nonverbal behavior
Analysis begins when the
data is first collected and is
used to guide decisions
related to further data
collection.

6. “
“
In communicating--or generating--the
data, the researcher must make the
process of the study accessible and
write descriptively so tacit knowledge
may best be communicated through
the use of rich, thick descriptions
(Myers, 2002)

7. What is Content Analysis?
 A formal methodology
• To discover, uncover, or answer
• Uncover unknown qualities about the data
• Looking for patterns in the content

8. A Little History
Systematic analysis of texts performed several times by
religious entities prior to 1900
Major growth periods in the 20th Century
Early 20th Century
Studies of newspaper content
Behavior sciences in 1930s and 1940s
Begin to study media effects
World War II
Propaganda studies
Post war expansion:
conversation analysis, personal document analysis, processes of
communication, and a generalized measure of meaning
Source: Krippendorf, 2004

9. More History
1960s:
Computer text analysis but
challenging beyond
quantitative analysis of text
Today:
extensive proliferation of
traditional, electronic, and
social media are leading to
strong interest in content
analysis and more powerful
software

10. Content Analysis Software Evolution
 Cards - counting
 N-vivo – pulled out paragraphs
 SPSS Text Analytics – computer generated counting increased the
quantity accessible
 Leximancer – text analytics plus context, same word with multiple
meanings, concepts instead of keywords, interactive and visual

11. What is Watson?
 We have a proliferation of Big Data,”
 90% of the world‘s information was
created in the last two years
 80% of that 90% is unstructured or
semi-structured information, like
doctor’s notes or product reviews on
Amazon
 IBM’s Watson is a genius at reading
unstructured information
 The ability to format and use these
knowledge banks will significantly
alter how and how well humans
make decision.

12. Application Areas Today
News Analysis
Sentiment
Social Media
Forensics
Historical Reviews
Political Documents
Conversations
Propaganda
Television Content
Bias Determination
Song Lyrics
National Security
Video Game Content
More ...

13. Content analysis is a
formal methodology
to study a collection of media to
discover, uncover, or answer

14. A Formal Methodology
A formal, objective method with rigor and repeatability
Many methods and processes are valid
Methodology example
1.
2.
3.
4.
5.
6.
Determine research question
Identify and collect samples
Perform quantitative analysis
Perform qualitative analysis
Draw conclusions
Summarize, publish, and share results

15. To Study A Collection of Media
Media is a method of information communication
Collections include the following (normalize formats to text)
Written media such as newspapers, magazines, websites, Blogs,
Tweets, Facebook pages, emails
Audio, such as radio programs, interview transcripts,
conversations (can be transcribed into text)
Video, such as television, movies, news footage, YouTube videos
(can be transcribed into text)
Images described in text

16. To Discover, Uncover, Answer...
Discover concepts, themes, and relationships in the
collection
Uncover unknown qualities about the data
Answer a specific research question
Concepts
Themes
Relationships

17. Key Points
All methods of content analysis share common components:
 Quantitative (counting) and qualitative (meaning) analyses
 Analysts can use one or both methods
 Content analysis is best when both quantitative and qualitative
approaches are combined
 Important study aspects include





Sampling
Units Of Measure
Coding
Validity
Reliability

18. Sampling: Taking a Subset
Sampling plans are needed to
reduce researcher bias
Select a type of sampling (e.g.,
random)
Sample size is important to be
representative
Split-half technique:
Two samples equal
the same result
Source: Krippendorf, 2004

19. Units of Measure
Sampling Part 2: Samples require a definition of data
resolution
Television comedies, 1/2 hour, Wednesday nights
Entire tweet, tweets from a user, collection of topical tweets
One blog entry, an entire blog, or consolidation of many blogs
Newspaper article, articles of a set timeline
Content analysts must determine these units to measure
Impacts relationships of words and coding
Concept discovery restricted to within units

20. Coding
Process of examining text in a specific unit and extracting
relevant data
Look for words, phrases, word sense, and categorize
units of text (i.e., words, sentences, paragraphs,
tweets)
Three methods of coding
1. Manual, by person(s) coding from codebooks, instructional guides,
intuition
2. Computer-assisted (N*Vivo) beginning with coding then often some
automation for remaining documents
3. Computer generated (Leximancer, CATPAC)

21. Reliability and Validity
For a formal analysis method to be sound, reliability and
validity must be addressed
Reliability refers to stability and reproducibility
Coding to be repeatable if manual or computer assisted
Inter-rater reliability for manual coding with multiple coders affects
reproducibility and must be ensured
Measure of accuracy is tied to statistical norms
Accuracy is the strongest form of reliability (Weber, 1990)

22. Validity
Major concern for qualitative analysis in general
Researcher chooses coding concepts --makes inferences
Researcher bias, errors, conclusions
Neuendorf listed external validity, face validity, criterion validity,
content validity, and construct validity
Are we measuring
what we want to measure?
“
Validity refers to general applicability of results and
conclusions obtained from inferences in the study
Neuendorf, 2002, p. 112

23. Quantitative Analysis in Qualitative Research
Counting and statistics: Numeric measures
Word frequencies
How many times does a word appear?
Specify stop-words to ignore (e.g., the,
and, others)
Need to consolidate synonyms, stems
(e.g., dog = dogs)
Compound words (i.e., word pairs) are
important
United States
not good
Categories (simply present or
frequencies)

24. Quantitative Analysis of Qualitative Information
Concept frequencies
How often do concepts occur?
Existence (occurs) or actual counts
Other Statistics
Proximity and co-occurrence frequencies can all be used to determine
concept relationships

25. Qualitative Analysis
Coding is performed to reduce text collection to categories (i.e.,
concepts)
Analyst can seed concepts or discover concepts during analysis
Often, the more discovery allowed the more objective the analysis
(grounded theory reduces researcher bias)
Concepts and their relationships form the foundations for extracting
meaning
Keyword in context (KWIC)
Which words and how used (Weber, 1990)

26. Qualitative Analysis
Coding is performed to reduce text collection to categories (i.e.,
concepts)
Analyst can seed concepts or discover concepts during analysis
Often, the more discovery allowed the more objective the analysis
(grounded theory reduces researcher bias)
Concepts and their relationships form the foundations for extracting
meaning
Keyword in context (KWIC)
Which words and how used (Weber, 1990)

27. What is a Concept?
Synthesis of a text representation
Key words, including consolidating synonyms, stems
Represents something meaningful
Found by examining word, compound word, and surrounding words in
a measurable unit
Useful to display on a graphical “map”

28. Role of the Computer Solutions
A content analysis can be done without a computer. Although...
At a minimum, a computer serves as a document file folder and backup
device
And a search tool for and within documents
Software can also assist with manual coding then continue coding
automatically (N*Vivo)
Or software can do coding automated
by statistical processing (Leximancer)
or networks (CATPAC)

29. Key Points Summary
A content analysis is best when both
quantitative and qualitative
approaches are combined (Weber,
1990).
Quantitative analysis counts and finds
statistics
Qualitative analysis determines
meaning
Important operational aspects include
sampling, units of measure, coding,
validity, and reliability

30. References
Krippendorf, K. (2004). Content analysis: An introduction to its methodology
(2nd ed.). Thousand Oaks, CA: Sage.
Neuendorf, K. A. (2002). The content analysis guidebook. Thousand Oaks, CA:
Sage.
Weber, R. P. (1990). Basic content analysis. Newbury Park, CA: Sage.
Willig, C. (2008). Introducing qualitative research in psychology: Adventures in
theory and method (2nd ed.). Philadelphia, PA: Open University Press.

31. Additional Reading
Evaluation of Unsupervised Semantic Mapping of Natural Language with
Leximancer Concept Mapping, Andrew Smith.
Conversations Between Careers and People With Schizophrenia: A Qualitative
Analysis Using Leximancer, Julia Cretchley, Cindy Gallois, Helen Chenery, and
Andrew Smith
Analysis of Asynchronous Discourse in Web-assisted and Web-based Courses,
David Thomas and Cleborne Maddux
Computer Aided Phenomenography: The Role of Leximancer Computer
Software in Phenomenographic Investigation, Sorrel Penn-Edwards
Content Analysis of a Random Day of Two News Sites: FoxNews.com and
MSNBC.com, Michael R. Neal

32. Qualitative Analysis
1. Grounded Theory
2. Phenomenological Analysis
3. Discourse Analysis
4. Narrative Research
5. Intuitive Inquiry

33. THANK YOU
THE PSYCHOLOGY OF AUDIENCE ENGAGEMENT
Dr. Pamela Rutledge | prutledge@fielding.edu
Dr. Jerri Lynn Hogg | jhogg@fielding.edu