Being a Data-Driven Communicator

Being a Data-Driven
Communicator
BYU-IDAHO MAY 2017
@valentinejames

Your superpower as a communication expert:
Tell a story that can
change the world.

Five lessons learned
along the way
COMMUNICATING
WITH DATA

Five lessons
learned along
the way

Prepare
Teach
One
Another
Ponder
and
Prove
Build and Maintain
Mental Models

The further you go out in time,uncertainty increases.

There are two types of
people in this world:
1. Those who can extrapolate from
incomplete data.

Tim,how do I
look?
RANDOM PERSON
ON THE STREET

Well if that’s
the look you
want,you’ve
got a good one.
TIM GUNN

What do I need to learn to be
successful?
INSTEAD ASK
AND REMEMBER…
Data doesn’t tell you what to do,but it can
give you an idea of what will be impacted by
your decisions.
Quantify tradeoffs: design experiments that
help you manage your risk going forward.

Analysis = Report + Recommendation

Observation
Inference
Decision
What we see.
What we think it means.
What we do as a result.
Basic mental model of decision-making.

What are they
seeing that
might be
reinforcing their
world view?
THINK PROBABILISTICALLY

1 . I N T U I T I O N
2 . R E A S O N I N G
Jonathan Haidt, “The Righteous Mind”
Find ways to trigger new intuitions, that
help prime the other(s) for new
reasonings.

M A K E A N O B S E R VAT I O N
A R T I C U L AT E T H E R I S K / O P P O R T U N I T Y
S U G G E S T P O S S I B L E N E X T S T E P S
RECOMMENDATION PATTERN

“The cause of
marketplace failure is
two-sided, with (1)
consumers
systematically
undervaluing and (2)
firms systematically
overvaluing the firm’s
innovation relative to
what an objective
analysis would
suggest.”
John Gourville,HBS

“The cause of
marketplace failure is
two-sided, with (1)
consumers
systematically
undervaluing and (2)
firms systematically
overvaluing the firm’s
innovation relative to
what an objective
analysis would
suggest.”
John Gourville,HBS
~FAIL~

Your
organization
is perfectly
designed to
get the results
it achieves.

1. Goals, metrics 2. Experiments, research
Aligning Success
Measure
Reality
Validation
Perceptions

Good strategy
is knowing to
what you are
saying yes and
to what you
are saying no.

Answering the Big Four
Questions will help you
and your team align
around a common
definition of success and
act on it.

What are we trying to
accomplish?
How will we know if
we’ve done that?
What do we think moves
the needle (up or down)?
How will we know if
we’re right or wrong?

Our Testing Strategy
• Outcomes
• Key Indicators
• Drivers
• Assessment Tools

Create mental models
Think probabilistically
Make recommendations
Calibrate continually
Champion process

T H R O U G H O U T T H E
R E M A I N D E R O F M Y L I F E , I
W I L L S E E K T O L E A R N B Y
W H AT I H E A R , S E E , A N D F E E L .
I W I L L W R I T E D O W N T H E
I M P O R TA N T T H I N G S I L E A R N ,
A N D I W I L L D O T H E M .
Richard G. Scott

Q&A
What is this form of testing called again?
I do A/B testing. The marketing/analytics specialization I work in
is “Conversion Rate Optimization”
Here are some industry resources:
• Conversion XL blog
• Optimizely's Optiverse

Q&A
How do you effectively test content?
Content can definitely be a little trickier than normal copy or testing design elements, but a lot of the basic
principles are the same. It is physically possible to A/B test an article where you have two versions and
randomly give your audience one version or the other and see if they do key indicators more or less often
compared to the other group. However, that usually takes a lot of time and you have to make a lot
assumptions in the process.
An alternative might be to write up an article two different ways (i.e. two different tones or differing levels of
detail/specifics) and then give each to two different test audiences ask them to rate it on things like clarity,
informativeness, entertainment or other attributes. See how the two different groups differ.
Another approach might be to introduce “mico-conversions” in the reading experience. For example, you’ll
see content publishers use “Continue Reading” buttons on their content. You read the first paragraph or so
and then it gives you the continue reading button. It’s a great feedback mechanism on the quality level of the
article. Also, you could introduce a poll on an article “Was this helpful? Yes or No” and then give them a
comment box. That can be really helpful because the user who comments will usually say what they were
expecting or hoping to find, which is really helpful information. Social sharing, email subscriptions and similar
metrics could also be used to see if content is doing its job.
Note: just in case anyone reading this isn’t aware—in
the industry, content usually refers to longer-former
written stuff: articles, blogs, etc and copy refers to
wording and verbiage on landing page. I wrote this
answer based on this definition of “content”

Q&A
Could you give an example of a use of the champion process?
Good decision making follows ﬁve basic steps:
1. You clearly deﬁne the problem.
2. You specify the criteria for a good solution to the problem.
3. You brainstorm possible solutions.
4. You evaluate the list of possible solutions against the criteria from step 2
5. You choose a solution.
Good process helps you not skip steps. My “big four questions” are key ways I try to make sure anything we
do maps back to what’s most important to the organization (the answer to question #1) and meets essential
criteria (what results matter, question #2) and then we work through possible solutions through testing and
measuring (question #3 and #4).
On the following slide I’ve included screenshots of a document that guides our conversation when we end an
A/B test. It helps keep us on topic and evaluate a test thoroughly. See if you can see follow along in how the
document helps our team reinforce the 5 steps above.

Q&A
Could you give an example of a use of the champion process?
(Answer continued)

Q&A
Does data testing involve a lot of trial and error?
A good testing program allows you to fail fast, fail often, but fail cheap. The old adage/joke in marketing/
advertising is that 50 percent of your advertising is wasted, the problem is you don’t know which half. Well,
testing takes that on directly. I would highly recommend “The Lean Start-Up” it does a great job of explaining
how you can use testing to manage uncertainty.
We all have to make assumptions in our work—good thoughtful testing allows you to reduce wasted effort
because you validate each step and manage risk.
Now if you don’t have a cohesive strategy behind what you are testing and why, absolutely it can be a lot of
trial and error that doesn’t lead anywhere. Lean Start-Up has a great framing question for testing: “What do I
need to learn to be successful?” If you build your tests around what you need to learn about your users,
you’ll ﬁnd even the errors are full of powerful insights that make your product better.

Q&A
Through ABC testing who do you get to test option
A, Option B etc when trying to find the right layout?
The testing platforms will randomly distribute folks between however many variations you make. That’s part of
my day job to set up the methodology. I also work on a lot on what each variation would be. That comes
down to identifying an opportunity and thinking through what will make the biggest difference to the user. For
example you might evaluate your current layout based on clarity and distractions and come up with a
variation that tests your current layout against a layout that you think reduces distraction.
Let’s say that test goes well, so you implement that change and now you think the next best thing you could
do is make it more readable or shareable or maybe users would stay on the site more if you suggested what
article to read next at the end of each article …
What you choose to test is going to be a function of what is important to the business and what you know
about the user’s experience. If something is confusing, you might make it clearer by doing this (A) or by
doing that (B). That’s the key—realizing there’s more than one way to solve a perceived problem and find
ways to test into a solution so you know what you are saying yes to and what you are saying no to.
Implied in that is a conversation about prioritization. What problems are more important to solve first?
Different groups have different ideas on that. Check out Wider Funnel’s Lift Model as an example or the PIE
framework or Conversion XL’s “PXL” (You should be able to google those). But prioritization is a key thing that
you don’t talk a lot about in school but is something key to how organizations function and make decisions
about what to focus on.

Q&A
What inspired you to pursue this career path?
I actually always thought I wanted to be a teacher. That began in high school when I read a
book over the summer called April 1865 that blew my mind about the Civil War. Then, I went
into American History and had such a watered down version of the same facts and basic plot
points. I was constantly turning to my friends in class and trying to help them appreciate it
more. I decided I wanted to be
On my mission, this evolved a little. I thought I might be a seminary teacher.
When I got into college, I started thinking, what if I taught college.
Then I realized: I just like teaching and communicating new ideas to my peers. And that’s
what helped me pivot into communication and public relations—telling the under-told story.
At the Church, to be a more strategic partner for my clients, I got into data because it was the
under-told story. I deep dived in analytics because others weren’t. This led a to a full-time
position on the analytics team (there were two of us). And then, once in charge of the
analytics tools, I realized we had this other tool, “Adobe Target” that was really powerful but
again under used and it’s ability to drive insight was under-told. So I invested time into getting
the organization to use it. Got to the point where I was given the green light to do that full-time.
And that’s been my specialization since.
(I still hope to teach one day, by the way.)
But maybe a general takeaway for students is realizing there’s what you learn and what you
do. There’s a whole lot more career options for an accounting major than just being an
accountant. The skills you develop as a comm major or as an analyst can be applied to a lot
of different directions. I realized I liked to talk to my peers and share with them new insights,
which helped me take the blinders off and think about my career options outside of traditional
terms. There were a lot of different ways to do the type of work that interested me.

How does strategy performance create effective permanence for a
company/organization/business, etc.?
Q&A
Another book suggestion: try Clayton Christiansen’s “Competing Against Luck.”
He argues that sometimes we treat innovation or success as something lucky or
something you can’t predict. He talks through how if we’re careful in how we frame the
problem, we can be much more systematic in driving performance. He’s developed a
theory called “Jobs Theory.” This theory developed out of his work on disruptive
innovation.
As an organization or product you have to ask yourself, “what job is my customer
hiring my product or service to do?”
If you understand what the true value your organization is providing (think of the
alignment between perception and reality in the venn diagram in above), then you
can measure the right things that indicate delivering that value. And if you are
optimizing your work around that value, you can position yourself for effective
permanence.

Q&A
Who's up for breakfast?
Anytime ya’ll are in the general Provo
area let me know, happy to grab
breakfast or lunch and talk shop.

0
25
50
75
100
SEO Paid Search: Brand Non-Brand Facebook Display
Q&A
How are you using Data to change Vivint from a summer sales
company to a technology company?
Our team is Acquisition Marketing, and we run vivint.com to drive business to our sales ﬂoor (call center).
We can track the entire purchase process from online to whether they call in and what happens after that
fact. We measure our digital marketing budgets by “Cost per install” So we take how much we spend on a
given channel (like Facebook or Google AdWords) and divide that spend by how many purchases it drove.
Each channel has it’s own CPI.
We manage it all using a portfolio approach. The entire portfolio of digital marketing spend needs to come in
at a goal CPI, but some channels are cheaper than others, which pretty much allows some channels to pay
for other channels. Visually it looks something like this:
Overall CPI GoalWe can spend over
our overall target CPI
in some channels
because of how well
other channels
perform.

0
25
50
75
100
SEO Paid Search: Brand Non-Brand Facebook Display
Q&A
How are you using Data to change Vivint from a summer sales
company to a technology company? (answer continued)
My contribution through all of this is to look at ways to bring down the CPI by making the webpages and
such more effective—optimized to drive leads to the ﬂoor. We test into better landing pages, which means if
more people install as a result, our cost per install goes down, allowing us to be more aggressive in that
channel and drive growth in that channel.
Overall CPI Goal

Q&A
Where were you located for your agency work? and What
dominated your strategic masters, was it client work or was
there a lot of busy homework?
Finch is a PPC agency located in Sugarhouse in Salt Lake.
My Masters in Strategic Communication was a combination of looking at rhetoric (what is persuasive?),
creating content for audiences, integrated marketing campaigns, organizational communication and digital
communication (website content, SEO, evaluation, etc.)
My program was very project based, which I appreciated it a lot. I worked very hands on with real clients.
For example, go check out peruvianhearts.org which was a non-proﬁt we worked with. You can see the
entire IMC plan we put together including branding, style guide and website measurement strategy at
peruvianhearts.org/planner.

Q&A
What were some of the metrics you came up with to measure
goals with the church?
There was a few things here that we did to make measurement mean something.
For the homepage we started by measuring clickthrough on articles to measure the homepage. If the
homepage’s job was to get someone to the next piece of content, let’s start there. It’s not the best metric in
other areas, but in this case it meant we were putting things forward that we’re relevant and interesting
enough for users to engage with.
We also tested into new data points. For example in the topics section, we would run a test and as part of
that test ask in the sidebar, “Was this helpful? Yes or No?” And then invite people to leave a comment. It
helped us start to quantify something that was otherwise hard to gauge.
If you use the feedback link in the footer, you’ll see a few survey questions. We added those so the feedback
had more context. “What were you here on the site to do today? We’re you able to do that?” And things like
that.
Finally, because “success” for the Church is changing hearts and minds and that’s not measurable, I’d frame
a lot of my results for tests in a way that helped quantify the tradeoffs between two different directions. For
example we might test a layout changes on the homepage where there are less links (more focus on the
articles themselves. I would measure clickthrough yes, but I also had a list of “Do not harm” metrics. These
were metrics that if we saw a dramatic swing in, we’d want to be aware of it. So while we’re measuring the
test by clickthrough on the homepage, I’m also keeping an eye on video views or % of users who logged in.
I’d then bring that additional context to the product manager and let them make as informed decision as
possible.

Being a Data-Driven Communicator

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (17)

Ähnlich wie Being a Data-Driven Communicator

Ähnlich wie Being a Data-Driven Communicator (20)

Mehr von James Valentine, MSC

Mehr von James Valentine, MSC (11)

Kürzlich hochgeladen

Kürzlich hochgeladen (10)

Being a Data-Driven Communicator