Part 1 of a 5 part lecture series on experimental design. This section deals with hypothesis generation, correlative vs manipulative experiments and choosing an appropriate model system. Text version of this content available at www.lantsandlaminins.com

1. Experimental Design

2. Welcome to the experimental
design lecture series
The series is broken up into 5
parts
Each part has examples and
questions associated with them to
help you test your knowledge
1. Experimental Preliminaries and
hypotheses
2. Measurements
3. Controls
4. Independence and assigning
groups
5. Sample size
determination
Kevin
LeeBurt
Krn
Conro

3. Experimental Preliminaries
and
Hypotheses
Experimental
design: part 1

4. Formulate a clear
primary research
question Big Question
Sub-question 1 Sub-question 2 Sub-question 3
Likely you won’t be
able to directly
answer this in one
experiment
Usually you will design
experiments to
answer a single sub
question
Experiment 1 Experiment 2
It’s better to answer 1
question well than 3
questions badly
Using multiple experiments
that answer the same
question will increase your
confidence in the findings

5. Most experimental
design happens at the
individual experiment
level
Big Question
Sub-question 2Each experiment
needs to be as robust
as possible for it to
contribute
meaningful data Experiment 1
There are three types
of experiment
1. Those that yield
interesting data, no
matter the outcome
2. Those that yield
interesting data, but only if
the experiment turns out
one way
3. Those that yield ambiguous,
uninterpretable results
regardless of outcome
Type 3 experiments are a
waste of time, money and
could be ethically
unforgiveable

6. Core Concept:
Hypothesis testing
We call the
prediction a
“hypothesis”
Experiments don’t
“prove” something
It’s (your predicted) answer to the
experimental question
Rather they are used
to test a prediction
At the end of an
experiment your
hypothesis is either
supported or refuted

7. Core Concept:
Hypothesis testing
Having a clear, well defined
hypothesis is a key first
step in experimental design
Hypotheses have
to TESTABLE
If you don’t know, and aren’t able to
explicitly state, what you are trying to
discover, it will be very difficult to
design you experiment well
Hypotheses have
to SPECIFIC
Key points:

8. How do you come
up with a
hypothesis?
First you need a key
set of observations
the first experiments of
an experimental series
might be designed to
generate these
observations
Descriptive
Studies
These could be
published work or
experimental data
We call these sort of
hypothesis generating
studies; descriptive
studies
Descriptive studies still
need to be carefully
designed if they are to be
useful

9. Descriptive
Studies
Often ‘omics or bioinformatics
type studies are viewed
“hypothesis generating”
These studies might be presented
as testing a hypothesis that
populations are different with
whatever manipulation is being
investigated
Gonna catch
me a
hypothesis
However, they may have a
more general aim of
characterising differences

10. Hypotheses have
to TESTABLE
Hypotheses have
to SPECIFIC
OK, let’s do it!Before you start,
remember:
Example time!

11. Examples
Normal
Cancer
Protein A Protein B
Protein A and Protein B are genetically related to each other. Previous
experiments have shown that in normal tissue protein A is highly
expressed (brown) whereas protein A shows low expression. In cancer
the pattern is reversed.

12. Examples
Normal
Cancer
Protein A Protein B
These sorts of
observations could
generate lots of
questions
Big Question
Sub-question 1 Sub-question 2 Sub-question 3
The big question might be
related to the clinical use
But you might also be
interested in how this switch
in expression is controlled or
what it means in terms of
cancer cell behaviour

13. Examples
Big Question
Sub-question 1 Sub-question 2 Sub-question 3
Let’s have a look at some
different types of questions
Normal
Cancer
Protein A Protein B

14. Examples
Q1
How often does this change happen? In which type of cancers? Does the
change in expression of protein A and/or B indicate disease severity?
Could measuring these proteins have diagnostic or prognostic value?
Q2
Q3
What causes the switch in expression to happen? Is it at the genetic
level or protein level? What is driving the switch?
What effect does the increase in protein B or decrease in protein A
have upon cancer cell invasion and metastasis?
Each of these questions are still
too big for a single experiment
so let’s simplify each to the first
part

15. Examples
Q1 How frequently does the change in expression of protein B happen?
Q2
Q3
Do the changes in expression happen at the mRNA level?
Does the increase in protein B influence cancer cell invasion?
Focus on the most
important questions
Normal
Cancer
Protein A Protein B

16. Examples
Q1
OK, we’ve got a questions, let’s
turn it them into hypotheses
Give it a go….
What would be
appropriate here?
How frequently does the change in expression of protein B happen?
Changes in expression of protein B occur in more
than 50% of cancers
Good start but you need your
hypothesis to be specific and
testable.
What type of cancer?
What direction will the change
be?

17. Examples
Q1 How frequently does the change in expression of protein B happen?
Protein B expression is increased in more than
50% of squamous cell carcinoma cancers
Better

18. Examples
Q1 How frequently do the changes in expression of protein A and protein B
happen?
Although we are studying protein B,
it might be possible to study protein
A at the same time.
Let’s change the experimental
question.
Can you come up with a
hypothesis that captures a
potential answer to this
question?

19. Examples
Q1 How frequently does the change in expression of protein A and
protein B happen?
Protein B expression is increased in more than
50% of squamous cell carcinoma cancers
Protein A expression is decreased in more than
50% of squamous cell carcinoma cancers
Protein A is decreased and Protein B expression
is increased in more than 50% of squamous cell
carcinoma cancers
An experiment can test multiple
hypotheses BUT make sure you
design it so that it can test all of
them effectively

20. Examples
Q1 How frequently does the change in expression of protein A and
protein B happen?
Protein A is decreased and Protein B expression
is increased in more than 50% of squamous cell
carcinoma cancers
Rank your hypotheses/questions based
on how important they are “primary
hypothesis, secondary hypotheses etc…”
Later in this series, we will discuss
sample sizes: you will use your primary
hypothesis for those calculations

21. Examples
Q1 How frequently does the change in expression of protein A and
protein B happen?
Why 50%?
Do be honest, it is
was just an arbitrary
number!
In situations like this, pick a
number that either comes from
your pilot data or is biologically
meaningful
What would be a useful
number?
For example here, would a certain
frequency of change mean the
information is useful as a
diagnostic or prognostic
biomarker

22. Examples
Q2
Q3
Do the changes in A and B protein expression also occur at the
mRNA level?
Does the increase in protein B influence cancer cell invasion?
What about the
other questions?
Try these now

23. Examples
Q2
Q3
Do the changes in A and B protein expression also occur at the
mRNA level?
Does the increase in protein B influence cancer cell invasion?
How about
something like:
The mRNA for protein A is decreased and mRNA for protein B is
increased in RNAs extracted from squamous cell carcinoma
tissue compared to RNAs isolated normal skin
Squamous cell carcinoma cells induced to overexpress protein B
display increased invasion compared with control treated cells.

24. OK, almost done with
hypotheses. But one final point:
Be careful with your
wording!
Note also that you may need to
revisit your hypothesis as you
proceed through the rest of the
design stages
When you say “due to” or
“influences” then your
experiment needs to
establish causality

25. Correlative or Manipulative?
My data shows a
positive correlation!
Great! What with what?
The number of
experiments I do and
how confused I
become!

26. Correlative:
Looking for associations
between one observation and
one or more other observations
For example:
“Smokers have lower lung
capacity than non-smokers”
In manipulative studies you
deliberately change something
to determine if it has an effect
For example:
“Lung capacity increases after
stopping smoking”
Let’s look at the two
main types of
experiment
Correlative or Manipulative?

27. Correlative or Manipulative?
Correlative studies are used
where manipulation is
impossible
Whereas manipulative studies
are used to gain mechanistic
insight
Correlative studies are often the
source of observations that
allow hypotheses to be
generated
Manipulative studies will allow
you to control for confounding
variables
Or allow you to determine
directionality, establish causality
or control for “reverse
causation”
How do you choose?
Let’s have a look at some of
these points

28. Key Questions
Is manipulation
possible?
Ethics, sample availability or
timeframe may mean you can’t
actually manipulate the system
For example; a study
investigating the impact of
childhood diet on lifespan in
humans would take 80 years to
completeOr where you predict a
manipulation could cause long-
term, unnecessary harm it
would be ethically wrong to do
the experiment

29. Key Questions
Next you should ask: Can
you actually produce
biologically realistic
manipulations?
Manipulations can introduce
things that would never happen
or in incomplete/irrelevant
contexts
If the data you obtain can’t be
interpreted in the real world
then is it worth asking?

30. Key Questions
Can you actually
infer causation?
Just because two things happen
together it doesn’t
automatically mean they are
causally linked!
What about correlative
studies? What do you
need to consider there?
http://www.tylervigen.com/spurious-correlations

31. Key Questions
Also, you need to
think about 3rd
variables
Also known as confounding
variables. These are other
things that could influence your
interpretation
We’ll talk about controls later, but if
you can’t find a way to eliminate 3rd
variables then you may not be able
to interpret your data.
Extra variables
Diet
Predators
Terrain
Weather
Genetics
For example, if you were comparing
experimental animals to wild
animals it might not be possible to
segregate one extra variable from
the others
Does X cause Y
Or is it
Z that causes Y

32. Key Questions
Does X cause Y or is it really
that Y that causes X?
You also need to ask; can
you control for reverse
causation?
Lower BMI
More
exercise
I exercise a lot
because I want to
stay thin
Causation
Lower BMI
More
exercise
I don’t exercise
because I am
embarrassed about
being overweight
Reverse
Causation

33. Correlative or Manipulative?
Q1
Q2
Q3
Protein B expression is increased in more than 50% of
squamous cell carcinoma cancers
The mRNA for protein A is decreased and mRNA for protein B is
decreased In RNA extracted from squamous cell carcinoma
tissue compared to RNA isolated normal skin
Squamous cell carcinoma cells induced to overexpress protein B
display increased invasion compared with control treated cells.
Your turn;
What approaches would you use
for these hypotheses?

34. Correlative or Manipulative?
Q1
Protein B expression is increased in more than 50% of
squamous cell carcinoma cancers
Correlative:
Measure protein B expression in
lots of squamous cell carcinoma
samples from human patients
Or, induce squamous cell
carcinoma in an animal
model and measure
protein B expression
Note that even though you
are manipulating (inducing
cancer) the output
measured is still a
correlation

35. Correlative or Manipulative?
Q2
The mRNA for protein A is decreased and the mRNA for protein
B is increased In RNA extracted from squamous cell carcinoma
tissue compared to RNA isolated normal skin
Correlative
Again, acquire samples and
analyse mRNA levels
If you wanted to do a
manipulative study you would
need a different hypothesis
For example: increasing
expression of the mRNA for
protein A causes a decrease in the
mRNA for protein B

36. Correlative or Manipulative?
Q3
Squamous cell carcinoma cells induced to overexpress protein B
display increased invasion compared with control treated cells.
Can you rephrase this hypothesis
to make it suitable for a
correlative study?
Manipulative
Here you would modify the
system: introducing
overexpression of protein B and
measuring the effect
Correlative
Squamous cell carcinomas with
increased protein B expression are
more likely to have metastatic spread
compared with those with normal
protein B expression

37. Choosing a Model System

38. Points to consider in your model
system choice
Ease of
manipulation
Direct vs indirect
effects
Cost and time
Ethics and approvals
Biological relevance
Complexity
Simplicity and tractability Biological
accuracy

39. The decision about what system to
use should be dependent on the
question you want to answer, not
the other way round!
Simplicity and tractability Biological
accuracy
Adding biological relevance by
using more complex systems may
seem good but won’t help if your
data become impossible to
interpret
You should also be
considering research
ethics in your study
design
Is the added benefit you get
from using animals/humans
in your studies actually
worth the cost to the subject

40. Squamous cell carcinoma cells induced to overexpress protein B
display increased invasion compared with control treated cells.
Example
Let’s look at one of our
examples from before…
In the literature there are
three widely used assays to
assess cell invasion
The end decision depends
on the question, and what
else is already known
Chick Amniotic
Membrane invasion
assay
$$
4-6 weeks
Variable
Invasion into more
biologically relevant substrate
Confounders
Migration through tissue
Colony growth and
proliferation
3D cell culture
invasion assay
$
2-3 weeks
Variable
Ability to invade
into a substrate
Confounders
Mouse graft model
$$$$
3-6 months
Variable
Invasion into more biologically
relevant substrates
Confounders
Migration through tissue
Colony growth and proliferation
Interaction with immune cells

41. Part 1 Recap.
Identify the important, interesting question you
want to answer
Form a clear discrete hypothesis that provides a plausible
answer to your main question
Decide the best approach;
manipulation or correlation?
Choose a model system that will allow you to address the
key question

42. Advice
from
Students
SamJess
Danielle
Adam