1. A graphical grammar +
graphical inference =
a grammar of graphical inference?
Hadley Wickham, Rice University

2. 1. Motivation
2. Resampling methods
(graphical inference)
3. Graphical display
(grammar of graphics)
4. Future work

3. 5
8
6
●
● ●
4 ● ●
●
● ●
●
●
●
● ●
●
● ●
●●
2 ● species
● ●
● ● Concinna
●
● ● ●
● Heikert.
●
0 ● ●
●
● ● ● ● ● Heptapot.
● ●
●● ● ●
● ●
● ● ●
●● ●
● ● ●
●
● ● ● ●
● ●
● ●●
−2 ●
● ●
●
● ●
●
●
●
● ●
● ●
−4
−6
−6 −4 −2 0 2 4 6

4. 1 2 3 4
8
6
4
● ●● ● ●
2 ●●
●●● ● ●●
● ●● ● ●
● ● ●●●
●● ●
●● ● ●
● ●
● ●
● ●
●●●●● ●
● ● ●● ●●●
●● ● ● ●
● ●● ● ●●● ●● ●
● ●
● ●
●
●● ● ● ● ● ●●●●
●● ● ●
●●
● ●●● ● ●
● ● ●●
●●●
●●●●● ● ●●●
●● ●●●
● ●●●● ●
●
●●●● ● ●
0 ● ● ●
●● ● ●●
● ●● ●●
●●● ●
●●
● ●● ● ●
●● ●●
●●
●●
●
●● ● ●
● ●● ● ● ●
● ●
● ●● ●
● ●●● ●
●● ●●
●● ●
●●
●●
●●
●
●●
● ●●
●● ●● ●● ●
● ● ● ●●
●●● ● ●
● ●
●● ●● ●
● ●●●●● ● ●●
●●● ●
●● ●
●
−2 ●●
●
●
●
●
−4
−6
5 6 7 8
8
6
4
●● ●
● ●
species
●●●
●● ●
●● ●
● ●
● ●
2 ●●
● ●
●
●● ●
●
●● ● ●
● ● ●●
●● ● Concinna
●
● ● ● ●
● ●●
●●
●
● ●
●● ●● ●
● ●
● ●●
●●●● ● ●
● ●
●● ●
●● ●
●● ● ● ●● ● ●●
0 ●● ●● ●
●
● ●
●●●● ●
●●● ●●● ●
●●●●●
●●●● ●
● ●● ●● ●
●● ●
● ● ●●
● ● ●●
● ●● ● ●●●●
●● ●
●●
●● ●● ●
● ● ●● ● ●
●●● ●● ●
●
●●●●● ● ● Heikert.
●
● ●
● ●● ●
●● ● ● ●●
● ●● ●
●● ● ● ●
● ●
●● ● ●
●
●● ●
●●●● ●
●●●● ● ●●● ●●
●● ●
● ●●●●● ●
● ● ●
● ● ●
●
●●● ● ● ●
●● ● ● ●●● ● ●●
●
−2 ●● ● ●
● ●● ●
● ●
● ●
● ●
●
● ●● ●● ● Heptapot.
−4
−6
9 10 11 12
8
6
4
●
●
2 ●●
● ●
● ●● ●
● ●● ● ●
●
●
● ●
● ●●●●●
●
● ●●● ●
●● ● ●
●●● ● ●
● ● ●●
●●● ● ●
● ●●
●● ●
● ● ● ●
●●●● ●●
●●● ● ●
● ● ● ● ●●●● ●
●●
●● ●
●●●●●
●●
● ●●
●●●●
●●●● ●●●●●
●
● ●● ●
●●
●
0 ●●● ●●
● ● ●
●● ●● ●
● ●● ●
● ●●●●● ●
●●●●●
● ● ●● ● ●
● ●●●● ●
●
● ● ●●
●● ● ●
● ●●●● ●● ●●
● ●● ●● ●● ●● ●
● ● ●● ●
●● ●
●●● ●●
●● ● ●
●
● ●●●● ● ●
● ●
●● ● ●
● ●●●●
● ●●●
●● ● ●●●● ●
● ●●
●●● ● ●●●
●●
●●● ● ●● ● ● ●●
−2 ● ●
●
−4
−6
−6 −4 −2 0 2 4 6 −6 −4 −2 0 2 4 6 −6 −4 −2 0 2 4 6 −6 −4 −2 0 2 4 6

5. 1 2 3 4
●
4 ● ●
● ● ● ●● ●
●● ●
● ●● ● ● ●
● ● ●
●
● ● ● ●
2 ●
● ●
● ●
●
●
●●
● ●
●
●
●● ●● ●
● ●●● ●
●
● ●
● ●● ● ●
●● ●
● ● ● ●● ● ●
● ●●● ●
●
● ●● ● ● ● ●● ● ●● ● ●●
● ● ● ●● ●●
● ● ● ● ●● ● ● ● ●● ● ●● ● ● ●● ●
● ● ● ● ● ● ● ● ●● ● ●
● ● ● ● ● ● ● ●
0 ● ● ●
●
● ● ● ●●●●
●●●● ● ●● ●
● ● ●● ●
●
●● ●
● ● ● ●
●
● ●
● ● ● ●●
● ●●
●● ●
●
●
●
● ●● ● ● ● ● ●● ● ●●
● ● ● ●●
● ● ● ● ●● ● ●●
●
● ●
●
● ●
●● ● ●● ● ●●
●●●● ●● ●●
● ●● ● ● ●
● ● ● ●
●● ● ● ●● ● ●
● ● ●●
● ● ●●● ●
−2 ●
●● ●
●● ●
●
● ●●
●
●
●
●●
●
● ●
●
● ● ● ●
●
−4
5 6 7 8
●
4 ●
●
●
● ● ● ●
●
●● ● ● ● species
● ● ● ● ● ● ●● ●
2 ● ●● ● ●● ●●
●● ●
●
●
●● ●
● ●● ● ● ●● ● ● ● ● ●●●
●●● ● ●●
● ● ● ●● ●● ● ● ● ●●
● ● ● ● ●●
●● ● ● ● ● ●●
● ●● ● ●
●● ● ● ●● ● ●
● Concinna
● ●● ●
● ●● ● ● ● ● ●●● ●●● ● ● ● ●
●● ● ● ●●
0 ● ● ●
●
● ●
● ● ●● ●
● ● ● ● ● ●● ●
● ●
● ● ● ● ●● ● ●
●●
● ●● ●
● ● ● ●● ●
● ● ● ● ● ● ●● ●●
●
●● ● ● ●● ●
● ●● ●●● ●● ● ●
●● ● ● ● ● ●
●
● Heikert.
●● ●
●●
● ● ●
● ●● ●
● ● ●● ● ●
● ●● ● ●●
● ● ●
● ● ●
● ● ● ●● ●●
● ● ● ●● ● ● ●
●
●●● ● ● ● ● ●● ●● ● ● ● ●
−2 ●
●
●
●●
● ●
●
● ●
● ●
● ●
●
● ● ● Heptapot.
● ● ● ● ●
●
−4
9 10 11 12
4 ● ● ● ●
● ● ● ● ●
● ● ● ● ●●
●
● ● ●● ● ● ●● ●
●
● ● ● ●
2 ●●
● ● ● ● ●
●
● ●● ● ●● ● ●
● ●●●
● ● ●● ● ● ● ●● ● ●
●● ●
●● ● ● ●● ●● ● ● ●● ● ● ●
● ● ●● ● ● ●● ● ●
● ● ●● ●●● ● ● ● ● ●
●●● ● ●● ● ● ● ● ● ● ● ●●
● ● ● ●
0 ●●
●● ● ● ● ●●
● ● ● ● ●● ●
● ●
● ●● ●
● ●
●
●● ●● ●●●●●● ● ●
●
● ●●
● ●● ● ● ● ●
●
● ● ● ●● ● ●
● ● ● ● ● ● ●● ●
●
● ● ● ●●
● ● ● ● ●
●● ●● ●● ● ● ●● ● ●
● ● ●●● ● ● ●
● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ●
● ●
● ●● ●● ● ●● ● ● ● ●● ● ● ● ● ● ●
● ● ●●
−2 ● ● ● ● ● ●
● ●
●● ●● ●● ● ●
● ● ● ● ●
● ●● ● ● ●● ●
●
●
−4
−4 −2 0 2 4 −4 −2 0 2 4 −4 −2 0 2 4 −4 −2 0 2 4

6. Problem
Want to make it very easy to make these
types of display - should be able to create
as easy as doing as simple numerical test.
With the right tools, it turns out to be really
easy!
Two basic problems: how to do
resampling under the null, and how to
define the graphic.

7. Resampling
Many resampling methods have
particularly elegant expression in R code.
For example, if null hypothesis is that of
independence:
df <- transform(df, resp = sample(resp))
However, better to generalise the pattern.

8. Generalisation
permute_var <- function(var) {
function(df) {
df[[var]] <- sample(df[[var]])
df
}
}
f <- permute_var(quot;mpgquot;)
f(mtcars)
permute_var(quot;mpgquot;)(mtcars)

9. Advantages
Separate description from
implementation.
Call describes action.
Can later replace implementation if better
approach discovered.
Unimportant details concealed.

10. n resamples
Each method gives us a single resample,
but we need n.
Trivial to repeat n times with rdply() from
the plyr package.
(rdply() is a generalisation of
replicate() that returns a data frame
with a column that labels each replicate.)

11. Overall
resamp <- function(true, method, n = 19, pos = sample(n + 1, 1)) {
samples <- rdply(n, method(true))
if (missing(pos)) {
message(quot;True data in position quot;, pos)
}
add_true(samples, true, pos)
}
add_true <- function(samples, true, n) {
samples$.n <- with(samples, ifelse(.n >= n, .n + 1, .n))
true$.n <- n
all <- rbind(samples, true)
all[order(all$.n), ]
}
Will see application shortly

12. Other nulls
Need functions for other common null
hypotheses.
Experimental null_model(), which
computes rotation residuals given a
specified linear model. (Demo a little
later)

13. Display
How is the plot of the simulated data with
true data different from a single plot?
Just need to repeat the same display n
times and label appropriately.
This is easy if we have a description of
the plot, independent of the data.

14. Grammar of Graphics
This is one principle of the grammar of
graphics: should describe the graphic we
want, not how to create it.
Implemented with the ggplot2 package in
R.
Easy to modify a graphic after it has been
created. For graphical inference, we need
to change the data and facetting.

15. Example

16. ●
35
●
30
●
● ●
●
●
●
25 ●
●
●
●
●
● ●
●
● factor(year)
●●
●
cty
●
●
● ●
●
●●
●
● 1999
● ●● ●●●●
●●
● ●
●● ●
●
20 ●
●
● ●
●
● 2008
● ●●
●
● ● ●●●● ● ●
●
● ● ● ●
●
● ●
● ●●●
● ●●
● ●● ● ● ●
● ●●
●●● ● ●●●
●
● ● ●●
●
●
● ● ● ● ●
●● ● ●
●
●● ● ● ●
● ● ● ●
● ●● ●
● ● ●
● ●
● ● ● ●
15 ●
●
● ●●
● ● ● ●
●
● ● ● ●
● ●
●
● ●● ● ● ●● ●●
●
● ●
● ●
● ● ● ● ●
● ● ●● ●
●● ●●
● ● ●
●
●●
●● ● ●●
● ● ●● ● ●
●
●
● ●●●
● ●
● ● ● ● ●
●
● ● ●
● ●
●
10
●
●
●
●
2 3 4 5 6 7
displ

17. ●
10
● ● ●● ●●● ● ● ●
● ● ● ● ● ●
8
●● ●● ● ● ●
1/cty * 100
● ● ●● ● ● ●● ●
factor(year)
● 1999
●● ● ●● ●● ● ● ● ● ●
● 2008
● ●● ● ● ● ● ● ● ●
6 ●● ●● ● ● ● ●
● ●● ● ●● ● ● ●
● ● ●● ● ● ●
● ●
● ● ● ●●
● ●
● ●
● ●
4 ● ●
●
● ●
●
●
●
2 3 4 5 6 7
displ

18. 1 2 3 4 5
● ● ● ● ●
10
● ● ● ● ●●
● ● ●
● ● ● ● ● ●●
● ● ●
● ● ● ● ● ●●
● ● ●
● ● ● ● ● ●●
● ● ●
● ● ● ● ● ●●
● ● ●
●
● ● ● ●
● ● ● ● ● ●
● ● ● ● ● ●
● ● ● ● ● ●
● ● ● ● ● ●
● ●
8 ● ● ● ●●
● ● ● ● ● ●●
● ● ● ● ● ●●
● ● ● ● ● ●●
● ● ● ● ● ●●
● ●
● ●● ● ●
●● ● ● ● ●● ● ●
●● ● ● ● ●● ● ●
●● ● ● ● ●● ● ●
●● ● ● ● ●● ● ●
●● ● ●
● ●●●● ●
● ● ● ● ● ● ● ●●●● ●
● ● ●●
● ● ● ● ● ●●●● ●
● ● ●●
● ● ● ● ● ●●●● ●
● ● ●●
● ● ● ● ● ●●●● ●
● ● ●●
● ● ● ●
● ● ● ●●
● ● ●● ● ● ● ● ●●
● ● ●● ● ● ● ● ●●
● ● ●● ● ● ● ● ●●
● ● ●● ● ● ● ● ●●
● ● ●● ●
6 ● ●●●● ●●
●●● ● ●
●●
●●●
● ● ●● ●●
● ●●
● ●● ● ●●●● ●●
● ●●● ●
●●●
● ● ●● ●●
● ●●
● ●●
●● ● ●●●● ●●
●●● ● ●
●●●
●●●
● ● ●● ●●
● ●●
● ●● ● ●●●● ●●
● ●●● ●
●●●
● ● ●● ●●
● ●●
● ●● ● ●●●● ●●
●●
●●●● ●
● ● ●● ●●
● ●●
● ●●
● ● ●●
● ●●
● ● ●● ●
● ●●
● ● ●● ●
● ●●
● ● ●● ●
● ●●
● ● ●● ●
● ●●
● ●
●●●
● ●
● ●●
●●
● ● ●●●
● ●
●● ●●●
● ●
●● ●●●
● ●
●●
4 ●●
●
●●
●
●●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
● ●
● ●
● ●
● ●
●
6 7 8 9 10
● ● ● ● ●
10
● ● ● ● ●●
● ● ●
● ● ● ● ● ●●
● ● ●
● ● ● ● ● ●●
● ● ●
● ● ● ● ● ●●
● ● ●
● ● ● ● ● ●●
● ● ●
●
● ● ● ●
● ● ● ● ● ●
● ● ● ● ● ●
● ● ● ● ● ●
● ● ● ● ● ●
● ●
8 ● ● ● ●●
● ● ● ● ● ●●
● ● ● ● ● ●●
● ● ● ● ● ●●
● ● ● ● ● ●●
● ●
● ●● ● ●
●● ● ● ● ●● ● ●
●● ● ● ● ●● ● ●
●● ● ● ● ●● ● ●
●● ● ● ● ●● ● ●
●● ● ●
● ●●●● ●
● ● ● ● ● ● ● ●●●● ●
● ● ● ● ● ● ● ●●●● ●
● ● ● ● ● ● ● ● ●● ●
●● ●
● ●●● ● ● ● ● ●●●● ●
● ● ●●
● ● ● ●
● ● ● ●●
● ● ●● ● ● ● ● ●●
● ● ●● ● ● ● ● ●●
● ● ●● ● ● ● ● ●●
● ● ●● ● ● ● ● ●●
● ● ●● ●
6 ● ●●●● ●●
● ●●● ●
●
● ● ●● ●●
● ●●
● ●● ● ●●●● ●●
●●
●●●● ●
●●
● ● ●● ●●
● ●●
● ●● ● ●●●● ●●
●●● ● ●
●●●
●●
● ● ●● ●●
● ●●
● ●●
●● ● ●●●● ●●
●●● ● ●
●●●
● ● ●● ●●
● ●●
● ●● ● ●●●● ●●
●●●● ●
● ● ●● ●●
● ●●
● ●●
●● ●
● ●●
● ● ● ● ●●
● ●●
● ● ●● ●
● ●●
●●● ●● ●
● ●●
● ● ● ● ●●
● ●●
● ●
●●●
● ●
● ●●●
● ●
● ●● ●
●● ●●●
● ●
● ●●●
● ●
●
4
1/cty * 100
●●
● ●●
● ●●
● ●●
● ●●
●
●●
● ●●
● ●●
● ●●
● ●●
●
●
● ●
● ●
● ●
● ●
●
factor(year)
11 12 13 14 15 ● 1999
● ● ● ● ●
● 2008
10
● ● ● ● ●●
● ● ●
● ● ● ● ● ●●
● ● ●
● ● ● ● ● ●●
● ● ●
● ● ● ● ● ●●
● ● ●
● ● ● ● ● ●●
● ● ●
●
● ● ● ●
● ● ● ● ● ●
● ● ● ● ● ●
● ● ● ● ● ●
● ● ● ● ● ●
● ●
8 ● ● ● ●●
● ● ● ● ● ●●
● ● ● ● ● ●●
● ● ● ● ● ●●
● ● ● ● ● ●●
● ●
● ●● ● ●
●● ● ● ● ●● ● ●
●● ● ● ● ●● ● ●
●● ● ● ● ●● ● ●
●● ● ● ● ●● ● ●
●● ● ●
● ●●●● ●
● ● ●●
● ● ● ● ● ●●●● ●
● ● ● ● ● ● ● ●●●● ●
● ● ● ● ● ● ● ●●●● ●
● ● ●●
● ● ● ● ● ●●●● ●
● ● ●●
● ● ● ●
● ● ● ●●
● ● ●● ● ● ● ● ●●
● ● ●● ● ● ● ● ●●
● ● ●● ● ● ● ● ●●
● ● ●● ● ● ● ● ●●
● ● ●● ●
6 ● ●●●● ●●
● ●●● ●
●●●
● ● ●● ●●
● ●● ● ●●●● ●●
●●● ● ●
●●
●●●
● ● ●● ●●
●●●●●
● ● ●●●● ●●
●●● ● ●
●●●
● ● ●● ●●
● ●● ● ●●●● ●●
●●●
●●●● ●
● ● ●● ●●
●●●● ● ●●●● ●●
●●●
●●●● ●
●●
● ● ●● ●●
●●●●
● ● ●●
●
● ●●
●● ● ● ●●
●
● ●●
●● ● ● ●●
●
● ●●
●● ● ● ●●
●
● ●●
●● ● ● ●●
●
● ●●
●●
●●●
● ●
● ●●●
● ●
● ●●●
● ●
● ●●●
● ●
● ●●●
● ●
●
4 ●●
●
●●
●
●●
●
●●
●
●●
●
●●
●
●●
●
●●
●
●●
●
●●
●
●
● ●
● ●
● ●
● ●
●
16 17 18 19 20
● ● ● ● ●
10
● ● ● ● ●●
● ● ●
● ● ● ● ● ●●
● ● ●
● ● ● ● ● ●●
● ● ●
● ● ● ● ● ●●
● ● ●
● ● ● ● ● ●●
● ● ●
●
● ● ● ●
● ● ● ● ● ●
● ● ● ● ● ●
● ● ● ● ● ●
● ● ● ● ● ●
● ●
8 ● ● ● ●●
● ● ● ● ● ●●
● ● ● ● ● ●●
● ● ● ● ● ●●
● ● ● ● ● ●●
● ●
● ●● ● ●
●● ● ● ● ●● ● ●
●● ● ● ● ●● ● ●
●● ● ● ● ●● ● ●
●● ● ● ● ●● ● ●
●● ● ●
● ●●●● ●
● ● ●●
● ● ● ● ● ●●●● ●
● ● ●●
● ● ● ● ● ● ●● ●
●●
● ●● ● ● ● ● ●●●● ●
● ● ● ● ● ● ● ●●●● ●
● ● ●●
● ● ● ●
● ● ● ●●
● ● ●● ● ● ● ● ●●
● ● ●● ● ● ● ● ●●
● ● ●● ● ● ● ● ●●
● ● ●● ● ● ● ● ●●
● ● ●● ●
6 ● ●●●● ●●
●●
●●●● ●
● ● ●● ●●
●●●●●
● ● ●●●● ●●
● ●●● ●
●
● ● ●● ●●
● ●● ● ●●●● ●●
●●● ● ●
●●●
● ● ●● ●●
●●●●● ● ●●●● ●●
● ●●● ●
●●
●
● ● ●● ●●
● ●● ● ●●●● ●●
●●● ● ●
●●
●●●
● ● ●● ●●
● ●●
● ● ●●
●
● ●●
●● ● ● ●●
●
● ●●
●● ● ● ●●
●
● ●●
●● ● ● ●●
●
● ●●
●● ● ● ●●
●
● ●●
●●
●●●
● ●
● ●●●
● ●
● ●●●
● ●
● ●●●
● ●
● ●●●
● ●
●
4 ●●
●
●●
●
●●
●
●●
●
●●
●
●●
●
●●
●
●●
●
●●
●
●●
●
●
● ●
● ●
● ●
● ●
●
2 3 4 5 6 7 2 3 4 5 6 7 2 3 4 5 6 7 2 3 4 5 6 7 2 3 4 5 6 7
displ