William Hudson is Principle Consultant at Syntagm Ltd and Director of User Experience for the Intranet Benchmarking Forum. He is the inventor of a popular card sorting technique (Computer Aided Paper Sorting – CAPS) and a related analysis package, SynCaps. The presentation will be of interest to anyone using or considering card sorting as a method of navigation design, but will be particularly relevant to CAPS and SynCaps users.

1. Advanced Topics in
Card Sorting
William Hudson
william.hudson@syntagm.co.uk
www.syntagm.co.uk/design/cardsort.shtml

2. Advanced Topics
Subgroups
Deviants
Navigational alignment
2
Copyright (c) 2008 Syntagm Ltd

3. Scope
Manual (where practical)
EZSort / EZCalc
Websort.net
SynCaps V1
SynCaps V2 (upcoming release)
3
Copyright (c) 2008 Syntagm Ltd

4. Subgroups
EZSort / EZCalc and SynCaps include
the notion of anonymous subgroups:
Vegetables
Carrots
Onions
Subgroup
Leeks
-----
Lettuce
Tomatoes
4
Copyright (c) 2008 Syntagm Ltd

5. Subgroups
Subgroups tantalize researchers with
the prospect of a three-level hierarchy:
groups, subgroups and items
The reality is not as promising
Subgroups are a means of refining
groups but the analysis is still only
monothetic (based on how often items
appear together)
5
Copyright (c) 2008 Syntagm Ltd

6. Proximity Matrix for Single Participant
n
no
vig
l
de
au
fan
Me e t a y
Sy ling io
do e
va t S
ar gn
be ai s
es ig
ar n
in
Cl n
Ca r n e
Ch p a
Ri t Gr
Ca uj ol
eZ
Pi at
Mu t
Wh
am
rlo
sc
no
ra
hit
a
Ch
Be
Beaujolais 1 1 1 1
Cabernet Sauvignon 1 1 1 1
Cava 1
Champagne 1
Chardonnay 1 1 1 1
Claret 1 1 1 1
Merlot 1 1 1 1
Muscat 1 1 1 1
Pinot Grigio 1 1 1 1
Riesling 1 1 1 1
Syrah 1 1 1 1
White Zinfandel 1 1 1 1
6
Copyright (c) 2008 Syntagm Ltd

7. Proximity Matrix for All Participants
n
no
vig
l
de
au
fan
Me e t a y
do e
Sy ling io
va t S
a r gn
be ai s
es ig
ar n
in
Cl n
Ca r ne
Ch p a
Ri t Gr
Ca u j o l
eZ
Pi a t
Mu o t
Wh
am
sc
no
ra
hit
rl
a
Ch
Be
Beaujolais 9 3 2 11 11 1 419
Cabernet Sauvignon 9 1 1 2 10 9 2 4 1 10 1
Cava 31 9 2314 1112
Champagne 21 9 1 13 21
Chardonnay 2 2 1 18 6 11 2 11
Claret 11 10 3 10 1 5 10 1
Merlot 11 9 1 1 1 10 4 2 10 1
Muscat 12 4 3 81 5828
Pinot Grigio 44 1 2 6545 637
Riesling 11 1 1 11 28 6 2 12
Syrah 9 10 1 2 10 10 2 32 2
White Zinfandel 1 2 11 1 1 8 7 12 2
7
Copyright (c) 2008 Syntagm Ltd

8. Subgroups
For EZSort / EZCalc and SynCaps
subgroups affect the values added to the
proximity matrix:
2 added for items in same subgroup
1 added for items in same group
8
Copyright (c) 2008 Syntagm Ltd

9. Proximity Matrix for Single
Participant (with Subgroups)
n
no
vig
el
nd
au
Me t y
Cl don e
Sy ling io
fa
va t S
ea
ar gn
be i s
es ig
ar n
in
Ca jola
Ca rne
Ch pa
Ri t Gr
eZ
Pi at
Mu ot
Wh
am
sc
au
no
ra
hit
rl
Ch
Be
Beaujolais 1 1 2 1
Cabernet Sauvignon 1 2 1 2
Cava 2
Champagne 2
Chardonnay 2 2 2 1
Claret 1 2 2 2
Merlot 2 2 2 2
Muscat 2 2 2 1
Pinot Grigio 2 2 2 1
Riesling 2 2 2 1
Syrah 1 2 2 2
White Zinfandel 1 1 1 1
9
Copyright (c) 2008 Syntagm Ltd

10. Dendrograms (SynCaps)
Single Participant
(no subgroups)
10
Copyright (c) 2008 Syntagm Ltd

11. Dendrograms (SynCaps)
Two Participants
Single Participant
(no subgroups)
(two subgroups)
11
Copyright (c) 2008 Syntagm Ltd

12. Dendrograms (SynCaps)
Notice that Apples and Plums
appear to be a subgroup
Three Participants
(subgroups and noise)
12
Copyright (c) 2008 Syntagm Ltd

13. Dendrograms
The ‘hierarchy’ of a dendrogram only
shows how frequently items were
grouped together
For well-aligned results, dendrograms
appears to reflect a ‘real’ hierarchy
Unfortunately, in practical use, this is
rarely the case – further examination of
the results is needed
13
Copyright (c) 2008 Syntagm Ltd

14. Items x Groups (SynCaps V2)
Breaks are for ease of reading
only (they occur every three
rows or columns)
14
Copyright (c) 2008 Syntagm Ltd

15. Pairs Map (SynCaps V2)
Deviant Results
Ideal Results
15
Copyright (c) 2008 Syntagm Ltd

16. Subgroups Summary
Open sorts
Subgroups are no substitute for multiple
sorts at different levels
Semi-open and closed sorts
Participants will be given some (semi-
open) or all (closed) group names
Subgroups can be safely used to refine
understanding
Remember that in EZCalc / EZSort and
SynCaps, subgroups are unnamed
16
Copyright (c) 2008 Syntagm Ltd

17. Dealing with Deviants
Deviants
Participants whose approach varies
dramatically from others
Items that participants do not group
consistently
In traditional cluster analysis no analysis
is performed on participants. Deviant
items appear only as weakly grouped.
17
Copyright (c) 2008 Syntagm Ltd

18. Deviant Items
‘Latest Updates’ and
‘Projects’ have very
week relationships
with other items
18
Copyright (c) 2008 Syntagm Ltd

19. Deviant Participants
Deviant participants have performed the
sorting task very differently to their
peers:
Very different number of groups
Extreme quality of fit (SynCaps)
Very low alignment (SynCaps)
19
Copyright (c) 2008 Syntagm Ltd

20. Quality of Fit
In Computer-Aided Paper
Sorting (Caps), each item can
be marked with a quality of fit
to its group
The fit is averaged across
items and participants
Average participant fit can
reveal extremes
20
Copyright (c) 2008 Syntagm Ltd

21. SynCaps Participant File
Groups Items Pairs AveSize %AveAlign %ExpertAlign
21 50 59 2.4 80% 73%
10 50 190 5 63% 59%
13 50 95 3.8 78% 69%
15 50 106 3.3 76% 67%
5 50 310 10 59% 50%
7 50 302 7.1 60% 52%
12 50 145 4.2 71% 65%
AveAlign shows agreement between participants
21
Copyright (c) 2008 Syntagm Ltd

22. Participant Scattergram (Excel)
%AveAlign by Group Count
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
0 5 10 15 20 25
22
Copyright (c) 2008 Syntagm Ltd

23. Pairs Maps (SynCaps V2)
All Participants Participants with
More than 10 Groups
23
Copyright (c) 2008 Syntagm Ltd

24. Pairs Map Reminder
Ideal Results
Deviant Results
24
Copyright (c) 2008 Syntagm Ltd

25. Deviants (200 Participants)
%AveAlign
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
Participants who may have been ‘in it to win it’
25
Copyright (c) 2008 Syntagm Ltd

26. Navigational Alignment
The SynCaps participants file includes
two measures of alignment:
Average alignment: how similar each
participant is relative to the average
across all participants
Expert alignment: how similar each
participant is relative to an ‘expert’ result
When the ‘expert’ result is the current
information architecture, we call the
measure ‘navigational alignment’
26
Copyright (c) 2008 Syntagm Ltd

27. Navigational Alignment
The pairs maps (SynCaps V2) is
adjusted to shown the ‘expert’ results set
as follows:
Participant
‘Expert’
27
Copyright (c) 2008 Syntagm Ltd

28. Navigational Alignment
Travel group
Expenses group
Travel Expenses
28
Copyright (c) 2008 Syntagm Ltd

29. SynCaps V1
Data & processing
Bar code or keyboard data entry
Simple file format
EZSort / EZCalc data supported
Output files
Cards, items, participants, proximity
matrices (weighted and unweighted)
Chart
Dendrogram
29
Copyright (c) 2008 Syntagm Ltd

30. New to SynCaps V2
Data capture & processing
Ad hoc items
Collapse items or groups
Duplicate items
Charts
Pairs map (with ‘expert’ markers)
Items x groups
Multi-page printing
30
Copyright (c) 2008 Syntagm Ltd

31. Items x Groups
31
Copyright (c) 2008 Syntagm Ltd

32. Workshops
Advances in Card Sorting (Workshop)
London, 12 May 2008
Edinburgh, 28 May 2008
Attendees will receive a fully licensed
copy of SynCaps V2
www.csadvances.com
www.syntagm.co.uk (design courses)
32
Copyright (c) 2008 Syntagm Ltd