Presentation at the Electronic Resources and Libraries Conference in Austin, TX, April 3-6, 2016

1. Assessing User Experience of E-Books in Academic Libraries
Tao Zhang, Marlen Promann
1
Xi Niu
Department of Software and Information Systems
University of North Carolina at Charlotte
Purdue University Libraries

2. Project motivation
• Growing e-books in library
collections
• 1.5 million, 41.9% of total
volumes at Purdue
• Physical space
• Cost
• Access
• Positive attitude of faculty
and students
• Actual user experience?
• Exploratory study
2

3. Research goals
• Explore how users search for e-books in the library’s discovery
tool.
• Understand how users use e-books.
• Understand how users find information in e-books.
3

4. Research goal
• Explore how users search for e-books in the library’s discovery
tool:
• Log analysis
• Observation in user tests
4

5. Discovery tool
5

6. Log analysis
• Apache http logs from Sep., 2014
• 284,912 queries
• 6,854,362 actions
• Measures
• Initial query distribution
• Query level: query length, facet selection
• Session level: number of queries and http requests in search sessions
6

7. Initial query distribution
7
Search Tab # Search Queries
All (default) 44,502 (77.1%)
Books & Media 8,342 (14.5%)
Articles 3,532 (6.1%)
Course Reserves 1,335 (2.3%)
Total initial queries: 57,623

8. Query-level results
8
All searches
(Sep. 1, 2014)
All searches
(Sep. 12, 2014)
Book searches
(Sep., 2014)
Average 3.75 3.79 3.35
Median 4 3 2
SD 3.92 3.66 3.61
Query length

9. Query-level results
9
0
2000
4000
6000
8000
10000
12000
14000
0 1 2 3 4 5 6 7 8 9
Facet depth level
Search queries with facet selec>ons

10. Query-level results
10
0
1000
2000
3000
4000
5000
6000
7000
8000
9000
10000
Publica>on
Date
Show Only:
Peer Review
Subject Show Only:
Online Access
Material Type:
Books
Language Author
Top facet selec>ons

11. Session-level results
11
All searches
(Sep. 1, 2014)
All searches
(Sep. 12, 2014)
Book searches
(Sep., 2014)
Average 2.96 3.30 2.04
Median 1 1 1
SD 5.15 6.07 2.99
Queries per session
All searches
(Sep. 1, 2014)
All searches
(Sep. 12, 2014)
Book searches
(Sep., 2014)
Average 41.42 41.80 102.13
Median 21 21 73
SD 69.49 73.48 131.07
HTTP requests per session

12. Log analysis summary
• Users most likely use “Search All” with no more than two facets
• Book searches vs. general searches:
• Query length (effect size ~ 0.10)
• Queries per session (effect size ~ 0.24)
• Actions per session (effect size ~ 0.33)
• Identify relevant results from similar titles
• Examine item details
12

13. Research goal
• Understand how users use e-books:
• How long do users use e-books?
• How many users choose to read online vs. download e-books?
• How many pages do users read?
13

14. Usage log analysis
• EBL e-book usage logs from Jan. 1 to Nov. 25, 2014
• 29,495 individual reading sessions
• Data fields:
• Date & time
• User ID
• E-book title, ID
• Publisher, subject category
• Page numbers browsed
14

15. Reading session duration
• 69% reading sessions lasted less than 10 minutes
• 92% sessions lasted less than 30 minutes
• 2,261 sessions (8%) had download action
15
0.00%
20.00%
40.00%
60.00%
80.00%
100.00%
120.00%
0
2000
4000
6000
8000
10000
12000
100
1400
2700
4000
5300
6600
7900
9200
10500
11800
13100
14400
15700
17000
18300
19600
20900
Frequency
Dura,on (seconds)
Dura,on
Frequency
Cumula>ve %

16. Pages browsed
• 80% sessions involved no more than 30 pages browsed by users
16
0.00%
20.00%
40.00%
60.00%
80.00%
100.00%
120.00%
0
2000
4000
6000
8000
10000
12000
10
50
90
130
170
210
250
290
330
370
410
450
490
530
570
610
Frequency
Pages
Pages browsed
Frequency
Cumula>ve %

17. Pages browsed
17
Page numbers browsed by users across e-books and grouped by publishers and subjects
https://goo.gl/EXij2lTableau visualization:

18. Usage log summary
• E-books were not read from beginning to end
• Short reading sessions, relatively small number of pages read
• Browsing patterns
• Initial parts of book – Table of Contents – Start of a chapter
• Non-sequential reading, flipping back and forth
• Focus on certain page ranges
Users mainly use e-books to find discrete information and sections
(“fact searching”)
18

19. Research goal
• Understand how users find information in e-books:
• How does users’ experience level affect their information seeking behavior?
• How do users use book navigation features (table of contents, list of figures/
tables, index)?
• How do users use full text search in e-books?
• What are users’ difficulties interacting with e-books?
19

20. User test
• 12 participants
• Faculty, undergraduate and graduate students
• Screening survey measured library and e-book experiences
• 4 randomly selected from each level: beginner, intermediate, and expert
• Exploratory search tasks on general topics
• Own topic, Apple iMovie software, Marketing, …
• Information finding tasks
In White’s (2008) book on neuroscience, find the definition of Ideational Apraxia.
Open Norris’s book “Electoral Engineering: Voting Rules and Political Behavior”
and find a table that lists the ideal functions of political institutions.
In Geismar’s (2011) book Identify Basic Principles of Identity Design in the Iconic
Trademarks of Chermayeff & Geismar, find the meaning behind the Chase
Manhattan Bank identity design in the 1960s.
…
20

21. User test
• TechSmith Morae for task performance measures
• Pre- and post-test interviews
• E-book use
• Overall experience, things to be improved
• Observation
• Search (discovery tool, within e-books)
• Navigation (page, table of contents, index, list of tables & figures)
• Copy, annotation, download
21

22. E-book interface
22

23. E-book interface
23

24. E-book interface
24

25. Search tasks
• Observations
• Start from default “Search All”
• Simple keyword search and browse first page of results
• Few further query formulation
• Low usage of facets
• Rare use of advanced search
• Users’ difficulties:
• Scan large number of results
• Identify material type (book vs. article, journal, report, etc.)
• Identify format (print vs. online access)
• Determine relevance of results (only title, author, publication year in results
list)
25

26. Information finding task - video
26 https://goo.gl/XnCVJu

27. Search functionality in e-books
• Any keyword matching for multi-term queries
• Exact phrase search not supported
• Long list of search results
27

28. Differences among experience levels
• Beginners searched more
• Beginners clicked more in text (page numbers, table of contents)
• Beginners scrolled more (not supported in some e-books)
• Experts relied more on index, table of contents, list of figures/
tables to navigate
28

29. Difficulties in interaction
29
Copy text

30. Difficulties in interaction
30
Copy text

31. Difficulties in interaction
31
Rotate, view change

32. Information finding tasks
• Users expected searching in e-books similar to web searches
• E-books designed for linear reading, not for flipping/scanning
• Single page view, limited context for skimming
• Copyright restrictions (copy, download, print, special software)
• Discomfort of reading on screen
32

33. Design recommendations
• Search
• More information in search results (table of contents, book summary,
reviews)
• Promote facet usage
• Reading
• Better full text search
• More emphasis on print book features (table of contents, lists, index)
• Flexible page view options (rotate, multiple-page view)
• Less restrictions on copy, annotation, print, download
33

34. 34
Project supported by the 2015 ER&L + EBSCO Library Fellowship
Thank you!