This session offers the results of a study that tests the assertion that the online dissemination of theses has a positive impact on the research profile of the institution. Based on a combination of primary and secondary research, with some fascinating statistical comparative information, the study outlines the types of metrics an institution may use to measure the impact of its corpus of digitised dissertations and examines how these metrics may be generated. It is the result of a year-long study undertaken with the London School of Economics which focuses on the outcomes achieved through its programme of theses digitisation, disseminated simultaneously through its institutional repository and through the ProQuest Dissertations and Theses Database (PDTD). Results achieved by the LSE will be compared with metrics gathered globally by ProQuest via its PDTD. The session will be of interest to all librarians and academics involved in the use of digitised theses as a research resource, digitisation projects (retrospective or ongoing) and university rankings.

1. MEASURING THE IMPACT OF
DIGITISED THESES? A CASE
STUDY FROM THE LONDON
SCHOOL OF ECONOMICS
DIMITY FLANAGAN AND LINDA BENNETT

2. INTRODUCTION: AIMS OF THE PROJECT
• The chief aims of this project were to:
• Gain a greater understanding of how digitised theses are used and how they fit into
the scholarly resources landscape
• Track the progress of a single digitisation project (ProQuest and the London School of
Economics in partnership) and draw conclusions of its successes and challenges
• Compare the LSE project with a similar project that took place at the University of
Surrey
• Provide some initial comparators with more established digitisation projects in the
USA
• Provide some initial pointers on getting the most from digitisation projects / addressing
the challenges.

3. INTRODUCTION:
PROQUEST’S DIGITISATION PROGRAMME AND THE
LSE
• ProQuest was founded in 1939. The vision of its founder was to enable and
promote the dissemination of scholarly works
• From the start, the digitisation of dissertations / theses programme was central
plank of this mission
• Early mandate to publish US dissertations; PQDT is the official off-site repository
for US dissertations for the Library of Congress
• LSE and Surrey projects: ProQuest paid for digitisation of approx. 2,000 theses
for each institution to kick-start programmes.

4. • 97% of US Dissertations & Theses
• 300,000 Canadian Dissertations & Theses
• 50,000 UK Theses
• Expanding Coverage from Europe, Latin America., Middle East & Asia Pacific
More than 2.0 million graduate works in full text
PDF
• Simple bibliographic citations available for dissertations dating from 1637
• 500,000 from UK
• 150,000 from China
• Dissertation records from over 30 countries
4.0 million searchable citations from 1861
through to present
• ProQuest partners with 2,700 academic institutions worldwide to publish their
graduate works
130,000 new full text graduate works added each
year
The ProQuest EDT database

5. METHODOLOGY
LSE PROJECT
• Statistics collected from LSE project
• Statistics collected from Surrey and some comparator US projects
• Focus groups carried out at the LSE, with undergraduates, postgraduates and
librarians
• Semi-structured telephone conversations with LSE academics
• Surrey case study prepared.

6. THE LSE DIGITISATION PROJECT
• Commenced in 2014
• Opt-out approach
• Contact details sourced for all authors
• 14 opt-outs
• Uploaded April/May 2015
• 5 takedowns processed
• Downloads have increased substantially

7. LSE THESES ONLINE (LSETO)
283
13399
59843
106019
162390
430551
2010 2011 2012 2013 2014 2015
0
50000
100000
150000
200000
250000
300000
350000
400000
450000
500000
Downloads

8. LSETO PERFORMANCE
0.00
5.00
10.00
15.00
20.00
25.00
Downloads per item

9. LSETO USERS
1. Germany
2. United Kingdom
3. United States
4. China
5. India
6. France
7. Netherlands
8. Canada
9. Australia
10. Japan
11. Ukraine
12. Italy
13. Pakistan
14. South Africa
15. Turkey
16. Russia
17. Malaysia
18. Nigeria
19. Indonesia
20. Israel

10. HOW USERS GET TO LSETO
• 2011-Feb 2016
=87.92%
• 2015-Feb 2016
=88.89%
69.48
12.69
2.92
2.77
1.03
Google
direct
ethos
Lse.ac.uk
Bing
67.83
12.37
5.6
2.12
Google
direct
Lse.ac.uk
ethos

11. GOOGLE SEARCHES DOMINATE

12. WHAT IS IMPACT?
0
2
4
6
8
10
12
14
16
18
20
0
2000
4000
6000
8000
10000
12000
14000
Downloads
GS Citations
Data collected
16/02/16

13. WHAT IS IMPACT?
Data collected 16/02/16

14. WHAT THESES WERE SOUGHT OUT?

15. SOCIAL REFERRALS
• Self promotion
• Topical
• Conversations: Google groups, blog comments.

16. SOCIAL REFERRALS
Screenshot from
02/03/16

17. SPIKES
19 June 2015 – 886 sessions.
473 went to
http://etheses.lse.ac.uk/view/year/
458 came from Taiwan
Conclusion: Looking for Presidential
candidate’s thesis
12 February 2016 – 847 sessions.
412 went to http://etheses.lse.ac.uk/1054/
387 redirected from social networks
Conclusion: Finance Minister of Finland
tweeted/FB posted his own thesis

18. CONVERSATIONS
174 sessions due to a comment in a blog http://judithcurry.com/2016/01/23/state-of-the-blog-discussion-thread-2/
Gloucestershire Society for Industrial
Archaeology Google Group
commemorating a former journal
editor by promoting his newly
digitised thesis in Aug ’15

19. FOCUS GROUPS:
UNDERGRADUATES
• 7 participants, none from the UK
• 4 knew of the digitisation project
• One had consulted a digitised thesis; another had looked for one but not found it
• None had been recommended to consult digitised theses for content; one lecturer
had recommended for layout
• Key enthusiasms: usefulness of digitised theses for ‘cutting-edge’ work; conversely, to
provide a long-time historical audit trail
• Key concerns: accuracy / authenticity of content; possible damage to chances of
‘traditional’ publication.

20. FOCUS GROUPS:
UNDERGRADUATES: SOME COMMENTS
• “I wouldn’t like to submit a piece of work to be digitised. If I wanted to publish it
elsewhere, I might be accused of self-plagiarising.”
• If you can get access to a work in the early stages of research, it’s more interesting.”
• “I’m very interested in sharing my work with academics, to see if it can be useful to
them.”
• “What you’ve disseminated may not be an academic paper [in the finished sense] but
it may be something you can record and something to expand on later.”
• “We have to produce stuff in a very short period of time. It’s not deep, but it may
contain some good ideas.”
• “It would be interesting to be able to put data online as well as the main thesis.”

21. FOCUS GROUPS:
POSTGRADUATES
• Six participants, none from the UK
• One knew of digitisation project
• None had been told of the project by academic supervisors
• Views on having their theses digitised ambivalent (some did not realise this was an LSE
requirement)
• Happier about using other theses than digitising their own
• Key enthusiasms: timeliness and accessibility; help with finding a topic / not duplicating;
developing a historical perspective
• Key concerns: damage to publication opportunities (several would use embargoes); damage to
future reputation; difficulties in handling copyright / understanding permissions.

22. FOCUS GROUPS: POSTGRADUATES
SOME COMMENTS
• “It’s hard to find data if you’re working on an unusual subject. I was working in my field before it
became topical, so I couldn’t find much published material. It’s useful to know what research other
people are doing. I rely on ProQuest quite a lot.”
• “Citations are more interesting than downloads. It shows that there is an argument specific to that
thesis which is worth publishing.””
• “Personally, I would prefer not to make it available and if it is at some point I’d want to make it
inaccessible. The work that I’m doing may be proved wrong by subsequent work that I or someone
else undertakes.”
• “I’m carrying out work on far-right politicians. If I decide late that I want to run as a leftist politician,
what effect will having published the thesis have on my chances?”
• “I was told to publish a book if I wanted my work to be read by anyone, but I realised that in order to
get a job what I had to do was publish a thesis.”

23. FOCUS GROUPS:
LIBRARIANS
• Six librarians from the LSE took part
• The project was not specifically promoted to academics
• It was agreed that having a bank of digitised theses would enhance the prestige
of the LSE; also the Library, because “it’s seen to have ownership of the theses
collection. Academics expect us to be there to sort out this kind of thing”.
• Better metrics are needed
• Librarians, in conjunction with research office, have to provide coaching on
copyright and permissions.

24. FOCUS GROUPS: LIBRARIANS
SOME COMMENTS
• “We have 3,500 digitised theses. We have that number again and more that
haven’t been digitised. This means that we can’t manage expectations.”
• We’ve enabled ProQuest to [sell] subscriptions to the intellectual output of the
LSE; but they have helped us. I really like the ProQuest database. And it carries
the LSE logo.”
• “If we were able to digitise the earlier ones, they’d become more visible and
usable.”
• “Several of our theses have been written by notable historical figures who are
now eminent in their fields. We have some Nobel prizewinners’ theses, for
example. The historical perspective adds value.”

25. TELEPHONE INTERVIEWS:
ACADEMICS
• Four academics took part (areas of research: Cold War; Social Psychology; Criminal
Law; Gender). All teachers; two PhD supervisors
• Two knew of the ProQuest project (all knew that the LSE digitises theses)
• Two supported digitisation of theses wholeheartedly; two had reservations (again
connected with copyright / publication issues)
• Three had themselves consulted digitised theses
• Agreed that it would help cutting-edge research and the provision of historical
perspective
• Number of downloads ‘doesn’t prove anything’. It is the citations that count.

26. TELEPHONE INTERVIEWS: ACADEMICS
SOME COMMENTS
• “I totally recommend it. It’s a good way of extending information to the wider
community. It’s hugely important if it encourages work across disciplines. It gives
students a sense of how PhDs are researched across different countries.”
• “The more that’s available, the easier it is to plagiarise. Policing is the issue: ensuring
that 3, 4 or 5 years of work is not improperly used by someone else.”
• “Some theses are not also published in book format, or if they are, it’s a very long
time afterwards. And even if they have been commissioned as a book, they’re useful
to cite in the meantime.”
• “If someone could send me a summary [of usage and citations], that would be
interesting.”

27. LSE AND SURREY

28. LSE AND SURREY: TOP 10
Data collected 08/03/16

29. THE SURREY DIGITISATION PROJECT
FACTS
• Of the original 2,000 theses chosen for the Surrey project, about one third excluded
owing to copyright issues
• Cut-and-dried view towards author permissions taken
• “Destructive” scanning or not?
• The Library carried out energetic promotion of the programme, partly through its
discovery tool, partly through general communications channels and social
networking sites
• The Library offers training sessions for new PhD students, Masters students and
postgraduate tutors as the students begin their research projects.

30. THE SURREY DIGITISATION PROJECT
OBSERVATIONS
• Main problems connected with authors’ disregard of third-party copyright
• Fiona Greig: “The time we took to identify copyright breaches in our historical theses
was significant; to have to spend staff time retrospectively gaining permissions or
redacting information is not practical, nor does it represent value for money.”
• Significant concerns among students that they will be able to protect their own
Intellectual Property and copyright: since Surrey has mandated online access,
requests for embargos have increased. [Surrey publisher project]
• The impact of the complex related areas of Open Access, mandates, funding sources
and research data publication: in their infancy; much more work to be done.

31. PROQUEST DISSERTATIONS & THESES
GLOBAL
US University 1 US University 2 LSE
0
100000
200000
300000
400000
500000
600000
700000
800000
900000 Downloads in PQDT (2015-2016 March)

32. SOME OBSERVATIONS FROM THE PROQUEST
EDT BOARD
• It’s essential to promote the PQDT collection in order to ensure usage. (How well this
is done varies from one university to another, even where it’s well-established.)
• Promotion is down to the efforts of individual librarians, sometimes in conjunction with
other departments (Graduate Division, Research Office). Even in the USA, there is
no concerted effort to promote digitised dissertations and theses as a separate area
of research. Some (few) Asian libraries are doing pioneering work on this
• Good citation data is key to success (ProQuest can supply this for more recent
theses)
• As well as promotion, training (both virtual and by means of workshops, etc.) on
copyright, intellectual property, permissions, embargoes, etc. is essential for success.

33. WHAT THE LSE HAS LEARNT
• ProQuest project has been beneficial for traffic to LSETO
• Impact is often on the micro level – but even a small impact can be important
• Citations and downloads are very different things in the thesis world
• Subsequent publishing is often more consequential
• The LSE could do more to promote the collection

34. WHAT NEXT? THE WAY FORWARD FOR LSE
• To boost referrals, departments could do more to promote their theses
• We have changed our policy with EThOS in order to increase the number of
theses digitised
• More student awareness needed on the implications for future publishing

35. HOW FAR DID WE SUCCEED IN FULFILLING THE AIMS OF
THE PROJECT?
• Gain a greater understanding of how digitised theses are used and how they fit into
the scholarly resources landscape
• Track the progress of a single digitisation project (ProQuest and the London School of
Economics in partnership) and draw conclusions of its successes and challenges
• Compare the LSE project with a similar project that took place at the University of
Surrey
• Provide some initial comparators with more established digitisation projects in the
USA
• Provide some initial pointers on getting the most from digitisation projects / addressing
the challenges.

36. TAKING THE RESEARCH FORWARD
• Better data and better promotion of digitisation of theses projects are needed in order
to find out more about how digitised theses are being used as a scholarly resource
• The relationship between publishing a thesis in digital format and opportunities for
converting it to a traditionally-published monograph needs to be better understood
• Universities and university libraries need more help with explaining permissions,
copyright, intellectual property rights, embargoes and other author and publication
issues. Although they are likely to take individual approaches to this, some standard
information and resources, such as webinars, would be beneficial
• As well as being important in its own right, to maximise use of theses as a scholarly
resource, future work will contribute directly to the impact of the Open Access
movement

37. SELECT BIBLIOGRAPHY
• ETD: total cost of ownership – collecting, archiving and providing access. Han, Yan. Library Management, 2014, Vol. 35 (4/5), pp. 250 – 259.
• Dissertations and Research in an Era of Change. Herther, Nancy K. Searcher, 2010, Vol. 18(2), pp. 22-36.
• Web Citation by Graduate Students: a comparison of print and electronic theses. Kushkowski, Jeffrey D. Libraries and the Academy, 2005, Vol 5 (2), pp. 259-277.
• Morphing Metadata: maximising access to electronic theses and dissertations. McCutcheon, Sean, et al. Librarty Hi Tech, 2008, Vol. 26(1), pp. 41 – 57.
• Mandatory Open Access Publishing for Electronic Theses and Dissertations: Ethics and enthusiasm. Hawkins, A. R., et al. The Journal of Academic Librarianship,
2013, Vol. 39(1), pp. 32-60.
• Citation Analysis of M.A. Theses and PhD Dissertations in Sociology and Anthropology: An assessment. Rosenberg, Zila. Journal of Academic Librarianship, 2015,
Vol. 41 (5), pp. 680 – 689.
• Electronic Theses and Dissertations: Promoting ‘hidden’ research. Copeland, Susan. Policy Futures in Education, 2008, Vo. 6(1), p.87.
• Open Access to Research Data in Electronic Theses and Dissertations: An overview. Schopfel, Joachim, et al. Library Hi Tech, Fall 2014, Vol. 32 (4), pp. 612 (16).
• American ETD Dissemination in the Age of Open Access. Clement, Gail P. College and Research Libraries News, 2013, Vol. 74 (11), pp. 562-567.
• Do Open Access Electronic Theses and Dissertations Diminish Publishing Opportunities in the Social Sciences and Humanities? Findings from a 2011 Survey of
Academic Publishers. Ramirez, Marisa at al. College and Research Libraries, 2013, Vol 74 (4), pp. 368 – 381.
• Do Open Access Electronic Theses and Dissertations Diminish Publishing Opportunities in the Sciences? Ramirez, Marisa et al. College and Research Libraries,
2014, Vol. 75 (6), pp. 808 – 822.

38. CONTACT DETAILS
If you’d like to contact us:
Dimity Flanagan: D.Flanagan@lse.ac.uk
Linda Bennett: linda@goldleaf.co.uk