The document discusses the development and management of the Trove online community platform in Australia. It summarizes how Trove began as the Australian Newspapers digitization project in 2007 and expanded in 2010 to become a single discovery service for libraries, archives and museums. It describes how Trove engaged users by allowing them to correct OCR text, add tags and comments, and contributed their own content like photos and videos. Over time, Trove saw increasing user contributions that helped improve and expand the collection.
Potential of AI (Generative AI) in Business: Learnings and Insights
Building and Managing Online Communities
1. BUILDING AND MANAGING ONLINE
COMMUNITIES
A case study: Australian Newspapers
and Trove
International Congress of Archives
Web 2.0 Workshop, Brisbane 24 August 2012
Presenter: Rose Holley
roseholley@naa.gov.au
2. Overview
National Library of Australia
• Australian Newspapers Service 2007- 2010
• Trove single discovery service 2010-2012
Presentation focuses on user engagement
and online community
2
4. National Program and Content
• Initial focus on
major titles from
each state and
territory Northern
Territory
Times
• ‘Regional’ titles
contributed by
libraries 2010
onwards Courier Mail
• Coverage: West Australian
Sydney Morning Herald
Advertiser Sydney Gazette
published between Canberra Times
1803 – 1954 - 1984
Argus
• 7 million pages = 70
million articles Mercury
4
9. Building National Infrastructure
• Storage
• Newspaper Content Management
system (digitisation workflow)
• Panel of digitisation contractors (mass
digi)
• Quality assurance processes and team
• Public delivery system
9
12. Greatest fears!
• No one will do it
OR
• People will deliberately vandalise the text.
Questions?
• Moderation?
• Login?
• Integration of user data?
12
20. Public feedback on the
feature
‘OCR text correction is great! I think I just found my new
hobby!’
‘It’s looking like it will be very cool and the text fixing and
tagging is quite addictive.’
‘An interesting way of using interested readers “labour”! I
really like it.’
‘A wonderful tool - the amount of user control is very
surprising but refreshing.’
‘I applaud the capability for readers to correct the text.’
20
21. Why do it?
• I love it
• It’s interesting and fun
• It is a worthy cause
• It’s addictive
• I am helping with something important e.g.
recording history, finding new things
• I want to do some voluntary work
• I want to help non-profit making
organisations like libraries
• I want to learn something
• It’s a challenge
• I want to give something back to the
community
21 • You trust me to do it so I’ll do it
22. “Who are the text correctors?”
22
Flickr: LucLeqay
27. Achievements
Aug 2012 (4 yrs since release)
60,000+ volunteer text correctors
92 million lines of text corrected
in 4 million articles
1.6 million tags added
45,500 comments added
4 million users
70 million articles
27
28. 92 million lines corrected Aug
2012
Lines corrected millions
100
90
80
70
60
50
40
30
20
10
0
Dec-08 Dec-09 Dec-10 Dec-11 Aug-12
29. Build on success – create Trove
• Same infrastructure and principles
• Digital AND non digital
• Australian Galleries, Libraries, Archives,
Museums (GLAM)
• Full-text (books, newspapers) GOOGLE
• User-generated content Flickr, YouTube,
29
Wikipedia
30. NLA Strategic Directions 2009
“We will explore new models for creating
and sharing information and for collecting
materials, including supporting the
creation of knowledge by our users. “
(not just NLA resources… all Australian
content)
“The changing expectations of users that
they will not be passive receivers of
information, but rather contributors and
participants in information services.”
30
31. Trove Strategy 2010 -2012
Grow Content
Develop Service
Engage with community
Promote
31
59. http://climatehistory.com.au This landmark project, spanning the sciences and the humanities, draws
together a team of leading climate scientists, water managers and historians to better understand south-
eastern Australian climate history over the past 200–500 years. It is the first study of its kind in Australia.
59
65. Monitoring activity in an average
day
7,000
6,000
5,000
4,000
3,000
2,000
1,000
0
1am 4am 7am 10am 1pm 4pm 7pm 10pm
Pageviews (mirrors searching and text correction activity)
August 2010: Searching peaks at 11,000 per hour, text correction at
9,000 lines per hour, average number of unique users per day is 10,000.
69. NOVEL IDEAS
“The earliest novel that I collected was 'The Miser's Daughter'
by William Harrison Ainsworth. It was serialised in The
Colonial Times, Hobart starting in Aug. 1842. It is enjoyable
being able to read a story while doing the text correction.
The added bonus is being able to put them online in e-book
format for others to read. The process is quite time-
consuming and can take longer than the original
newspaper text-correcting, but it is very rewarding. So far I
have managed to create 105 e-books from the newspaper
text in the last 18 months. I upload the e-books to Project
Gutenberg Australia. I found recently that some of the
stories that we have uploaded have since been copied by
some other websites and set up in different file formats.
Some of the stories have even been put onto
Amazon.com.”
70. 14 Keys to success for
crowdsourcing
1. Clear and big goal on homepage
2. Progress towards goal is visible
3. Site is quick and reliable
4. Activity easy and fun
5. Results/outcome visible
6. Simple rewards and acknowledgements given
7. Content or topic is interesting (history, science,
animals, personal)
71. 8. Volunteer profiles are open and visible
9. Volunteers have an online team
environment e.g. wiki, forum
10. Volunteers have choices on how to work
11. We assume work will be done well and
volunteers can be trusted
12. The site is alive with new content
13. Site owner listens to ‘super’ volunteers
carefully
14. Topical/news events are used to your
advantage
72. Management of the crowd
• Volunteers largely manage each other
• Make their activity transparent to everyone
• Be IT savvy use – forums, blogs, wiki’s to
help
• Make it easy for volunteers to contact you
direct if they see an issue.
• ‘Shepherd’ rather than manage volunteers
– can be done an hour a week.
73. The Crowd
• Once content is liberated
anyone can become a
‘researcher’.
• The ‘ivory tower’ of gated
and protected knowledge
is gone.
• ‘Formal’ scholars are
replaced by the crowd in
the cloud.
• Today’s public are
educated and engaged,
demonstrated by their
participation in citizen
science projects.
73
74. Expectations of online users
“Self service, satisfaction and seamlessness are definitive
of information seekers expectations. Ease of use,
convenience and availability are equally as important to
information seekers as information quality and
trustworthiness.”
2003 OCLC Environmental Scan
To interact with content,
other users and the
organisation (web 2.0)
To be able to annotate content
and contribute their own
74
75. Important Things
• Connections • Sharing
• Linkages • Re-purposing
• Related • Mashing
• Context • Adding
Giving users
• Access to resources
• Tools to do stuff
• Freedom and choices
• Ways to work collaboratively
75
together
76. Where are the walls?
There are no walls only
bridges:
• People outside your
building are accessing
information within it.
• People inside your
building are accessing
information from
outside.
• Changing use of
spaces.
76
77. Power vs Freedom
“Freedom is actually a bigger game than power.
Power is about what you can control.
Freedom is about what you can unleash.”
Harriet Rubin
Rose says: We are gatekeepers who need to focus on
opening rather than closing doors….We need to change
institutional thinking.
77
Thank you for inviting me to speak here today. I am going to talk about changing roles for librarians and users, drawing to a large extent on my own personal experiences, with particular reference to Trove and Australian Newspapers.
ANDP started 2007. Website gives all project information. A great enabler in this program was the existence of ANPlan – the Australian Newspaper Preservation Plan. This is also a national collaboration between the state and territory libraries. For a number of years ANPlan had been working to ensure that significant newspapers were preserved via microfilming. This meant that the microfilms could be sourced and used for digitisation. Digitising newspapers from microfilm costs about a third of that from hard copy, so this has enabled much more content to be digitised.
The program is national and collaborative. Every state and territory library in Australia is involved. Each state and territory has made their selections of titles and provided microfilm to the NNLA and by 2011, 40 million articles will be available. After this it is hoped that state, territory and public libraries will begin to contribute regional titles. Key Features of this service will be that: It is accessible online Access is free The service is full text searchable. Up until now people wanting to research historic Australian newspapers needed to go to libraries across Australia and scroll through reels of microfilm. This program aimed to provide an online service that will let people anywhere, anytime access these newspapers via the internet. The service is now available. It is free. You can full text search across every page of every newspaper in the service, including advertising, cartoons, letters to the editor as well as the news and sports articles.
State titles 7 million pages
A complete microfilm of the Sydney Morning Herald did not exist but due to a very generous donation of $1 million by the Vincent Fairfax Family Foundation we have been able to source the hard copies of the SMH and include these in the service. This title is now complete except for 3500 missing issues - now found in stacks which will be digitised and made available soon.
Of all the items you can digitise newspapers are one of the hardest to digitise and deliver for a number of reasons. The NLA decided that setting up a national infrastructure and aiming to have all Australian Newspapers accessible from one place would be the best thing for the nation. This was a big challenge and one that no other country had attempted to take. The national infrastructure consisted of 5 things: Online and offline storage for digital files Development of a NCM to manage the digitisation workflows Development of a public delivery system to provide access to the newspapers Quality assurance processes and team Establishing a panel of newspaper digitisation contractors
Beta service. Quickly found, no publicity. Agile development, new content new features added.
One of the innovative features that was in the first release was the ability for members of the public to correct or enhance the OCR text. When digitising old newspapers the process is to convert a digital image into full-text by use of Optical Character Recognition software (OCR). This works well on new clear documents but on old newspapers where the font and paper is of poor quality and microfilms may be out of focus the translation often goes into gibberish. After investigating every possible way technically of being able to improve this we came to the conclusion that the best way was by hand and human eye. We could not possibly afford to pay contractors to do this ‘re-keying’ so the lead programmer Kent Fitch suggested we open it up for the public to do. If text was made accurate the searching would be instantly improved for everyone since the search works over the OCR text.
This was very controversial since no such thing had ever been done by a library or archive before and was considered high risk. The risks were identified as: No one will do it OR People will deliberately vandalise the text. The likelihood that people would just do lots of it well was considered extremely unlikely. Because it was unknown if people would do it, it was decided not to put valuable time into developing a moderation module which may be unnecessary but to just see how it went without moderation. Also to lesson barriers it was decided registration would not be necessary. To reduce risk the added data would be kept in layers and not integrated into the original metadata (although it would appear to users as if they had changed the actual metadata). All layers of data would be searched on. While we were doing this we also decided to add in the ability for people to add comments and tags to articles as well. NEXT – quick demo.
Now I have zoomed in on the image and if the OCR text was inaccurate I would edit it in the box on the left. This is what we call the power edit mode. In this article the text is actually very accurate so has either OCR’d very well, or already been corrected by someone else.
This is the text correction history of this article, showing all the different users and what parts they corrected.
This is the article view. Users can zoom in or out and choose to view the article in the context of the entire page. They can also navigate to any other page within the newspaper issue. The electronically generated text created through the OCR process is displayed on the left hand side. This is also where the users can use the 3 enhancement features. They can drag the viewing pane to see more of the or less. Users can tag the article with keywords and they can write comments and notes about the article. If users login they will be able to choose to make their tags and comments public or private. So they can share their comments with all users or they can add their own private research notes that only they can access. One feature that we believe is innovative and not available in any other online newspaper service, is the ability for the user to correct the electronically generated text. There are a number of reasons why the electronically created text is not always 100% accurate, mainly due to the quality of the original newspaper that the image was created from. Users can correct the text by clicking on the ‘Help fix this text’ button. We will now use these features on this article. The article we are looking at is the first report in an Australian paper of the sinking of the titantic.It’s in the Northern Territory Times on 19 April 1912.
I want to tag the article with ‘titantic sinking’. If a user does not login when they first enter the service then the first time they want to enhance an article they will be offered the option to login. At this point they can either login or enter the captcha to verify they are human (and not a robot – attempting to do something undesirable).
Once logged in or verified with captcha a user can enter their tags.
Now I want to add a comment. Those of you who read this article may have noticed that it was reported that all passengers were safely rescued from the titanic and the weather was calm. I’ll just add a comment to say this was unfortunately not the case.
Now we can review the article with all the enhancements we have made showing on the left. Tags, comments and corrections. We can view the history of all the enhancements (both ours and other peoples history).
We did not prompt the public to give their opinion on text correction, though many did anyway. Although no-one had encountered anything like this before they quickly understood the aim and thought it was a good idea. Once people understood they likened it to Wikipedia, though it was not quite the same since the original image is always there for verification.
But also people gave these reasons for doing so much – in some cases up to 40 hrs a week. Because after all if you don’t enjoy it you wouldn’t keep at it, so loving it and finding it interesting and fun were really important.
So after all this activity the most common question people kept asking me was “Who are these people?” and also “Why do they do it?” Some people even suspected that the text correctors were really library staff, which is not the case. The text correctors are real, normal people. We sent some of them a survey to find answers to our questions about how long they spend correcting, why they do it, what motivates them, what would motivate them to do more or less? The responses were very interesting. The majority of the work is done by ‘super’ users or volunteers. The top 10% of volunteers can do as much as 89% of the work. Their age varies. It is not all older people as some imagine, in fact it is highly likely that moderators or those with extra responsibilities, or the super users are dynamic young professionals who have full-time jobs. There are retired people, but also stay at home mums and disabled or sick people. The volunteer profile is broad.
In July 2010 a survey was undertaken on Trove to find out more about the user base. This was primarily to help with the usability testing. The results show that the majority of users consider themselves to be researchers, with a significant portion interested in family history. Librarians also featured and many of these were acting as intermediaries on behalf of users. The survey showed there was very little use of Trove by educators, or primary or secondary level students. This audience group has not yet been targeted in the marketing strategy. The survey results may not be 100% accurate since they only reflect those users who chose to fill out the survey.
Julie the top corrector has featured in the media and become a star. She loves correcting articles about Bendigo murders.
Because the work these people are doing is so invaluable the Director General of the NLA decided to honour the top correctors in the annual NLA Australia Day Awards (which is usually for library staff). The text correctors are considered part of the NLA family for the invaluable work they are doing. It was very interesting for me to finally meet these individuals in person and be able to thank them. They had not necessarily realised what an impact they were making. The busiest corrector Ann Manley achieved a personal best of 1 million lines single-handedly in Dec 2011. The day she achieved this was International Volunteers Day.
This graph shows the rising rate of text correction. Text correction peaks over the christmas period and long weekends. There has been no time since release of service when text correction is not taking place. It goes on 24/7. In 2010 – huge increase in text correction. 2 million lines per month
Build on success – utlisise newspaper infrastructure for everything else. Single search. Integrates existing services into it. Fully migrate services 2 needs – efficiency in maintenance of single service behind scenes Satisfy public need to have single google style search
And now we are in 2010. We have firmly acknowledged that in order to give users the best service, collaboration and data sharing are key. But more than this. The two direction statements the NLA is working towards are as follows: We will explore new models for creating and sharing information and for collecting materials, including supporting the creation of knowledge by our users . “ “ The changing expectations of users that they will not be passive receivers of information, but rather contributors and participants in information services.”
I am now going to talk about Trove – the search engine for Australian resources. Australian Newspapers service was in fact a test bed for the idea to transform service delivery and our internal IT infrastructure in the future. Because Australian Newspaper worked so well the beta model of software development, the underlying IT infrastructure and the application of user engagement has been applied to all the other discovery services the library manages which are rolled into Trove. Content of zones The key features of Trove are that 1. Firstly, and most importantly it is a single search. In one click you can simultaneously search across several groups of information- books, journals, magazines and articles: images: australian digitised newspapers: diaries, letters, archives: maps: music, sound, video: archived websites, about people and organisations. 2. Secondly you can browse through these groups or zones one at a time if you prefer to only seek one type of content for example newspapers. 3. Thirdly you are able to restrict your searches to – online content only, and/or content held in locations near to you. This is very useful feature for the large majority of users.
Results are unbiased – best and most relevant info possible – relevancy ranking. Similar to values of a good reference librarian (subject to initial choices made by user eg location, immediacy). Results are returned in the same zones that we saw on the home page. You can see in each zone how many results are found. Most searches retrieve vast numbers of results because of the wealth and richness of the repository that is being searched. It is likely that you will want to refine or limit your search results and you can do this by using the facets on the left hand side of the screen. The facets change depending what content you are looking at, so for example the book, journal, magazine and article zone has a facet to refine by braille book or audio book. We recognise that many people just want items that are immediately accessible ie digitised or online, as fast as possible, so the links to online content appear immediately at this stage although we haven’t yet drilled down to a detailed results screen. The check boxes to restrict the content to online or Australian are always visible so that they can be checked or unchecked at any point in the search. Here is an example – Merino sheep originating from Spain. The first twenty six merino type sheep were introduced in Australia in 1797 from Cape Hope, South Africa. Merinos are originally the typical Spanish sheep; bred five thousand years ago by the Tartessos at the valley of the river Guadalquivir, southern Spain. Thank you Spain! Numbers have grown in last 200 years and now have over 100 million in Australia.
New home page 2011 – Contribute has greater prominence The recent interactions users have made are also displayed on the homepage for everyone to see. You can see the number of searches in the last hour, newspaper article corrections so far today, works merged or split this week, items tagged this week, and comments this month.
Monthly hall of fame – requested by users
Profile enhanced with activity – total rankings
About 50,000 new digitised articles are added each week to the Newspaper zone. Many users have requested alerts for new content based on keyword search terms for the whole of Trove. This is still in progress. In the meantime alerting for individual articles has been implemented. Because these items in digitisation progress appear as ‘coming soon’ in the results list an user can choose to be notified when the item arrives. This radically improves the ‘get process’ for newspaper articles. Also subject keyword feeds
Lists – adaptable, different purposes, by user demand. Personal- private or public
This is a newspaper title information page – showing the date range digitised, title pi and information being pulled from wikipedia. Rich info rather than just bib data
This is what you would see for the Argus. We’ve added the link to the digitised newspapers as an external link at the bottom. For most newspapers there is already a page existing, but if there wasn’t I created one and it quickly got populated.
Nov 2010
Forum so users can discuss and contact each other
3000 comments and feedback
Current photos 150,000 added – about 3,000 a month
Old photos – especially for identifying people
Peoples names in comments. Searchable separately. For example users can comment on items as well as newspaper articles now eg images, books and archives and share valuable information, and rate items.
Inst – virtual exhib
Educators, replicate trails, slideshows except can do themselves, teaching resources for classroom
Re-purpose trove data. Smithsonian commons
Interconnected to forum and service, purpose to promote service and content within it. Build virtual environment.
Tweet
Happy, sad??
Linked to tweets and social enagement, Raised money in few weeks
Physical group – met during floods
Leading climate research project, harnesses power of service and users
National news and SMH
Users start to make screencasts!
T-shirt not good – too expensive too many sizes. Ask me about Trove – too challenging. I love trove good. Gold star system like blood donor bank
Notes: weekend usage is slightly lower than weekday usage. Weekend usage – more activity in the small hours. Text corrections – 75,000 per day and 1.5 million a month. Searching 11,000 per hour peaks at 3pm. This is very similar to the OCR correction table done 2 years ago. 10, 000 unique visitors in a day.
Trove enables anyone to view the search terms in recent searches. The most popular search terms do not vary much throughout the year and reflect the heavy usage of the digitised Newspapers zone. They included people’s names (especially William, George, Henry and Smith), births, deaths, family notices, shipping, cricket and railways. In January 2011 there were three significant spikes in usage, one caused by new content and the other two by Australian news items. The Wordle below shows these terms: ‘Lionel Logue’ (starring in The King’s Speech movie), 1974 Brisbane Flood (as a comparison to current floods) and ‘knitting patterns’ now available in the recently digitised Australian Women’s Weekly.
knitting regains popularity and we see the resurgence in ‘retro’ fashion. The knitting community are falling with glee on digitised historic Australian newspapers and the Australian Women’s Weekly, particularly from the 1950’s. Someone has added the instructions into Ravelry for how to find vintage knitting patterns in Trove, which is search for knit+"cast on" or knitting patterns (now one of the top search terms – see the wordle above). If you do this in Trove you get nearly 73,000 results for knitting patterns. Most newspaper included at least one pattern a week. Of all those patterns the community has chosen to add some of the more popular ones into Ravelry so more community engagement can happen. So far 290 have been added from Australian newspapers and the Australian Women’s Weekly. A classic number ‘a cosy cardigan’ which appeared in the Sydney Morning Herald of 1953. It has been favourited by 145 people, one has knitted it and 93 people have added it to their queue of things to knit next. The person who has knitted it has added notes and instructions on how they did it with a colour picture of the finished garment.
The most favourited pattern added to Ravelry from the National Library of Australia’s digitised Australian Women’s Weekly collection is… wait for it…… the ‘Elegant Elephant’. It has been favourited by 690 people, rated 4 out of 5 and easy to knit, knitted by 21 people in a variety of colours, and 174 more people intend to knit it soon. If you click on the pattern you will see uploaded photos of finished knitted elephants….
Feedback and compliments – about 10 a day. Questions – 3 a day.
Whilst we focused on digitisation and delivering things online we did not seem to realise that the public lost valuable social interactions that they had before in the physical environment. Web 2.0 is really about building those social engagements back into the digital world. To give you an example when I worked in the reference library users often used to underline words in books and write their own notes by the side. If they did this they would get a stiff fine, and if they did it repeatedly they would be banned from the library, but people still continued to do it. Now on a digital book it is possible to add a virtual post it note, or an annotation or comment and we encourage this!! It has taken us a long time to realise that what the users want to do, has not actually changed, and it is good to let people do these things.
Users rate finding information EQUALLY as important as doing stuff with it. ‘Doing stuff with it’ is not the icing on the cake it’s core.
Access services remotely. Access services on mobile devices Digital Commons NLB Singapore ‘Library in my pocket’
Don’t under-estimate the power of people. People who join together can accomplish amazing things. We can give them the tools to do this. (video next)