ILS on a shoe-string budget: open source software in a non-profit organization
Jolene Bennett and Zachary Osborne
Without the budgetary means to purchase proprietary software or hire consultants, the
Weston Family Library at the Toronto Botanical Garden embarked upon an
ambitious project to migrate its catalogue from InMagic to Koha, an open-source ILS platform. This session outlines the resources and processes that were used,
including volunteer technical expertise, freely available software, and other web
applications in order to migrate non-MARC bibliographic data from InMagic to a usable
format for the new Koha platform. We describe the win-win process of using
volunteer talent in a tight labour market to create an ILS in a non-profit organization.
By recruiting library technician and librarian volunteers, the library gained its necessary catalogue upgrade, and volunteers gained incomparable and marketable experience.
6. Weston Family Library
• Context and History
– History
– Library in a non-profit
organization
• staffing, collection
development, budget
– Collections
– Users
– Systems
7.
8.
9. Vision: Library Updates
• Update and modernize both the physical
and virtual space of the library
– Introduce new seating options
– Re-consider space and furniture
– Introduce rotating art gallery exhibitions to
draw visitors into the library
– Expand web presence
– Modernize the library catalogue
10. Vision: Library Updates
• Physical Space
– Laptop bar with stools for added seating
– Couch and quiet reading area
– New lamps and lighting
– Artwork
– Removal of storage cabinets to create more
comfortable environment
– Improved signage
11.
12.
13.
14.
15.
16. Vision: Library Updates
• Virtual Space
– New web-based catalogue with added
discovery and improved end-user experience
• Something that the library volunteers could use
more intuitively and with fewer challenges.
• Financially sustainable alternative to proprietary
library software.
• An integrated library system (ILS) that is dynamic,
flexible, and interoperable with the TBG’s website
and the Internet.
17.
18. Research: What?
“Which platform should we implement?”
“What are our options?”
“What are similar small libraries using?”
Considerations:
• Budget
• Proprietary or Open Source?
• IT or Vendor support options?
19. Research: Why?
• Open source ILS:
– Free (i.e. no related costs to download & use)
– Customizable
– Powerful enough for our small library
• Why :
– Established community of support
– Web-based, z39.50 compatible, modern features
– Intuitive layout, and familiar design
20. Research: Why?
Inmagic
• Bibliographic fields
• Limited authority control
• No interoperability with
library or TBG website
• Challenging for library
volunteers and users
• Annual maintenance fees
Koha
• MARC fields
• Authorized values
• Seamlessly embedded
within TBG website, and
interoperable.
• Intuitive and basic design
• Free, and open source
21. Research:
Now for the big question: How?
How will we…
• …Carry out data migration from Inmagic to Koha?
• …Without a budget for vendor or IT support?
• …Or a budget for dedicated library staff to
support the new ILS system and conversion
process?
22. Research: How?
• Volunteer assistance:
dfd
Brian Stewart:
– Download and install a Debian server, Apache
software, MySQL, and Koha
dfdf
Library System Support Analysts:
– To help establish Koha parameters for the library and
migrate approximately 10,000 records
• Identified readily-available, inexpensive or free
resources to help carry out the project
• Flexible attitude, creative problem-solving, and
openness to problems and challenges as they arose
25. Methodology - First Steps
1. Download & Install Koha
2. Set System Preferences
3. Export records from InMagic
26. Spreadsheet “Before”
• 9704 records
• Each record with 67 fields
• Many fields blank (null)
• General bibliographic fields
27. Sample Record (portion)
Title: The rose :
ready to be mapped to MARC field 245 subfield a
Subtitle: myth, folklore and legend
Ready to be mapped to MARC field 245 subfield b
Author: Mayhew, Ann | Pollard, Michael, ill.
Subject: Roses – Folklore | Roses – Mythology
Pub. Place: New York ready to be mapped
Publisher: Walker ready to be mapped
Pub. Date: c1979 ready to be mapped
Notes: Out of print 1981. | Includes bibliographical
references.
29. Excel
i. Reorganize records
ii. Analyze records and field structure
iii. Restructure fields using Excel Functions
iv. Restructure fields manually where appropriate
31. Reorganizing Records
Decision:
• Separate out records with more than 1 subject
• Separate out records with more than 1 copy
• What records remain?
-5,017 records (over half our records) contain just 1
subject and 1 copy
32. Field Deletions
• Null fields
• Irrelevant data
• Outside library scope
• 36 fields deleted
33. Label Info Retention
Copy Management Indexed
Source Special Issue
When Approved Holdings
Order # Renewal Note
Supplier Title No. Expiration
Internal Acct. Number Start Date
P.O. Frequency
Requestor due date Q1, Q2, Q3, Q4 Cost
Order Date LC Card
Order Received Record Created
Order Received Status
Sent to Requestor Due Date
Order Type Recatalogue Date
Supplier List Price
Invoice # Actual Cost
Order Notes
34. Field Additions
• Koha field requirements
– Mandatory Fields (i.e. 003, 040, 942)
– Authorized Values (item and location)
– Item Records (952 field)
• Parsing data from one field across multiple fields
in Excel
Author: Mayhew, Ann | Pollard, Michael, ill.
36. Advanced Sort Function
Data analysis:
• List of unique entries
• Find inconsistencies
Answers the Question: “What have we got in our data?”
i.e. the Location field in spreadsheet
39. 1. Main Shelving Area
2. Back room (please ask at desk for assistance)
3. Reference shelving area
4. Special collections area (west wall)
5. Behind circulation desk (ask for assistance)
6. Children’s Corner
7. Multimedia area (near printer)
8. Green Roofs (by magazines)
Now what’s left?
46. Text to Column Function
• Some fields include multiple items that could be re-
allocated to more specific MARC fields
• Parse data over multiple columns in Excel
• i.e. Authors:
Grossman, Mary Louise|Grossman, Shelly, jnt. aut.|Hamlet, John N., jnt. aut.
• We can use Text to Columns function in Excel and
choose the vertical bar as our delimiter
51. “There‟s more than one way to peel a banana:
we can build a machine to do it for us,
or use „BFI‟– Brute Force & Ignorance”
~Lee Benson
52. Parsing Fields Manually
• Some fields had numerous unique entries
• Must parse manually
• Out of the 9704 records there were:
1647 unique entries in the Distributor field
3871 unique entries in the Subjects field
5009 unique entries in the Notes field
• Notes Field:
Values parsed into various 5XX fields
Some parsed into various non-5XX fields
• i.e. edition (250), varying title (246), ISBN (020)
53. Limited edition, no. 333 of 496 | Signed by author
Reprint of the 1706 ed., London|Limited ed., no. 309|Or, the gard'ner's almanac, directing what he
is to do monthly throughout the year; and what fruits and flowers are in prime
Subtitle: A world directory
Text in German. Von der Schonheit der Pflanzen und Garten.
Includes index. |Elementary and junior high school.
Includes index.|Describes birds found commonly in North America.|Ages 7-10
Reprinted from the Paris ed., 1635|Text in Latin.
Introduction by Charlotte Gray|Signed by the author|Includes bibliographic references and index.
Copy 1 autographed by the author. Ideas and inspirations for your garden
Includes index.|Foreword by Beth Chatto.|Preface by Fergus Garrett and Rosemary
Alexander.|With contributions by Anna Pavord, Alan Titchmarsh, Dan Hinkley, and Helen
Dillon.|Book Contents: Arrivals - Gardens & gardening - The plants - The house - Family - Visits -
Christopher - Travels.
Foreword by C. Colston Burrell|Autographed by the author Plants and ideas for natural and
created water gardens
54.
55.
56. Excel - Technical Difficulties
1. Cell Formatting
• Set to Text : “Text format cells treated as text even when a number is
in the cell. The cell is displayed exactly as entered”
• Fields with numerical or alphanumeric values had to be formatted to
the “Text” Setting i.e. ISBN: 9.78E+12
2. Truncation of data
• Abstract Field: lengthy descriptions cut
• Data Recovery using previous spreadsheet iteration
3. DTF (date time format) - yyyy-mm-dd
• International Standard Date and Time Notation
59. Spreadsheet “After”
• 5017 records
• Approximately 150+ volunteer hours
• Started with 67 fields
– 36 Field Deletions
– 47 Field Additions
• Finished with 78 total fields
Now we‟re ready to begin mapping fields!
60.
61. 1. Delimited Text Translator
Mapping fields
Joining subfields to construct MARC fields
2. Marc Editor
View / edit MARC records
3. Marc Maker
Create the file format needed for upload to Koha
3 Functions Used in Marc Edit
64. Joining fields in MarcEdit was necessary to:
• Create strings of Marc subfields
• Create item records
BUT…..
When an Excel field is being mapped to a joined
field in MarcEdit, it cannot be empty (null)
• When encountering a null cell in the Excel
spreadsheet, a joined MarcEdit field pulls a value
from the next available cell in Excel
• “work-around” : used underscores in null cells
• Removed underscores after mapping process
Joining Fields
65. • You can use .csv or any Excel file formats
• After the mapping process you end up with:
.mrd this is your mapping template generated by the Delimited
Text Translator function
.mrk Contains your MARC records, the .mrk is created using the
Delimited Text Translator, but can be opened, viewed and edited
using the MarcEditor Function
.mrc this file format is created using the MarcMaker function
and is the file format needed for importing records into Koha
Keeping File Formats Straight!
66.
67.
68.
69.
70.
71.
72.
73.
74.
75.
76.
77.
78.
79.
80.
81.
82.
83.
84.
85. Importing Records
• Uploaded 5017 records into the catalogue
• We searched the records that had just
been added to the catalogue
BUT …..
86.
87.
88. Detour:
Where are the records?
• None of the 5017 newly created MARC
records added to Koha were searchable...
• We had a mild heart attack
• Reached out to the Koha community for help
• Zebra indexing was required to index the
newly added records, so that they could be
retrieved in catalogue searches
• So we scheduled a command in the Crontab
to rebuild the index every week.
89.
90.
91. Destination:
What we accomplished
• Through a combination of resources, we created
usable MARC records from Inmagic’s output of
bibliographic fields.
• The library gained an upgraded web-based ILS
that offers:
– Increased interoperability
– Seamless branding between catalogue and website
– Publisher, book covers, and other links within
catalogue records
– Z39.50 capability
– Improved ease of use for staff and users
92. Destination:
Next steps
• Complete OPAC customization
• Import patrons
• Finalize training documentation for library
volunteers
• Complete the migration/addition of
remaining catalogue records
93. For the Future
• Project is ongoing – Cataloguing “Flash
Mob” event, May 4th and 5th, 2013.
