As service providers and primary code contributors in the Islandora Community, discoverygarden encounters customers who are ingesting, accessing, and storing high volumes of data. For example, a customer who had 150,000 objects in 2012 now has three million objects and expectations to grow to five million in the very short term. This is increasingly common. As repositories grow in size they can encounter poor performance, particularly during large ingests and derivative generation. To accommodate growing repositories caching mechanisms, infrastructure changes, and code updates are necessary. The presentation will explore customer case studies that demonstrate interim solutions and the extensive, ongoing research and development to find long-term solutions.

2. ❏
❏
❏
❏
❏

7. ● Inactive project
● Performance suffers with
triplestore managing 50
million or more triples
● No ability to cluster or
balance load
● Incomplete implementation
of Sparql

11. ● Isolate derivative generation
to 20 server cluster
● Asynchronous and parallel
derivative creation with
Islandora Jobs (Gearman)
● Using Parallels instead of
Drush scripts

12. Asynchronous and Parallel Derivative Creation

14. ● Benchmark Testing
● Block Obscure Crawlers
● Manage Caching
● Ensure Sufficient Server
Resources

19. Room to improve scaling and
performance through exploration:
● Media Streaming
● Load Balancing
● More Aggressive Caching
Configuration
● Clustering Solr

21. Developing a scalable repository
back end to satisfy growing
performance requirements
Will accommodate:
● More concurrent users
● Many millions of objects
● Faster response times
May be used as a Fedora
replacement
Continued work on Fedora 4
integration