Presented at the Bootcamp session of PASIG 2013 conference, Washington, May 21-24, 2013

1. Design Principles for
Digital Preservation Systems
Stephen Abrams
University of California Curation Center
California Digital Library
www.cdlib.org/uc3
SystemsSystems?

2. Design Principles for
Digital Preservation Systems
Stephen Abrams
University of California Curation Center
California Digital Library
www.cdlib.org/uc3
Programs

3. Washington
May 22-24, 2013
What’s the problem we’re trying to solve?
 Connecting people with digital content in meaningful ways
across barriers of space and time
 Preservation is the end, systems are just the means
Robbi Verte, Gesture of hand holding a flash disk, www.123rf.com/
Pietro Izzo, Open hand, www.flickr.com/photos/pietroizzo/482812880

4. Washington
May 22-24, 2013
Think outside the (system) box
 Worry about designing your overall preservation program
before considering the systems that will implement parts of it
Luxmart, Working together team puzzle concept,
http://commons.wikimedia.org/wiki/File:Working_Together_Teamwork_Puzzle_Concept.jpg
« Integration »

5. Washington
May 22-24, 2013
Think outside the (system) box
 Worry about designing your overall preservation program
before considering the systems that will implement parts of it
 A preservation program should provide effective control over
managed content in several key areas
 Technical control
 Managing bits, descriptions of bits, relationships between bits, etc.
 Analysis, planning, monitoring, intervention, etc.
 Intellectual (or curatorial) control
 Creation, selection/acquisition, arrangement, cataloging, etc.
Cf. Kenney & McGovern (2003), “Five organizational stages of digital preservation,” Digital Libraries: A Vision
for the 21st Century (Ann Arbor: MPublishing,), hdl:2027/spo.bbv9812.0001.001
« Integration »

6. Washington
May 22-24, 2013
Good design is good design
 Any principles for preservation system design should be
informed by general principles for system design and
engineering design
 Beginning with a clearly defined need
 Leading to a creative design in response to the need
 Resulting in a system fully meeting the need
Cf. Royal Academy of Engineering (1999), Principles of Engineering Design
www.raeng.org.uk/education/vps/principles/pdf/armstrong_keynote.pdf
Jason DeRusha, Crying baby shot, www.flickr.com/photos/derusha/1465953800
Izumi Mitatami, The hamburger, www.flickr.com/photos/marvin_izumi/3881467402
Sean Dreilinger, Big hamburger, little kid, www.flickr.com/photos/seandreilinger/3002176844

7. Washington
May 22-24, 2013
Design principles for preservation systems
 Integration  Least surprise
 Definition  Elegance
 Generality  Community
 Parsimony
 Modularity
 Granularity
 Orthogonality
 Emergence
 Redundancy
 Evolution
 Transience
Yes, some of these may sound, or even
be, somewhat inconsistent or
contradictory

8. Washington
May 22-24, 2013
Know what you (really) need
 What you need, not just what you want
 Tie requirements to specific use cases
 Actual, expected, anticipated, or probable
 Hypothetical?
 Acceptance criteria
 Did you get what you asked for?
« Definition »
Guardian, Five reasons why waiters don’t write down your order,
www.guardian.co.uk/lifeandstyle/2011/aug/14/waiters-dont-write-orders-down

9. Washington
May 22-24, 2013
Know what you don’t need
 Necessity vs. sufficiency vs. superfluity
 “A scientific theory should be as simple as possible, but no
simpler” – Einstein
 “It is futile to do with more things that which can be done with
fewer” – William of Ockham
 “Not too big, not too small, just right” – Goldilocks
« Parsimony »
Wikimedia Commons, Goldilocks 1912, commons.wikimedia.org/wiki/File:Goldilocks_1912.jpg

10. Washington
May 22-24, 2013
Inclusive applicability
 Solve a general problem today, to avoid having to solve a
specific problem tomorrow
 Support for re-configuration or self-configuration
 Facilitate (re)use, potentially in novel ways
Cf. Yourdon and Constantine (1979), Structure Design: Fundamentals of a Discipline of Computer
Programming (Prentice-Hall), www.win.tue.nl/ ~wstomv/ quotes/structured-design.html#19
« Generality »
Brian Snelson, Lovely new metric spanners and torque wrench,
www.flickr.com/photos/exfordy/353771860
LoggerHead Tools, Bionic wrench, loggerheadtools.com

11. Washington
May 22-24, 2013
The Unix philosophy
 Make each program do one thing well
 To do a new job, build afresh rather than complicate old
programs by adding new features
 Expect the output of every program to become the input to
another, as yet unknown, program
 Don't hesitate to throw away the clumsy parts and rebuild
them
Cf. McIllroy et al. (1978), “Unix time-sharing system: Forward,” Bell Systems Technical Journal 57(6): 1899–
1904, www3.alcatel-lucent.com/bstj/vol57-1978/articles/bstj57-6-1899.pdf
« Modularity / Granularity / Orthogonality / Emergence »
Windell Oskay, Inside-out Lego brick, www.flickr.com/photos/oskay/265899811

12. Washington
May 22-24, 2013
Just in case, just in case
 Plan for failure
 Replication to avoid single points of failure
 Decorrelation to avoid cascade failure
Cf. Rosenthal (2010), “LOCKSS: lots of copies keeps stuff safe,” US Workshop on Roadmap for Digital
Preservation Interoperability Framework, NIST, Gaithersburg, MD
lockss.org/locksswiki/files/NIST2010.pdf
« Redundancy »
Energy Press, PWC: Το δάνειο στο ΛΑΓΗΕ δεν αρκεί για να μην καταρρεύσει η αγορά
www.energypress.gr/news/lianikh-reymatos/PWC:-To-daneio-sto-LAGHE-den-arkei-gia-na-mhn-katarreysei-h-agora
NCinDC, Life is one big balancing act, www.flickr.com/photos/ncindc/3229050640

13. Washington
May 22-24, 2013
First make it work, then make it work better
 Configuration
 Customization
 Iterative enhancement
Cf. May and Zimmer (1996), “Evolutional development model for software,” HP Journal (August): 39-45,
www.hpl.hp.com/hpjournal/96aug/aug96a4.pdf
« Evolution »
Wikimedia Commons, commons.wikimedia.org/wiki/File:Human_evolution_scheme.svg

14. Washington
May 22-24, 2013
Easy come, easy go
 Preservation systems are inherently ephemeral and
expendable; the content managed in them is not
 Avoid system lock-in
 Standardized content representation
 Standardized APIs
 Smooth migration paths
 Your (aging) system’s DIP should be a replacement system’s SIP
 Preferably, change at a time and place of your choosing
Cf. Janée (2009), “Relay-supporting archives: Requirements and progress,” International Journal of Digital
Information 4(1), www.ijdc.net/index.php/ijdc/article/view/102
« Transience »

15. Washington
May 22-24, 2013
Keep the customer satisfied
 Default system behaviors should conform to implicit user
expectations
 Know the communities you are seeking to serve
 Consistency
 Treat like things alike
Cf. Raymond (2003), “Applying the rule of least surprise,” Art of Unix Programming (Addison-Wesley),
http://www.faqs.org/docs/artu/ch11s01.html
« Least surprise »
Narufag, Naruto in the scream, narufag.deviantart.com/art/naruto-in-the-scream-267479366

16. Washington
May 22-24, 2013
Commodity, firmness, and delight
 “Well building hath three conditions: firmness, commodity,
and delight” – Vitruvius, De architectura [trans. Wotten, 1694]
 The analogous conditions for computer, rather than structural,
architecture are…
 Utility
 Resilience
 Elegance
Cf. Madni (2012), “Elegant systems design: Creative fusion of simplicity and power,” Systems Engineering
15(3): 347-54, doi:10.1002/sys.21209
« Elegance »
Dominic 2007, Pantheon Dome, www.flickr.com/photos/9556741@N03/3157684854

17. Washington
May 22-24, 2013
Beg, borrow, or steal
 Learn from the solutions and experience of the community
 Support the community and contribute back
Cf. Anderson (2011), “National Digital Stewardship Alliance: Community, content, commitment,” CENDI
Principals and Alternatives, Washington, DC
www.cendi.gov/presentations/03_06_11_Anderson_Martha_NDSA.pdf
« Community »
Enrique Martinez Bermejo, Community-manager,
www.flickr.com/photos/kikemb/5428414543
Duncan Hall, Attention aux PickPockets,
www.flickr.com/photos/dullhunk/4575707721

18. Washington
May 22-24, 2013
Principles in action: micro-services
 Decomposition of infrastructure function into a granular set of
independent, but highly interoperable services
Cf. Abrams, Cruse, Kunze, and Minor (2011), “Curation micro-services: A pipeline metaphor for
repositories,” Journal of Digital Information 12(2), journals.tdl.org/jodi/article/view/1605
“Archivematica implements a micro-service
approach to digital preservation”
http://www.archivematica.org/
“Curation Architecture Prototype Services (CAPS), is
built on the micro-services approach to digital
curation” http://www.libraries.psu.edu/
“The micro-services approach … seemed similar to
the SDR 2.0 principle of making services more
modular” http://library.stanford.edu/
“*SDB] provides a viable solution to the challenges
of long term digital preservation by delivering a
flexible, extensible set of micro-services”
http://www.tessella.com/
“The University of North Texas (UNT) has
implemented a robust architecture for digital library
initiatives utilizing the Curation Micro Services
methodology for building repository infrastructure”
http://www.library.unt.edu/
Mode Focus Value Service Valence Visibility
Curation
Value
Accretion Annotation
UI/Accesscontrol/Messagequeuing
Interoperation
User-facing
Visibility Notification
Utility
Accessibility Access
Application
Derivation Transformation
Selectivity Search
Actionability Index
Stewardship Ingest
Preservation
Context
Epistemology Characterization
Interpretation
Provider-facing
Ontology Inventory
State
Reliability Replication
Protection
Fixity Fixity
Stability Storage
Identity Identity

19. Principles in action: Merritt repository
Storage node
Storage
broker
Inventory
Storage node
Storage node
UI/API
UI/API
UI/API
LDAP
LDAP
LDAP
RDBMS
Fixity
User
agent
Message
queue
RDBMS
Load
balancer
Ingest
Load
balancer
Ingest
Ingest
EZID
No-SQL
DataCite
DataONE
member node
RDBMS
RDBMS
DataONE
coord’ing node
DAS
SAN
Cloud
IDF
http://www.cdlib.org/uc3/merritt
http://merritt.cdlib.org/
RDBMS
RDBMS

20. Washington
May 22-24, 2013
Design principles for preservation systems
 Integration  Least surprise
 Definition  Elegance
 Generality  Community
 Parsimony
 Modularity
 Granularity
 Orthogonality
 Emergence
 Redundancy
 Evolution
 Transience
Note that these are principles, not rules
Their applicability will depend on local
needs, conditions, expertise, resources,
etc.
—
Rely on your intuition and experience
www.slideshare.net/UC3/pasig-2013abramsdesignprinciplesforpreservationsystems

21. Washington
May 22-24, 2013
UC Curation Center
 www.cdlib.org/uc3
uc3@ucop.edu
Stephen Abrams David Loy
Patricia Cruse Mark Reyes
Shirin Faenza Joan Starr
Scott Fisher Carly Strasser
Erik Hetzner Marisa Strong
Joshua Hubbard Adrian Turner
Greg Janée Bhavitavya Vedula
John Kunze Kenneth Weiss
Rosalie Lack Perry Willet
www.slideshare.net/UC3/pasig-2013abramsdesignprinciplesforpreservationsystems