1. Determining Requirements for
Managing Unstructured Data
Christine Connors
TriviumRLG LLC
Information Management Consulting
March 22, 2012
Thursday, March 22, 12

2. Overview
✤ Triggers
✤ Techniques
✤ Input
✤ Output
✤ Scale
✤ Mapping requirements to capabilities
Thursday, March 22, 12

3. Triggers
✤ “Didn’t we already do that?”
✤ “I found it once. It’s in there somewhere.”
✤ “Who knows how to do this?”
✤ “We maintain how many document management systems?!?”
✤ “Why can’t we use this content to do ... ?”
✤ “Which customer wanted that feature?”
Thursday, March 22, 12

4. As true today...
✤ “The search engine is poor to inadequate. I needed to find an appropriations data
sheet and was returned 366 entries, none which had anything to do with
appropriations. I spend far too much time looking through the search results for this
engine to be effective. If I could find this document on the INTERNET I would do
so, but this is an internal document that is successfully hidden somewhere in the
archives with the Ark of the Covenant.”
Unidentified search and browse survey participant, June, 2003
✤ “Who gets more hits: www.amazon.com or
www.thequaintbookstoredownthestreet.com? Listen up people: Our intranet is a
wasteland of information. We need to unify - we need to standardize. Information is
power - but only if it is on my desktop, not hidden away in some server waiting for
a lucky adventurer to uncover it like some lost continent.”
Another unidentified search and browse survey participant, June, 2003
Thursday, March 22, 12

5. Wonderful
objects with no
metadata (context)
A secret garden
“Secret Garden” by wonderlane | Flickr |
CC Attribution 2.0 Generic
Thursday, March 22, 12

6. Objects with
can’t-be-bothered
metadata
A maze
“Longleat Maze” by odolphie | Flickr |
CC Attribution 2.0 Generic
Thursday, March 22, 12

7. Lots of
unmarked
repositories
Silos
“Silo” by Plano Light | Flickr | CC
Attribution 2.0 Generic
Thursday, March 22, 12

8. Techniques
Thursday, March 22, 12

9. Sometimes, it’s obvious
✤ Environmental scan
✤ Do we really need 40 document management systems?
➡ We need to reduce the number of systems
➡ Improve the finability of the objects contained
✤ Budget analysis
✤ Projections indicate un-supportable costs of maintaining servers
✤ Costs are going down, but not as fast as our rate of acquisition
➡ We need to archive or compress the data, intelligently
Thursday, March 22, 12

10. Here there be dragons...
Thursday, March 22, 12

11. Standard
Techniques
✤ Surveys
✤ Focus groups
✤ Observation
✤ SWOT
✤ Capabilities analysis
Thursday, March 22, 12

12. User Personas
✤ Craft fictional characters
based on your key user
groups
✤ These archetypes will
represent the users of your
new system or process
✤ Give them attributes and
stories
✤ Figure out what you need to
solve their problems
Thursday, March 22, 12

13. Knowledge
Audits
✤ Identify what types of
information are critical for the
organization
✤ Dashboards?
✤ Note gaps
✤ Note overlaps - redundancies,
duplication and collaborate
Thursday, March 22, 12

14. Business
Process Map
✤ Document the steps in
standard business
processes
✤ Identify where
unstructured data is used
and created
✤ Identify critical inputs/
outputs
✤ Identify breaks and blocks
in the system Photo by ottonassar | CC Attribution-Share Alike
http://www.fotopedia.com/items/flickr-3417427945
Thursday, March 22, 12

15. Social Tagging
Analysis
✤ Analyze the metadata and
folksonomy - the organic
hierarchies and social tags
that have been created ad
hoc in the systems
✤ Are there synonymous or
near-synonymous terms?
✤ Are there trends by date or
location?
Thursday, March 22, 12

16. Survey Stakeholders
✤ What problem(s) are you solving?
✤ What are the pain points in the digital asset management strategy?
Discovery, re-use, IP management?
✤ What are the benefits?
✤ New products, increased customer and/or employee satisfaction?
✤ Are there restrictions on how it gets done?
Thursday, March 22, 12

17. Typical Project Structure
✤ Analysis of needs & wants
✤ Define requirements
✤ Commit
✤ Resourcing
✤ Develop and Deploy
✤ Define & Publish Maintenance Processes and Governance Rules
Thursday, March 22, 12

18. Improve Efficiencies
Reduce Costs
Thursday, March 22, 12

19. Input - Lay of the Land
✤ Data discovery in an 80k employee multi-national
✤ 85% of the data “unstructured”
✤ 90% had no metadata
✤ most of that was “bad” metadata
✤ 13% exact duplicate
✤ True age of object hard to determine due to web scripting, server
migrations, shared access
Thursday, March 22, 12

20. Input
✤ Qualify searches by
✤ function, organization, and business
✤ date
✤ document type (especially web pages)
✤ category (tags)
✤ Provide sorting of results by date, document type
✤ Do not change URLs of pages (users bookmarked)
Thursday, March 22, 12

21. Improved Efficiencies
✤ Delphi Group:
✤ Business professionals spend more than 2 hours per day searching for
information
✤ Half of that time – 1 hour per day is wasted by failure to find what
they seek
✤ The single factor most attributed to the large amount of time wasted
was
✤ data changes (location 35%) and
✤ bad tools (ineffective search and lack of labeling 28%)
Thursday, March 22, 12

22. Output
✤ Objects must have metadata
✤ Title, Author, Subject
✤ Repositories should be created for organization/business/function
✤ Objects must be stored in one location to reduce duplicates
✤ Objects need to be shared to many locations
✤ Search & browse UI tools must provide filters for the index created
✤ File naming conventions need to be created and enforced
Thursday, March 22, 12

23. Improved Efficiencies
Dollars Returned to the Business for Growth (1 hour per year per general employee plus 1 hour per month)
$4,000,000
$3,000,000
$2,000,000
$1,000,000
$0
1.2k 2k 10.4k 12.3k 4k 11.6k 11.9k 8.3k 13.9k
Thursday, March 22, 12

24. Reduce Storage Costs
Data growth assuming 60% annual growth rate
$90 3000
T1 Only
General tiered move
Unintelligent Move
Policy based Move
$68 2250
2228
Millions (Annual Cost)
$45 1500
$44.3
1393
$27.7
870
$23 750
544 $17.3
$12.2 $10.8
340
$0 0
Year 1 Year 2 Year 3 Year 4 Year 5
Relative of starting point, growth curves represent storage acquisition cost increases over time.
Thursday, March 22, 12

25. Identify Opportunities
Thursday, March 22, 12

26. Input
✤ Curate the content for me
✤ Allow me to reuse content easily
✤ a part, not the whole
✤ in a new package
✤ without copying/pasting
✤ with citations
✤ Allow me to annotate content
✤ Allow me to refine content based on my needs
Thursday, March 22, 12

27. Content Re-use and Re-purposing
✤ Skills: people do not learn at the same pace nor neatly align to ‘grade’
levels
✤ Product catalog: name and image as a tile on a sale page as well as in a
detailed product description
✤ A taxonomy focused on a subject from introductory to mastery levels of
understanding can be used to tag content fragments
✤ Combined with a taxonomy of skill levels, the content can be aggregated
into packages consistently addressing the right audience in the right order
✤ These fragments can be re-used in a variety of products: multiple skill
levels, multiple assessments, multiple delivery channels
Thursday, March 22, 12

28. Output
✤ CRM content must be indexed and categorized
✤ Objects must have metadata
✤ Title, Author, Subject, Skill Level, Process Step
✤ Objects need to be shared to many locations
✤ Objects must be usable in multiple systems and platforms
✤ File naming conventions need to be created and enforced
✤ Source data/citations must be available
✤ Objects must be written in a re-usable, neutral voice
Thursday, March 22, 12

29. Define Requirements
✤ Functional Requirements
✤ User Requirements
✤ Administrative Requirements
✤ Authentication/Authorization/Security
✤ Metrics
✤ Documentation requirements
✤ Technical Requirements
✤ Back End
✤ Front End
✤ Platform
✤ Interoperability
Thursday, March 22, 12

30. Authentication, Authorization and
Security
✤ Consider the content collections that will be part of the program.
✤ Do you anticipate any of it having restrictions?
✤ If so, then what are those restrictions?
✤ How will authorized users authenticate and gain access?
✤ Will you restrict access by entity type?
✤ By rules-based classification?
✤ By system access and control policies?
Thursday, March 22, 12

31. Back End
✤ How will you architect the back end to scale effectively?
✤ Will it be easily repeated on additional clusters?
✤ What OS and software will it need to run?
✤ Will it fail over?
✤ Can it scale to handle the number of users, documents and entities
predicted for the anticipated life of the hardware?
Thursday, March 22, 12

32. Front End
✤ How will users interact with the system?
✤ Create - Read - Update - Delete as permissioned
✤ Search, browse, publish, integrate, migrate and import to and from other
systems.
✤ What tools are needed to support these actions?
✤ Should select users be able to perform administrative tasks via a client or browser
interface?
✤ How about the ability to generate reports?
✤ What operating system(s) does this interface need to function on?
✤ Mobile? Offline?
Thursday, March 22, 12

33. Interoperability
✤ How are you going to package and publish the data?
✤ File servers?
✤ Cloud?
✤ XML? Office suites? Analytics packages? Other tools?
✤ What other applications need to use the data created by one of the
above?
✤ DMS/DAM/CMS/CRM
Thursday, March 22, 12

34. Metadata Management
✤ What kinds of information is important to manage - what metadata
elements?
✤ Title, Author, Subject, Process, Skill, Dates, Business, Function...
✤ Will you need a taxonomy?
✤ Enforce some control on the description of attributes
✤ Do you need an external tool or is there a module within your CMS,
DMS or portal solution that will suffice?
Thursday, March 22, 12

35. Resourcing
✤ Build vs. buy
✤ Human resources - staff or contractors needed
✤ Technology needs
✤ Hardware? Software? Network? Costs?
Thursday, March 22, 12

36. Define & Publish Processes and
Rules
✤ Maintenance processes
✤ Schedule for review and updates
✤ Rules for additions, changes, deletions
✤ Implementation and publishing process
✤ Governance rules
✤ Editor? Committee? User input?
✤ Standards compliance?
Thursday, March 22, 12

37. Scale
Thursday, March 22, 12

38. Scale
✤ According to the 2011 Digital Universe study by IDC/Sponsored by
EMC, by 2020 the world will generate 50x the amount of information
we have now, on 75x the number of containers, and increase IT
support for those systems only by a factor of 1.5.
Thursday, March 22, 12

39. Scale Using Tools
✤ Compression technologies
✤ Metadata management
✤ Indexing, NLP, Search
✤ Business rule generation and application
✤ Virtualization
Thursday, March 22, 12

40. Scale Using Processes
✤ Standards
✤ Metadata governance
✤ Schema
✤ Taxonomy
✤ Subject Matter Experts
✤ Editorial Boards
✤ Product development
Thursday, March 22, 12

41. Mapping requirements to
technologies
Thursday, March 22, 12

42. What’s available?
✤ Latest technologies
✤ Information management frameworks
✤ Business process best practices
Thursday, March 22, 12

43. Questions?
Thursday, March 22, 12

44. Thank you for attending!
Christine Connors
TriviumRLG.com
christine@triviumrlg.com
http://www.slideshare.net/triviumrlg
Thursday, March 22, 12