SlideShare ist ein Scribd-Unternehmen logo
1 von 83
Downloaden Sie, um offline zu lesen
BUILDING AN ACADEMIC
PROGRAM DATABASE AND API
Dave Olsen / @dmolsen
West Virginia University
OUR STORY BEGINS IN 2015
GUIDING PRINCIPLE
ACADEMICS
COST
= VALUE
IMPORTANCE OF ACADEMICS
of our admits ranked academic
reputation as “very important” when
selecting a school.
72%
of enrolling students rated us as “very
good” or “excellent” for academic
reputation.
75%
of non-enrolling students described
WVU as having a good academic
reputation.
24%
vs.
GUIDING PRINCIPLE
ACADEMICS
COST
= VALUE
PROGRAM PAGE EXAMPLE
ORIGINAL PROGRAM DB
SUMMER OF 2016
LESSONS LEARNED:
3RD PARTY CONTENT TOOL
The bad:
• Could not build DRY content. No content
relationships.
• API very limited. No search or preview.
The good:
• Content modeling was the right direction. 👌
• Having a 3rd party host and maintain an
editing interface was awesome. 👍
LESSONS LEARNED:
UNIVERSITY CONTENT
The bad:
• No one could agree on how many programs
we had or their names. #
• No data to fill in blanks we had (e.g. outcomes)
• Program search not robust enough.
The good:
• So much content related back to programs.
We had an opportunity.
SPOILER ALERT
• Still no agreement on how many programs
we have or their names. # # #
• Still no true outcome data.
OUR PLAN OF ACTION
• Gather a team
• Develop goals and requirements
• Evaluate 3rd party and technical options
• Source content from domain experts
• Build technical solution and design new
program pages
• Maintain!
Team: The Duo
(actual team headshot)
The writer and organizer
Sarah Gould
Developer and ne’er-do-well
Dave Olsen
Team: The “Creatives”
(actual team headshot)
Front end dev #1
Adam Johnson Front end dev #3
Dustin Mazon
Front end dev #2 Adam Glenn
Team: Supporting Cast
(actual team headshot)
Team: The Champion
(actual champion headshot)
Project Champion
PRO TIP: SOURCING ACADEMIC
CONTENT REQUIRES ACADEMICS
It was nearly impossible to source content
with communicators and recruiters alone.
Get someone from Provost Office or
similar on the team.
OUR GOALS
• Standardize content.
• Highlight our value proposition.
• Address full life-cycle of a student.
• Optimize for search engines.
• Deploy accurate content quickly.
• Expand to all three regional campuses.
OPTIMAL FINAL OUTCOME
Course catalog
SSC data
O*NET career data
Program pages Major Maps
Other campus outlets via API
Share what we’ve
collected
Consume content
from others
Make connections
Curriculum matrix
Admissions Registrar
CLASS
Programs
Database
WHAT WASN’T A GOAL?
We didn’t want this to be a primary source
of truth for academic content for internal
stakeholders.
That is asking for trouble.
OUR REQUIREMENTS
• Build glue, not CRUD.
• 3rd party
• Robust API system
• Deep content relationships (COPE).
• Ability to preview changes.
• Improved program search.
OUR PLAN OF ACTION
• Gather a team
• Develop goals and requirements
• Evaluate 3rd party and technical options
• Source content from domain experts
• Build technical solution and design new
program pages
• Maintain!
ENTER CONTENTFUL
CONTENTFUL: THE GOOD
• APIs for images and content preview, delivery and
management plus webhook support.
• Click-and-edit content modeling.
• Rich, pre-built field types.
• Can also create UI Extensions to extend their interface.
• Media management with generous storage.
• Multi-language using “locales.”
• Free to try. 😎
CONTENTFUL:
BUILDING A CONTENT MODEL
CONTENTFUL: THE BAD
• Need to use their SDKs to access API and data.
• No easy way to limit how much data was
returned when using API. Objects > 7MB.
• 50 max fields per content type.
• Rate limiting when posting and getting data.
• No XML option.
• API calls were limited. 💵
COMING SOON: GRAPHQL
MEASURING UP
Build glue, not CRUD.
3rd party
Robust API system
Deep content relationships (COPE).
Ability to preview changes.
Improved program search.
Hmm, Contentful…
•Is great for content organization! 😁
•Has a high bar for content re-use by myself and
other campus developers. 😞
ENTER AWS LAMBDA
ENTER AWS DYNAMODB
ENTER AWS CLOUDSEARCH
ENTER AWS API GATEWAY
ENTER AWS SNS
AWS SERVICES: THE GOOD
• It’s all glue:
• No hardware to manage. “Serverless.” 🎉
• Events/SNS - easily stitch products together.
• Testable
• Define my own API and become dumb data endpoint.
• Click-and-edit configurations.
• Could implement Contentful workarounds:
• “Reverse GraphQL” and “post-process” rules to optimize content
for outlets. Allows for inheritance, key building and richer objects.
• No rate limiting and layer of resiliency.
• XML and JSON
AWS SERVICES: THE GOOD
“Reverse GraphQL” “Post-process”
AWS SERVICES: THE GOOD
“Reverse GraphQL” “Post-process”
AWS SERVICES: THE BAD
• Actual content is further away from final outlet.
Sometimes need to force pull data to update.
• Need to update “reverse GraphQL”
configurations when content models change.
• DynamoDB doesn’t like large objects when
scanning. Have had to develop *Thin tables.
• Supported programming languages are limited.
PRO TIP: EMBRACE NODE.JS
PRO TIP: BETA/STABLE
• Set-up BETA alias in Lambda to point at $LATEST
version.
• Use versions in Lambda to publish stable code.
• Set-up STABLE alias to point at latest stable
version.
• Add stage variables to the targeted Lambda
function in API Gateway.
• Extend to DynamoDB for beta and stable datasets.
OUR PLAN OF ACTION
• Gather a team
• Develop goals and requirements
• Evaluate 3rd party and technical options
• Source content from domain experts
• Build technical solution and design new
program pages
• Maintain!
• Read every major’s entry in the course catalog.
Twice. Take notes.
• Design or content first? 🐔 or 🥚?
• Use meetings and workshops over worksheets.
A traveling circus. 🎪
• Identify owners of certain types of content (e.g.
tuition and requirements). Always defer to them
and their processes.
LESSONS LEARNED:
SOURCING CONTENT
• Automate the import of content whenever possible.
• Find proxies for missing content. Our career outcomes
come from O*NET.
• Find experts to review the proxies. Our Career
Services Office chose the careers for each major.
• Prep for naming complaints and groups wanting to
“opt-out.” This won’t happen until you’re done and
reality hits.
LESSONS LEARNED:
SOURCING CONTENT
OUR PLAN OF ACTION
• Gather a team
• Develop goals and requirements
• Evaluate 3rd party and technical options
• Source content from domain experts
• Build technical solution and design new
program pages
• Maintain!
CONTENTFUL CONTENT MODELS
An early attempt to document content models.
• Admission Requirement Rules
• Announcements
• Areas of Emphasis
• Building Types
• Buildings
• Campuses
• Capstone Projects
• Career Abilities
• Career Interests
• Career Knowledge
• Careers
• College and Schools
• Companies
• Cost of Living Rules
CONTENTFUL CONTENT MODELS
68Content Models
AWS ARCHITECTURE
Poor diagram of the
system architecture.
AWS ARCHITECTURE: INPUT
Store Data
Lambda, Events,
DynamoDB and
CloudSearch
Post-Process
Lambda
Validate
Lambda and
SNS
Invoke
Contentful
AWS ARCHITECTURE: INPUT
Store Data
Lambda, Events,
DynamoDB and
CloudSearch
Validate
Lambda and
SNS
Invoke
Contentful
Post-Process
Lambda
AWS ARCHITECTURE: INPUT
Post-Process
Lambda
Validate
Lambda and
SNS
Invoke
Contentful
Store Data
Lambda, Events,
DynamoDB and
CloudSearch
AWS ARCHITECTURE: INPUT
Validate
Lambda and
SNS
Invoke
Contentful
Post-Process
Lambda
Store Data
Lambda, Events,
DynamoDB and
CloudSearch
AWS ARCHITECTURE: INPUT
Store Data
Lambda, Events,
DynamoDB and
CloudSearch
Validate
Lambda and
SNS
Invoke
Contentful
Post-Process
Lambda
AWS ARCHITECTURE: OUTPUT
Request and
Render
PHP, JavaScript
and Ruby
Route, Transform
and Deliver
API Gateway
Validate and
Process
Lambda
Fetch Data
DynamoDB and
CloudSearch
AWS ARCHITECTURE: OUTPUT
Request and
Render
PHP, JavaScript
and Ruby
Route, Transform
and Deliver
API Gateway
Validate and
Process
Lambda
Fetch Data
DynamoDB and
CloudSearch
AWS ARCHITECTURE: OUTPUT
Request and
Render
PHP, JavaScript
and Ruby
Route, Transform
and Deliver
API Gateway
Validate and
Process
Lambda
Fetch Data
DynamoDB and
CloudSearch
AWS ARCHITECTURE: OUTPUT
Request and
Render
PHP, JavaScript
and Ruby
Route, Transform
and Deliver
API Gateway
Validate and
Process
Lambda
Fetch Data
DynamoDB and
CloudSearch
AWS ARCHITECTURE: OUTPUT
Request and
Render
PHP, JavaScript
and Ruby
Route, Transform
and Deliver
API Gateway
Validate and
Process
Lambda
Fetch Data
DynamoDB and
CloudSearch
AWS ARCHITECTURE: OUTPUT
Request and
Render
PHP, JavaScript
and Ruby
Route, Transform
and Deliver
API Gateway
Validate and
Process
Lambda
Fetch Data
DynamoDB and
CloudSearch
API ENDPOINTS
Scan:
Search:
By key:
Endpoints support query string options like “format,”
“sortby” or “groupby.”
The {contentType} path variable gets matched to a
config by the Lambda function. {contentKey} is used
in the DynamoDB look-up.
/{contentType}?options
/{contentType}/search?options
/{contentType}/{contentKey}?options
FINISHED PRODUCT:
API CONTENT
FINISHED PRODUCT:
PROGRAM PAGES
go.wvu.edu/program-page
FINISHED PRODUCT:
ENGAGEMENT?
Heat map
Scroll tracking
FINISHED PRODUCT:
MAJOR MAPS
go.wvu.edu/major-map
FINISHED PRODUCT:
CAREER PATHWAYS
FINISHED PRODUCT:
SCHOLARSHIP CALCULATOR
FINISHED PRODUCT:
LANDING PAGE FEATURES
OUR PLAN OF ACTION
• Gather a team
• Develop goals and requirements
• Evaluate 3rd party and technical options
• Source content from domain experts
• Build technical solution and design new
program pages
• Maintain!
Maintaining the Database
• Contact primary sources of central data and get
updates.
• Three times a year ask recruiters and
communicators to update content.
• Give them access to an internal website and
Word docs with content.
• Almost zero feedback. 😢
REVIEW OUR PLAN OF ACTION
• Gather a team
• Develop goals and requirements
• Evaluate 3rd party and technical options
• Source content from domain experts
• Build technical solution and design new
program pages
• Maintain!
REVIEW OUR GOALS
Standardize content.
Highlight our value proposition.
Address full life-cycle of a student.
Optimize for search engines.
Deploy accurate content quickly.
Expand to all three regional campuses.
Hero Block
Model & Pattern
Quicklinks Block
Model & Pattern
Profile Block
Model & Pattern
OUR FUTURE:
CONTENT MODELS + LAYOUT MODELS
OUR FUTURE:
CONTENT MODELS + LAYOUT MODELS
Program
Title
Degree Designation
Course Delivery Option
Advisement Sheets
Areas of Emphasis
… more …
Name
Background Image
Description
Program
Interests
… more …
Blocks - Student
🎩
Content Model Layout Model Post-Process
Magic
Design System Pattern/Layout - Student
OUR FUTURE:
CONTENT MODELS + LAYOUT MODELS
power.wvu.edu
wisdom.wvu.edu
courage.wvu.edu
Static Sites
Design system-based

static site generator
“Reverse GraphQL” /
post-process
Contentful DynamoDB
Build Pipeline
OUR FUTURE:
CONTENT MODELS + LAYOUT MODELS
THANK YOU / QUESTIONS?
Dave Olsen / david.olsen@mail.wvu.edu

Weitere ähnliche Inhalte

Ähnlich wie Building an Academic Program Database and API with Contentful and Amazon Web Services

Web tech weblamp_infosession_2012-13
Web tech weblamp_infosession_2012-13Web tech weblamp_infosession_2012-13
Web tech weblamp_infosession_2012-13
Konrad Roeder
 

Ähnlich wie Building an Academic Program Database and API with Contentful and Amazon Web Services (20)

Untangling - fall2017 - week 9
Untangling - fall2017 - week 9Untangling - fall2017 - week 9
Untangling - fall2017 - week 9
 
Web tech weblamp_infosession_2012-13
Web tech weblamp_infosession_2012-13Web tech weblamp_infosession_2012-13
Web tech weblamp_infosession_2012-13
 
Ankur Bajad
Ankur BajadAnkur Bajad
Ankur Bajad
 
summer internship
summer internshipsummer internship
summer internship
 
Scribe online 03 scribe online cdk and api overview
Scribe online 03   scribe online cdk and api overviewScribe online 03   scribe online cdk and api overview
Scribe online 03 scribe online cdk and api overview
 
Emerging technologies in academic libraries
Emerging technologies in academic librariesEmerging technologies in academic libraries
Emerging technologies in academic libraries
 
Lessons learned on the Azure API Stewardship Journey.pptx
Lessons learned on the Azure API Stewardship Journey.pptxLessons learned on the Azure API Stewardship Journey.pptx
Lessons learned on the Azure API Stewardship Journey.pptx
 
Docs Like Code: Strategies and Stories
Docs Like Code: Strategies and StoriesDocs Like Code: Strategies and Stories
Docs Like Code: Strategies and Stories
 
apidays LIVE Paris 2021 - Lessons from the API Stewardship Journey in Azure b...
apidays LIVE Paris 2021 - Lessons from the API Stewardship Journey in Azure b...apidays LIVE Paris 2021 - Lessons from the API Stewardship Journey in Azure b...
apidays LIVE Paris 2021 - Lessons from the API Stewardship Journey in Azure b...
 
Enterprise Serverless Adoption. An Experience Report
Enterprise Serverless Adoption. An Experience ReportEnterprise Serverless Adoption. An Experience Report
Enterprise Serverless Adoption. An Experience Report
 
Web-D club Orientation
Web-D club OrientationWeb-D club Orientation
Web-D club Orientation
 
[DevDay2018] High quality mindset in software development - By: Phat Vu, Scru...
[DevDay2018] High quality mindset in software development - By: Phat Vu, Scru...[DevDay2018] High quality mindset in software development - By: Phat Vu, Scru...
[DevDay2018] High quality mindset in software development - By: Phat Vu, Scru...
 
Pega online training in canada
Pega online training in canadaPega online training in canada
Pega online training in canada
 
Software Quality without Testing
Software Quality without TestingSoftware Quality without Testing
Software Quality without Testing
 
#RADC4L16: An API-First Archives Approach at NPR
#RADC4L16: An API-First Archives Approach at NPR#RADC4L16: An API-First Archives Approach at NPR
#RADC4L16: An API-First Archives Approach at NPR
 
Microservices and the Art of Taming the Dependency Hell Monster
Microservices and the Art of Taming the Dependency Hell MonsterMicroservices and the Art of Taming the Dependency Hell Monster
Microservices and the Art of Taming the Dependency Hell Monster
 
Mark Tortoricci - Talent42 2015
Mark Tortoricci - Talent42 2015Mark Tortoricci - Talent42 2015
Mark Tortoricci - Talent42 2015
 
SRV318_Research at PNNL Powered by AWS
SRV318_Research at PNNL Powered by AWSSRV318_Research at PNNL Powered by AWS
SRV318_Research at PNNL Powered by AWS
 
Research at PNNL: Powered by AWS - SRV318 - re:Invent 2017
Research at PNNL: Powered by AWS - SRV318 - re:Invent 2017Research at PNNL: Powered by AWS - SRV318 - re:Invent 2017
Research at PNNL: Powered by AWS - SRV318 - re:Invent 2017
 
AWS User Group - Survey Results and Building APIs on AWS
AWS User Group - Survey Results and Building APIs on AWSAWS User Group - Survey Results and Building APIs on AWS
AWS User Group - Survey Results and Building APIs on AWS
 

Mehr von Dave Olsen

The Death of Lorem Ipsum & Pixel Perfect Content
The Death of Lorem Ipsum & Pixel Perfect ContentThe Death of Lorem Ipsum & Pixel Perfect Content
The Death of Lorem Ipsum & Pixel Perfect Content
Dave Olsen
 
The Squishy Future of Content - Key Communicators Edition
The Squishy Future of Content - Key Communicators EditionThe Squishy Future of Content - Key Communicators Edition
The Squishy Future of Content - Key Communicators Edition
Dave Olsen
 

Mehr von Dave Olsen (20)

Taking Your HTML Email Communications from "Ew" to "Wow"
Taking Your HTML Email Communications from "Ew" to "Wow"Taking Your HTML Email Communications from "Ew" to "Wow"
Taking Your HTML Email Communications from "Ew" to "Wow"
 
The Google Marketing Workflow Workshop
The Google Marketing Workflow WorkshopThe Google Marketing Workflow Workshop
The Google Marketing Workflow Workshop
 
Reimagining Your Website: What are prospective students looking for and how a...
Reimagining Your Website: What are prospective students looking for and how a...Reimagining Your Website: What are prospective students looking for and how a...
Reimagining Your Website: What are prospective students looking for and how a...
 
Progressive Mobile Strategy Redux: The Future Friendly Enterprise
Progressive Mobile Strategy Redux: The Future Friendly EnterpriseProgressive Mobile Strategy Redux: The Future Friendly Enterprise
Progressive Mobile Strategy Redux: The Future Friendly Enterprise
 
Case Study: Rebuilding an Admissions Web Presence
Case Study: Rebuilding an Admissions Web PresenceCase Study: Rebuilding an Admissions Web Presence
Case Study: Rebuilding an Admissions Web Presence
 
Admissions Brain Dump
Admissions Brain DumpAdmissions Brain Dump
Admissions Brain Dump
 
Implementing Brand Patterns
Implementing Brand PatternsImplementing Brand Patterns
Implementing Brand Patterns
 
The Death of Lorem Ipsum and Pixel-Perfect Content (MinneWebCon version)
The Death of Lorem Ipsum and Pixel-Perfect Content (MinneWebCon version)The Death of Lorem Ipsum and Pixel-Perfect Content (MinneWebCon version)
The Death of Lorem Ipsum and Pixel-Perfect Content (MinneWebCon version)
 
Case Study: Automating Outage Monitoring & Communication
Case Study: Automating Outage Monitoring & CommunicationCase Study: Automating Outage Monitoring & Communication
Case Study: Automating Outage Monitoring & Communication
 
The Death of Lorem Ipsum & Pixel Perfect Content
The Death of Lorem Ipsum & Pixel Perfect ContentThe Death of Lorem Ipsum & Pixel Perfect Content
The Death of Lorem Ipsum & Pixel Perfect Content
 
Optimizing web performance (Fronteers edition)
Optimizing web performance (Fronteers edition)Optimizing web performance (Fronteers edition)
Optimizing web performance (Fronteers edition)
 
The Why and What of Pattern Lab
The Why and What of Pattern LabThe Why and What of Pattern Lab
The Why and What of Pattern Lab
 
The Squishy Future of Content - HEEMAC Edition
The Squishy Future of Content - HEEMAC EditionThe Squishy Future of Content - HEEMAC Edition
The Squishy Future of Content - HEEMAC Edition
 
The What & Why of Pattern Lab
The What & Why of Pattern LabThe What & Why of Pattern Lab
The What & Why of Pattern Lab
 
The Squishy Future of Content - Key Communicators Edition
The Squishy Future of Content - Key Communicators EditionThe Squishy Future of Content - Key Communicators Edition
The Squishy Future of Content - Key Communicators Edition
 
The Squishy Future of Content - Penn State Edition
The Squishy Future of Content - Penn State EditionThe Squishy Future of Content - Penn State Edition
The Squishy Future of Content - Penn State Edition
 
The Squishy Future of Content
The Squishy Future of ContentThe Squishy Future of Content
The Squishy Future of Content
 
The Server Side of Responsive Web Design
The Server Side of Responsive Web DesignThe Server Side of Responsive Web Design
The Server Side of Responsive Web Design
 
Measuring Web Performance - HighEdWeb Edition
Measuring Web Performance - HighEdWeb EditionMeasuring Web Performance - HighEdWeb Edition
Measuring Web Performance - HighEdWeb Edition
 
Web Performance & You - HighEdWeb Arkansas Version
Web Performance & You - HighEdWeb Arkansas VersionWeb Performance & You - HighEdWeb Arkansas Version
Web Performance & You - HighEdWeb Arkansas Version
 

Kürzlich hochgeladen

EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
Earley Information Science
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI Solutions
Enterprise Knowledge
 
CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of ServiceCNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Service
giselly40
 

Kürzlich hochgeladen (20)

Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI Solutions
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
 
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024
 
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men
 
CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of ServiceCNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Service
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day Presentation
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed texts
 
What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?
 
Real Time Object Detection Using Open CV
Real Time Object Detection Using Open CVReal Time Object Detection Using Open CV
Real Time Object Detection Using Open CV
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
 
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfThe Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
 

Building an Academic Program Database and API with Contentful and Amazon Web Services

  • 1. BUILDING AN ACADEMIC PROGRAM DATABASE AND API Dave Olsen / @dmolsen West Virginia University
  • 4. IMPORTANCE OF ACADEMICS of our admits ranked academic reputation as “very important” when selecting a school. 72% of enrolling students rated us as “very good” or “excellent” for academic reputation. 75% of non-enrolling students described WVU as having a good academic reputation. 24% vs.
  • 8.
  • 10. LESSONS LEARNED: 3RD PARTY CONTENT TOOL The bad: • Could not build DRY content. No content relationships. • API very limited. No search or preview. The good: • Content modeling was the right direction. 👌 • Having a 3rd party host and maintain an editing interface was awesome. 👍
  • 11. LESSONS LEARNED: UNIVERSITY CONTENT The bad: • No one could agree on how many programs we had or their names. # • No data to fill in blanks we had (e.g. outcomes) • Program search not robust enough. The good: • So much content related back to programs. We had an opportunity.
  • 12. SPOILER ALERT • Still no agreement on how many programs we have or their names. # # # • Still no true outcome data.
  • 13. OUR PLAN OF ACTION • Gather a team • Develop goals and requirements • Evaluate 3rd party and technical options • Source content from domain experts • Build technical solution and design new program pages • Maintain!
  • 14. Team: The Duo (actual team headshot) The writer and organizer Sarah Gould Developer and ne’er-do-well Dave Olsen
  • 15. Team: The “Creatives” (actual team headshot) Front end dev #1 Adam Johnson Front end dev #3 Dustin Mazon Front end dev #2 Adam Glenn
  • 17. Team: The Champion (actual champion headshot) Project Champion
  • 18. PRO TIP: SOURCING ACADEMIC CONTENT REQUIRES ACADEMICS It was nearly impossible to source content with communicators and recruiters alone. Get someone from Provost Office or similar on the team.
  • 19. OUR GOALS • Standardize content. • Highlight our value proposition. • Address full life-cycle of a student. • Optimize for search engines. • Deploy accurate content quickly. • Expand to all three regional campuses.
  • 20. OPTIMAL FINAL OUTCOME Course catalog SSC data O*NET career data Program pages Major Maps Other campus outlets via API Share what we’ve collected Consume content from others Make connections Curriculum matrix Admissions Registrar CLASS Programs Database
  • 21. WHAT WASN’T A GOAL? We didn’t want this to be a primary source of truth for academic content for internal stakeholders. That is asking for trouble.
  • 22. OUR REQUIREMENTS • Build glue, not CRUD. • 3rd party • Robust API system • Deep content relationships (COPE). • Ability to preview changes. • Improved program search.
  • 23. OUR PLAN OF ACTION • Gather a team • Develop goals and requirements • Evaluate 3rd party and technical options • Source content from domain experts • Build technical solution and design new program pages • Maintain!
  • 25. CONTENTFUL: THE GOOD • APIs for images and content preview, delivery and management plus webhook support. • Click-and-edit content modeling. • Rich, pre-built field types. • Can also create UI Extensions to extend their interface. • Media management with generous storage. • Multi-language using “locales.” • Free to try. 😎
  • 27. CONTENTFUL: THE BAD • Need to use their SDKs to access API and data. • No easy way to limit how much data was returned when using API. Objects > 7MB. • 50 max fields per content type. • Rate limiting when posting and getting data. • No XML option. • API calls were limited. 💵
  • 29. MEASURING UP Build glue, not CRUD. 3rd party Robust API system Deep content relationships (COPE). Ability to preview changes. Improved program search.
  • 30. Hmm, Contentful… •Is great for content organization! 😁 •Has a high bar for content re-use by myself and other campus developers. 😞
  • 31.
  • 35. ENTER AWS API GATEWAY
  • 37. AWS SERVICES: THE GOOD • It’s all glue: • No hardware to manage. “Serverless.” 🎉 • Events/SNS - easily stitch products together. • Testable • Define my own API and become dumb data endpoint. • Click-and-edit configurations. • Could implement Contentful workarounds: • “Reverse GraphQL” and “post-process” rules to optimize content for outlets. Allows for inheritance, key building and richer objects. • No rate limiting and layer of resiliency. • XML and JSON
  • 38. AWS SERVICES: THE GOOD “Reverse GraphQL” “Post-process”
  • 39. AWS SERVICES: THE GOOD “Reverse GraphQL” “Post-process”
  • 40. AWS SERVICES: THE BAD • Actual content is further away from final outlet. Sometimes need to force pull data to update. • Need to update “reverse GraphQL” configurations when content models change. • DynamoDB doesn’t like large objects when scanning. Have had to develop *Thin tables. • Supported programming languages are limited.
  • 41. PRO TIP: EMBRACE NODE.JS
  • 42. PRO TIP: BETA/STABLE • Set-up BETA alias in Lambda to point at $LATEST version. • Use versions in Lambda to publish stable code. • Set-up STABLE alias to point at latest stable version. • Add stage variables to the targeted Lambda function in API Gateway. • Extend to DynamoDB for beta and stable datasets.
  • 43.
  • 44. OUR PLAN OF ACTION • Gather a team • Develop goals and requirements • Evaluate 3rd party and technical options • Source content from domain experts • Build technical solution and design new program pages • Maintain!
  • 45.
  • 46. • Read every major’s entry in the course catalog. Twice. Take notes. • Design or content first? 🐔 or 🥚? • Use meetings and workshops over worksheets. A traveling circus. 🎪 • Identify owners of certain types of content (e.g. tuition and requirements). Always defer to them and their processes. LESSONS LEARNED: SOURCING CONTENT
  • 47. • Automate the import of content whenever possible. • Find proxies for missing content. Our career outcomes come from O*NET. • Find experts to review the proxies. Our Career Services Office chose the careers for each major. • Prep for naming complaints and groups wanting to “opt-out.” This won’t happen until you’re done and reality hits. LESSONS LEARNED: SOURCING CONTENT
  • 48. OUR PLAN OF ACTION • Gather a team • Develop goals and requirements • Evaluate 3rd party and technical options • Source content from domain experts • Build technical solution and design new program pages • Maintain!
  • 49.
  • 50. CONTENTFUL CONTENT MODELS An early attempt to document content models.
  • 51. • Admission Requirement Rules • Announcements • Areas of Emphasis • Building Types • Buildings • Campuses • Capstone Projects • Career Abilities • Career Interests • Career Knowledge • Careers • College and Schools • Companies • Cost of Living Rules CONTENTFUL CONTENT MODELS 68Content Models
  • 52. AWS ARCHITECTURE Poor diagram of the system architecture.
  • 53. AWS ARCHITECTURE: INPUT Store Data Lambda, Events, DynamoDB and CloudSearch Post-Process Lambda Validate Lambda and SNS Invoke Contentful
  • 54. AWS ARCHITECTURE: INPUT Store Data Lambda, Events, DynamoDB and CloudSearch Validate Lambda and SNS Invoke Contentful Post-Process Lambda
  • 55. AWS ARCHITECTURE: INPUT Post-Process Lambda Validate Lambda and SNS Invoke Contentful Store Data Lambda, Events, DynamoDB and CloudSearch
  • 56. AWS ARCHITECTURE: INPUT Validate Lambda and SNS Invoke Contentful Post-Process Lambda Store Data Lambda, Events, DynamoDB and CloudSearch
  • 57. AWS ARCHITECTURE: INPUT Store Data Lambda, Events, DynamoDB and CloudSearch Validate Lambda and SNS Invoke Contentful Post-Process Lambda
  • 58. AWS ARCHITECTURE: OUTPUT Request and Render PHP, JavaScript and Ruby Route, Transform and Deliver API Gateway Validate and Process Lambda Fetch Data DynamoDB and CloudSearch
  • 59. AWS ARCHITECTURE: OUTPUT Request and Render PHP, JavaScript and Ruby Route, Transform and Deliver API Gateway Validate and Process Lambda Fetch Data DynamoDB and CloudSearch
  • 60. AWS ARCHITECTURE: OUTPUT Request and Render PHP, JavaScript and Ruby Route, Transform and Deliver API Gateway Validate and Process Lambda Fetch Data DynamoDB and CloudSearch
  • 61. AWS ARCHITECTURE: OUTPUT Request and Render PHP, JavaScript and Ruby Route, Transform and Deliver API Gateway Validate and Process Lambda Fetch Data DynamoDB and CloudSearch
  • 62. AWS ARCHITECTURE: OUTPUT Request and Render PHP, JavaScript and Ruby Route, Transform and Deliver API Gateway Validate and Process Lambda Fetch Data DynamoDB and CloudSearch
  • 63. AWS ARCHITECTURE: OUTPUT Request and Render PHP, JavaScript and Ruby Route, Transform and Deliver API Gateway Validate and Process Lambda Fetch Data DynamoDB and CloudSearch
  • 64. API ENDPOINTS Scan: Search: By key: Endpoints support query string options like “format,” “sortby” or “groupby.” The {contentType} path variable gets matched to a config by the Lambda function. {contentKey} is used in the DynamoDB look-up. /{contentType}?options /{contentType}/search?options /{contentType}/{contentKey}?options
  • 72.
  • 73. OUR PLAN OF ACTION • Gather a team • Develop goals and requirements • Evaluate 3rd party and technical options • Source content from domain experts • Build technical solution and design new program pages • Maintain!
  • 74. Maintaining the Database • Contact primary sources of central data and get updates. • Three times a year ask recruiters and communicators to update content. • Give them access to an internal website and Word docs with content. • Almost zero feedback. 😢
  • 75.
  • 76. REVIEW OUR PLAN OF ACTION • Gather a team • Develop goals and requirements • Evaluate 3rd party and technical options • Source content from domain experts • Build technical solution and design new program pages • Maintain!
  • 77. REVIEW OUR GOALS Standardize content. Highlight our value proposition. Address full life-cycle of a student. Optimize for search engines. Deploy accurate content quickly. Expand to all three regional campuses.
  • 78.
  • 79. Hero Block Model & Pattern Quicklinks Block Model & Pattern Profile Block Model & Pattern OUR FUTURE: CONTENT MODELS + LAYOUT MODELS
  • 80. OUR FUTURE: CONTENT MODELS + LAYOUT MODELS Program Title Degree Designation Course Delivery Option Advisement Sheets Areas of Emphasis … more … Name Background Image Description Program Interests … more … Blocks - Student 🎩 Content Model Layout Model Post-Process Magic Design System Pattern/Layout - Student
  • 81. OUR FUTURE: CONTENT MODELS + LAYOUT MODELS power.wvu.edu wisdom.wvu.edu courage.wvu.edu Static Sites Design system-based static site generator “Reverse GraphQL” / post-process Contentful DynamoDB Build Pipeline
  • 82. OUR FUTURE: CONTENT MODELS + LAYOUT MODELS
  • 83. THANK YOU / QUESTIONS? Dave Olsen / david.olsen@mail.wvu.edu