ScalaCourses.com has been serving online Scala and Play training material to students for over two years. ScalaCourses.com teaches courses on the same technology stack that the web site runs on. The Cadenza application that powers ScalaCourses.com is a Play Framework 2 application, written in Scala and using Akka, Slick, AWS and Postgres. Some of the architectural features in Cadenza that allow a modest-sized Play application to serve large amounts of multimedia data efficiently is discussed, including technical details of how to work with an immutable domain model that can be modified. Over the last 2+ years the underlying technology has changed a lot; a brief history of Play Framework will be recounted, and how that impacted Cadenza. The talk concludes with a proposal regarding Play Framework's future.

1. Lessons Learned from a 2-year-old Play App
Aug 14, 2015 Scala By The Bay Oakland CA
Play Architecture, Implementation,
Shiny Objects, and Proposal

2. • About me
• About the Cadenza web app that powers
ScalaCourses.com
• Brief History of Play Framework
• Architectural Features for Multimedia
Scalability
• Standards and Processes
Outline

3. Who is this guy, and why should I care?
About Mike Slinn

4. About Mike Slinn – Today
• Architect and lead developer of Cadenza, the webapp that powers ScalaCourses.com
• Co-ordinates the ScalaCourses.com instructors
• Contributor to the Scala / Play courses provided through ScalaCourses.com

5. About Mike Slinn – Selected History
• Co-founder of Vancouver AutoCAD User Society (“Oldest and Most Dangerous AutoCAD User
Group”, world’s largest)
• Co-Chair of Silicon Valley Ruby Conference (world’s largest)
• Former primary organizer Bay Area Scala Enthusiasts (grew to world’s largest)
• Former organizer of SF Scala , Scala for Startups

6. About Mike Slinn – Author & Expert
• Wrote 3 books, including “Composable Futures With Akka 2.0”
• Wrote over 5 dozen technical articles for computer magazines
• Former product marketing manager at Borland for C, C++, .NET & Ruby on Rails IDEs
• Traveled the world for decades teaching computer software commercially
• Now travel much less and uses video conferencing instead
• Often retained as a software expert for court cases and arbitrations involving software

7. Powers ScalaCourses.com
About Cadenza

8. ScalaCourses.com

9. ScalaCourses.com
• Best learning experience for Scala and Play Framework
• Most cost-effective option
• Material is maintained up-to-date
• Students have access for a year at a time
• We are looking for course authors

10. "Better to train outstanding people into new areas than to accept
marginal people with some skills" – NASA

11. Cadenza – Powers ScalaCourses.com
• ScalaCourses.com has been serving online Scala and Play training material for 2+ years
• Cadenza is:
• Online content editor and multimedia delivery mechanism
• Play Framework 2 application
• Written in Scala, uses Akka, Slick, AWS and Postgres
• Originally deployed app to Heroku, now on Digital Ocean (1GB droplet)
• Still using Heroku database
• Some functionality is broken out into microservices
• View free Cadenza course at ScalaCourses.com to learn more

12. Cadenza Features – A&A
• Sophisticated and fine-grained authentication and authorization (A&A)
• Hierarchical permissions control access to hierarchy of entities
• Entities: Lecture, Section, Course, Group, PublishingConfiguration, User, Roles
• Next release adds: StudyGuide, Exam
• Roles: Visitor, Student, TA, Instructor, Videographer, CopyEditor, Publisher, SuperUser
• Deep integration of webapp framework, A&A dependencies and persistence mechanism
• Breaking changes in Play Framework seriously impacted plugin developers, which locked us
in Play 2.2.6 – more on this later

13. Architectural Features for Multimedia Scalability
• Cadenza serves large amounts of multimedia data quickly and economically
• Play app does not handle any data
• Play app only handles authorization and authentication
• Use AWS S3 and CloudFront to serve data
• Signed URLs
• Avoid enumeratees, very slow; perhaps Akka-HTTP for Play will eventually be an alternative?

14. Cadenza Features – Online Editors
• Quick demo of lecture editor and section organizer

15. Cadenza Features – Live Content Caching
• Content is parsed, summarized and cached
• Lots of lazy vals, memoization and usage of caches
• Decision of which caching mechanism and cache type to use depends on artifact purpose
• Tabs in web pages are demand-loaded to increase responsiveness and decrease system load
• Students can download a course notes by printing to PDF on demand
• Current content is printed
• Content is aggregated by course

16. Cadenza Features – White Label Branding
• Designed to be branded for third parties who use Cadenza for course authoring and delivery
of their own courses

17. Cadenza Features – Asset Controller
• Videos and images are delivered by AWS CloudFront
• Cadenza’s asset manager only delivers assets when in development mode
• AWS signed URLs provide time-limited access to remote assets
• App is optimized for data retrieval (read access), so students do not present a significant load
• A separate instance for course authoring is recommended, with more memory

18. Google Guava to the rescue!
Concurrency, Persistence & Caching

19. Cadenza Features – Database
• No joined tables; domain model is created in memory
• Cadenza’s AbstractTrainingEntity trait provides write-through caching support
• Most database requests are served from local in-memory domain model cache
• Domain model is immutable; stale entities are flushed from cache and replaced as required
• Fast and scalable
• “Immutable domain model with write-through caching” (IMWC) pattern is easy to program to
• IMWC requires a deep integration of webapp framework and database

20. Cadenza Features – Immutable / Lazy Domain Model

21. Cadenza Features – Immutable / Lazy Domain Model
• A lazily instantiated, immutable domain model starts fast
• Naturally optimized for read access
• Naturally supports concurrency
• Stateless so it scales horizontally
• Best when coupled with or implemented by local cache
• Must differentiate between:
• Core domain model, running out of local cache
• User data, run from a distributed cache or loaded from a database
• Transient data, not cached

22. Caches and Concurrency
• The Play cache interface is not threadsafe (affects all implementations, such as EHCache).
• This does not matter if multiple threads attempt to set the same value, however.
• Better to implement a threadsafe cache using Google Guava’s CacheBuilder.

23. Cadenza Front Page (Cached)
def apply(pubConf: PublisherConfig, referrer: String="")
(implicit request: RequestHeader): Html =
views.html.main(Model.frontPageTitle, enableGoogleAnalytics=true) { Html.empty } {
Html(s"""<div class="row-fluid">
| <div class="span6">
| ${pubConf.cache("courseBlurbs", courseBlurbs)}
| </div>
| <div class="span6">
| ${ pubConf.comments.getOrElse("") }
| </div>
|</div>
|${ request.flash.get("signedUp").map { _ => signedUp }.mkString("n") }
|""".stripMargin)
}

24. Those ignorant of history are doomed to repeat it
Brief History of Play Framework

25. Play Framework is Popular

26. Play versions
• 1.0 Oct 19 2009
• 1.1 Nov 01 2010
• 1.2 Apr 13 2011
• 2.0 Mar 13 2012
• 2.1.0 Feb 06 2013
• 2.2.0 Sep 20 2013
• 2.3.0 May 30 2014
• 2.4.0 May 26 2015
• 3.0 May 2016
Huge API churn

27. 2010-11-01 v1.1 RC3

28. 2012-04-07 v1.2.4

29. 2012-04-13 - Play 2.0 announced

30. 2013-05-18 – v2.1.1

31. 2013-09-29 – v2.2.0

32. 2014-06-25 – v2.3.0

33. 2015-08-14 – v2.4.x

34. Next year: Play 3.0
• Biggest feature: Reworked dependency injection
• No details yet, even though ship date is likely 7 months away.
• No indication if CDI 2.0 (Contexts and Dependency Injection), which implements JSR 365, will
be supported
• Initial reaction to my suggestion was that nothing could be learned from standards

35. Scala Ecosystem generally lacks S&P
Standards and Processes

36. Hmm…
• Go-Karts - Feel fast, are slow
• 747s - Feel slow, are fast
• “Ship it!” - Feels fast, is slow
• “Test + Document it!” - Feels slow, is fast
• “Define standards and engage community” – Feels slow, is powerful
• Build the right thing (product management, standards) vs. build the thing right (agile)
• High agile velocity without adult supervision => API churn

38. Standards Engage Ecosystems
• “Lack of standards threatens to derail the big data innovation train” – Jonas Bonér
• Standards also important for Scala ecosystem in general and Play in particular
• Jonas only recently has begun to profess the standards religion
• The Akka project is the only one within Typesafe that shows evidence of interest in standards

39. Standards and Process
• Standards bring people together: core developers, customers, 3rd party developers
• Result (of a healthy standards process):
• Well thought out specifications
• Quality reference implementations
• This drives:
• Customer adoption
• Lower cost of operation
• Less API churn

40. Play Framework is Special
• Play Framework requires integrating language features, persistence, security, A&A, etc
• Integration success metrics are hugely affected by availability of good standards
• Change in mindset required:
• What can be learned from existing standards?
• How can customer usage patterns be codified into standards with competing implementations?
• Result:
• Huge reduction in API churn
• Much greater adoption by enterprises
• Much higher quality product

41. Standards and Process Can Be Powerful
• Sun was bureaucratic but that allowed them to foster standards
• Java is still hugely important decades later as a result
• Ruby was initiated by young developers
• Disdained standards (our projects are so different that standards do not apply)
• Ruby became merely the language that powered Ruby on Rails
• 40% drop in demand for Rubyists in 2014, downward trend is continuing
• Scala’s ecosystem also lacks standards and a vehicle to provide process

42. Just Enough Bureaucracy
• Bureaucracy is boring, but when used appropriately it allows many participants to interact
effectively
• Project management requires bureaucracy
• Release planning
• Experimentation prior to making design decisions
• Communicating and engaging users
• Integration with other products

43. “The bug wasn't ours. It was in an open source project we use,
but do not fund or contribute to in any way.”
Proposal: Play Framework Foundation

44. Set Play Free!
• Open source is becoming even more open – because it has to
• Joyent gave away Node.js to the Node.js Foundation
• Google gave away all rights for Kubernetes to the Cloud Native Computing Foundation
• Backed by Cisco, eBay, Goldman Sachs, IBM, Intel, Joyent, Twitter, Google and Docker
• VMware gave away Cloud Foundry to the Cloud Foundry Foundation
• All of these foundations are managed by the Linux Foundation, which is now a meta-foundation
• Typesafe should give Play a boost by doing the same:
The Play Framework Foundation

48. Thank you!
• Mike Slinn
• mslinn@scalacourses.com
• @scalacourses