at Scala Love in the City 2021

1. GitBucket
Open source self-hosting Git server built by Scala
Naoki Takezoe
@takezoen

2. Who am I?
Naoki Takezoe
● Software Engineer at Treasure Data
○ Presto (Trino) and Spark
● 10 years experience with Scala
○ GitBucket, Scalatra, Apache PredictionIO, etc
○ Japanese edition of Scala Puzzlers

3. GitBucket is...

4. GitBucket is...
● Open source self-hosting Git server
● Initial commit was April 2013
● Built by Scala and Java technologies

5. Current status (Feb 6, 2021)
● 8266 stars
● 158 contributors
● 719 gitter users
● 98 releases

6. Start GitBucket right now!
● Download gitbucket.war from
○ https://github.com/gitbucket/gitbucket
● Run
● Official Docker image is available
$ java -jar gitbucket.war
$ docker run -d -p 8080:8080 gitbucket/gitbucket

7. Motivation

8. Motivation
● In-house Git repository
○ Due to company's security policy?
○ Due to contract with customers?
● Solutions
○ GitHub Enterprise was expensive
○ GitLab installation was awkward
I must make it myself! by

9. Why ?
From First Principles: Why Scala? by Li Haoyi
https://www.lihaoyi.com/post/FromFirstPrinciplesWhyScala.html
● "A Compiled Language that feels Dynamic"
○ Compiled language + Scripting launguage
○ Easy to write with safety
● "A Broad and Deep Ecosystem"
○ Existing Java resources can be leveraged
○ JGit: Pure Java Git implementation
○ Zero dependencies (except JVM) inspired by Jenkins
Personal reasons:
● Wanted to write a real-world applications in Scala
● Just for fun :-)

10. Tour of GitBucket

11. Repository viewer

12. Issues

13. Pull requests

14. Major Features
● Public and private repository hosting
● Dashboard includes activity timeline
● Repository viewer which supports online file editing
● Markdown available Wiki
● Issues and pull requests
● Comment on source code
● Notification via E-mail
● User and group management
● LDAP integration
● Gravator integration
● GitHub compatible Web API
● Plugin system
● External database support (MySQL and PostgreSQL)

15. Kanban and Gantt chart

16. Continuous Integration

17. Major plugins
Name Description
Gist plugin Add code snippet Gist-like functionality to GitBucket
Asciidoc plugin Add AsciiDoc support to GitBucket
Bugspots plugin Apply Google Bugspots to code in GitBucket repositories
Pages plugin Publish repository contents as web sites
Network plugin Add the commit graph view to GitBucket
Emoji plugin Emoji support in Wiki or Issues
RST plugin Add ReStructuredText support to GitBucket
Explorer plugin Add the tree view for repositories on GitBucket
PlantUML plugin Render PlantUML files on GitBucket
Jupyter plugin Render Jupyter or IPython files on GitBucket
Fess Plugin Add full text search capability to GitBucket
Maven repository plugin Host in-house maven repositories on GitBucket
Visit https://gitbucket-plugins.github.io/ to find other plugins!!

18. Technology

19. Technology stack
Git Repository
RDBMS
(H2 / MySQL / PostgreSQL)
JGit Slick
Apache MINA SSHD
Jetty
GitServlet (JGit) Scalatra + Twirl
Git Client Web Browser
SSH HTTP

20. Core technologies are Java components
● Jetty
● H2
● JGit
● Apache MINA SSHD

21. Technology stack (Java parts)
Git Repository
RDBMS
(H2 / MySQL / PostgreSQL)
JGit Slick
Apache MINA SSHD
Jetty
GitServlet (JGit) Scalatra + Twirl
Git Client Web Browser
SSH HTTP

22. Key to minimize development cost
Minimizing dev cost is very important for sustainability of
personal OSS projects
● Java interoperability
○ Scala has good Java interoperability
○ Benefit from existing Java software resources
● Plugin architecture
○ Keep core features minimum for maintainability
○ Leverage community resources
Because GitBucket users are not necessarily mature Scala users, we
avoid too much FP flavor in GitBucket in order to open the door to
contribution and plugin development to them.

23. Challenges in
long-life application with Scala

24. Upgrading Scala and libraries
Scala's source code level backward compatibility is great, but…
● Need to rebuild libraries for new Scala major version
● Abandoned or not-well maintained libraries can be blockers
● Some libraries changed its public interface significantly
(e.g. Slick2 -> Slick3)

25. Experienced Scala major upgrade 3 times
● 2.10 -> 2.11
● 2.11 -> 2.12
● 2.12 -> 2.13
The most painful upgrade!

26. Why upgrading to Scala 2.12 was so painful?
Git Repository
RDBMS
(H2 / MySQL / PostgreSQL)
JGit Slick
Apache MINA SSHD
Jetty
GitServlet (JGit) Scalatra + Twirl
Git Client Web Browser
SSH HTTP
Scalatra development was going down
Destructive change in Slick3

27. Scalatra
● Simple we framework for Scala inspired by Ruby's Sinatra
○ Traditional Java servlet based framework
○ Declarative input validation and mapping framework like Play2
○ json4s based JSON support

28. Scalatra development is going down
● Rise of reactive and functional programming in Scala
○ Emerge of new frameworks such as finagle, http4s and akka-http
○ Main Scalatra developers shifted to http4s
● Became a Scalatra committer
○ Boosted migration to Scala 2.12, and eventually Scala 2.13
● Reduced maintenance cost to make it sustainable
○ Dropped minor features and library dependencies
○ Forked abandoned libraries, took some into Scalatra source tree if
small enough

29. Slick
● Advanced Type-safe SQL builder (former Scala-Query)
○ Very powerful and flexible type-safe API
○ Sometimes generated SQL can cause performance issues,
especially on MySQL, though...
● Super painful upgrade in Slick2 -> Slick3
○ Monadic DBIO introduced in Slick3 affected all existing code

30. Slick2 -> Slick3 migration
● Amount of code that needs to be migrated
○ Affected all existing Slick2 based code including community
developed plugins
● Difficulty of DBIO for GitBucket users
○ GitBucket users are not mature Scala programmers
○ We wanted to keep the bar low
● Scala 2.12 version of Slick2 was not available back then
○ Eventually Scala 2.12 version of Slick2 was released, though
○ Scala 2.13 version has not been released

31. How we migrated to Slick3?
● Created blocking-slick library
○ Slick2 compatible blocking API on the top of Slick3
https://github.com/takezoe/blocking-slick
● We could migrate to Slick3 with minimum effort
https://github.com/gitbucket/gitbucket/pull/1381/files
○ Also, minimized the negative impact on plugin developers

32. Created Java libraries, not Scala libraries
● Markedj (GitHub flavored markdown parser, Java-port of marked.js)
● Solidbase (Multi-tenant and multi-database supported migration tool
based on Liquibase)
● If the library interface is simple enough, no benefit to write in Scala
for library users.
● We don't need to rebuild a library for each major Scala version.

33. Effective strategy for long-term maintenance
● Minimize library dependencies
● Use Java libraries if possible
● Fork or take over library maintenance if needed
These strategies would be effective even for migration to Scala3!

34. Try GitBucket!
● GitHub: https://github.com/gitbucket/gitbucket
● Demo site: https://gitbucket.herokuapp.com/
● Gitter: https://gitter.im/gitbucket/gitbucket
● Blog: https://gitbucket.github.io/gitbucket-news/
● Community Plugins: https://gitbucket-plugins.github.io/

35. Appendix:
How to create GitBucket plugin

36. Create project
● build.sbt
● project/plugins.sbt
This sbt plugin adds necessary library dependencies to the project
and provide configuration and sbt task useful for GitBucket plugin
development.

37. Define plugin manifest
● Plugin.scala (plugin manifest)
Register new controller via
extension point

38. Implement plugin
● HelloWorldController.scala (Typical Scalatra controller)
Bunch of extension points are available, such as:
● Add menus and tabs
● Inject JavaScript
● Register event hooks
● etc

39. Build and Test
● Create a package
● Install to local GitBucket
● Template project
○ https://github.com/gitbucket/gitbucket-plugin-template
● Tutorial
○ https://gitbucket.github.io/gitbucket-news/gitbucket/2015/06/29/ho
w-to-create-plugin.html
$ sbt assembly
$ sbt install