Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Ähnlich wie How to successfully migrate to Bazel from Maven or Gradle - Riga Dev Days
Ähnlich wie How to successfully migrate to Bazel from Maven or Gradle - Riga Dev Days (20)
Mehr von Natan Silnitsky
Mehr von Natan Silnitsky (20)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
How to successfully migrate to Bazel from Maven or Gradle - Riga Dev Days
- 1. Natan Silnitsky Backend Infra Developer, Wix.com How to successfully migrate to Bazel from Maven or Gradle The wait is over natans@wix.com twitter@NSilnitsky linkedin/natansilnitsky github.com/natansil
- 2. Maven 45 Bazel BAZEL IS FAST avg build time (minutes) 1
- 3. WIX BACKEND ~250 developers ~750 micro-services >10M LoC, Scala Java, Node.js K’s of builds every day
- 4. Maven + Teamcity Bazel + Google Cloud WE MIGRATED Infra change
- 6. Maven + Teamcity Bazel + Google Cloud WE MIGRATED hours few minutesInfra change
- 7. AGENDA Bazel fast, correct, and scalable Migration to Bazel What you’ll need to do
- 8. In Bazel You control the level of granularity to build. java_project ├── WORKSPACE └── src └── main └── java └── com ├── example │ ├── A.java │ ├── B.java │ └── C.java └── example2 ├── D.java ├── E.java └── F.java
- 9. The basic build unit is called a target java_project ├── WORKSPACE └── src └── main └── java └── com ├── example │ ├── A.java │ ├── B.java │ ├── BUILD │ └── C.java └── example2 ├── D.java ├── E.java └── F.java Target Targets are instances of rules.
- 10. A RULE f(A,B,C)f( ) Source files env’ variables dependencies A B C java library | C++ binary | python test | shell script | many many more... wix
- 11. java_project ├── WORKSPACE └── src ├── main │ └── java │ └── com │ └── example │ ├── A.java │ ├── B.java │ ├── BUILD FILE │ ├── C.java │ └── Example.java └── test A C A.java + :c -> java_library -> a.jar java_library( name="a", srcs=[ "A.java", ], deps = [":c"]) $ bazel build :a
- 12. A RULE f(A,B,C)f( ) A B C INPUTs source files | dependencies | tools configurations
- 13. A RULE A B C f( ) f(A,B,C) Sandbox source files | dependencies | tools configurations
- 14. A RULE f(A,B,C) OUTPUTs f( ) A B C f(A,B,C) f( ) A B C SAME
- 15. Maven / Gradle BAZEL MANY SMALL TARGETS
- 16. MUCH MORE PARALLEL Maven / Gradle BAZEL
- 17. MUCH MORE PARALLEL Maven / Gradle Compilation Test Run BAZEL Pending
- 18. ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ BAZEL INCREMENTAL
- 19. ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ GOOGLE CLOUD / OSS BAZEL Cache REMOTE EXECUTION 100s Workers CI/local tradeoff
- 20. maven local clean 45 min IT’S WORTH IT
- 21. 45 minmaven local clean 20 minbazel local clean 8.5 minbazel remote clean IT’S WORTH IT bazel fully cached 0.1 min
- 22. IT’S WORTH IT IN SOME CASES Startup Small Codebase Medium Codebase Large Codebase WAIT
- 23. IT’S WORTH IT IN SOME CASES Startup Small Codebase Medium Codebase Large Codebase WAIT CONSIDER
- 24. IT’S WORTH IT IN SOME CASES Startup Small Codebase Medium Codebase Large Codebase WAIT CONSIDER ADOPT
- 25. HERE IS WHAT YOU’LL NEED TO DO. #1 #2 #3 #4 #5 Choose granularity for migration
- 28. Package Granularity Java / Scala package More sophisticated tool, more build configs to maintain Performance gain? Easy to migrate? Keep maven/gradle Granularity
- 29. Package Granularity Java / Scala package Code analysis tool, more build configs to maintain Performance gain? Easy to migrate? Keep maven/gradle Granularity
- 31. DEMO
- 32. HERE IS WHAT YOU’LL NEED TO DO. Choose granularity for migration #1 #2 #3 #4 #5 Decide on amount of source control repos
- 33. 1 repo A few repos Many repos Bazel is optimized for 1 repo repo per project
- 34. 1 repo A few repos Many repos Bazel is optimized for 1 repo repo per project git_repository( name = "some_repo", remote = "git@github.com:org/repo.git", commit = "4d97fc0f22c651...4a2db52", )
- 35. 1 repo A few repos Triggering Highly inter- dependant Naive - False triggers Many repos repo per project
- 36. 1 repo A few repos Need to manage external source dependencies? Static list Dynamically updated list Many repos repo per project Triggering Highly inter- dependant False triggers
- 37. 1 repo A few repos Static list Dynamically updated list May require Mercurial / VFS Need to manage external source dependencies? Supported by Git? Many repos repo per project Triggering Highly inter- dependant False triggers
- 38. 1 repo A few repos Static list Dynamically updated list May require Mercurial / VFS Need to manage external source dependencies? Supported by Git? Many repos repo per project Triggering Highly inter- dependant False triggers
- 39. HERE IS WHAT YOU’LL NEED TO DO. Choose granularity for migration #1 #2 #3 #4 #5 Decide on amount of source control repos Decide on CI Server and remote execution / caching
- 40. OSS solutions Google Cloud build $$$free
- 41. OSS solutions Google Cloud build $$$free Google Results Store UI - per target results view/Filter?
- 42. Google Cloud build UI
- 43. OSS solutions Google Cloud build $$$free Google Results Store UI - per target results view/Filter? Dashboard Build History filters Jenkins Dashboard
- 44. Google Cloud Build History
- 45. OSS solutions Google Cloud build Google Results Store UI - per target results view/Filter? Dashboard Remote Build Execution? $$$free BuildFarm / Buildbarn / Buildgrid - immature/DIY Google RBE Alpha Build History filters Jenkins Dashboard
- 46. OSS solutions Google Cloud build $$$free UI - per target results view/Filter? Dashboard Remote Build Execution? Google Results Store BuildFarm / Buildbarn / Buildgrid - Still immature Google RBE Alpha Build History filters Jenkins Dashboard
- 47. HERE IS WHAT YOU’LL NEED TO DO. Choose granularity for migration #1 #2 #3 Decide on amount of source control repos Decide on CI Server and remote execution / caching #4 Think about local dev experience #5
- 48. Linux Mac CI Remote cache Use toolchains Separate Cache
- 49. Linux Mac Remote cache Compile time On second build Use toolchains Separate Cache
- 50. Linux Mac Remote cache Compile time Parallel tests On second build Port collision Sandbox Use toolchains Separate Cache
- 51. Linux Mac Remote cache Compile time Parallel tests Intellij Support On second build Port collision Sandbox Use toolchains Separate Cache
- 52. Linux Mac Linux in Docker Remote cache Compile time Parallel tests Intellij Support On second build VM overhead + File sync Port collision Sandbox File sync Use toolchains Separate Cache
- 53. HERE IS WHAT YOU’LL NEED TO DO. Choose granularity for migration #1 #2 #3 Decide on amount of source control repos Decide on CI Server and remote execution / caching Think about 3rd-party version resolution #4 #5 Think about local dev experience
- 54. 1 version per repo Multiple versions per repo jarBazel is optimized for 1 version of each external jar in the repo
- 55. 1 version per repo Multiple versions per repo Must be declared - exodus tool Transitive dep. resolution? jar
- 56. 1 version per repo Multiple versions per repo Must be declared - bazel deps tool jar Transitive dep. resolution? Trade Offs? Simple config. Fix code breakage Flexible but hard to maintain, can easily break
- 57. Multiple versions per repo Must be declared - bazel deps tool jar Transitive dep. resolution? Trade Offs? 1 version per repo Simple config. Fix code breakage Flexible but hard to maintain, can easily break
- 58. WITH THE RIGHT CHOICES YOU CAN DO IT. Choose granularity for migration #1 #2 #3 #4 #5 Decide on amount of source control repos Decide on CI Server and remote execution / caching Think about 3rd-party version resolution Think about local dev experience
- 59. Faster, Reliable Builds Ecosystem is becoming better Migration is becoming easier BAZEL IS WORTH IT
- 60. Migration Stories Migrating to Bazel? 5 crucial questions... - My Blog Series on Medium We Switched from Maven to Bazel and Builds Got 10x Faster - Redfin Migrating From Maven to Bazel at Salesforce
- 61. Thank You natans@wix.com twitter@NSilnitsky linkedin/natansilnitsky github.com/natansil
- 62. Q&A natans@wix.com twitter@NSilnitsky linkedin/natansilnitsky github.com/natansil