Presentation from AUSLUG 2015 / Inform 2015 on Build Automation for XPages.
Building NSF and Plugins using Jenkins, Ant, Headless Designer and Headless Eclipse PDE Build
Learn the Fundamentals of XCUITest Framework_ A Beginner's Guide.pdf
Build automation for XPages - AUSLUG 2015
1. Build Automation for XPages
Cameron Gregor | Senior Systems Engineer
Jord International Pty Ltd
www.jord.com.au
2. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
Cameron Gregor
• Blog – gregorbyte.com
• Twitter - @gregorbyte
• Github - camac
3. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
Agenda
● What is Build Automation?
● Introduction to APACHE ANT
● Introduction to Jenkins CI Server
● Building NSF from Source with Headless Designer
● Building OSGi plugins with PDE Build
● Deployment Considerations
4. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
WHAT IS BUILD AUTOMATION?
5. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
What do we mean by build automation?
• Automation of common tasks in software development
– Compiling Source Code
– Running Tests
– Generating Documentation
– Deploying to Production / Testing
• Repeatable
– Avoid human error
– Avoid dependency on key personnel
6. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
Basic Tools involved for a Build ‘Job’
• A Script of Tasks to follow
• A Computer to run the script
• Some programs to execute the Tasks
• A mechanism to Trigger the Job
7. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
Example Build Job
Checkout
Source Code
Ensure
Dependencies
Met
Compile
Package into
an Artifact
8. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
INTRODUCTION TO APACHE ANT
9. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
http://ant.apache.org/
• ANT = Another Neat Tool
• Written in Java
– Platform independent
• Mainly used for building Java applications
– However: can be used for anything
• Easily Extensible via ‘antlibs’ (ant libraries)
• Run from command line
ant
10. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
Why ANT?
• It worked!
• Documentation was easy to follow
• Extending it was easy to follow
• Installation was easy
• Very Flexible (Some say this is a disadvantage though)
• Alternatives
– Gradle: I hear this is pretty good but I don’t know much about it
– Maven: I tried to give this a go but It was too big a learning curve for too little gain
– others
12. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
Example Tasks
• Print Hello World to the Console
<echo message=“Hello, world”/>
• Make a new Directory
<mkdir dir=“lib”/>
• Compile java
<javac srcdir="${src}" destdir="${build}" classpath="xyz.jar" debug="on"
source="1.4" />
• Create a Zip File
<zip destfile="${dist}/manual.zip“
basedir="htdocs/manual"/>
• Execute a program
<exec executable=“some.exe” failonerror=“true”/>
13. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
Example Build File
14. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
Example Build File Output
> ant
Buildfile: V:ProjectsSimpleBuildFilebuild.xml
clean:
[delete] Deleting directory V:ProjectsSimpleBuildFilebuild
init:
[mkdir] Created dir: V:ProjectsSimpleBuildFilebuild
build:
[touch] Creating V:ProjectsSimpleBuildFilebuildbar.txt
BUILD SUCCESSFUL
Total time: 0 seconds
15. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
Properties
• Similar to a ‘parameter’ or argument
• Provide a way to allow customisation of a build,
– e.g. specify different directory locations, different options
• Reduce duplication of strings within the build file
16. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
Setting Properties within Build File
• Set a property to a Value
– <property name=“propname” value=“propvalue”/>
• Set a property to a File or Directory Location (relative to base directory)
– <property name=“propname” location=“someDir/someFile”/>
• Properties can be read from a ‘properties’ file from within build file
<property file=“build.properties”/>
17. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
Setting Properties when Calling Ant
• Properties can be passed in via command line when calling ant using –D flag
ant –Dproperty=value
• Or read from properties file from command line
ant –propertyfile build.properties
18. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
Build File Using Property
19. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
Buildfile: V:ProjectsSimpleBuildFilebuildUsingProperty.xml
clean:
[delete] Deleting directory
V:ProjectsSimpleBuildFilebuildPropertyCmdLine
init:
[mkdir] Created dir:
V:ProjectsSimpleBuildFilebuildPropertyCmdLine
build:
[touch] Creating
V:ProjectsSimpleBuildFilebuildPropertyCmdLinebar.txt
BUILD SUCCESSFUL
Total time: 0 seconds
Build with Property Output
>ant –buildfile buildUsingProperty.xml -DbuildDir=buildPropertyCmdLine
20. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
INTRODUCTION TO JENKINS CI
21. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
Jenkins CI Server
• An application that monitors executions of repeated jobs with focus on
– Building / Testing Software projects continuously
– Monitoring executions of externally run jobs such as cron jobs
• Easy installation, no database required
• Distributed builds, master / slave possible
• Extensible via Plugins, with a plugin directory ready to access.
• Web Interface with rest-like api as well
• Requires JRE of 1.6 or later
• FREE! As in Speech and Beer – MIT License, very permissive
22. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
Jenkins vs Hudson
• https://jenkins-ci.org/
• Originally created at Sun Microsystems as ‘Hudson’
• Dispute between Project Team and Oracle
• ‘Jenkins’ created as a Fork
• Kohsuke Kawaguchi is Creator and went with Jenkins
• Hudson is still available too http://hudson-ci.org/
@kohsukekawa
23. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
Note: the following Screen shots are
taken from a Jenkins installation with
additional plugins installed so may look
different to your install
24. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
Home Page
25. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
Status Icons
• S = Last run status
– Grey = Disabled
– Blue = Success
– Red = Fail
• W = Health of Recent Builds (last 5)
– Sunny - all succeeded
– Sunny with Cloud = 1 failed
– Cloud = 2 failed
– Rain 3 failed
– Lightning – all recent builds failed
39. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
Jenkins - Conclusion
• Price is right
• Good community – lots of plugins
• Easy to install and backup
• Works for us!
40. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
BUILDING NSF FROM SOURCE
WITH HEADLESS DESIGNER
41. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
Domino Designer Source Control Enablement
NSF ODP
42. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
The .project file
43. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
Headless Designer
• Not Really Headless though
• New feature since 9.0.1
• Allows for automated building of NSF from On Disk Project
• Still a bit clunky but it works
• http://www-10.lotus.com/ldd/ddwiki.nsf/dx/Headless_Designer_Wiki
• 2 Options for running
– Single Command
– Command File (Multiple commands)
• Built NSF can only end up in the Notes Data Folder (not in subdirectory)
44. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
Requirements on machine
• Domino Designer 9.0.1
• NOTES.INI entry
– DESIGNER_AUTO_ENABLED=true
• Notes ID with NO PASSWORD
• Check Source Control preferences
• Maybe add Notes Program directory to PATH environment variable so you can
run ‘designer’ from anywhere
• Disable any dialogs that prompt on start-up and shutdown
45. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
Setting No Password
46. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
Source Control Preferences
53. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
You can maximize it manually
54. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
Click Progress Icon to Monitor
55. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
Oh no – Designer doesn’t have latest plugins
56. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
INSTALL LATEST PLUGINS
TRY AGAIN
57. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
Good – No problem now
58. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
Plugin management for Headless Designer
• Wiki previously mentioned some ‘Headless’ commands that auto installed /
removed plugins
• Removed in version 27 of
Wiki article
• Read version 26 to see them
• Probably still work?
• I haven’t tried yet
59. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
A Minor Problem
• Most build tasks will let you know when they finish and whether success or fail
• Headless Designer has short attention span
• designer.exe actually starts notes2.exe in parallel and then shuts down
• Need a way to monitor when notes2 shuts down
• Egor Margineanu @egmar came up with a Powershell script to do so
– http://www.egmar.ro/blog/2014/02/10/using-continous-integration-in-xpages-projects/
60. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
Powershell script
• Erases previous HEADLESS0.log file
• Launches Headless Designer
• Finds the ‘notes2’ process
• Waits until the notes2 process has exited
61. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
Jenkins Installation Notes
• Jenkins can be installed as Service or started as normal application
• I had trouble running Headless Designer when installed as service
– Others have said it works for them if they tick ‘Service can interact with desktop’
• Instead I launch Jenkins from a logged in user
62. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
BUILDING OSGI PLUGINS WITH
PDE BUILD
63. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
Building Plugins != Building Java
• OSGi plugins are complicated
• Bundles need to ‘resolve’ dependencies
• Eclipse has PDE Build
– Can build using Eclipse UI
– Can also build Headless
64. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
Plugins, Features and Update Sites
Plugins
PluginA
PluginB
PluginC
Features
Feature A
• Plugin A
Feature B
• Plugin A
• Plugin B
Feature C
• Plugin C
65. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
Building Normal Java
• Java Runtime Environment
• Classpath
• javac!
66. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
Building OSGi Plugins
• Java Runtime Environment
• Target Platform (Other OSGi plugins)
• Your Plugin(s) Source Code
• A Feature Project pointing to your plugins
67. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
Manually Building Plugins
• Create Feature Project
– Add Plugins to feature
• Create Update Site Project
– Add Features to Update Site
• Build Update Site
68. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
69. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
After running Build All
70. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
What do you need to build Domino OSGi Plugins
• Eclipse for RCP and RAP Developers
– Eclipse.org, latest version is ‘Luna’
• Source Code of your plugins
• A Feature Project which specifies the plugins that are to be built
• Your Target Platform (all the other Plugins will be co-located)
• Your Target Java Runtime Environment (all the jars on the bootclasspath)
– We run our headless builds on a development server that has a Domino instance
• A Build Configuration directory
• A Build Directory
73. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
Automated Building of Features
• Basic Setup
– Create a Build Configuration Directory in your project
– Copy the build.properties template file to your build configuration dir
– Modify build.properties as needed
• buildDirectory property specifies the directory to perform build in
• Before Each ‘Build’
– Wipe BuildDirectory clean
– Copy Source of Plugins and feature into Build directory
– Structure is
• <buildDirectory>plugins<your plugin projects here>
• <buildDirectoyr>features<your feature project here>
75. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
Build.Properties for Domino Plugins – in a Nutshell
• Please Build feature ‘X.Y.Z’
• Please do not to download Source from CVS
• We are not creating a ‘Product’ so don’t run the packager
• Please make plugins and features into Jars
• By the way this is our:
– Runtime Environment ( jar’s that are on the boot class path),
• these are the jars in <domino/jvm/* subfolders
– Target Platform (Plugins that our plugins would like to play with)
• These are the plugins in
– <domino>/osgi/rcp/eclipse
– <domino>/osgi/shared/eclipse
– Any other plugins that are depended on
76. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
Running the build
• Eclipse contains an embedded ‘ant’
• This command invokes the embedded ‘ant’ and tells it to use the pde build
build.xml file
– <eclipse> = Eclipse Directory
– <v> = Version number
java
-jar <eclipse>/plugins/org.eclipse.equinox.launcher_<v>.jar
-application org.eclipse.ant.core.antRunner
-buildfile <<eclipse>/plugins/org.eclipse.pde.build_<v>/scripts/build.xml>
-Dbuilder=<path to the build configuration folder>
77. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
Running build from Ant File
79. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
What to do with Built NSF and Plugins?
• Copying NSF to Server
• Refreshing Designs
– Set template name on built nsf
– Set template inheritance of target nsf
– Refresh design
• Single Copy Xpage Design
• Restarting Http
80. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
81. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
BuildXPages
• Ant Library of tasks for use in deploying and building Xpages apps
• Uses Java Native Access to make calls to Notes C Api
• Only tested on Windows! But give it a try
• Tasks include
– Setting Template Names (inherit from, is master, Single Copy Xpage)
– Copy NSF to server from Local Filesystem
– Deletensf
– Generate Site Xml for plugins
82. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
Generating a Site.xml
• Site.xml file is needed to import plugins and features
• PDE Headless build only generates plugins and features, not site.xml
• Need to generate site.xml yourself
83. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
NSF Update Site
• By Default you must manually import to NSF Update Site
• Karsten Lehmann made slight modification to allow ‘Headless’ import
• http://www.openntf.org/main.nsf/project.xsp?r=project/Open%20Eclipse%20Up
date%20Site
84. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
Conclusion
• If you want to automate some Build Processes you can
• It is fiddly to set up but once you have it there it works
• Contact me if you have any questions!
85. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
APPENDIX A
Notes on Build Properties for OSGi Headless Build
86. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
Notes on Build.Properties
• topLevelElementId would correspond to the feature which defines the plugins to
build
• <BuildDirectory>/features/com.your.feature.id/
Property Value
topLevelElementType feature
topLevelElementId com.your.feature.id
87. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
Notes on Build.Properties
• The Packager is used when building a whole ‘Rich Client Platform’ product
– E.g. Eclipse, IBM Notes, IBM Domino Designer are Rich Client Platform Products
• We are not building a ‘Product’ we are only building plugins
Property Value
runPackager false
88. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
Notes on Build.Properties
• PDE build has an in-built mechanism to ‘fetch’ Source from CVS
• Keep these set to true to avoid any CVS operations
Property Value
skipFetch true
skipMaps true
89. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
Notes on Build.Properties
• outputUpdateJars will generate the plugins and features as ‘.jar’ files
– if set to false, features will not be made into ‘jar’ files
– if set to false, plugins with ‘unpack the plug-in archive after the installation’ will not be
made into ‘jar’ files
– If you plan on loading into an Update site I recommened setting to ‘true’
• resolution.devMode may be needed to be used to allow for less strict validation
of bundle dependencies
Property Value
outputUpdateJars true
resolution.devMode true
90. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
Notes on Build.Properties
• PDE Build matches the Execution
Environment in the Plugin Manifest, to
the related property in build.properties
• These Jars form the ‘boot class path’
• skipBase tells
Property Value
JavaSE-1.6 List of all jars from <domino>/jvm and
subdirectories
skipBase true
91. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
Notes on Build.Properties
• pluginPath is a list of folders which contain features/ and plugins/
– This is equivalent to your ‘Target Platform’
– By Default you will have
• <dominoProgDir>/osgi/rcp/eclipse
• <dominoProgDir>/osgi/shared/eclipse
– You could also include other 3rd party plugin dependencies
• <someotherDir>/someproject/eclipse
Property Value
pluginPath List of Eclipse Plugin folders
92. June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015
Notes on Build.Properties
• If you are building plugins for production, then you may want javacDebugInfo to
be false
• If you are building plugins and want to ‘Remote Java Debug’ them, you will
need line numbers included, and should set javacDebugInfo to true
Property Value
javacDebugInfo true or false
compilerArg -g