Oracle Developers APAC Meetup #1 - Working with Wercker
This is the hands-on work exercise example used for the Oracle APAC Developers Meetup #1 held in Singapore on 7th February 2018.
The worksheets are intended to be used in conjunction with the slides.
You can find the slides at :
https://www.slideshare.net/DarrelChia1/oracle-apac-developers-meetup-1-working-with-wercker-slides
Meetup Site:
https://www.meetup.com/Oracle-Developers-APAC/events/247111220/
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Oracle Developers APAC Meetup #1 - Working with Wercker Worksheets
1. Wercker Hands On Session
Objectives
The objective of this sheet is to introduce the users to the general features of Wercker.
We hope that through the session, it builds up sufficient foundation on the product that users can
easily explore the more advanced features of wercker.
Pre-requisites
The following software and services are required to complete the hands-on exercise:
1. Hashicorp Vagrant (https://vagrantup.com)
2. Oracle VirtualBox (VirtualBox is used as the provider for Vagrant)
3. GitHub account (http://GitHub.com)
4. DockerHub account (http://hub.docker.com)
5. Oracle Container Cloud Services (optional)
6. Your preferred Text Editor.
In addition, the user should have some knowledge with
git
docker
basic Linux shell commands
What the session is not intended to cover
The session takes the user through a simple exercise to build a wercker pipeline. In the example we
make use of a variety of tools, such as vagrant, git or docker. The tutorial will NOT be covering the
use of these tools in detail.
Working with Vagrant
The hands on session leverages on Vagrant to provision a working environment for the hands on
session.
To help reduce the setup time, we’ve prepared a vagrant box that uses the VirtualBox provider. The
box can be downloaded at:
The box comes installed with the following:
1. Oracle Enterprise Linux 7.4
2. Wercker CLI
3. golang 1.9.4
4. git 1.9.3
The above toolsets is sufficient to get through the entire hands on session.
See Appendix A for some commonly used commands used by the tools.
Additional Notes and Observations about Vagrant
For our exercises, we use Vagrant with the VirtualBox provider. Before we get started, we
should ensure that we are able to get VirtualBox running. Especially for Windows machines.
Windows uses Hyper-V as the underlying hypervisor for Docker, and it conflicts with
2. VirtualBox directly. To use VirtualBox, users need to disable the Hyper-V service, and ensure
that the VT-X settings on the BIOS is enabled.
Getting started
Provision a container cloud instance
At the last few steps of the hands-on, we will try to build the wercker pipeline to provision to an
Oracle Container Cloud service. Before we start working on the exercise proper, we should provision
the service, as it may take some time for the service to start up. Use the following steps to create a
container cloud.
Note: Wercker does support pipelines to other container services, but for our purposes, we have
chosen to work with Oracle Container Cloud Service.
Hands On
The hands on involves building a simple wercker workflow that builds a simple golang code,
containerizes it, then uploads the container to a container repository, in our case, DockerHub.
Stage 1: Setup the working environment
Note:
Our Vagrantfile uses virtualbox as the provider. You need to ensure that you are able to run VirtualBox before starting these steps,
especially if you are working on a windows environment. VirtualBox is known to use technologies that conflict with Docker.
To fix these docker-virtualbox issues, you may need to ensure the following:
Docker services are stopped
Hyper-V services are stopped
VT-X options are enabled. You’ll need to changes the settings in your bios, and reboot.
Let’s work on getting the working environment up first. To do this step, you need to have Vagrant,
and VirtualBox installed, and a working GitHub account
1. Create your own GitHub repository, and import the codes from
https://github.com/darrelchia/wercker-meetup-exercise into your new repository.
2. Clone the repository onto your working machine. The repo contains a simple “Hello World”
golang program (main.go), a unit test file (main_test.go) and a Vagrantfile.
3. Now open up the Vagrantfile. It should look something like the text below.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
# -*- mode: ruby -*-
# vi: set ft=ruby :
# All Vagrant configuration is done below. The "2" in Vagrant.configure
# configures the configuration version (we support older styles for
# backwards compatibility). Please don't change it unless you know what
# you're doing.
Vagrant.configure("2") do |config|
Vagrant.configure("2") do |config|
config.vm.box = "darrelchia/wercker-meetup-go"
config.vm.box_version = "1.0.0"
# config.vm.box = "wercker-meetup-go"
# config.vm.box_url = "file://D:wercker-meetup-go.box"
config.vm.network "forwarded_port", guest: 9000, host: 9000, "id": "portainer-port"
3. 18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
config.vm.network "forwarded_port", guest: 5000, host: 5000, "id": "golang-port"
config.vm.network "forwarded_port", guest: 22, host: 2222, "id": "ssh"
# You may want to install the vagrant proxyconf plugin.
# At the command line, type : vagrant plugin install vagrant-proxyconf
if Vagrant.has_plugin?("vagrant-proxyconf")
config.proxy.http = ""
config.proxy.https = ""
config.proxy.no_proxy = ""
end
config.vm.synced_folder "D:Exercisewercker-meetup", "/home/vagrant/wercker-
exercise"
config.vm.box_check_update = false
# Example for VirtualBox:
#
config.vm.provider "virtualbox" do |vbox|
vbox.memory = 2048
vbox.cpus = 2
vbox.name = "Wercker-Meetup-Golang2"
end
end
4. If you have received a box image (either through a USB, or via a cloud download link and have
access to the .box file, do the steps below. If you have not, and want to download the Vagrant
Box from Vagrant Cloud, you may skip this step.
Comment out lines 11 config.vm.box and line 12 config.vm.box_version .
Uncomment out line 14 config.vm.box and line 15 config.vm.box_url Update the value
to point to the location of the downloaded box. Use the file:// syntax. If you are doing this
in windows, please use a double slash as the directory separator. (e.g. C:Program
Fileswercker-meetup-go.box”)
What these steps do, is to make sure vagrant uses the local box instead of download from
Vagrant Cloud.
5. Change the value of config.vm.synced_folder to point to the location where you have
cloned the GitHub repository. By doing this, we’re going to share the directory on our host
machine, with the vagrant box. This gives us the convenience of using a preferred editing tool on
our host machine.
For example, if we cloned our git repository to the directory D:ExerciseWercker-Meetup, or
config.vm.synced_folder in line 27 will look like this:
27 config.vm.synced_folder "D:ExerciseWercker-Meetup", "/home/vagrant/wercker-
exercise"
6. Startup the development environment from the command prompt using the following command
as an Administrator. This will import the box into VirtualBox and start up the environment.
vagrant up
4. Note: All vagrant commands should be run in the same directory as the Vagrantfile.
7. When the command has finished executing without any errors, you’re ready to start. Log into
the environment using the following command:
vagrant ssh
This opens up a ssh session to the running Vagrant Box. Note: You can open multiple ssh
sessions into the same box.
Phase 2: Building a simple wercker.yml file
For the second phase, we’re going to write a simple workflow to test and build our sample code. We
will start off with using the Wercker CLI tool on our local environment to make sure we get the
workflow right first.
8. Create an empty text based file called wercker.yml. This file should be located in your working
directory (the place where you cloned the repository.
Note
Wercker.yml uses the yml format. yml is very strict on formatting, and does not accept the <tab> character. For
indenting the code, we MUST use a double <space>. If you fail to do so, you will find that your wercker workflows may
fail.
9. In the wercker.yml file, add the following :
1
2
3
4
box:
id: golang
ports:
- "5000"
10. This defines the base docker image that we will be working with. For our example, we’re working
with golang, hence, we defined a box with the id: golang. When wercker runs, it will retrieve the
golang docker image from DockerHub.
11. Now, let’s add some pipelines. Add the following into the wercker.yml file.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
box:
id: golang
ports:
- "5000"
build:
# The steps that will be executed on build
steps:
# golint step!
- wercker/golint
# Test the project
- script:
name: go test
code: |
go test ./...
5. 17
18
19
20
21
22
23
24
25
26
27
28
29
30
# Build the project
- script:
name: go build
code: |
go build –o wercker-example main.go
dev:
steps:
- internal/watch:
code: |
go build –o wercker-example main.go
./wercker-example
reload: true
12. Save the file, and we can test it locally.
13. Go to our command prompt the vagrant ssh session, and execute the command wercker
build on the same location as the wercker.yml file. This executes the build pipeline defined
with the build: label (line 6).
14. The initial build will take some time, the Wercker CLI needs to download the containers to get it
to work.
15. Now, let’s run the dev: pipeline using the command: wercker dev -–expose-ports
16. Once you see that the pipeline is running, open a browser on the host machine, and access
http://localhost:5000. You should see the output of the go application “Hello World”. Also,
access http://localhost:5000/cities.json. You should also see a list of cities in a json format.
17. When using the internal/watch step, the CLI is able to watch for changes to the source code, and
rebuild and refresh the running code. You can try this by opening another vagrant ssh session.
Modify main.go:15 and adding one more city into the array. If you access
http://localhost:5000/cities.json again. You will find that the list of cities has been updated.
Note: This does not work if you are using a shared folder between the vagrant box and the host
machine. We’re not sure if it’s an issue with VirtualBox synced folders, or with Wercker.
18. Terminate the wercker dev session with a ctrl+c.
19. Now that we’re able to execute the wercker pipeline using the CLI, let’s bring it into wercker.
20. First let’s update the code changes into our repository. If you’re going to push the code from the
vagrant ssh session, you’ll need to configure git first. Run the following set of commands to
push the code back to GitHub.
git config –global user.name “<your name>”
git config –global user.email “<your email address>”
git add wercker.yml
git commit –m “Added wercker.yml”
git push
Once this is done, let’s go the wercker web console.
6. 21. Open a browser and log in to http://app.wercker.com. You’ll see a screen similar to the one
below.
Click on the “Create your first application” to create the application.
22.
23. In the Select user and SCM screen, select GitHub, and click Next.
If it shows “Not Connected”, you may need to link your GitHub account to Wercker first.
24. In the Select Repository page, select the repository that you have pushed our wercker.yml to,
and click Next.
25. In the Configure Access page, select “wercker will check out the code without using an SSH key”
and click Next.
26. Click on Create to create the application. You should see a popup saying the Application was
created successfully. Wercker will import your code from GitHub and bring you to a
configuration page.
7. On the page, click on the “I already have a wercker.yml, trigger a build now.
27. You’ll see your workflow running. (Hopefully successfully)
Stage 3: Adding a new pipeline to push to DockerHub
28. By now you will have realized that just building is not enough, we need to push the artifacts
somewhere. Let’s add one more step to push the build artifacts to a container repository.
29. Let’s get back to our wercker.yml file. Add the following pipeline into the wercker file to push
the built container onto DockerHub.
31
32
33
34
35
36
37
38
39
40
41
42
push-docker:
steps:
- internal/docker-push:
username: $DOCKER_USERNAME
password: $DOCKER_PASSWORD
repository: $DOCKER_USERNAME/wercker-sg-meetup
ports: "5000"
registry: https://registry.hub.docker.com/v2
tags: latest
cmd: “/pipeline/source/wercker-example”
30. Save and push the file into the GitHub repo. Pushing changes to GitHub automatically triggers
the wercker workflow to run. Wercker automatically creates a web hook that picks up code
commits and executes the build.
31. Let’s modify our workflow now. Go back to the browser and access the Workflows tab.
8. 32. In the editor, you should be able to see the build step. What we want to do is add a new
pipeline.
33. Click on the Add new pipeline button. Enter a name, and the pipeline name (as per what you
have created in the wercker.yml file. Click Create.
34. Then you will be asked to add pipeline environment variables. The pipeline environment
variables are what we define in the wercker.yml file using by prefixing the $ symbol (e.g.
$DOCKER_USERNAME). By adding the variables, here it means only this pipeline can use these
variables. If you wish to create workflow-wide variables, click on the Environment tab.
9. 35. Create the variables for DOCKER_USERNAME and DOCKER_PASSWORD. When you’re done, click
on the workflows tab at the top again.
36. Now, let’s add the pipeline into our workflow. In the Editor, click on the + icon after the build
step.
37. On the popup, enter the following details:
On branch(es) : *
No on branch(es): <blank>
Execute pipeline: push-docker
Click on the add button.
10. 38. Now, let’s trigger the build again.
39. We’ll see that the wercker workflow executes and successfully publishes the build artifacts to
DockerHub.
40. You may wish to pull the image down to test it. Our vagrant box has docker installed, so you can
pull it there to check that it’s working.
Stage 4: Complete it with Oracle Container Cloud
41. As a last step, we want to configure our container to run on a container cloud service. For our
exercise, we have chosen to work with Oracle Container cloud.
42. Before we start, we need to configure an OCCS service first. We can make use of wercker’s
custom steps, or using the script function to do this automatically, but for simplicity sake, we’re
going to do it manually.
This assumes that you already have an oracle container cloud provisioned. If you don’t you may
need to log in to provision it.
43. Log in to your Oracle Container Cloud Service.
11. 44. In the OCCS console, Click on the Services menu on the sidebar, and click the New Service
Button.
45. Let’s create a new service called wercker-example. Enter the location of the image to use for
this service. The image should be the same one that we have used in the previous stage.
12. Configure the exposed ports by clicking on the Ports checkbox under Available Options.
46. Click on the Add button from the Ports section. In the popup, expose the host port 5000 to the
container port 5000 over TCP. This is where our Hello World is running.
47. Click on the deploy button to start up the container service. This make take a minute or two to
work.
48. Click on the hosts menu on the sidebar and locate our service. Click on the hostname, and look
for the public_ip for our instance. We’ll need this information to access our example later.
13. 49. Now to work on our pipeline. Add the following pipeline to your wercker.yml file. We’re using a
custom step that interacts with the Oracle Container cloud service.
43
44
45
46
47
48
49
50
51
52
occs-restart:
steps:
# Manage Oracle Container Cloud Service container
- peternagy/oracle-occs-container-util:
occs_user: $OCCS_USERNAME
occs_password: $OCCS_PASSWORD
rest_server_url: $REST_SERVER_URL
function: stop
deployment_name: $DEPLOYMENT_NAME
50. Using the steps we’ve learnt in the earlier phases, do the following:
push the changes into the repository
log in to the wercker web console, and create the pipeline
remember to add the environment variables: The values map in the following way:
o $OCCS_USERNAME – The username to log into the OCCS instance. This is different
from your oracle cloud account name.
o $OCCS_PASSWORD – The password for the OCCS instance.
o $REST_SERVER_URL – This is the IP address to access your OCCS instance. See step
43. The format of this should be https://<OCCS IP address (e.g. https://129.10.11.12)
o $DEPLOYMENT_NAME – This is the name of the service we created in step 45.
51. Add the pipeline to our workflow and we’re done!
You can try going to main.go on our vagrant box and make some changes to the code and push it
back to our github repository, and you can watch the process deploy.
For this particular stage, what we’re doing is making our OCCS deployment restart. On restart, it will
go fetch the latest image from DockerHub and re-deploy it again, completing our CD workflow.
There are also different types of steps that you can choose to explore, e.g. deploy to a rolling router
for blue-green deployments, deploy to a K8s, just browse through the marketplace for interesting
steps that others have contribute.
This concludes the hands on.
14. Appendix A: Commonly used commands for the various tools
This section lists the commands and resources that we have used throughout this exercise. During
the hands on, please print out this section and distribute to the attendees.
Today’s WIFI Access:
SSID: clear-guest
Username: guest
Password:
GitHub Repository containing the sample codes:
https://github.com/darrelchia/oracledeveloper-apac-wercker.git
Vagrant
vagrant up Initializes the box, downloads and/or imports any images and starts up the
instance.
vagrant ssh Starts a SSH connection to our vagrant session
vagrant halt Stop (shutdown) the vagrant session.
vagrant destroy Cleanup and delete the vagrant session. The next time you run a vagrant
up, it will be a entirely new instance.
GIT
git config –global user.email <email address> This configures the email address of the using
git.
git config –global user.name <user name> This configures the username of the using git.
git add <file> This adds a file into the local repository.
git commit –m <commit message> This commits a file into the local repository
git push This pushes the changes from our local
repository to our remote one (GitHub)
Code Blocks
This section provides the code snippets used in the example
dev:
steps:
- internal/watch:
code: |
go build -o wercker_example main.go
./wercker_example
reload: true
# Build definition
build:
# The steps that will be executed on build
steps:
# golint step!. This re-formats the go code
- wercker/golint
# Run the go unit tests
15. - script:
name: go test
code: |
go test ./...
# Build the project
- script:
name: go build
code: |
go build -o wercker_example main.go
push:
steps:
# Push to public docker repo
- internal/docker-push:
ports: "5000"
username: $DOCKER_USERNAME
password: $DOCKER_PASSWORD
tag: latest
repository: $DOCKER_REPOSITORY
registry: https://index.docker.io/v2/
cmd: /pipeline/source/wercker_example
You can use the following OCC account:
Address: _________________________________________
Username: occ_admin
Password: ________________________________________
restart-occs:
steps:
# Manage Oracle Container Cloud Service container
- peternagy/oracle-occs-container-util:
occs_user: $OCCS_USER
occs_password: $OCCS_PASSWORD
rest_server_url: $REST_SERVER_URL
function: $FUNCTION
deployment_name: $DEPLOYMENT_NAME