Gitlab CI build on merge request and push to master

Introduction

In this post, I am going to show how to setup gitlab CI so that a build is triggered whenever a new feature or bug fix branch is opened for review (i.e. merge request, note: if you use Github, it is called a pull request). The build should also be triggered whenever new (ex. after review) commits are pushed to the merge request branch. In addition to merge requests, the build should be triggered whenever a commit is pushed to master (ex. when the merge request is accepted) so that master is always kept as a golden copy ready for release.

Installing Gitlab server on a Mac using Docker

Typically, you should have Gitlab account if your company uses Gitlab for source control. The reason why we are going to install Gitlab server locally is because it gives us flexibility to demonstrate all the concepts in this tutorial (ex. We do not need IT help to configure IP addresses or do port forwarding on firewalls).

Let us get started...

  • Download Docker for Mac. You can click the following link (let me know in the comments if the link is not working)
  • Install Docker by running the installer dmg file
  • Execute the following command for single container deployment. Note that you need to edit the command so that everything is on one line. In the snippet below, the command options are occupying separate lines for the purpose of reading clarity.
docker run --detach --name gitlab--hostname gitlab.example.com 
--publish 30080:30080 --publish 30022:22 -env GITLAB_OMNIBUS_CONFIG=
"external_url 'http://gitlab.example.com:30080';
gitlab_rails['gitlab_shell_ssh_port']=30022;" gitlab/gitlab-ce:9.1.0-ce.0
  • We are going to use few Docker commands but note that this article is not a Docker tutorial, for more information about Docker and Docker compose, you may refer to docker.com
  • Type: docker images you should see the following image listed
REPOSITORY        TAG         IMAGE ID      CREATED       SIZE
gitlab/gitlab-ce  9.1.0-ce.0  d488a27e1c94  7 months ago  1.1GB
  • If Docker instance is stopped for any reason, you can start it by typing: docker start gitlab
  • To make sure it is running, you can type: docker ps
CONTAINER ID        IMAGE                         ...   NAMES
dc23553b8065        gitlab/gitlab-ce:9.1.0-ce.0   ...   gitlab
  • To get a shell in the running container you can type: docker exec -it gitlab bash
  • To access the container by host name, edit the following file:
//vi /etc/hosts
127.0.0.1       gitlab.example.com
  • In order for the container to be able to reach the host (Mac machine), execute the following command:

ifconfig lo0 alias 169.254.254.254
  • Now launch your browser and point to:

http://gitlab.example.com:30080/

  • Click New project, give it a name then click Create project. In the next page, follow the instructions to add files to the repository

Install Gitlab CI Runner

  • Download the binary to some directory

curl --output /usr/local/bin/gitlab-runner https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-darwin-amd64
  • Make it executable

  chmod +x /usr/local/bin/gitlab-runner
  • Install/Register the runner

gitlab-runner install
gitlab-runner start

  • The installer will ask some questions. You can get the needed information by opening the gitlab project that we just created, then go to settings, CI/CI pipelines, Specific Runners
  • To make sure the runner is installed and registered successfully, under Specific Runners, make sure your runner is listed there

Configuring gitlab CI

  • Configure .gitlab-ci.yml which is the file used by GitLab Runner to manage project's jobs. To do that, you can navigate to the file directly or use the menu by clicking on Project then CI configuration

  • For more information about .gitlab-ci.yml, you can read gitlab documentation
  • In the example below, we have a build pipe line that consists from 4 stages (clean, test, build, deploy). For each stage, the script specifies what needs to be done, the "only" keyword decides when to trigger a build. For example, master here means trigger the build when commits are pushed to master, triggers means trigger the build when a gitlab trigger is (well) triggered. Will talk about gitlab triggers later. The strategy variable is used to disable cloning the repo between stages

#-------------------------------------------------------------------------------
# Our project CI pipeline consists from the following stages
stages:
  - clean
  - test
  - build
  - deploy
#-------------------------------------------------------------------------------
Clean or setup:
  script:
    - cp file_01.json $CI_PROJECT_DIR
    - cp file_02.py $CI_PROJECT_DIR/app
    - ./gradle clean
  stage: clean
  only:
    - master
    - triggers
#-------------------------------------------------------------------------------
Run Tests:
  variables:
    GIT_STRATEGY: none
  script:
    - ./gradle test
  stage: test
  only:
    - master
    - triggers
#-------------------------------------------------------------------------------
Build Project:
  variables:
    GIT_STRATEGY: none
  script:
    - ./gradle build
  stage: build
  only:
    - master
    - triggers
#-------------------------------------------------------------------------------
Deploy app:
  variables:
    GIT_STRATEGY: none
  script:
    - ./gradle deploy
  stage: deploy
  only:
    - master
    - triggers
#-------------------------------------------------------------------------------
  • To test: clone your repo using the following command (if you have not done that)

git clone ssh://git@gitlab.example.com:30022/username/test_project.git
  • Go to the cloned repo, create a new file, "git add ." then "git commit -m" then "git push"
  • If everything is ok, build pipeline for master branch should run. To check that, open your project then go to Pipelines
  • You can even click each stage and see the progress as it happens

How to build on merge request

Gitlab CI does not support building a branch on merge request out of the box, however there is a trick to do that. Follow the steps below:

  • Gitlab supports triggering a job programmatically using triggers (that is why we used triggers when we specified the "only" keyword in the yml file). The trick is to call the trigger from an external service (a web hook). Gitlab supports web hooks for the sake of integrating with 3rd party tools on events like merge requests so we can exploit that and solve our problem. Here is how to do that in details
  • Open your project, go to settings, go to CI/CD pipelines, scroll down to triggers section then click Add trigger

  • Go to settings, integrations. Make sure "Merge Requests events" checkbox is checked. Type the URL of the web hook service, in our case it should be http://169.254.254.254:8080/hook
  • We are going to use an open source web hook (written in golang) that exposes the /hook end point

  • If you scroll down, there is a button to create a web hook. Click on Add webhook

  • Now let us setup the web hook before we click test and see if the connection is ok
  • Go to gitlab-ci-build-on-merge-request and download that script
  • Make sure golang for mac is installed, if not you can get it from here
  • To launch the webhook on your mac, type the following command:
	go run gitlab-ci-build-on-merge-request.go --url=http://gitlab.example.com:30080 —private_token=Your private token
  • To get your private token, go to your user settings, under account

  • Note: the golang script mentioned above did not work for me right the way, I had to fix few lines in the script (let me know if you want my script)
  • How to test that ? create a new branch, add some files, add, commit and push
  • Go to gitlab, open your project, you will see the new branch and the option to create a merge request. Click Create merge request

  • In the next page, just give it a name and description then click submit
  • Your merge request branch should be automatically built by CI. Like we did for master, you can go to the pipelines section and see the build in progress
  • If the build is not triggered, take a look at the golang script. It should be listening on port 8080. When the merge request is opened, the golang script should get some payload from gitlab. Take a look at that, it should look normal without errors

  • Similarly, you can open the project, go to pipelines and see the merge request build pipeline running in progress

References

That is it. Thanks for reading. Please use the comments section below for questions and feedback.

Tags:

Leave a Reply