Continuous Integration

In this post, we describe how we got our continuous integration pipeline running on GitLab. To get started, we first need to install gitlab-runner, which on Linux is done with the following steps:

# Add GitLab's official repository
curl -L https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh | sudo bash

sudo apt install gitlab-runner

Once gitlab-runner is installed, we can create our first runner, which is basically a server that will run the jobs corresponding to the stages of the pipeline. In GitLab CI parlance, a pipeline is made up of a succession of stages, e.g. a build stage followed by a deployment stage. Strictly speaking, these stages do not have to be consecutive: as long as there are no dependencies between them, the runner can run the jobs in parallel. One can get really fancy and define a whole dependency graph apparently. Runners are characterized by two main properties: whether they are specific to a project or shared, and what type of executor they use. We went for the simplest option, and created a runner specific to the Sempervirens project with a shell executor. The latter means that we will execute our builds on our local machine where all the required dependencies are installed. To create the runner, we need to use the register functionality of gitlab-runner:

sudo gitlab-runner register

Once prompted for the token associated to the runner, we entered the token that was generated for the Sempervirens project, and that can be found under the Settings/CI/CD tab in the Runners section. To check that we registered our runner properly we can use either of:

sudo gitlab-runner list 
sudo gitlab-runner verify

The first one simply lists the runners currently set up, while the second checks if the runner is alive. We can also check that the runner has an active status in our projects Settings/CI/CD tab.

We wanted to set up a simple pipeline that assures first, that each time we push something on the repo, the whole code is built (i.e. a build stage), and second, that a suite of tests is run along (i.e. a test stage). To set that up, the first thing we need is a .gitlab-ci.yml file that defines these two stages:

stages:
  - build-Debug-Linux
  - test

build-Debug-Linux:
  stage: build-Debug-Linux
  script:
    - rm -rf build-Debug-Linux  
    - mkdir build-Debug-Linux && cd build-Debug-Linux
    - cmake -G "Unix Makefiles" -DCMAKE_BUILD_TYPE="Debug" ../
    - cmake --build . --config Debug --parallel 8
  artifacts:
    untracked: true
    paths:
      - build-Debug-Linux/

test:
  stage: test
  dependencies:
    - build-Debug-Linux
  script:
    - cd build-Debug-Linux
    - ctest -V -N -E 'test-window'

This file is to be pushed in the root directory of the project.

Our second stage calls ctest to run a suite of unit tests. As ctest needs to be called from the build directory of the project, the latter needs to be passed along from the first to the second stage of our pipeline. This would be no problem if that directory was tracked by git, but because it is not, it will be removed between the stages when the project repository is checked out again in the working directory of the runner. To avoid that, we create what is called an artifact to declare the relative path of the build directory, and we declare the build stage as a dependency of the test stage. That way the build directory will be uploaded to the server at the end of the build stage, and downloaded again at the beginning of the test stage.

One can start/stop the server with:

sudo gitlab-runner start/stop

It can be useful to test stages of the pipeline locally. For example to test the build stage:

gitlab-runner exec shell build-Debug-Linux

Finally, to delete a runner, one has to remove it from the project’s Settings/CI/CD tab. The runner will still be registered after that. To get rid of it completely one needs to use:

sudo gitlab-runner unregister -t 'token' -u 'URL'

where ‘token’ and ‘URL’ stand for the corresponding fields that can be seen when listing the runners currently set up (with gitlab-runner list).