Publish Binary Releases on Gitlab

I switched to Gitlab when Github was bought by Microsoft. Call me extremist, stupid, and all the names you like, I personally felt the need to do it.
And quite honestly, I’m pretty happy with that choice, Gitlab is an incredibly useful suite, we use the community version at work and would never go back. But there’s one thing that’s pretty annoying with Gitlab, their documentation organization. Honestly it’s like they don’t want you to figure out how to do things. It’s often split into tens of various links and you end up reading issues discussions without really knowing if the feature you’d like is implemented or just an idea at that point.

This happened to me when trying to automate binary releases of goxplorer, a Bitcoin blockchain explorer / parser I began writing some months ago.

What I wanted to achieve was pretty simple, whenever a tag is set, produce by-OS binaries and make them publicly available via the release page of the project. Turns out it is possible, but it took me an insane amount of time to put the pieces together. But hey, being the good guy I am, I’ll give them to you just like that.

The first thing I learned is that building Go binaries for any platform is the easiest thing on earth:

GOOS=netbsd GOARCH=amd64 go build

Isn’t it the most beautiful thing? Here’s the corresponding Makefile target:

PROGNAME=goxplorer
OUTPUT=${PROGNAME}_${CI_COMMIT_TAG}_${GOOS}_${GOARCH}

# other targets

cross:
	go build -o ${OUTPUT}
	sha256sum ${OUTPUT} >${OUTPUT}.sha256

And the release stage in .gitlab-ci.yml is like this:

before_script:
  - export GOPATH=${CI_PROJECT_DIR}/.cache
  - export GOARCH=amd64
  - export OSS="linux freebsd netbsd darwin"
  - apt-get update -qq && apt-get -y -qq install jq xxd

# build and test stages, unrelated to this article

release:
  stage: release
  script:
    - for os in ${OSS}; do GOOS=$os GOARCH=${GOARCH} make cross; done
    - apt-get -qq -y install curl
    - /bin/sh mkrel.sh goxplorer
  artifacts:
    paths:
      - goxplorer_*
  only:
    - tags

As you can see, I wrote a loop that will build binaries for common platforms (if you want more PR please!). Those binaries need to be kept in order to make them downloadable, and that’s the job of the artifact section, where I specify that files which path is goxplorer_* must be kept.

Now about the release page, my understanding is that as of today, it must be created via the HTTPS API, using the /api/v4/ path, and a private token that you’ll create in your profile’s settings. Note that this access token must be granted API and write_repository permissions:

Personal Access Tokens

And then declare it in your project’s CI/CD Settings:

CI/CD Settings

In order to keep things as clean as possible, I wrote a separate script to build the JSON data needed for the release creation along with the curl call to Gitlab‘s API:

#!/bin/sh

progname=$1

baseurl="https://gitlab.com/api/v4/projects/${CI_PROJECT_ID}"

for os in ${OSS}
do
	fullname="${progname}_${CI_COMMIT_TAG}_${os}_${GOARCH}"
	linkname="${fullname}\ (SHA256 $(cut -f1 -d' ' ${fullname}.sha256))"
	linkurl="${baseurl}/jobs/${CI_JOB_ID}/artifacts/${fullname}"
	linklist="${linklist}{\"name\": \"${linkname}\", \"url\": \"${linkurl}\"}"
done

links="[$(echo ${linklist}|sed 's/}{/}, {/g')]"

descr="$(curl -H \"PRIVATE-TOKEN:\ ${PRIVATE_TOKEN}\" ${baseurl}/repository/tags/${CI_COMMIT_TAG}|jq -r '.message')"

DATA="
{
  \"name\": \"${progname} version ${CI_COMMIT_TAG}\",
  \"description\": \"${descr}\",
  \"tag_name\": \"${CI_COMMIT_TAG}\",
  \"assets\": {
    \"links\": "${links}"
  }
}
"
curl -H 'Content-Type: application/json' -X POST -H "PRIVATE-TOKEN: ${PRIVATE_TOKEN}" "${baseurl}/releases" -d "${DATA}"

Here I use:

  • ${CI_COMMIT_TAG}, a variable set by Gitlab with the tag being released
  • ${OSS} and ${GOARCH} which I declared in .gitlab-ci.yml
  • ${CI_PROJECT_ID}, another variable set by Gitlab containing the project ID
  • ${CI_JOB_ID}, again, a CI variable with the current job ID

Those are used to build the URLs to the previously generated artifacts, our binaries. The release description will be taken from the annotated commit message via a query to /repository/tags/${CI_COMMIT_TAG}.

With all this put together, when creating a tag and pushing it

$ git tag -a v0.3.1-test -m "a test release"
$ git push --tags

Gitlab will trigger the creation of the new release and update the Release page accordingly.

If you are trying this and fail like I did many times, you can clean your previously created release with curl alone:

$ curl -X DELETE -H "PRIVATE-TOKEN: ${PRIVATE_TOKEN}" "https://gitlab.com/api/v4/projects/15030361/releases/v0.3.1-test"

When in trial and error phase you might also want to clean up your tags locally and remotely:

# delete local tag
$ git tag -d 0.3.1-test
# delete remote tag
$ git push --delete origin 0.3.1-test