We are running our Gitlab CI infrastructure on top of OpenStack . To not use a fixed number of VMs, we use Gitlab CI with docker-machine to create VMs as needed by the build jobs. This blog post will describe how to enable docker-machine to properly talk to the keystone v3 API in this setup.
The old configuration looked something like this:
[[runners]]
name = "Your Runner”
url = "https://your.gitlab.url/"
token = "your-gitlab-ci-runner-token"
executor = "docker+machine"
[runners.machine]
MachineDriver = "openstack"
MachineOptions = [
"openstack-auth-url=https://your.keystone.url:5000/v3",
"openstack-flavor-id=build-vm-flavor",
"openstack-image-name=build-vm-image",
"openstack-tenant-name=your-openstack-project",
"openstack-domain-name=Default",
"openstack-net-id=your-ci-project-network",
"openstack-sec-groups=your-sec-groups",
"openstack-username=openstack-username",
"openstack-password=openstack-password",
"openstack-keypair-name=your-ssh-keypair",
"openstack-private-key-file=your-ssh-private-key-file",
"openstack-ssh-user=your-ssh-user"
The above setup uses the keystone v2 API wording and also connects to the keystone v2 API. But starting with OpenStack Version Queens, keystone v2 API is removed and you have to use the v3 API. The problem is: docker-machine does not provide any config parameters to use the v3 API. The version number in the auth-url is not sufficient.
But docker-machine respects environment variables for the OpenStack client used under the hood. Luckily for us, Gitlab gives us the possibility to provide environment variables to the docker-machine process.
We had to change the configuration in two sections:
[[runners]]
environment = ["OS_PROJECT_NAME=your-openstack-project","OS_IDENTITY_API_VERSION=3","OS_USER_DOMAIN_NAME=Default"]
[runners.machine]
MachineDriver = "openstack"
MachineOptions = [
"openstack-auth-url=https://your.keystone.url:5000/v3",
"openstack-flavor-id=build-vm-flavor",
"openstack-image-name=build-vm-image",
"openstack-tenant-id=your-openstack-project-id",
"openstack-domain-name=Default",
"openstack-net-id=your-ci-project-network",
"openstack-sec-groups=your-sec-groups",
"openstack-username=openstack-username",
"openstack-password=openstack-password",
"openstack-keypair-name=your-ssh-keypair",
"openstack-private-key-file=your-ssh-private-key-file",
"openstack-ssh-user=your-ssh-user"
We previously used the --openstack-tenant-name parameter. With this parameter, docker-machine tries to call the removed apis to resolve the tenant name to its id . By using the --openstack-tenant-id parameter the code doesn’t get executed.
We have to confess, we learned it the hard way. We found the problem because of our recent upgrade from OpenStack Pike to Queens. Our Gitlab CI stopped working with this error message in the gitlab-runner logs:
Nov 21 07:31:07 master-scheduler gitlab-runner[20426]: ERROR: Error creating machine: Error in driver during machine creation: Expected HTTP response code [200 204] when accessing [GET https:://your.keystone.url:5000/v2.0/tenants], but got 404 instead driver=openstack name=runner-b099e1bc-master-privileged-1542787257-7b239e1a operation=create Nov 21 07:31:07 master-scheduler gitlab-runner[20426]: ERROR: {"error": {"message": "(https://your.keystone.url:5000/v2.0/tenants): The resource could not be found.", "code": 404, "title": "Not Found"}} driver=openstack name=runner-b099e1bc-master-privileged-1542787257-7b239e1a operation=create
We hope this type of posts helps other people save some time by learning from our experiences.
Was this post helpful?
More articles
fromChristian Zunker
9.8.2021 Your job at codecentric?
Jobs
Agile Developer und Consultant (w/d/m)
Alle Standorte
Gemeinsam bessere Projekte umsetzen.
Wir helfen deinem Unternehmen.
Du stehst vor einer großen IT-Herausforderung? Wir sorgen für eine maßgeschneiderte Unterstützung. Informiere dich jetzt.
Hilf uns, noch besser zu werden.
Wir sind immer auf der Suche nach neuen Talenten. Auch für dich ist die passende Stelle dabei.
Blog author
Christian Zunker
Do you still have questions? Just send me a message.
Do you still have questions? Just send me a message.