How to Configure Jenkins with Google Cloud Platform (GCP)

Our colleague João Ferreira, Site Reliability Engineer of dareCode, guides us through this post on how to configure Jenkins with Google Cloud Platform (GCP).

In this article we will cover the basics on configuring Jenkins to use GCP to create agents on-demand.

We will start by covering the requirements to fulfil this setup and then move to how to configure a GCP service account.

After that comes the Jenkins configuration which I will start by the plugin installation, then the creation of the service account credentials and the cloud configuration.

At the end there’s a small test job to verify the configuration.

Requirements for setting up Jenkins with Google Cloud Platform

The following resources are required in order to set this up:
  1. Account on a Jenkins instance*: With administrator permissions.
  2. Account on Google Cloud Platform: Capable of using Compute Engine and create service accounts.
  3. Optional: gcloud command-line tool.

*This instance can be in any cloud or in on-premise. For readers with less experience we recommend  this article which describe the process of creating a brand new Jenkins instance.

Do you need help with the implementation of DevOps on your teams?

Discover all our DevOps World services to help you to achieve all your goals and evolve towards the software organization of the future.

Creating a Google Cloud Platform service account

This first thing to do is to create a service account in GCP.

This account will be used by Jenkins to communicate with GCP, whenever is necessary to create an agent.

This account will have the following roles:

  • instanceAdmin
  • networkAdmin
  • ServiceAccountUser

Using gcloud command-line tool or cloud shell this is easily achieved by running the following commands.

In this example we are going to use a cloud shell and the steps are as follows:

1. Create the service account:

gcloud iam service-accounts create jenkins-gce

2. Assign the required roles to the service account:

export PROJECT=$(gcloud info --format='value(config.project)') 
export SA_EMAIL=$(gcloud iam service-accounts list --filter="name:jenkins-gce" \
 --format='value(email)') 
gcloud projects add-iam-policy-binding --member serviceAccount:$SA_EMAIL \
 --role roles/compute.instanceAdmin $PROJECT
gcloud projects add-iam-policy-binding --member serviceAccount:$SA_EMAIL \
 --role roles/compute.networkAdmin $PROJECT
gcloud projects add-iam-policy-binding --member serviceAccount:$SA_EMAIL \
 --role roles/iam.serviceAccountUser $PROJECT
 gcloud projects get-iam-policy $PROJECT

You should see:

- members:
  - serviceAccount:jenkins-gce@$PROJECT.iam.gserviceaccount.com
  role: roles/compute.instanceAdmin
- members:
  - serviceAccount:jenkins-gce@$PROJECT.iam.gserviceaccount.com
  role: roles/compute.networkAdmin
- members:
  - serviceAccount:jenkins-gce@$PROJECT.iam.gserviceaccount.com
  - user:<user>
  role: roles/iam.serviceAccountUser

3. Grab the JSON service account key:

gcloud iam service-accounts keys create --iam-account $SA_EMAIL jenkins-gce.json

If you are using cloud shell, use the following command to download the file:

cloudshell download jenkins-gce.json

Using this service account Jenkins will be able to manage all the resources required to create agents on-demand.

Installing Google Compute Engine (GCE) plugin

The plugin required for this setup is called Google Compute Engine which is responsible to communicate with GCP, create the resources and configure the agent on those resources.

Go to Manage Jenkins > Manage Plugins > Available and look for “Google Compute Engine” as we show you in the following image:

instalacion plugin google compute engine gce

Service account credentials setup

In order for Jenkins to authenticate against GCP it’s required to add the service account key to the Credentials section in Jenkins.

Go to Manage Jenkins > Manage Credentials > Global > Add Credentials of kind “Google Service Account from private key”.

Set your project name and upload the json file previously downloaded.

Configurar credenciales de la service account para google cloud platform y jenkins

Jenkins Cloud Setup

Now that we have all the requirements for Jenkins to communicate with GCP and create resources, we need to configure how the agents are created and of what type.

Go to Manage Jenkins > Manage Nodes and Clouds > Configure Clouds > Add a new cloud > Google Compute Engine and cover the fields with your information.

The Service Account Credentials should be the one created in the previous step and you can use an instance template or configure the instance directly in Jenkins.

Note: The instance requires java8 installed and on the default path. Create a VM, install Java8 and create a new OS image in your GCP project. Packer can also be used to achieve this. More info here.

This is a example of a Jenkins Cloud setup:

Example in Configuration as Code, replace PROJECT and REGION with the ones you are using:

clouds:
- computeEngine:
    cloudName: "GCP"
    configurations:
    - bootDiskAutoDelete: true
      bootDiskSizeGb: 10
      bootDiskSizeGbStr: "10"
      bootDiskType: "https://www.googleapis.com/compute/v1/projects/$PROJECT/zones/$REGION/diskTypes/pd-balanced"
      description: "CICD"
      externalAddress: true
      javaExecPath: "java"
      labelSet:
      - name: "linux"
      labelString: "linux"
      labels: "linux"
      launchTimeoutSeconds: 300
      launchTimeoutSecondsStr: "300"
      mode: EXCLUSIVE
      namePrefix: "cicd"
      numExecutors: 1
      numExecutorsStr: "1"
      region: "https://www.googleapis.com/compute/v1/projects/$PROJECT/regions/europe-west1"
      retentionTimeMinutes: 6
      retentionTimeMinutesStr: "6"
      runAsUser: "jenkins"
      serviceAccountEmail: "jenkins-gce@$PROJECT.iam.gserviceaccount.com"
      template: "https://www.googleapis.com/compute/v1/projects/$PROJECT/global/instanceTemplates/jenkins-template-1"
      zone: "https://www.googleapis.com/compute/v1/projects/$PROJECT/zones/$REGION"
    credentialsId: "$PROJECT"
    instanceId: ""
    projectId: "$PROJECT"

Setting up a test job

We are going to configure an small test job to verify that Jenkins is capable of creating agents on demand in GCP as well as configure the agents properly.

1. Create a new Freestyle Project and called “Test GCP agents”, for example:

2. Restrict where the job can be run and select the label for GCP agents (gcp in this example):

restringir donde se puede correr el job para configurar jenkins gcp

3. In build section select “Execute shell” and add this small script to check the agent IP and hostname.

#!/bin/bash
echo "This is my IP"
curl -s ifconfig.co
echo "This is my hostname"
hostname -f

It is as shown in the image:

4. Run the job and verify the output:

You should be able to see the new VM in the GCP console and the job will output information about the instance which is also available on the console.

With this test we verify that everything works correctly.

Final remarks

Configuring Jenkins to use agents on demand has a lot of advantages, both on the technical and financial side.

Different type of agents can be created depending on the job requirements and, since the agents are ephemeral, the cost is reduced to the minimal required to run the job.

The configuration is simple, the complex task of create new agents is abstracted and can be achieved by a simple click or trigger.

Do you want to ask us something?

Contact us completely free of charge and without obligation for any kind of question, our team is looking forward to hearing from you!