vsz
vsz
Tire Kicker
Jul 6, 2021 5 min read

Cloud Build Demystified

thumbnail for this post

[Cover Image: “adam wallride” by Rob React is licensed under CC BY 2.0]

Skate or die

Do you remember learning to ride a bicycle or a skateboard? I remember as a kid in the 80s giving skating a try when it became popular (again). Board? Check. Pads? Check. Vans? Check. Helmet? Nah, come on, it was the 80s. Skill? Please. And I remember those first few times out on the board being pretty awkward (and painful). But I persevered and after a few months at it I could pass as semi-cool in the neighborhood.

When I first encountered Google Cloud Build I kind of felt the same way. What’s this thing that runs build steps in containers in the cloud? What’s it designed to do? What can I use it for? I’ve been playing with Cloud Build for a while now and I am hoping this helps you bootstrap quickly. Please feel free to jump in with your own pointers in the comments.

Managed

Cloud Build, as the name suggests, is a cloud service offered by Google Cloud. You don’t run or maintain a “server”; even the “runners” are in the cloud.

Containers → Swiss Army Knife

The first thing you have to get your head around is that Cloud Build runs everything in containers. And you define a “pipeline” as a series of steps where each step runs in its own container. There is a common workspace which persists between steps so that you can pass data/artifacts between steps.

Cloud Build workflow

I use the term “pipeline” loosely because it’s really a sequence of steps to do… sort of anything you want. A survey of Cloud Build use cases is beyond the scope here but think Swiss Army knife DevOps automation platform instead of strictly just CI/CD pipelines.

Cloud Builders

But what are these container steps? We’ll go through a few examples below, but for now just know that the containers used in build steps can be purpose-built for the task at hand like building a binary (e.g. Maven, Go, C#, NPM), creating a docker container, or performing a task or command (e.g., bash, gcloud CLI, terraform).

In fact, Google provides some common container starting points, called Cloud Builders (see what they did there?). There is also a community contributed collection of Cloud Builders. Which of course means you can build your own, too. Basically, if it runs in a container, you can run it with Cloud Build as part of some DevOps automation. I think of this as a form of extensibility.

Community Builders

Simple config

Here’s a simple example of the declarative yaml configuration file used by Cloud Build:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
# To build and deploy to Cloud Run, run from this directory:
# $ gcloud builds submit . --config=cr-cloudbuild.yaml
steps:
# Install dependencies
- name: python:3.8-alpine
  entrypoint: pip
  args: ["install", "-r", "requirements.txt", "--user"]
# Run unit tests
- name: python:3.8-alpine
  entrypoint: python
  args: ["unit_tests.py"]
# Build the container image
- name: 'gcr.io/cloud-builders/docker'
  args: ['build', '.', '-t', 'gcr.io/$PROJECT_ID/pop-stats', '-f', 'docker/pythonalpine.Dockerfile']
# Push the container image to Container Registry
- name: 'gcr.io/cloud-builders/docker'
  args: ['push', 'gcr.io/$PROJECT_ID/pop-stats']
# Deploy container image to Cloud Run
- name: 'gcr.io/google.com/cloudsdktool/cloud-sdk'
  entrypoint: gcloud
  args: ['run', 'deploy', 'pop-stats', '--image', 'gcr.io/$PROJECT_ID/pop-stats', '--region', 'us-west1', '--platform', 'managed', '--allow-unauthenticated']
Github link

The highlighted line (3) indicates that this is a collection of steps. The name label refers to a container image, which is by default in docker hub, or explicitly referenced like the gcr.io urls. Notice that some steps specify an entrypoint which is a way to call a specific command available to that container.

  • The first two steps load our app files and dependencies into a python-alpine docker container and then run unit_tests.py
  • If those pass, then we build the image we intend to deploy in step 3, this time using a docker builder container and Dockerfile.
  • We then push to gcr.io registry in step 4 (line 16)
  • Finally we use the cloud-sdk Cloud Builder image, which has Google Cloud command line loaded into it, to deploy to Cloud Run (line 19). Notice the entrypoint gcloud is specified

Here’s what a unit test failure looks like:

And for my next trick

Here’s another example my colleague Dave Stanke pulled together.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
steps:
- name: 'gcr.io/cloud-builders/docker'
  id: docker build
  args: ['build', '-t', 'gcr.io/$PROJECT_ID/us-covid-stats', '.']
  dir: 'app'
  waitFor: ['-']

- name: 'gcr.io/cloud-builders/docker'
  id: docker push
  args: ['push', 'gcr.io/$PROJECT_ID/us-covid-stats']
  dir: 'app'
  waitFor: ['docker build']

- name: 'gcr.io/$PROJECT_ID/terraform'
  id: terraform init
  args: [
    'init',
    '-backend-config=bucket=$PROJECT_ID-tf-state',
    '-backend-config=prefix=terraform/state'
  ]
  waitFor: ['-']

- name: 'gcr.io/$PROJECT_ID/terraform'
  id: terraform apply
  args: [
    'apply', '--auto-approve',
    '-var=google_project_id=$PROJECT_ID',
    '-var=google_region=us-east1'
  ]
  waitFor: ['docker push','terraform init']
Github link

So what’s different here? First, we’re using Cloud Build here to lay down some terraform using the community contributed terraform Cloud Builder, which we have pulled into our project (and perhaps customized for our purposes).

Also an id attribute surfaces, which is referenced by waitFor. Until now every step progressed sequentially, but waitFor provides some control over that. If an explicit step is referenced (like line 12), the current step “waits for” the referenced step. But it also provides a way to run steps in parallel; in line 21 the - means “run at the same time as the previous step”. This makes sense because there’s no reason we can’t push our image while terraform init runs.

Old dog, new tricks

So that’s a wrap for my first look at Cloud Build. I have to say I’m intrigued by the possibilities and this old dog is learning some new tricks. I’m curious what others are doing with Cloud Build, so leave me your thoughts - good, bad, or ugly. Much more to follow in this space.

comments powered by Disqus