Cloud Build Demystified
[Cover Image: “adam wallride” by Rob React is licensed under CC BY 2.0]
Skate or die
Do you remember learning to ride a bicycle or a skateboard? I remember as a kid in the 80s giving skating a try when it became popular (again). Board? Check. Pads? Check. Vans? Check. Helmet? Nah, come on, it was the 80s. Skill? Please. And I remember those first few times out on the board being pretty awkward (and painful). But I persevered and after a few months at it I could pass as semi-cool in the neighborhood.
When I first encountered Google Cloud Build I kind of felt the same way. What’s this thing that runs build steps in containers in the cloud? What’s it designed to do? What can I use it for? I’ve been playing with Cloud Build for a while now and I am hoping this helps you bootstrap quickly. Please feel free to jump in with your own pointers in the comments.
Managed
Cloud Build, as the name suggests, is a cloud service offered by Google Cloud. You don’t run or maintain a “server”; even the “runners” are in the cloud.
Containers → Swiss Army Knife
The first thing you have to get your head around is that Cloud Build runs everything in containers. And you define a “pipeline” as a series of steps where each step runs in its own container. There is a common workspace which persists between steps so that you can pass data/artifacts between steps.
I use the term “pipeline” loosely because it’s really a sequence of steps to do… sort of anything you want. A survey of Cloud Build use cases is beyond the scope here but think Swiss Army knife DevOps automation platform instead of strictly just CI/CD pipelines.
Cloud Builders
But what are these container steps? We’ll go through a few examples below, but for now just know that the containers used in build steps can be purpose-built for the task at hand like building a binary (e.g. Maven, Go, C#, NPM), creating a docker container, or performing a task or command (e.g., bash, gcloud CLI, terraform).
In fact, Google provides some common container starting points, called Cloud Builders (see what they did there?). There is also a community contributed collection of Cloud Builders. Which of course means you can build your own, too. Basically, if it runs in a container, you can run it with Cloud Build as part of some DevOps automation. I think of this as a form of extensibility.
Simple config
Here’s a simple example of the declarative yaml configuration file used by Cloud Build:
|
|
The highlighted line (3) indicates that this is a collection of steps. The name
label refers to a container image, which is by default in docker hub, or explicitly referenced like the gcr.io urls. Notice that some steps specify an entrypoint
which is a way to call a specific command available to that container.
- The first two steps load our app files and dependencies into a python-alpine docker container and then run
unit_tests.py
- If those pass, then we build the image we intend to deploy in step 3, this time using a docker builder container and Dockerfile.
- We then push to gcr.io registry in step 4 (line 16)
- Finally we use the cloud-sdk Cloud Builder image, which has Google Cloud command line loaded into it, to deploy to Cloud Run (line 19). Notice the
entrypoint
gcloud is specified
Here’s what a unit test failure looks like:
And for my next trick
Here’s another example my colleague Dave Stanke pulled together.
|
|
So what’s different here? First, we’re using Cloud Build here to lay down some terraform using the community contributed terraform Cloud Builder, which we have pulled into our project (and perhaps customized for our purposes).
Also an id
attribute surfaces, which is referenced by waitFor
. Until now every step progressed sequentially, but waitFor
provides some control over that. If an explicit step is referenced (like line 12), the current step “waits for” the referenced step. But it also provides a way to run steps in parallel; in line 21 the -
means “run at the same time as the previous step”. This makes sense because there’s no reason we can’t push our image while terraform init runs.
Old dog, new tricks
So that’s a wrap for my first look at Cloud Build. I have to say I’m intrigued by the possibilities and this old dog is learning some new tricks. I’m curious what others are doing with Cloud Build, so leave me your thoughts - good, bad, or ugly. Much more to follow in this space.