[Image attribution: Dan Maharry ]

Last time, we looked at Google Cloud Code’s “Run as Kubernetes” - aka Skaffold - as an elegant way to develop and iterate a Kubernetes application locally. We saw how Skaffold configuration could be used to abstract the build and local deployment process. But we were left with aching questions like:

• What if we need to build and deploy into multiple, differing environments?
• Can this Skaffold thing be used by my other team members to onboard quickly?

Oh, the suspense. But first a quick story about cars again. Back when I was learning to fix my 80s Land Cruiser, I got the crazy idea that I could fix modern cars. I had a bunch of tools I bought for the Cruiser and, well why not, say, try to replace the water pump on an ‘08 BMW? I was bordering on arrogant as I cracked the hood, trying to understand the insanely complicated serpentine belt and idler system. Things were easier on the Cruiser because it was far easier to access everything, but also because it was just mechanically… simpler. In over my head, I soon realized that I didn’t have the right tools, either. It turns out BMWs use the torx ratchet systems and all I had were my old school hex ratchets. I also needed a special low profile jack to lift the BMW, and the list of tooling differences grew from there.

[Image attribution: phtorxp]

I raise this to point out that just because you have some tools that work well for one purpose, they may not work well for another. And if there’s anything I learned from cars it’s that the smart money invests in the right tools for the job. Sure, you could go hokey and try to jerry-rig some insane workaround, but you’ll very likely pay for it in the end in both stress and time.

Kubernetes environments can also vary from one another. How does Skaffold help? Enter Skaffold profiles. Skaffold configuration typically is scoped to an application or workload. But there are likely environmental differences moving from local dev, to CI, to various staging and pre-production environments, and ultimately prod. Secrets and environmental variables will differ, replica sets may be low in dev and high in prod, build characteristics may differ, etc.

Skaffold Profiles can be added to any Skaffold configuration file in the profiles stanza. Each profile has a name, and under that, you can define the profile’s build, test, portForwarding, and deploy sections. In other words, it allows you to customize nearly every aspect of configuration we talked about last time.

Example Skaffold file with profiles:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

apiVersion: skaffold/v2beta15
kind: Config
metadata:
 name: my-app
build:
 artifacts:
 - image: my-service
   context: .
   docker:
     dockerfile: path/to/Dockerfile
deploy:
 kubectl:
   manifests: ["path/to/default.yaml"]
profiles:
  - name: packs
    build:
      artifacts:
      - image: my-service
        buildpacks:
          builder: gcr.io/buildpacks/builder
  - name: prod
    build:
      googleCloudBuild: 
        projectId: my-google-cloud-project
      artifacts:
      - image: my-service
        docker:
          dockerfile: docker/distroless.Dockerfile
    deploy:
      kubectl:
        manifests: ["path/to/prod.yaml"]

But what about the configuration that sits above the profiles stanza? Well, think of that as your default configuration. In fact, your profiles can simply contain variances that override your default configuration; everything else is inherited from the default config.

Using Skaffold Profiles

To build and deploy with a specific profile from command line:

$ skaffold dev --profile [profile-name]

Or set the profile attribute in launch.json (vscode) and then use the Cloud Code “Run as Kubernetes” menu. In this example, I’m going to use my “packs” profile which use buildpacks as opposed to the default Dockerfile approach:

Profiles can also be auto-activated, say, in the presence of a specific environmental variable, Kubernetes context, or Skaffold command.

Build with profiles

There’s a lot going on in the example above, I have a variety of build examples to show just how different your profiles can be, including:

• Dockerfile (the default - line 10)
• Buildpacks (“packs” profile - line 20)
• even a Distroless option for production (“prod” profile - line 28)

Distroless is a cool concept and likely a smart move from a security standpoint (now SLSA 2 compliant), but have you ever tried developing with it? No shell, no tar, no nothing. So in my example, I have only my “prod” profile set to use distroless.

Offloading builds to the cloud

I’ll point out another cool feature of Skaffold build. You can offload your builds to Google Cloud Build (and turn off your local Docker daemon) by adding a simple configuration parameter to your build config:

21
22
23
24
25

  - name: prod
    build:
      googleCloudBuild: 
        projectId: my-google-cloud-project
...

Deploy with profiles

The various environments you deploy to (local, remote cloud clusters, etc.) may require configuration differences, and once again Skaffold profiles help simplify deployment across diverse environments.

What kinds of differences? Replica counts, environment variables, secrets… really anything supported by the Kubernetes API. Profiles let you specify separate manifest files per environment if you’re using kubectl. For example, the domain name of your database will absolutely be different in staging and prod. In addition to my kubectl examples, Helm and Kustomize are also supported.

In my example above, I’m using an environment specific Kubernetes configuration for my “prod” profile:

21
22
23
24
25
26
27
28
29
30
31
32

  - name: prod
    build:
      googleCloudBuild: 
        projectId: my-google-cloud-project
      artifacts:
      - image: pop-stats
        docker:
          dockerfile: docker/distroless.Dockerfile
    deploy:
      kubectl:
        manifests: ["path/to/prod.yaml"]
...

Here’s a kustomize example.

How do I actually deploy to remote clusters? Skaffold uses your kube context and deploys to it, so all you need to do is change your kube context and re-run skaffold dev or skaffold run. You could use the kubectx utility to update the context, or you can use Cloud Code’s context selector UI in your favorite IDE.

Try the sample app

If you followed along from part 1 of this series, you’ll recall my “pop stats” sample app. The skaffold.yaml file is similar to the one I posted above here. And Cloud Shell will give you a chance to play around with these ideas (Google account required):

Onboarding others into your application

Skaffold also really helps with onboarding teammates. This may be abundantly obvious from the fact that you can take my app and start building it in minutes. But in case it’s not, the suggestion is to check skaffold.yaml into your repo. If your teammates get Cloud Code installed, building and deploying your app becomes as simple as calling ‘skaffold dev’. In other words, set this up once and let your entire team enjoy the benefits and unify their dev approach in the process (hopefully resulting in less “works for me” bug status updates).

Next time

OK, so hopefully I’ve piqued your interest on this Skaffold thing. But what about CI/CD? Can Skaffold play a part in unifying your development and DevOps loops? We’ll explore that in the next part of this Skaffold series.