Google Kubernetes Engine & GCP

I’ve been using Google Compute Platform (GCP) & Google Kubernetes Engine (GKE) for years.
And I love it, I hated AWS since the first release to these days, naming is cumbersome, UX is terrible…

Here are a quick survey of the tools and how I’ve used them for small to medium projects:

Cloud Build

Cloud Build is super easy, it’s a scriptable CI/CD, very similar to Github Actions, there is a one hour free tier per day, the default machine is very slow, it can be tuned in the cloudbuild.yaml file as the following example:

steps:
- name: 'akhenakh/gotools'
  args:  ['make build-only']
  env: ['GO111MODULE=on', 'VERSION=$SHORT_SHA']
- name: 'gcr.io/cloud-builders/docker'
  args: ['build', 
  '-t', 'gcr.io/$PROJECT_ID/authord:$SHORT_SHA',
  '-t', 'gcr.io/$PROJECT_ID/authord:latest',
  'author/cmd/authord']
  env: ['PROJECT_ROOT=.']
images:
- 'gcr.io/$PROJECT_ID/authord'
options:  
  machineType: N1_HIGHCPU_8

Add a repository (can be Github), add a hook on commit or tag, it will get the image akhenakh/gotools and execute make build-only, then create a docker image, resulting artifacts will be pushed to the ‘gcr.io/$PROJECT_ID/authord’

Note the akhenakh/gotools is a custom image with several tools like protobuf, golangci-lint and more. It’s really easy to provide your own build system since Cloud Build is spawning an instance of your docker image then executes the commands your provide.

Container Registry

Not much to say here, it’s a docker private container registry, accessible at this address: ‘gcr.io/$PROJECT_ID/myimage’
Visble directly from your Kubernetes cluster if they both are in the same GCP project.

Google Kubernetes Engine

No surprise since the Kubernetes project is mainly handled by Google, GKE is the most up to date provider, a delight to use almost on every subjects but metrics…

Logging & Metrics

Google is offering a service call StackDriver which is supposed to be a fully integrated, metrics, logs, traces, but … it’s terrible, unfinished/WIP product Google bought.

The provided fluentd is consuming 300MiB per nodes.

The required OpenCensus library to push metrics & traces to StackDriver is “stodgy”, prefer Prometheus & OpenTracing, I’ve migrated several projects to OpenCensus for StackDriver support and regret it.

Having to manage my own tools is costly but I’m back to completely disable StackDriver at cluster creation and use Prometheus + Grafana & Jaeger.
Remember to enable prometheus KubeStateMetrics=true to collect your Kub cluster states.
Don’t worry the kubectl log command still works, also give a shot to Loki for handling your logs history.

In Grafana install the great Kubernetes dashboard.

Preemptive VM

Preemptive VMs are cheap instances with no guarantee from Google, they can be shutdown at any time, it can be use for ephemeral jobs.
It has a nice side effect as acting as a Chaos monkey tool.

Octant to the rescue

The state of the Kubernetes dashboards are unclear, the Web UI is not supported anymore, Google recommends to use the Cloud console which is okay but won’t be available on your developer instance (Minikube, k3s…).

I’m a big fan of the Octant frontend, no installation in your cluster required, it runs on your client machine.

Cloud SQL

Not everybody wants to manage a database, especially inside Kubernetes, Cloud SQL offers managed Mysql and Postgresql instances with HA & backups.
Note that Postgresql extensions like Postgis are supported !

Big Query

Big Query is an analytics data crunching solution with an SQL frontend.
Any situations you have some analytics to do you can probably use Big Query, capable of live streaming insertions and GIS.
It’s also very cheap compared to in house solution.

Overall

Don’t be fooled by this post, I had to report the negative parts but I will never exchange GCP for AWS nor Azure !

I still don’t get why people would chose AWS over GCP for new projects, other than bad habits and conservatism.

And by the way I’m not affiliated with Google in any ways :)