TL;DR: Checkout the template engine at github.
When working with pods in Kubernetes I had the need to define environment variables for the pod from a file. This is currently not supported in the currently released version of Kubernetes (1.1). What I wanted to do was to have a template file, for example `pod-template.yaml`, that looked like this:
kind: "ReplicationController"
apiVersion: "v1"
metadata:
name: "my-pod-{{VERSION}}"
labels:
name: "my-pod"
version: "{{VERSION}}"
spec:
replicas: 2
selector:
name: "my-pod"
version: "{{VERSION}}"
template:
metadata:
labels:
name: "my-pod"
version: "{{VERSION}}"
spec:
containers:
- name: "my-pod"
image: "gcr.io/my-project/my-pod:{{VERSION}}"
env:
- name: "TIMEOUT"
value: {{TIMEOUT}}
- name: "URL"
value: "{{URL}}"
ports:
- containerPort: {{API_PORT}}
name: "api"
protocol: "TCP"
- containerPort: {{ADMIN_PORT}}
name: "admin"
protocol: "TCP"
and be able to replace things inside the double brackets (placeholders) with values defined externally to the `pod-template.yaml`. This is of course nothing new and there’s a gazillion of various templating engines out there for various programming languages. But after quite a lot of googling and looking through stackoverflow I couldn’t find anything that worked the way I wanted to in bash.
What made this particular use case a bit tricky was that some of the placeholder values were defined in a file while others had to be passed as parameters to the template engine on usage. For example the value of the `{{VERSION}}` placeholder is generated by our continuous integration server while all others (`{{TIMEOUT}}`, `{{URL}}` etc) are defined in an (encrypted) configuration file checked into our code repository. So I needed a template engine that could handle both of these cases at the same time. The closest thing I found was the bash-templater projected created by Sébastien Lavoie which allowed me to do like this:
$ VERSION=1.2.3 TIMEOUT=1000 URL="http://www.google.com" API_PORT=8080 ADMIN_PORT=8081 \
./templater.sh pod-template.yaml
I.e. it takes environment variables and replaces the placeholders defined in the template file with these values. The outcome looks like this:
kind: "ReplicationController"
apiVersion: "v1"
metadata:
name: "my-pod-1.2.3"
labels:
name: "my-pod"
version: "1.2.3"
spec:
replicas: 2
selector:
name: "my-pod"
version: "1.2.3"
template:
metadata:
labels:
name: "my-pod"
version: "1.2.3"
spec:
containers:
- name: "my-pod"
image: "gcr.io/my-project/my-pod:1.2.3"
env:
- name: "TIMEOUT"
value: 1000
- name: "URL"
value: "http://www.google.se"
ports:
- containerPort: 8080
name: "api"
protocol: "TCP"
- containerPort: 8081
name: "admin"
protocol: "TCP"
The problem (again) was that our configuration was defined in a file (`config.properties`) that looked something like this:
# The timeout in milliseconds
TIMEOUT=1000
# The URL
URL="http://www.google.com"
# External API port
API_PORT=8080
# Internal admin port
ADMIN_PORT=8081
So I ended up forking `bash-templater` to my github repository and added support for passing in values as parameters as well as reading values from a file. Usage:
$ VERSION=1.2.3 ./templater.sh pod-template.yaml -f config.properties > pod.yaml
Volia, all requirements fulfilled! The result of the transformation is stored in the `pod.yaml` file.
I also added some other tweaks, for example suppression of warning messages (that will otherwise be included in the resulting file and mess up the outcome) if you define entries in `config.properties` that are not found as placeholders in template. Just add the `-s` (silent) argument to ignore these warnings:
$ VERSION=1.2.3 ./templater.sh pod-template.yaml -f config.properties -s > pod.yaml
I hope that this will come to use for someone else as well.
13 thoughts on “Simple templating engine in Bash”
Why not just use `envsubst` ?
I’m not familiar with envsubst, could you please elaborate?
A simple solution: http://pempek.net/articles/2013/07/08/bash-sh-as-template-engine
export var1=”test”
envsubst generated.yaml
$var1 in template.yaml, and will substituted as “test” in generated.yaml
But, i cannot find a way to escape if using envsubst.
e.g. i want to keep ${var_to_keep} not substituted by envsubst
Johan, hello. Once again, thanks.
I’ve run into an issue where supplying a template with no variables (ie: {{}}) throws a warning and an error:
Warning: No variable was found in services/portal.yml, syntax is {{VAR}}
sed: 1: “services/portal.yml”: unterminated substitute in regular expression
I meant to open an issue in the repo but they’re disabled. This is just a heads up. I’ll try to submit a PR asap.
Hi Luis,
Thanks for your comment. I actually didn’t know that issues were disabled, I’ll fix this. Also a PR would be great 🙂
Regards,
/Johan
Thanks for your article! Are you going to submit your work upstream to Sébastien’s repo?
I already have provided a PR, see https://github.com/lavoiesl/bash-templater/pull/1, but it has not been accepted yet.
Ah, I missed that. Cool!
I drew some inspiration from your and lavoiesl’s work: https://github.com/relaxdiego/renderest
Just wanted to say that j2cli does the same, as uses jinja syntax for templating.
You can also source properties files as well:
https://github.com/kolypto/j2cli
Thanks for the tip