TL;DR: Checkout the template engine at github.
When working with pods in Kubernetes I had the need to define environment variables for the pod from a file. This is currently not supported in the currently released version of Kubernetes (1.1). What I wanted to do was to have a template file, for example pod-template.yaml
, that looked like this:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 |
kind: "ReplicationController" apiVersion: "v1" metadata: name: "my-pod-{{VERSION}}" labels: name: "my-pod" version: "{{VERSION}}" spec: replicas: 2 selector: name: "my-pod" version: "{{VERSION}}" template: metadata: labels: name: "my-pod" version: "{{VERSION}}" spec: containers: - name: "my-pod" image: "gcr.io/my-project/my-pod:{{VERSION}}" env: - name: "TIMEOUT" value: {{TIMEOUT}} - name: "URL" value: "{{URL}}" ports: - containerPort: {{API_PORT}} name: "api" protocol: "TCP" - containerPort: {{ADMIN_PORT}} name: "admin" protocol: "TCP" |
and be able to replace things inside the double brackets (placeholders) with values defined externally to the pod-template.yaml
. This is of course nothing new and there’s a gazillion of various templating engines out there for various programming languages. But after quite a lot of googling and looking through stackoverflow I couldn’t find anything that worked the way I wanted to in bash.
What made this particular use case a bit tricky was that some of the placeholder values were defined in a file while others had to be passed as parameters to the template engine on usage. For example the value of the {{VERSION}}
placeholder is generated by our continuous integration server while all others ({{TIMEOUT}}
, {{URL}}
etc) are defined in an (encrypted) configuration file checked into our code repository. So I needed a template engine that could handle both of these cases at the same time. The closest thing I found was the bash-templater projected created by Sébastien Lavoie which allowed me to do like this:
1 2 |
$ VERSION=1.2.3 TIMEOUT=1000 URL="http://www.google.com" API_PORT=8080 ADMIN_PORT=8081 \ ./templater.sh pod-template.yaml |
I.e. it takes environment variables and replaces the placeholders defined in the template file with these values. The outcome looks like this:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 |
kind: "ReplicationController" apiVersion: "v1" metadata: name: "my-pod-1.2.3" labels: name: "my-pod" version: "1.2.3" spec: replicas: 2 selector: name: "my-pod" version: "1.2.3" template: metadata: labels: name: "my-pod" version: "1.2.3" spec: containers: - name: "my-pod" image: "gcr.io/my-project/my-pod:1.2.3" env: - name: "TIMEOUT" value: 1000 - name: "URL" value: "http://www.google.se" ports: - containerPort: 8080 name: "api" protocol: "TCP" - containerPort: 8081 name: "admin" protocol: "TCP" |
The problem (again) was that our configuration was defined in a file (config.properties
) that looked something like this:
1 2 3 4 5 6 7 8 |
# The timeout in milliseconds TIMEOUT=1000 # The URL URL="http://www.google.com" # External API port API_PORT=8080 # Internal admin port ADMIN_PORT=8081 |
So I ended up forking bash-templater
to my github repository and added support for passing in values as parameters as well as reading values from a file. Usage:
1 |
$ VERSION=1.2.3 ./templater.sh pod-template.yaml -f config.properties > pod.yaml |
Volia, all requirements fulfilled! The result of the transformation is stored in the pod.yaml
file.
I also added some other tweaks, for example suppression of warning messages (that will otherwise be included in the resulting file and mess up the outcome) if you define entries in config.properties
that are not found as placeholders in template. Just add the -s
(silent) argument to ignore these warnings:
1 |
$ VERSION=1.2.3 ./templater.sh pod-template.yaml -f config.properties -s > pod.yaml |
I hope that this will come to use for someone else as well.
Why not just use
envsubst
?I’m not familiar with envsubst, could you please elaborate?
[…] resides in the app-config.properties file. I’ve described this workaround in more detail here. Kubernetes 1.2 will give us ConfigMaps that’ll probably address the need for this […]
A simple solution: http://pempek.net/articles/2013/07/08/bash-sh-as-template-engine
export var1=”test”
envsubst generated.yaml
$var1 in template.yaml, and will substituted as “test” in generated.yaml
But, i cannot find a way to escape if using envsubst.
e.g. i want to keep ${var_to_keep} not substituted by envsubst
Johan, hello. Once again, thanks.
I’ve run into an issue where supplying a template with no variables (ie: {{}}) throws a warning and an error:
Warning: No variable was found in services/portal.yml, syntax is {{VAR}}
sed: 1: “services/portal.yml”: unterminated substitute in regular expression
I meant to open an issue in the repo but they’re disabled. This is just a heads up. I’ll try to submit a PR asap.
Hi Luis,
Thanks for your comment. I actually didn’t know that issues were disabled, I’ll fix this. Also a PR would be great 🙂
Regards,
/Johan
Thanks for your article! Are you going to submit your work upstream to Sébastien’s repo?
I already have provided a PR, see https://github.com/lavoiesl/bash-templater/pull/1, but it has not been accepted yet.
Ah, I missed that. Cool!
I drew some inspiration from your and lavoiesl’s work: https://github.com/relaxdiego/renderest
Just wanted to say that j2cli does the same, as uses jinja syntax for templating.
You can also source properties files as well:
https://github.com/kolypto/j2cli
Thanks for the tip
[…] johanhaleby/bash-templater & Simple templating engine in Bash […]