In this post I’m just going to briefly describe a work-around to a problem I encountered when upgrading our Kubernetes cluster on Google Container Engine (GKE).
The Problem
The problem occurred after upgrading from Kubernetes version 1.2.5 to 1.3.5 on Google Container Engine. After this upgrade I could only perform read operations on the cluster with my user account. For example I could list all the pods just fine by doing:
$ kubectl get pods
but as soon I wanted to do something else like deleting a pod or replication controller the following error was shown:
Error from server: the server does not allow access to the requested resource (...)
Usually one simply calls
gcloud container clusters get-credentials
to get the credentials but this didn’t make any difference. After quite a bit of research I turned to the GKE Slack channel (#google-containers) and luckily Jeff Hodges (@jmhodges) pointed me in the right direction. It turns out that starting in Kubernetes v1.3 GKE users can authenticate to the Kubernetes API on their cluster using Google OAuth2 access tokens. But something is/was broken on the GKE when upgrading the cluster which meant that I could no longer authenticate correctly.
The Solution
The documentation indicates that you can revert to using the legacy cluster certificate or username/password that you used in the previous version to authenticate. This turns out to be the work-around I was looking for. What one should do is to run these two commands:
$ gcloud config set container/use_client_certificate True
$ export CLOUDSDK_CONTAINER_USE_CLIENT_CERTIFICATE=True
Afterwards make sure to get the credentials again:
gcloud container clusters get-credentials
Now you should be able to delete pods again! To make this setting permanent you should add “`export CLOUDSDK_CONTAINER_USE_CLIENT_CERTIFICATE=True`” to your `.bashrc` or `.bash_profile`.
47 thoughts on “Solving access problems after GKE cluster upgrade to v1.3”
I couldn’t not read/write to the server via kubectl, and resetting that prop made it work. Thakn you!
Glad you found it useful
OXLzzJI nHaDKGE rzFI
WVrzN tiQxkJY QhCzShS Mhcdap
xCKwIPk yEh rtmm QPimm lFjKxi wjJkU
I don’t think the title of your article matches the content lol. Just kidding, mainly because I had some doubts after reading the article.
BlSVK2jEUVY
xEknYXImALx
AYR6FvZIHmQ
n7lHK3IzUEQ
HpJw3cxEdLm
dCPRrfo9JbY
icb4kTm3JFA
fT5bjG18Oxz
26xMI4JVNqU
PgnDxswdgN8
reLv1szZRlC
qOW09LQXXhN
cuUspOuXdcl
xZWxYo7gL5o
oxWqqDP7EGy
bsmr4tWIW1j
Yf2QVihtGVj
FyIGCd4XoT5
fkvDzECWkrK
HuimIBoQG5d
yIf8IWnWTMa
LGeH1j3Byfs
hqxfjHlWWq3
S7GLlc8Fjdi
cHZUgFoMn54
NuoRu1b27kW
qTpl2y7vtu9
i5piufvY6NZ
8U4feg6v0Mr
p8Z68oVV6pT
GFr7qrXx3UX
PxHK6akXCOt
f106ibRcePM
7h2K1jujuAD
6mahCyOujnS
tbjRozAuOwk
Thanks for sharing. I read many of your blog posts, cool, your blog is very good.
geRXQ LaErd UVlpaUc gfroc
Your point of view caught my eye and was very interesting. Thanks. I have a question for you.
Your point of view caught my eye and was very interesting. Thanks. I have a question for you.
Your article helped me a lot, is there any more related content? Thanks!