How to fix kubernetes dashboard forbidden 403 error – message services https kubernetes-dashboard is forbidden User

Jun 30, 2020 · 4 min read · Kubernetes  ·
Share on:

The objective of this article is to troubleshoot the https kubernetes-dashboard is forbidden error which you might have encountered after setting up the kubernetes dashboard but while trying to access it via browser it is throwing you forbidden error message.

This blog is divided into two section -

  1. The actual kubernetes dashboard forbidden error
  2. Root Cause
  3. How to Troubleshoot

1. The actual kubernetes dashboard forbidden error

Let me get straight to the point - You are trying to setup Kubernetes dashboard. You have installed the kubernetes dashboard using kubectl command -

1kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.4.0/aio/deploy/recommended.yaml

Also you have started your kubernetes API server using

1kubectl proxy

But after above steps when you tried to access the kubernetes dashboard URL (http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/) , you got the following error message.

 1{
 2  "kind": "Status",
 3  "apiVersion": "v1",
 4  "metadata": {
 5    
 6  },
 7  "status": "Failure",
 8  "message": "services \"https:kubernetes-dashboard:\" is forbidden: User \"system:anonymous\" cannot get services/proxy in the namespace \"kube-system\"",
 9  "reason": "Forbidden",
10  "details": {
11    "name": "https:kubernetes-dashboard:",
12    "kind": "services"
13  },
14  "code": 403
15}

2. Root Cause of Error

You do not have user certificate installed at your end and that is the reason kubernetes does not trust you, eventually leading to HTTP 403 forbidden error .


3. How to Troubleshoot

Install the certificate into your browser, so that kubernetes trust your request. But the questions comes from where to get the certificate ?

So we are going to generate the certificate using kubeconfig file which is generated by kubeadm. Follow the below steps for generating the certificate

(If you are interested more in visual way to troubleshoot then please refer to following lab session as well as follow the guide.)

Step 1 - Locate your kubeconfig file

If you are using vagrant setup just like me than you should go and look for /home/vagrant/.kube/config or /etc/kubernetes/admin.conf

If you couldn’t find it then use the following command to search

1find / -name '*.kube*' 2>/dev/null

It will lead to the kubeconfig file location

Step 2 - Generate kubecfg.crt

I am assuming that your kubeconfig file is located at /home/vagrant/.kube/config

Switch to /home/vagrant

1cd /home/vagrant

Run the following certificate generation command

1grep 'client-certificate-data' ~/.kube/config | head -n 1 | awk '{print $2}' | base64 -d >> kubecfg.crt

Now you should have kubecfg.crt generated at /home/vagrant

Step 3 - Generate kubecfg.key

To generate the kubecfg.key use the following command

1grep 'client-key-data' ~/.kube/config | head -n 1 | awk '{print $2}' | base64 -d >> kubecfg.key

Now you should have kubecfg.key generated at /home/vagrant

Step 4 - Generate kubecfg.p12

Use the following command to generate the kubecfg.p12

1openssl pkcs12 -export -clcerts -inkey kubecfg.key -in kubecfg.crt -out kubecfg.p12 -name "kubernetes-client"

Now you should have kubecfg.p12 generated at /home/vagrant

Step 5 - Copy generated kubecfg.p12 to you local development machine

Use the scp command for coping the kubecfg.p12 file to your local development machine

1scp kubecfg.p12 rahul@192.168.1.125:/home/rahul/Jhooq/Kubernetes/vagrant

So now you obtained your kubecfg.p12 file.

Step 6 - Import the kubecfg.p12 to your browser

In my case i am using Firefox so following steps are applicable for firefox but if your using Chrome then i would suggest to refer this link for importing the certificate in Google Chrome

Goto Firefox->Preferences

In the Find in Preferences search box search for Certificates

Now it will open Certificate Manager window

Now click on Import and import your kubecfg.p12 file here. It will ask for the password so please supply the same password which you used while at the time of creation.

And after the successful import it should look like this

That it is. Now you can go back and try accessing the URL again .i.e. - http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/.

You should be able to access that URL successfully.

Summary

To summarize this article we need following file -

  • .crt (kubecfg.crt)
  • .key(kubecfg.key)
  • .p12(kubecfg.p12)

For fixing the kubernetes dashboard error - “message”: “services "https:kubernetes-dashboard:" is forbidden: User "system:anonymous" cannot get services/proxy in the namespace "kube-system"”


Learn more On Kubernetes -

  1. Setup kubernetes on Ubuntu
  2. Setup Kubernetes on CentOs
  3. Setup HA Kubernetes Cluster with Kubespray
  4. Setup HA Kubernetes with Minikube
  5. Setup Kubernetes Dashboard for local kubernetes cluster
  6. Setup Kubernetes Dashboard On GCP(Google Cloud Platform)
  7. How to use Persistent Volume and Persistent Volume Claims in Kubernetes
  8. Deploy Spring Boot Microservice on local Kubernetes cluster
  9. Deploy Spring Boot Microservice on Cloud Platform(GCP)
  10. Setting up Ingress controller NGINX along with HAproxy inside Kubernetes cluster
  11. CI/CD Kubernetes | Setting up CI/CD Jenkins pipeline for kubernetes
  12. kubectl export YAML | Get YAML for deployed kubernetes resources(service, deployment, PV, PVC….)
  13. How to setup kubernetes jenkins pipeline on AWS?
  14. Implementing Kubernetes liveness, Readiness and Startup probes with Spring Boot Microservice Application?
  15. How to fix kubernetes pods getting recreated?
  16. How to delete all kubernetes PODS?
  17. How to use Kubernetes secrets?
  18. Share kubernetes secrets between namespaces?
  19. How to Delete PV(Persistent Volume) and PVC(Persistent Volume Claim) stuck in terminating state?
  20. Delete Kubernetes POD stuck in terminating state?