Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't get status, inspect stuck #2125

Closed
kwonmha opened this issue Mar 24, 2021 · 15 comments
Closed

Can't get status, inspect stuck #2125

kwonmha opened this issue Mar 24, 2021 · 15 comments
Labels

Comments

@kwonmha
Copy link

kwonmha commented Mar 24, 2021

microk8s status doesn't show anything.
It says, microk8s isn't running.
I stopped and started microk8s but it didn't work.
And, I can't run microk8s inspect either.

(base) user01@user-X299-WU8:/var/snap/microk8s/current/args$ microk8s status
microk8s is not running. Use microk8s inspect for a deeper inspection.
(base) user01@user-X299-WU8:/var/snap/microk8s/current/args$ microk8s stop
Stopped.
(base) user01@user-X299-WU8:/var/snap/microk8s/current/args$ microk8s start
Started.
(base) user01@user-X299-WU8:/var/snap/microk8s/current/args$ microk8s status
microk8s is not running. Use microk8s inspect for a deeper inspection.

(base) user01@user-X299-WU8:/var/snap/microk8s/current/args$ microk8s inspect
Inspecting Certificates
Inspecting services
  Service snap.microk8s.daemon-cluster-agent is running
  Service snap.microk8s.daemon-containerd is running
  Service snap.microk8s.daemon-apiserver is running
  Service snap.microk8s.daemon-apiserver-kicker is running
  Service snap.microk8s.daemon-control-plane-kicker is running
  Service snap.microk8s.daemon-proxy is running
  Service snap.microk8s.daemon-kubelet is running
  Service snap.microk8s.daemon-scheduler is running
  Service snap.microk8s.daemon-controller-manager is running
  Copy service arguments to the final report tarball
Inspecting AppArmor configuration
Gathering system information
  Copy processes list to the final report tarball
  Copy snap list to the final report tarball
  Copy VM name (or none) to the final report tarball
  Copy disk usage information to the final report tarball
  Copy memory usage information to the final report tarball
  Copy server uptime to the final report tarball
  Copy current linux distribution to the final report tarball
  Copy openSSL information to the final report tarball
  Copy network configuration to the final report tarball
Inspecting kubernetes cluster
  Inspect kubernetes cluster
Inspecting juju
  Inspect Juju

It stucks more than 10 minutes.

@balchua
Copy link
Collaborator

balchua commented Apr 8, 2021

@kwonmha just wanted to check if you are still facing this issue. Thanks

@kwonmha
Copy link
Author

kwonmha commented Apr 12, 2021

I can't reproduce this issue now.
I'll report later if this happens again.

@balchua
Copy link
Collaborator

balchua commented Apr 12, 2021

Great to hear that. Thanks 👍

@Pezhvak
Copy link

Pezhvak commented Apr 12, 2021

something is off, my cluster is not working at all, tried re installing all the nodes but it keeps happening, here is my tarball:
inspection-report-20210412_150020.tar.gz

@balchua
Copy link
Collaborator

balchua commented Apr 12, 2021

@Pezhvak can you try to remove it with sudo snap remove microk8s --purge? Then do sudo snap install --classic --channel <the channel you want>

@betsegaw
Copy link

I have run into something similar and I get inconsistent results...

Basically, a low powered (1 cpu 1 gb ram) Azure VM with ubuntu server 20 lts never returns a value for microk8s status or microk8s inspect... and instead hangs.

Is that behavior expected?

@balchua
Copy link
Collaborator

balchua commented Apr 13, 2021

@betsegaw my guess is that the kernel is killing the apiserver due to memory pressure. You may want to try enabling swap. But that is highly discouraged by kubernetes. Imho 1GB is too small for kubernetes.

@betsegaw
Copy link

@balchua - thank you for the reply and you are correct, increasing the memory of the VM fixed things. Sorry for the late confirmation but I had my github notifications off and didn't see someone had replied.

@RaghuMeda
Copy link

I have run into something similar and I get inconsistent results...

Basically, a low powered (1 cpu 1 gb ram) Azure VM with ubuntu server 20 lts never returns a value for microk8s status or microk8s inspect... and instead hangs.

Is that behavior expected?

It's the same issue for me. I am using Raspberry Pi3 with 2 cpu and 1 GB ram and microk8s status and microk8s inspect always hangs out. Its pure luck when it works. I did replicate the same issue happens in AWS EC2 with 2 cpu and 1 GB ARM64 system. I beleieve Microk8s is not tested throughly yet for ARM64 systems with low power. Although it says it works for low power devices thats not true.

@fritzprix
Copy link

fritzprix commented Mar 12, 2022

have same issue from 1.23/stable with Raspberry Pi3. Is there specific version that works with rpi3? or any plan to fix issue?
or any walkaround for this?

microk8s inspect
Inspecting Certificates
Inspecting services
  Service snap.microk8s.daemon-cluster-agent is running
  Service snap.microk8s.daemon-containerd is running
  Service snap.microk8s.daemon-k8s-dqlite is running
  Service snap.microk8s.daemon-kubelite is running
  Service snap.microk8s.daemon-apiserver-kicker is running
  Copy service arguments to the final report tarball
Inspecting AppArmor configuration
Gathering system information
  Copy processes list to the final report tarball
  Copy snap list to the final report tarball
  Copy VM name (or none) to the final report tarball
  Copy disk usage information to the final report tarball
  Copy memory usage information to the final report tarball
  Copy server uptime to the final report tarball
  Copy current linux distribution to the final report tarball
  Copy openSSL information to the final report tarball
  Copy network configuration to the final report tarball
Inspecting kubernetes cluster
  Inspect kubernetes cluster
HANGS HERE!!

@stale
Copy link

stale bot commented Feb 5, 2023

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the inactive label Feb 5, 2023
@stale stale bot closed this as completed Mar 7, 2023
@felipegouveiae
Copy link

I'm having the same issue after rebooting my Mac m1 over Multipass.

@xvilo
Copy link

xvilo commented Jun 2, 2024

I'm having similar issues on an RPi 5 8GB, after my internal network's ip changed. I'm in the process of trying to get it all working again but it just hangs:

$ microk8s inspect
Inspecting system
Inspecting Certificates
Inspecting services
  Service snap.microk8s.daemon-cluster-agent is running
  Service snap.microk8s.daemon-containerd is running
  Service snap.microk8s.daemon-kubelite is running
  Service snap.microk8s.daemon-k8s-dqlite is running
 FAIL:  Service snap.microk8s.daemon-apiserver-proxy is not running
For more details look at: sudo journalctl -u snap.microk8s.daemon-apiserver-proxy
  Service snap.microk8s.daemon-apiserver-kicker is running
  Copy service arguments to the final report tarball
Inspecting AppArmor configuration
Gathering system information
  Copy processes list to the final report tarball
  Copy disk usage information to the final report tarball
  Copy memory usage information to the final report tarball
  Copy server uptime to the final report tarball
  Copy openSSL information to the final report tarball
  Copy snap list to the final report tarball
  Copy VM name (or none) to the final report tarball
  Copy current linux distribution to the final report tarball
  Copy asnycio usage and limits to the final report tarball
  Copy inotify max_user_instances and max_user_watches to the final report tarball
  Copy network configuration to the final report tarball
Inspecting kubernetes cluster
  Inspect kubernetes cluster

@TheMickeyMike
Copy link

I had a similar problem. What worked for me:

microk8s stop
microk8s refresh-certs --check
The CA certificate will expire in X days.
The server certificate will expire in X days.
The front proxy client certificate will expire in X days.

microk8s refresh-certs --cert server.crt
microk8s start

@Kavlings
Copy link

Kavlings commented Nov 1, 2024

have same issue from 1.23/stable with Raspberry Pi3. Is there specific version that works with rpi3? or any plan to fix issue? or any walkaround for this?

microk8s inspect
Inspecting Certificates
Inspecting services
  Service snap.microk8s.daemon-cluster-agent is running
  Service snap.microk8s.daemon-containerd is running
  Service snap.microk8s.daemon-k8s-dqlite is running
  Service snap.microk8s.daemon-kubelite is running
  Service snap.microk8s.daemon-apiserver-kicker is running
  Copy service arguments to the final report tarball
Inspecting AppArmor configuration
Gathering system information
  Copy processes list to the final report tarball
  Copy snap list to the final report tarball
  Copy VM name (or none) to the final report tarball
  Copy disk usage information to the final report tarball
  Copy memory usage information to the final report tarball
  Copy server uptime to the final report tarball
  Copy current linux distribution to the final report tarball
  Copy openSSL information to the final report tarball
  Copy network configuration to the final report tarball
Inspecting kubernetes cluster
  Inspect kubernetes cluster
HANGS HERE!!

Having the same issue here on an Pi3 as well, anyone found a fix yet?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

10 participants