[Starlingx-discuss] Diagnostic metrics of a healthy StarlingX system.

[Cordoba Malibran, Erich]

Hi,

After some time debugging issues in StarlingX I think it would be
helpful to have a tool (if there isn't one yet) to get a diagnostic of
the health of the system. Something that an user, with a limited
knowledge of the system like me, can run and get the status of the
different components. 


I would like to start writing some code for this and I have a general
idea on which components to check.

As an initial approach I plan to: 

  - Check that configuration has been applied without errors: I plan to
check for puppet/ansible logs and look for errors there.
  - Check for services up and running: Look at the output of sm-dump.
  - Verify the health of k8s: Checking that no pods are in Init state.
  - Check connectivity with additional nodes: To check that nodes
booted and has network access.
  - Not sure yet how to check storage status beyond "ceph -s"

I know this might be a naive approach and I'll appreciate any guidance
to get in the correct path.

Thank you in advance.

-Erich