[Starlingx-discuss] User feedback and questions
Hi StarlingX Community, I’m reaching out to share some information with you from a call I had with a potential new user. * Industry area, use cases and requirements - The company is in the telecom industry, the person I talked to has been in the area of research - StarlingX use cases + Primarily private wireless and 5G core + Later on potential use cases might include AI, healthcare and more - They are currently prioritizing containerized workloads and later on will be looking into deploying OpenStack as well * Evaluation status - Early stages, they deployed StarlingX, both simplex and duplex, in virtual environments - Looking into bare metal deployment as well * Feedback and questions so far - As they have been looking into the platform, they didn’t seem to be able to find the level of multi-tenancy that they needed + Does the platform support multiple tenants within one Kubernetes cluster and one StarlingX instance? Is there any documentation that describes the available tenant and user configurations within StarlingX? - The networking setup has been hard to figure out for StarlingX, they found that it's not very well documented. + Is it in the roadmap to document the networking setup for StarlingX with more details? + Would it be possible to automate the creation of networks for the platform? Especially for the virtual deployment option, since that is most commonly deployed by new users, who are evaluating the platform. - They had an intern who was working on deploying the platform and has been using the community’s communication channels, including IRC. The intern had a hard time getting responses to their questions, which made it harder to figure out how to overcome the deployment challenges. Best Regards, Ildikó ——— Ildikó Váncsa Director of Community Open Infrastructure Foundation
Hi, Ildikó and StarlingX Community. I'll add my 2 cents regarding their feedback.
- The networking setup has been hard to figure out for StarlingX, they found that it's not very well documented. + Is it in the roadmap to document the networking setup for StarlingX with more details?
Regarding the above, the Docs team recently discussed a problem with the Virtual guides being outdated when compared to the bare metal guides. If someone was following the Virtual guides, it could partially account for their confusion. This problem is being tracked as a bug and it's actively being discussed in the Docs team meetings. While I don't represent the Docs team, I participated in the discussion, and I'm assigned to this bug while awaiting the resolution of other blocking factors. The bug in question can be found here: https://bugs.launchpad.net/starlingx/+bug/2028272. Therefore, even though there might be additional problems not covered by this bug, addressing the concern with the Virtual guides is indeed part of the community's plan. Regarding the query below:
+ Would it be possible to automate the creation of networks for the platform? Especially for the virtual deployment option, since that is most commonly deployed by new users, who are evaluating the platform.
I'm not sure I fully understand what exactly they meant by "automate the creation of networks for the platform". Could we gather more specific details on this? Regards, -- Bruno. ________________________________ From: Ildiko Vancsa <ildiko@openinfra.dev> Sent: Tuesday, August 8, 2023 19:06 To: StarlingX ML <starlingx-discuss@lists.starlingx.io> Subject: [Starlingx-discuss] User feedback and questions External Mail: Do not click links or open attachments unless you recognize the sender and know the content is safe. Hi StarlingX Community, I’m reaching out to share some information with you from a call I had with a potential new user. * Industry area, use cases and requirements - The company is in the telecom industry, the person I talked to has been in the area of research - StarlingX use cases + Primarily private wireless and 5G core + Later on potential use cases might include AI, healthcare and more - They are currently prioritizing containerized workloads and later on will be looking into deploying OpenStack as well * Evaluation status - Early stages, they deployed StarlingX, both simplex and duplex, in virtual environments - Looking into bare metal deployment as well * Feedback and questions so far - As they have been looking into the platform, they didn’t seem to be able to find the level of multi-tenancy that they needed + Does the platform support multiple tenants within one Kubernetes cluster and one StarlingX instance? Is there any documentation that describes the available tenant and user configurations within StarlingX? - The networking setup has been hard to figure out for StarlingX, they found that it's not very well documented. + Is it in the roadmap to document the networking setup for StarlingX with more details? + Would it be possible to automate the creation of networks for the platform? Especially for the virtual deployment option, since that is most commonly deployed by new users, who are evaluating the platform. - They had an intern who was working on deploying the platform and has been using the community’s communication channels, including IRC. The intern had a hard time getting responses to their questions, which made it harder to figure out how to overcome the deployment challenges. Best Regards, Ildikó ——— Ildikó Váncsa Director of Community Open Infrastructure Foundation _______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io
Hi Bruno, Thank you for the quick comments. A few notes in line.
[…]
Hi, Ildikó and StarlingX Community.
I'll add my 2 cents regarding their feedback.
- The networking setup has been hard to figure out for StarlingX, they found that it's not very well documented. + Is it in the roadmap to document the networking setup for StarlingX with more details?
Regarding the above, the Docs team recently discussed a problem with the Virtual guides being outdated when compared to the bare metal guides. If someone was following the Virtual guides, it could partially account for their confusion. This problem is being tracked as a bug and it's actively being discussed in the Docs team meetings. While I don't represent the Docs team, I participated in the discussion, and I'm assigned to this bug while awaiting the resolution of other blocking factors. The bug in question can be found here: https://bugs.launchpad.net/starlingx/+bug/2028272.
Therefore, even though there might be additional problems not covered by this bug, addressing the concern with the Virtual guides is indeed part of the community's plan.
[Ildiko] That is great to hear, thank you for the updates!
Regarding the query below:
+ Would it be possible to automate the creation of networks for the platform? Especially for the virtual deployment option, since that is most commonly deployed by new users, who are evaluating the platform.
I'm not sure I fully understand what exactly they meant by "automate the creation of networks for the platform". Could we gather more specific details on this?
[…]
[Ildiko] While we did not go into deep details on this during our conversation. The person said that the L2 network setup is fine, but the L3 is a bit lacking in terms of setting up NAT, bridges, etc. I’m not sure if the comments were maybe specific to the duplex system or just in general applicable to either virtual deployment options. Does this help? Thanks, Ildikó ——— Ildikó Váncsa Director of Community Open Infrastructure Foundation
Hi again, folks. Based on our conversation at the Community call today, we split the feedback regarding the network problems into two different things: 1) Get a GPU passthrough for NVIDIA to work on a virtual setup with VirtualBox. I don't have anything that I can add here, but would love to hear from others 🙂 2) Correctly set up networking for VirtualBox setup. On this second topic, I want to provide some additional info. If you had at any point a working StarlingX setup with VirtualBox (either using the recently merged automated guide<https://docs.starlingx.io/deploy_install_guides/release/virtual/automated_install.html> or manually installing with the canonical guides) and you're now having problems accessing services running in the VM(s) it could be this: https://www.virtualbox.org/ticket/20337#comment:1. In order to figure this out, I check if there's anything listening, on my host, in a port that I know is set up with a port forward in VirtualBox's NatNetwork (8080, for example): netstat -tunlp | grep 8080 If I don't get anything back from that, I then try to restart the NatNetworks with (notice that NatNetwork is the name of the Nat Network that is assigned to one of the VM's network adapters): vboxmanage natnetwork stop --netname NatNetwork vboxmanage natnetwork start --netname NatNetwork If after that the "netstat" is still not showing anything listening on ports that are forwarded between the host and the VM(s), then I also try to recreate the Nat Network from scratch (that worked already at least twice for me): vboxmanage natnetwork remove --netname NatNetwork || true vboxmanage natnetwork add --netname NatNetwork --dhcp off --network 10.10.10.0/24 vboxmanage natnetwork modify --netname NatNetwork --port-forward-4 "SSH:tcp:[]:3122:[10.10.10.3]:22" vboxmanage natnetwork modify --netname NatNetwork --port-forward-4 "HTTP:tcp:[]:8080:[10.10.10.3]:8080" ... and so on. If this is the case with your setup, I'm sorry. I know how frustrating this is and that's why I'm trying to follow up with the VirtualBox community (both on https://www.virtualbox.org/ticket/20337 and https://www.virtualbox.org/ticket/19651). If not, please let us know and share more details here 🙂. Regards, -- Bruno. ________________________________ From: Ildiko Vancsa <ildiko@openinfra.dev> Sent: Tuesday, August 8, 2023 19:56 To: Bruno Drugowick Muniz <Bruno.Muniz@encora.com> Cc: StarlingX ML <starlingx-discuss@lists.starlingx.io> Subject: Re: [Starlingx-discuss] User feedback and questions External Mail: Do not click links or open attachments unless you recognize the sender and know the content is safe. Hi Bruno, Thank you for the quick comments. A few notes in line.
[…]
Hi, Ildikó and StarlingX Community.
I'll add my 2 cents regarding their feedback.
- The networking setup has been hard to figure out for StarlingX, they found that it's not very well documented. + Is it in the roadmap to document the networking setup for StarlingX with more details?
Regarding the above, the Docs team recently discussed a problem with the Virtual guides being outdated when compared to the bare metal guides. If someone was following the Virtual guides, it could partially account for their confusion. This problem is being tracked as a bug and it's actively being discussed in the Docs team meetings. While I don't represent the Docs team, I participated in the discussion, and I'm assigned to this bug while awaiting the resolution of other blocking factors. The bug in question can be found here: https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugs.launchpad.net%2Fstarlingx%2F%2Bbug%2F2028272&data=05%7C01%7CBruno.Muniz%40encora.com%7C9439c06058ad4fcfd01108db9862bfe9%7C1e0c92b0f1bd441ebbe6c778f9ced553%7C0%7C0%7C638271322132779517%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=a%2FbPhdkTmLoXT83DCMqAQpvrau4DkcpldQqswm9PmX4%3D&reserved=0<https://bugs.launchpad.net/starlingx/+bug/2028272>.
Therefore, even though there might be additional problems not covered by this bug, addressing the concern with the Virtual guides is indeed part of the community's plan.
[Ildiko] That is great to hear, thank you for the updates!
Regarding the query below:
+ Would it be possible to automate the creation of networks for the platform? Especially for the virtual deployment option, since that is most commonly deployed by new users, who are evaluating the platform.
I'm not sure I fully understand what exactly they meant by "automate the creation of networks for the platform". Could we gather more specific details on this?
[…]
[Ildiko] While we did not go into deep details on this during our conversation. The person said that the L2 network setup is fine, but the L3 is a bit lacking in terms of setting up NAT, bridges, etc. I’m not sure if the comments were maybe specific to the duplex system or just in general applicable to either virtual deployment options. Does this help? Thanks, Ildikó ——— Ildikó Váncsa Director of Community Open Infrastructure Foundation
See in-lined comments/responses below, Greg. -----Original Message----- From: Ildiko Vancsa <ildiko@openinfra.dev> Sent: Tuesday, August 8, 2023 6:06 PM To: StarlingX ML <starlingx-discuss@lists.starlingx.io> Subject: [Starlingx-discuss] User feedback and questions CAUTION: This email comes from a non Wind River email account! Do not click links or open attachments unless you recognize the sender and know the content is safe. Hi StarlingX Community, I’m reaching out to share some information with you from a call I had with a potential new user. * Industry area, use cases and requirements - The company is in the telecom industry, the person I talked to has been in the area of research - StarlingX use cases + Primarily private wireless and 5G core + Later on potential use cases might include AI, healthcare and more - They are currently prioritizing containerized workloads and later on will be looking into deploying OpenStack as well * Evaluation status - Early stages, they deployed StarlingX, both simplex and duplex, in virtual environments - Looking into bare metal deployment as well * Feedback and questions so far - As they have been looking into the platform, they didn’t seem to be able to find the level of multi-tenancy that they needed + Does the platform support multiple tenants within one Kubernetes cluster and one StarlingX instance? Is there any documentation that describes the available tenant and user configurations within StarlingX? [Greg] In the context of StarlingX infrastructure management, there is absolutely no multiple tenant support. There is one, and only one, tenant that manages the hosts and the services that make up the StarlingX Bare Metal Infrastructure Management. In the context of the StarlingX Kubernetes cluster, … sort of, but nothing more than vanilla K8S gives you … • there is some documentation on how you can manage non-admin type k8s users, and groups of k8s users, in different namespaces, with restricted RBAC, restricted security policies and restricted resource management, etc. ( https://docs.starlingx.io/security/kubernetes/index-security-kub-81153c1254c... ) o … all using common K8S mechanisms • … sort of a very soft multi-tenancy • … but we have not done anything else o areas that we would LIKE to look into for kubernetes multi-tenancy is to basically look at the projects under the Kubernetes Working Group for Multi-Tenancy ( https://github.com/kubernetes-sigs/multi-tenancy/ ), e.g. • Benchmarks: a set of benchmarks (i.e., compliance tests) to determine if your clusters are well-configured for multitenancy. • Hierararchical namespaces (aka HNC): allows namespaces to own each other, policy propagation between related namespaces, and delegated namespace creation. • Virtual clusters: run multiple virtualized cluster on a single underlying cluster, allowing for hard(er) multitenancy. - The networking setup has been hard to figure out for StarlingX, they found that it's not very well documented. + Is it in the roadmap to document the networking setup for StarlingX with more details? + Would it be possible to automate the creation of networks for the platform? Especially for the virtual deployment option, since that is most commonly deployed by new users, who are evaluating the platform. [Greg] Would be interesting to have a discussion with them to understand their difficulty in understanding the platform networks. I.e. I’m assuming it is the ‘platform’ networks that they are having trouble understanding … i.e. https://docs.starlingx.io/_images/starlingx-deployment-options-controller-st... - They had an intern who was working on deploying the platform and has been using the community’s communication channels, including IRC. The intern had a hard time getting responses to their questions, which made it harder to figure out how to overcome the deployment challenges. Best Regards, Ildikó ——— Ildikó Váncsa Director of Community Open Infrastructure Foundation _______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io<mailto:Starlingx-discuss@lists.starlingx.io>
Hi Greg, Thank you for the clarifications. Please see a few more notes in line. Best Regards, Ildikó ——— Ildikó Váncsa Director of Community Open Infrastructure Foundation
[…]
* Feedback and questions so far - As they have been looking into the platform, they didn’t seem to be able to find the level of multi-tenancy that they needed + Does the platform support multiple tenants within one Kubernetes cluster and one StarlingX instance? Is there any documentation that describes the available tenant and user configurations within StarlingX? [Greg] In the context of StarlingX infrastructure management, there is absolutely no multiple tenant support. There is one, and only one, tenant that manages the hosts and the services that make up the StarlingX Bare Metal Infrastructure Management.
In the context of the StarlingX Kubernetes cluster, … sort of, but nothing more than vanilla K8S gives you … • there is some documentation on how you can manage non-admin type k8s users, and groups of k8s users, in different namespaces, with restricted RBAC, restricted security policies and restricted resource management, etc. ( https://docs.starlingx.io/security/kubernetes/index-security-kub-81153c1254c... ) • … all using common K8S mechanisms • … sort of a very soft multi-tenancy • … but we have not done anything else • areas that we would LIKE to look into for kubernetes multi-tenancy is to basically look at the projects under the Kubernetes Working Group for Multi-Tenancy ( https://github.com/kubernetes-sigs/multi-tenancy/ ), e.g. • Benchmarks: a set of benchmarks (i.e., compliance tests) to determine if your clusters are well-configured for multitenancy. • Hierararchical namespaces (aka HNC): allows namespaces to own each other, policy propagation between related namespaces, and delegated namespace creation. • Virtual clusters: run multiple virtualized cluster on a single underlying cluster, allowing for hard(er) multitenancy.
[Ildiko] I understand the infrastructure management constraints. When it comes to Kubernetes, is there a way to utilize the multi-tenancy concept from OpenStack, without running the Kubernetes pieces and containers in virtual machines?
- The networking setup has been hard to figure out for StarlingX, they found that it's not very well documented. + Is it in the roadmap to document the networking setup for StarlingX with more details? + Would it be possible to automate the creation of networks for the platform? Especially for the virtual deployment option, since that is most commonly deployed by new users, who are evaluating the platform. [Greg] Would be interesting to have a discussion with them to understand their difficulty in understanding the platform networks. I.e. I’m assuming it is the ‘platform’ networks that they are having trouble understanding … i.e. https://docs.starlingx.io/_images/starlingx-deployment-options-controller-st...
[Ildiko] We haven’t gone into deep details on this topic. They also had an intern working on setting StarlingX up, who might’v had less overall experience in networking that could’ve contributed to some of the challenges. Networking related questions are common on IRC too, so it is definitely an area that can benefit from more examples for people who are less experienced.
[…]
participants (3)
-
Bruno Drugowick Muniz
-
Ildiko Vancsa
-
Waines, Greg