[Starlingx-discuss] networking across pods / OpenStack VMs

[Webster, Steven]

Forwarding to the list - didn't 'reply all'

________________________________
From: Webster, Steven <Steven.Webster at windriver.com>
Sent: Thursday, April 30, 2020 6:09 PM
To: Jones, Bruce E <bruce.e.jones at intel.com>
Subject: Re: networking across pods / OpenStack VMs

Hi Bruce,

It's an interesting and (I think) valid question.  The key issue is that there is no coordination between openstack (neutron) networking and kubernetes.  Here are some thoughts - keep in mind I have not tried any of this!

1. In StarlingX as it stands today, it would probably be possible to connect a container and VM via SR-IOV.  Similar to the neutron issue, there's no coordination between openstack (nova) and the kubernetes SR-IOV device plugin.  So you'd probably want to dedicate an entire NIC to SR-IOV use for kubernetes, and an entire NIC to openstack, on separate data networks.  Otherwise, it would be possible for kubernetes and openstack to grab the same SR-IOV VF.  Of course, you'd want to configure the external switch such that the SR-IOV ports dedicated to kubernetes could reach those dedicated to openstack.

2. I think it would be possible to expose a nodePort to an application running in a kubernetes service.  Then, the openstack VMs could communicate via with the app via the nodePort by essentially going 'external'  [1]

3. Run the application in a kubernetes pod/container running inside the VM (that has joined the bare-metal cluster).  The VM would essentially be seen as another node that has joined the cluster.

4. There's a project which aims to solve some of the neutron/k8s integration - kubernetes-kuyr.  This aims to enable neutron networking awareness in a kubernetes cluster [2].  We don't integrate or support this project in StarlingX 3.0+

Happy to continue the discussion if anybody else has any thoughts,

Steve

[1] https://kubernetes.io/docs/concepts/services-networking/connect-applications-service/
Connecting Applications with Services - Kubernetes<https://kubernetes.io/docs/concepts/services-networking/connect-applications-service/>
This specification will create a Service which targets TCP port 80 on any Pod with the run: my-nginx label, and expose it on an abstracted Service port (targetPort: is the port the container accepts traffic on, port: is the abstracted Service port, which can be any port other pods use to access the Service).View Service API object to see the list of supported fields in service definition.
kubernetes.io

[2] https://github.com/openstack/kuryr-kubernetes
[https://avatars2.githubusercontent.com/u/324574?s=400&v=4]<https://github.com/openstack/kuryr-kubernetes>
GitHub - openstack/kuryr-kubernetes: Kubernetes integration with OpenStack networking<https://github.com/openstack/kuryr-kubernetes>
The OpenStack Kuryr project enables native Neutron-based networking in Kubernetes. With Kuryr-Kubernetes it's now possible to choose to run both OpenStack VMs and Kubernetes Pods on the same Neutron network if your workloads require it or to use different segments and, for example, route between ...
github.com

________________________________
From: Jones, Bruce E <bruce.e.jones at intel.com>
Sent: Thursday, April 30, 2020 1:36 PM
To: starlingx-discuss at lists.starlingx.io <starlingx-discuss at lists.starlingx.io>
Subject: [Starlingx-discuss] networking across pods / OpenStack VMs


I have a question from a potential user for the Networking team.  Is it possible in stx.3.0 to connect applications running in K8S pods to apps running in VMs under OpenStack?  If so, how?  If not, is that something that we could solve in StarlingX or is this something we’d need the respective upstreams to address?



Thank you!


         brucej
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.starlingx.io/pipermail/starlingx-discuss/attachments/20200430/3aadea89/attachment.html>