[Starlingx-discuss] proposal for Intel GPU K8s device plugin support in StarlingX

[Victor Rodriguez]

On Wed, Jul 17, 2019 at 12:24 PM Yong Hu <yong.hu at intel.com> wrote:
>
> pls see my comments.
>
> On 17/07/2019 9:08 AM, Victor Rodriguez wrote:
> > On Mon, Jul 15, 2019 at 3:37 AM An, Ran1 <ran1.an at intel.com> wrote:
> >>
> >> Hi All
> >>
> >>    here is the proposal for enabling intel-gpu-plugin on StarlingX ,  welcome suggestion and advise.
> >>
> >>
> >>
> >> The background:
> >>
> >> As a part of resource management, kubernetes provides a device plugin framework [2] for vendors to advertise their resources to the kubelet since version 1.8. StarlingX has already supported SR-IOV CNI plugins now [3].
> >>
> >> Intel-gpu-plugins is a device plugin implementation [4] for intel GPU (with driver i915). Users could deploy their pods with Intel GPU resource requests or limits, if intel-gpu-plugins was integrated into StarlingX.
> >>
> >
> > +1
> >
> >>
> >>
> >> proposal:
> >>
> >> Deploy intel-gpu-plugins as a daemon set with node selector “intelgpu: enabled”. Kubernetes label “intelgpu: enabled” will be set automatically once the node detected supported GPU device.
> >>
> >> Details are shown as follows:
> >>
> >> 1.      Build StarlingX plugin docker image based on [5], the implement in starlingx are [6] and [7]
> >>
> >
> > ok
> >
> >> 2.      Deploy Intel-gpu-plugins daemon set in tasks “bringup_kubemaster” after kubernetes master has been initialized during ansible bootstrap process. Add value “import_plugins” and value list “kube_plugins” as condition of deploying Intel-gpu-plugins daemon set, so user could determine whether Intel-gpu-plugins would be enabled. Create file “/etc/platform/enabled_kube_plugins” and write list “kube_plugins” into the file after active Intel-gpu-plugin daemon if “import_plugins” is true. Partical Implement is [8]
> >>
> >> 3.      Detect supported GPU device with the help of sysinv agent and request to set kubernetes label “intelgpu: enabled” for specific node by calling sysinv conductor rpcapi. Sysinv conductor will check file “/etc/platform/enabled_kube_plugins”, and set kubernetes label if the file is exist and intel-gpu-plugins is in list. Partial implement is [9]
> >>
> >
> > I like the approach and the demo, I just have a question:
> >
> > One question, what kind of workload support the container running in
> > the GPU, do we have to write it in cuda? Do we have some example of
> > source code that will be run inside the container that will run on the
> > GPU ?
> >
> No, this is Intel GPU and we don't support Cuda APIs. We can use OpenCL.
> You can find Intel GPU device plugin demos
> here:https://github.com/intel/intel-device-plugins-for-kubernetes/tree/8310d84f96f9fc7b7487b2dfb9059905638c58fe/demo
>

Thanks , this clarify my questions :)

> > regards
> >
> >>
> >>
> >> [1] https://storyboard.openstack.org/#!/story/2005937
> >>
> >> [2] https://kubernetes.io/docs/concepts/extend-kubernetes/compute-storage-net/device-plugins/
> >>
> >>        https://github.com/kubernetes/community/blob/master/contributors/design-proposals/resource-management/device-plugin.md
> >>
> >> [3] https://review.opendev.org/#/c/655495/
> >>
> >> [4] https://github.com/intel/intel-device-plugins-for-kubernetes
> >>
> >> [5] https://github.com/intel/intel-device-plugins-for-kubernetes/blob/master/cmd/gpu_plugin/README.md
> >>
> >> [6] https://review.opendev.org/668803
> >>
> >> [7] https://review.opendev.org/668808
> >>
> >> [8] https://review.opendev.org/666510
> >>
> >> [9] https://review.opendev.org/666511
> >>
> >>
> >>
> >> Thanks
> >>
> >> Ran
> >>
> >>
> >>
> >> _______________________________________________
> >> Starlingx-discuss mailing list
> >> Starlingx-discuss at lists.starlingx.io
> >> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
> >
> > _______________________________________________
> > Starlingx-discuss mailing list
> > Starlingx-discuss at lists.starlingx.io
> > http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
> >