[Starlingx-discuss] proposal for Intel GPU K8s device plugin support in StarlingX

[An, Ran1]

Hi All
  here is the proposal for enabling intel-gpu-plugin on StarlingX ,  welcome suggestion and advise.

The background:
As a part of resource management, kubernetes provides a device plugin framework [2] for vendors to advertise their resources to the kubelet since version 1.8. StarlingX has already supported SR-IOV CNI plugins now [3].
Intel-gpu-plugins is a device plugin implementation [4] for intel GPU (with driver i915). Users could deploy their pods with Intel GPU resource requests or limits, if intel-gpu-plugins was integrated into StarlingX.

proposal:
Deploy intel-gpu-plugins as a daemon set with node selector "intelgpu: enabled". Kubernetes label "intelgpu: enabled" will be set automatically once the node detected supported GPU device.
Details are shown as follows:

1.      Build StarlingX plugin docker image based on [5], the implement in starlingx are [6] and [7]

2.      Deploy Intel-gpu-plugins daemon set in tasks "bringup_kubemaster" after kubernetes master has been initialized during ansible bootstrap process. Add value "import_plugins" and value list "kube_plugins" as condition of deploying Intel-gpu-plugins daemon set, so user could determine whether Intel-gpu-plugins would be enabled. Create file "/etc/platform/enabled_kube_plugins" and write list "kube_plugins" into the file after active Intel-gpu-plugin daemon if "import_plugins" is true. Partical Implement is [8]

3.      Detect supported GPU device with the help of sysinv agent and request to set kubernetes label "intelgpu: enabled" for specific node by calling sysinv conductor rpcapi. Sysinv conductor will check file "/etc/platform/enabled_kube_plugins", and set kubernetes label if the file is exist and intel-gpu-plugins is in list. Partial implement is [9]

[1] https://storyboard.openstack.org/#!/story/2005937
[2] https://kubernetes.io/docs/concepts/extend-kubernetes/compute-storage-net/device-plugins/
      https://github.com/kubernetes/community/blob/master/contributors/design-proposals/resource-management/device-plugin.md
[3] https://review.opendev.org/#/c/655495/
[4] https://github.com/intel/intel-device-plugins-for-kubernetes
[5] https://github.com/intel/intel-device-plugins-for-kubernetes/blob/master/cmd/gpu_plugin/README.md
[6] https://review.opendev.org/668803
[7] https://review.opendev.org/668808
[8] https://review.opendev.org/666510
[9] https://review.opendev.org/666511

Thanks
Ran

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.starlingx.io/pipermail/starlingx-discuss/attachments/20190715/e0053584/attachment-0001.html>