[Starlingx-discuss] proposal for Intel GPU K8s device plugin support in StarlingX

[Yong Hu]

pls see my comments.

On 17/07/2019 9:08 AM, Victor Rodriguez wrote:
> On Mon, Jul 15, 2019 at 3:37 AM An, Ran1 <ran1.an at intel.com> wrote:
>>
>> Hi All
>>
>>    here is the proposal for enabling intel-gpu-plugin on StarlingX ,  welcome suggestion and advise.
>>
>>
>>
>> The background:
>>
>> As a part of resource management, kubernetes provides a device plugin framework [2] for vendors to advertise their resources to the kubelet since version 1.8. StarlingX has already supported SR-IOV CNI plugins now [3].
>>
>> Intel-gpu-plugins is a device plugin implementation [4] for intel GPU (with driver i915). Users could deploy their pods with Intel GPU resource requests or limits, if intel-gpu-plugins was integrated into StarlingX.
>>
> 
> +1
> 
>>
>>
>> proposal:
>>
>> Deploy intel-gpu-plugins as a daemon set with node selector “intelgpu: enabled”. Kubernetes label “intelgpu: enabled” will be set automatically once the node detected supported GPU device.
>>
>> Details are shown as follows:
>>
>> 1.      Build StarlingX plugin docker image based on [5], the implement in starlingx are [6] and [7]
>>
> 
> ok
> 
>> 2.      Deploy Intel-gpu-plugins daemon set in tasks “bringup_kubemaster” after kubernetes master has been initialized during ansible bootstrap process. Add value “import_plugins” and value list “kube_plugins” as condition of deploying Intel-gpu-plugins daemon set, so user could determine whether Intel-gpu-plugins would be enabled. Create file “/etc/platform/enabled_kube_plugins” and write list “kube_plugins” into the file after active Intel-gpu-plugin daemon if “import_plugins” is true. Partical Implement is [8]
>>
>> 3.      Detect supported GPU device with the help of sysinv agent and request to set kubernetes label “intelgpu: enabled” for specific node by calling sysinv conductor rpcapi. Sysinv conductor will check file “/etc/platform/enabled_kube_plugins”, and set kubernetes label if the file is exist and intel-gpu-plugins is in list. Partial implement is [9]
>>
> 
> I like the approach and the demo, I just have a question:
> 
> One question, what kind of workload support the container running in
> the GPU, do we have to write it in cuda? Do we have some example of
> source code that will be run inside the container that will run on the
> GPU ?
> 
No, this is Intel GPU and we don't support Cuda APIs. We can use OpenCL.
You can find Intel GPU device plugin demos 
here:https://github.com/intel/intel-device-plugins-for-kubernetes/tree/8310d84f96f9fc7b7487b2dfb9059905638c58fe/demo

> regards
> 
>>
>>
>> [1] https://storyboard.openstack.org/#!/story/2005937
>>
>> [2] https://kubernetes.io/docs/concepts/extend-kubernetes/compute-storage-net/device-plugins/
>>
>>        https://github.com/kubernetes/community/blob/master/contributors/design-proposals/resource-management/device-plugin.md
>>
>> [3] https://review.opendev.org/#/c/655495/
>>
>> [4] https://github.com/intel/intel-device-plugins-for-kubernetes
>>
>> [5] https://github.com/intel/intel-device-plugins-for-kubernetes/blob/master/cmd/gpu_plugin/README.md
>>
>> [6] https://review.opendev.org/668803
>>
>> [7] https://review.opendev.org/668808
>>
>> [8] https://review.opendev.org/666510
>>
>> [9] https://review.opendev.org/666511
>>
>>
>>
>> Thanks
>>
>> Ran
>>
>>
>>
>> _______________________________________________
>> Starlingx-discuss mailing list
>> Starlingx-discuss at lists.starlingx.io
>> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
> 
> _______________________________________________
> Starlingx-discuss mailing list
> Starlingx-discuss at lists.starlingx.io
> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
>