[Starlingx-discuss] [STARLINGX 4.0] APPLICATION UPLOAD GETTING FAILED - with LAG Interface

Thu Feb 11 04:12:18 UTC 2021

Hi Lokendra, Anirudh, and team,
Thanks for submitting the STX Documentation launchpad for your issue:  https://bugs.launchpad.net/starlingx/+bug/1915285

We discussed it at our Docs meeting and wanted to point you to the Node Management Guide for more info.
Here are some relevant sections that may be helpful:
https://docs.starlingx.io/node_management/starlingx-kubernetes/node_interfaces/interface-provisioning.html
https://docs.starlingx.io/node_management/starlingx-kubernetes/node_interfaces/link-aggregation-settings.html
https://docs.starlingx.io/node_management/starlingx-kubernetes/node_interfaces/configuring-aggregated-ethernet-interfaces-using-the-cli.html

Good luck,
Mary Camp
Kelly Services Technical Writer | maryx.camp at intel.com<mailto:maryx.camp at intel.com>

From: Lokendra Singh Rathour <Lokendra.Rathour at hsc.com>
Sent: Monday, February 8, 2021 11:49 PM
To: starlingx-discuss at lists.starlingx.io
Subject: Re: [Starlingx-discuss] [STARLINGX 4.0] APPLICATION UPLOAD GETTING FAILED - with LAG Interface

Hi Team,
Any update with respect the Query raised!!

Best Regards,
Lokendra

From: Lokendra Singh Rathour
Sent: Friday, February 5, 2021 5:27 PM
To: starlingx-discuss at lists.starlingx.io<mailto:starlingx-discuss at lists.starlingx.io>
Subject: [STARLINGX 4.0] APPLICATION UPLOAD GETTING FAILED - with LAG Interface

Hello Team,
We are trying to setup dedicated Storage setup using StarlingX 4.0 over which we have certain observations/Error during the time of worker Node Configuration for the LAG type Data interfaces. Though we have tweaked the procedure a bit and successfully unlocked the Worker Nodes,we are facing the error in upload the STX Application packages. Steps as followed:

STEP 1:
We have created bonds for Data Network on Worker Node(reference document: https://docs.starlingx.io/configuration/host_interface_network_config.html#:~:text=When%20a%20host%20is%20added,system%20host%2Dif%2Dadd.)

     *   system host-if-add worker-1 -m 1500 -a active_standby data1bond ae eth1 eth2
     *   system host-if-add worker-1 -m 1500 -a active_standby data2bond ae eth3 eth4

STEP 2:
Then further as per the main documents when Configuring data interfaces for worker nodes ( reference document: https://docs.starlingx.io/deploy_install_guides/r4_release/bare_metal/dedicated_storage_install_kubernetes.html)

  *   SPL=/tmp/tmp-system-port-list
  *   SPIL=/tmp/tmp-system-host-if-list

Above files does not have the information of the created Bond interfaces for data Nodes,

Further in order to get the the value of  ${DATA0IFUUID} and ${DATA1IFUUID} below mentioned commands are executed:

DATA0IF=<DATA-0-PORT>
DATA1IF=<DATA-1-PORT>
PHYSNET0='physnet0'
PHYSNET1='physnet1'
SPL=/tmp/tmp-system-port-list
SPIL=/tmp/tmp-system-host-if-list

# configure the datanetworks in sysinv, prior to referencing it
# in the ``system host-if-modify`` command'.
system datanetwork-add ${PHYSNET0} vlan
system datanetwork-add ${PHYSNET1} vlan

for NODE in worker-0 worker-1; do
  echo "Configuring interface for: $NODE"
  set -ex
  system host-port-list ${NODE} --nowrap > ${SPL}
  system host-if-list -a ${NODE} --nowrap > ${SPIL}
  DATA0PCIADDR=$(cat $SPL | grep $DATA0IF |awk '{print $8}')
  DATA1PCIADDR=$(cat $SPL | grep $DATA1IF |awk '{print $8}')
  DATA0PORTUUID=$(cat $SPL | grep ${DATA0PCIADDR} | awk '{print $2}')
  DATA1PORTUUID=$(cat $SPL | grep ${DATA1PCIADDR} | awk '{print $2}')
  DATA0PORTNAME=$(cat $SPL | grep ${DATA0PCIADDR} | awk '{print $4}')
  DATA1PORTNAME=$(cat $SPL | grep ${DATA1PCIADDR} | awk '{print $4}')
  DATA0IFUUID=$(cat $SPIL | awk -v DATA0PORTNAME=$DATA0PORTNAME '($12 ~ DATA0PORTNAME) {print $2}')
  DATA1IFUUID=$(cat $SPIL | awk -v DATA1PORTNAME=$DATA1PORTNAME '($12 ~ DATA1PORTNAME) {print $2}')
  system host-if-modify -m 1500 -n data0 -c data ${NODE} ${DATA0IFUUID}
  system host-if-modify -m 1500 -n data1 -c data ${NODE} ${DATA1IFUUID}
  system interface-datanetwork-assign ${NODE} ${DATA0IFUUID} ${PHYSNET0}
  system interface-datanetwork-assign ${NODE} ${DATA1IFUUID} ${PHYSNET1}
  set +ex
done

But in our case these values in the variable(${DATA0IFUUID} and ${DATA1IFUUID}) were not getting populated, therefore we have used the  UUID received from command :
              System host-if-list worker-0
              System host-if-list worker-1

We observed that  UUID of any ethernet interface mentioned in the file (SPIL=/tmp/tmp-system-host-if-list
) matches with the UUID obtained by running the above command(System host-if-list worker-0 ) , so as we did not get the UUID from the file in case LAG Bonds we passed the Values of UUID directly.

STEP 3:

Using the change, we were able to successfully unlock the worker Nodes.

[sysadmin at controller-0 ~(keystone_admin)]$ system host-list
+----+--------------+-------------+----------------+-------------+--------------+
| id | hostname     | personality | administrative | operational | availability |
+----+--------------+-------------+----------------+-------------+--------------+
| 1  | controller-0 | controller  | unlocked       | enabled     | available    |
| 2  | controller-1 | controller  | unlocked       | enabled     | available    |
| 3  | storage-0    | storage     | unlocked       | enabled     | available    |
| 4  | storage-1    | storage     | unlocked       | enabled     | available    |
| 5  | worker-0     | worker      | unlocked       | enabled     | available    |
| 6  | worker-1     | worker      | unlocked       | enabled     | available    |
+----+--------------+-------------+----------------+-------------+--------------+

STEP 4:
Then Further while running the application-upload:
              system application-upload stx-openstack-<version>-centos-stable-versioned.tgz

Error as below was seen
Error:
2021-02-05 07:46:40.318 113059 ERROR sysinv.conductor.kube_app   File "/usr/lib64/python2.7/site-packages/sysinv/helm/helm.py", line 317, in _get_helm_chart_overrides
2021-02-05 07:46:40.318 113059 ERROR sysinv.conductor.kube_app     cnamespace))
2021-02-05 07:46:40.318 113059 ERROR sysinv.conductor.kube_app   File "/opt/platform/helm/20.06/stx-openstack/1.0-49-centos-stable-versioned/plugins/k8sapp_openstack/helm/neutron.py", line 45, in get_overrides
2021-02-05 07:46:40.318 113059 ERROR sysinv.conductor.kube_app     'hosts': self._get_per_host_overrides()
2021-02-05 07:46:40.318 113059 ERROR sysinv.conductor.kube_app   File "/opt/platform/helm/20.06/stx-openstack/1.0-49-centos-stable-versioned/plugins/k8sapp_openstack/helm/neutron.py", line 105, in _get_per_host_overrides
2021-02-05 07:46:40.318 113059 ERROR sysinv.conductor.kube_app     'auto_bridge_add': self._get_host_bridges(host)})
2021-02-05 07:46:40.318 113059 ERROR sysinv.conductor.kube_app   File "/opt/platform/helm/20.06/stx-openstack/1.0-49-centos-stable-versioned/plugins/k8sapp_openstack/helm/neutron.py", line 141, in _get_host_bridges
2021-02-05 07:46:40.318 113059 ERROR sysinv.conductor.kube_app     port_name = self._get_interface_port_name(iface)
2021-02-05 07:46:40.318 113059 ERROR sysinv.conductor.kube_app   File "/opt/platform/helm/20.06/stx-openstack/1.0-49-centos-stable-versioned/plugins/k8sapp_openstack/helm/neutron.py", line 280, in _get_interface_port_name
2021-02-05 07:46:40.318 113059 ERROR sysinv.conductor.kube_app     assert iface['iftype'] == constants.INTERFACE_TYPE_ETHERNET
2021-02-05 07:46:40.318 113059 ERROR sysinv.conductor.kube_app AssertionError
2021-02-05 07:46:40.318 113059 ERROR sysinv.conductor.kube_app

Kindly check and help in advising the way forward here w.r.t. as

  1.  does starlingx support for the bond interface for the Data Networks ?
     *   if yes then do we have any supported document w.r.t the same.

Reference Documents:
For Deployment: https://docs.starlingx.io/deploy_install_guides/r4_release/bare_metal/dedicated_storage_install_kubernetes.html
For LAG : : https://docs.starlingx.io/configuration/host_interface_network_config.html#:~:text=When%20a%20host%20is%20added,system%20host%2Dif%2Dadd
DISCLAIMER: This electronic message and all of its contents, contains information which is privileged, confidential or otherwise protected from disclosure. The information contained in this electronic mail transmission is intended for use only by the individual or entity to which it is addressed. If you are not the intended recipient or may have received this electronic mail transmission in error, please notify the sender immediately and delete / destroy all copies of this electronic mail transmission without disclosing, copying, distributing, forwarding, printing or retaining any part of it. Hughes Systique accepts no responsibility for loss or damage arising from the use of the information transmitted by this email including damage from virus.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.starlingx.io/pipermail/starlingx-discuss/attachments/20210211/c70099a4/attachment-0001.html>