[Starlingx-discuss] unable to add 3ed edge on central-cloud

Wensley, Barton Barton.Wensley at windriver.com
Thu Mar 25 11:12:58 UTC 2021


Make the change from the fix I referenced below to /usr/lib/ocf/resource.d/openstack/dcmanager-manager (on both controllers) and then restart the dcmanager-manager on the active controller with:
sm-restart service dcmanager-manager

Bart

From: Rai, Ankush <Ankush.Rai at commscope.com>
Sent: Thursday, March 25, 2021 6:07 AM
To: Wensley, Barton <Barton.Wensley at windriver.com>; starlingx-discuss at lists.starlingx.io
Cc: Sambandan, Devaraj <Devaraj.Sambandan at commscope.com>; Dharwadkar, Sriram <Sriram.Dharwadkar at commscope.com>
Subject: RE: unable to add 3ed edge on central-cloud

[Please note: This e-mail is from an EXTERNAL e-mail address]
Please suggest any work-around for this issue.

Thanks,
Ankush

From: Rai, Ankush
Sent: Wednesday, March 24, 2021 10:32 PM
To: Wensley, Barton <Barton.Wensley at windriver.com<mailto:Barton.Wensley at windriver.com>>; starlingx-discuss at lists.starlingx.io<mailto:starlingx-discuss at lists.starlingx.io>
Subject: RE: unable to add 3ed edge on central-cloud

Will disabling this clean-up service avoid this issue?

systemctl  disable systemd-tmpfiles-clean.timer

Thanks,
Ankush

From: Wensley, Barton <Barton.Wensley at windriver.com<mailto:Barton.Wensley at windriver.com>>
Sent: Wednesday, March 24, 2021 9:18 PM
To: Rai, Ankush <Ankush.Rai at commscope.com<mailto:Ankush.Rai at commscope.com>>; starlingx-discuss at lists.starlingx.io<mailto:starlingx-discuss at lists.starlingx.io>
Subject: RE: unable to add 3ed edge on central-cloud


My mistake - the fix you need did not make it into stx.4.0:
https://opendev.org/starlingx/distcloud/commit/e8d916756d439cf63eed52f347a2a547fc5124cb<https://secure-web.cisco.com/18Nni-eUAIx-l9wQk3zAnso8hjxCFhb8Q9BlphAA7uJsqvKD8zGOz0MFTa0F16KE7Df3u9LkTe2OU4btLuGKmPF1d_4LHMEyTC4nUK1srBnohRNYssK4ZPfVkdHleADWz0tRfXnM_bFPQlyQZAZi5QLCai3PUILdl7DWBEW7nsXvRc69W3top9dxdT4L50A9geFntYy7GUteevSsQeWgjrJ2RBfbPvmpFUZMgUFNiuRA1bUMp_MLk1ZKrwURDed7SoUeO7qxAldVVNeFhhmH-BgsrRvi8jJ_qSsbnodXmTk54RLjcH8T-POT-k8Gw5dB9/https%3A%2F%2Fopendev.org%2Fstarlingx%2Fdistcloud%2Fcommit%2Fe8d916756d439cf63eed52f347a2a547fc5124cb>

The fix will be available in the upcoming stx.5.0 release.

Bart

From: Rai, Ankush <Ankush.Rai at commscope.com<mailto:Ankush.Rai at commscope.com>>
Sent: Wednesday, March 24, 2021 11:41 AM
To: Wensley, Barton <Barton.Wensley at windriver.com<mailto:Barton.Wensley at windriver.com>>; starlingx-discuss at lists.starlingx.io<mailto:starlingx-discuss at lists.starlingx.io>
Subject: Re: unable to add 3ed edge on central-cloud

[Please note: This e-mail is from an EXTERNAL e-mail address]
No it's 4.0 only.
20.06

Thanks
Ankush


Get Outlook for iOS<https://secure-web.cisco.com/1X2WSa7uNw-o03_A0bVDolfynbOhgObwmUoyAdOHWwvKf9I7qsvsdPQDrZdFf1d0AM7UE8XDOzBziGkU0T5UD1jHdtK-lXVCOJnZ15nmi3m4aKMJ38mtfZq7Ngo1mtJIcMm2u7m5wCK9_wwUFfIxBiw-4VxWHl4JYVXXqHCwfcJjifX_cubluYcblY_-Do8VUoM2B92XvMVJ0pq879E7CnL7UXu0WIiIvdzruxZCB0YO4G-H_8gWtsCwAJPNCqcXeaYsKb-Nla4iyjocUsKljQxBT6djF0zAL-xN5LWjd2ljTrVWHg6-DZXf7rEnL0gck/https%3A%2F%2Faka.ms%2Fo0ukef>
________________________________
From: Wensley, Barton <Barton.Wensley at windriver.com<mailto:Barton.Wensley at windriver.com>>
Sent: Wednesday, March 24, 2021 9:09:04 PM
To: Rai, Ankush <Ankush.Rai at commscope.com<mailto:Ankush.Rai at commscope.com>>; starlingx-discuss at lists.starlingx.io<mailto:starlingx-discuss at lists.starlingx.io> <starlingx-discuss at lists.starlingx.io<mailto:starlingx-discuss at lists.starlingx.io>>
Subject: RE: unable to add 3ed edge on central-cloud


It looks like you may have hit this issue: https://bugs.launchpad.net/starlingx/+bug/1883599<https://secure-web.cisco.com/1aPpINLnbONoKiUQZQ5RPYFkqL5v5_ksBOFPp0JjlWdSeWfkRO9HNvZS17mcVuP4o7qyxuuThVBwGubwqW8rwdZR2Oo3WSx1zAWVoVpJvHKDHiPQNR29CSG8-qeZBsJS-zeS_M51wDT2yB2Lr6vnaKPZqQ4twxIkC8Uwb2L_-ep_9acPxCoPIJelY3sTWW0Zv4N21SlJaZogtTn1KLqXASNFrR1gqSnTM6LZCh6hYidDSMcU2rnwYgX-0nQ9_mjjIIh7FzyL6nljIPMSN79xKwSu4C9g_wdqiMVzn4KpHsKlWmqu98kpz7L3sNGpmwDDS/https%3A%2F%2Fbugs.launchpad.net%2Fstarlingx%2F%2Bbug%2F1883599>



I believe this was fixed in stx.4.0 - are you using an earlier release?



Bart



From: Rai, Ankush <Ankush.Rai at commscope.com<mailto:Ankush.Rai at commscope.com>>
Sent: Wednesday, March 24, 2021 2:04 AM
To: starlingx-discuss at lists.starlingx.io<mailto:starlingx-discuss at lists.starlingx.io>
Subject: Re: [Starlingx-discuss] unable to add 3ed edge on central-cloud



[Please note: This e-mail is from an EXTERNAL e-mail address]

Please reply on this. I am stuck here.



Thanks,

Ankush

From: Rai, Ankush
Sent: Tuesday, March 23, 2021 5:00 PM
To: starlingx-discuss at lists.starlingx.io<mailto:starlingx-discuss at lists.starlingx.io>
Subject: unable to add 3ed edge on central-cloud



Hi,



Sub-cloud addition is failing for 3ed edge. Not understanding what is going wrong here. Please help me to debug the issue.





dcmanager subcloud add --bootstrap-address 10.214.5.1 --bootstrap-values $HOME/edge3/bootstrap.yml

Enter the sysadmin password for the subcloud:

Re-enter sysadmin password to confirm:

+-----------------------------+----------------------------+

| Field                       | Value                      |

+-----------------------------+----------------------------+

| id                          | 13                         |

| name                        | edge3                      |

| description                 | StarlingX edge3            |

| location                    | COMMSCOPE_LAB              |

| software_version            | 20.06                      |

| management                  | unmanaged                  |

| availability                | offline                    |

| deploy_status               | not-deployed               |

| management_subnet           | 10.214.6.0/24              |

| management_start_ip         | 10.214.6.2                 |

| management_end_ip           | 10.214.6.20                |

| management_gateway_ip       | 10.214.6.254               |

| systemcontroller_gateway_ip | 10.214.2.254               |

| group_id                    | 1                          |

| created_at                  | 2021-03-23 11:16:56.943669 |

| updated_at                  | None                       |

+-----------------------------+----------------------------+





subcloud list

+----+-------+------------+--------------+--------------------+---------+

| id | name  | management | availability | deploy status      | sync    |

+----+-------+------------+--------------+--------------------+---------+

|  3 | edge1 | managed    | online       | complete           | in-sync |

|  6 | edge  | managed    | offline      | complete           | unknown |

| 13 | edge3 | unmanaged  | offline      | deploy-prep-failed | unknown |

+----+-------+------------+--------------+--------------------+---------+







==> /var/log/dcmanager/dcmanager.log <==

2021-03-23 11:16:56.949 118088 INFO eventlet.wsgi.server [req-4dbde86f-545a-4e34-948a-401bf7795450 8a78866e123740abaa939956db6a949b - - default default] 10.214.2.3 "POST /v1.0/subclouds/ HTTP/1.1" status: 200  len: 770 time: 2.0502820

2021-03-23 11:16:56.950 116829 INFO dcmanager.manager.service [req-4dbde86f-545a-4e34-948a-401bf7795450 8a78866e123740abaa939956db6a949b - - default default] Handling add_subcloud request for: edge3

2021-03-23 11:16:56.951 116829 INFO dcmanager.manager.subcloud_manager [req-4dbde86f-545a-4e34-948a-401bf7795450 8a78866e123740abaa939956db6a949b - - default default] Adding subcloud edge3.

2021-03-23 11:16:58.859 116829 INFO dccommon.drivers.openstack.sysinv_v1 [req-4dbde86f-545a-4e34-948a-401bf7795450 8a78866e123740abaa939956db6a949b - - default default] Creating route: interface: 814ed6e8-d700-4140-9c62-0d3d18535664 dest: 10.214.6.0/24 gateway: 10.214.2.254 metric 1

2021-03-23 11:17:05.067 116829 INFO dccommon.drivers.openstack.sysinv_v1 [req-4dbde86f-545a-4e34-948a-401bf7795450 8a78866e123740abaa939956db6a949b - - default default] Creating route: interface: 14e4aec4-2f41-4bc9-9ec9-d704ac460787 dest: 10.214.6.0/24 gateway: 10.214.2.254 metric 1

2021-03-23 11:17:12.880 116829 ERROR dcmanager.manager.subcloud_manager [req-4dbde86f-545a-4e34-948a-401bf7795450 8a78866e123740abaa939956db6a949b - - default default] Failed to create subcloud edge3: ConfigException: File does not exists: /tmp/tmpADel38

2021-03-23 11:17:12.880 116829 ERROR dcmanager.manager.subcloud_manager Traceback (most recent call last):

2021-03-23 11:17:12.880 116829 ERROR dcmanager.manager.subcloud_manager   File "/usr/lib/python2.7/site-packages/dcmanager/manager/subcloud_manager.py", line 353, in add_subcloud

2021-03-23 11:17:12.880 116829 ERROR dcmanager.manager.subcloud_manager     self._create_intermediate_ca_cert(payload)

2021-03-23 11:17:12.880 116829 ERROR dcmanager.manager.subcloud_manager   File "/usr/lib/python2.7/site-packages/dcmanager/manager/subcloud_manager.py", line 160, in _create_intermediate_ca_cert

2021-03-23 11:17:12.880 116829 ERROR dcmanager.manager.subcloud_manager     kube.apply_cert_manager_certificate(CERT_NAMESPACE, cert_name, cert)

2021-03-23 11:17:12.880 116829 ERROR dcmanager.manager.subcloud_manager   File "/usr/lib/python2.7/site-packages/dccommon/kubeoperator.py", line 112, in apply_cert_manager_certificate

2021-03-23 11:17:12.880 116829 ERROR dcmanager.manager.subcloud_manager     custom_object_api = self._get_kubernetesclient_custom_objects()

2021-03-23 11:17:12.880 116829 ERROR dcmanager.manager.subcloud_manager   File "/usr/lib/python2.7/site-packages/dccommon/kubeoperator.py", line 52, in _get_kubernetesclient_custom_objects

2021-03-23 11:17:12.880 116829 ERROR dcmanager.manager.subcloud_manager     self._load_kube_config()

2021-03-23 11:17:12.880 116829 ERROR dcmanager.manager.subcloud_manager   File "/usr/lib/python2.7/site-packages/dccommon/kubeoperator.py", line 31, in _load_kube_config

2021-03-23 11:17:12.880 116829 ERROR dcmanager.manager.subcloud_manager     config.load_kube_config(KUBE_CONFIG_PATH)

2021-03-23 11:17:12.880 116829 ERROR dcmanager.manager.subcloud_manager   File "/usr/lib/python2.7/site-packages/kubernetes/config/kube_config.py", line 531, in load_kube_config

2021-03-23 11:17:12.880 116829 ERROR dcmanager.manager.subcloud_manager     loader.load_and_set(config)

2021-03-23 11:17:12.880 116829 ERROR dcmanager.manager.subcloud_manager   File "/usr/lib/python2.7/site-packages/kubernetes/config/kube_config.py", line 413, in load_and_set

2021-03-23 11:17:12.880 116829 ERROR dcmanager.manager.subcloud_manager     self._load_cluster_info()

2021-03-23 11:17:12.880 116829 ERROR dcmanager.manager.subcloud_manager   File "/usr/lib/python2.7/site-packages/kubernetes/config/kube_config.py", line 395, in _load_cluster_info

2021-03-23 11:17:12.880 116829 ERROR dcmanager.manager.subcloud_manager     file_base_path=self._config_base_path).as_file()

2021-03-23 11:17:12.880 116829 ERROR dcmanager.manager.subcloud_manager   File "/usr/lib/python2.7/site-packages/kubernetes/config/kube_config.py", line 116, in as_file

2021-03-23 11:17:12.880 116829 ERROR dcmanager.manager.subcloud_manager     raise ConfigException("File does not exists: %s" % self._file)

2021-03-23 11:17:12.880 116829 ERROR dcmanager.manager.subcloud_manager ConfigException: File does not exists: /tmp/tmpADel38

2021-03-23 11:17:12.880 116829 ERROR dcmanager.manager.subcloud_manager

2021-03-23 11:17:20.778 118089 INFO eventlet.wsgi.server [req-59e03ba0-a0ca-4194-9b04-21b62dffe4bd 8a78866e123740abaa939956db6a949b - - default default] 10.214.2.3,10.214.2.3 "GET /v1.0/alarms/ HTTP/1.1" status: 200  len: 728 time: 0.0075271

2021-03-23 11:17:20.785 118088 INFO eventlet.wsgi.server [req-eeb598dd-38d7-4165-af67-f663cf513828 8a78866e123740abaa939956db6a949b - - default default] 10.214.2.3,10.214.2.3 "GET /v1.0/subclouds/ HTTP/1.1" status: 200  len: 3023 time: 0.0140100



+----+-------+-----------------+---------------+------------------+------------+--------------+--------------------+-------------------+---------------------+-------------------+-----------------------+-----------------------------+----------+----------------------------+----------------------------+

| id | name  | description     | location      | software_version | management | availability | deploy_status      | management_subnet | management_start_ip | management_end_ip | management_gateway_ip | systemcontroller_gateway_ip | group_id | created_at                 | updated_at                 |

+----+-------+-----------------+---------------+------------------+------------+--------------+--------------------+-------------------+---------------------+-------------------+-----------------------+-----------------------------+----------+----------------------------+----------------------------+

|  3 | edge1 | StarlingX edge1 | COMMSCOPE_LAB | 20.06            | managed    | online       | complete           | 10.214.4.0/24     | 10.214.4.2          | 10.214.4.20       | 10.214.4.254          | 10.214.2.254                |        1 | 2021-02-24 08:17:01.698138 | 2021-03-12 11:21:57.684272 |

|  6 | edge  | StarlingX edge1 | COMMSCOPE_LAB | 20.06            | managed    | offline      | complete           | 10.207.7.0/24     | 10.207.7.2          | 10.207.7.20       | 10.207.7.254          | 10.214.2.254                |        1 | 2021-03-03 18:56:52.779913 | 2021-03-23 11:18:51.674364 |

| 13 | edge3 | StarlingX edge3 | COMMSCOPE_LAB | 20.06            | unmanaged  | offline      | deploy-prep-failed | 10.214.6.0/24     | 10.214.6.2          | 10.214.6.20       | 10.214.6.254          | 10.214.2.254                |        1 | 2021-03-23 11:16:56.943669 | 2021-03-23 11:17:12.899548 |

+----+-------+-----------------+---------------+------------------+------------+--------------+--------------------+-------------------+---------------------+-------------------+-----------------------+-----------------------------+----------+----------------------------+----------------------------+





cat  /home/sysadmin/edge3/bootstrap.yml

system_mode: duplex

name: "edge3"

description: "StarlingX edge3"

location: "COMMSCOPE_LAB"



management_subnet: 10.214.6.0/24

management_floating_address: 10.214.6.1

management_start_address: 10.214.6.2

management_end_address: 10.214.6.20

management_gateway_address: 10.214.6.254



external_oam_subnet: 10.214.5.0/24

external_oam_gateway_address: 10.214.5.254

external_oam_floating_address: 10.214.5.1

external_oam_start_address: 10.214.5.2

external_oam_end_address:   10.214.5.7



systemcontroller_gateway_address: 10.214.2.254



docker_registries:

  k8s.gcr.io:

    url: 10.214.1.1/starlingx

  quay.io:

    url: 10.214.1.1/starlingx

  docker.io:

    url: 10.214.1.1/starlingx

  gcr.io:

    url: 10.214.1.1/starlingx

  defaults:

    url: 10.214.1.1/starlingx

    username: admin

    password: XYZ



ssl_ca_cert: /home/sysadmin/centralharborcert/ca.crt







Thanks,

Ankush
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.starlingx.io/pipermail/starlingx-discuss/attachments/20210325/2524fdbd/attachment-0001.html>


More information about the Starlingx-discuss mailing list