Will disabling this clean-up service avoid this issue?
systemctl disable systemd-tmpfiles-clean.timer
Thanks,
Ankush
From: Wensley, Barton <Barton.Wensley@windriver.com>
Sent: Wednesday, March 24, 2021 9:18 PM
To: Rai, Ankush <Ankush.Rai@commscope.com>; starlingx-discuss@lists.starlingx.io
Subject: RE: unable to add 3ed edge on central-cloud
My mistake – the fix you need did not make it into stx.4.0:
https://opendev.org/starlingx/distcloud/commit/e8d916756d439cf63eed52f347a2a547fc5124cb
The fix will be available in the upcoming stx.5.0 release.
Bart
From: Rai, Ankush <Ankush.Rai@commscope.com>
Sent: Wednesday, March 24, 2021 11:41 AM
To: Wensley, Barton <Barton.Wensley@windriver.com>;
starlingx-discuss@lists.starlingx.io
Subject: Re: unable to add 3ed edge on central-cloud
[Please note: This e-mail is from an EXTERNAL e-mail address]
From: Wensley, Barton <Barton.Wensley@windriver.com>
Sent: Wednesday, March 24, 2021 9:09:04 PM
To: Rai, Ankush <Ankush.Rai@commscope.com>;
starlingx-discuss@lists.starlingx.io <starlingx-discuss@lists.starlingx.io>
Subject: RE: unable to add 3ed edge on central-cloud
It looks like you may have hit this issue:
https://bugs.launchpad.net/starlingx/+bug/1883599
I believe this was fixed in stx.4.0 – are you using an earlier release?
Bart
From: Rai, Ankush <Ankush.Rai@commscope.com>
Sent: Wednesday, March 24, 2021 2:04 AM
To: starlingx-discuss@lists.starlingx.io
Subject: Re: [Starlingx-discuss] unable to add 3ed edge on central-cloud
[Please note: This e-mail is from an EXTERNAL e-mail address]
Please reply on this. I am stuck here.
Thanks,
Ankush
From: Rai, Ankush
Sent: Tuesday, March 23, 2021 5:00 PM
To: starlingx-discuss@lists.starlingx.io
Subject: unable to add 3ed edge on central-cloud
Hi,
Sub-cloud addition is failing for 3ed edge. Not understanding what is going wrong here. Please help me to debug the issue.
|
dcmanager subcloud add --bootstrap-address 10.214.5.1 --bootstrap-values $HOME/edge3/bootstrap.yml Enter the sysadmin password for the subcloud: Re-enter sysadmin password to confirm: +-----------------------------+----------------------------+ | Field | Value | +-----------------------------+----------------------------+ | id | 13 | | name | edge3 | | description | StarlingX edge3 | | location | COMMSCOPE_LAB | | software_version | 20.06 | | management | unmanaged | | availability | offline | | deploy_status | not-deployed | | management_subnet | 10.214.6.0/24 | | management_start_ip | 10.214.6.2 | | management_end_ip | 10.214.6.20 | | management_gateway_ip | 10.214.6.254 | | systemcontroller_gateway_ip | 10.214.2.254 | | group_id | 1 | | created_at | 2021-03-23 11:16:56.943669 | | updated_at | None | +-----------------------------+----------------------------+ |
|
subcloud list +----+-------+------------+--------------+--------------------+---------+ | id | name | management | availability | deploy status | sync | +----+-------+------------+--------------+--------------------+---------+ | 3 | edge1 | managed | online | complete | in-sync | | 6 | edge | managed | offline | complete | unknown | | 13 | edge3 | unmanaged | offline | deploy-prep-failed | unknown | +----+-------+------------+--------------+--------------------+---------+ |
|
==> /var/log/dcmanager/dcmanager.log <== 2021-03-23 11:16:56.949 118088 INFO eventlet.wsgi.server [req-4dbde86f-545a-4e34-948a-401bf7795450 8a78866e123740abaa939956db6a949b - - default default] 10.214.2.3 "POST /v1.0/subclouds/ HTTP/1.1" status: 200
len: 770 time: 2.0502820 2021-03-23 11:16:56.950 116829 INFO dcmanager.manager.service [req-4dbde86f-545a-4e34-948a-401bf7795450 8a78866e123740abaa939956db6a949b - - default default] Handling add_subcloud request for: edge3 2021-03-23 11:16:56.951 116829 INFO dcmanager.manager.subcloud_manager [req-4dbde86f-545a-4e34-948a-401bf7795450 8a78866e123740abaa939956db6a949b - - default default] Adding subcloud edge3. 2021-03-23 11:16:58.859 116829 INFO dccommon.drivers.openstack.sysinv_v1 [req-4dbde86f-545a-4e34-948a-401bf7795450 8a78866e123740abaa939956db6a949b - - default default] Creating route: interface: 814ed6e8-d700-4140-9c62-0d3d18535664
dest: 10.214.6.0/24 gateway: 10.214.2.254 metric 1 2021-03-23 11:17:05.067 116829 INFO dccommon.drivers.openstack.sysinv_v1 [req-4dbde86f-545a-4e34-948a-401bf7795450 8a78866e123740abaa939956db6a949b - - default default] Creating route: interface: 14e4aec4-2f41-4bc9-9ec9-d704ac460787
dest: 10.214.6.0/24 gateway: 10.214.2.254 metric 1 2021-03-23 11:17:12.880 116829 ERROR dcmanager.manager.subcloud_manager [req-4dbde86f-545a-4e34-948a-401bf7795450 8a78866e123740abaa939956db6a949b - - default default] Failed to create subcloud edge3: ConfigException:
File does not exists: /tmp/tmpADel38 2021-03-23 11:17:12.880 116829 ERROR dcmanager.manager.subcloud_manager Traceback (most recent call last): 2021-03-23 11:17:12.880 116829 ERROR dcmanager.manager.subcloud_manager File "/usr/lib/python2.7/site-packages/dcmanager/manager/subcloud_manager.py", line 353, in add_subcloud 2021-03-23 11:17:12.880 116829 ERROR dcmanager.manager.subcloud_manager self._create_intermediate_ca_cert(payload) 2021-03-23 11:17:12.880 116829 ERROR dcmanager.manager.subcloud_manager File "/usr/lib/python2.7/site-packages/dcmanager/manager/subcloud_manager.py", line 160, in _create_intermediate_ca_cert 2021-03-23 11:17:12.880 116829 ERROR dcmanager.manager.subcloud_manager kube.apply_cert_manager_certificate(CERT_NAMESPACE, cert_name, cert) 2021-03-23 11:17:12.880 116829 ERROR dcmanager.manager.subcloud_manager File "/usr/lib/python2.7/site-packages/dccommon/kubeoperator.py", line 112, in apply_cert_manager_certificate 2021-03-23 11:17:12.880 116829 ERROR dcmanager.manager.subcloud_manager custom_object_api = self._get_kubernetesclient_custom_objects() 2021-03-23 11:17:12.880 116829 ERROR dcmanager.manager.subcloud_manager File "/usr/lib/python2.7/site-packages/dccommon/kubeoperator.py", line 52, in _get_kubernetesclient_custom_objects 2021-03-23 11:17:12.880 116829 ERROR dcmanager.manager.subcloud_manager self._load_kube_config() 2021-03-23 11:17:12.880 116829 ERROR dcmanager.manager.subcloud_manager File "/usr/lib/python2.7/site-packages/dccommon/kubeoperator.py", line 31, in _load_kube_config 2021-03-23 11:17:12.880 116829 ERROR dcmanager.manager.subcloud_manager config.load_kube_config(KUBE_CONFIG_PATH) 2021-03-23 11:17:12.880 116829 ERROR dcmanager.manager.subcloud_manager File "/usr/lib/python2.7/site-packages/kubernetes/config/kube_config.py", line 531, in load_kube_config 2021-03-23 11:17:12.880 116829 ERROR dcmanager.manager.subcloud_manager loader.load_and_set(config) 2021-03-23 11:17:12.880 116829 ERROR dcmanager.manager.subcloud_manager File "/usr/lib/python2.7/site-packages/kubernetes/config/kube_config.py", line 413, in load_and_set 2021-03-23 11:17:12.880 116829 ERROR dcmanager.manager.subcloud_manager self._load_cluster_info() 2021-03-23 11:17:12.880 116829 ERROR dcmanager.manager.subcloud_manager File "/usr/lib/python2.7/site-packages/kubernetes/config/kube_config.py", line 395, in _load_cluster_info 2021-03-23 11:17:12.880 116829 ERROR dcmanager.manager.subcloud_manager file_base_path=self._config_base_path).as_file() 2021-03-23 11:17:12.880 116829 ERROR dcmanager.manager.subcloud_manager File "/usr/lib/python2.7/site-packages/kubernetes/config/kube_config.py", line 116, in as_file 2021-03-23 11:17:12.880 116829 ERROR dcmanager.manager.subcloud_manager raise ConfigException("File does not exists: %s" % self._file) 2021-03-23 11:17:12.880 116829 ERROR dcmanager.manager.subcloud_manager ConfigException: File does not exists: /tmp/tmpADel38 2021-03-23 11:17:12.880 116829 ERROR dcmanager.manager.subcloud_manager 2021-03-23 11:17:20.778 118089 INFO eventlet.wsgi.server [req-59e03ba0-a0ca-4194-9b04-21b62dffe4bd 8a78866e123740abaa939956db6a949b - - default default] 10.214.2.3,10.214.2.3 "GET /v1.0/alarms/ HTTP/1.1" status:
200 len: 728 time: 0.0075271 2021-03-23 11:17:20.785 118088 INFO eventlet.wsgi.server [req-eeb598dd-38d7-4165-af67-f663cf513828 8a78866e123740abaa939956db6a949b - - default default] 10.214.2.3,10.214.2.3 "GET /v1.0/subclouds/ HTTP/1.1" status:
200 len: 3023 time: 0.0140100 |
|
+----+-------+-----------------+---------------+------------------+------------+--------------+--------------------+-------------------+---------------------+-------------------+-----------------------+-----------------------------+----------+----------------------------+----------------------------+ | id | name | description | location | software_version | management | availability | deploy_status | management_subnet | management_start_ip | management_end_ip | management_gateway_ip | systemcontroller_gateway_ip
| group_id | created_at | updated_at | +----+-------+-----------------+---------------+------------------+------------+--------------+--------------------+-------------------+---------------------+-------------------+-----------------------+-----------------------------+----------+----------------------------+----------------------------+ | 3 | edge1 | StarlingX edge1 | COMMSCOPE_LAB | 20.06 | managed | online | complete | 10.214.4.0/24 | 10.214.4.2 | 10.214.4.20 | 10.214.4.254 | 10.214.2.254
| 1 | 2021-02-24 08:17:01.698138 | 2021-03-12 11:21:57.684272 | | 6 | edge | StarlingX edge1 | COMMSCOPE_LAB | 20.06 | managed | offline | complete | 10.207.7.0/24 | 10.207.7.2 | 10.207.7.20 | 10.207.7.254 | 10.214.2.254
| 1 | 2021-03-03 18:56:52.779913 | 2021-03-23 11:18:51.674364 | | 13 | edge3 | StarlingX edge3 | COMMSCOPE_LAB | 20.06 | unmanaged | offline | deploy-prep-failed | 10.214.6.0/24 | 10.214.6.2 | 10.214.6.20 | 10.214.6.254 | 10.214.2.254
| 1 | 2021-03-23 11:16:56.943669 | 2021-03-23 11:17:12.899548 | +----+-------+-----------------+---------------+------------------+------------+--------------+--------------------+-------------------+---------------------+-------------------+-----------------------+-----------------------------+----------+----------------------------+----------------------------+ |
|
cat /home/sysadmin/edge3/bootstrap.yml system_mode: duplex name: "edge3" description: "StarlingX edge3" location: "COMMSCOPE_LAB" management_subnet: 10.214.6.0/24 management_floating_address: 10.214.6.1 management_start_address: 10.214.6.2 management_end_address: 10.214.6.20 management_gateway_address: 10.214.6.254 external_oam_subnet: 10.214.5.0/24 external_oam_gateway_address: 10.214.5.254 external_oam_floating_address: 10.214.5.1 external_oam_start_address: 10.214.5.2 external_oam_end_address: 10.214.5.7 systemcontroller_gateway_address: 10.214.2.254 docker_registries: k8s.gcr.io: url: 10.214.1.1/starlingx quay.io: url: 10.214.1.1/starlingx docker.io: url: 10.214.1.1/starlingx gcr.io: url: 10.214.1.1/starlingx defaults: url: 10.214.1.1/starlingx username: admin password: XYZ ssl_ca_cert: /home/sysadmin/centralharborcert/ca.crt |
Thanks,
Ankush