[Starlingx-discuss] Starlingx-discuss Digest, Vol 38, Issue 2
Chen, Haochuan Z
haochuan.z.chen at intel.com
Sun Jul 4 23:48:41 UTC 2021
HI
You can check what's the failure by this command firstly.
$ Kubectl describe po rook-ceph-mon-a-65b58876db-98jg5 -n kube-system
BR!
Martin, Chen
IOTG, Software Engineer
021-61164330
-----Original Message-----
From: starlingx-discuss-request at lists.starlingx.io <starlingx-discuss-request at lists.starlingx.io>
Sent: Thursday, July 1, 2021 8:30 PM
To: starlingx-discuss at lists.starlingx.io
Subject: Starlingx-discuss Digest, Vol 38, Issue 2
Send Starlingx-discuss mailing list submissions to
starlingx-discuss at lists.starlingx.io
To subscribe or unsubscribe via the World Wide Web, visit
http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
or, via email, send a message with subject or body 'help' to
starlingx-discuss-request at lists.starlingx.io
You can reach the person managing the list at
starlingx-discuss-owner at lists.starlingx.io
When replying, please edit your Subject line so it is more specific than "Re: Contents of Starlingx-discuss digest..."
Today's Topics:
1. rook-ceph deployment failure (Embedded Devel)
----------------------------------------------------------------------
Message: 1
Date: Thu, 01 Jul 2021 02:39:10 +0000
From: Embedded Devel <lists at optimcloud.com>
To: Starlingx-discuss at lists.starlingx.io
Subject: [Starlingx-discuss] rook-ceph deployment failure
Message-ID: <1625106963669.939601506.1886162123 at optimcloud.com>
Content-Type: text/plain; charset=utf-8; format=flowed
stx 5.0 simplex bare metal fais so 1) how to resolv, or 2) how to back out and just use normal ceph
kube-system rook-ceph-crashcollector-controller-0-96bb8b6df-cz5qf 1/1
Running 1 45h
kube-system rook-ceph-mgr-a-7b84cc65d6-bt5mq 1/1
Running 55 45h
kube-system rook-ceph-mon-a-65b58876db-98jg5 0/1
CrashLoopBackOff 520 45h
kube-system rook-ceph-operator-79fb8559-k7w8z 1/1
Running 1 45h
kube-system rook-ceph-tools-5778d7f6c-d5pwp 1/1
Running 1 45h
kube-system rook-discover-djv7r 1/1
Running 1 45h
controller-0:~$ cat
/var/log/armada/rook-ceph-apps-apply_2021-06-30-12-41-10.log
2021-06-30 12:41:11.316 68 DEBUG armada.handlers.document [-] Resolving reference /tmp/manifests/rook-ceph-apps/1.0-5/rook-ceph-apps-manifest.yaml.
resolve_reference
/usr/local/lib/python3.6/dist-packages/armada/handlers/document.py:49
2021-06-30 12:41:11.327 68 DEBUG armada.handlers.tiller [-] Using Tiller host IP: 127.0.0.1 _get_tiller_ip
/usr/local/lib/python3.6/dist-packages/armada/handlers/tiller.py:165
2021-06-30 12:41:11.328 68 DEBUG armada.handlers.tiller [-] Using Tiller host port: 24134 _get_tiller_port
/usr/local/lib/python3.6/dist-packages/armada/handlers/tiller.py:174
2021-06-30 12:41:11.328 68 DEBUG armada.handlers.tiller [-] Tiller getting gRPC insecure channel at 127.0.0.1:24134 with options:
[grpc.max_send_message_length=429496729,
grpc.max_receive_message_length=429496729] get_channel
/usr/local/lib/python3.6/dist-packages/armada/handlers/tiller.py:127
2021-06-30 12:41:11.334 68 DEBUG armada.handlers.tiller [-] Armada is using Tiller at: 127.0.0.1:24134, namespace=kube-system, timeout=300 __init__
/usr/local/lib/python3.6/dist-packages/armada/handlers/tiller.py:107
2021-06-30 12:41:11.334 68 INFO armada.handlers.lock [-] Acquiring lock
2021-06-30 12:41:11.350 68 DEBUG armada.utils.validate [-] Validating document [armada/Chart/v1] kube-system-rook-operator validate_armada_document
/usr/local/lib/python3.6/dist-packages/armada/utils/validate.py:108
2021-06-30 12:41:11.351 68 DEBUG armada.utils.validate [-] Validating document [armada/Chart/v1] kube-system-rook-ceph validate_armada_document
/usr/local/lib/python3.6/dist-packages/armada/utils/validate.py:108
2021-06-30 12:41:11.351 68 DEBUG armada.utils.validate [-] Validating document [armada/Chart/v1] kube-system-rook-ceph-provisioner validate_armada_document
/usr/local/lib/python3.6/dist-packages/armada/utils/validate.py:108
2021-06-30 12:41:11.351 68 DEBUG armada.utils.validate [-] Validating document [armada/ChartGroup/v1] starlingx-rook-charts validate_armada_document
/usr/local/lib/python3.6/dist-packages/armada/utils/validate.py:108
2021-06-30 12:41:11.351 68 DEBUG armada.utils.validate [-] Validating document [armada/Manifest/v1] rook-ceph-manifest validate_armada_document
/usr/local/lib/python3.6/dist-packages/armada/utils/validate.py:108
2021-06-30 12:41:11.352 68 DEBUG armada.utils.validate [-] Validating document [armada/Chart/v1] kube-system-rook-operator validate_armada_document
/usr/local/lib/python3.6/dist-packages/armada/utils/validate.py:108
2021-06-30 12:41:11.352 68 DEBUG armada.utils.validate [-] Validating document [armada/Chart/v1] kube-system-rook-ceph validate_armada_document
/usr/local/lib/python3.6/dist-packages/armada/utils/validate.py:108
2021-06-30 12:41:11.352 68 DEBUG armada.utils.validate [-] Validating document [armada/Chart/v1] kube-system-rook-ceph-provisioner validate_armada_document
/usr/local/lib/python3.6/dist-packages/armada/utils/validate.py:108
2021-06-30 12:41:11.353 68 DEBUG armada.utils.validate [-] Validating document [armada/ChartGroup/v1] starlingx-rook-charts validate_armada_document
/usr/local/lib/python3.6/dist-packages/armada/utils/validate.py:108
2021-06-30 12:41:11.353 68 DEBUG armada.utils.validate [-] Validating document [armada/Manifest/v1] rook-ceph-manifest validate_armada_document
/usr/local/lib/python3.6/dist-packages/armada/utils/validate.py:108
2021-06-30 12:41:11.353 68 INFO armada.handlers.armada [-] Performing pre-flight operations.
2021-06-30 12:41:11.353 68 DEBUG armada.handlers.tiller [-] Using Tiller host IP: 127.0.0.1 _get_tiller_ip
/usr/local/lib/python3.6/dist-packages/armada/handlers/tiller.py:165
2021-06-30 12:41:11.353 68 DEBUG armada.handlers.tiller [-] Getting Tiller
Status: Tiller exists tiller_status
/usr/local/lib/python3.6/dist-packages/armada/handlers/tiller.py:186
2021-06-30 12:41:11.353 68 INFO armada.handlers.armada [-] Downloading tarball from:
http://192.168.206.1:8080/helm_charts/stx-platform/rook-operator-0.1.0.tgz
2021-06-30 12:41:11.354 68 WARNING armada.handlers.armada [-] Disabling server validation certs to extract charts
2021-06-30 12:41:11.359 68 INFO armada.handlers.armada [-] Downloading tarball from:
http://192.168.206.1:8080/helm_charts/stx-platform/rook-ceph-0.1.0.tgz
2021-06-30 12:41:11.359 68 WARNING armada.handlers.armada [-] Disabling server validation certs to extract charts
2021-06-30 12:41:11.362 68 INFO armada.handlers.armada [-] Downloading tarball from:
http://192.168.206.1:8080/helm_charts/stx-platform/rook-ceph-provisioner-0.1.0.tgz
2021-06-30 12:41:11.362 68 WARNING armada.handlers.armada [-] Disabling server validation certs to extract charts
2021-06-30 12:41:11.365 68 DEBUG armada.handlers.tiller [-] Tiller
ListReleases() with timeout=300, request=limit: 32
status_codes: UNKNOWN
status_codes: DEPLOYED
status_codes: DELETED
status_codes: DELETING
status_codes: FAILED
status_codes: PENDING_INSTALL
status_codes: PENDING_UPGRADE
status_codes: PENDING_ROLLBACK
get_results
/usr/local/lib/python3.6/dist-packages/armada/handlers/tiller.py:215
2021-06-30 12:41:11.402 68 DEBUG armada.handlers.tiller [-] Found release stx-rook-ceph, version 1, status: DEPLOYED list_releases
/usr/local/lib/python3.6/dist-packages/armada/handlers/tiller.py:276
2021-06-30 12:41:11.402 68 DEBUG armada.handlers.tiller [-] Found release stx-rook-operator, version 1, status: DEPLOYED list_releases
/usr/local/lib/python3.6/dist-packages/armada/handlers/tiller.py:276
2021-06-30 12:41:11.402 68 DEBUG armada.handlers.tiller [-] Found release osh-openstack-mariadb, version 1, status: PENDING_INSTALL list_releases
/usr/local/lib/python3.6/dist-packages/armada/handlers/tiller.py:276
2021-06-30 12:41:11.402 68 DEBUG armada.handlers.tiller [-] Found release osh-openstack-ingress, version 1, status: DEPLOYED list_releases
/usr/local/lib/python3.6/dist-packages/armada/handlers/tiller.py:276
2021-06-30 12:41:11.402 68 DEBUG armada.handlers.tiller [-] Found release osh-openstack-nginx-ports-control, version 1, status: DEPLOYED list_releases
/usr/local/lib/python3.6/dist-packages/armada/handlers/tiller.py:276
2021-06-30 12:41:11.402 68 DEBUG armada.handlers.tiller [-] Found release osh-openstack-psp-rolebinding, version 1, status: DEPLOYED list_releases
/usr/local/lib/python3.6/dist-packages/armada/handlers/tiller.py:276
2021-06-30 12:41:11.402 68 DEBUG armada.handlers.tiller [-] Found release stx-cephfs-provisioner, version 1, status: DEPLOYED list_releases
/usr/local/lib/python3.6/dist-packages/armada/handlers/tiller.py:276
2021-06-30 12:41:11.402 68 DEBUG armada.handlers.tiller [-] Found release stx-ceph-pools-audit, version 1, status: DEPLOYED list_releases
/usr/local/lib/python3.6/dist-packages/armada/handlers/tiller.py:276
2021-06-30 12:41:11.402 68 DEBUG armada.handlers.tiller [-] Found release stx-rbd-provisioner, version 1, status: DEPLOYED list_releases
/usr/local/lib/python3.6/dist-packages/armada/handlers/tiller.py:276
2021-06-30 12:41:11.402 68 DEBUG armada.handlers.tiller [-] Found release cm-cert-manager-psp-rolebinding, version 1, status: DEPLOYED list_releases
/usr/local/lib/python3.6/dist-packages/armada/handlers/tiller.py:276
2021-06-30 12:41:11.402 68 DEBUG armada.handlers.tiller [-] Found release cm-cert-manager, version 1, status: DEPLOYED list_releases
/usr/local/lib/python3.6/dist-packages/armada/handlers/tiller.py:276
2021-06-30 12:41:11.402 68 DEBUG armada.handlers.tiller [-] Found release ic-nginx-ingress, version 1, status: DEPLOYED list_releases
/usr/local/lib/python3.6/dist-packages/armada/handlers/tiller.py:276
2021-06-30 12:41:11.403 68 INFO armada.handlers.armada [-] Processing
ChartGroup: starlingx-rook-charts (StarlingX Rook Ceph Charts), sequenced=True
2021-06-30 12:41:11.403 68 INFO armada.handlers.chart_deploy [-]
[chart=kube-system-rook-operator]: Processing Chart, release=stx-rook-operator
2021-06-30 12:41:11.403 68 DEBUG armada.handlers.wait [-]
[chart=kube-system-rook-operator]: Resolved `wait.resources` list:
[{'type': 'pod', 'labels': {'app': 'rook-ceph-operator'}}] __init__
/usr/local/lib/python3.6/dist-packages/armada/handlers/wait.py:89
2021-06-30 12:41:11.404 68 INFO armada.handlers.chart_deploy [-]
[chart=kube-system-rook-operator]: Existing release stx-rook-operator found in namespace kube-system
2021-06-30 12:41:11.406 68 INFO armada.handlers.chart_deploy [-]
[chart=kube-system-rook-operator]: Checking for updates to chart release inputs.
2021-06-30 12:41:11.425 68 INFO armada.handlers.chart_deploy [-]
[chart=kube-system-rook-operator]: Found no updates to chart release inputs
2021-06-30 12:41:11.425 68 INFO armada.handlers.wait [-]
[chart=kube-system-rook-operator]: Waiting for resource type=pod, namespace=kube-system labels=app=rook-ceph-operator required=True for 1800s
2021-06-30 12:41:11.425 68 DEBUG armada.handlers.wait [-]
[chart=kube-system-rook-operator]: Starting to wait on:
namespace=kube-system, resource type=pod, label_selector=(app=rook-ceph-operator), timeout=1800 _watch_resource_completions
/usr/local/lib/python3.6/dist-packages/armada/handlers/wait.py:367
2021-06-30 12:41:11.435 68 DEBUG armada.handlers.wait [-]
[chart=kube-system-rook-operator]: pod rook-ceph-operator-79fb8559-k7w8z is ready! handle_resource
/usr/local/lib/python3.6/dist-packages/armada/handlers/wait.py:258
2021-06-30 12:41:11.436 68 DEBUG armada.handlers.wait [-]
[chart=kube-system-rook-operator]: Found no modified resources. wait
/usr/local/lib/python3.6/dist-packages/armada/handlers/wait.py:302
2021-06-30 12:41:11.436 68 INFO armada.handlers.chart_deploy [-]
[chart=kube-system-rook-ceph]: Processing Chart, release=stx-rook-ceph
2021-06-30 12:41:11.436 68 DEBUG armada.handlers.wait [-]
[chart=kube-system-rook-ceph]: Resolved `wait.resources` list: [{'type':
'pod', 'labels': {'app': 'rook-ceph-mgr'}}, {'type': 'pod', 'labels':
{'app': 'rook-ceph-mon'}}, {'type': 'pod', 'labels': {'app':
'rook-ceph-osd'}}] __init__
/usr/local/lib/python3.6/dist-packages/armada/handlers/wait.py:89
2021-06-30 12:41:11.437 68 INFO armada.handlers.chart_deploy [-]
[chart=kube-system-rook-ceph]: Existing release stx-rook-ceph found in namespace kube-system
2021-06-30 12:41:11.439 68 INFO armada.handlers.chart_deploy [-]
[chart=kube-system-rook-ceph]: Checking for updates to chart release inputs.
2021-06-30 12:41:11.448 68 INFO armada.handlers.chart_deploy [-]
[chart=kube-system-rook-ceph]: Found no updates to chart release inputs
2021-06-30 12:41:11.448 68 INFO armada.handlers.wait [-]
[chart=kube-system-rook-ceph]: Waiting for resource type=pod, namespace=kube-system labels=app=rook-ceph-mgr required=True for 1800s
2021-06-30 12:41:11.448 68 DEBUG armada.handlers.wait [-]
[chart=kube-system-rook-ceph]: Starting to wait on: namespace=kube-system, resource type=pod, label_selector=(app=rook-ceph-mgr), timeout=1800 _watch_resource_completions
/usr/local/lib/python3.6/dist-packages/armada/handlers/wait.py:367
2021-06-30 12:41:11.476 68 DEBUG armada.handlers.wait [-]
[chart=kube-system-rook-ceph]: pod rook-ceph-mgr-a-7b84cc65d6-bt5mq is ready! handle_resource
/usr/local/lib/python3.6/dist-packages/armada/handlers/wait.py:258
2021-06-30 12:41:11.476 68 DEBUG armada.handlers.wait [-]
[chart=kube-system-rook-ceph]: Found no modified resources. wait
/usr/local/lib/python3.6/dist-packages/armada/handlers/wait.py:302
2021-06-30 12:41:11.476 68 INFO armada.handlers.wait [-]
[chart=kube-system-rook-ceph]: Waiting for resource type=pod, namespace=kube-system labels=app=rook-ceph-mon required=True for 1800s
2021-06-30 12:41:11.476 68 DEBUG armada.handlers.wait [-]
[chart=kube-system-rook-ceph]: Starting to wait on: namespace=kube-system, resource type=pod, label_selector=(app=rook-ceph-mon), timeout=1800 _watch_resource_completions
/usr/local/lib/python3.6/dist-packages/armada/handlers/wait.py:367
2021-06-30 12:41:11.483 68 DEBUG armada.handlers.wait [-]
[chart=kube-system-rook-ceph]: pod rook-ceph-mon-a-65b58876db-98jg5 not
ready: Waiting for pod rook-ceph-mon-a-65b58876db-98jg5 to be ready...
handle_resource
/usr/local/lib/python3.6/dist-packages/armada/handlers/wait.py:260
2021-06-30 12:41:22.131 68 DEBUG armada.handlers.wait [-]
[chart=kube-system-rook-ceph]: Watch event: type=MODIFIED, name=rook-ceph-mon-a-65b58876db-98jg5,
namespace=kube-system,resource_version=2005320 _watch_resource_completions
/usr/local/lib/python3.6/dist-packages/armada/handlers/wait.py:412
2021-06-30 12:41:22.131 68 DEBUG armada.handlers.wait [-]
[chart=kube-system-rook-ceph]: pod rook-ceph-mon-a-65b58876db-98jg5 not
ready: Waiting for pod rook-ceph-mon-a-65b58876db-98jg5 to be ready...
handle_resource
/usr/local/lib/python3.6/dist-packages/armada/handlers/wait.py:260
2021-06-30 12:41:35.357 68 DEBUG armada.handlers.wait [-]
[chart=kube-system-rook-ceph]: Watch event: type=MODIFIED, name=rook-ceph-mon-a-65b58876db-98jg5,
namespace=kube-system,resource_version=2005430 _watch_resource_completions
/usr/local/lib/python3.6/dist-packages/armada/handlers/wait.py:412
2021-06-30 12:41:35.358 68 DEBUG armada.handlers.wait [-]
[chart=kube-system-rook-ceph]: pod rook-ceph-mon-a-65b58876db-98jg5 is ready! handle_resource
/usr/local/lib/python3.6/dist-packages/armada/handlers/wait.py:258
2021-06-30 12:41:35.358 68 DEBUG armada.handlers.wait [-]
[chart=kube-system-rook-ceph]: Found modified resources:
['rook-ceph-mon-a-65b58876db-98jg5'] wait
/usr/local/lib/python3.6/dist-packages/armada/handlers/wait.py:299
2021-06-30 12:41:35.358 68 DEBUG armada.handlers.wait [-]
[chart=kube-system-rook-ceph]: Continuing to wait: 0 consecutive attempts without modified resources of 1 required. wait
/usr/local/lib/python3.6/dist-packages/armada/handlers/wait.py:310
2021-06-30 12:41:36.359 68 DEBUG armada.handlers.wait [-]
[chart=kube-system-rook-ceph]: Starting to wait on: namespace=kube-system, resource type=pod, label_selector=(app=rook-ceph-mon), timeout=1775 _watch_resource_completions
/usr/local/lib/python3.6/dist-packages/armada/handlers/wait.py:367
2021-06-30 12:41:36.367 68 DEBUG armada.handlers.wait [-]
[chart=kube-system-rook-ceph]: pod rook-ceph-mon-a-65b58876db-98jg5 is ready! handle_resource
/usr/local/lib/python3.6/dist-packages/armada/handlers/wait.py:258
2021-06-30 12:41:36.367 68 DEBUG armada.handlers.wait [-]
[chart=kube-system-rook-ceph]: Found no modified resources. wait
/usr/local/lib/python3.6/dist-packages/armada/handlers/wait.py:302
2021-06-30 12:41:36.367 68 INFO armada.handlers.wait [-]
[chart=kube-system-rook-ceph]: Waiting for resource type=pod, namespace=kube-system labels=app=rook-ceph-osd required=True for 1775s
2021-06-30 12:41:36.367 68 DEBUG armada.handlers.wait [-]
[chart=kube-system-rook-ceph]: Starting to wait on: namespace=kube-system, resource type=pod, label_selector=(app=rook-ceph-osd), timeout=1775 _watch_resource_completions
/usr/local/lib/python3.6/dist-packages/armada/handlers/wait.py:367
2021-06-30 12:42:11.404 68 DEBUG armada.handlers.lock [-] Updating lock update_lock
/usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py:176
2021-06-30 12:43:11.466 68 DEBUG armada.handlers.lock [-] Updating lock update_lock
/usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py:176
2021-06-30 12:44:11.528 68 DEBUG armada.handlers.lock [-] Updating lock update_lock
/usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py:176
2021-06-30 12:45:11.589 68 DEBUG armada.handlers.lock [-] Updating lock update_lock
/usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py:176
2021-06-30 12:46:11.651 68 DEBUG armada.handlers.lock [-] Updating lock update_lock
/usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py:176
2021-06-30 12:47:11.712 68 DEBUG armada.handlers.lock [-] Updating lock update_lock
/usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py:176
2021-06-30 12:48:11.774 68 DEBUG armada.handlers.lock [-] Updating lock update_lock
/usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py:176
2021-06-30 12:49:11.835 68 DEBUG armada.handlers.lock [-] Updating lock update_lock
/usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py:176
2021-06-30 12:50:11.897 68 DEBUG armada.handlers.lock [-] Updating lock update_lock
/usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py:176
2021-06-30 12:51:11.958 68 DEBUG armada.handlers.lock [-] Updating lock update_lock
/usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py:176
2021-06-30 12:52:12.019 68 DEBUG armada.handlers.lock [-] Updating lock update_lock
/usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py:176
2021-06-30 12:53:12.081 68 DEBUG armada.handlers.lock [-] Updating lock update_lock
/usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py:176
2021-06-30 12:54:12.144 68 DEBUG armada.handlers.lock [-] Updating lock update_lock
/usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py:176
2021-06-30 12:55:12.205 68 DEBUG armada.handlers.lock [-] Updating lock update_lock
/usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py:176
2021-06-30 12:56:12.268 68 DEBUG armada.handlers.lock [-] Updating lock update_lock
/usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py:176
2021-06-30 12:57:12.329 68 DEBUG armada.handlers.lock [-] Updating lock update_lock
/usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py:176
2021-06-30 12:58:12.392 68 DEBUG armada.handlers.lock [-] Updating lock update_lock
/usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py:176
2021-06-30 12:59:12.454 68 DEBUG armada.handlers.lock [-] Updating lock update_lock
/usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py:176
2021-06-30 13:00:12.517 68 DEBUG armada.handlers.lock [-] Updating lock update_lock
/usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py:176
2021-06-30 13:01:12.578 68 DEBUG armada.handlers.lock [-] Updating lock update_lock
/usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py:176
2021-06-30 13:02:12.639 68 DEBUG armada.handlers.lock [-] Updating lock update_lock
/usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py:176
2021-06-30 13:03:12.701 68 DEBUG armada.handlers.lock [-] Updating lock update_lock
/usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py:176
2021-06-30 13:04:12.762 68 DEBUG armada.handlers.lock [-] Updating lock update_lock
/usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py:176
2021-06-30 13:05:12.824 68 DEBUG armada.handlers.lock [-] Updating lock update_lock
/usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py:176
2021-06-30 13:06:12.886 68 DEBUG armada.handlers.lock [-] Updating lock update_lock
/usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py:176
2021-06-30 13:07:12.946 68 DEBUG armada.handlers.lock [-] Updating lock update_lock
/usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py:176
2021-06-30 13:08:13.007 68 DEBUG armada.handlers.lock [-] Updating lock update_lock
/usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py:176
2021-06-30 13:09:13.069 68 DEBUG armada.handlers.lock [-] Updating lock update_lock
/usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py:176
2021-06-30 13:10:13.132 68 DEBUG armada.handlers.lock [-] Updating lock update_lock
/usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py:176
2021-06-30 13:11:11.371 68 ERROR armada.handlers.wait [-]
[chart=kube-system-rook-ceph]: Timed out waiting for pods (namespace=kube-system, labels=(app=rook-ceph-osd)). None found! Are `wait.labels` correct? Does `wait.resources` need to exclude `type: pod`?
2021-06-30 13:11:11.371 68 ERROR armada.handlers.armada [-] Chart deploy [kube-system-rook-ceph] failed:
armada.exceptions.k8s_exceptions.KubernetesWatchTimeoutException: Timed out waiting for pods (namespace=kube-system, labels=(app=rook-ceph-osd)). None found! Are `wait.labels` correct? Does `wait.resources` need to exclude
`type: pod`?
2021-06-30 13:11:11.371 68 ERROR armada.handlers.armada Traceback (most recent call last):
2021-06-30 13:11:11.371 68 ERROR armada.handlers.armada File
"/usr/local/lib/python3.6/dist-packages/armada/handlers/armada.py", line 225, in handle_result
2021-06-30 13:11:11.371 68 ERROR armada.handlers.armada result =
get_result()
2021-06-30 13:11:11.371 68 ERROR armada.handlers.armada File
"/usr/local/lib/python3.6/dist-packages/armada/handlers/armada.py", line 236, in <lambda>
2021-06-30 13:11:11.371 68 ERROR armada.handlers.armada if
(handle_result(chart, lambda: deploy_chart(chart))):
2021-06-30 13:11:11.371 68 ERROR armada.handlers.armada File
"/usr/local/lib/python3.6/dist-packages/armada/handlers/armada.py", line 214, in deploy_chart
2021-06-30 13:11:11.371 68 ERROR armada.handlers.armada chart,
cg_test_all_charts, prefix, known_releases)
2021-06-30 13:11:11.371 68 ERROR armada.handlers.armada File
"/usr/local/lib/python3.6/dist-packages/armada/handlers/chart_deploy.py",
line 248, in execute
2021-06-30 13:11:11.371 68 ERROR armada.handlers.armada
chart_wait.wait(timer)
2021-06-30 13:11:11.371 68 ERROR armada.handlers.armada File
"/usr/local/lib/python3.6/dist-packages/armada/handlers/wait.py", line 134, in wait
2021-06-30 13:11:11.371 68 ERROR armada.handlers.armada
wait.wait(timeout=timeout)
2021-06-30 13:11:11.371 68 ERROR armada.handlers.armada File
"/usr/local/lib/python3.6/dist-packages/armada/handlers/wait.py", line 294, in wait
2021-06-30 13:11:11.371 68 ERROR armada.handlers.armada modified =
self._wait(deadline)
2021-06-30 13:11:11.371 68 ERROR armada.handlers.armada File
"/usr/local/lib/python3.6/dist-packages/armada/handlers/wait.py", line 354, in _wait
2021-06-30 13:11:11.371 68 ERROR armada.handlers.armada raise
k8s_exceptions.KubernetesWatchTimeoutException(error)
2021-06-30 13:11:11.371 68 ERROR armada.handlers.armada
armada.exceptions.k8s_exceptions.KubernetesWatchTimeoutException: Timed out waiting for pods (namespace=kube-system, labels=(app=rook-ceph-osd)). None found! Are `wait.labels` correct? Does `wait.resources` need to exclude
`type: pod`?
2021-06-30 13:11:11.371 68 ERROR armada.handlers.armada
2021-06-30 13:11:11.373 68 ERROR armada.handlers.armada [-] Chart deploy(s)
failed: ['kube-system-rook-ceph']
2021-06-30 13:11:12.193 68 INFO armada.handlers.lock [-] Releasing lock
2021-06-30 13:11:12.196 68 ERROR armada.cli [-] Caught internal exception:
armada.exceptions.armada_exceptions.ChartDeployException: Exception deploying charts: ['kube-system-rook-ceph']
2021-06-30 13:11:12.196 68 ERROR armada.cli Traceback (most recent call
last):
2021-06-30 13:11:12.196 68 ERROR armada.cli File
"/usr/local/lib/python3.6/dist-packages/armada/cli/__init__.py", line 38, in safe_invoke
2021-06-30 13:11:12.196 68 ERROR armada.cli self.invoke()
2021-06-30 13:11:12.196 68 ERROR armada.cli File
"/usr/local/lib/python3.6/dist-packages/armada/cli/apply.py", line 213, in invoke
2021-06-30 13:11:12.196 68 ERROR armada.cli resp =
self.handle(documents, tiller)
2021-06-30 13:11:12.196 68 ERROR armada.cli File
"/usr/local/lib/python3.6/dist-packages/armada/handlers/lock.py", line 81, in func_wrapper
2021-06-30 13:11:12.196 68 ERROR armada.cli return future.result()
2021-06-30 13:11:12.196 68 ERROR armada.cli File
"/usr/lib/python3.6/concurrent/futures/_base.py", line 425, in result
2021-06-30 13:11:12.196 68 ERROR armada.cli return self.__get_result()
2021-06-30 13:11:12.196 68 ERROR armada.cli File
"/usr/lib/python3.6/concurrent/futures/_base.py", line 384, in __get_result
2021-06-30 13:11:12.196 68 ERROR armada.cli raise self._exception
2021-06-30 13:11:12.196 68 ERROR armada.cli File
"/usr/lib/python3.6/concurrent/futures/thread.py", line 56, in run
2021-06-30 13:11:12.196 68 ERROR armada.cli result =
self.fn(*self.args, **self.kwargs)
2021-06-30 13:11:12.196 68 ERROR armada.cli File
"/usr/local/lib/python3.6/dist-packages/armada/cli/apply.py", line 256, in handle
2021-06-30 13:11:12.196 68 ERROR armada.cli return armada.sync()
2021-06-30 13:11:12.196 68 ERROR armada.cli File
"/usr/local/lib/python3.6/dist-packages/armada/handlers/armada.py", line 252, in sync
2021-06-30 13:11:12.196 68 ERROR armada.cli raise
armada_exceptions.ChartDeployException(failures)
2021-06-30 13:11:12.196 68 ERROR armada.cli
armada.exceptions.armada_exceptions.ChartDeployException: Exception deploying charts: ['kube-system-rook-ceph']
2021-06-30 13:11:12.196 68 ERROR armada.cli command terminated with exit code 1
--
Sent with Vivaldi Mail. Download Vivaldi for free at vivaldi.com
------------------------------
Subject: Digest Footer
_______________________________________________
Starlingx-discuss mailing list
Starlingx-discuss at lists.starlingx.io
http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
------------------------------
End of Starlingx-discuss Digest, Vol 38, Issue 2
************************************************
More information about the Starlingx-discuss
mailing list