Hi,
We are performing FPGA image update following the procedure defined below
After running the below command, we see that image status stays in pending state for the last 2 images (although we see the first image update is completed). Although the doc says that it may take about 40 minutes, this has been there for
2 hours the image is in pending state. Any suggestions are appreciated. More information below.
~(keystone)admin)$ system host-device-image-update controller-0
[sysadmin@controller-0 ~(keystone_admin)]$ date;system device-image-state-list
Wed Jan 27 22:41:30 UTC 2021
+--------------+--------------------+--------------------------------------+-----------+----------------------------------+----------------------------------+
| hostname | PCI device address | Device image uuid | status | Update start time | updated_at |
+--------------+--------------------+--------------------------------------+-----------+----------------------------------+----------------------------------+
| controller-0 | 0000:1c:00.0 | 58bcd3f5-213b-4c16-ac5e-6d6313cc2712 | completed | 2021-01-27T20:18:43.269482+00:00 | 2021-01-27T20:18:50.399593+00:00 |
| controller-0 | 0000:1c:00.0 | d31086d2-f7d6-4129-8421-a0d298fa21f1 | pending | None | None |
| controller-0 | 0000:1c:00.0 | 3a04429f-916e-46b0-bff9-2f26c0a597c5 | pending | None | None |
+--------------+--------------------+--------------------------------------+-----------+----------------------------------+----------------------------------+
[sysadmin@controller-0 ~(keystone_admin)]$ system host-show controller-0 |grep device_image_update
| device_image_update | in-progress
The image update was triggered at the below timestamp.
bash info
2021-01-27T20:22:48.000 controller-0 -sh: info HISTORY: PID=1380366 UID=42425 system host-device-image-update controller-0
sysinv API has been triggered at the below timestamp.
2021-01-27 20:22:49.440 87255 INFO sysinv.api.controllers.v1.host [-] device_image_update host_uuid=26a50590-bbaa-41e8-a902-533250cca708
Checked after 2 hours nearly and the status is still the same.
Wed Jan 27 22:41:30 UTC 2021
+--------------+--------------------+--------------------------------------+-----------+----------------------------------+----------------------------------+
| hostname | PCI device address | Device image uuid | status | Update start time | updated_at |
+--------------+--------------------+--------------------------------------+-----------+----------------------------------+----------------------------------+
| controller-0 | 0000:1c:00.0 | 58bcd3f5-213b-4c16-ac5e-6d6313cc2712 | completed | 2021-01-27T20:18:43.269482+00:00 | 2021-01-27T20:18:50.399593+00:00 |
| controller-0 | 0000:1c:00.0 | d31086d2-f7d6-4129-8421-a0d298fa21f1 | pending | None | None |
| controller-0 | 0000:1c:00.0 | 3a04429f-916e-46b0-bff9-2f26c0a597c5 | pending | None | None |
+--------------+--------------------+--------------------------------------+-----------+----------------------------------+----------------------------------+
Our understanding is that “host-device-image-update”
is the command triggered once for all the pending images for controller-0 and that should automatically update all the pending images in one go.
Is there a place in /var/log where we can see the logs on the progress and what might be going wrong?. Is it a good idea to do a force lock & unlock to recover from this state?
Below are the images that are being updated.
controller-0:/opt/platform/device_images# ls -lrt
total 44044
-rw-r--r-- 1 root root 1152 Jan 27 19:35 root-key-8086-0b30-58bcd3f5-213b-4c16-ac5e-6d6313cc2712.bit
-rw-r--r-- 1 root root 45089792 Jan 27 19:40 functional-8086-0b30-d31086d2-f7d6-4129-8421-a0d298fa21f1.bit
-rw-r--r-- 1 root root 1152 Jan 27 19:41 key-revocation-8086-0b30-3a04429f-916e-46b0-bff9-2f26c0a597c5.bit
Venkata Veldanda