Hi,

 

We are performing FPGA image update following the procedure defined below

https://docs.starlingx.io/node_management/starlingx-kubernetes/hardware_acceleration_devices/updating-an-intel-n3000-fpga-image.html

 

After running the below command, we see that image status stays in pending state for the last 2 images (although we see the first image update is completed). Although the doc says that it may take about 40 minutes, this has been there for 2 hours the image is in pending state. Any suggestions are appreciated. More information below.

~(keystone)admin)$ system host-device-image-update controller-0

 

[sysadmin@controller-0 ~(keystone_admin)]$ date;system device-image-state-list

Wed Jan 27 22:41:30 UTC 2021

+--------------+--------------------+--------------------------------------+-----------+----------------------------------+----------------------------------+

| hostname     | PCI device address | Device image uuid                    | status    | Update start time                | updated_at                       |

+--------------+--------------------+--------------------------------------+-----------+----------------------------------+----------------------------------+

| controller-0 | 0000:1c:00.0       | 58bcd3f5-213b-4c16-ac5e-6d6313cc2712 | completed | 2021-01-27T20:18:43.269482+00:00 | 2021-01-27T20:18:50.399593+00:00 |

| controller-0 | 0000:1c:00.0       | d31086d2-f7d6-4129-8421-a0d298fa21f1 | pending   | None                             | None                             |

| controller-0 | 0000:1c:00.0       | 3a04429f-916e-46b0-bff9-2f26c0a597c5 | pending   | None                             | None                             |

+--------------+--------------------+--------------------------------------+-----------+----------------------------------+----------------------------------+

 

[sysadmin@controller-0 ~(keystone_admin)]$ system host-show controller-0 |grep device_image_update

| device_image_update   | in-progress

 

 

The image update was triggered at the below timestamp.

bash info

2021-01-27T20:22:48.000 controller-0 -sh: info HISTORY: PID=1380366 UID=42425 system host-device-image-update controller-0

 

sysinv API has been triggered at the below timestamp.

2021-01-27 20:22:49.440 87255 INFO sysinv.api.controllers.v1.host [-] device_image_update host_uuid=26a50590-bbaa-41e8-a902-533250cca708

 

Checked after 2 hours nearly and the status is still the same.

Wed Jan 27 22:41:30 UTC 2021

+--------------+--------------------+--------------------------------------+-----------+----------------------------------+----------------------------------+

| hostname     | PCI device address | Device image uuid                    | status    | Update start time                | updated_at                       |

+--------------+--------------------+--------------------------------------+-----------+----------------------------------+----------------------------------+

| controller-0 | 0000:1c:00.0       | 58bcd3f5-213b-4c16-ac5e-6d6313cc2712 | completed | 2021-01-27T20:18:43.269482+00:00 | 2021-01-27T20:18:50.399593+00:00 |

| controller-0 | 0000:1c:00.0       | d31086d2-f7d6-4129-8421-a0d298fa21f1 | pending   | None                             | None                             |

| controller-0 | 0000:1c:00.0       | 3a04429f-916e-46b0-bff9-2f26c0a597c5 | pending   | None                             | None                             |

+--------------+--------------------+--------------------------------------+-----------+----------------------------------+----------------------------------+

 

Our understanding is that “host-device-image-update” is the command triggered once for all the pending images for controller-0 and that should automatically update all the pending images in one go.

 

Is there a place in /var/log where we can see the logs on the progress  and what might be going wrong?. Is it a good idea to do a force lock & unlock to recover from this state?

 

Below are the images that are being updated.

controller-0:/opt/platform/device_images# ls -lrt
total 44044
-rw-r--r-- 1 root root 1152 Jan 27 19:35 root-key-8086-0b30-58bcd3f5-213b-4c16-ac5e-6d6313cc2712.bit
-rw-r--r-- 1 root root 45089792 Jan 27 19:40 functional-8086-0b30-d31086d2-f7d6-4129-8421-a0d298fa21f1.bit
-rw-r--r-- 1 root root 1152 Jan 27 19:41 key-revocation-8086-0b30-3a04429f-916e-46b0-bff9-2f26c0a597c5.bit

 

 

Venkata Veldanda