<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:dt="uuid:C2F41010-65B3-11d1-A29F-00AA00C14882" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-IN" link="#0563C1" vlink="#954F72" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal">Hi,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">We are performing FPGA image update following the procedure defined below
<o:p></o:p></p>
<p class="MsoNormal"><a href="https://docs.starlingx.io/node_management/starlingx-kubernetes/hardware_acceleration_devices/updating-an-intel-n3000-fpga-image.html">https://docs.starlingx.io/node_management/starlingx-kubernetes/hardware_acceleration_devices/updating-an-intel-n3000-fpga-image.html</a><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">After running the below command, we see that image status stays in pending state for the last 2 images (although we see the first image update is completed). Although the doc says that it may take about 40 minutes, this has been there for
2 hours the image is in pending state. Any suggestions are appreciated. More information below.<o:p></o:p></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">~(keystone)admin)$ system host-device-image-update controller-0
<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">[sysadmin@controller-0 ~(keystone_admin)]$ date;system device-image-state-list<o:p></o:p></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">Wed Jan 27 22:41:30 UTC 2021<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">+--------------+--------------------+--------------------------------------+-----------+----------------------------------+----------------------------------+<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">| hostname | PCI device address | Device image uuid | status | Update start time | updated_at |<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">+--------------+--------------------+--------------------------------------+-----------+----------------------------------+----------------------------------+<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">| controller-0 | 0000:1c:00.0 | 58bcd3f5-213b-4c16-ac5e-6d6313cc2712 | completed | 2021-01-27T20:18:43.269482+00:00 | 2021-01-27T20:18:50.399593+00:00 |<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">| controller-0 | 0000:1c:00.0 | d31086d2-f7d6-4129-8421-a0d298fa21f1 | pending | None | None |<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">| controller-0 | 0000:1c:00.0 | 3a04429f-916e-46b0-bff9-2f26c0a597c5 | pending | None | None |<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">+--------------+--------------------+--------------------------------------+-----------+----------------------------------+----------------------------------+<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">[sysadmin@controller-0 ~(keystone_admin)]$ system host-show controller-0 |grep device_image_update<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">| device_image_update | in-progress<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">The image update was triggered at the below timestamp.
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">bash info <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">2021-01-27T20:22:48.000 controller-0 -sh: info HISTORY: PID=1380366 UID=42425 system host-device-image-update controller-0<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">sysinv API has been triggered at the below timestamp.
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">2021-01-27 20:22:49.440 87255 INFO sysinv.api.controllers.v1.host [-] device_image_update host_uuid=26a50590-bbaa-41e8-a902-533250cca708<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Checked after 2 hours nearly and the status is still the same.<o:p></o:p></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">Wed Jan 27 22:41:30 UTC 2021<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">+--------------+--------------------+--------------------------------------+-----------+----------------------------------+----------------------------------+<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">| hostname | PCI device address | Device image uuid | status | Update start time | updated_at |<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">+--------------+--------------------+--------------------------------------+-----------+----------------------------------+----------------------------------+<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">| controller-0 | 0000:1c:00.0 | 58bcd3f5-213b-4c16-ac5e-6d6313cc2712 | completed | 2021-01-27T20:18:43.269482+00:00 | 2021-01-27T20:18:50.399593+00:00 |<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">| controller-0 | 0000:1c:00.0 | d31086d2-f7d6-4129-8421-a0d298fa21f1 | pending | None | None |<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">| controller-0 | 0000:1c:00.0 | 3a04429f-916e-46b0-bff9-2f26c0a597c5 | pending | None | None |<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">+--------------+--------------------+--------------------------------------+-----------+----------------------------------+----------------------------------+<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Our understanding is that “<span style="font-family:"Courier New"">host-device-image-update”
</span>is the command triggered once for all the pending images for controller-0 and that should automatically update all the pending images in one go.
<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Is there a place in /var/log where we can see the logs on the progress and what might be going wrong?. Is it a good idea to do a force lock & unlock to recover from this state?<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Below are the images that are being updated. <o:p></o:p></p>
<p class="MsoNormal"><span style="font-family:"Courier New"">controller-0:/opt/platform/device_images# ls -lrt<br>
total 44044<br>
-rw-r--r-- 1 root root 1152 Jan 27 19:35 root-key-8086-0b30-58bcd3f5-213b-4c16-ac5e-6d6313cc2712.bit<br>
-rw-r--r-- 1 root root 45089792 Jan 27 19:40 functional-8086-0b30-d31086d2-f7d6-4129-8421-a0d298fa21f1.bit<br>
-rw-r--r-- 1 root root 1152 Jan 27 19:41 key-revocation-8086-0b30-3a04429f-916e-46b0-bff9-2f26c0a597c5.bit<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-IN">Venkata Veldanda
<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</body>
</html>