<html dir="ltr">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
<style>
<!--
@font-face
{font-family:Wingdings}
@font-face
{font-family:Wingdings}
@font-face
{font-family:Calibri}
@font-face
{font-family:Tahoma}
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif"}
a:link, span.MsoHyperlink
{color:blue;
text-decoration:underline}
a:visited, span.MsoHyperlinkFollowed
{color:purple;
text-decoration:underline}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif"}
span.EmailStyle18
{font-family:"Calibri","sans-serif";
color:#1F497D}
span.EmailStyle19
{font-family:"Calibri","sans-serif";
color:#1F497D}
span.EmailStyle20
{font-family:"Calibri","sans-serif";
color:#1F497D}
span.EmailStyle21
{font-family:"Calibri","sans-serif";
color:windowtext}
span.EmailStyle22
{font-family:"Calibri","sans-serif";
color:#1F497D}
span.EmailStyle23
{font-family:"Calibri","sans-serif";
color:#1F497D}
span.EmailStyle24
{font-family:"Calibri","sans-serif";
color:windowtext}
span.EmailStyle25
{font-family:"Calibri","sans-serif";
color:#1F497D}
.MsoChpDefault
{font-size:10.0pt}
@page WordSection1
{margin:1.0in 1.0in 1.0in 1.0in}
-->
</style><style type="text/css" id="owaParaStyle"></style>
</head>
<body lang="EN-US" link="blue" vlink="purple" fpstyle="1" ocsi="0">
<div style="direction: ltr;font-family: Tahoma;color: #000000;font-size: 10pt;">SM only requires mtce to reboot a failed controller during a failover, i.e, in a duplex environment, the survivor controller requires mtce to reboot the failed controller.
<div>SM won't require reboot when there is only 1 controller available (e.g an aio simplex).
<div><br>
</div>
<div>Bin<br>
<div style="font-family: Times New Roman; color: #000000; font-size: 16px">
<hr tabindex="-1">
<div id="divRpF628194" style="direction: ltr;"><font face="Tahoma" size="2" color="#000000"><b>From:</b> MacDonald, Eric [Eric.MacDonald@windriver.com]<br>
<b>Sent:</b> Monday, June 24, 2019 5:15 AM<br>
<b>To:</b> Lin, Shuicheng; Liu, Tao; Ezpeer Chen; starlingx-discuss@lists.starlingx.io<br>
<b>Subject:</b> Re: [Starlingx-discuss] How to turn off fault management?<br>
</font><br>
</div>
<div></div>
<div>
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:14.0pt; font-family:"Calibri","sans-serif"; color:#1F497D">Yes, SM can request mtce to reboot a controller and there will be an explicit mtce log for that (below)</span></p>
<p class="MsoNormal"><span style="font-size:14.0pt; font-family:"Calibri","sans-serif"; color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="font-size:14.0pt; font-family:"Courier New"; color:#1F497D">controller-? is being force failed by SM</span></p>
<p class="MsoNormal"><span style="font-size:14.0pt; font-family:"Calibri","sans-serif"; color:#1F497D"> </span></p>
<div style="border:none; border-left:solid blue 1.5pt; padding:0in 0in 0in 4.0pt">
<div>
<div style="border:none; border-top:solid #B5C4DF 1.0pt; padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt; font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt; font-family:"Tahoma","sans-serif""> Lin, Shuicheng [mailto:shuicheng.lin@intel.com]
<br>
<b>Sent:</b> Monday, June 24, 2019 7:48 AM<br>
<b>To:</b> MacDonald, Eric; Liu, Tao; Ezpeer Chen; starlingx-discuss@lists.starlingx.io<br>
<b>Subject:</b> RE: [Starlingx-discuss] How to turn off fault management?<br>
<b>Importance:</b> High</span></p>
</div>
</div>
<p class="MsoNormal"> </p>
<p class="MsoNormal"><a name="_MailEndCompose"><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D">Hi Eric,</span></a><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D"></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D">I guess you mean sm should reboot a host, not shutdown. Is it correct?</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D">But per Ezpeer’s description, it seems like shutdown.</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D">BTW, it is with STX 1.0, from 2010.10 release branch.</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D"> </span></p>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D">Best Regards</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D">Shuicheng</span></p>
</div>
<p class="MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D"> </span></p>
<div>
<div style="border:none; border-top:solid #E1E1E1 1.0pt; padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><a name="_____replyseparator"></a><b><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"">From:</span></b><span style="font-size:11.0pt; font-family:"Calibri","sans-serif""> MacDonald, Eric [mailto:Eric.MacDonald@windriver.com]
<br>
<b>Sent:</b> Monday, June 24, 2019 7:40 PM<br>
<b>To:</b> Lin, Shuicheng <shuicheng.lin@intel.com>; Liu, Tao <Tao.Liu@windriver.com>; Ezpeer Chen <ezpeerchen@gmail.com>; starlingx-discuss@lists.starlingx.io<br>
<b>Subject:</b> RE: [Starlingx-discuss] How to turn off fault management?</span></p>
</div>
</div>
<p class="MsoNormal"> </p>
<p class="MsoNormal"><span style="font-size:14.0pt; font-family:"Calibri","sans-serif"; color:#1F497D">SM does not power off a host.</span></p>
<p class="MsoNormal"><span style="font-size:14.0pt; font-family:"Calibri","sans-serif"; color:#1F497D">When you say shutdown you mean power off correct ?</span></p>
<p class="MsoNormal"><span style="font-size:14.0pt; font-family:"Calibri","sans-serif"; color:#1F497D"> </span></p>
<div style="border:none; border-left:solid blue 1.5pt; padding:0in 0in 0in 4.0pt">
<div>
<div style="border:none; border-top:solid #B5C4DF 1.0pt; padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt; font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt; font-family:"Tahoma","sans-serif""> Lin, Shuicheng [<a href="mailto:shuicheng.lin@intel.com" target="_blank" rel="noopener noreferrer">mailto:shuicheng.lin@intel.com</a>]
<br>
<b>Sent:</b> Monday, June 24, 2019 7:38 AM<br>
<b>To:</b> MacDonald, Eric; Liu, Tao; Ezpeer Chen; <a href="mailto:starlingx-discuss@lists.starlingx.io" target="_blank" rel="noopener noreferrer">
starlingx-discuss@lists.starlingx.io</a><br>
<b>Subject:</b> RE: [Starlingx-discuss] How to turn off fault management?<br>
<b>Importance:</b> High</span></p>
</div>
</div>
<p class="MsoNormal"> </p>
<p class="MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D">Hi Eric,</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D">Here is the collect log for the issue.</span></p>
<p class="MsoNormal">Log files:<br>
<a href="https://drive.google.com/open?id=1QViho0khiMQpYDOcF5ACZV2cwymgNEMS" target="_blank" rel="noopener noreferrer">https://drive.google.com/open?id=1QViho0khiMQpYDOcF5ACZV2cwymgNEMS</a><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D"></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D">You could find the issue reproduce step in below mail:</span></p>
<p class="MsoNormal"><a href="http://lists.starlingx.io/pipermail/starlingx-discuss/2019-June/005033.html" target="_blank" rel="noopener noreferrer">http://lists.starlingx.io/pipermail/starlingx-discuss/2019-June/005033.html</a><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D"></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D">It seems it is pci-sriov related issue. From the sm.log, it seems Ethernet interface is not stable, and cause several services cannot run successfully, and
lead to the shutdown.</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D">Maybe you could provide some workaround suggestion for him.</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D">Thanks.</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D"> </span></p>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D">Best Regards</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D">Shuicheng</span></p>
</div>
<p class="MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D"> </span></p>
<div>
<div style="border:none; border-top:solid #E1E1E1 1.0pt; padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"">From:</span></b><span style="font-size:11.0pt; font-family:"Calibri","sans-serif""> MacDonald, Eric [<a href="mailto:Eric.MacDonald@windriver.com" target="_blank" rel="noopener noreferrer">mailto:Eric.MacDonald@windriver.com</a>]
<br>
<b>Sent:</b> Monday, June 24, 2019 7:18 PM<br>
<b>To:</b> Liu, Tao <<a href="mailto:Tao.Liu@windriver.com" target="_blank" rel="noopener noreferrer">Tao.Liu@windriver.com</a>>; Ezpeer Chen <<a href="mailto:ezpeerchen@gmail.com" target="_blank" rel="noopener noreferrer">ezpeerchen@gmail.com</a>>;
<a href="mailto:starlingx-discuss@lists.starlingx.io" target="_blank" rel="noopener noreferrer">
starlingx-discuss@lists.starlingx.io</a><br>
<b>Subject:</b> Re: [Starlingx-discuss] How to turn off fault management?</span></p>
</div>
</div>
<p class="MsoNormal"> </p>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif"; color:#1F497D">Hi Ezpeer,</span></p>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif"; color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif"; color:#1F497D">In addition to Tao’s point …</span></p>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif"; color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif"; color:#1F497D">The only time maintenance will power off a host outside of explicit administrative action is if that host’s board management controller is provisioned, the critical action
for a sensor group has been changed to power cycle AND a sensor in that sensor group reports a debounced critical severity.</span></p>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif"; color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif"; color:#1F497D">If you are experiencing heartbeat failures that you are trying to debug you can change the heartbeat failure action to ‘degrade’ or ‘alarm’ only to avoid the recovery reboot.
Not recommended, but available for debug.</span></p>
<p class="MsoListParagraph" style="text-indent:-.25in"><span style="font-family:Wingdings; color:#1F497D">Ř</span><span style="font-size:7.0pt; color:#1F497D">
</span><span style="font-family:"Calibri","sans-serif"; color:#1F497D">system service-parameter-modify platform maintenance heartbeat_failure_action=degrade</span></p>
<p class="MsoListParagraph" style="text-indent:-.25in"><span style="font-family:Wingdings; color:#1F497D">Ř</span><span style="font-size:7.0pt; color:#1F497D">
</span><span style="font-family:"Calibri","sans-serif"; color:#1F497D">system service-parameter-apply platform</span></p>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif"; color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif"; color:#1F497D">Locking a host will prevent host watchdog reboot due to quorum process failure or watchdog pet failure/timeout</span></p>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif"; color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif"; color:#1F497D">If you are experiencing autonomous host power off then I would look at the BMC logs for critical or fatal event reports.</span></p>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif"; color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif"; color:#1F497D">Eric.</span></p>
<p class="MsoNormal"><span style="font-size:14.0pt; font-family:"Calibri","sans-serif"; color:#1F497D"> </span></p>
<div style="border:none; border-left:solid blue 1.5pt; padding:0in 0in 0in 4.0pt">
<div>
<div style="border:none; border-top:solid #B5C4DF 1.0pt; padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt; font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt; font-family:"Tahoma","sans-serif""> Liu, Tao [<a href="mailto:Tao.Liu@windriver.com" target="_blank" rel="noopener noreferrer">mailto:Tao.Liu@windriver.com</a>]
<br>
<b>Sent:</b> Friday, June 21, 2019 9:33 AM<br>
<b>To:</b> Ezpeer Chen; <a href="mailto:starlingx-discuss@lists.starlingx.io" target="_blank" rel="noopener noreferrer">
starlingx-discuss@lists.starlingx.io</a><br>
<b>Subject:</b> Re: [Starlingx-discuss] How to turn off fault management?</span></p>
</div>
</div>
<p class="MsoNormal"> </p>
<p class="MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D">Hi Ezpeer,</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D">The fault management reports fault conditions and significant events in the system and it does not reboot or power off the controller. The maintenance system
takes proper actions to recover the system When necessary. </span></p>
<p class="MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D">I suggest you to view the active alarms and event history to see what failures might lead to reboot the controller for recovery. (could it be a configuration
failure?). </span></p>
<p class="MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D">fm alarm-list</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D">fm event-list</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D">In addition, /var/log/mtcAgent.log provides more details on why the host is reboot or power-off.</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D">Regards,</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D">Tao</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D"> </span></p>
<p class="MsoNormal"><b><span style="font-size:10.0pt; font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt; font-family:"Tahoma","sans-serif""> Ezpeer Chen [<a href="mailto:ezpeerchen@gmail.com" target="_blank" rel="noopener noreferrer">mailto:ezpeerchen@gmail.com</a>]
<br>
<b>Sent:</b> Friday, June 21, 2019 4:08 AM<br>
<b>To:</b> <a href="mailto:starlingx-discuss@lists.starlingx.io" target="_blank" rel="noopener noreferrer">
starlingx-discuss@lists.starlingx.io</a><br>
<b>Subject:</b> [Starlingx-discuss] How to turn off fault management?</span></p>
<p class="MsoNormal"> </p>
<div>
<p class="MsoNormal">Dear all,<br>
<br>
Environment: STX 1.0 (2018/10) all-in-one simplex<br>
<br>
<br>
How could i turn off fault management which cause my system(controller-0) reboot or power-off?<br>
<br>
<br>
<br>
Best Regards </p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</body>
</html>