SM only requires mtce to reboot a failed controller during a failover, i.e, in a duplex environment, the survivor controller requires mtce to reboot the failed controller. SM won't require reboot when there is only 1 controller available (e.g an aio simplex). Bin ________________________________ From: MacDonald, Eric [Eric.MacDonald@windriver.com] Sent: Monday, June 24, 2019 5:15 AM To: Lin, Shuicheng; Liu, Tao; Ezpeer Chen; starlingx-discuss@lists.starlingx.io Subject: Re: [Starlingx-discuss] How to turn off fault management? Yes, SM can request mtce to reboot a controller and there will be an explicit mtce log for that (below) controller-? is being force failed by SM From: Lin, Shuicheng [mailto:shuicheng.lin@intel.com] Sent: Monday, June 24, 2019 7:48 AM To: MacDonald, Eric; Liu, Tao; Ezpeer Chen; starlingx-discuss@lists.starlingx.io Subject: RE: [Starlingx-discuss] How to turn off fault management? Importance: High Hi Eric, I guess you mean sm should reboot a host, not shutdown. Is it correct? But per Ezpeer’s description, it seems like shutdown. BTW, it is with STX 1.0, from 2010.10 release branch. Best Regards Shuicheng From: MacDonald, Eric [mailto:Eric.MacDonald@windriver.com] Sent: Monday, June 24, 2019 7:40 PM To: Lin, Shuicheng <shuicheng.lin@intel.com>; Liu, Tao <Tao.Liu@windriver.com>; Ezpeer Chen <ezpeerchen@gmail.com>; starlingx-discuss@lists.starlingx.io Subject: RE: [Starlingx-discuss] How to turn off fault management? SM does not power off a host. When you say shutdown you mean power off correct ? From: Lin, Shuicheng [mailto:shuicheng.lin@intel.com] Sent: Monday, June 24, 2019 7:38 AM To: MacDonald, Eric; Liu, Tao; Ezpeer Chen; starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Subject: RE: [Starlingx-discuss] How to turn off fault management? Importance: High Hi Eric, Here is the collect log for the issue. Log files: https://drive.google.com/open?id=1QViho0khiMQpYDOcF5ACZV2cwymgNEMS You could find the issue reproduce step in below mail: http://lists.starlingx.io/pipermail/starlingx-discuss/2019-June/005033.html It seems it is pci-sriov related issue. From the sm.log, it seems Ethernet interface is not stable, and cause several services cannot run successfully, and lead to the shutdown. Maybe you could provide some workaround suggestion for him. Thanks. Best Regards Shuicheng From: MacDonald, Eric [mailto:Eric.MacDonald@windriver.com] Sent: Monday, June 24, 2019 7:18 PM To: Liu, Tao <Tao.Liu@windriver.com<mailto:Tao.Liu@windriver.com>>; Ezpeer Chen <ezpeerchen@gmail.com<mailto:ezpeerchen@gmail.com>>; starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Subject: Re: [Starlingx-discuss] How to turn off fault management? Hi Ezpeer, In addition to Tao’s point … The only time maintenance will power off a host outside of explicit administrative action is if that host’s board management controller is provisioned, the critical action for a sensor group has been changed to power cycle AND a sensor in that sensor group reports a debounced critical severity. If you are experiencing heartbeat failures that you are trying to debug you can change the heartbeat failure action to ‘degrade’ or ‘alarm’ only to avoid the recovery reboot. Not recommended, but available for debug.
system service-parameter-modify platform maintenance heartbeat_failure_action=degrade
system service-parameter-apply platform
Locking a host will prevent host watchdog reboot due to quorum process failure or watchdog pet failure/timeout If you are experiencing autonomous host power off then I would look at the BMC logs for critical or fatal event reports. Eric. From: Liu, Tao [mailto:Tao.Liu@windriver.com] Sent: Friday, June 21, 2019 9:33 AM To: Ezpeer Chen; starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Subject: Re: [Starlingx-discuss] How to turn off fault management? Hi Ezpeer, The fault management reports fault conditions and significant events in the system and it does not reboot or power off the controller. The maintenance system takes proper actions to recover the system When necessary. I suggest you to view the active alarms and event history to see what failures might lead to reboot the controller for recovery. (could it be a configuration failure?). fm alarm-list fm event-list In addition, /var/log/mtcAgent.log provides more details on why the host is reboot or power-off. Regards, Tao From: Ezpeer Chen [mailto:ezpeerchen@gmail.com] Sent: Friday, June 21, 2019 4:08 AM To: starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Subject: [Starlingx-discuss] How to turn off fault management? Dear all, Environment: STX 1.0 (2018/10) all-in-one simplex How could i turn off fault management which cause my system(controller-0) reboot or power-off? Best Regards