[Starlingx-discuss] Train patch elimination

MacDonald, Eric Eric.MacDonald at windriver.com
Mon Nov 11 19:52:34 UTC 2019


Do you see the same kind of traceback in hwmond_controller-0_thermal_sensor_data  ?



Your issue looks like mtcAgent selected RedFish as the BMC access protocol after getting a good response but when the hardware monitor queries sensors from this server it gets a redfishtool python exception.



Please send me the group of logs for controller-0 from mtcAgent  containing the 'redfishUtil_get_bmc_info' label so that I can understand more about the server you have.



If you like you can execute the following 2 commands to force the system to use ipmitool instead of redfish.



system service-parameter-modify platform maintenance bmc_access_method=ipmi

system service-parameter-apply platform



Wait a few minutes and then try and read your sensors with the system-sensor-list controller-0



I'd be interested in seeing the raw result of the sensor read commands against this server using redfishtool.



time redfishtool -r <ip> -u <un> -p <pw> Chassis Thermal

time redfishtool -r <ip> -u <un> -p <pw> Chassis Power



Basically the same thing but outside daemon control.



Eric.



-----Original Message-----
From: Martinez Monroy, Elio [mailto:elio.martinez.monroy at intel.com]
Sent: Monday, November 11, 2019 12:12 PM
To: MacDonald, Eric; Hu, Yong; Alonso, Juan Carlos; Chen, Yan; starlingx-discuss at lists.starlingx.io
Cc: Cabrales, Ada
Subject: RE: [Starlingx-discuss] Train patch elimination
Importance: High



Hi guys, i checked everything in the way that you suggested.



MtcAgent.log is showing, im working in a Duplex configuration locked and trying to get sensors from controller-0:



p   ( 601) redfishUtil_get_bmc_info: Info : controller-0 has  2 Processors ; Ena                                                                                                                               bled and OK:OK

2019-11-11T10:38:49.554 [1945193.00281] controller-1 mtcAgent --- redfishUtil.cp                                                                                                                               p   ( 624) redfishUtil_get_bmc_info: Info : controller-0 has 96 GiB Memory ; Ena                                                                                                                               bled and OK:OK

2019-11-11T10:38:49.554 [1945193.00282] controller-1 mtcAgent |-| mtcNodeHdlrs.c                                                                                                                               pp  (6492) bmc_handler             : Info : controller-0 bmc audit timer started                                                                                                                                (120 secs)

2019-11-11T10:38:49.554 [1945193.00283] controller-1 mtcAgent |-| mtcNodeHdlrs.c                                                                                                                               pp  (6507) bmc_handler             : Info : controller-0 bmc is accessible using



cd /var/run/bmc/

[sysadmin at controller-1 bmc(keystone_admin)]$ ls

controller-0  controller-1  ipmitool  redfishtool



cat /var/run/bmc/

cat controller-0

{"protocol":"redfishtool","power_state":"on"}



cat controller-1

{"protocol":"redfishtool","power_state":"on"}



cd ipmitool

controller-1:/var/run/bmc/ipmitool$

controller-1:/var/run/bmc/ipmitool$

controller-1:/var/run/bmc/ipmitool$ ls

controller-1:/var/run/bmc/ipmitool$



edfishtool$ ls

hwmond_controller-0_power_sensor_data    hwmond_controller-1_power_sensor_data    mtcAgent_controller-0_bmc_info    mtcAgent_controller-1_bmc_info

hwmond_controller-0_thermal_sensor_data  hwmond_controller-1_thermal_sensor_data  mtcAgent_controller-0_root_query  mtcAgent_controller-1_root_query





If I check any of the files I got this:



cat hwmond_controller-0_power_sensor_data

  redfishtool:Transport: Fatal timeout waiting for response from rhostTraceback (most recent call last):

  File "/usr/bin/redfishtool", line 18, in <module>

    main(sys.argv)

  File "/usr/lib64/python2.7/site-packages/redfishtool/redfishtoolMain.py", line 372, in main

    rc,r,j,d=runSubCmd(rft)

  File "/usr/lib64/python2.7/site-packages/redfishtool/redfishtoolMain.py", line 449, in runSubCmd

    rc,r,j,d=subCmdTable[rft.subcommand](rft,cmdTop=True)

  File "/usr/lib64/python2.7/site-packages/redfishtool/Chassis.py", line 157, in ChassisMain

    rc,r,j,d = self.runOperation(rft)

  File "/usr/lib64/python2.7/site-packages/redfishtool/Chassis.py", line 113, in runOperation

    rc,r,j,d=operationTable[self.operation](self, op, rft, cmdTop=True)

  File "/usr/lib64/python2.7/site-packages/redfishtool/Chassis.py", line 428, in getPower

    rc,r,j,d=op.get(sc, op, rft, cmdTop, resName)

  File "/usr/lib64/python2.7/site-packages/redfishtool/Chassis.py", line 234, in get

    sysPath,rc,r,j,d=rft.getPathBy(rft, r, d, prop)

  File "/usr/lib64/python2.7/site-packages/redfishtool/redfishtoolTransport.py", line 885, in getPathBy

    rc, r, j, d = rft.listCollection(rft, r, coll, prop)

  File "/usr/lib64/python2.7/site-packages/redfishtool/redfishtoolTransport.py", line 1025, in listCollection

    rc,r,j,d=rft.rftSendRecvRequest(rft.AUTHENTICATED_API, 'GET', r.url, relPath=path)

AttributeError: 'NoneType' object has no attribute 'url'





So , my question should be, is the redfishtool blocking ipmitool to show the values? Because impitool is showing sensors by itself.



BR



Elio





-----Original Message-----

From: MacDonald, Eric <Eric.MacDonald at windriver.com>

Sent: Monday, November 11, 2019 8:33 AM

To: Hu, Yong <yong.hu at intel.com>; Alonso, Juan Carlos <juan.carlos.alonso at intel.com>; Chen, Yan <yan.chen at intel.com>; starlingx-discuss at lists.starlingx.io

Cc: Cabrales, Ada <ada.cabrales at intel.com>

Subject: Re: [Starlingx-discuss] Train patch elimination



Yong's suggestion to check the server's BMC settings is good.

Some servers such as Dell, have an 'Enable IPMI over LAN setting.



Do you see a BMC access alarm ?

> fm alarm-list



Try to ping the target server BMC.



If that works then I suggest you try and read the sensors manually using ipmitool.



                ipmitool -I lanplus -H <ip address> -U <username> -P <password> sensor list



If a list of sensors is displayed and you can't see the sensor list with 'system host-sensor-list' then



1. Inspect recent logs in /var/log/mtcAgent.log and var/log/hwmond.log looking for logs that pertain to BMC accesses.



2. Check that your provisioned username, ip address and password are correct.

     Try deprovisioning and reprovisioning the BMC.



    > system host-update <hostname> bm_type=none

    And then provision again ensuring the credentials are correct.



   Wait 5-10 minutes



   Tail mtcAgent and hwmond.log files. If there is a failure the logs will help us narrow in on the issue.



3. Provide output of the following commands



   cat /var/run/bmc/*

   ls -r /var/run/bmc/*



4. Ideally compress and attach the mtcAgent.log file



Eric.



-----Original Message-----

From: Yong Hu [mailto:yong.hu at intel.com]

Sent: Sunday, November 10, 2019 8:27 PM

To: Alonso, Juan Carlos; Chen, Yan; starlingx-discuss at lists.starlingx.io

Cc: Cabrales, Ada

Subject: Re: [Starlingx-discuss] Train patch elimination



JC,

Have you enabled BMC config in your test servers and BIOS?

If so, please use "ipmitool" utility to have a try to read sensor info.



regards,

Yong





On 2019/11/8 11:29 PM, Alonso, Juan Carlos wrote:

> Hello Yan/community,

>

> Any update on this? (:

>

> Regards.

>

> Juan Carlos Alonso

>

> *From:* Alonso, Juan Carlos

> *Sent:* Thursday, November 7, 2019 9:56 AM

> *To:* Chen, Yan <yan.chen at intel.com>; Hu, Yong <yong.hu at intel.com>;

> Gerry.Kopec at windriver.com; starlingx-discuss at lists.starlingx.io

> *Cc:* Cabrales, Ada <ada.cabrales at intel.com>

> *Subject:* RE: Train patch elimination

>

> Hello Yan,

>

> Thank you for your response.

>

> Same question now for:

>

> _Horizon/Custom Confirmation Messages : Custom confirmation message

> for Suppress Sensor Group, Relearn Sensor Model and Suppress Sensor._

>

> I added the bmc_ip, bmc_type, bmc_username and bmc_password for each

> node, but I cannot display any sensors on Horizon nor CLI.

>

> $ system host-update compute-0 bm_username=<user_name>

> bm_password=<password> bm_type=bmc bm_ip=<bmc_ip>

>

> $ system host-lock compute-0

>

> $ system host-sensor-list compute-0

>

> <none>

>

> $ system host-unlock compute-0

>

> $ system host-sensor-list compute-0

>

> <none>

>

> Is there any other configuration needed or is this feature not

> supported anymore?

>

> Regards.

>

> Juan Carlos Alonso

>

> *From:* Chen, Yan <yan.chen at intel.com <mailto:yan.chen at intel.com>>

> *Sent:* Thursday, November 7, 2019 12:15 AM

> *To:* Alonso, Juan Carlos <juan.carlos.alonso at intel.com

> <mailto:juan.carlos.alonso at intel.com>>; Hu, Yong <yong.hu at intel.com

> <mailto:yong.hu at intel.com>>; Gerry.Kopec at windriver.com

> <mailto:Gerry.Kopec at windriver.com>;

> starlingx-discuss at lists.starlingx.io

> <mailto:starlingx-discuss at lists.starlingx.io>

> *Cc:* Cabrales, Ada <ada.cabrales at intel.com

> <mailto:ada.cabrales at intel.com>>

> *Subject:* RE: Train patch elimination

>

> Carlos,

>

> Please remove _Horizon/CSRF AngularJS Fixes_ related cases in your

> test plan, which are originally planned for STX additional patches, as

> those patches are already removed from our project and not upstreamed.

>

> Yan

>

> *From:* Alonso, Juan Carlos <juan.carlos.alonso at intel.com

> <mailto:juan.carlos.alonso at intel.com>>

> *Sent:* Wednesday, November 6, 2019 23:18

> *To:* Hu, Yong <yong.hu at intel.com <mailto:yong.hu at intel.com>>; Chen,

> Yan <yan.chen at intel.com <mailto:yan.chen at intel.com>>;

> Gerry.Kopec at windriver.com <mailto:Gerry.Kopec at windriver.com>;

> starlingx-discuss at lists.starlingx.io

> <mailto:starlingx-discuss at lists.starlingx.io>

> *Cc:* Cabrales, Ada <ada.cabrales at intel.com

> <mailto:ada.cabrales at intel.com>>

> *Subject:* RE: Train patch elimination

>

> Hello,

>

> I hope also the community can help with this questions.

>

> @Chen, Yan <mailto:yan.chen at intel.com>

>

> Can you help me to verify if this feature is still disabled on Train

> version: _Horizon/CSRF AngularJS Fixes_?

>

> On last test plan for Stein this feature was no longer required on

> Horizon and was disabled on SarlingX code.

>

> @Gerry.Kopec at windriver.com <mailto:Gerry.Kopec at windriver.com>

>

> About the Live Migration bugs I mentioned on the last meeting, almost

> all of them are merged, except the last three.

>

> All those patches are related to live migration, so my question is if

> I perform live migrations such patches will be covered or should be

> tested in a different way?

>

> The last three were marked as Unlikely and was not merged on Stein. I

> don’t have more info about such patches.

>

> Enable delete bound trunk for linux bridge agent:

> https://review.opendev.org/#/c/605589

>

> 0040-rpc-removing-timeout-backoff-multiplier.patch (2e4ca00):

> https://review.opendev.org/#/c/623401 and

> https://review.opendev.org/#/c/626109

>

> 0062-dvr-force-admin-state-update-before-distri.patch (edf9731):

> https://review.opendev.org/#/c/625134

>

> 0065-dvr-do-not-create-agent-gateway-ports-unle.patch (2857911):

> https://review.opendev.org/#/c/628071

>

> 0077-api-reject-routes-with-invalid-network-val.patch (6ed8fe6):

> https://review.opendev.org/#/c/623415

>

> 0014-add-support-for-querying-quotas-with-usage.patch (71c07d7):

> _https://review.opendev.org/#/c/621507/_

>

> Prevent DHCP from processing stale RPC messages: ??

>

> 0039-dhcp-handle-concurrent-port-creation-error.patch (a38f89): ??

>

> 0047-wsgi-prevent-accepting-socket-without-a-gr.patch (8e72491): ??

>

> Regards.

>

> Juan Carlos Alonso

>



_______________________________________________

Starlingx-discuss mailing list

Starlingx-discuss at lists.starlingx.io

http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss

_______________________________________________

Starlingx-discuss mailing list

Starlingx-discuss at lists.starlingx.io

http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.starlingx.io/pipermail/starlingx-discuss/attachments/20191111/9c7aee3f/attachment-0001.html>


More information about the Starlingx-discuss mailing list