[Starlingx-discuss] Duplex Configuration: Unlock of controller-0 causes reboot
Hi, when I unlock the controller-0 after installation following this tutorial https://docs.starlingx.io/installation_guide/duplex.html#duplex the system is rebooting automatically with a lot of errors from drdb. Controller-1 is not configured at this point in time. Must I configure the secondary controller before unlocking controller-0? Thanks Marcel
Yes, this is the correct behavior. As for the drbd warnings, there are some state change failures that occur as the system shuts down, but aren't a problem (aside from the noise). I've managed to clean up most of these in a recent update. What build are you using? -----Original Message----- From: Marcel Schaible [mailto:marcel@schaible-consulting.de] Sent: Wednesday, February 13, 2019 11:28 AM To: starlingx-discuss@lists.starlingx.io Subject: [Starlingx-discuss] Duplex Configuration: Unlock of controller-0 causes reboot Hi, when I unlock the controller-0 after installation following this tutorial https://docs.starlingx.io/installation_guide/duplex.html#duplex the system is rebooting automatically with a lot of errors from drdb. Controller-1 is not configured at this point in time. Must I configure the secondary controller before unlocking controller-0? Thanks Marcel _______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
I am using the build centos-20190206T060000Z. The main problem, beside of the noise is, that the system comes up and reboots immediately again.
"Penney, Don" <Don.Penney@windriver.com> hat am 13. Februar 2019 um 17:41 geschrieben:
Yes, this is the correct behavior.
As for the drbd warnings, there are some state change failures that occur as the system shuts down, but aren't a problem (aside from the noise). I've managed to clean up most of these in a recent update. What build are you using?
-----Original Message----- From: Marcel Schaible [mailto:marcel@schaible-consulting.de] Sent: Wednesday, February 13, 2019 11:28 AM To: starlingx-discuss@lists.starlingx.io Subject: [Starlingx-discuss] Duplex Configuration: Unlock of controller-0 causes reboot
Hi,
when I unlock the controller-0 after installation following this tutorial
https://docs.starlingx.io/installation_guide/duplex.html#duplex
the system is rebooting automatically with a lot of errors from drdb.
Controller-1 is not configured at this point in time.
Must I configure the secondary controller before unlocking controller-0?
Thanks
Marcel
_______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
Ok, my updates related to the DRBD state change failures merged Feb 8th. I wouldn't expect to see an additional reboot. So what you're seeing, just to confirm, is: * install node, run config_controller, do config steps * system host-unlock controller-0 * system reboots * system reboots a second time? At what point is it rebooting a second time? Maybe a reboot is being triggered due to config changes made when the puppet manifests apply, which I think we could confirm from the puppet logs. -----Original Message----- From: Marcel Schaible [mailto:marcel@schaible-consulting.de] Sent: Wednesday, February 13, 2019 11:48 AM To: Penney, Don; starlingx-discuss@lists.starlingx.io Subject: Re: [Starlingx-discuss] Duplex Configuration: Unlock of controller-0 causes reboot I am using the build centos-20190206T060000Z. The main problem, beside of the noise is, that the system comes up and reboots immediately again.
"Penney, Don" <Don.Penney@windriver.com> hat am 13. Februar 2019 um 17:41 geschrieben:
Yes, this is the correct behavior.
As for the drbd warnings, there are some state change failures that occur as the system shuts down, but aren't a problem (aside from the noise). I've managed to clean up most of these in a recent update. What build are you using?
-----Original Message----- From: Marcel Schaible [mailto:marcel@schaible-consulting.de] Sent: Wednesday, February 13, 2019 11:28 AM To: starlingx-discuss@lists.starlingx.io Subject: [Starlingx-discuss] Duplex Configuration: Unlock of controller-0 causes reboot
Hi,
when I unlock the controller-0 after installation following this tutorial
https://docs.starlingx.io/installation_guide/duplex.html#duplex
the system is rebooting automatically with a lot of errors from drdb.
Controller-1 is not configured at this point in time.
Must I configure the secondary controller before unlocking controller-0?
Thanks
Marcel
_______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
_______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
At the moment the system seems to stablize after 2 reboots. I'll keep an eye on that. Is the alarm list looking reasonable after the controller-0 is unlocked and rebooted? If yes, I would start to configure the controller-1. root@controller-0 ~(keystone_admin)]# fm alarm-list +----------+-------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------+----------+----------------------------+ | Alarm ID | Reason Text | Entity ID | Severity | Time Stamp | +----------+-------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------+----------+----------------------------+ | 200.011 | controller-0 experienced a configuration failure. | host=controller-0 | critical | 2019-02-13T18:57:13.772108 | | 400.002 | Service group controller-services loss of redundancy; expected 1 standby member but no standby members available | service_domain=controller.service_group=controller-services | major | 2019-02-13T18:56:15.215083 | | 400.002 | Service group vim-services loss of redundancy; expected 1 standby member but no standby members available | service_domain=controller.service_group=vim-services | major | 2019-02-13T18:56:15.174083 | | 400.002 | Service group cloud-services loss of redundancy; expected 1 standby member but no standby members available | service_domain=controller.service_group=cloud-services | major | 2019-02-13T18:56:12.213065 | | 400.002 | Service group oam-services loss of redundancy; expected 1 standby member but no standby members available | service_domain=controller.service_group=oam-services | major | 2019-02-13T18:56:05.330099 | | 400.002 | Service group patching-services loss of redundancy; expected 1 standby member but no standby members available | service_domain=controller.service_group=patching-services | major | 2019-02-13T18:56:04.276424 | | 400.002 | Service group directory-services loss of redundancy; expected 2 active members but only 1 active member available | service_domain=controller.service_group=directory-services | major | 2019-02-13T18:56:04.195081 | | 400.002 | Service group web-services loss of redundancy; expected 2 active members but only 1 active member available | service_domain=controller.service_group=web-services | major | 2019-02-13T18:56:04.114064 | | 400.005 | Communication failure detected with peer over port enp9s20f7 on host controller-0 | host=controller-0.network=mgmt | major | 2019-02-13T18:56:03.790070 | | 100.106 | 'OAM' Port failed. | host=controller-0.port=cd202069-f5a0-43bb-a445-d3b9b58ce631 | major | 2019-02-13T18:56:00.563067 | | 100.107 | 'OAM' Interface degraded. | host=controller-0.interface=oam | major | 2019-02-13T18:56:00.515836 | | 300.004 | No enabled compute host with connectivity to provider network. | service=networking.providernet=e9eef26b-d266-4b3f-b1b7-6d9c2cd3a86b | major | 2019-02-13T17:20:21.110405 | +----------+-------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------+----------+----------------------------+ [root@controller-0 ~(keystone_admin)]#
"Penney, Don" <Don.Penney@windriver.com> hat am 13. Februar 2019 um 18:02 geschrieben:
Ok, my updates related to the DRBD state change failures merged Feb 8th.
I wouldn't expect to see an additional reboot. So what you're seeing, just to confirm, is: * install node, run config_controller, do config steps * system host-unlock controller-0 * system reboots * system reboots a second time?
Yes, exactly.
At what point is it rebooting a second time? Maybe a reboot is being triggered due to config changes made when the puppet manifests apply, which I think we could confirm from the puppet logs.
The second reboot starts in about 1 minute after the login prompt is shown.
-----Original Message----- From: Marcel Schaible [mailto:marcel@schaible-consulting.de] Sent: Wednesday, February 13, 2019 11:48 AM To: Penney, Don; starlingx-discuss@lists.starlingx.io Subject: Re: [Starlingx-discuss] Duplex Configuration: Unlock of controller-0 causes reboot
I am using the build centos-20190206T060000Z.
The main problem, beside of the noise is, that the system comes up and reboots immediately again.
"Penney, Don" <Don.Penney@windriver.com> hat am 13. Februar 2019 um 17:41 geschrieben:
Yes, this is the correct behavior.
As for the drbd warnings, there are some state change failures that occur as the system shuts down, but aren't a problem (aside from the noise). I've managed to clean up most of these in a recent update. What build are you using?
-----Original Message----- From: Marcel Schaible [mailto:marcel@schaible-consulting.de] Sent: Wednesday, February 13, 2019 11:28 AM To: starlingx-discuss@lists.starlingx.io Subject: [Starlingx-discuss] Duplex Configuration: Unlock of controller-0 causes reboot
Hi,
when I unlock the controller-0 after installation following this tutorial
https://docs.starlingx.io/installation_guide/duplex.html#duplex
the system is rebooting automatically with a lot of errors from drdb.
Controller-1 is not configured at this point in time.
Must I configure the secondary controller before unlocking controller-0?
Thanks
Marcel
_______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
_______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
I would suggest checking the puppet logs for a failure.... grep for Error. The redundancy alarms are because you don't have controller-1 yet, and can be ignored. The port failure alarm is concerning, I think... maybe that's related to the configuration error. -----Original Message----- From: Marcel Schaible [mailto:marcel@schaible-consulting.de] Sent: Wednesday, February 13, 2019 12:18 PM To: Penney, Don; starlingx-discuss@lists.starlingx.io Subject: RE: [Starlingx-discuss] Duplex Configuration: Unlock of controller-0 causes reboot At the moment the system seems to stablize after 2 reboots. I'll keep an eye on that. Is the alarm list looking reasonable after the controller-0 is unlocked and rebooted? If yes, I would start to configure the controller-1. root@controller-0 ~(keystone_admin)]# fm alarm-list +----------+-------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------+----------+----------------------------+ | Alarm ID | Reason Text | Entity ID | Severity | Time Stamp | +----------+-------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------+----------+----------------------------+ | 200.011 | controller-0 experienced a configuration failure. | host=controller-0 | critical | 2019-02-13T18:57:13.772108 | | 400.002 | Service group controller-services loss of redundancy; expected 1 standby member but no standby members available | service_domain=controller.service_group=controller-services | major | 2019-02-13T18:56:15.215083 | | 400.002 | Service group vim-services loss of redundancy; expected 1 standby member but no standby members available | service_domain=controller.service_group=vim-services | major | 2019-02-13T18:56:15.174083 | | 400.002 | Service group cloud-services loss of redundancy; expected 1 standby member but no standby members available | service_domain=controller.service_group=cloud-services | major | 2019-02-13T18:56:12.213065 | | 400.002 | Service group oam-services loss of redundancy; expected 1 standby member but no standby members available | service_domain=controller.service_group=oam-services | major | 2019-02-13T18:56:05.330099 | | 400.002 | Service group patching-services loss of redundancy; expected 1 standby member but no standby members available | service_domain=controller.service_group=patching-services | major | 2019-02-13T18:56:04.276424 | | 400.002 | Service group directory-services loss of redundancy; expected 2 active members but only 1 active member available | service_domain=controller.service_group=directory-services | major | 2019-02-13T18:56:04.195081 | | 400.002 | Service group web-services loss of redundancy; expected 2 active members but only 1 active member available | service_domain=controller.service_group=web-services | major | 2019-02-13T18:56:04.114064 | | 400.005 | Communication failure detected with peer over port enp9s20f7 on host controller-0 | host=controller-0.network=mgmt | major | 2019-02-13T18:56:03.790070 | | 100.106 | 'OAM' Port failed. | host=controller-0.port=cd202069-f5a0-43bb-a445-d3b9b58ce631 | major | 2019-02-13T18:56:00.563067 | | 100.107 | 'OAM' Interface degraded. | host=controller-0.interface=oam | major | 2019-02-13T18:56:00.515836 | | 300.004 | No enabled compute host with connectivity to provider network. | service=networking.providernet=e9eef26b-d266-4b3f-b1b7-6d9c2cd3a86b | major | 2019-02-13T17:20:21.110405 | +----------+-------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------+----------+----------------------------+ [root@controller-0 ~(keystone_admin)]#
"Penney, Don" <Don.Penney@windriver.com> hat am 13. Februar 2019 um 18:02 geschrieben:
Ok, my updates related to the DRBD state change failures merged Feb 8th.
I wouldn't expect to see an additional reboot. So what you're seeing, just to confirm, is: * install node, run config_controller, do config steps * system host-unlock controller-0 * system reboots * system reboots a second time?
Yes, exactly.
At what point is it rebooting a second time? Maybe a reboot is being triggered due to config changes made when the puppet manifests apply, which I think we could confirm from the puppet logs.
The second reboot starts in about 1 minute after the login prompt is shown.
-----Original Message----- From: Marcel Schaible [mailto:marcel@schaible-consulting.de] Sent: Wednesday, February 13, 2019 11:48 AM To: Penney, Don; starlingx-discuss@lists.starlingx.io Subject: Re: [Starlingx-discuss] Duplex Configuration: Unlock of controller-0 causes reboot
I am using the build centos-20190206T060000Z.
The main problem, beside of the noise is, that the system comes up and reboots immediately again.
"Penney, Don" <Don.Penney@windriver.com> hat am 13. Februar 2019 um 17:41 geschrieben:
Yes, this is the correct behavior.
As for the drbd warnings, there are some state change failures that occur as the system shuts down, but aren't a problem (aside from the noise). I've managed to clean up most of these in a recent update. What build are you using?
-----Original Message----- From: Marcel Schaible [mailto:marcel@schaible-consulting.de] Sent: Wednesday, February 13, 2019 11:28 AM To: starlingx-discuss@lists.starlingx.io Subject: [Starlingx-discuss] Duplex Configuration: Unlock of controller-0 causes reboot
Hi,
when I unlock the controller-0 after installation following this tutorial
https://docs.starlingx.io/installation_guide/duplex.html#duplex
the system is rebooting automatically with a lot of errors from drdb.
Controller-1 is not configured at this point in time.
Must I configure the secondary controller before unlocking controller-0?
Thanks
Marcel
_______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
_______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
Ok. The latest puppet logs from controller-0 do not contain any errors. Should I try to install a newer image?
"Penney, Don" <Don.Penney@windriver.com> hat am 13. Februar 2019 um 18:20 geschrieben:
I would suggest checking the puppet logs for a failure.... grep for Error. The redundancy alarms are because you don't have controller-1 yet, and can be ignored. The port failure alarm is concerning, I think... maybe that's related to the configuration error.
-----Original Message----- From: Marcel Schaible [mailto:marcel@schaible-consulting.de] Sent: Wednesday, February 13, 2019 12:18 PM To: Penney, Don; starlingx-discuss@lists.starlingx.io Subject: RE: [Starlingx-discuss] Duplex Configuration: Unlock of controller-0 causes reboot
At the moment the system seems to stablize after 2 reboots. I'll keep an eye on that.
Is the alarm list looking reasonable after the controller-0 is unlocked and rebooted? If yes, I would start to configure the controller-1.
root@controller-0 ~(keystone_admin)]# fm alarm-list +----------+-------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------+----------+----------------------------+ | Alarm ID | Reason Text | Entity ID | Severity | Time Stamp | +----------+-------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------+----------+----------------------------+ | 200.011 | controller-0 experienced a configuration failure. | host=controller-0 | critical | 2019-02-13T18:57:13.772108 | | 400.002 | Service group controller-services loss of redundancy; expected 1 standby member but no standby members available | service_domain=controller.service_group=controller-services | major | 2019-02-13T18:56:15.215083 | | 400.002 | Service group vim-services loss of redundancy; expected 1 standby member but no standby members available | service_domain=controller.service_group=vim-services | major | 2019-02-13T18:56:15.174083 | | 400.002 | Service group cloud-services loss of redundancy; expected 1 standby member but no standby members available | service_domain=controller.service_group=cloud-services | major | 2019-02-13T18:56:12.213065 | | 400.002 | Service group oam-services loss of redundancy; expected 1 standby member but no standby members available | service_domain=controller.service_group=oam-services | major | 2019-02-13T18:56:05.330099 | | 400.002 | Service group patching-services loss of redundancy; expected 1 standby member but no standby members available | service_domain=controller.service_group=patching-services | major | 2019-02-13T18:56:04.276424 | | 400.002 | Service group directory-services loss of redundancy; expected 2 active members but only 1 active member available | service_domain=controller.service_group=directory-services | major | 2019-02-13T18:56:04.195081 | | 400.002 | Service group web-services loss of redundancy; expected 2 active members but only 1 active member available | service_domain=controller.service_group=web-services | major | 2019-02-13T18:56:04.114064 | | 400.005 | Communication failure detected with peer over port enp9s20f7 on host controller-0 | host=controller-0.network=mgmt | major | 2019-02-13T18:56:03.790070 | | 100.106 | 'OAM' Port failed. | host=controller-0.port=cd202069-f5a0-43bb-a445-d3b9b58ce631 | major | 2019-02-13T18:56:00.563067 | | 100.107 | 'OAM' Interface degraded. | host=controller-0.interface=oam | major | 2019-02-13T18:56:00.515836 | | 300.004 | No enabled compute host with connectivity to provider network. | service=networking.providernet=e9eef26b-d266-4b3f-b1b7-6d9c2cd3a86b | major | 2019-02-13T17:20:21.110405 | +----------+-------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------+----------+----------------------------+ [root@controller-0 ~(keystone_admin)]#
"Penney, Don" <Don.Penney@windriver.com> hat am 13. Februar 2019 um 18:02 geschrieben:
Ok, my updates related to the DRBD state change failures merged Feb 8th.
I wouldn't expect to see an additional reboot. So what you're seeing, just to confirm, is: * install node, run config_controller, do config steps * system host-unlock controller-0 * system reboots * system reboots a second time?
Yes, exactly.
At what point is it rebooting a second time? Maybe a reboot is being triggered due to config changes made when the puppet manifests apply, which I think we could confirm from the puppet logs.
The second reboot starts in about 1 minute after the login prompt is shown.
-----Original Message----- From: Marcel Schaible [mailto:marcel@schaible-consulting.de] Sent: Wednesday, February 13, 2019 11:48 AM To: Penney, Don; starlingx-discuss@lists.starlingx.io Subject: Re: [Starlingx-discuss] Duplex Configuration: Unlock of controller-0 causes reboot
I am using the build centos-20190206T060000Z.
The main problem, beside of the noise is, that the system comes up and reboots immediately again.
"Penney, Don" <Don.Penney@windriver.com> hat am 13. Februar 2019 um 17:41 geschrieben:
Yes, this is the correct behavior.
As for the drbd warnings, there are some state change failures that occur as the system shuts down, but aren't a problem (aside from the noise). I've managed to clean up most of these in a recent update. What build are you using?
-----Original Message----- From: Marcel Schaible [mailto:marcel@schaible-consulting.de] Sent: Wednesday, February 13, 2019 11:28 AM To: starlingx-discuss@lists.starlingx.io Subject: [Starlingx-discuss] Duplex Configuration: Unlock of controller-0 causes reboot
Hi,
when I unlock the controller-0 after installation following this tutorial
https://docs.starlingx.io/installation_guide/duplex.html#duplex
the system is rebooting automatically with a lot of errors from drdb.
Controller-1 is not configured at this point in time.
Must I configure the secondary controller before unlocking controller-0?
Thanks
Marcel
_______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
_______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
You'd need to look at the puppet log from the first reboot to see if some failure occurred there, or whether the reboot was triggered by a config change. I don't know if the configuration failure alarm is related to puppet or not. Given the OAM port alarm, you should double-check your OAM interface and configuration. Maybe John Kung has better advice on what to check... John? -----Original Message----- From: Marcel Schaible [mailto:marcel@schaible-consulting.de] Sent: Wednesday, February 13, 2019 12:27 PM To: Penney, Don; starlingx-discuss@lists.starlingx.io Subject: RE: [Starlingx-discuss] Duplex Configuration: Unlock of controller-0 causes reboot Ok. The latest puppet logs from controller-0 do not contain any errors. Should I try to install a newer image?
"Penney, Don" <Don.Penney@windriver.com> hat am 13. Februar 2019 um 18:20 geschrieben:
I would suggest checking the puppet logs for a failure.... grep for Error. The redundancy alarms are because you don't have controller-1 yet, and can be ignored. The port failure alarm is concerning, I think... maybe that's related to the configuration error.
-----Original Message----- From: Marcel Schaible [mailto:marcel@schaible-consulting.de] Sent: Wednesday, February 13, 2019 12:18 PM To: Penney, Don; starlingx-discuss@lists.starlingx.io Subject: RE: [Starlingx-discuss] Duplex Configuration: Unlock of controller-0 causes reboot
At the moment the system seems to stablize after 2 reboots. I'll keep an eye on that.
Is the alarm list looking reasonable after the controller-0 is unlocked and rebooted? If yes, I would start to configure the controller-1.
root@controller-0 ~(keystone_admin)]# fm alarm-list +----------+-------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------+----------+----------------------------+ | Alarm ID | Reason Text | Entity ID | Severity | Time Stamp | +----------+-------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------+----------+----------------------------+ | 200.011 | controller-0 experienced a configuration failure. | host=controller-0 | critical | 2019-02-13T18:57:13.772108 | | 400.002 | Service group controller-services loss of redundancy; expected 1 standby member but no standby members available | service_domain=controller.service_group=controller-services | major | 2019-02-13T18:56:15.215083 | | 400.002 | Service group vim-services loss of redundancy; expected 1 standby member but no standby members available | service_domain=controller.service_group=vim-services | major | 2019-02-13T18:56:15.174083 | | 400.002 | Service group cloud-services loss of redundancy; expected 1 standby member but no standby members available | service_domain=controller.service_group=cloud-services | major | 2019-02-13T18:56:12.213065 | | 400.002 | Service group oam-services loss of redundancy; expected 1 standby member but no standby members available | service_domain=controller.service_group=oam-services | major | 2019-02-13T18:56:05.330099 | | 400.002 | Service group patching-services loss of redundancy; expected 1 standby member but no standby members available | service_domain=controller.service_group=patching-services | major | 2019-02-13T18:56:04.276424 | | 400.002 | Service group directory-services loss of redundancy; expected 2 active members but only 1 active member available | service_domain=controller.service_group=directory-services | major | 2019-02-13T18:56:04.195081 | | 400.002 | Service group web-services loss of redundancy; expected 2 active members but only 1 active member available | service_domain=controller.service_group=web-services | major | 2019-02-13T18:56:04.114064 | | 400.005 | Communication failure detected with peer over port enp9s20f7 on host controller-0 | host=controller-0.network=mgmt | major | 2019-02-13T18:56:03.790070 | | 100.106 | 'OAM' Port failed. | host=controller-0.port=cd202069-f5a0-43bb-a445-d3b9b58ce631 | major | 2019-02-13T18:56:00.563067 | | 100.107 | 'OAM' Interface degraded. | host=controller-0.interface=oam | major | 2019-02-13T18:56:00.515836 | | 300.004 | No enabled compute host with connectivity to provider network. | service=networking.providernet=e9eef26b-d266-4b3f-b1b7-6d9c2cd3a86b | major | 2019-02-13T17:20:21.110405 | +----------+-------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------+----------+----------------------------+ [root@controller-0 ~(keystone_admin)]#
"Penney, Don" <Don.Penney@windriver.com> hat am 13. Februar 2019 um 18:02 geschrieben:
Ok, my updates related to the DRBD state change failures merged Feb 8th.
I wouldn't expect to see an additional reboot. So what you're seeing, just to confirm, is: * install node, run config_controller, do config steps * system host-unlock controller-0 * system reboots * system reboots a second time?
Yes, exactly.
At what point is it rebooting a second time? Maybe a reboot is being triggered due to config changes made when the puppet manifests apply, which I think we could confirm from the puppet logs.
The second reboot starts in about 1 minute after the login prompt is shown.
-----Original Message----- From: Marcel Schaible [mailto:marcel@schaible-consulting.de] Sent: Wednesday, February 13, 2019 11:48 AM To: Penney, Don; starlingx-discuss@lists.starlingx.io Subject: Re: [Starlingx-discuss] Duplex Configuration: Unlock of controller-0 causes reboot
I am using the build centos-20190206T060000Z.
The main problem, beside of the noise is, that the system comes up and reboots immediately again.
"Penney, Don" <Don.Penney@windriver.com> hat am 13. Februar 2019 um 17:41 geschrieben:
Yes, this is the correct behavior.
As for the drbd warnings, there are some state change failures that occur as the system shuts down, but aren't a problem (aside from the noise). I've managed to clean up most of these in a recent update. What build are you using?
-----Original Message----- From: Marcel Schaible [mailto:marcel@schaible-consulting.de] Sent: Wednesday, February 13, 2019 11:28 AM To: starlingx-discuss@lists.starlingx.io Subject: [Starlingx-discuss] Duplex Configuration: Unlock of controller-0 causes reboot
Hi,
when I unlock the controller-0 after installation following this tutorial
https://docs.starlingx.io/installation_guide/duplex.html#duplex
the system is rebooting automatically with a lot of errors from drdb.
Controller-1 is not configured at this point in time.
Must I configure the secondary controller before unlocking controller-0?
Thanks
Marcel
_______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
_______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
Hi Marcel Can you please share us the output of the drbd error? Thanks. Best regards. Mario. ________________________________________ From: Marcel Schaible [marcel@schaible-consulting.de] Sent: Wednesday, February 13, 2019 8:28 AM To: starlingx-discuss@lists.starlingx.io Subject: [Starlingx-discuss] Duplex Configuration: Unlock of controller-0 causes reboot Hi, when I unlock the controller-0 after installation following this tutorial https://docs.starlingx.io/installation_guide/duplex.html#duplex the system is rebooting automatically with a lot of errors from drdb. Controller-1 is not configured at this point in time. Must I configure the secondary controller before unlocking controller-0? Thanks Marcel _______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
controller-0 login: wrsroot Password: [ 447.539473] block drbd5: State change failed: Device is held open by someone [ 447.547353] block drbd5: state = { cs:WFConnection ro:Primary/Unknown ds:UpToDate/DUnknown r----- } [ 447.557656] block drbd5: wanted = { cs:WFConnection ro:Secondary/Unknown ds:UpToDate/DUnknown r----- } [ 447.616216] block drbd1: State change failed: Device is held open by someone [ 447.624093] block drbd1: state = { cs:WFConnection ro:Primary/Unknown ds:UpToDate/DUnknown r----- } [ 447.625520] block drbd2: State change failed: Device is held open by someone [ 447.625523] block drbd2: state = { cs:WFConnection ro:Primary/Unknown ds:UpToDate/DUnknown r----- } [ 447.625524] block drbd2: wanted = { cs:WFConnection ro:Secondary/Unknown ds:UpToDate/DUnknown r----- } [ 447.663063] block drbd1: wanted = { cs:WFConnection ro:Secondary/Unknown ds:UpToDate/DUnknown r----- } [ 447.695930] block drbd0: State change failed: Device is held open by someone [ 447.703809] block drbd0: state = { cs:WFConnection ro:Primary/Unknown ds:UpToDate/DUnknown r----- } [ 447.714117] block drbd0: wanted = { cs:WFConnection ro:Secondary/Unknown ds:UpToDate/DUnknown r----- } [ 447.717735] block drbd5: State change failed: Device is held open by someone [ 447.717737] block drbd5: state = { cs:WFConnection ro:Primary/Unknown ds:UpToDate/DUnknown r----- } [ 447.717739] block drbd5: wanted = { cs:WFConnection ro:Secondary/Unknown ds:UpToDate/DUnknown r----- } [ 447.744998] block drbd2: State change failed: Device is held open by someone [ 447.745001] block drbd2: state = { cs:WFConnection ro:Primary/Unknown ds:UpToDate/DUnknown r----- } [ 447.745002] block drbd2: wanted = { cs:WFConnection ro:Secondary/Unknown ds:UpToDate/DUnknown r----- } [ 447.790092] block drbd1: State change failed: Device is held open by someone [ 447.792927] block drbd0: State change failed: Device is held open by someone [ 447.792930] block drbd0: state = { cs:WFConnection ro:Primary/Unknown ds:UpToDate/DUnknown r----- } [ 447.792931] block drbd0: wanted = { cs:WFConnection ro:Secondary/Unknown ds:UpToDate/DUnknown r----- } [ 447.826631] block drbd1: state = { cs:WFConnection ro:Primary/Unknown ds:UpToDate/DUnknown r----- } [ 447.826633] block drbd1: wanted = { cs:WFConnection ro:Secondary/Unknown ds:UpToDate/DUnknown r----- } [ 448.813520] block drbd2: State change failed: Device is held open by someone [ 448.816397] block drbd0: State change failed: Device is held open by someone [ 448.816399] block drbd0: state = { cs:WFConnection ro:Primary/Unknown ds:UpToDate/DUnknown r----- } [ 448.816400] block drbd0: wanted = { cs:WFConnection ro:Secondary/Unknown ds:UpToDate/DUnknown r----- } [ 448.850061] block drbd2: state = { cs:WFConnection ro:Primary/Unknown ds:UpToDate/DUnknown r----- } [ 448.850062] block drbd2: wanted = { cs:WFConnection ro:Secondary/Unknown ds:UpToDate/DUnknown r----- } [ 448.851141] block drbd1: State change failed: Device is held open by someone [ 448.851143] block drbd1: state = { cs:WFConnection ro:Primary/Unknown ds:UpToDate/DUnknown r----- } [ 448.851144] block drbd1: wanted = { cs:WFConnection ro:Secondary/Unknown ds:UpToDate/DUnknown r----- } [ 449.902198] block drbd0: State change failed: Device is held open by someone [ 449.910077] block drbd0: state = { cs:WFConnection ro:Primary/Unknown ds:UpToDate/DUnknown r----- } [ 449.920380] block drbd0: wanted = { cs:WFConnection ro:Secondary/Unknown ds:UpToDate/DUnknown r----- } [ 449.933114] block drbd1: State change failed: Device is held open by someone [ 449.935475] block drbd2: State change failed: Device is held open by someone [ 449.935478] block drbd2: state = { cs:WFConnection ro:Primary/Unknown ds:UpToDate/DUnknown r----- } [ 449.935479] block drbd2: wanted = { cs:WFConnection ro:Secondary/Unknown ds:UpToDate/DUnknown r----- } [ 449.969656] block drbd1: state = { cs:WFConnection ro:Primary/Unknown ds:UpToDate/DUnknown r----- } [ 449.969658] block drbd1: wanted = { cs:WFConnection ro:Secondary/Unknown ds:UpToDate/DUnknown r----- } [ 450.975871] block drbd2: State change failed: Device is held open by someone [ 450.983748] block drbd2: state = { cs:WFConnection ro:Primary/Unknown ds:UpToDate/DUnknown r----- } [ 450.994052] block drbd2: wanted = { cs:WFConnection ro:Secondary/Unknown ds:UpToDate/DUnknown r----- } [ 451.015768] block drbd0: State change failed: Device is held open by someone [ 451.023643] block drbd0: state = { cs:WFConnection ro:Primary/Unknown ds:UpToDate/DUnknown r----- } [ 451.033947] block drbd0: wanted = { cs:WFConnection ro:Secondary/Unknown ds:UpToDate/DUnknown r----- } [ 451.038700] block drbd1: State change failed: Device is held open by someone [ 451.038702] block drbd1: state = { cs:WFConnection ro:Primary/Unknown ds:UpToDate/DUnknown r----- } [ 451.038703] block drbd1: wanted = { cs:WFConnection ro:Secondary/Unknown ds:UpToDate/DUnknown r----- } [ 451.954434] block drbd3: State change failed: Device is held open by someone [ 451.962309] block drbd3: state = { cs:WFConnection ro:Primary/Unknown ds:UpToDate/DUnknown r----- } [ 451.972608] block drbd3: wanted = { cs:WFConnection ro:Secondary/Unknown ds:UpToDate/DUnknown r----- } [ 452.007079] block drbd3: State change failed: Device is held open by someone [ 452.012542] block drbd2: State change failed: Device is held open by someone [ 452.012544] block drbd2: state = { cs:WFConnection ro:Primary/Unknown ds:UpToDate/DUnknown r----- } [ 452.012545] block drbd2: wanted = { cs:WFConnection ro:Secondary/Unknown ds:UpToDate/DUnknown r----- } [ 452.043617] block drbd3: state = { cs:WFConnection ro:Primary/Unknown ds:UpToDate/DUnknown r----- } [ 452.043619] block drbd3: wanted = { cs:WFConnection ro:Secondary/Unknown ds:UpToDate/DUnknown r----- } [ 452.074582] block drbd1: State change failed: Device is held open by someone [ 452.080401] block drbd0: State change failed: Device is held open by someone [ 452.080403] block drbd0: state = { cs:WFConnection ro:Primary/Unknown ds:UpToDate/DUnknown r----- } [ 452.080404] block drbd0: wanted = { cs:WFConnection ro:Secondary/Unknown ds:UpToDate/DUnknown r----- } [ 452.111118] block drbd1: state = { cs:WFConnection ro:Primary/Unknown ds:UpToDate/DUnknown r----- } [ 452.111120] block drbd1: wanted = { cs:WFConnection ro:Secondary/Unknown ds:UpToDate/DUnknown r----- } [ 453.045601] block drbd2: State change failed: Device is held open by someone [ 453.053478] block drbd2: state = { cs:WFConnection ro:Primary/Unknown ds:UpToDate/DUnknown r----- } [ 453.063780] block drbd2: wanted = { cs:WFConnection ro:Secondary/Unknown ds:UpToDate/DUnknown r----- } [ 453.096723] block drbd3: State change failed: Device is held open by someone [ 453.104599] block drbd3: state = { cs:WFConnection ro:Primary/Unknown ds:UpToDate/DUnknown r----- } [ 453.114898] block drbd3: wanted = { cs:WFConnection ro:Secondary/Unknown ds:UpToDate/DUnknown r----- } [ 453.137325] block drbd0: State change failed: Device is held open by someone [ 453.142791] block drbd1: State change failed: Device is held open by someone [ 453.142793] block drbd1: state = { cs:WFConnection ro:Primary/Unknown ds:UpToDate/DUnknown r----- } [ 453.142794] block drbd1: wanted = { cs:WFConnection ro:Secondary/Unknown ds:UpToDate/DUnknown r----- } [ 453.173863] block drbd0: state = { cs:WFConnection ro:Primary/Unknown ds:UpToDate/DUnknown r----- } [ 453.173864] block drbd0: wanted = { cs:WFConnection ro:Secondary/Unknown ds:UpToDate/DUnknown r----- } [ 454.091972] block drbd2: State change failed: Device is held open by someone [ 454.099849] block drbd2: state = { cs:WFConnection ro:Primary/Unknown ds:UpToDate/DUnknown r----- } [ 454.110150] block drbd2: wanted = { cs:WFConnection ro:Secondary/Unknown ds:UpToDate/DUnknown r----- } [ 454.145690] block drbd3: State change failed: Device is held open by someone [ 454.153561] block drbd3: state = { cs:WFConnection ro:Primary/Unknown ds:UpToDate/DUnknown r----- } [ 454.163867] block drbd3: wanted = { cs:WFConnection ro:Secondary/Unknown ds:UpToDate/DUnknown r----- } [ 454.176301] block drbd1: State change failed: Device is held open by someone [ 454.184182] block drbd1: state = { cs:WFConnection ro:Primary/Unknown ds:UpToDate/DUnknown r----- } [ 454.184184] block drbd1: wanted = { cs:WFConnection ro:Secondary/Unknown ds:UpToDate/DUnknown r----- } [ 454.207939] block drbd0: State change failed: Device is held open by someone [ 454.215814] block drbd0: state = { cs:WFConnection ro:Primary/Unknown ds:UpToDate/DUnknown r----- } [ 454.226114] block drbd0: wanted = { cs:WFConnection ro:Secondary/Unknown ds:UpToDate/DUnknown r----- } [ 455.139180] block drbd2: State change failed: Device is held open by someone [ 455.147055] block drbd2: state = { cs:WFConnection ro:Primary/Unknown ds:UpToDate/DUnknown r----- } [ 455.157356] block drbd2: wanted = { cs:WFConnection ro:Secondary/Unknown ds:UpToDate/DUnknown r----- } [ 455.187247] block drbd3: State change failed: Device is held open by someone [ 455.195124] block drbd3: state = { cs:WFConnection ro:Primary/Unknown ds:UpToDate/DUnknown r----- } [ 455.205430] block drbd3: wanted = { cs:WFConnection ro:Secondary/Unknown ds:UpToDate/DUnknown r----- } [ 455.206796] block drbd1: State change failed: Device is held open by someone [ 455.206798] block drbd1: state = { cs:WFConnection ro:Primary/Unknown ds:UpToDate/DUnknown r----- } [ 455.206800] block drbd1: wanted = { cs:WFConnection ro:Secondary/Unknown ds:UpToDate/DUnknown r----- } [ 455.254455] block drbd0: State change failed: Device is held open by someone [ 455.262332] block drbd0: state = { cs:WFConnection ro:Primary/Unknown ds:UpToDate/DUnknown r----- } [ 455.272633] block drbd0: wanted = { cs:WFConnection ro:Secondary/Unknown ds:UpToDate/DUnknown r----- } [ 456.186309] block drbd2: State change failed: Device is held open by someone [ 456.194185] block drbd2: state = { cs:WFConnection ro:Primary/Unknown ds:UpToDate/DUnknown r----- } [ 456.204488] block drbd2: wanted = { cs:WFConnection ro:Secondary/Unknown ds:UpToDate/DUnknown r----- } [ 456.223846] block drbd1: State change failed: Device is held open by someone [ 456.224096] block drbd3: State change failed: Device is held open by someone [ 456.224098] block drbd3: state = { cs:WFConnection ro:Primary/Unknown ds:UpToDate/DUnknown r----- } [ 456.224099] block drbd3: wanted = { cs:WFConnection ro:Secondary/Unknown ds:UpToDate/DUnknown r----- } [ 456.260386] block drbd1: state = { cs:WFConnection ro:Primary/Unknown ds:UpToDate/DUnknown r----- } [ 456.260388] block drbd1: wanted = { cs:WFConnection ro:Secondary/Unknown ds:UpToDate/DUnknown r----- } [ 456.300789] block drbd0: State change failed: Device is held open by someone [ 456.308664] block drbd0: state = { cs:WFConnection ro:Primary/Unknown ds:UpToDate/DUnknown r----- } [ 456.318965] block drbd0: wanted = { cs:WFConnection ro:Secondary/Unknown ds:UpToDate/DUnknown r----- } [ 457.242948] block drbd3: State change failed: Device is held open by someone [ 457.250834] block drbd3: state = { cs:WFConnection ro:Primary/Unknown ds:UpToDate/DUnknown r----- } [ 457.250835] block drbd3: wanted = { cs:WFConnection ro:Secondary/Unknown ds:UpToDate/DUnknown r----- } [ 457.279143] block drbd1: State change failed: Device is held open by someone [ 457.287019] block drbd1: state = { cs:WFConnection ro:Primary/Unknown ds:UpToDate/DUnknown r----- } [ 457.297319] block drbd1: wanted = { cs:WFConnection ro:Secondary/Unknown ds:UpToDate/DUnknown r----- } [ 457.346954] block drbd0: State change failed: Device is held open by someone [ 457.354829] block drbd0: state = { cs:WFConnection ro:Primary/Unknown ds:UpToDate/DUnknown r----- } [ 457.365129] block drbd0: wanted = { cs:WFConnection ro:Secondary/Unknown ds:UpToDate/DUnknown r----- } [ 458.310997] block drbd3: State change failed: Device is held open by someone [ 458.318872] block drbd3: state = { cs:WFConnection ro:Primary/Unknown ds:UpToDate/DUnknown r----- } [ 458.326017] block drbd1: State change failed: Device is held open by someone [ 458.326019] block drbd1: state = { cs:WFConnection ro:Primary/Unknown ds:UpToDate/DUnknown r----- } [ 458.326020] block drbd1: wanted = { cs:WFConnection ro:Secondary/Unknown ds:UpToDate/DUnknown r----- } [ 458.357835] block drbd3: wanted = { cs:WFConnection ro:Secondary/Unknown ds:UpToDate/DUnknown r----- } [ 458.393913] block drbd0: State change failed: Device is held open by someone [ 458.401786] block drbd0: state = { cs:WFConnection ro:Primary/Unknown ds:UpToDate/DUnknown r----- } [ 458.412085] block drbd0: wanted = { cs:WFConnection ro:Secondary/Unknown ds:UpToDate/DUnknown r----- } [ 459.346078] block drbd1: State change failed: Device is held open by someone [ 459.353953] block drbd1: state = { cs:WFConnection ro:Primary/Unknown ds:UpToDate/DUnknown r----- } [ 459.364252] block drbd1: wanted = { cs:WFConnection ro:Secondary/Unknown ds:UpToDate/DUnknown r----- } [ 459.375826] block drbd3: State change failed: Device is held open by someone [ 459.383716] block drbd3: state = { cs:WFConnection ro:Primary/Unknown ds:UpToDate/DUnknown r----- } [ 459.383717] block drbd3: wanted = { cs:WFConnection ro:Secondary/Unknown ds:UpToDate/DUnknown r----- } [ 459.440444] block drbd0: State change failed: Device is held open by someone [ 459.448319] block drbd0: state = { cs:WFConnection ro:Primary/Unknown ds:UpToDate/DUnknown r----- } [ 459.458619] block drbd0: wanted = { cs:WFConnection ro:Secondary/Unknown ds:UpToDate/DUnknown r----- } [ 460.394649] block drbd1: State change failed: Device is held open by someone [ 460.402529] block drbd1: state = { cs:WFConnection ro:Primary/Unknown ds:UpToDate/DUnknown r----- } [ 460.402530] block drbd1: wanted = { cs:WFConnection ro:Secondary/Unknown ds:UpToDate/DUnknown r----- } [ 460.498012] block drbd0: State change failed: Device is held open by someone [ 460.505890] block drbd0: state = { cs:WFConnection ro:Primary/Unknown ds:UpToDate/DUnknown r----- } [ 460.505891] block drbd0: wanted = { cs:WFConnection ro:Secondary/Unknown ds:UpToDate/DUnknown r----- } [ 461.458950] block drbd1: State change failed: Device is held open by someone [ 461.466830] block drbd1: state = { cs:WFConnection ro:Primary/Unknown ds:UpToDate/DUnknown r----- } [ 461.466831] block drbd1: wanted = { cs:WFConnection ro:Secondary/Unknown ds:UpToDate/DUnknown r----- } [ 461.529731] block drbd0: State change failed: Device is held open by someone [ 461.537611] block drbd0: state = { cs:WFConnection ro:Primary/Unknown ds:UpToDate/DUnknown r----- } [ 461.537612] block drbd0: wanted = { cs:WFConnection ro:Secondary/Unknown ds:UpToDate/DUnknown r----- }
"Arevalo, Mario Alfredo C" <mario.alfredo.c.arevalo@intel.com> hat am 13. Februar 2019 um 17:55 geschrieben:
Hi Marcel
Can you please share us the output of the drbd error?
Thanks.
Best regards. Mario. ________________________________________ From: Marcel Schaible [marcel@schaible-consulting.de] Sent: Wednesday, February 13, 2019 8:28 AM To: starlingx-discuss@lists.starlingx.io Subject: [Starlingx-discuss] Duplex Configuration: Unlock of controller-0 causes reboot
Hi,
when I unlock the controller-0 after installation following this tutorial
https://docs.starlingx.io/installation_guide/duplex.html#duplex
the system is rebooting automatically with a lot of errors from drdb.
Controller-1 is not configured at this point in time.
Must I configure the secondary controller before unlocking controller-0?
Thanks
Marcel
_______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
participants (3)
-
Arevalo, Mario Alfredo C
-
Marcel Schaible
-
Penney, Don