[Starlingx-discuss] Duplex Configuration: Unlock of controller-0 causes

Kung, John John.Kung at windriver.com
Wed Feb 13 18:27:48 UTC 2019 

Marcel,

controller-0 should be unlocked successfully for the first time before attempting to provision/configure controller-1

As noted, the presence  of 'Error' logs on controller-0 (under directory /var/log/puppet ) would indicate the reason for the configuration failure (200.011).

In regards to the OAM interface configured, please check 'system host-if-list -a controller-0' and 'system host-ethernet-port-list controller-0' to determine the current interface configuration.  Furthermore, after the 'host-unlock controller-0', the configured OAM interface IP should be reachable from your OAM network 'system host-addr-list controller-0' to see the addresses allocated for that host.


John
----------------------------------------------------------------------

Message: 1
Date: Wed, 13 Feb 2019 17:02:18 +0000
From: "Penney, Don" <Don.Penney at windriver.com>
To: Marcel Schaible <marcel at schaible-consulting.de>,
	"starlingx-discuss at lists.starlingx.io"
	<starlingx-discuss at lists.starlingx.io>
Subject: Re: [Starlingx-discuss] Duplex Configuration: Unlock of
	controller-0 causes reboot
Message-ID:
	<6703202FD9FDFF4A8DA9ACF104AE129FBA43D23F at ALA-MBD.corp.ad.wrs.com>
Content-Type: text/plain; charset="utf-8"

Ok, my updates related to the DRBD state change failures merged Feb 8th.

I wouldn't expect to see an additional reboot. So what you're seeing, just to confirm, is:
* install node, run config_controller, do config steps
* system host-unlock controller-0
* system reboots
* system reboots a second time?

At what point is it rebooting a second time? Maybe a reboot is being triggered due to config changes made when the puppet manifests apply, which I think we could confirm from the puppet logs.

-----Original Message-----
From: Marcel Schaible [mailto:marcel at schaible-consulting.de] 
Sent: Wednesday, February 13, 2019 11:48 AM
To: Penney, Don; starlingx-discuss at lists.starlingx.io
Subject: Re: [Starlingx-discuss] Duplex Configuration: Unlock of controller-0 causes reboot

I am using the build centos-20190206T060000Z.

The main problem, beside of the noise is, that the system comes up and reboots immediately again.



> "Penney, Don" <Don.Penney at windriver.com> hat am 13. Februar 2019 um 17:41 geschrieben:
> 
> 
> Yes, this is the correct behavior.
> 
> As for the drbd warnings, there are some state change failures that occur as the system shuts down, but aren't a problem (aside from the noise). I've managed to clean up most of these in a recent update. What build are you using?
> 
> -----Original Message-----
> From: Marcel Schaible [mailto:marcel at schaible-consulting.de] 
> Sent: Wednesday, February 13, 2019 11:28 AM
> To: starlingx-discuss at lists.starlingx.io
> Subject: [Starlingx-discuss] Duplex Configuration: Unlock of controller-0 causes reboot
> 
> Hi,
> 
> when I unlock the controller-0 after installation following this tutorial
> 
> https://docs.starlingx.io/installation_guide/duplex.html#duplex
> 
> the system is rebooting automatically with a lot of errors from drdb.
> 
> Controller-1 is not configured at this point in time.
> 
> Must I configure the secondary controller before unlocking controller-0?
> 
> Thanks
> 
> Marcel
> 
> _______________________________________________
> Starlingx-discuss mailing list
> Starlingx-discuss at lists.starlingx.io
> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss

_______________________________________________
Starlingx-discuss mailing list
Starlingx-discuss at lists.starlingx.io
http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss

------------------------------

Message: 2
Date: Wed, 13 Feb 2019 18:17:39 +0100 (CET)
From: Marcel Schaible <marcel at schaible-consulting.de>
To: "Penney, Don" <Don.Penney at windriver.com>,
	starlingx-discuss at lists.starlingx.io
Subject: Re: [Starlingx-discuss] Duplex Configuration: Unlock of
	controller-0 causes reboot
Message-ID: <23037907.438773.1550078259297 at communicator.strato.com>
Content-Type: text/plain; charset=UTF-8

At the moment the system seems to stablize after 2 reboots. I'll keep an eye on that.

Is the alarm list looking reasonable after the controller-0 is unlocked and rebooted?
If yes, I would start to configure the controller-1.


root at controller-0 ~(keystone_admin)]# fm alarm-list
+----------+-------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------+----------+----------------------------+
| Alarm ID | Reason Text                                                                                                       | Entity ID                                                           | Severity | Time Stamp                 |
+----------+-------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------+----------+----------------------------+
| 200.011  | controller-0 experienced a configuration failure.                                                                 | host=controller-0                                                   | critical | 2019-02-13T18:57:13.772108 |
| 400.002  | Service group controller-services loss of redundancy; expected 1 standby member but no standby members available  | service_domain=controller.service_group=controller-services         | major    | 2019-02-13T18:56:15.215083 |
| 400.002  | Service group vim-services loss of redundancy; expected 1 standby member but no standby members available         | service_domain=controller.service_group=vim-services                | major    | 2019-02-13T18:56:15.174083 |
| 400.002  | Service group cloud-services loss of redundancy; expected 1 standby member but no standby members available       | service_domain=controller.service_group=cloud-services              | major    | 2019-02-13T18:56:12.213065 |
| 400.002  | Service group oam-services loss of redundancy; expected 1 standby member but no standby members available         | service_domain=controller.service_group=oam-services                | major    | 2019-02-13T18:56:05.330099 |
| 400.002  | Service group patching-services loss of redundancy; expected 1 standby member but no standby members available    | service_domain=controller.service_group=patching-services           | major    | 2019-02-13T18:56:04.276424 |
| 400.002  | Service group directory-services loss of redundancy; expected 2 active members but only 1 active member available | service_domain=controller.service_group=directory-services          | major    | 2019-02-13T18:56:04.195081 |
| 400.002  | Service group web-services loss of redundancy; expected 2 active members but only 1 active member available       | service_domain=controller.service_group=web-services                | major    | 2019-02-13T18:56:04.114064 |
| 400.005  | Communication failure detected with peer over port enp9s20f7 on host controller-0                                 | host=controller-0.network=mgmt                                      | major    | 2019-02-13T18:56:03.790070 |
| 100.106  | 'OAM' Port failed.                                                                                                | host=controller-0.port=cd202069-f5a0-43bb-a445-d3b9b58ce631         | major    | 2019-02-13T18:56:00.563067 |
| 100.107  | 'OAM' Interface degraded.                                                                                         | host=controller-0.interface=oam                                     | major    | 2019-02-13T18:56:00.515836 |
| 300.004  | No enabled compute host with connectivity to provider network.                                                    | service=networking.providernet=e9eef26b-d266-4b3f-b1b7-6d9c2cd3a86b | major    | 2019-02-13T17:20:21.110405 |
+----------+-------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------+----------+----------------------------+
[root at controller-0 ~(keystone_admin)]#



> "Penney, Don" <Don.Penney at windriver.com> hat am 13. Februar 2019 um 18:02 geschrieben:
> 
> 
> Ok, my updates related to the DRBD state change failures merged Feb 8th.
> 
> I wouldn't expect to see an additional reboot. So what you're seeing, just to confirm, is:
> * install node, run config_controller, do config steps
> * system host-unlock controller-0
> * system reboots
> * system reboots a second time?

Yes, exactly.

> 
> At what point is it rebooting a second time? Maybe a reboot is being triggered due to config changes made when the puppet manifests apply, which I think we could confirm from the puppet logs.

The second reboot starts in about 1 minute after the login prompt is shown.


> 
> -----Original Message-----
> From: Marcel Schaible [mailto:marcel at schaible-consulting.de] 
> Sent: Wednesday, February 13, 2019 11:48 AM
> To: Penney, Don; starlingx-discuss at lists.starlingx.io
> Subject: Re: [Starlingx-discuss] Duplex Configuration: Unlock of controller-0 causes reboot
> 
> I am using the build centos-20190206T060000Z.
> 
> The main problem, beside of the noise is, that the system comes up and reboots immediately again.
> 
> 
> 
> > "Penney, Don" <Don.Penney at windriver.com> hat am 13. Februar 2019 um 17:41 geschrieben:
> > 
> > 
> > Yes, this is the correct behavior.
> > 
> > As for the drbd warnings, there are some state change failures that occur as the system shuts down, but aren't a problem (aside from the noise). I've managed to clean up most of these in a recent update. What build are you using?
> > 
> > -----Original Message-----
> > From: Marcel Schaible [mailto:marcel at schaible-consulting.de] 
> > Sent: Wednesday, February 13, 2019 11:28 AM
> > To: starlingx-discuss at lists.starlingx.io
> > Subject: [Starlingx-discuss] Duplex Configuration: Unlock of controller-0 causes reboot
> > 
> > Hi,
> > 
> > when I unlock the controller-0 after installation following this tutorial
> > 
> > https://docs.starlingx.io/installation_guide/duplex.html#duplex
> > 
> > the system is rebooting automatically with a lot of errors from drdb.
> > 
> > Controller-1 is not configured at this point in time.
> > 
> > Must I configure the secondary controller before unlocking controller-0?
> > 
> > Thanks
> > 
> > Marcel
> > 
> > _______________________________________________
> > Starlingx-discuss mailing list
> > Starlingx-discuss at lists.starlingx.io
> > http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
> 
> _______________________________________________
> Starlingx-discuss mailing list
> Starlingx-discuss at lists.starlingx.io
> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss

More information about the Starlingx-discuss mailing list