[Starlingx-discuss] 2nd node stuck at pxe installation

Marcel Schaible marcel at schaible-consulting.de
Thu May 2 13:40:07 UTC 2019


We solved our problem by setting the management network start and end address during Installation of controller-0. The default ip address 172.27.0.2 collides on our server with a dhcp server running on non-starlingx managament cpu.

> "Penney, Don" <Don.Penney at windriver.com> hat am 2. Mai 2019 um 15:31 geschrieben:
> 
> 
> Running a "collect" and providing the tarball might help us to understand what's happening. Can you provide a "system host-if-list" for your controllers, at a minimum? And maybe the content of /opt/platform/config/19.01/cgcs_config
> 
> -----Original Message-----
> From: Sun, Austin [mailto:austin.sun at intel.com] 
> Sent: Wednesday, May 01, 2019 10:08 PM
> To: Penney, Don; Marcel Schaible; starlingx-discuss at lists.starlingx.io
> Subject: RE: [Starlingx-discuss] 2nd node stuck at pxe installation
> 
> Hi Penney:
>      Yes.   pxecontrooler IP is active on controller-0 management interface .  The environment is virtual environment .
> 
> The issue is that in controller-1 , the ip has been assigned to mgmt_ip if it is already failed.   From my understanding , the ip in controller-1 when install.
> The ip should be in range pxe subnet. 
> 
> Thanks.
> BR
> Austin Sun. 
> 
> -----Original Message-----
> From: Penney, Don [mailto:Don.Penney at windriver.com] 
> Sent: Wednesday, May 1, 2019 10:11 PM
> To: Sun, Austin <austin.sun at intel.com>; Marcel Schaible <marcel at schaible-consulting.de>; starlingx-discuss at lists.starlingx.io
> Subject: RE: [Starlingx-discuss] 2nd node stuck at pxe installation
> 
> Is the pxecontroller IP active on your management interface? For example (if your management isn't on a vlan):
> 
> controller-0:/home/wrsroot# grep management_interface /etc/platform/platform.conf
> management_interface=eno2
> controller-0:/home/wrsroot# grep pxecontroller /etc/hosts
> 169.254.202.2   pxecontroller
> controller-0:/home/wrsroot# ip addr show eno2 | grep 169.254.202.2
>     inet 169.254.202.2/24 scope global eno2
> 
> 
> If not, is the pxeboot-ip service in SM active?
> controller-0:/home/wrsroot# sm-dump |grep pxe
> pxeboot-ip                       enabled-active       enabled-active
> 
> 
> Cheers,
> Don.
> 
> 
> -----Original Message-----
> From: Penney, Don
> Sent: Wednesday, May 01, 2019 9:54 AM
> To: 'Sun, Austin'; Marcel Schaible; starlingx-discuss at lists.starlingx.io
> Subject: RE: [Starlingx-discuss] 2nd node stuck at pxe installation
> 
> The pxecontroller address is purposefully on a separate subnet. Our networking team could provide a better answer, but I believe it was due to issues with pxeboot and VLANs. This address should absolutely be available from the installer.
> 
> 
> -----Original Message-----
> From: Sun, Austin [mailto:austin.sun at intel.com]
> Sent: Tuesday, April 30, 2019 10:59 PM
> To: Penney, Don; Marcel Schaible; starlingx-discuss at lists.starlingx.io
> Subject: RE: [Starlingx-discuss] 2nd node stuck at pxe installation
> 
> Hi Penny :
>     Thanks a lot.  some update from my side. 
> From log Dracut [1];  squashfs.img was not successfully download on controller-1. 
> And the ip on controller-1 during pxe boot was 192.168.204.4 , if use curl command "curl http://192.168.204.2:8080/feed/rel-19.01//LiveOS/squashfs.img -o mytest.img"  can successfully download squashfs.img . 
> But if running command "curl http://pxecontroller:8080/feed/rel-19.01//LiveOS/squashfs.img" , it will failed. 
> 
> From controller "nslookup pxecontroller" , the pxecontroller ip addr is 169.254.202.2 , not mgmt ip(192.168.204.2).  so it will make sense pxecontroller can not be accessed by controller-1(pxe boot stage).
> 
> If  hack sysinv-conductor manager.py change pxecontroller to mgmt. ip (192.168.204.2) and restart sysinv-conductor, then the controller-1 can be installed successfully. 
> 
> BTW: This issue is 100% reproducible on certain servers,  but not for all kinds of servers. 
> 
> [1] http://paste.openstack.org/show/750134/
> [2] http://paste.openstack.org/show/750135/
> 
> 
> Thanks.
> BR
> Austin Sun. 
> 
> -----Original Message-----
> From: Penney, Don [mailto:Don.Penney at windriver.com]
> Sent: Tuesday, April 30, 2019 11:04 PM
> To: Marcel Schaible <marcel at schaible-consulting.de>; starlingx-discuss at lists.starlingx.io
> Subject: Re: [Starlingx-discuss] 2nd node stuck at pxe installation
> 
> You can also look in /var/log/daemon.log for TFTP logs showing the requests and transfers of files, and /www/var/log/lighttpd-access.log for transfer of the squashfs.img.
> 
> Since the console showed "Loading rel-19.01/installer-initrd...........ready", it seems like the TFTP transfer was fine, which is why I suggested modifying the console setting to ensure you're seeing what's happening. This is what I'd expect to see if the console is set to serial, but there's no serial console attached (such as using virtual box without a serial console).
> 
> For example, you can see the bzImage and initrd here being transferred via TFTP in daemon.log:
> 2019-04-26T16:27:34.000 controller-0 dnsmasq-tftp[96915]: info sent /pxeboot/rel-19.01/installer-bzImage to 192.168.202.24
> 2019-04-26T16:27:36.000 controller-0 dnsmasq-tftp[96915]: info sent /pxeboot/rel-19.01/installer-initrd to 192.168.202.24
> 
> followed by the squashfs.img in /www/var/log/lighttpd-access.log:
> 192.168.202.24 pxecontroller:8080 - [26/Apr/2019:16:28:12 +0000] "GET /feed/rel-19.01//LiveOS/squashfs.img HTTP/1.1" 200 368152576 "-" "curl/7.29.0"
> 
> If only the bzImage and initrd is transferred, without seeing the logs on console, my guess would be an unsupported NIC preventing the initrd from accessing the management network in order to request the squashfs.img (which provides the installer rootfs)
> 
> 
> -----Original Message-----
> From: Marcel Schaible [mailto:marcel at schaible-consulting.de]
> Sent: Tuesday, April 30, 2019 10:46 AM
> To: starlingx-discuss at lists.starlingx.io
> Subject: Re: [Starlingx-discuss] 2nd node stuck at pxe installation
> 
> Hi,
> 
> we have the same problem with the image from 20190411. It Looks like that controller-1 is not finding the pxe boot image.
> 
> Kind regards
> 
> Marcel
> 
> > Message: 1
> > Date: Tue, 30 Apr 2019 13:54:17 +0000
> > From: "Penney, Don" <Don.Penney at windriver.com>
> > To: "Lin, Shuicheng" <shuicheng.lin at intel.com>,
> > 	"starlingx-discuss at lists.starlingx.io"
> > 	<starlingx-discuss at lists.starlingx.io>
> > Subject: Re: [Starlingx-discuss] 2nd node stuck at pxe installation
> > Message-ID:
> > 	<6703202FD9FDFF4A8DA9ACF104AE129FBA48BB86 at ALA-MBD.corp.ad.wrs.com>
> > Content-Type: text/plain; charset="utf-8"
> > 
> > You can set the console either via the web browser in the installation parameters, or at command-line, such as:
> > system host-update 2 personality=controller console=
> > 
> > The default is the serial console, "ttyS0,115200". If you're using a graphical console, you can set it blank, or console=tty0, for example. This should allow you to see if there are some errors occurring trying to load the installer.
> > 
> > 
> > From: Lin, Shuicheng [mailto:shuicheng.lin at intel.com]
> > Sent: Tuesday, April 30, 2019 1:03 AM
> > To: starlingx-discuss at lists.starlingx.io
> > Subject: [Starlingx-discuss] 2nd node stuck at pxe installation
> > 
> > Hi all,
> > I try to do duplex deploy, and find controller-1 stuck at pxe installation step.
> > The screen stuck at "Loading rel-19.01/installer-initrd...........ready". But package installation is not started as host-show command shown in controller-0.
> > I meet this issue with latest code. Do you know what cause the issue, or how to debug it?
> > Thanks.
> > 
> > [wrsroot at controller-0 ~(keystone_admin)]$ system host-list
> > +----+--------------+-------------+----------------+-------------+--------------+
> > | id | hostname     | personality | administrative | operational | availability |
> > +----+--------------+-------------+----------------+-------------+--------------+
> > | 1  | controller-0 | controller  | unlocked       | enabled     | available    |
> > | 2  | controller-1 | controller  | locked         | disabled    | offline      |
> > +----+--------------+-------------+----------------+-------------+--------------+
> > [wrsroot at controller-0 ~(keystone_admin)]$ system host-show
> > controller-1 | grep install
> > | install_output      | text                                    |
> > | install_state       | None                                    |
> > | install_state_info  | None                                    |
> > | subfunction_avail   | not-installed
> > 
> > Best Regards
> > Shuicheng
> >
> 
> _______________________________________________
> Starlingx-discuss mailing list
> Starlingx-discuss at lists.starlingx.io
> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
> _______________________________________________
> Starlingx-discuss mailing list
> Starlingx-discuss at lists.starlingx.io
> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss



More information about the Starlingx-discuss mailing list