1) can i boot multiple aio duplex from a pxe boot server - You can ONLY pxeboot the first controller of a system (controller-0) from an external pxe boot server - after controller-0 is up, controller-0 is used to pxeboot all the remaining starlingx hosts (e.g. controller-1, worker-0, worker-1, etc.) 2) once controller-0 is up, how does controller-1 connect to it. - controller-0 pxeboots controller-1 over mgmt. network (by default ... you can use a separate pxeboot network if your mgmt. network needs to be configured in a fashion that does not support pxebooting, e.g. ipv6 or vlan tagged) - controller-0 then communicates with controller-1 over mgmt. network and cluster-host network for starlingx infrastructure reasons and kubernetes reasons respectively 3) which then i have to inquire how does discovery work for controller-1 on controller-0 - controller-1 bios is configured to network boot on mgmt. network - controller-0 dnsmasq pxeboots controller-1 and kicks starlingx systeminventory to auto-discover node 4) in your opinions, which are welcome, how should i be deploying 6 nodes with this much memery and storage per node, to use 3 as controllers, and all 6 as compute/storage nodes? - starlingx currently only supports only up to 2 controllers - in order to us 6x nodes as both compute/storage ... for a 6 node cluster, you would have to: * 2x AIO controllers (controller and worker/compute function) * 4x workers (worker/compute function) * the only way to use storage on all nodes is to use ROOK (and not the host-based ceph) for your storage backend Greg. -----Original Message----- From: Embedded Devel <lists@optimcloud.com> Sent: Friday, May 13, 2022 10:15 PM To: starlingx-discuss@lists.starlingx.io Subject: [Starlingx-discuss] pxe boot network [Please note: This e-mail is from an EXTERNAL e-mail address] Im trying to deploy 6 nodes with 512G ram and 24TB disk each Im envisioning 2-3 AIO DUPLEX Controller and 4 WORKER/STORAGE SO i deployed a single AIO duplex node, works fine, then had issues with pxe booting other nodes from it, due to switch configuration, without having to reinvent the switch topology to accomodate the pxe 169.254.202.1 pxe network, im reading below where it states PXE Boot Network VERSION You can set up a PXE boot network for booting all nodes to allow a non-standard management network configuration. The internal management network is used for PXE booting of new hosts and the PXE boot network is not required. However there are scenarios where the internal management network cannot be used for PXE booting of new hosts. For example, if the internal management network needs to be on a VLAN-tagged network for deployment reasons, or if it must support IPv6, you must configure the optional untagged PXE boot network for PXE booting of new hosts using IPv4. According to: https://docs.starlingx.io/deploy_install_guides/r6_release/bare_metal/config... Okay so it seems i need to go this route, and pxe boot all nodes, however.. reading the configure a pxe boot server doc, it states "You can optionally set up a PXE Boot Server to support controller-0 initialization." so this tells me that it can only be used for pxe booting controller-0 ???? https://docs.starlingx.io/deploy_install_guides/r6_release/bare_metal/config... so questions: 1) can i boot multiple aio duplex from a pxe boot server 2) once controller-0 is up, how does controller-1 connect to it. 3) which then i have to inquire how does discovery work for controller-1 on controller-0 4) in your opinions, which are welcome, how should i be deploying 6 nodes with this much memery and storage per node, to use 3 as controllers, and all 6 as compute/storage nodes? -- Sent with Vivaldi Mail. Download Vivaldi for free at vivaldi.com _______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss