[Starlingx-discuss] multi-node AIO / PXE booting
not sure why my emails not going through so again.... I have 6 nodes, 512Gb Memory and 24TB disk 8x3TB per node can i deploy a duplex aio on 2-3 nodes? and worker/storage on 4 nodes second to that, since we cannot confirm our network environment to meet the pxe boot on 169.254 for other controller-1, worker nodes it states here https://docs.starlingx.io/planning/kubernetes/network-planning-the-pxe-boot-... that we can create a pxe boot environment to boot all nodes... okay PXE Boot Network VERSION You can set up a PXE boot network for booting all nodes to allow a non-standard management network configuration. however, this doc states https://docs.starlingx.io/deploy_install_guides/r6_release/bare_metal/config... Configure a PXE Boot Server VERSION You can optionally set up a PXE Boot Server to support controller-0 initialization. so questions: 1) can i boot all nodes from pxe server? 2) after booting installing the controller-0 node i can then pxe boot install controller-1 node 3) in this scenerio how does controller-0 discover controller-1 so i can set the host "system host-update 2 personality=controller" 4) does the same apply to workers 5) if you have 6 fully configured server each with 24TB and 512Gb memory, how would you deploy to get a full 6 nodes with storage and compute?
On Mon, May 9, 2022 at 4:01 PM Outback Dingo <outbackdingo@gmail.com> wrote:
can i deploy a duplex aio on 2-3 nodes? and worker/storage on 4 nodes
Duplex AIO = Controller (and storage and workers) on 2 nodes. There is no 3-controller option in StarlingX. So typically you would get 2 controllers and 6 workers. Pl. see: https://docs.starlingx.io/deploy/index-deploy-da06a98b83b1.html
so questions: 1) can i boot all nodes from pxe server?
In the standard/typical deployment, only controller-0 is PXE booted from an "external" PXE server. Other hosts PXE boot off the controller-0 on the cluster-internal mgmt/cluster network. The second link you refer to, is about PXE booting the controller-0. But, the first link you refer to, allows PXE booting ALL nodes from an external network. I haven't tried this but it seems supported. For e.g. https://docs.starlingx.io/developer_resources/stx_ipv6_deployment.html
2) after booting installing the controller-0 node i can then pxe boot install controller-1 node 3) in this scenerio how does controller-0 discover controller-1 so i can set the host "system host-update 2 personality=controller"
In the typical deployment, once controller-0 is up, it runs a DHCP and TFTP server on the internal mgmt/cluster network. The controller-1 and other worker nodes are supposed to be configured to network boot off the NIC connected to this internal network. When they boot, controller-0 will "discover" them and add them to the host inventory. https://docs.starlingx.io/planning/kubernetes/starlingx-boot-sequence-consid... https://docs.starlingx.io/planning/kubernetes/internal-management-network-pl...
4) does the same apply to workers
Yes
5) if you have 6 fully configured server each with 24TB and 512Gb memory, how would you deploy to get a full 6 nodes with storage and compute?
You want more storage than controllers can provide? Pl see if this helps: https://docs.starlingx.io/planning/kubernetes/storage-planning-storage-resou... Thanks
On Tue, May 10, 2022 at 6:48 PM Pratik M. <pvmpublic@gmail.com> wrote:
On Mon, May 9, 2022 at 4:01 PM Outback Dingo <outbackdingo@gmail.com> wrote:
can i deploy a duplex aio on 2-3 nodes? and worker/storage on 4 nodes
Duplex AIO = Controller (and storage and workers) on 2 nodes. There is no 3-controller option in StarlingX. So typically you would get 2 controllers and 6 workers. Pl. see: https://docs.starlingx.io/deploy/index-deploy-da06a98b83b1.html
so never more then 2 controllers.... i can pxe boot and manually add all hosts from a central pxe server, since we cant use stx 169.254, due to bonds and vlans. that leaves me with what do i do for storage can a compute node be a storage node, as in all 6 nodes being compute/storage (ceph), what about controller-0 storage... meaning none except for boot drives. ie... control-1 2x128Gb boot ssd, 1 x 1TB, document states i need 2 500gb + drives control-2 2x128Gb boot ssd, 1 x 1TB, document states i need 2 500gb + drives compute-1 1x1TB boot drive, and 8x3.8TB drives compute-2 1x1TB boot drive, and 8x3.8TB drives compute-3 1x1TB boot drive, and 8x3.8TB drives compute-4 1x1TB boot drive, and 8x3.8TB drives compute-5 1x1TB boot drive, and 8x3.8TB drives compute-6 1x1TB boot drive, and 8x3.8TB drives not sure this scenerio is even viable without controller-0/controller-1 having 0 ceph storage, as it would then all be on the nodes
so questions: 1) can i boot all nodes from pxe server?
In the standard/typical deployment, only controller-0 is PXE booted from an "external" PXE server. Other hosts PXE boot off the controller-0 on the cluster-internal mgmt/cluster network. The second link you refer to, is about PXE booting the controller-0.
But, the first link you refer to, allows PXE booting ALL nodes from an external network. I haven't tried this but it seems supported. For e.g. https://docs.starlingx.io/developer_resources/stx_ipv6_deployment.html
2) after booting installing the controller-0 node i can then pxe boot install controller-1 node 3) in this scenerio how does controller-0 discover controller-1 so i can set the host "system host-update 2 personality=controller"
In the typical deployment, once controller-0 is up, it runs a DHCP and TFTP server on the internal mgmt/cluster network. The controller-1 and other worker nodes are supposed to be configured to network boot off the NIC connected to this internal network. When they boot, controller-0 will "discover" them and add them to the host inventory.
https://docs.starlingx.io/planning/kubernetes/starlingx-boot-sequence-consid... https://docs.starlingx.io/planning/kubernetes/internal-management-network-pl...
4) does the same apply to workers
Yes
5) if you have 6 fully configured server each with 24TB and 512Gb memory, how would you deploy to get a full 6 nodes with storage and compute?
You want more storage than controllers can provide? Pl see if this helps: https://docs.starlingx.io/planning/kubernetes/storage-planning-storage-resou...
Thanks
On Tue, May 10, 2022 at 6:48 PM Pratik M. <pvmpublic@gmail.com> wrote:
On Mon, May 9, 2022 at 4:01 PM Outback Dingo <outbackdingo@gmail.com> wrote:
can i deploy a duplex aio on 2-3 nodes? and worker/storage on 4 nodes
Duplex AIO = Controller (and storage and workers) on 2 nodes. There is no 3-controller option in StarlingX. So typically you would get 2 controllers and 6 workers. Pl. see: https://docs.starlingx.io/deploy/index-deploy-da06a98b83b1.html
according to this document Standard with Storage Cluster on Controller Nodes A two node HA controller + storage node cluster, managing up to 200 worker nodes. Standard with Storage Cluster on dedicated Storage Nodes A two node HA controller node cluster with a 2-9 node Ceph storage cluster, managing up to 200 worker nodes. which basically says i cant even put storage on compute/worker nodes
so questions: 1) can i boot all nodes from pxe server?
In the standard/typical deployment, only controller-0 is PXE booted from an "external" PXE server. Other hosts PXE boot off the controller-0 on the cluster-internal mgmt/cluster network. The second link you refer to, is about PXE booting the controller-0.
But, the first link you refer to, allows PXE booting ALL nodes from an external network. I haven't tried this but it seems supported. For e.g. https://docs.starlingx.io/developer_resources/stx_ipv6_deployment.html
2) after booting installing the controller-0 node i can then pxe boot install controller-1 node 3) in this scenerio how does controller-0 discover controller-1 so i can set the host "system host-update 2 personality=controller"
In the typical deployment, once controller-0 is up, it runs a DHCP and TFTP server on the internal mgmt/cluster network. The controller-1 and other worker nodes are supposed to be configured to network boot off the NIC connected to this internal network. When they boot, controller-0 will "discover" them and add them to the host inventory.
https://docs.starlingx.io/planning/kubernetes/starlingx-boot-sequence-consid... https://docs.starlingx.io/planning/kubernetes/internal-management-network-pl...
4) does the same apply to workers
Yes
5) if you have 6 fully configured server each with 24TB and 512Gb memory, how would you deploy to get a full 6 nodes with storage and compute?
You want more storage than controllers can provide? Pl see if this helps: https://docs.starlingx.io/planning/kubernetes/storage-planning-storage-resou...
Thanks
participants (2)
-
Outback Dingo
-
Pratik M.