Hi team,

I installed StarlingX using following the March 5th ISO image and the issue still existshttp://mirror.starlingx.cengn.ca/mirror/starlingx/master/centos/20190305T060000Z/outputs/iso/

Are there any steps not been written on the guide? I just followed the following guide:

https://wiki.openstack.org/wiki/StarlingX/Containers/InstallationOnStandard

 

I checked the ovs-vswitchd.log and the following line maybe the cause:

23127 hugepages of size 2097152 reserved, but no mounted hugetlbfs found for that size

It seems that something went wrong with hugepage. Some logs have been attached. If you have any idea about hugepage, please let me know.

 

Peng has been added and hope Peng can provide some information about his environment.
https://bugs.launchpad.net/starlingx/+bug/1819031

 

 

Best Regards,

Xu, Chenjie

 

From: Xu, Chenjie
Sent: Tuesday, March 12, 2019 12:39 AM
To: Peters, Matt <Matt.Peters@windriver.com>; Khalil, Ghada <Ghada.Khalil@windriver.com>; Xie, Cindy <cindy.xie@intel.com>; Zhao, Forrest <forrest.zhao@intel.com>; Richard, Joseph <Joseph.Richard@windriver.com>; Winnicki, Chris <Chris.Winnicki@windriver.com>; Lin, Shuicheng <shuicheng.lin@intel.com>
Cc: starlingx-discuss@lists.starlingx.io; Martinez Monroy, Elio <elio.martinez.monroy@intel.com>; Cabrales, Ada <ada.cabrales@intel.com>; Jones, Bruce E <bruce.e.jones@intel.com>; Qin, Kailun <kailun.qin@intel.com>; Guo, Ruijing <ruijing.guo@intel.com>; Le, Huifeng <huifeng.le@intel.com>
Subject: RE: [Starlingx-discuss] OVS-DPDK Upgrade Testing

 

Hi team,

For Matt, I just tried to lock compute-1 to ensure the VMs is on the same compute host. But it still doesn’t work. The DHCP, router and VMs all located on compute-0.

 

For Ghada, I checked the test cases listed in sanity test. I see that there is no test case to test the connectivity between VMs. Need confirmation from Ada’s team.
https://wiki.openstack.org/wiki/StarlingX/Test/SanityTests

 

Best Regards,

Xu, Chenjie

 

From: Peters, Matt [mailto:Matt.Peters@windriver.com]
Sent: Tuesday, March 12, 2019 12:28 AM
To: Khalil, Ghada <Ghada.Khalil@windriver.com>; Xu, Chenjie <chenjie.xu@intel.com>; Xie, Cindy <cindy.xie@intel.com>; Zhao, Forrest <forrest.zhao@intel.com>; Richard, Joseph <Joseph.Richard@windriver.com>; Winnicki, Chris <Chris.Winnicki@windriver.com>; Lin, Shuicheng <shuicheng.lin@intel.com>
Cc: starlingx-discuss@lists.starlingx.io; Martinez Monroy, Elio <elio.martinez.monroy@intel.com>; Cabrales, Ada <ada.cabrales@intel.com>; Jones, Bruce E <bruce.e.jones@intel.com>; Qin, Kailun <kailun.qin@intel.com>; Guo, Ruijing <ruijing.guo@intel.com>; Le, Huifeng <huifeng.le@intel.com>
Subject: Re: [Starlingx-discuss] OVS-DPDK Upgrade Testing

 

If you co-locate the VMs on the same compute host (lock the other computes) do things work as expected?  I.e. is it only when you are traversing the physical network that things are not pingable?

 

-Matt

 

From: Ghada Khalil <Ghada.Khalil@windriver.com>
Date: Monday, March 11, 2019 at 10:51 AM
To: "Xu, Chenjie" <chenjie.xu@intel.com>, "Xie, Cindy" <cindy.xie@intel.com>, "Zhao, Forrest" <forrest.zhao@intel.com>, "Peters, Matt" <Matt.Peters@windriver.com>, "Richard, Joseph" <Joseph.Richard@windriver.com>, "Winnicki, Chris" <Chris.Winnicki@windriver.com>, "Lin, Shuicheng" <shuicheng.lin@intel.com>
Cc: "starlingx-discuss@lists.starlingx.io" <starlingx-discuss@lists.starlingx.io>, "Martinez Monroy, Elio" <elio.martinez.monroy@intel.com>, "Cabrales, Ada" <ada.cabrales@intel.com>, "Jones, Bruce E" <bruce.e.jones@intel.com>, "Qin, Kailun" <kailun.qin@intel.com>, "Guo, Ruijing" <ruijing.guo@intel.com>, "Le, Huifeng" <huifeng.le@intel.com>
Subject: RE: [Starlingx-discuss] OVS-DPDK Upgrade Testing

 

To answer Forrest’s question:

               Do you know if OVS/DPDK has been tested after cut-over to container?

I have seen Green sanity reports from Ada’s team since the cut-over. I expect this covers basic ovs-dpdk functional testing. Ada, please correct me if I am wrong.

 

Regarding https://bugs.launchpad.net/starlingx/+bug/1819031, we’ve requested information from the reporter on how reproducible this issue is. As far as I know, it was reported on one system only. We’re waiting for more information from Peng.

 

Regards,

Ghada

 

 

From: Xu, Chenjie [mailto:chenjie.xu@intel.com]
Sent: Monday, March 11, 2019 10:40 AM
To: Xie, Cindy; Zhao, Forrest; Peters, Matt; Richard, Joseph; Khalil, Ghada; Winnicki, Chris; Lin, Shuicheng
Cc: starlingx-discuss@lists.starlingx.io; Martinez Monroy, Elio; Cabrales, Ada; Jones, Bruce E; Qin, Kailun; Guo, Ruijing; Le, Huifeng
Subject: RE: [Starlingx-discuss] OVS-DPDK Upgrade Testing

 

Hi Cindy,

Thank you for your information! The following issue is not exactly the same as mine:

https://bugs.launchpad.net/starlingx/+bug/1819031

 

In this issue, the DHCP works and external network can reach the VMs. However in my case, VMs can’t get IP from DHCP and external network can’t reach VMs even with an IP allocated by myself.

 

Best Regards,

Xu, Chenjie

 

From: Xie, Cindy
Sent: Monday, March 11, 2019 10:05 PM
To: Zhao, Forrest <forrest.zhao@intel.com>; Xu, Chenjie <chenjie.xu@intel.com>; Peters, Matt <Matt.Peters@windriver.com>; Richard, Joseph <Joseph.Richard@windriver.com>; Khalil, Ghada <Ghada.Khalil@windriver.com>; Chris.Winnicki@windriver.com; Lin, Shuicheng <shuicheng.lin@intel.com>
Cc: starlingx-discuss@lists.starlingx.io; Martinez Monroy, Elio <elio.martinez.monroy@intel.com>; Cabrales, Ada <ada.cabrales@intel.com>; Jones, Bruce E <bruce.e.jones@intel.com>; Qin, Kailun <kailun.qin@intel.com>; Guo, Ruijing <ruijing.guo@intel.com>; Le, Huifeng <huifeng.le@intel.com>
Subject: RE: [Starlingx-discuss] OVS-DPDK Upgrade Testing

 

Is the below issue similar?

 

https://bugs.launchpad.net/starlingx/+bug/1819031

 

 

From: Zhao, Forrest
Sent: Monday, March 11, 2019 9:54 PM
To: Xu, Chenjie <chenjie.xu@intel.com>; Peters, Matt <Matt.Peters@windriver.com>; Richard, Joseph <Joseph.Richard@windriver.com>; Khalil, Ghada <Ghada.Khalil@windriver.com>; Chris.Winnicki@windriver.com; Lin, Shuicheng <shuicheng.lin@intel.com>
Cc: starlingx-discuss@lists.starlingx.io; Martinez Monroy, Elio <elio.martinez.monroy@intel.com>; Xie, Cindy <cindy.xie@intel.com>; Cabrales, Ada <ada.cabrales@intel.com>; Jones, Bruce E <bruce.e.jones@intel.com>; Qin, Kailun <kailun.qin@intel.com>; Guo, Ruijing <ruijing.guo@intel.com>; Le, Huifeng <huifeng.le@intel.com>
Subject: RE: [Starlingx-discuss] OVS-DPDK Upgrade Testing

 

Ghada, Matt, Joseph and validation team,

 

Do you know if OVS/DPDK has been tested after cut-over to container? I think the below issue reported by Chenjie might already exist after cut-over to container.

 

Thanks,

Forrest

 

From: Xu, Chenjie
Sent: Monday, March 11, 2019 9:17 PM
To: Peters, Matt <Matt.Peters@windriver.com>; Richard, Joseph <Joseph.Richard@windriver.com>; Khalil, Ghada <Ghada.Khalil@windriver.com>; Chris.Winnicki@windriver.com; Lin, Shuicheng <shuicheng.lin@intel.com>
Cc: starlingx-discuss@lists.starlingx.io; Martinez Monroy, Elio <elio.martinez.monroy@intel.com>; Zhao, Forrest <forrest.zhao@intel.com>; Xie, Cindy <cindy.xie@intel.com>; Cabrales, Ada <ada.cabrales@intel.com>; Jones, Bruce E <bruce.e.jones@intel.com>; Qin, Kailun <kailun.qin@intel.com>; Guo, Ruijing <ruijing.guo@intel.com>; Le, Huifeng <huifeng.le@intel.com>
Subject: RE: [Starlingx-discuss] OVS-DPDK Upgrade Testing

 

Hi team,

I have installed StarlingX multimode with new ISO image on 4 bare metals (2 controllers and 2 computes). However the VMs can’t get an IP address. I allocate the IP address which is same as neutron allocated to the VM, but the VMs still can’t ping each other. The DHCP exists and VMs can’t ping DHCP or router. Some logs have been attached.

 

This issue blocks basic functional testing for OVS-DPDK Upgrade and please let me know if you have any ideas.

 

Best Regards,

Xu, Chenjie

 

From: Lin, Shuicheng
Sent: Friday, March 8, 2019 9:49 AM
To: Xu, Chenjie <chenjie.xu@intel.com>; Khalil, Ghada <Ghada.Khalil@windriver.com>; Richard, Joseph <Joseph.Richard@windriver.com>; Peters, Matt <Matt.Peters@windriver.com>; Chris.Winnicki@windriver.com
Cc: starlingx-discuss@lists.starlingx.io; Martinez Monroy, Elio <elio.martinez.monroy@intel.com>; Zhao, Forrest <forrest.zhao@intel.com>; Xie, Cindy <cindy.xie@intel.com>; Cabrales, Ada <ada.cabrales@intel.com>; Jones, Bruce E <bruce.e.jones@intel.com>; Qin, Kailun <kailun.qin@intel.com>; Guo, Ruijing <ruijing.guo@intel.com>; Le, Huifeng <huifeng.le@intel.com>
Subject: RE: [Starlingx-discuss] OVS-DPDK Upgrade Testing

 

Hi all,

I just shared the new test ISO with Chenjie/Elio in another mail thread.

It is built from centos76 feature branch with latest master code rebased.

 

Best Regards

Shuicheng

 

From: Xu, Chenjie
Sent: Wednesday, March 6, 2019 11:07 AM
To: Khalil, Ghada <Ghada.Khalil@windriver.com>; Richard, Joseph <Joseph.Richard@windriver.com>; Peters, Matt <Matt.Peters@windriver.com>; Chris.Winnicki@windriver.com
Cc: starlingx-discuss@lists.starlingx.io; Lin, Shuicheng <shuicheng.lin@intel.com>; Martinez Monroy, Elio <elio.martinez.monroy@intel.com>; Zhao, Forrest <forrest.zhao@intel.com>; Xie, Cindy <cindy.xie@intel.com>; Cabrales, Ada <ada.cabrales@intel.com>; Jones, Bruce E <bruce.e.jones@intel.com>; Qin, Kailun <kailun.qin@intel.com>; Guo, Ruijing <ruijing.guo@intel.com>; Le, Huifeng <huifeng.le@intel.com>
Subject: RE: [Starlingx-discuss] OVS-DPDK Upgrade Testing

 

Hi Ghada,

That will be very helpful and many thanks to you and Joseph! Looking forward to Chris’s guide.

 

Best Regards,

Xu, Chenjie

 

From: Khalil, Ghada [mailto:Ghada.Khalil@windriver.com]
Sent: Wednesday, March 6, 2019 12:07 AM
To: Xu, Chenjie <chenjie.xu@intel.com>; Richard, Joseph <Joseph.Richard@windriver.com>; Peters, Matt <Matt.Peters@windriver.com>; Winnicki, Chris <Chris.Winnicki@windriver.com>
Cc: starlingx-discuss@lists.starlingx.io; Lin, Shuicheng <shuicheng.lin@intel.com>; Martinez Monroy, Elio <elio.martinez.monroy@intel.com>; Zhao, Forrest <forrest.zhao@intel.com>; Xie, Cindy <cindy.xie@intel.com>; Cabrales, Ada <ada.cabrales@intel.com>; Jones, Bruce E <bruce.e.jones@intel.com>; Qin, Kailun <kailun.qin@intel.com>; Guo, Ruijing <ruijing.guo@intel.com>; Le, Huifeng <huifeng.le@intel.com>
Subject: RE: [Starlingx-discuss] OVS-DPDK Upgrade Testing

 

Hi Chenjie,

I spoke with Joseph. We don’t believe you need to make any puppet changes to test qos.

Storyboard https://storyboard.openstack.org/#!/story/2003948 (which you already noted below) removed the STX/WR custom QoS extensions, so StarlingX is now aligned with the upstream neutron behavior.

 

I have added Chris Winnicki to this email thread. He recently verified QoS functionality in StarlingX after the above Story was merged. He can provide you with the steps. Chris is away today, but should be able to respond tomorrow.

 

Regards,

Ghada

 

From: Xu, Chenjie [mailto:chenjie.xu@intel.com]
Sent: Tuesday, March 05, 2019 8:54 AM
To: Richard, Joseph; Khalil, Ghada; Peters, Matt
Cc: starlingx-discuss@lists.starlingx.io; Lin, Shuicheng; Martinez Monroy, Elio; Zhao, Forrest; Xie, Cindy; Cabrales, Ada; Jones, Bruce E; Qin, Kailun; Guo, Ruijing; Le, Huifeng
Subject: RE: [Starlingx-discuss] OVS-DPDK Upgrade Testing

 

Hi Joseph,
About the configuration of QOS, I don’t know which puppet file should be modified to add the following configuration into the file /etc/neutron/plugins/ml2/openvswitch.ini:
extensions = qos

Could you please tell which puppet file I should modify?

 

Best Regards,

Xu, Chenjie

 

From: Xu, Chenjie
Sent: Monday, March 4, 2019 11:39 PM
To: 'Richard, Joseph' <Joseph.Richard@windriver.com>; 'Ghada.Khalil@windriver.com' <Ghada.Khalil@windriver.com>; 'Peters, Matt' <Matt.Peters@windriver.com>
Cc: starlingx-discuss@lists.starlingx.io; Lin, Shuicheng <shuicheng.lin@intel.com>; Martinez Monroy, Elio <elio.martinez.monroy@intel.com>; Zhao, Forrest <forrest.zhao@intel.com>; Xie, Cindy <cindy.xie@intel.com>; Cabrales, Ada <ada.cabrales@intel.com>; Jones, Bruce E <bruce.e.jones@intel.com>; Qin, Kailun <kailun.qin@intel.com>; Guo, Ruijing <ruijing.guo@intel.com>; Le, Huifeng <huifeng.le@intel.com>
Subject: FW: [Starlingx-discuss] OVS-DPDK Upgrade Testing

 

Hi Joseph,

For the following bug sent on last Friday:

system host-port-list compute-1
'pdevice'

I attached the /var/log/sysinv.log of controller-0 and compute-1. According to the logs, the compute-1 is not correctly installed. Could you please take a look at the logs and provide some comments?

 

Best Regards,

Xu, Chenjie

 

From: Xu, Chenjie
Sent: Monday, March 4, 2019 10:08 PM
To: Richard, Joseph <Joseph.Richard@windriver.com>; Ghada.Khalil@windriver.com; Peters, Matt <Matt.Peters@windriver.com>
Cc: starlingx-discuss@lists.starlingx.io; Lin, Shuicheng <shuicheng.lin@intel.com>; Martinez Monroy, Elio <elio.martinez.monroy@intel.com>; Zhao, Forrest <forrest.zhao@intel.com>; Xie, Cindy <cindy.xie@intel.com>; Cabrales, Ada <ada.cabrales@intel.com>; Jones, Bruce E <bruce.e.jones@intel.com>; Qin, Kailun <kailun.qin@intel.com>; Guo, Ruijing <ruijing.guo@intel.com>; Le, Huifeng <huifeng.le@intel.com>
Subject: RE: [Starlingx-discuss] OVS-DPDK Upgrade Testing

 

Hi Joseph,

Thank you for your reply! For now I’m trying to configure QOS in StarlingX by changing puppet to change neutron configuration files. But I think this is not a good way to configure QOS in StarlingX. Could you please provide some guide on how to configure QOS in StarlingX? I noticed that the StarlingX custom QOS extension has been removed. (https://storyboard.openstack.org/#!/story/2003948)

 

Best Regards,

Xu, Chenjie

 

From: Richard, Joseph [mailto:Joseph.Richard@windriver.com]
Sent: Saturday, March 2, 2019 12:41 AM
To: Xu, Chenjie <chenjie.xu@intel.com>; Khalil, Ghada <Ghada.Khalil@windriver.com>; Peters, Matt <Matt.Peters@windriver.com>
Cc: starlingx-discuss@lists.starlingx.io; Lin, Shuicheng <shuicheng.lin@intel.com>; Martinez Monroy, Elio <elio.martinez.monroy@intel.com>; Zhao, Forrest <forrest.zhao@intel.com>; Xie, Cindy <cindy.xie@intel.com>; Cabrales, Ada <ada.cabrales@intel.com>; Jones, Bruce E <bruce.e.jones@intel.com>; Qin, Kailun <kailun.qin@intel.com>; Guo, Ruijing <ruijing.guo@intel.com>; Le, Huifeng <huifeng.le@intel.com>
Subject: RE: [Starlingx-discuss] OVS-DPDK Upgrade Testing

 

1:

You need to use sysinv to add the address, not add it using ip commands.

usage: system host-addr-add <hostname or id> <interface name or id> <ipv4/ipv6

                            address> <prefix length>

 

2:

Check in /var/log/sysinv.log for the traceback at that time.

 

From: Xu, Chenjie [mailto:chenjie.xu@intel.com]
Sent: Friday, March 1, 2019 9:55 AM
To: Richard, Joseph; Khalil, Ghada; Peters, Matt
Cc:
starlingx-discuss@lists.starlingx.io; Lin, Shuicheng; Martinez Monroy, Elio; Zhao, Forrest; Xie, Cindy; Cabrales, Ada; Jones, Bruce E; Qin, Kailun; Guo, Ruijing; Le, Huifeng
Subject: [Starlingx-discuss] OVS-DPDK Upgrade Testing

 

Hi Joseph,

I’m responsible for the Basic Functional Testing for OVS-DPDK Upgrade. I have set up a StarlingX multimode environment (1 controller and 2 computes) using the ISO image for OVS/DPDK upgrade test. I have 2 questions for you:

 

1. I don’t know how associate a vxlan providernet with compute node. Could you please provide the steps how to associate a vxlan providernet with compute node? The commands I use are following:
     neutron providernet-create tenant_vxlan --type=vxlan --mtu=1400

neutron providernet-range-create --name tenant_vxlan-a-range1 --range 100-400 --group 224.0.0.1 --ttl 255  tenant_vxlan

system datanetwork-add tenant_vxlan vxlan --multicast_group 224.0.0.1 --ttl 255 --port_num 4789

system host-if-modify -p tenant_vxlan -c data compute-0 enp4s0f1

system host-unlock compute-0

After executing “system host-unlock compute-0”, it will report “Interface enp4s0f1 is associated to VXLAN data network tenant_vxlan which requires an IPv4 address”. But I do allocate a IP address for enp4s0f1. You can check the attached images.


2. I have found one bug and will investigate it. If you have any ideas please let me know. After installing the compute-1, I try to list the interface on compute-1 by following command:

system host-port-list compute-1
'pdevice'

The normal output should be:
system host-port-list compute-0
+--------------------------------------+----------+----------+--------------+--------+-----------+-------------+------------------------------------------------+

| uuid                                 | name     | type     | pci address  | device | processor | accelerated | device type                                    |

+--------------------------------------+----------+----------+--------------+--------+-----------+-------------+------------------------------------------------+

| e072d576-90a6-4040-87df-66a913076df0 | eno1     | ethernet | 0000:07:00.0 | 0      | 0         | True        | I350 Gigabit Network Connection                |

| 485b6e35-5b75-4b73-93dc-7deae0fb5c5c | eno2     | ethernet | 0000:07:00.1 | 0      | 0         | True        | I350 Gigabit Network Connection                |

| 8b448ab6-e9df-4ce7-84a4-256d97f6fd5b | enp4s0f0 | ethernet | 0000:04:00.0 | 0      | 0         | True        | 82599ES 10-Gigabit SFI/SFP+   Network Connection |

| 6d5b1ce7-ae76-4781-9e51-93bfe8417dc0 | enp4s0f1 | ethernet | 0000:04:00.1 | 0      | 0         | True        | 82599ES 10-Gigabit SFI/SFP+ Network Connection |

+--------------------------------------+----------+----------+--------------+--------+-----------+-------------+------------------------------------------------+

 

Best Regards,

Xu, Chenjie

 

From: Xu, Chenjie
Sent: Friday, March 1, 2019 2:32 PM
To: Lin, Shuicheng <shuicheng.lin@intel.com>
Cc: Martinez Monroy, Elio <elio.martinez.monroy@intel.com>; Zhao, Forrest <forrest.zhao@intel.com>; Xie, Cindy <cindy.xie@intel.com>; Cabrales, Ada <ada.cabrales@intel.com>; Jones, Bruce E <bruce.e.jones@intel.com>; Qin, Kailun <kailun.qin@intel.com>; Guo, Ruijing <ruijing.guo@intel.com>; Le, Huifeng <huifeng.le@intel.com>
Subject: RE: ISO image for OVS/DPDK upgrade

 

Hi Shuicheng,

Thank you for your information! I will start to do the Basic Functional Test.

 

Best Regards,

Xu, Chenjie

 

 

From: Lin, Shuicheng
Sent: Friday, March 1, 2019 2:03 PM
To: Xu, Chenjie <chenjie.xu@intel.com>; Martinez Monroy, Elio <elio.martinez.monroy@intel.com>
Cc: Zhao, Forrest <forrest.zhao@intel.com>; Xie, Cindy <cindy.xie@intel.com>; Cabrales, Ada <ada.cabrales@intel.com>; Jones, Bruce E <bruce.e.jones@intel.com>; Qin, Kailun <kailun.qin@intel.com>; Guo, Ruijing <ruijing.guo@intel.com>
Subject: ISO image for OVS/DPDK upgrade

 

Hi Chenjie/Elio,

Here is the ISO image for OVS/DPDK upgrade test.

I have done basic deploy test, and could ping VM from each other successfully.

Please help have a try with it, and notify me if there is any issue.

It is built with recent code of centos76 feature branch.

 

ec@10.239.154.150:/home/ec/bootimage_ovs.iso

scp passwd: 1

 

Here is the patch list for the upgrade in case you want to have a check:

https://review.openstack.org/640244

https://review.openstack.org/640245

 

Best Regards

Shuicheng