<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:dt="uuid:C2F41010-65B3-11d1-A29F-00AA00C14882" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:SimSun;
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:"\@SimSun";
panose-1:2 1 6 0 3 1 1 1 1 1;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle18
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:windowtext;}
span.EmailStyle19
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:#1F497D;}
span.EmailStyle20
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:windowtext;}
span.EmailStyle21
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:#1F497D;}
span.EmailStyle22
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:#1F497D;}
span.EmailStyle23
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:windowtext;}
span.EmailStyle24
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:#1F497D;}
span.EmailStyle25
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:#1F497D;}
span.EmailStyle26
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:#1F497D;}
span.EmailStyle27
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:#1F497D;}
span.EmailStyle28
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:#1F497D;}
span.EmailStyle29
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:#1F497D;}
span.EmailStyle30
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:#1F497D;}
span.EmailStyle31
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:windowtext;}
span.EmailStyle32
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:#1F497D;}
span.EmailStyle33
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:#1F497D;}
span.EmailStyle34
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:windowtext;}
span.EmailStyle35
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:#1F497D;}
span.EmailStyle36
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:#1F497D;}
span.EmailStyle37
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:#1F497D;}
span.EmailStyle38
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:#1F497D;}
span.EmailStyle39
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:#1F497D;}
span.EmailStyle40
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:windowtext;}
span.EmailStyle41
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:windowtext;}
span.EmailStyle42
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:#1F497D;}
span.EmailStyle43
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:#1F497D;}
span.EmailStyle44
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:windowtext;}
span.EmailStyle45
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:#1F497D;}
span.EmailStyle46
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang="EN-CA" link="#0563C1" vlink="#954F72">
<div class="WordSection1">
<p class="MsoNormal">Hi Chenjie,<o:p></o:p></p>
<p class="MsoNormal">Sorry for the delayed reply.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">For the port that is created / plugged by OpenStack, can you confirm the flows that are visible to the OVS datapath? Are the packets seen by OVS br-int and then not switched properly or are the packets not being processed by the vhost
port at all? If you are seeing packets reach the bridge, can you confirm if all packet types are not getting through, i.e. is ARP working, but not ICMP.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">You can dump the datapath ports and flows with the following command:<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p style="margin:0cm;margin-bottom:.0001pt"><span lang="EN-US">ovs-appctl dpif/show<o:p></o:p></span></p>
<p style="margin:0cm;margin-bottom:.0001pt"><span lang="EN-US">ovs-appctl dpif/dump-flows
</span>br-int<span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Also, when you did your manual test, did you have the other VM present as well, or was it a fully manual setup with no OpenStack agents present?<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Regards, Matt<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span style="font-size:12.0pt;color:black">From: </span></b><span style="font-size:12.0pt;color:black">"Xu, Chenjie" <chenjie.xu@intel.com><br>
<b>Date: </b>Friday, March 15, 2019 at 2:48 PM<br>
<b>To: </b>"Peters, Matt" <Matt.Peters@windriver.com>, Ghada Khalil <Ghada.Khalil@windriver.com>, "Xie, Cindy" <cindy.xie@intel.com>, "Zhao, Forrest" <forrest.zhao@intel.com>, "Richard, Joseph" <Joseph.Richard@windriver.com>, "Winnicki, Chris" <Chris.Winnicki@windriver.com>,
"Lin, Shuicheng" <shuicheng.lin@intel.com>, "Peng, Peng" <Peng.Peng@windriver.com><br>
<b>Cc: </b>"'starlingx-discuss@lists.starlingx.io'" <starlingx-discuss@lists.starlingx.io>, "Martinez Monroy, Elio" <elio.martinez.monroy@intel.com>, "Cabrales, Ada" <ada.cabrales@intel.com>, "Jones, Bruce E" <bruce.e.jones@intel.com>, "Qin, Kailun" <kailun.qin@intel.com>,
"Guo, Ruijing" <ruijing.guo@intel.com>, "Le, Huifeng" <huifeng.le@intel.com><br>
<b>Subject: </b>RE: [Starlingx-discuss] OVS-DPDK Upgrade Testing<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;color:#1F497D">Hi Matt,</span><o:p></o:p></p>
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:10.5pt;color:#1F497D">We create vhostuser port and VM manually. And the connectivity between manually created VM and br-int works correctly. But the vhostuser port created by StarlingX doesn’t work</span></b><span lang="EN-US" style="font-size:10.5pt;color:#1F497D">.
Could you please help review our following debugging steps and leave a comment?</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;color:#1F497D"><br>
1. We manually assigned correct IP to the StarlingX VM’s interface eth0 and ping DHCP server in VM. No packet can be captured by tcpdump and thus we think vhostuser port doesn’t work.</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:21.0pt;text-indent:21.0pt"><span lang="EN-US" style="font-size:10.5pt;color:#1F497D">Bring up br-int to verify the connectivity: sudo ifconfig br-int 192.168.101.1/24 up</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;color:#1F497D"> Check the IP assigned to the VM: openstack server list</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;color:#1F497D"> Manually assign IP to VM: sudo ifconfig 192.168.101.216/24 up</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;color:#1F497D"> Tcpdump in dhcp namespace: sudo ip netns exec $dhcp_namespace tcpdump –i $tap_device</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;color:#1F497D"> Ping DHCP server in VM: ping 192.168.101.1</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;color:#1F497D"> No packets can be captured.</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;color:#1F497D">2. We added our own vhost user port to br-int and use virsh command and cirros-dpdk-vhostuserclient.xml (you can find in attached files) to create VM:</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:126.0pt;text-indent:-84.0pt"><span lang="EN-US" style="font-size:10.5pt;color:#1F497D">Create vhostuser port: ovs-vsctl add-port br-int vhost-user-1 -- set Interface vhost-user-1
</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:231.0pt;text-indent:21.0pt"><span lang="EN-US" style="font-size:10.5pt;color:#1F497D">type=dpdkvhostuserclient
</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:231.0pt;text-indent:21.0pt"><span lang="EN-US" style="font-size:10.5pt;color:#1F497D">options:vhost-server-path=/var/run/openvswitch/vhost-user-1</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:21.0pt;text-indent:21.0pt"><span lang="EN-US" style="font-size:10.5pt;color:#1F497D">Create VM: virsh create cirros-dpdk-vhostuserclient.xml</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:21.0pt;text-indent:21.0pt"><span lang="EN-US" style="font-size:10.5pt;color:#1F497D">Bring up br-int to verify the connectivity: sudo ifconfig br-int 192.168.101.1/24 up</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:21.0pt;text-indent:21.0pt"><span lang="EN-US" style="font-size:10.5pt;color:#1F497D">Manually assign an IP to VM: sudo ifconfig eth0 192.168.101.18/24 up</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:21.0pt;text-indent:21.0pt"><span lang="EN-US" style="font-size:10.5pt;color:#1F497D">In created VM, ping br-int: ping 192.168.101.1</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:21.0pt;text-indent:21.0pt"><span lang="EN-US" style="font-size:10.5pt;color:#1F497D">And the ping succeeds.</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;color:#1F497D">Best Regards,</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;color:#1F497D">Xu, Chenjie</span><o:p></o:p></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><a name="_____replyseparator"></a><b><span lang="EN-US">From:</span></b><span lang="EN-US"> Peters, Matt [mailto:Matt.Peters@windriver.com]
<br>
<b>Sent:</b> Wednesday, March 13, 2019 9:13 PM<br>
<b>To:</b> Xu, Chenjie <chenjie.xu@intel.com>; Khalil, Ghada <Ghada.Khalil@windriver.com>; Xie, Cindy <cindy.xie@intel.com>; Zhao, Forrest <forrest.zhao@intel.com>; Richard, Joseph <Joseph.Richard@windriver.com>; Winnicki, Chris <Chris.Winnicki@windriver.com>;
Lin, Shuicheng <shuicheng.lin@intel.com>; Peng, Peng <Peng.Peng@windriver.com><br>
<b>Cc:</b> 'starlingx-discuss@lists.starlingx.io' <starlingx-discuss@lists.starlingx.io>; Martinez Monroy, Elio <elio.martinez.monroy@intel.com>; Cabrales, Ada <ada.cabrales@intel.com>; Jones, Bruce E <bruce.e.jones@intel.com>; Qin, Kailun <kailun.qin@intel.com>;
Guo, Ruijing <ruijing.guo@intel.com>; Le, Huifeng <huifeng.le@intel.com><br>
<b>Subject:</b> Re: [Starlingx-discuss] OVS-DPDK Upgrade Testing</span><o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><span lang="EN-US"> </span><o:p></o:p></p>
<p class="MsoNormal">Hello,<o:p></o:p></p>
<p class="MsoNormal">That message is a normal message from DPDK if you have multiple huge page sizes configured in your system. Openvswitch is configured to use 1G huge pages and will ignore the 2M huge pages.<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal">You can confirm this by looking at the Puppet logs to see what was configured and to ensure the 1G huge pages were mounted prior to openvswitch starting.<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal">-Matt<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span style="font-size:12.0pt;color:black">From: </span></b><span style="font-size:12.0pt;color:black">"Xu, Chenjie" <</span><span lang="EN-US"><a href="mailto:chenjie.xu@intel.com"><span lang="EN-US" style="font-size:12.0pt"><span lang="EN-US">chenjie.xu@intel.com</span></span></a></span><span style="font-size:12.0pt;color:black">><br>
<b>Date: </b>Wednesday, March 13, 2019 at 8:55 AM<br>
<b>To: </b>"Peters, Matt" <</span><span lang="EN-US"><a href="mailto:Matt.Peters@windriver.com"><span lang="EN-US" style="font-size:12.0pt"><span lang="EN-US">Matt.Peters@windriver.com</span></span></a></span><span style="font-size:12.0pt;color:black">>, Ghada
Khalil <</span><span lang="EN-US"><a href="mailto:Ghada.Khalil@windriver.com"><span lang="EN-US" style="font-size:12.0pt"><span lang="EN-US">Ghada.Khalil@windriver.com</span></span></a></span><span style="font-size:12.0pt;color:black">>, "Xie, Cindy" <</span><span lang="EN-US"><a href="mailto:cindy.xie@intel.com"><span lang="EN-US" style="font-size:12.0pt"><span lang="EN-US">cindy.xie@intel.com</span></span></a></span><span style="font-size:12.0pt;color:black">>,
"Zhao, Forrest" <</span><span lang="EN-US"><a href="mailto:forrest.zhao@intel.com"><span lang="EN-US" style="font-size:12.0pt"><span lang="EN-US">forrest.zhao@intel.com</span></span></a></span><span style="font-size:12.0pt;color:black">>, "Richard, Joseph"
<</span><span lang="EN-US"><a href="mailto:Joseph.Richard@windriver.com"><span lang="EN-US" style="font-size:12.0pt"><span lang="EN-US">Joseph.Richard@windriver.com</span></span></a></span><span style="font-size:12.0pt;color:black">>, "Winnicki, Chris" <</span><span lang="EN-US"><a href="mailto:Chris.Winnicki@windriver.com"><span lang="EN-US" style="font-size:12.0pt"><span lang="EN-US">Chris.Winnicki@windriver.com</span></span></a></span><span style="font-size:12.0pt;color:black">>,
"Lin, Shuicheng" <</span><span lang="EN-US"><a href="mailto:shuicheng.lin@intel.com"><span lang="EN-US" style="font-size:12.0pt"><span lang="EN-US">shuicheng.lin@intel.com</span></span></a></span><span style="font-size:12.0pt;color:black">>, "Peng, Peng" <</span><span lang="EN-US"><a href="mailto:Peng.Peng@windriver.com"><span lang="EN-US" style="font-size:12.0pt"><span lang="EN-US">Peng.Peng@windrive<span lang="EN-US">r.com</span></span></span></a></span><span style="font-size:12.0pt;color:black">><br>
<b>Cc: </b>"'starlingx-discuss@lists.starlingx.io'" <</span><span lang="EN-US"><a href="mailto:starlingx-discuss@lists.starlingx.io"><span lang="EN-US" style="font-size:12.0pt"><span lang="EN-US">starlingx-discuss@lists.starlingx.io</span></span></a></span><span style="font-size:12.0pt;color:black">>,
"Martinez Monroy, Elio" <</span><span lang="EN-US"><a href="mailto:elio.martinez.monroy@intel.com"><span lang="EN-US" style="font-size:12.0pt"><span lang="EN-US">elio.martinez.monroy@inte<span lang="EN-US">l.com</span></span></span></a></span><span style="font-size:12.0pt;color:black">>,
"Cabrales, Ada" <</span><span lang="EN-US"><a href="mailto:ada.cabrales@intel.com"><span lang="EN-US" style="font-size:12.0pt"><span lang="EN-US">ada.cabrales@intel.com</span></span></a></span><span style="font-size:12.0pt;color:black">>, "Jones, Bruce E"
<</span><span lang="EN-US"><a href="mailto:bruce.e.jones@intel.com"><span lang="EN-US" style="font-size:12.0pt"><span lang="EN-US">bruce.e.jones@intel.com</span></span></a></span><span style="font-size:12.0pt;color:black">>, "Qin, Kailun" <</span><span lang="EN-US"><a href="mailto:kailun.qin@intel.com"><span lang="EN-US" style="font-size:12.0pt"><span lang="EN-US">kailun.qin@intel.com</span></span></a></span><span style="font-size:12.0pt;color:black">>,
"Guo, Ruijing" <</span><span lang="EN-US"><a href="mailto:ruijing.guo@intel.com"><span lang="EN-US" style="font-size:12.0pt"><span lang="EN-US">ruijing.guo@intel.com</span></span></a></span><span style="font-size:12.0pt;color:black">>, "Le, Huifeng" <</span><span lang="EN-US"><a href="mailto:huifeng.le@intel.com"><span lang="EN-US" style="font-size:12.0pt"><span lang="EN-US">huifeng.le@intel.com</span></span></a></span><span style="font-size:12.0pt;color:black">><br>
<b>Subject: </b>RE: [Starlingx-discuss] OVS-DPDK Upgrade Testing</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;color:#1F497D">Hi team,</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;color:#1F497D">I installed StarlingX using following the March 5<sup>th</sup> ISO image and the issue still exists</span><span lang="EN-US" style="font-size:10.5pt;font-family:SimSun;color:#1F497D">：</span><span lang="EN-US"><a href="http://mirror.starlingx.cengn.ca/mirror/starlingx/master/centos/20190305T060000Z/outputs/iso/"><span style="font-size:10.5pt">http://mirror.starlingx.cengn.ca/mirror/starlingx/master/centos/20190305T060000Z/outputs/iso/</span></a></span><o:p></o:p></p>
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:10.5pt;color:#1F497D">Are there any steps not been written on the guide?
</span></b><span lang="EN-US" style="font-size:10.5pt;color:#1F497D">I just followed the following guide:</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US"><a href="https://wiki.openstack.org/wiki/StarlingX/Containers/InstallationOnStandard"><span style="font-size:10.5pt">https://wiki.openstack.org/wiki/StarlingX/Containers/InstallationOnStandard</span></a></span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;color:#1F497D">I checked the ovs-vswitchd.log and the following line maybe the cause:</span><o:p></o:p></p>
<p class="MsoNormal" style="text-indent:21.0pt"><span lang="EN-US" style="font-size:10.5pt;color:#1F497D">23127 hugepages of size 2097152 reserved, but no mounted hugetlbfs found for that size</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;color:#1F497D">It seems that something went wrong with hugepage. Some logs have been attached.
<b>If you have any idea about hugepage, please let me know.</b></span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;color:#1F497D">Peng has been added and hope Peng can provide some information about his environment.<br>
</span><span lang="EN-US"><a href="https://bugs.launchpad.net/starlingx/+bug/1819031"><span style="font-size:10.5pt">https://bugs.launchpad.net/starlingx/+bug/1819031</span></a></span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;color:#1F497D">Best Regards,
</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;color:#1F497D">Xu, Chenjie</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;color:#1F497D"> </span><o:p></o:p></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span lang="EN-US">From:</span></b><span lang="EN-US"> Xu, Chenjie
<br>
<b>Sent:</b> Tuesday, March 12, 2019 12:39 AM<br>
<b>To:</b> Peters, Matt <<a href="mailto:Matt.Peters@windriver.com">Matt.Peters@windriver.com</a>>; Khalil, Ghada <<a href="mailto:Ghada.Khalil@windriver.com">Ghada.Khalil@windriver.com</a>>; Xie, Cindy <<a href="mailto:cindy.xie@intel.com">cindy.xie@intel.com</a>>;
Zhao, Forrest <<a href="mailto:forrest.zhao@intel.com">forrest.zhao@intel.com</a>>; Richard, Joseph <<a href="mailto:Joseph.Richard@windriver.com">Joseph.Richard@windriver.com</a>>; Winnicki, Chris <<a href="mailto:Chris.Winnicki@windriver.com">Chris.Winnicki@windriver.com</a>>;
Lin, Shuicheng <<a href="mailto:shuicheng.lin@intel.com">shuicheng.lin@intel.com</a>><br>
<b>Cc:</b> <a href="mailto:starlingx-discuss@lists.starlingx.io">starlingx-discuss@lists.starlingx.io</a>; Martinez Monroy, Elio <<a href="mailto:elio.martinez.monroy@intel.com">elio.martinez.monroy@intel.com</a>>; Cabrales, Ada <<a href="mailto:ada.cabrales@intel.com">ada.cabrales@intel.com</a>>;
Jones, Bruce E <<a href="mailto:bruce.e.jones@intel.com">bruce.e.jones@intel.com</a>>; Qin, Kailun <<a href="mailto:kailun.qin@intel.com">kailun.qin@intel.com</a>>; Guo, Ruijing <<a href="mailto:ruijing.guo@intel.com">ruijing.guo@intel.com</a>>; Le, Huifeng
<<a href="mailto:huifeng.le@intel.com">huifeng.le@intel.com</a>><br>
<b>Subject:</b> RE: [Starlingx-discuss] OVS-DPDK Upgrade Testing</span><o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><span lang="EN-US"> </span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;color:#1F497D">Hi team,</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;color:#1F497D">For Matt, I just tried to lock compute-1 to ensure the VMs is on the same compute host. But it still doesn’t work. The DHCP, router and VMs all located on compute-0.</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;color:#1F497D">For Ghada, I checked the test cases listed in sanity test. I see that there is no test case to test the connectivity between VMs. Need confirmation from Ada’s team.<br>
</span><span lang="EN-US"><a href="https://wiki.openstack.org/wiki/StarlingX/Test/SanityTests"><span style="font-size:10.5pt">https://wiki.openstack.org/wiki/StarlingX/Test/SanityTests</span></a></span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;color:#1F497D">Best Regards,</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;color:#1F497D">Xu, Chenjie</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;color:#1F497D"> </span><o:p></o:p></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span lang="EN-US">From:</span></b><span lang="EN-US"> Peters, Matt [<a href="mailto:Matt.Peters@windriver.com">mailto:Matt.Peters@windriver.com</a>]
<br>
<b>Sent:</b> Tuesday, March 12, 2019 12:28 AM<br>
<b>To:</b> Khalil, Ghada <<a href="mailto:Ghada.Khalil@windriver.com">Ghada.Khalil@windriver.com</a>>; Xu, Chenjie <<a href="mailto:chenjie.xu@intel.com">chenjie.xu@intel.com</a>>; Xie, Cindy <<a href="mailto:cindy.xie@intel.com">cindy.xie@intel.com</a>>; Zhao,
Forrest <<a href="mailto:forrest.zhao@intel.com">forrest.zhao@intel.com</a>>; Richard, Joseph <<a href="mailto:Joseph.Richard@windriver.com">Joseph.Richard@windriver.com</a>>; Winnicki, Chris <<a href="mailto:Chris.Winnicki@windriver.com">Chris.Winnicki@windriver.com</a>>;
Lin, Shuicheng <<a href="mailto:shuicheng.lin@intel.com">shuicheng.lin@intel.com</a>><br>
<b>Cc:</b> <a href="mailto:starlingx-discuss@lists.starlingx.io">starlingx-discuss@lists.starlingx.io</a>; Martinez Monroy, Elio <<a href="mailto:elio.martinez.monroy@intel.com">elio.martinez.monroy@intel.com</a>>; Cabrales, Ada <<a href="mailto:ada.cabrales@intel.com">ada.cabrales@intel.com</a>>;
Jones, Bruce E <<a href="mailto:bruce.e.jones@intel.com">bruce.e.jones@intel.com</a>>; Qin, Kailun <<a href="mailto:kailun.qin@intel.com">kailun.qin@intel.com</a>>; Guo, Ruijing <<a href="mailto:ruijing.guo@intel.com">ruijing.guo@intel.com</a>>; Le, Huifeng
<<a href="mailto:huifeng.le@intel.com">huifeng.le@intel.com</a>><br>
<b>Subject:</b> Re: [Starlingx-discuss] OVS-DPDK Upgrade Testing</span><o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><span lang="EN-US"> </span><o:p></o:p></p>
<p class="MsoNormal">If you co-locate the VMs on the same compute host (lock the other computes) do things work as expected? I.e. is it only when you are traversing the physical network that things are not pingable?<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal">-Matt<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span style="font-size:12.0pt;color:black">From: </span></b><span style="font-size:12.0pt;color:black">Ghada Khalil <</span><span lang="EN-US"><a href="mailto:Ghada.Khalil@windriver.com"><span lang="EN-US" style="font-size:12.0pt"><span lang="EN-US">Ghada.Khalil@windriver.com</span></span></a></span><span style="font-size:12.0pt;color:black">><br>
<b>Date: </b>Monday, March 11, 2019 at 10:51 AM<br>
<b>To: </b>"Xu, Chenjie" <</span><span lang="EN-US"><a href="mailto:chenjie.xu@intel.com"><span lang="EN-US" style="font-size:12.0pt"><span lang="EN-US">chenjie.xu@intel.com</span></span></a></span><span style="font-size:12.0pt;color:black">>, "Xie, Cindy"
<</span><span lang="EN-US"><a href="mailto:cindy.xie@intel.com"><span lang="EN-US" style="font-size:12.0pt"><span lang="EN-US">cindy.xie@intel.com</span></span></a></span><span style="font-size:12.0pt;color:black">>, "Zhao, Forrest" <</span><span lang="EN-US"><a href="mailto:forrest.zhao@intel.com"><span lang="EN-US" style="font-size:12.0pt"><span lang="EN-US">forrest.zhao@intel.com</span></span></a></span><span style="font-size:12.0pt;color:black">>,
"Peters, Matt" <</span><span lang="EN-US"><a href="mailto:Matt.Peters@windriver.com"><span lang="EN-US" style="font-size:12.0pt"><span lang="EN-US">Matt.Peters@windriver.com</span></span></a></span><span style="font-size:12.0pt;color:black">>, "Richard, Joseph"
<</span><span lang="EN-US"><a href="mailto:Joseph.Richard@windriver.com"><span lang="EN-US" style="font-size:12.0pt"><span lang="EN-US">Joseph.Richard@windriver.com</span></span></a></span><span style="font-size:12.0pt;color:black">>, "Winnicki, Chris" <</span><span lang="EN-US"><a href="mailto:Chris.Winnicki@windriver.com"><span lang="EN-US" style="font-size:12.0pt"><span lang="EN-US">Chris.Winnicki@windriver.com</span></span></a></span><span style="font-size:12.0pt;color:black">>,
"Lin, Shuicheng" <</span><span lang="EN-US"><a href="mailto:shuicheng.lin@intel.com"><span lang="EN-US" style="font-size:12.0pt"><span lang="EN-US">shuicheng.lin@intel.com</span></span></a></span><span style="font-size:12.0pt;color:black">><br>
<b>Cc: </b>"</span><span lang="EN-US"><a href="mailto:starlingx-discuss@lists.starlingx.io"><span lang="EN-US" style="font-size:12.0pt"><span lang="EN-US">starlingx-discuss@lists.starlingx.io</span></span></a></span><span style="font-size:12.0pt;color:black">"
<</span><span lang="EN-US"><a href="mailto:starlingx-discuss@lists.starlingx.io"><span lang="EN-US" style="font-size:12.0pt"><span lang="EN-US">starlingx-discuss@lists.starlingx.io</span></span></a></span><span style="font-size:12.0pt;color:black">>, "Martinez
Monroy, Elio" <</span><span lang="EN-US"><a href="mailto:elio.martinez.monroy@intel.com"><span lang="EN-US" style="font-size:12.0pt"><span lang="EN-US">elio.martinez.monroy@intel.com</span></span></a></span><span style="font-size:12.0pt;color:black">>, "Cabrales,
Ada" <</span><span lang="EN-US"><a href="mailto:ada.cabrales@intel.com"><span lang="EN-US" style="font-size:12.0pt"><span lang="EN-US">ada.cabrales@intel.com</span></span></a></span><span style="font-size:12.0pt;color:black">>, "Jones, Bruce E" <</span><span lang="EN-US"><a href="mailto:bruce.e.jones@intel.com"><span lang="EN-US" style="font-size:12.0pt"><span lang="EN-US">bruce.<span lang="EN-US">e.jones@intel.com</span></span></span></a></span><span style="font-size:12.0pt;color:black">>,
"Qin, Kailun" <</span><span lang="EN-US"><a href="mailto:kailun.qin@intel.com"><span lang="EN-US" style="font-size:12.0pt"><span lang="EN-US">kailun.qin@intel.com</span></span></a></span><span style="font-size:12.0pt;color:black">>, "Guo, Ruijing" <</span><span lang="EN-US"><a href="mailto:ruijing.guo@intel.com"><span lang="EN-US" style="font-size:12.0pt"><span lang="EN-US">ruijing.guo@intel.com</span></span></a></span><span style="font-size:12.0pt;color:black">>,
"Le, Huifeng" <</span><span lang="EN-US"><a href="mailto:huifeng.le@intel.com"><span lang="EN-US" style="font-size:12.0pt"><span lang="EN-US">huifeng.le<span lang="EN-US">@intel.com</span></span></span></a></span><span style="font-size:12.0pt;color:black">><br>
<b>Subject: </b>RE: [Starlingx-discuss] OVS-DPDK Upgrade Testing</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<p class="MsoNormal"><span style="color:#1F497D">To answer Forrest’s question:</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D"> Do you know if OVS/DPDK has been tested after cut-over to container?</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">I have seen Green sanity reports from Ada’s team since the cut-over. I expect this covers basic ovs-dpdk functional testing. Ada, please correct me if I am wrong.</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">Regarding </span><span lang="EN-US"><a href="https://bugs.launchpad.net/starlingx/+bug/1819031"><span lang="EN-US"><span lang="EN-US">https://bugs.launchpad.net/starlingx/+bug/1819031</span></span></a></span><span style="color:#1F497D">,
we’ve requested information from the reporter on how reproducible this issue is. As far as I know, it was reported on one system only. We’re waiting for more information from Peng.</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">Regards,</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">Ghada</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span><o:p></o:p></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b>From:</b> Xu, Chenjie [<span lang="EN-US"><a href="mailto:chenjie.xu@intel.com"><span lang="EN-US"><span lang="EN-US">mailto:chenjie.xu@intel.com</span></span></a></span>]
<br>
<b>Sent:</b> Monday, March 11, 2019 10:40 AM<br>
<b>To:</b> Xie, Cindy; Zhao, Forrest; Peters, Matt; Richard, Joseph; Khalil, Ghada; Winnicki, Chris; Lin, Shuicheng<br>
<b>Cc:</b> <span lang="EN-US"><a href="mailto:starlingx-discuss@lists.starlingx.io"><span lang="EN-US"><span lang="EN-US">starlingx-discuss@lists.starlingx.io</span></span></a></span>; Martinez Monroy, Elio; Cabrales, Ada; Jones, Bruce E; Qin, Kailun; Guo,
Ruijing; Le, Huifeng<br>
<b>Subject:</b> RE: [Starlingx-discuss] OVS-DPDK Upgrade Testing<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1F497D">Hi Cindy,</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1F497D">Thank you for your information! The following issue is not exactly the same as mine:</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US"><a href="https://bugs.launchpad.net/starlingx/+bug/1819031"><span lang="EN-US"><span lang="EN-US">https://bugs.launchpad.net/starlingx/+bug/1819031</span></span></a></span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1F497D">In this issue, the DHCP works and external network can reach the VMs. However in my case, VMs can’t get IP from DHCP and external network can’t reach VMs even with an IP allocated by myself.
</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1F497D">Best Regards,</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1F497D">Xu, Chenjie</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1F497D"> </span><o:p></o:p></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b>From:</b> Xie, Cindy <br>
<b>Sent:</b> Monday, March 11, 2019 10:05 PM<br>
<b>To:</b> Zhao, Forrest <<span lang="EN-US"><a href="mailto:forrest.zhao@intel.com"><span lang="EN-US"><span lang="EN-US">forrest.zhao@intel.com</span></span></a></span>>; Xu, Chenjie <<span lang="EN-US"><a href="mailto:chenjie.xu@intel.com"><span lang="EN-US"><span lang="EN-US">chenjie.xu@intel.com</span></span></a></span>>;
Peters, Matt <<span lang="EN-US"><a href="mailto:Matt.Peters@windriver.com"><span lang="EN-US"><span lang="EN-US">M<span lang="EN-US">att.Peters@windriver.com</span></span></span></a></span>>; Richard, Joseph <<span lang="EN-US"><a href="mailto:Joseph.Richard@windriver.com"><span lang="EN-US"><span lang="EN-US">Joseph.Richard@windriver.com</span></span></a></span>>;
Khalil, Ghada <<span lang="EN-US"><a href="mailto:Ghada.Khalil@windriver.com"><span lang="EN-US"><span lang="EN-US">Ghada.Khalil@windriver.com</span></span></a></span>>;
<span lang="EN-US"><a href="mailto:Chris.Winnicki@windriver.com"><span lang="EN-US"><span lang="EN-US">Chris.Winnicki@windriver.com</span></span></a></span>; Lin, Shuicheng <<span lang="EN-US"><a href="mailto:shuicheng.lin@intel.com"><span lang="EN-US"><span lang="EN-US">shuicheng.lin@intel.com</span></span></a></span>><br>
<b>Cc:</b> <span lang="EN-US"><a href="mailto:starlingx-discuss@lists.starlingx.io"><span lang="EN-US"><span lang="EN-US">starlingx-discuss@list<span lang="EN-US">s.starlingx.io</span></span></span></a></span>; Martinez Monroy, Elio <<span lang="EN-US"><a href="mailto:elio.martinez.monroy@intel.com"><span lang="EN-US"><span lang="EN-US">elio.martinez.monroy@intel.com</span></span></a></span>>;
Cabrales, Ada <<span lang="EN-US"><a href="mailto:ada.cabrales@intel.com"><span lang="EN-US"><span lang="EN-US">ada.cabrales@intel.com</span></span></a></span>>; Jones, Bruce E <<span lang="EN-US"><a href="mailto:bruce.e.jones@intel.com"><span lang="EN-US"><span lang="EN-US">bruce.e.jones@intel.com</span></span></a></span>>;
Qin, Kailun <<span lang="EN-US"><a href="mailto:kailun.qin@intel.com"><span lang="EN-US"><span lang="EN-US">kailun.qin@intel.com</span></span></a></span>>; Guo, Ruijing <<span lang="EN-US"><a href="mailto:ruijing.guo@intel.com"><span lang="EN-US"><span lang="EN-US">ruijing.guo@intel.com</span></span></a></span>>;
Le, Huifeng <<span lang="EN-US"><a href="mailto:huifeng.le@intel.com"><span lang="EN-US"><span lang="EN-US">huifeng.le@intel.com</span></span></a></span>><br>
<b>Subject:</b> RE: [Starlingx-discuss] OVS-DPDK Upgrade Testing<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">Is the below issue similar?</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US"><a href="https://bugs.launchpad.net/starlingx/+bug/1819031"><span lang="EN-US"><span lang="EN-US">https://bugs.launchpad.net/starlingx/+bug/1819031</span></span></a></span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span><o:p></o:p></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b>From:</b> Zhao, Forrest <br>
<b>Sent:</b> Monday, March 11, 2019 9:54 PM<br>
<b>To:</b> Xu, Chenjie <<span lang="EN-US"><a href="mailto:chenjie.xu@intel.com"><span lang="EN-US"><span lang="EN-US">chenjie.xu@intel.com</span></span></a></span>>; Peters, Matt <<span lang="EN-US"><a href="mailto:Matt.Peters@windriver.com"><span lang="EN-US"><span lang="EN-US">Matt.Peters@windriver.com</span></span></a></span>>;
Richard, Joseph <<span lang="EN-US"><a href="mailto:Joseph.Richard@windriver.com"><span lang="EN-US"><span lang="EN-US">Joseph.Richard@windriver.com</span></span></a></span>>; Khalil, Ghada <<span lang="EN-US"><a href="mailto:Ghada.Khalil@windriver.com"><span lang="EN-US"><span lang="EN-US">Ghada.Khalil@windriver.com</span></span></a></span>>;
<span lang="EN-US"><a href="mailto:Chris.Winnicki@windriver.com"><span lang="EN-US"><span lang="EN-US">Chris.Winnicki@windriver.co<span lang="EN-US">m</span></span></span></a></span>; Lin, Shuicheng <<span lang="EN-US"><a href="mailto:shuicheng.lin@intel.com"><span lang="EN-US"><span lang="EN-US">shuicheng.lin@intel.com</span></span></a></span>><br>
<b>Cc:</b> <span lang="EN-US"><a href="mailto:starlingx-discuss@lists.starlingx.io"><span lang="EN-US"><span lang="EN-US">starlingx-discuss@lists.starlingx.io</span></span></a></span>; Martinez Monroy, Elio <<span lang="EN-US"><a href="mailto:elio.martinez.monroy@intel.com"><span lang="EN-US"><span lang="EN-US">elio.martinez.monroy@intel.com</span></span></a></span>>;
Xie, Cindy <<span lang="EN-US"><a href="mailto:cindy.xie@intel.com"><span lang="EN-US"><span lang="EN-US">cindy.xie@intel.com</span></span></a></span>>; Cabrales, Ada <<span lang="EN-US"><a href="mailto:ada.cabrales@intel.com"><span lang="EN-US"><span lang="EN-US">ada.cabrales@intel.com</span></span></a></span>>;
Jones, Bruce E <<span lang="EN-US"><a href="mailto:bruce.e.jones@intel.com"><span lang="EN-US"><span lang="EN-US">bruce.e.jones@intel.com</span></span></a></span>>; Qin, Kailun <<span lang="EN-US"><a href="mailto:kailun.qin@intel.com"><span lang="EN-US"><span lang="EN-US">kailun.qin@intel.com</span></span></a></span>>;
Guo, Ruijing <<span lang="EN-US"><a href="mailto:ruijing.guo@intel.com"><span lang="EN-US"><span lang="EN-US">ruijing.guo@intel.com</span></span></a></span>>; Le, Huifeng <<span lang="EN-US"><a href="mailto:huifeng.le@intel.com"><span lang="EN-US"><span lang="EN-US">huifeng.le@intel.com</span></span></a></span>><br>
<b>Subject:</b> RE: [Starlingx-discuss] OVS-DPDK Upgrade Testing<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">Ghada, Matt, Joseph and validation team,</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">Do you know if OVS/DPDK has been tested after cut-over to container? I think the below issue reported by Chenjie might already exist after cut-over to container.</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">Thanks,</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">Forrest</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span><o:p></o:p></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b>From:</b> Xu, Chenjie <br>
<b>Sent:</b> Monday, March 11, 2019 9:17 PM<br>
<b>To:</b> Peters, Matt <<span lang="EN-US"><a href="mailto:Matt.Peters@windriver.com"><span lang="EN-US"><span lang="EN-US">Matt.Peters@windriver.com</span></span></a></span>>; Richard, Joseph <<span lang="EN-US"><a href="mailto:Joseph.Richard@windriver.com"><span lang="EN-US"><span lang="EN-US">Joseph.Richard@windriver.com</span></span></a></span>>;
Khalil, Ghada <<span lang="EN-US"><a href="mailto:Ghada.Khalil@windriver.com"><span lang="EN-US"><span lang="EN-US">Ghada.Khalil@w<span lang="EN-US">indriver.com</span></span></span></a></span>>;
<span lang="EN-US"><a href="mailto:Chris.Winnicki@windriver.com"><span lang="EN-US"><span lang="EN-US">Chris.Winnicki@windriver.com</span></span></a></span>; Lin, Shuicheng <<span lang="EN-US"><a href="mailto:shuicheng.lin@intel.com"><span lang="EN-US"><span lang="EN-US">shuicheng.lin@intel.com</span></span></a></span>><br>
<b>Cc:</b> <span lang="EN-US"><a href="mailto:starlingx-discuss@lists.starlingx.io"><span lang="EN-US"><span lang="EN-US">starlin<span lang="EN-US">gx-discuss@lists.starlingx.io</span></span></span></a></span>; Martinez Monroy, Elio <<span lang="EN-US"><a href="mailto:elio.martinez.monroy@intel.com"><span lang="EN-US"><span lang="EN-US">elio.martinez.monroy@intel.com</span></span></a></span>>;
Zhao, Forrest <<span lang="EN-US"><a href="mailto:forrest.zhao@intel.com"><span lang="EN-US"><span lang="EN-US">forrest.zhao@intel.com</span></span></a></span>>; Xie, Cindy <<span lang="EN-US"><a href="mailto:cindy.xie@intel.com"><span lang="EN-US"><span lang="EN-US">cindy.xie@intel.com</span></span></a></span>>;
Cabrales, Ada <<span lang="EN-US"><a href="mailto:ada.cabrales@intel.com"><span lang="EN-US"><span lang="EN-US">ada.cabrales@intel.com</span></span></a></span>>; Jones, Bruce E <<span lang="EN-US"><a href="mailto:bruce.e.jones@intel.com"><span lang="EN-US"><span lang="EN-US">bruce.e.jones@intel.com</span></span></a></span>>;
Qin, Kailun <<span lang="EN-US"><a href="mailto:kailun.qin@intel.com"><span lang="EN-US"><span lang="EN-US">kailun.qin@intel.com</span></span></a></span>>; Guo, Ruijing <<span lang="EN-US"><a href="mailto:ruijing.guo@intel.com"><span lang="EN-US"><span lang="EN-US">ruijing.guo@intel.com</span></span></a></span>>;
Le, Huifeng <<span lang="EN-US"><a href="mailto:huifeng.le@intel.com"><span lang="EN-US"><span lang="EN-US">huifeng.le@intel.com</span></span></a></span>><br>
<b>Subject:</b> RE: [Starlingx-discuss] OVS-DPDK Upgrade Testing<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1F497D">Hi team,</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1F497D">I have installed StarlingX multimode with new ISO image on 4 bare metals (2 controllers and 2 computes). However the VMs can’t get an IP address. I allocate the IP address which is same as neutron
allocated to the VM, but the VMs still can’t ping each other. The DHCP exists and VMs can’t ping DHCP or router. Some logs have been attached.</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><b><span style="font-size:10.5pt;color:#1F497D">This issue blocks basic functional testing for OVS-DPDK Upgrade and please let me know if you have any ideas.</span></b><span style="font-size:10.5pt;color:#1F497D">
</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1F497D">Best Regards,</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1F497D">Xu, Chenjie</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1F497D"> </span><o:p></o:p></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b>From:</b> Lin, Shuicheng <br>
<b>Sent:</b> Friday, March 8, 2019 9:49 AM<br>
<b>To:</b> Xu, Chenjie <<span lang="EN-US"><a href="mailto:chenjie.xu@intel.com"><span lang="EN-US"><span lang="EN-US">chenjie.xu@intel.com</span></span></a></span>>; Khalil, Ghada <<span lang="EN-US"><a href="mailto:Ghada.Khalil@windriver.com"><span lang="EN-US"><span lang="EN-US">Ghada.Khalil@windriver.com</span></span></a></span>>;
Richard, Joseph <<span lang="EN-US"><a href="mailto:Joseph.Richard@windriver.com"><span lang="EN-US"><span lang="EN-US">Joseph.Richard@windriver.com</span></span></a></span>>; Peters, Matt <<span lang="EN-US"><a href="mailto:Matt.Peters@windriver.com"><span lang="EN-US"><span lang="EN-US">Matt.Peters@wi<span lang="EN-US">ndriver.com</span></span></span></a></span>>;
<span lang="EN-US"><a href="mailto:Chris.Winnicki@windriver.com"><span lang="EN-US"><span lang="EN-US">Chris.Winnicki@windriver.com</span></span></a></span><br>
<b>Cc:</b> <span lang="EN-US"><a href="mailto:starlingx-discuss@lists.starlingx.io"><span lang="EN-US"><span lang="EN-US">starlingx-discuss@lists.starlingx.io</span></span></a></span>; Martinez Monroy, Elio <<span lang="EN-US"><a href="mailto:elio.martinez.monroy@intel.com"><span lang="EN-US"><span lang="EN-US">elio.martinez.monroy@intel.com</span></span></a></span>>;
Zhao, Forrest <<span lang="EN-US"><a href="mailto:forrest.zhao@intel.com"><span lang="EN-US"><span lang="EN-US">forrest.zhao@intel.com</span></span></a></span>>; Xie, Cindy <<span lang="EN-US"><a href="mailto:cindy.xie@intel.com"><span lang="EN-US"><span lang="EN-US">cindy.xie@intel.com</span></span></a></span>>;
Cabrales, Ada <<span lang="EN-US"><a href="mailto:ada.cabrales@intel.com"><span lang="EN-US"><span lang="EN-US">ada.cabrales@intel.com</span></span></a></span>>; Jones, Bruce E <<span lang="EN-US"><a href="mailto:bruce.e.jones@intel.com"><span lang="EN-US"><span lang="EN-US">bruce.e.jones@intel.com</span></span></a></span>>;
Qin, Kailun <<span lang="EN-US"><a href="mailto:kailun.qin@intel.com"><span lang="EN-US"><span lang="EN-US">kailun.qin@intel.com</span></span></a></span>>; Guo, Ruijing <<span lang="EN-US"><a href="mailto:ruijing.guo@intel.com"><span lang="EN-US"><span lang="EN-US">ruijing.guo@intel.com</span></span></a></span>>;
Le, Huifeng <<span lang="EN-US"><a href="mailto:huifeng.le@intel.com"><span lang="EN-US"><span lang="EN-US">huifeng.le@intel.com</span></span></a></span>><br>
<b>Subject:</b> RE: [Starlingx-discuss] OVS-DPDK Upgrade Testing<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">Hi all,</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">I just shared the new test ISO with Chenjie/Elio in another mail thread.</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">It is built from centos76 feature branch with latest master code rebased.</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span><o:p></o:p></p>
<div>
<p class="MsoNormal"><span style="color:#1F497D">Best Regards</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">Shuicheng</span><o:p></o:p></p>
</div>
<p class="MsoNormal"><span style="color:#1F497D"> </span><o:p></o:p></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b>From:</b> Xu, Chenjie <br>
<b>Sent:</b> Wednesday, March 6, 2019 11:07 AM<br>
<b>To:</b> Khalil, Ghada <<span lang="EN-US"><a href="mailto:Ghada.Khalil@windriver.com"><span lang="EN-US"><span lang="EN-US">Ghada.Khalil@windriver.com</span></span></a></span>>; Richard, Joseph <<span lang="EN-US"><a href="mailto:Joseph.Richard@windriver.com"><span lang="EN-US"><span lang="EN-US">Joseph.Richard@windriver.com</span></span></a></span>>;
Peters, Matt <<span lang="EN-US"><a href="mailto:Matt.Peters@windriver.com"><span lang="EN-US"><span lang="EN-US">Matt.Peters@windriver.com</span></span></a></span>>;
<span lang="EN-US"><a href="mailto:Chris.Winnicki@windriver.com"><span lang="EN-US"><span lang="EN-US">Chris.Winnicki@windriver.com</span></span></a></span><br>
<b>Cc:</b> <span lang="EN-US"><a href="mailto:starlingx-discuss@lists.starlingx.io"><span lang="EN-US"><span lang="EN-US">starlingx-discuss@lists.starlingx.io</span></span></a></span>; Lin, Shuicheng <<span lang="EN-US"><a href="mailto:shuicheng.lin@intel.com"><span lang="EN-US"><span lang="EN-US">shuicheng.lin@intel.com</span></span></a></span>>;
Martinez Monroy, Elio <<span lang="EN-US"><a href="mailto:elio.martinez.monroy@intel.com"><span lang="EN-US"><span lang="EN-US">elio.martinez.monroy@intel.com</span></span></a></span>>; Zhao, Forrest <<span lang="EN-US"><a href="mailto:forrest.zhao@intel.com"><span lang="EN-US"><span lang="EN-US">forrest.zhao@intel.com</span></span></a></span>>;
Xie, Cindy <<span lang="EN-US"><a href="mailto:cindy.xie@intel.com"><span lang="EN-US"><span lang="EN-US">cindy.xie@intel.com</span></span></a></span>>; Cabrales, Ada <<span lang="EN-US"><a href="mailto:ada.cabrales@intel.com"><span lang="EN-US"><span lang="EN-US">ada.cabrales@intel.com</span></span></a></span>>;
Jones, Bruce E <<span lang="EN-US"><a href="mailto:bruce.e.jones@intel.com"><span lang="EN-US"><span lang="EN-US">bruce.e.jones@intel.com</span></span></a></span>>; Qin, Kailun <<span lang="EN-US"><a href="mailto:kailun.qin@intel.com"><span lang="EN-US"><span lang="EN-US">kailun.qin@intel.com</span></span></a></span>>;
Guo, Ruijing <<span lang="EN-US"><a href="mailto:ruijing.guo@intel.com"><span lang="EN-US"><span lang="EN-US">ruijing.guo@intel.com</span></span></a></span>>; Le, Huifeng <<span lang="EN-US"><a href="mailto:huifeng.le@intel.com"><span lang="EN-US"><span lang="EN-US">huifeng.le@intel.com</span></span></a></span>><br>
<b>Subject:</b> RE: [Starlingx-discuss] OVS-DPDK Upgrade Testing<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1F497D">Hi Ghada,</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1F497D">That will be very helpful and many thanks to you and Joseph! Looking forward to Chris’s guide.</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1F497D">Best Regards,</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1F497D">Xu, Chenjie</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1F497D"> </span><o:p></o:p></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b>From:</b> Khalil, Ghada [<span lang="EN-US"><a href="mailto:Ghada.Khalil@windriver.com"><span lang="EN-US"><span lang="EN-US">mailto:Ghada.Khalil@windriver.com</span></span></a></span>]
<br>
<b>Sent:</b> Wednesday, March 6, 2019 12:07 AM<br>
<b>To:</b> Xu, Chenjie <<span lang="EN-US"><a href="mailto:chenjie.xu@intel.com"><span lang="EN-US"><span lang="EN-US">chenjie.xu@intel.com</span></span></a></span>>; Richard, Joseph <<span lang="EN-US"><a href="mailto:Joseph.Richard@windriver.com"><span lang="EN-US"><span lang="EN-US">Joseph.Richard@windriver.com</span></span></a></span>>;
Peters, Matt <<span lang="EN-US"><a href="mailto:Matt.Peters@windriver.com"><span lang="EN-US"><span lang="EN-US">Matt.Peters@windriver.com</span></span></a></span>>; Winnicki, Chris <<span lang="EN-US"><a href="mailto:Chris.Winnicki@windriver.com"><span lang="EN-US"><span lang="EN-US">Chris.Winnicki@windriver.com</span></span></a></span>><br>
<b>Cc:</b> <span lang="EN-US"><a href="mailto:starlingx-discuss@lists.starlingx.io"><span lang="EN-US"><span lang="EN-US">starlingx-discus<span lang="EN-US">s@lists.starlingx.io</span></span></span></a></span>; Lin, Shuicheng <<span lang="EN-US"><a href="mailto:shuicheng.lin@intel.com"><span lang="EN-US"><span lang="EN-US">shuicheng.lin@intel.com</span></span></a></span>>;
Martinez Monroy, Elio <<span lang="EN-US"><a href="mailto:elio.martinez.monroy@intel.com"><span lang="EN-US"><span lang="EN-US">elio.martinez.monroy@intel.com</span></span></a></span>>; Zhao, Forrest <<span lang="EN-US"><a href="mailto:forrest.zhao@intel.com"><span lang="EN-US"><span lang="EN-US">forrest.zhao@intel.com</span></span></a></span>>;
Xie, Cindy <<span lang="EN-US"><a href="mailto:cindy.xie@intel.com"><span lang="EN-US"><span lang="EN-US">cindy.xie@intel.com</span></span></a></span>>; Cabrales, Ada <<span lang="EN-US"><a href="mailto:ada.cabrales@intel.com"><span lang="EN-US"><span lang="EN-US">ada.cabrales@intel.com</span></span></a></span>>;
Jones, Bruce E <<span lang="EN-US"><a href="mailto:bruce.e.jones@intel.com"><span lang="EN-US"><span lang="EN-US">bruce.e.jones@intel.com</span></span></a></span>>; Qin, Kailun <<span lang="EN-US"><a href="mailto:kailun.qin@intel.com"><span lang="EN-US"><span lang="EN-US">kailun.qin@intel.com</span></span></a></span>>;
Guo, Ruijing <<span lang="EN-US"><a href="mailto:ruijing.guo@intel.com"><span lang="EN-US"><span lang="EN-US">ruijing.guo@intel.com</span></span></a></span>>; Le, Huifeng <<span lang="EN-US"><a href="mailto:huifeng.le@intel.com"><span lang="EN-US"><span lang="EN-US">huifeng.le@intel.com</span></span></a></span>><br>
<b>Subject:</b> RE: [Starlingx-discuss] OVS-DPDK Upgrade Testing<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">Hi Chenjie,</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">I spoke with Joseph. We don’t believe you need to make any puppet changes to test qos.
</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">Storyboard </span><span lang="EN-US"><a href="https://storyboard.openstack.org/#!/story/2003948"><span lang="EN-US" style="font-size:10.5pt"><span lang="EN-US">https://storyboard.openstack.org/#!/story/2003948</span></span></a></span><span lang="EN-US" style="color:#1F497D">
</span><span style="color:#1F497D">(which you already noted below) removed the STX/WR custom QoS extensions, so StarlingX is now aligned with the upstream neutron behavior.</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">I have added Chris Winnicki to this email thread. He recently verified QoS functionality in StarlingX after the above Story was merged. He can provide you with the steps. Chris is away today, but should be able
to respond tomorrow.</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">Regards,</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">Ghada</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span><o:p></o:p></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b>From:</b> Xu, Chenjie [<span lang="EN-US"><a href="mailto:chenjie.xu@intel.com"><span lang="EN-US"><span lang="EN-US">mailto:chenjie.xu@intel.com</span></span></a></span>]
<br>
<b>Sent:</b> Tuesday, March 05, 2019 8:54 AM<br>
<b>To:</b> Richard, Joseph; Khalil, Ghada; Peters, Matt<br>
<b>Cc:</b> <span lang="EN-US"><a href="mailto:starlingx-discuss@lists.starlingx.io"><span lang="EN-US"><span lang="EN-US">starlingx-discuss@lists.starlingx.io</span></span></a></span>; Lin, Shuicheng; Martinez Monroy, Elio; Zhao, Forrest; Xie, Cindy; Cabrales,
Ada; Jones, Bruce E; Qin, Kailun; Guo, Ruijing; Le, Huifeng<br>
<b>Subject:</b> RE: [Starlingx-discuss] OVS-DPDK Upgrade Testing<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1F497D">Hi Joseph,<br>
About the configuration of QOS, I don’t know which puppet file should be modified to add the following configuration into the file /etc/neutron/plugins/ml2/openvswitch.ini:<br>
extensions = qos</span><o:p></o:p></p>
<p class="MsoNormal"><b><span style="font-size:10.5pt;color:#1F497D">Could you please tell which puppet file I should modify?</span></b><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1F497D">Best Regards,</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1F497D">Xu, Chenjie</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1F497D"> </span><o:p></o:p></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b>From:</b> Xu, Chenjie <br>
<b>Sent:</b> Monday, March 4, 2019 11:39 PM<br>
<b>To:</b> 'Richard, Joseph' <<span lang="EN-US"><a href="mailto:Joseph.Richard@windriver.com"><span lang="EN-US"><span lang="EN-US">Joseph.Richard@windriver.com</span></span></a></span>>; 'Ghada.Khalil@windriver.com' <<span lang="EN-US"><a href="mailto:Ghada.Khalil@windriver.com"><span lang="EN-US"><span lang="EN-US">Ghada.Khalil@windriver.com</span></span></a></span>>;
'Peters, Matt' <<span lang="EN-US"><a href="mailto:Matt.Peters@windriver.com"><span lang="EN-US"><span lang="EN-US">Matt.Peters@windriver<span lang="EN-US">.com</span></span></span></a></span>><br>
<b>Cc:</b> <span lang="EN-US"><a href="mailto:starlingx-discuss@lists.starlingx.io"><span lang="EN-US"><span lang="EN-US">starlingx-discuss@lists.starlingx.io</span></span></a></span>; Lin, Shuicheng <<span lang="EN-US"><a href="mailto:shuicheng.lin@intel.com"><span lang="EN-US"><span lang="EN-US">shuicheng.lin@intel.com</span></span></a></span>>;
Martinez Monroy, Elio <<span lang="EN-US"><a href="mailto:elio.martinez.monroy@intel.com"><span lang="EN-US"><span lang="EN-US">elio.martinez.monroy@intel.com</span></span></a></span>>; Zhao, Forrest <<span lang="EN-US"><a href="mailto:forrest.zhao@intel.com"><span lang="EN-US"><span lang="EN-US">forrest.zhao@intel.com</span></span></a></span>>;
Xie, Cindy <<span lang="EN-US"><a href="mailto:cindy.xie@intel.com"><span lang="EN-US"><span lang="EN-US">cindy.xie@intel.com</span></span></a></span>>; Cabrales, Ada <<span lang="EN-US"><a href="mailto:ada.cabrales@intel.com"><span lang="EN-US"><span lang="EN-US">ada.cabrales@intel.com</span></span></a></span>>;
Jones, Bruce E <<span lang="EN-US"><a href="mailto:bruce.e.jones@intel.com"><span lang="EN-US"><span lang="EN-US">bruce.e.jones@intel.com</span></span></a></span>>; Qin, Kailun <<span lang="EN-US"><a href="mailto:kailun.qin@intel.com"><span lang="EN-US"><span lang="EN-US">kailun.qin@intel.com</span></span></a></span>>;
Guo, Ruijing <<span lang="EN-US"><a href="mailto:ruijing.guo@intel.com"><span lang="EN-US"><span lang="EN-US">ruijing.guo@intel.com</span></span></a></span>>; Le, Huifeng <<span lang="EN-US"><a href="mailto:huifeng.le@intel.com"><span lang="EN-US"><span lang="EN-US">huifeng.le@intel.com</span></span></a></span>><br>
<b>Subject:</b> FW: [Starlingx-discuss] OVS-DPDK Upgrade Testing<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1F497D">Hi Joseph,</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1F497D">For the following bug sent on last Friday:</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:11.0pt;mso-para-margin-left:1.0gd"><span style="font-size:10.5pt;color:#1F497D">system host-port-list compute-1<br>
'pdevice'</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1F497D">I attached the /var/log/sysinv.log of controller-0 and compute-1. According to the logs, the compute-1 is not correctly installed.
<b>Could you please take a look at the logs and provide some comments?</b></span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1F497D">Best Regards,</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1F497D">Xu, Chenjie</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1F497D"> </span><o:p></o:p></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b>From:</b> Xu, Chenjie <br>
<b>Sent:</b> Monday, March 4, 2019 10:08 PM<br>
<b>To:</b> Richard, Joseph <<span lang="EN-US"><a href="mailto:Joseph.Richard@windriver.com"><span lang="EN-US"><span lang="EN-US">Joseph.Richard@windriver.com</span></span></a></span>>;
<span lang="EN-US"><a href="mailto:Ghada.Khalil@windriver.com"><span lang="EN-US"><span lang="EN-US">Ghada.Khalil@windriver.com</span></span></a></span>; Peters, Matt <<span lang="EN-US"><a href="mailto:Matt.Peters@windriver.com"><span lang="EN-US"><span lang="EN-US">Matt.Peters@windriver.com</span></span></a></span>><br>
<b>Cc:</b> <span lang="EN-US"><a href="mailto:starlingx-discuss@lists.starlingx.io"><span lang="EN-US"><span lang="EN-US">starlingx-discuss@lists.<span lang="EN-US">starlingx.io</span></span></span></a></span>; Lin, Shuicheng <<span lang="EN-US"><a href="mailto:shuicheng.lin@intel.com"><span lang="EN-US"><span lang="EN-US">shuicheng.lin@intel.com</span></span></a></span>>;
Martinez Monroy, Elio <<span lang="EN-US"><a href="mailto:elio.martinez.monroy@intel.com"><span lang="EN-US"><span lang="EN-US">elio.martinez.monroy@intel.com</span></span></a></span>>; Zhao, Forrest <<span lang="EN-US"><a href="mailto:forrest.zhao@intel.com"><span lang="EN-US"><span lang="EN-US">forrest.zhao@intel.com</span></span></a></span>>;
Xie, Cindy <<span lang="EN-US"><a href="mailto:cindy.xie@intel.com"><span lang="EN-US"><span lang="EN-US">cindy.xie@intel.com</span></span></a></span>>; Cabrales, Ada <<span lang="EN-US"><a href="mailto:ada.cabrales@intel.com"><span lang="EN-US"><span lang="EN-US">ada.cabrales@intel.com</span></span></a></span>>;
Jones, Bruce E <<span lang="EN-US"><a href="mailto:bruce.e.jones@intel.com"><span lang="EN-US"><span lang="EN-US">bruce.e.jones@intel.com</span></span></a></span>>; Qin, Kailun <<span lang="EN-US"><a href="mailto:kailun.qin@intel.com"><span lang="EN-US"><span lang="EN-US">kailun.qin@intel.com</span></span></a></span>>;
Guo, Ruijing <<span lang="EN-US"><a href="mailto:ruijing.guo@intel.com"><span lang="EN-US"><span lang="EN-US">ruijing.guo@intel.com</span></span></a></span>>; Le, Huifeng <<span lang="EN-US"><a href="mailto:huifeng.le@intel.com"><span lang="EN-US"><span lang="EN-US">huifeng.le@intel.com</span></span></a></span>><br>
<b>Subject:</b> RE: [Starlingx-discuss] OVS-DPDK Upgrade Testing<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1F497D">Hi Joseph,</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1F497D">Thank you for your reply! For now I’m trying to configure QOS in StarlingX by changing puppet to change neutron configuration files.</span>
<span style="font-size:10.5pt;color:#1F497D">But I think this is not a good way to configure QOS in StarlingX.
<b>Could you please provide some guide on how to configure QOS in StarlingX?</b> I noticed that the StarlingX custom QOS extension has been removed. (</span><span lang="EN-US"><a href="https://storyboard.openstack.org/#!/story/2003948"><span lang="EN-US" style="font-size:10.5pt"><span lang="EN-US">https://storyboard.openstack.org/#!/story/2003948</span></span></a></span><span style="font-size:10.5pt;color:#1F497D">)</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1F497D">Best Regards,</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1F497D">Xu, Chenjie</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1F497D"> </span><o:p></o:p></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b>From:</b> Richard, Joseph [<span lang="EN-US"><a href="mailto:Joseph.Richard@windriver.com"><span lang="EN-US"><span lang="EN-US">mailto:Joseph.Richard@windriver.com</span></span></a></span>]
<br>
<b>Sent:</b> Saturday, March 2, 2019 12:41 AM<br>
<b>To:</b> Xu, Chenjie <<span lang="EN-US"><a href="mailto:chenjie.xu@intel.com"><span lang="EN-US"><span lang="EN-US">chenjie.xu@intel.com</span></span></a></span>>; Khalil, Ghada <<span lang="EN-US"><a href="mailto:Ghada.Khalil@windriver.com"><span lang="EN-US"><span lang="EN-US">Ghada.Khalil@windriver.com</span></span></a></span>>;
Peters, Matt <<span lang="EN-US"><a href="mailto:Matt.Peters@windriver.com"><span lang="EN-US"><span lang="EN-US">Matt.Peters@windriver.com</span></span></a></span>><br>
<b>Cc:</b> <span lang="EN-US"><a href="mailto:starlingx-discuss@lists.starlingx.io"><span lang="EN-US"><span lang="EN-US">starlingx-discuss@lists.starlingx.io</span></span></a></span>; Lin, Shuicheng <<span lang="EN-US"><a href="mailto:shuicheng.lin@intel.com"><span lang="EN-US"><span lang="EN-US">shuicheng.lin@intel.com</span></span></a></span>>;
Martinez Monroy, Elio <<span lang="EN-US"><a href="mailto:elio.martinez.monroy@intel.com"><span lang="EN-US"><span lang="EN-US">elio.martinez.monroy@intel.com</span></span></a></span>>; Zhao, Forrest <<span lang="EN-US"><a href="mailto:forrest.zhao@intel.com"><span lang="EN-US"><span lang="EN-US">forrest.z<span lang="EN-US">hao@intel.com</span></span></span></a></span>>;
Xie, Cindy <<span lang="EN-US"><a href="mailto:cindy.xie@intel.com"><span lang="EN-US"><span lang="EN-US">cindy.xie@intel.com</span></span></a></span>>; Cabrales, Ada <<span lang="EN-US"><a href="mailto:ada.cabrales@intel.com"><span lang="EN-US"><span lang="EN-US">ada.cabrales@intel.com</span></span></a></span>>;
Jones, Bruce E <<span lang="EN-US"><a href="mailto:bruce.e.jones@intel.com"><span lang="EN-US"><span lang="EN-US">bruce.e.jones@intel.com</span></span></a></span>>; Qin, Kailun <<span lang="EN-US"><a href="mailto:kailun.qin@intel.com"><span lang="EN-US"><span lang="EN-US">kailun.qin@intel.com</span></span></a></span>>;
Guo, Ruijing <<span lang="EN-US"><a href="mailto:ruijing.guo@intel.com"><span lang="EN-US"><span lang="EN-US">ruijing.guo@intel.com</span></span></a></span>>; Le, Huifeng <<span lang="EN-US"><a href="mailto:huifeng.le@intel.com"><span lang="EN-US"><span lang="EN-US">huifeng.le@intel.com</span></span></a></span>><br>
<b>Subject:</b> RE: [Starlingx-discuss] OVS-DPDK Upgrade Testing<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">1:</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">You need to use sysinv to add the address, not add it using ip commands.</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">usage: system host-addr-add <hostname or id> <interface name or id> <ipv4/ipv6</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D"> address> <prefix length></span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">2:</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">Check in /var/log/sysinv.log for the traceback at that time.</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span><o:p></o:p></p>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma",sans-serif">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma",sans-serif"> Xu, Chenjie [</span><span lang="EN-US"><a href="mailto:chenjie.xu@intel.com"><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma",sans-serif"><span lang="EN-US">mailto:chenjie.xu@intel.com</span></span></a></span><span style="font-size:10.0pt;font-family:"Tahoma",sans-serif">]
<br>
<b>Sent:</b> Friday, March 1, 2019 9:55 AM<br>
<b>To:</b> Richard, Joseph; Khalil, Ghada; Peters, Matt<br>
<b>Cc:</b> </span><span lang="EN-US"><a href="mailto:starlingx-discuss@lists.starlingx.io"><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma",sans-serif"><span lang="EN-US">starlingx-discuss@lists.starlingx.io</span></span></a></span><span style="font-size:10.0pt;font-family:"Tahoma",sans-serif">;
Lin, Shuicheng; Martinez Monroy, Elio; Zhao, Forrest; Xie, Cindy; Cabrales, Ada; Jones, Bruce E; Qin, Kailun; Guo, Ruijing; Le, Huifeng<br>
<b>Subject:</b> [Starlingx-discuss] OVS-DPDK Upgrade Testing</span><o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1F497D">Hi Joseph,</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1F497D">I’m responsible for the Basic Functional Testing for OVS-DPDK Upgrade. I have set up a StarlingX multimode environment (1 controller and 2 computes) using the ISO image
</span><span style="color:#1F497D">for OVS/DPDK upgrade test</span><span style="font-size:10.5pt;color:#1F497D">. I have
<b>2 questions for you</b>:</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1F497D">1. I don’t know how associate a vxlan providernet with compute node.
<b>Could you please provide the steps how to associate a vxlan providernet with compute node?</b> The commands I use are following:<br>
neutron providernet-create tenant_vxlan --type=vxlan --mtu=1400</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:11.0pt;mso-para-margin-left:1.0gd"><span style="font-size:10.5pt;color:#1F497D">neutron providernet-range-create --name tenant_vxlan-a-range1 --range 100-400 --group 224.0.0.1 --ttl 255 tenant_vxlan</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:11.0pt;mso-para-margin-left:1.0gd"><span style="font-size:10.5pt;color:#1F497D">system datanetwork-add tenant_vxlan vxlan --multicast_group 224.0.0.1 --ttl 255 --port_num 4789</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:11.0pt;mso-para-margin-left:1.0gd"><span style="font-size:10.5pt;color:#1F497D">system host-if-modify -p tenant_vxlan -c data compute-0 enp4s0f1</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:11.0pt;mso-para-margin-left:1.0gd"><span style="font-size:10.5pt;color:#1F497D">system host-unlock compute-0</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1F497D">After executing “system host-unlock compute-0”, it will report “Interface enp4s0f1 is associated to VXLAN data network tenant_vxlan which requires an IPv4 address”. But I do allocate a IP address
for enp4s0f1. You can check the attached images. </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1F497D"><br>
2. I have <b>found one bug and will investigate it. If you have any ideas please let me know</b>. After installing the compute-1, I try to list the interface on compute-1 by following command:</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:11.0pt;mso-para-margin-left:1.0gd"><span style="font-size:10.5pt;color:#1F497D">system host-port-list compute-1<br>
'pdevice'</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:10.5pt;text-indent:-10.5pt"><span style="font-size:10.5pt;color:#1F497D">The normal output should be:<br>
system host-port-list compute-0<br>
+--------------------------------------+----------+----------+--------------+--------+-----------+-------------+------------------------------------------------+</span><o:p></o:p></p>
<p class="MsoNormal" style="text-indent:10.5pt"><span style="font-size:10.5pt;color:#1F497D">| uuid | name | type | pci address | device | processor | accelerated | device type |</span><o:p></o:p></p>
<p class="MsoNormal" style="text-indent:10.5pt"><span style="font-size:10.5pt;color:#1F497D">+--------------------------------------+----------+----------+--------------+--------+-----------+-------------+------------------------------------------------+</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:11.0pt;mso-para-margin-left:1.0gd"><span style="font-size:10.5pt;color:#1F497D">| e072d576-90a6-4040-87df-66a913076df0 | eno1 | ethernet | 0000:07:00.0 | 0 | 0 | True | I350 Gigabit Network Connection
|</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:11.0pt;mso-para-margin-left:1.0gd"><span style="font-size:10.5pt;color:#1F497D">| 485b6e35-5b75-4b73-93dc-7deae0fb5c5c | eno2 | ethernet | 0000:07:00.1 | 0 | 0 | True | I350 Gigabit Network Connection
|</span><o:p></o:p></p>
<p class="MsoNormal" style="text-indent:10.5pt"><span style="font-size:10.5pt;color:#1F497D">| 8b448ab6-e9df-4ce7-84a4-256d97f6fd5b | enp4s0f0 | ethernet | 0000:04:00.0 | 0 | 0 | True | 82599ES 10-Gigabit SFI/SFP+ Network Connection |</span><o:p></o:p></p>
<p class="MsoNormal" style="text-indent:10.5pt"><span style="font-size:10.5pt;color:#1F497D">| 6d5b1ce7-ae76-4781-9e51-93bfe8417dc0 | enp4s0f1 | ethernet | 0000:04:00.1 | 0 | 0 | True | 82599ES 10-Gigabit SFI/SFP+ Network Connection |</span><o:p></o:p></p>
<p class="MsoNormal" style="text-indent:10.5pt"><span style="font-size:10.5pt;color:#1F497D">+--------------------------------------+----------+----------+--------------+--------+-----------+-------------+------------------------------------------------+</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1F497D">Best Regards,</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1F497D">Xu, Chenjie</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1F497D"> </span><o:p></o:p></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b>From:</b> Xu, Chenjie <br>
<b>Sent:</b> Friday, March 1, 2019 2:32 PM<br>
<b>To:</b> Lin, Shuicheng <<span lang="EN-US"><a href="mailto:shuicheng.lin@intel.com"><span lang="EN-US"><span lang="EN-US">shuicheng.lin@intel.com</span></span></a></span>><br>
<b>Cc:</b> Martinez Monroy, Elio <<span lang="EN-US"><a href="mailto:elio.martinez.monroy@intel.com"><span lang="EN-US"><span lang="EN-US">elio.martinez.monroy@intel.com</span></span></a></span>>; Zhao, Forrest <<span lang="EN-US"><a href="mailto:forrest.zhao@intel.com"><span lang="EN-US"><span lang="EN-US">forrest.zhao@intel.com</span></span></a></span>>;
Xie, Cindy <<span lang="EN-US"><a href="mailto:cindy.xie@intel.com"><span lang="EN-US"><span lang="EN-US">cindy.xie@intel.com</span></span></a></span>>; Cabrales, Ada <<span lang="EN-US"><a href="mailto:ada.cabrales@intel.com"><span lang="EN-US"><span lang="EN-US">ada.cabrales@intel.com</span></span></a></span>>;
Jones, Bruce E <<span lang="EN-US"><a href="mailto:bruce.e.jones@intel.com"><span lang="EN-US"><span lang="EN-US">bruce.e.jones@intel.com</span></span></a></span>>; Qin, Kailun <<span lang="EN-US"><a href="mailto:kailun.qin@intel.com"><span lang="EN-US"><span lang="EN-US">kailun.qin@intel.com</span></span></a></span>>;
Guo, Ruijing <<span lang="EN-US"><a href="mailto:ruijing.guo@intel.com"><span lang="EN-US"><span lang="EN-US">ruijing.guo@intel.com</span></span></a></span>>; Le, Huifeng <<span lang="EN-US"><a href="mailto:huifeng.le@intel.com"><span lang="EN-US"><span lang="EN-US">huifeng.le@intel.com</span></span></a></span>><br>
<b>Subject:</b> RE: ISO image for OVS/DPDK upgrade<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1F497D">Hi Shuicheng,</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1F497D">Thank you for your information! I will start to do the Basic Functional Test.</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1F497D">Best Regards,</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1F497D">Xu, Chenjie</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:#1F497D"> </span><o:p></o:p></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b>From:</b> Lin, Shuicheng <br>
<b>Sent:</b> Friday, March 1, 2019 2:03 PM<br>
<b>To:</b> Xu, Chenjie <<span lang="EN-US"><a href="mailto:chenjie.xu@intel.com"><span lang="EN-US"><span lang="EN-US">chenjie.xu@intel.com</span></span></a></span>>; Martinez Monroy, Elio <<span lang="EN-US"><a href="mailto:elio.martinez.monroy@intel.com"><span lang="EN-US"><span lang="EN-US">elio.martinez.monroy@intel.com</span></span></a></span>><br>
<b>Cc:</b> Zhao, Forrest <<span lang="EN-US"><a href="mailto:forrest.zhao@intel.com"><span lang="EN-US"><span lang="EN-US">forrest.zhao@intel.com</span></span></a></span>>; Xie, Cindy <<span lang="EN-US"><a href="mailto:cindy.xie@intel.com"><span lang="EN-US"><span lang="EN-US">cindy.xie@intel.c<span lang="EN-US">om</span></span></span></a></span>>;
Cabrales, Ada <<span lang="EN-US"><a href="mailto:ada.cabrales@intel.com"><span lang="EN-US"><span lang="EN-US">ada.cabrales@intel.com</span></span></a></span>>; Jones, Bruce E <<span lang="EN-US"><a href="mailto:bruce.e.jones@intel.com"><span lang="EN-US"><span lang="EN-US">bruce.e.jones@intel.com</span></span></a></span>>;
Qin, Kailun <<span lang="EN-US"><a href="mailto:kailun.qin@intel.com"><span lang="EN-US"><span lang="EN-US">kailun.qin@intel.com</span></span></a></span>>; Guo, Ruijing <<span lang="EN-US"><a href="mailto:ruijing.guo@intel.com"><span lang="EN-US"><span lang="EN-US">ruijing.guo@intel.com</span></span></a></span>><br>
<b>Subject:</b> ISO image for OVS/DPDK upgrade<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">Hi Chenjie/Elio,</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">Here is the ISO image for OVS/DPDK upgrade test.</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">I have done basic deploy test, and could ping VM from each other successfully.</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">Please help have a try with it, and notify me if there is any issue.</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">It is built with recent code of centos76 feature branch.</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US"><a href="mailto:ec@10.239.154.150:/home/ec/bootimage_ovs.iso"><span lang="EN-US"><span lang="EN-US">ec@10.239.154.150:/home/ec/bootimage_ovs.iso</span></span></a></span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">scp passwd: 1</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">Here is the patch list for the upgrade in case you want to have a check:</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US"><a href="https://review.openstack.org/640244"><span lang="EN-US"><span lang="EN-US">https://review.openstack.org/640244</span></span></a></span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US"><a href="https://review.openstack.org/640245"><span lang="EN-US"><span lang="EN-US">https://review.openstack.org/640245</span></span></a></span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">Best Regards</span><o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">Shuicheng</span><o:p></o:p></p>
</div>
</body>
</html>