Hi Allain,

Thank you for your advice! By following your suggestion, the FDB for floating IP can be installed on compute node. However there is a problem within the use case for RFE “Add l2pop support for floating IP resources”. The use case can be summarized as two VM instances, residing on different networks, communicate via their respective FIP addresses. The flows for floating IP on compute node:

To use the flows for floating IP, the VM on compute node needs to have the same dl_vlan (local VLAN ID) which is 1. According to your previous email, the VM must be launched on the external network. Thus for the use case, the two VM resides on different external networks where the floating IPs for two VMs are. Thus another VM’s dl_vlan won’t be 1 and the flows for floating IP can’t be matched.

Based on the above analysis, the two VM instances, residing on different networks, can’t communicate via their respective FIP addresses. Could you please help review and comment?

Best Regards,

Xu, Chenjie

From: Legacy, Allain [mailto:Allain.Legacy@windriver.com]
Sent: Friday, December 14, 2018 8:55 PM
To: Xu, Chenjie <chenjie.xu@intel.com>
Cc: Peters, Matt <Matt.Peters@windriver.com>; starlingx-discuss@lists.starlingx.io
Subject: RE: [Starlingx-discuss] Analysis of patch 9f926a5 for StartlingX upstreaming

An FDB entry for a Floating IP resource would only be required on nodes with ports attached to the related external network.   Normally that means only the network nodes that are hosting virtual routers attached to those external networks rather than compute nodes.  However, there are less frequently used scenarios that involve customers launching VM instances directly on external networks and in those scenarios the external network will be instantiated in some compute nodes as well as the network nodes. 

In your test scenario, the external network will only be present on the network node and therefore only the agent running on the network node will be capable of processing any related FDB entries. 

Regards,

Allain

Allain Legacy, Software Developer, Wind River
direct 613.270.2279  fax 613.492.7870 skype allain.legacy

350 Terry Fox Drive, Suite 200, Ottawa, Ontario, K2K 2W5

From: Xu, Chenjie [mailto:chenjie.xu@intel.com]
Sent: Friday, December 14, 2018 3:24 AM
To: Legacy, Allain
Cc: Peters, Matt; starlingx-discuss@lists.starlingx.io
Subject: FW: [Starlingx-discuss] Analysis of patch 9f926a5 for StartlingX upstreaming

Hi Allain,

As you suggested, I’m testing the 2 changes together. However I find that the fdb for floatingip can’t be installed on the br-tun of the compute node. Because the network-id in fdb is not in the LocalVlanManager. The details are below and could you please help review and comment?

The environment: latest DevStack with 1 controller node and 1 compute node

The network topology is below:

Steps:

1. Create an external network external-net

neutron net-create external-net  --router:external True  --provider:network_type vxlan

2. Create a subnet on external-net

neutron subnet-create external-net 192.168.25.0/24 --name external-subnet --allocation-pool start=192.168.25.200,end=192.168.25.250

3. Create an internal network, create a subnet on net4

neutron net-create net4

neutron subnet-create net4 192.168.2.0/24 --name subnet4

4. Create a router, set router gateway as external-net, add subnet4 to router

neutron router-create router

neutron router-gateway-set $router-id $external-net-id

neutron router-interface-add $router-id $subnet4-id

5. Create a VM vm-1 on net4 (vm-1 runs on compute node)

6. Allocate floating IP FIP-1 on external-net through horizon

7. Associate FIP-1 with vm-1 through horizon

The fdb is below:

The OVS agent on the compute node receives the FDB:

In OVS agent, LocalVlanManager is used to map tunnel ids or vlan ids to internal vlans:

https://github.com/openstack/neutron/blob/master/neutron/plugins/ml2/drivers/openvswitch/agent/vlanmanager.py

After OVS agent receiving the FDB, it will try to get LocalVlanMapping from LocalVlanMannager:

https://github.com/openstack/neutron/blob/master/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py#L558

https://github.com/openstack/neutron/blob/master/neutron/plugins/ml2/drivers/l2pop/rpc_manager/l2population_rpc.py#L237

However the network-id f2ebf82a-e788-4456-8516-c95b12f91d49 is not in the LocalVlanManager. Thus the fdb can’t be installed on br-tun. The mapping in the LocalVlanManager in the OVS agent which is on the compute node is below:

Analysis:

The vm-1 is created on internal network net4 . When creating vm-1, a port in the net4 will be bound to vm-1. Thus the network-id can be added to the LocalVlanManager. The network id for floating IP is used in FDB but the corresponding network is an external network. And this external network’s network id is not in the LocalVlanManager in the OVS agent which is on the compute node.

Best Regard,

Xu, Chenjie

From: Legacy, Allain [mailto:Allain.Legacy@windriver.com]
Sent: Friday, December 7, 2018 10:20 PM
To: Xu, Chenjie <chenjie.xu@intel.com>; Peters, Matt <Matt.Peters@windriver.com>
Cc: starlingx-discuss@lists.starlingx.io
Subject: RE: [Starlingx-discuss] Analysis of patch 9f926a5 for StartlingX upstreaming

The change that is being reviewed here was originally a part of a larger commit (9f926a5d253).   They should be implemented together or at least tested together.   I seem to remember that there was information missing in case 1 that prevented a proper FDB notification from being generated.

Please retest your scenarios and capture the input parameters to add_fdb_entries(), remove_fdb_entries(), and update_fdb_entries() in neutron/plugins/ml2/drivers/l2pop/rpc.py:L2populationAgentNotifyAPI to be sure that expected notifications are published.

Regards,

Allain

Allain Legacy, Software Developer, Wind River
direct 613.270.2279  fax 613.492.7870 skype allain.legacy

350 Terry Fox Drive, Suite 200, Ottawa, Ontario, K2K 2W5

From: Xu, Chenjie [mailto:chenjie.xu@intel.com]
Sent: Friday, December 07, 2018 3:42 AM
To: Peters, Matt
Cc: starlingx-discuss@lists.starlingx.io
Subject: Re: [Starlingx-discuss] Analysis of patch 9f926a5 for StartlingX upstreaming

Hi Matt,

Ryan Tidwell comments on this patch and he thinks that AFTER_DELETE notification can be used to trigger l2pop.

https://review.openstack.org/#/c/611261/

https://review.openstack.org/#/c/611261/4/neutron/db/l3_db.py

From the comment in the following line:

https://github.com/starlingx-staging/stx-neutron/blob/master/neutron/services/l3_router/service_providers/l2pop.py#L276

It seems that  the router_id and port_id in AFTER_DELETE notification are None. As a result of that, the last_known_router_id and last_fixed_port_id should be used to construct FDB entries which are used to remove FDBs on each host. However, I print the notification in the following 2 cases:

Case-1:

1) Allocate floating ip fip-1

2) Associate fip-1 with vm-1

3) Delete fip-1

Case-2:

1) Allocate floating ip fip-1

2) Associate fip-1 with vm-1

3) Disassociate fip-1 with vm-1

4) Delete fip-1

The notification for case1 and case 2 are attached. router_id and port_id are not None in case-1 and are None in case-2. Thus in case-1, AFTER_DELETE notification can be used. In case-2, FDB will be removed by step 3, thus no need to remove again.

Based on the above analysis, I think we can use AFTER_DELETE notification. Could you please comment and review?

Best Regards,

Xu, Chenjie

From: Peters, Matt [mailto:Matt.Peters@windriver.com]
Sent: Monday, November 12, 2018 11:19 PM
To: Xu, Chenjie <chenjie.xu@intel.com>
Cc: starlingx-discuss@lists.starlingx.io; Legacy, Allain <Allain.Legacy@windriver.com>
Subject: Re: [Starlingx-discuss] Analysis of patch 9f926a5 for StartlingX upstreaming

Hi Chenjie,

The latest RFE looks good to me.

Regards, Matt

From: "Xu, Chenjie" <chenjie.xu@intel.com>
Date: Monday, November 12, 2018 at 1:23 AM
To: "Peters, Matt" <Matt.Peters@windriver.com>
Cc: "starlingx-discuss@lists.starlingx.io" <starlingx-discuss@lists.starlingx.io>, Allain Legacy <Allain.Legacy@windriver.com>
Subject: RE: [Starlingx-discuss] Analysis of patch 9f926a5 for StartlingX upstreaming

Hi Matt,

The RFE has been updated and is attached. Could you please help review and comment?

Best Regards,

Xu, Chenjie

From: Peters, Matt [mailto:Matt.Peters@windriver.com]
Sent: Friday, November 9, 2018 9:22 PM
To: Xu, Chenjie <chenjie.xu@intel.com>; Legacy, Allain <Allain.Legacy@windriver.com>
Cc: starlingx-discuss@lists.starlingx.io
Subject: Re: [Starlingx-discuss] Analysis of patch 9f926a5 for StartlingX upstreaming

Hi Chenjie,

The RFE looks good.  The use cases are clear and detailed.

I only have a few minor review comments (see attached).

Regards, Matt

From: "Xu, Chenjie" <chenjie.xu@intel.com>
Date: Thursday, November 1, 2018 at 4:28 AM
To: "Peters, Matt" <Matt.Peters@windriver.com>, Allain Legacy <Allain.Legacy@windriver.com>
Cc: "starlingx-discuss@lists.starlingx.io" <starlingx-discuss@lists.starlingx.io>
Subject: [Starlingx-discuss] Analysis of patch 9f926a5 for StartlingX upstreaming

Hi Matt/Allain,

We analyze the patch 9f926a5 related to l2pop. An RFE “Add l2pop support for floating ip resources” has been written and is attached. The test case is provided by Allain. Could you please help to review and comment? Thanks very much!

Best Regards,

Xu, Chenjie