Hi Chenjie, The latest RFE looks good to me. Regards, Matt From: "Xu, Chenjie" <chenjie.xu@intel.com> Date: Monday, November 12, 2018 at 1:23 AM To: "Peters, Matt" <Matt.Peters@windriver.com> Cc: "starlingx-discuss@lists.starlingx.io" <starlingx-discuss@lists.starlingx.io>, Allain Legacy <Allain.Legacy@windriver.com> Subject: RE: [Starlingx-discuss] Analysis of patch 9f926a5 for StartlingX upstreaming Hi Matt, The RFE has been updated and is attached. Could you please help review and comment? Best Regards, Xu, Chenjie From: Peters, Matt [mailto:Matt.Peters@windriver.com] Sent: Friday, November 9, 2018 9:22 PM To: Xu, Chenjie <chenjie.xu@intel.com>; Legacy, Allain <Allain.Legacy@windriver.com> Cc: starlingx-discuss@lists.starlingx.io Subject: Re: [Starlingx-discuss] Analysis of patch 9f926a5 for StartlingX upstreaming Hi Chenjie, The RFE looks good. The use cases are clear and detailed. I only have a few minor review comments (see attached). Regards, Matt From: "Xu, Chenjie" <chenjie.xu@intel.com<mailto:chenjie.xu@intel.com>> Date: Thursday, November 1, 2018 at 4:28 AM To: "Peters, Matt" <Matt.Peters@windriver.com<mailto:Matt.Peters@windriver.com>>, Allain Legacy <Allain.Legacy@windriver.com<mailto:Allain.Legacy@windriver.com>> Cc: "starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>" <starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>> Subject: [Starlingx-discuss] Analysis of patch 9f926a5 for StartlingX upstreaming Hi Matt/Allain, We analyze the patch 9f926a5 related to l2pop. An RFE “Add l2pop support for floating ip resources” has been written and is attached. The test case is provided by Allain. Could you please help to review and comment? Thanks very much! Best Regards, Xu, Chenjie

The change that is being reviewed here was originally a part of a larger commit (9f926a5d253<http://stash.wrs.com/projects/CGCS/repos/github.com.openstack.neutron/commits/9f926a5d2539b0b0451c50053244a8fa6c43bf3c>). They should be implemented together or at least tested together. I seem to remember that there was information missing in case 1 that prevented a proper FDB notification from being generated. Please retest your scenarios and capture the input parameters to add_fdb_entries(), remove_fdb_entries(), and update_fdb_entries() in neutron/plugins/ml2/drivers/l2pop/rpc.py:L2populationAgentNotifyAPI to be sure that expected notifications are published. Regards, Allain Allain Legacy, Software Developer, Wind River direct 613.270.2279 fax 613.492.7870 skype allain.legacy 350 Terry Fox Drive, Suite 200, Ottawa, Ontario, K2K 2W5 [WIND]<http://www.windriver.com/> From: Xu, Chenjie [mailto:chenjie.xu@intel.com] Sent: Friday, December 07, 2018 3:42 AM To: Peters, Matt Cc: starlingx-discuss@lists.starlingx.io Subject: Re: [Starlingx-discuss] Analysis of patch 9f926a5 for StartlingX upstreaming Hi Matt, Ryan Tidwell comments on this patch and he thinks that AFTER_DELETE notification can be used to trigger l2pop. https://review.openstack.org/#/c/611261/ https://review.openstack.org/#/c/611261/4/neutron/db/l3_db.py From the comment in the following line: https://github.com/starlingx-staging/stx-neutron/blob/master/neutron/service... It seems that the router_id and port_id in AFTER_DELETE notification are None. As a result of that, the last_known_router_id and last_fixed_port_id should be used to construct FDB entries which are used to remove FDBs on each host. However, I print the notification in the following 2 cases: Case-1: 1) Allocate floating ip fip-1 2) Associate fip-1 with vm-1 3) Delete fip-1 Case-2: 1) Allocate floating ip fip-1 2) Associate fip-1 with vm-1 3) Disassociate fip-1 with vm-1 4) Delete fip-1 The notification for case1 and case 2 are attached. router_id and port_id are not None in case-1 and are None in case-2. Thus in case-1, AFTER_DELETE notification can be used. In case-2, FDB will be removed by step 3, thus no need to remove again. Based on the above analysis, I think we can use AFTER_DELETE notification. Could you please comment and review? Best Regards, Xu, Chenjie From: Peters, Matt [mailto:Matt.Peters@windriver.com] Sent: Monday, November 12, 2018 11:19 PM To: Xu, Chenjie <chenjie.xu@intel.com<mailto:chenjie.xu@intel.com>> Cc: starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>; Legacy, Allain <Allain.Legacy@windriver.com<mailto:Allain.Legacy@windriver.com>> Subject: Re: [Starlingx-discuss] Analysis of patch 9f926a5 for StartlingX upstreaming Hi Chenjie, The latest RFE looks good to me. Regards, Matt From: "Xu, Chenjie" <chenjie.xu@intel.com<mailto:chenjie.xu@intel.com>> Date: Monday, November 12, 2018 at 1:23 AM To: "Peters, Matt" <Matt.Peters@windriver.com<mailto:Matt.Peters@windriver.com>> Cc: "starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>" <starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>>, Allain Legacy <Allain.Legacy@windriver.com<mailto:Allain.Legacy@windriver.com>> Subject: RE: [Starlingx-discuss] Analysis of patch 9f926a5 for StartlingX upstreaming Hi Matt, The RFE has been updated and is attached. Could you please help review and comment? Best Regards, Xu, Chenjie From: Peters, Matt [mailto:Matt.Peters@windriver.com] Sent: Friday, November 9, 2018 9:22 PM To: Xu, Chenjie <chenjie.xu@intel.com<mailto:chenjie.xu@intel.com>>; Legacy, Allain <Allain.Legacy@windriver.com<mailto:Allain.Legacy@windriver.com>> Cc: starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Subject: Re: [Starlingx-discuss] Analysis of patch 9f926a5 for StartlingX upstreaming Hi Chenjie, The RFE looks good. The use cases are clear and detailed. I only have a few minor review comments (see attached). Regards, Matt From: "Xu, Chenjie" <chenjie.xu@intel.com<mailto:chenjie.xu@intel.com>> Date: Thursday, November 1, 2018 at 4:28 AM To: "Peters, Matt" <Matt.Peters@windriver.com<mailto:Matt.Peters@windriver.com>>, Allain Legacy <Allain.Legacy@windriver.com<mailto:Allain.Legacy@windriver.com>> Cc: "starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>" <starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>> Subject: [Starlingx-discuss] Analysis of patch 9f926a5 for StartlingX upstreaming Hi Matt/Allain, We analyze the patch 9f926a5 related to l2pop. An RFE “Add l2pop support for floating ip resources” has been written and is attached. The test case is provided by Allain. Could you please help to review and comment? Thanks very much! Best Regards, Xu, Chenjie

An FDB entry for a Floating IP resource would only be required on nodes with ports attached to the related external network. Normally that means only the network nodes that are hosting virtual routers attached to those external networks rather than compute nodes. However, there are less frequently used scenarios that involve customers launching VM instances directly on external networks and in those scenarios the external network will be instantiated in some compute nodes as well as the network nodes. In your test scenario, the external network will only be present on the network node and therefore only the agent running on the network node will be capable of processing any related FDB entries. Regards, Allain Allain Legacy, Software Developer, Wind River direct 613.270.2279 fax 613.492.7870 skype allain.legacy 350 Terry Fox Drive, Suite 200, Ottawa, Ontario, K2K 2W5 [WIND]<http://www.windriver.com/> From: Xu, Chenjie [mailto:chenjie.xu@intel.com] Sent: Friday, December 14, 2018 3:24 AM To: Legacy, Allain Cc: Peters, Matt; starlingx-discuss@lists.starlingx.io Subject: FW: [Starlingx-discuss] Analysis of patch 9f926a5 for StartlingX upstreaming Hi Allain, As you suggested, I’m testing the 2 changes together. However I find that the fdb for floatingip can’t be installed on the br-tun of the compute node. Because the network-id in fdb is not in the LocalVlanManager. The details are below and could you please help review and comment? The environment: latest DevStack with 1 controller node and 1 compute node The network topology is below: [cid:image010.jpg@01D49382.52015E80] Steps: 1. Create an external network external-net neutron net-create external-net --router:external True --provider:network_type vxlan 2. Create a subnet on external-net neutron subnet-create external-net 192.168.25.0/24 --name external-subnet --allocation-pool start=192.168.25.200,end=192.168.25.250 3. Create an internal network, create a subnet on net4 neutron net-create net4 neutron subnet-create net4 192.168.2.0/24 --name subnet4 4. Create a router, set router gateway as external-net, add subnet4 to router neutron router-create router neutron router-gateway-set $router-id $external-net-id neutron router-interface-add $router-id $subnet4-id 5. Create a VM vm-1 on net4 (vm-1 runs on compute node) 6. Allocate floating IP FIP-1 on external-net through horizon 7. Associate FIP-1 with vm-1 through horizon The fdb is below: [cid:image011.jpg@01D49382.52015E80] The OVS agent on the compute node receives the FDB: [cid:image012.jpg@01D49382.52015E80] In OVS agent, LocalVlanManager is used to map tunnel ids or vlan ids to internal vlans: https://github.com/openstack/neutron/blob/master/neutron/plugins/ml2/drivers... After OVS agent receiving the FDB, it will try to get LocalVlanMapping from LocalVlanMannager: https://github.com/openstack/neutron/blob/master/neutron/plugins/ml2/drivers... https://github.com/openstack/neutron/blob/master/neutron/plugins/ml2/drivers... However the network-id f2ebf82a-e788-4456-8516-c95b12f91d49 is not in the LocalVlanManager. Thus the fdb can’t be installed on br-tun. The mapping in the LocalVlanManager in the OVS agent which is on the compute node is below: [cid:image013.jpg@01D49382.52015E80] Analysis: The vm-1 is created on internal network net4 . When creating vm-1, a port in the net4 will be bound to vm-1. Thus the network-id can be added to the LocalVlanManager. The network id for floating IP is used in FDB but the corresponding network is an external network. And this external network’s network id is not in the LocalVlanManager in the OVS agent which is on the compute node. Best Regard, Xu, Chenjie From: Legacy, Allain [mailto:Allain.Legacy@windriver.com] Sent: Friday, December 7, 2018 10:20 PM To: Xu, Chenjie <chenjie.xu@intel.com<mailto:chenjie.xu@intel.com>>; Peters, Matt <Matt.Peters@windriver.com<mailto:Matt.Peters@windriver.com>> Cc: starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Subject: RE: [Starlingx-discuss] Analysis of patch 9f926a5 for StartlingX upstreaming The change that is being reviewed here was originally a part of a larger commit (9f926a5d253<http://stash.wrs.com/projects/CGCS/repos/github.com.openstack.neutron/commits/9f926a5d2539b0b0451c50053244a8fa6c43bf3c>). They should be implemented together or at least tested together. I seem to remember that there was information missing in case 1 that prevented a proper FDB notification from being generated. Please retest your scenarios and capture the input parameters to add_fdb_entries(), remove_fdb_entries(), and update_fdb_entries() in neutron/plugins/ml2/drivers/l2pop/rpc.py:L2populationAgentNotifyAPI to be sure that expected notifications are published. Regards, Allain Allain Legacy, Software Developer, Wind River direct 613.270.2279 fax 613.492.7870 skype allain.legacy 350 Terry Fox Drive, Suite 200, Ottawa, Ontario, K2K 2W5 [WIND]<http://www.windriver.com/> From: Xu, Chenjie [mailto:chenjie.xu@intel.com] Sent: Friday, December 07, 2018 3:42 AM To: Peters, Matt Cc: starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Subject: Re: [Starlingx-discuss] Analysis of patch 9f926a5 for StartlingX upstreaming Hi Matt, Ryan Tidwell comments on this patch and he thinks that AFTER_DELETE notification can be used to trigger l2pop. https://review.openstack.org/#/c/611261/ https://review.openstack.org/#/c/611261/4/neutron/db/l3_db.py From the comment in the following line: https://github.com/starlingx-staging/stx-neutron/blob/master/neutron/service... It seems that the router_id and port_id in AFTER_DELETE notification are None. As a result of that, the last_known_router_id and last_fixed_port_id should be used to construct FDB entries which are used to remove FDBs on each host. However, I print the notification in the following 2 cases: Case-1: 1) Allocate floating ip fip-1 2) Associate fip-1 with vm-1 3) Delete fip-1 Case-2: 1) Allocate floating ip fip-1 2) Associate fip-1 with vm-1 3) Disassociate fip-1 with vm-1 4) Delete fip-1 The notification for case1 and case 2 are attached. router_id and port_id are not None in case-1 and are None in case-2. Thus in case-1, AFTER_DELETE notification can be used. In case-2, FDB will be removed by step 3, thus no need to remove again. Based on the above analysis, I think we can use AFTER_DELETE notification. Could you please comment and review? Best Regards, Xu, Chenjie From: Peters, Matt [mailto:Matt.Peters@windriver.com] Sent: Monday, November 12, 2018 11:19 PM To: Xu, Chenjie <chenjie.xu@intel.com<mailto:chenjie.xu@intel.com>> Cc: starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>; Legacy, Allain <Allain.Legacy@windriver.com<mailto:Allain.Legacy@windriver.com>> Subject: Re: [Starlingx-discuss] Analysis of patch 9f926a5 for StartlingX upstreaming Hi Chenjie, The latest RFE looks good to me. Regards, Matt From: "Xu, Chenjie" <chenjie.xu@intel.com<mailto:chenjie.xu@intel.com>> Date: Monday, November 12, 2018 at 1:23 AM To: "Peters, Matt" <Matt.Peters@windriver.com<mailto:Matt.Peters@windriver.com>> Cc: "starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>" <starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>>, Allain Legacy <Allain.Legacy@windriver.com<mailto:Allain.Legacy@windriver.com>> Subject: RE: [Starlingx-discuss] Analysis of patch 9f926a5 for StartlingX upstreaming Hi Matt, The RFE has been updated and is attached. Could you please help review and comment? Best Regards, Xu, Chenjie From: Peters, Matt [mailto:Matt.Peters@windriver.com] Sent: Friday, November 9, 2018 9:22 PM To: Xu, Chenjie <chenjie.xu@intel.com<mailto:chenjie.xu@intel.com>>; Legacy, Allain <Allain.Legacy@windriver.com<mailto:Allain.Legacy@windriver.com>> Cc: starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Subject: Re: [Starlingx-discuss] Analysis of patch 9f926a5 for StartlingX upstreaming Hi Chenjie, The RFE looks good. The use cases are clear and detailed. I only have a few minor review comments (see attached). Regards, Matt From: "Xu, Chenjie" <chenjie.xu@intel.com<mailto:chenjie.xu@intel.com>> Date: Thursday, November 1, 2018 at 4:28 AM To: "Peters, Matt" <Matt.Peters@windriver.com<mailto:Matt.Peters@windriver.com>>, Allain Legacy <Allain.Legacy@windriver.com<mailto:Allain.Legacy@windriver.com>> Cc: "starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>" <starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>> Subject: [Starlingx-discuss] Analysis of patch 9f926a5 for StartlingX upstreaming Hi Matt/Allain, We analyze the patch 9f926a5 related to l2pop. An RFE “Add l2pop support for floating ip resources” has been written and is attached. The test case is provided by Allain. Could you please help to review and comment? Thanks very much! Best Regards, Xu, Chenjie

Sorry, I think I confused things by mentioning the scenario of launching VM instances directly on an external network. The original scenario that was being discussed was two different VM instances, on two different tenant networks, communicating via their respective FIP address over the same external tenant network. In that scenario, it is the respective virtual routers of both VM instances that will be sending out ARP requests for the destination FIP addresses. It is those broadcast ARP requests that need to be eliminated by using updated L2POP FDB information. Therefore, the FDB updates need to be generated properly at the neutron-server and received by the neutron-layer2-agent on the node which is hosting the virtual routers; not necessarily the nodes hosting the VM instances. In the following diagram, that means that the FDB on network node 1 and 2 need to be updated with information for FIP1 and FIP2. The FDB on compute node 1 and 2 do not need to be updated because there are no ports directly on the external network on those nodes. FIP1 FIP2 + + | | +-------+ +------+ | | +------+ +-------+ | | | | | | | | | | | VM1 +---network1--+ R1 +-+-ext-network-+-+ R2 +----network2-----+ VM2 | | | | | | | | | +---+---+ +---+--+ +---+--+ +---+---+ | | | | | | | | v v v v compute network network compute node 1 node 1 node 2 node 2 The other scenario that I mentioned, which was to launch VM instances directly on the external network, was only to illustrate that it is possible for a VM to communicate directly for another server’s FIP rather than having to go through a virtual router. In this type of scenario, the FDB of the node hosting the VM instance that resides directly on the external network needs to be updated whenever the FIP changes state. In the following diagram, that means that the FDB on compute2 needs to be updated with information for FIP1, and the FDB on network node 1 needs to be updated with information for VM2’s port on the external network. The FDB on compute node 1 does not need to be updated for VM2’s port information because there are no ports on compute node 1 that are directly attached to the external network. FIP1 + | +-------+ +------+ | +-------+ | | | | | | | | VM1 +---network1--+ R1 +-+-ext-network---+ VM2 | | | | | | | +---+---+ +---+--+ +---+---+ | | | | | | v v v compute network compute node 1 node 1 node 2 Allain Allain Legacy, Software Developer, Wind River direct 613.270.2279 fax 613.492.7870 skype allain.legacy 350 Terry Fox Drive, Suite 200, Ottawa, Ontario, K2K 2W5 [WIND]<http://www.windriver.com/> From: Xu, Chenjie [mailto:chenjie.xu@intel.com] Sent: Monday, December 17, 2018 5:35 AM To: Legacy, Allain Cc: Peters, Matt; starlingx-discuss@lists.starlingx.io Subject: RE: [Starlingx-discuss] Analysis of patch 9f926a5 for StartlingX upstreaming Hi Allain, Thank you for your advice! By following your suggestion, the FDB for floating IP can be installed on compute node. However there is a problem within the use case for RFE “Add l2pop support for floating IP resources”. The use case can be summarized as two VM instances, residing on different networks, communicate via their respective FIP addresses. The flows for floating IP on compute node: [cid:image002.jpg@01D495EB.7889B190] To use the flows for floating IP, the VM on compute node needs to have the same dl_vlan (local VLAN ID) which is 1. According to your previous email, the VM must be launched on the external network. Thus for the use case, the two VM resides on different external networks where the floating IPs for two VMs are. Thus another VM’s dl_vlan won’t be 1 and the flows for floating IP can’t be matched. Based on the above analysis, the two VM instances, residing on different networks, can’t communicate via their respective FIP addresses. Could you please help review and comment? Best Regards, Xu, Chenjie From: Legacy, Allain [mailto:Allain.Legacy@windriver.com] Sent: Friday, December 14, 2018 8:55 PM To: Xu, Chenjie <chenjie.xu@intel.com<mailto:chenjie.xu@intel.com>> Cc: Peters, Matt <Matt.Peters@windriver.com<mailto:Matt.Peters@windriver.com>>; starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Subject: RE: [Starlingx-discuss] Analysis of patch 9f926a5 for StartlingX upstreaming An FDB entry for a Floating IP resource would only be required on nodes with ports attached to the related external network. Normally that means only the network nodes that are hosting virtual routers attached to those external networks rather than compute nodes. However, there are less frequently used scenarios that involve customers launching VM instances directly on external networks and in those scenarios the external network will be instantiated in some compute nodes as well as the network nodes. In your test scenario, the external network will only be present on the network node and therefore only the agent running on the network node will be capable of processing any related FDB entries. Regards, Allain Allain Legacy, Software Developer, Wind River direct 613.270.2279 fax 613.492.7870 skype allain.legacy 350 Terry Fox Drive, Suite 200, Ottawa, Ontario, K2K 2W5 [WIND]<http://www.windriver.com/> From: Xu, Chenjie [mailto:chenjie.xu@intel.com] Sent: Friday, December 14, 2018 3:24 AM To: Legacy, Allain Cc: Peters, Matt; starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Subject: FW: [Starlingx-discuss] Analysis of patch 9f926a5 for StartlingX upstreaming Hi Allain, As you suggested, I’m testing the 2 changes together. However I find that the fdb for floatingip can’t be installed on the br-tun of the compute node. Because the network-id in fdb is not in the LocalVlanManager. The details are below and could you please help review and comment? The environment: latest DevStack with 1 controller node and 1 compute node The network topology is below: [cid:image002.jpg@01D49632.CEB88F40] Steps: 1. Create an external network external-net neutron net-create external-net --router:external True --provider:network_type vxlan 2. Create a subnet on external-net neutron subnet-create external-net 192.168.25.0/24 --name external-subnet --allocation-pool start=192.168.25.200,end=192.168.25.250 3. Create an internal network, create a subnet on net4 neutron net-create net4 neutron subnet-create net4 192.168.2.0/24 --name subnet4 4. Create a router, set router gateway as external-net, add subnet4 to router neutron router-create router neutron router-gateway-set $router-id $external-net-id neutron router-interface-add $router-id $subnet4-id 5. Create a VM vm-1 on net4 (vm-1 runs on compute node) 6. Allocate floating IP FIP-1 on external-net through horizon 7. Associate FIP-1 with vm-1 through horizon The fdb is below: [cid:image003.jpg@01D49632.CEB88F40] The OVS agent on the compute node receives the FDB: [cid:image004.jpg@01D49632.CEB88F40] In OVS agent, LocalVlanManager is used to map tunnel ids or vlan ids to internal vlans: https://github.com/openstack/neutron/blob/master/neutron/plugins/ml2/drivers... After OVS agent receiving the FDB, it will try to get LocalVlanMapping from LocalVlanMannager: https://github.com/openstack/neutron/blob/master/neutron/plugins/ml2/drivers... https://github.com/openstack/neutron/blob/master/neutron/plugins/ml2/drivers... However the network-id f2ebf82a-e788-4456-8516-c95b12f91d49 is not in the LocalVlanManager. Thus the fdb can’t be installed on br-tun. The mapping in the LocalVlanManager in the OVS agent which is on the compute node is below: [cid:image005.jpg@01D49632.CEB88F40] Analysis: The vm-1 is created on internal network net4 . When creating vm-1, a port in the net4 will be bound to vm-1. Thus the network-id can be added to the LocalVlanManager. The network id for floating IP is used in FDB but the corresponding network is an external network. And this external network’s network id is not in the LocalVlanManager in the OVS agent which is on the compute node. Best Regard, Xu, Chenjie From: Legacy, Allain [mailto:Allain.Legacy@windriver.com] Sent: Friday, December 7, 2018 10:20 PM To: Xu, Chenjie <chenjie.xu@intel.com<mailto:chenjie.xu@intel.com>>; Peters, Matt <Matt.Peters@windriver.com<mailto:Matt.Peters@windriver.com>> Cc: starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Subject: RE: [Starlingx-discuss] Analysis of patch 9f926a5 for StartlingX upstreaming The change that is being reviewed here was originally a part of a larger commit (9f926a5d253<http://stash.wrs.com/projects/CGCS/repos/github.com.openstack.neutron/commits/9f926a5d2539b0b0451c50053244a8fa6c43bf3c>). They should be implemented together or at least tested together. I seem to remember that there was information missing in case 1 that prevented a proper FDB notification from being generated. Please retest your scenarios and capture the input parameters to add_fdb_entries(), remove_fdb_entries(), and update_fdb_entries() in neutron/plugins/ml2/drivers/l2pop/rpc.py:L2populationAgentNotifyAPI to be sure that expected notifications are published. Regards, Allain Allain Legacy, Software Developer, Wind River direct 613.270.2279 fax 613.492.7870 skype allain.legacy 350 Terry Fox Drive, Suite 200, Ottawa, Ontario, K2K 2W5 [WIND]<http://www.windriver.com/> From: Xu, Chenjie [mailto:chenjie.xu@intel.com] Sent: Friday, December 07, 2018 3:42 AM To: Peters, Matt Cc: starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Subject: Re: [Starlingx-discuss] Analysis of patch 9f926a5 for StartlingX upstreaming Hi Matt, Ryan Tidwell comments on this patch and he thinks that AFTER_DELETE notification can be used to trigger l2pop. https://review.openstack.org/#/c/611261/ https://review.openstack.org/#/c/611261/4/neutron/db/l3_db.py From the comment in the following line: https://github.com/starlingx-staging/stx-neutron/blob/master/neutron/service... It seems that the router_id and port_id in AFTER_DELETE notification are None. As a result of that, the last_known_router_id and last_fixed_port_id should be used to construct FDB entries which are used to remove FDBs on each host. However, I print the notification in the following 2 cases: Case-1: 1) Allocate floating ip fip-1 2) Associate fip-1 with vm-1 3) Delete fip-1 Case-2: 1) Allocate floating ip fip-1 2) Associate fip-1 with vm-1 3) Disassociate fip-1 with vm-1 4) Delete fip-1 The notification for case1 and case 2 are attached. router_id and port_id are not None in case-1 and are None in case-2. Thus in case-1, AFTER_DELETE notification can be used. In case-2, FDB will be removed by step 3, thus no need to remove again. Based on the above analysis, I think we can use AFTER_DELETE notification. Could you please comment and review? Best Regards, Xu, Chenjie From: Peters, Matt [mailto:Matt.Peters@windriver.com] Sent: Monday, November 12, 2018 11:19 PM To: Xu, Chenjie <chenjie.xu@intel.com<mailto:chenjie.xu@intel.com>> Cc: starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>; Legacy, Allain <Allain.Legacy@windriver.com<mailto:Allain.Legacy@windriver.com>> Subject: Re: [Starlingx-discuss] Analysis of patch 9f926a5 for StartlingX upstreaming Hi Chenjie, The latest RFE looks good to me. Regards, Matt From: "Xu, Chenjie" <chenjie.xu@intel.com<mailto:chenjie.xu@intel.com>> Date: Monday, November 12, 2018 at 1:23 AM To: "Peters, Matt" <Matt.Peters@windriver.com<mailto:Matt.Peters@windriver.com>> Cc: "starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>" <starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>>, Allain Legacy <Allain.Legacy@windriver.com<mailto:Allain.Legacy@windriver.com>> Subject: RE: [Starlingx-discuss] Analysis of patch 9f926a5 for StartlingX upstreaming Hi Matt, The RFE has been updated and is attached. Could you please help review and comment? Best Regards, Xu, Chenjie From: Peters, Matt [mailto:Matt.Peters@windriver.com] Sent: Friday, November 9, 2018 9:22 PM To: Xu, Chenjie <chenjie.xu@intel.com<mailto:chenjie.xu@intel.com>>; Legacy, Allain <Allain.Legacy@windriver.com<mailto:Allain.Legacy@windriver.com>> Cc: starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Subject: Re: [Starlingx-discuss] Analysis of patch 9f926a5 for StartlingX upstreaming Hi Chenjie, The RFE looks good. The use cases are clear and detailed. I only have a few minor review comments (see attached). Regards, Matt From: "Xu, Chenjie" <chenjie.xu@intel.com<mailto:chenjie.xu@intel.com>> Date: Thursday, November 1, 2018 at 4:28 AM To: "Peters, Matt" <Matt.Peters@windriver.com<mailto:Matt.Peters@windriver.com>>, Allain Legacy <Allain.Legacy@windriver.com<mailto:Allain.Legacy@windriver.com>> Cc: "starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>" <starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>> Subject: [Starlingx-discuss] Analysis of patch 9f926a5 for StartlingX upstreaming Hi Matt/Allain, We analyze the patch 9f926a5 related to l2pop. An RFE “Add l2pop support for floating ip resources” has been written and is attached. The test case is provided by Allain. Could you please help to review and comment? Thanks very much! Best Regards, Xu, Chenjie