[Starlingx-discuss] [ Test ] meeting notes - 03/31/2020

Jones, Bruce E bruce.e.jones at intel.com
Fri Apr 3 16:43:52 UTC 2020


Yang, Nic - thank you for the update on this.  Can we please work toward one common Sanity test suite and setup?  I know Nic's team is working on updating our automation to Pytest which I assume is the long pole - but I'd like to see us take on the goal of fully common test cases and setups for Sanity.  Can we do that for 4.0?

          bruej

From: Liu, Yang (YOW) [mailto:yang.liu at windriver.com]
Sent: Friday, April 3, 2020 6:53 AM
To: Jascanu, Nicolae <nicolae.jascanu at intel.com>; Jones, Bruce E <bruce.e.jones at intel.com>
Cc: 'starlingx-discuss at lists.starlingx.io' <starlingx-discuss at lists.starlingx.io>
Subject: RE: [ Test ] meeting notes - 03/31/2020

There was a comparison done by previous Intel test team, and on a high level, sanity test cases were mostly the same between us with 1 extra test in WR sanity suite.

The main difference may be our configurations and verifications.

There are couple of things I know of that might cause the differences in results:

*         OpenStack neutron configurations:

o   In our config, the external traffic goes through neutron routers to reach the vms, and connectivity verification is included in most openstack sanity test we run. Perhaps that is something Nic's team can look into.

*         Couple of the platform sanity issues we reported were more frequently seen on IPv6 systems (Christopher originally reported in a different LP). This is another item Nic can consider.

*         1868567 is a stx-monitor test case we recently added to the sanity suite to test system applications, this test is available in stx-test. Nic could pick up this test if needed.

*         1869739 is a configuration alarm issue without obvious system impact, this can only be found if the overall system health including alarms were verified explicitly in every test. Nic could also pick up this one easily because it's already included in pytest framework.

Hope that helps.

Regards,
Yang


From: Jascanu, Nicolae [mailto:nicolae.jascanu at intel.com]
Sent: April-03-20 7:49 AM
To: Jones, Bruce E; Liu, Yang (YOW)
Cc: 'starlingx-discuss at lists.starlingx.io'
Subject: RE: [ Test ] meeting notes - 03/31/2020

Hi Bruce,
Hi Yang,
Looks like the bugs 1863795 and 1859645  are duplicates of 1856078. This was raised by Cristopher and each time we encountered, we've stated this in the report. Seems that we need an upgrade to Helm V3.
For the rest of bugs, we need to replicate them after the daily finishes. I guess that Yang is executing also other sanity tests.

Regards,
Nicolae Jascanu, Ph.D.
TSD Engineering Manager

[intel-logo]

Internet Of Things Group
Galati, Romania

From: Jones, Bruce E <bruce.e.jones at intel.com<mailto:bruce.e.jones at intel.com>>
Sent: Thursday, April 2, 2020 22:53
To: Liu, Yang (YOW) <yang.liu at windriver.com<mailto:yang.liu at windriver.com>>; Jascanu, Nicolae <nicolae.jascanu at intel.com<mailto:nicolae.jascanu at intel.com>>
Cc: 'starlingx-discuss at lists.starlingx.io' <starlingx-discuss at lists.starlingx.io<mailto:starlingx-discuss at lists.starlingx.io>>
Subject: RE: [ Test ] meeting notes - 03/31/2020

Yang, I'm puzzled by the different Sanity results.  Do you and Nic know why the Intel team is seeing Green Sanity runs, and the WR team is seeing multiple issues?  Are we running different tests?  Different images?

Meanwhile, congratulations on pulling the Feature and Regression plan together.  It's looking good, thanks to great work by you and the teams!

            bruej

From: Liu, Yang (YOW) [mailto:yang.liu at windriver.com]
Sent: Tuesday, March 31, 2020 9:09 AM
To: 'starlingx-discuss at lists.starlingx.io' <starlingx-discuss at lists.starlingx.io<mailto:starlingx-discuss at lists.starlingx.io>>
Subject: [Starlingx-discuss] [ Test ] meeting notes - 03/31/2020

Agenda/meeting notes are posted here:

https://etherpad.openstack.org/p/stx-test


Agenda for 03/31/2020

Attendees: Yang, Mihail, Nicolae, George, Cosmin, Sharath, Claudiu, Bruce, ChrisW, Wendy

 1. Sanity Status:
*         Sanity issues reported by WR:

  *   1855474  -  OpenStack pods were not recovered after force reboot active controller

  *   1859645 - platform-integ-apps fail to apply initially due to could not find the requested resource (get configmaps)

  *   https://bugs.launchpad.net/starlingx/+bug/1863795  -  "helm list" failed by network is unreachable after host-unlock

  *   https://bugs.launchpad.net/starlingx/+bug/1868567  -  CMD application-delete failed by Timeout while waiting on RPC response

  *   https://bugs.launchpad.net/starlingx/+bug/1869739  -  250.001 controller Configuration is out-of-date alarm not cleared after system installation
*         Sanity issues reported by Intel:

  *   Green

  *   Private registry used by Intel: some container images changed and not picked up in private registry - now resolved

  *   May look into adding ipv6 configs for the long term

 2. stx4.0 testing
*         Release plan:

  *   https://docs.google.com/spreadsheets/d/1a93wt0XO0_JvajnPzQwnqFkXfdDysKVnHpbrEc17_yg/edit#gid=0

  *   Kata: intend to start early. target test complete by 05/08

  *   Rook: intend to start early. target test complete by 05/22

  *   Fault containerization: target test complete by 05/29

  *   Centos8: no server with TPM, mellanox cards; integrity may not be testable

  *   TSN support in kata container: target test complete by 05/29

  *   OpenStack clients and platform services: eng build will be available in a week. testing: 04-13 - 05-04

  *   OpenStack upversion:
*  automated and manual test update needed, may be time consuming
*  target 06-05

  *   Layered build is tested daily with issues being found and resolved
*  image did not build - fixed
*  03-31 build still failed, but went further
*         Targeted feature completion date: 06-08

  *   Risk: OpenStack upversion to Ussuris coming in late, and requires large amount of testing
*         Targeted regression completion date: 06-15  (pushing out 1 week)

  *   Propose no openstack regression between 06-08 to 06-15

3. Unified santiy

  *   Intel will bring in robot sanity as is into pytest framework

  *   For feature test cases porting from robot framework in future, should look into organizing/porting by feature, and porting only the missing ones

  *   Need to look into external traffic to/from openstack vm via neutron router in future to enable neutron testing in Intel environment

4. Opens
*         Servers/test environment setup within Nic's team:

  *   In progress - cabling, configuring ports, etc

  *   Will know more by end of week.

  *   Risk reduced compared to last week.



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.starlingx.io/pipermail/starlingx-discuss/attachments/20200403/47fc63c0/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 3923 bytes
Desc: image001.png
URL: <http://lists.starlingx.io/pipermail/starlingx-discuss/attachments/20200403/47fc63c0/attachment-0001.png>


More information about the Starlingx-discuss mailing list