Hi All

As discussed on weekly test meeting we decide to consolidate in only one repo all the test for StarlingX. The main Idea is to have a common place to contribute with different test cases. Here is a draft document where you can add your comments and we can discuss on 02/05 Test Meeting to have an official repo structure.  

https://docs.google.com/document/d/1TuBZHSv0zerLp17PqIxk8HKXh0XsQJ_VAolO5rMGiac/edit?usp=sharing

Regards,

José

From: Liu, Yang [mailto:yang.liu@windriver.com]
Sent: Monday, January 28, 2019 8:11 AM
To: Perez Carranza, Jose <jose.perez.carranza@intel.com>; Waheed, Numan <Numan.Waheed@windriver.com>; starlingx-discuss@lists.starlingx.io; Cabrales, Ada <ada.cabrales@intel.com>
Subject: RE: [Test] Proposal for Test Repository

Hi Jose,

I think 3 separate repos will be easier from maintenance and usage perspective.

- Manual/robot/pytest have very different installation/usage/contribution requirements, it would be much easier to maintain them with standard repo structure, i.e., package requirements and README in root directory of a repo.

- User won’t be forced to download everything if they are only interested in contributing to one of them.

- Easier to separate code reviews.

BR,

Yang

From: Perez Carranza, Jose [mailto:jose.perez.carranza@intel.com]
Sent: January-28-19 8:50 AM
To: Waheed, Numan; starlingx-discuss@lists.starlingx.io; Cabrales, Ada
Subject: Re: [Starlingx-discuss] [Test] Proposal for Test Repository

Hi Newman

Some comments inline

Regards,

José

From: Waheed, Numan [mailto:Numan.Waheed@windriver.com]
Sent: Friday, January 25, 2019 3:18 PM
To: starlingx-discuss@lists.starlingx.io; Cabrales, Ada <ada.cabrales@intel.com>
Subject: [Starlingx-discuss] [Test] Proposal for Test Repository

Hi Ada and Christopher,

After investigating existing openstack test projects, I think we should have:

* separate repo for test cases that are independent from each other. Since they require different instructions for almost everything (install, usage, rules, etc.)

* one repo for one auto project. Subrepos are not recommended due to complicity involved in updating common libraries(keywords, fixtures, etc.) and their usages(test cases).

I think having 3 different repositories can become hard to maintain, also could be confusing for people who is contributing to the project.

Maybe we can consolidate all in one repo divided by directories , below an example:

-> stx-test

     - manual-tests

     - automated-tests

         - robot-suite

           - pytets-suite

Thus I suggest 3 different repositories:

- repo for manual test cases

- Is this section going to have actual scripts or just Test Specifications (preconditions, steps and expected result) in a plain text?

- repo for robot test cases

- repo for pytest test cases

Inside automated test repository, I would suggest the following structure:

README.rst

LICENSE

setup.py

tox.ini                        # pep8, py27, etc

requirements.txt               # project package requirements

consts/…                      # directory for various constants modules

keywords/…                     # directory for helper modules

testfixtures/                  # directory for commonly used test fixtures modules

testcases/cli/mtc/…            # directory for mtc test cases that is mainly using cli

testcases/cli/heat/…  

testcases/cli/nova/…

testcases/cli/networking/…

testcases/cli/security/…

testcases/cli/storage/…

testcases/cli/sysinv/…

testcases/rest/…               # directory for restAPI test cases

testcases/horizon/…            # directory for horizon test cases

testcases/system_test/…        # directory for complex system test scenarios

Agree with this structure inside of the automated test suites.

Thanks,

Numan