[Starlingx-discuss] [Test] Proposal for Test Repository

[Perez Carranza, Jose]

Hi All

As discussed on weekly test meeting we decide to consolidate in only one repo all the test for StarlingX. The main Idea is to have a common place to contribute with different test cases. Here is a draft document where you can add your comments and we can discuss on 02/05 Test Meeting to have an official repo structure.

https://docs.google.com/document/d/1TuBZHSv0zerLp17PqIxk8HKXh0XsQJ_VAolO5rMGiac/edit?usp=sharing

Regards,
José

From: Liu, Yang [mailto:yang.liu at windriver.com]
Sent: Monday, January 28, 2019 8:11 AM
To: Perez Carranza, Jose <jose.perez.carranza at intel.com>; Waheed, Numan <Numan.Waheed at windriver.com>; starlingx-discuss at lists.starlingx.io; Cabrales, Ada <ada.cabrales at intel.com>
Subject: RE: [Test] Proposal for Test Repository

Hi Jose,

I think 3 separate repos will be easier from maintenance and usage perspective.
- Manual/robot/pytest have very different installation/usage/contribution requirements, it would be much easier to maintain them with standard repo structure, i.e., package requirements and README in root directory of a repo.
- User won't be forced to download everything if they are only interested in contributing to one of them.
- Easier to separate code reviews.

BR,
Yang

From: Perez Carranza, Jose [mailto:jose.perez.carranza at intel.com]
Sent: January-28-19 8:50 AM
To: Waheed, Numan; starlingx-discuss at lists.starlingx.io<mailto:starlingx-discuss at lists.starlingx.io>; Cabrales, Ada
Subject: Re: [Starlingx-discuss] [Test] Proposal for Test Repository

Hi Newman

Some comments inline

Regards,
José

From: Waheed, Numan [mailto:Numan.Waheed at windriver.com]
Sent: Friday, January 25, 2019 3:18 PM
To: starlingx-discuss at lists.starlingx.io<mailto:starlingx-discuss at lists.starlingx.io>; Cabrales, Ada <ada.cabrales at intel.com<mailto:ada.cabrales at intel.com>>
Subject: [Starlingx-discuss] [Test] Proposal for Test Repository

Hi Ada and Christopher,

After investigating existing openstack test projects, I think we should have:
* separate repo for test cases that are independent from each other. Since they require different instructions for almost everything (install, usage, rules, etc.)
* one repo for one auto project. Subrepos are not recommended due to complicity involved in updating common libraries(keywords, fixtures, etc.) and their usages(test cases).

I think having 3 different repositories can become hard to maintain, also could be confusing for people who is contributing to the project.

Maybe we can consolidate all in one repo divided by directories , below an example:
-> stx-test
     - manual-tests
     - automated-tests
         - robot-suite
           - pytets-suite

Thus I suggest 3 different repositories:
- repo for manual test cases
- Is this section going to have actual scripts or just Test Specifications (preconditions, steps and expected result) in a plain text?
- repo for robot test cases
- repo for pytest test cases


Inside automated test repository, I would suggest the following structure:

README.rst
LICENSE
setup.py
tox.ini                        # pep8, py27, etc
requirements.txt               # project package requirements
consts/...                      # directory for various constants modules
keywords/...                     # directory for helper modules
testfixtures/                  # directory for commonly used test fixtures modules
testcases/cli/mtc/...            # directory for mtc test cases that is mainly using cli
testcases/cli/heat/...
testcases/cli/nova/...
testcases/cli/networking/...
testcases/cli/security/...
testcases/cli/storage/...
testcases/cli/sysinv/...
testcases/rest/...               # directory for restAPI test cases
testcases/horizon/...            # directory for horizon test cases
testcases/system_test/...        # directory for complex system test scenarios

Agree with this structure inside of the automated test suites.

Thanks,

Numan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.starlingx.io/pipermail/starlingx-discuss/attachments/20190204/58b531ec/attachment-0001.html>