[Starlingx-discuss] [Test] Proposal for Test Repository
Hi Ada and Christopher,
After investigating existing openstack test projects, I think we should have: * separate repo for test cases that are independent from each other. Since they require different instructions for almost everything (install, usage, rules, etc.) * one repo for one auto project. Subrepos are not recommended due to complicity involved in updating common libraries(keywords, fixtures, etc.) and their usages(test cases).
Thus I suggest 3 different repositories: - repo for manual test cases - repo for robot test cases - repo for pytest test cases
Inside automated test repository, I would suggest the following structure:
README.rst LICENSE setup.py tox.ini # pep8, py27, etc requirements.txt # project package requirements consts/... # directory for various constants modules keywords/... # directory for helper modules testfixtures/ # directory for commonly used test fixtures modules testcases/cli/mtc/... # directory for mtc test cases that is mainly using cli testcases/cli/heat/... testcases/cli/nova/... testcases/cli/networking/... testcases/cli/security/... testcases/cli/storage/... testcases/cli/sysinv/... testcases/rest/... # directory for restAPI test cases testcases/horizon/... # directory for horizon test cases testcases/system_test/... # directory for complex system test scenarios
Thanks,
Numan
Hi Newman
Some comments inline
Regards, José
From: Waheed, Numan [mailto:Numan.Waheed@windriver.com] Sent: Friday, January 25, 2019 3:18 PM To: starlingx-discuss@lists.starlingx.io; Cabrales, Ada ada.cabrales@intel.com Subject: [Starlingx-discuss] [Test] Proposal for Test Repository
Hi Ada and Christopher,
After investigating existing openstack test projects, I think we should have: * separate repo for test cases that are independent from each other. Since they require different instructions for almost everything (install, usage, rules, etc.) * one repo for one auto project. Subrepos are not recommended due to complicity involved in updating common libraries(keywords, fixtures, etc.) and their usages(test cases).
I think having 3 different repositories can become hard to maintain, also could be confusing for people who is contributing to the project.
Maybe we can consolidate all in one repo divided by directories , below an example: -> stx-test - manual-tests - automated-tests - robot-suite - pytets-suite
Thus I suggest 3 different repositories: - repo for manual test cases - Is this section going to have actual scripts or just Test Specifications (preconditions, steps and expected result) in a plain text? - repo for robot test cases - repo for pytest test cases
Inside automated test repository, I would suggest the following structure:
README.rst LICENSE setup.py tox.ini # pep8, py27, etc requirements.txt # project package requirements consts/... # directory for various constants modules keywords/... # directory for helper modules testfixtures/ # directory for commonly used test fixtures modules testcases/cli/mtc/... # directory for mtc test cases that is mainly using cli testcases/cli/heat/... testcases/cli/nova/... testcases/cli/networking/... testcases/cli/security/... testcases/cli/storage/... testcases/cli/sysinv/... testcases/rest/... # directory for restAPI test cases testcases/horizon/... # directory for horizon test cases testcases/system_test/... # directory for complex system test scenarios
Agree with this structure inside of the automated test suites.
Thanks,
Numan
Hi Jose,
I think 3 separate repos will be easier from maintenance and usage perspective. - Manual/robot/pytest have very different installation/usage/contribution requirements, it would be much easier to maintain them with standard repo structure, i.e., package requirements and README in root directory of a repo. - User won't be forced to download everything if they are only interested in contributing to one of them. - Easier to separate code reviews.
BR, Yang
From: Perez Carranza, Jose [mailto:jose.perez.carranza@intel.com] Sent: January-28-19 8:50 AM To: Waheed, Numan; starlingx-discuss@lists.starlingx.io; Cabrales, Ada Subject: Re: [Starlingx-discuss] [Test] Proposal for Test Repository
Hi Newman
Some comments inline
Regards, José
From: Waheed, Numan [mailto:Numan.Waheed@windriver.com] Sent: Friday, January 25, 2019 3:18 PM To: starlingx-discuss@lists.starlingx.io; Cabrales, Ada ada.cabrales@intel.com Subject: [Starlingx-discuss] [Test] Proposal for Test Repository
Hi Ada and Christopher,
After investigating existing openstack test projects, I think we should have: * separate repo for test cases that are independent from each other. Since they require different instructions for almost everything (install, usage, rules, etc.) * one repo for one auto project. Subrepos are not recommended due to complicity involved in updating common libraries(keywords, fixtures, etc.) and their usages(test cases).
I think having 3 different repositories can become hard to maintain, also could be confusing for people who is contributing to the project.
Maybe we can consolidate all in one repo divided by directories , below an example: -> stx-test - manual-tests - automated-tests - robot-suite - pytets-suite
Thus I suggest 3 different repositories: - repo for manual test cases - Is this section going to have actual scripts or just Test Specifications (preconditions, steps and expected result) in a plain text? - repo for robot test cases - repo for pytest test cases
Inside automated test repository, I would suggest the following structure:
README.rst LICENSE setup.py tox.ini # pep8, py27, etc requirements.txt # project package requirements consts/... # directory for various constants modules keywords/... # directory for helper modules testfixtures/ # directory for commonly used test fixtures modules testcases/cli/mtc/... # directory for mtc test cases that is mainly using cli testcases/cli/heat/... testcases/cli/nova/... testcases/cli/networking/... testcases/cli/security/... testcases/cli/storage/... testcases/cli/sysinv/... testcases/rest/... # directory for restAPI test cases testcases/horizon/... # directory for horizon test cases testcases/system_test/... # directory for complex system test scenarios
Agree with this structure inside of the automated test suites.
Thanks,
Numan
Hi All
As discussed on weekly test meeting we decide to consolidate in only one repo all the test for StarlingX. The main Idea is to have a common place to contribute with different test cases. Here is a draft document where you can add your comments and we can discuss on 02/05 Test Meeting to have an official repo structure.
https://docs.google.com/document/d/1TuBZHSv0zerLp17PqIxk8HKXh0XsQJ_VAolO5rMG...
Regards, José
From: Liu, Yang [mailto:yang.liu@windriver.com] Sent: Monday, January 28, 2019 8:11 AM To: Perez Carranza, Jose jose.perez.carranza@intel.com; Waheed, Numan Numan.Waheed@windriver.com; starlingx-discuss@lists.starlingx.io; Cabrales, Ada ada.cabrales@intel.com Subject: RE: [Test] Proposal for Test Repository
Hi Jose,
I think 3 separate repos will be easier from maintenance and usage perspective. - Manual/robot/pytest have very different installation/usage/contribution requirements, it would be much easier to maintain them with standard repo structure, i.e., package requirements and README in root directory of a repo. - User won't be forced to download everything if they are only interested in contributing to one of them. - Easier to separate code reviews.
BR, Yang
From: Perez Carranza, Jose [mailto:jose.perez.carranza@intel.com] Sent: January-28-19 8:50 AM To: Waheed, Numan; starlingx-discuss@lists.starlingx.iomailto:starlingx-discuss@lists.starlingx.io; Cabrales, Ada Subject: Re: [Starlingx-discuss] [Test] Proposal for Test Repository
Hi Newman
Some comments inline
Regards, José
From: Waheed, Numan [mailto:Numan.Waheed@windriver.com] Sent: Friday, January 25, 2019 3:18 PM To: starlingx-discuss@lists.starlingx.iomailto:starlingx-discuss@lists.starlingx.io; Cabrales, Ada <ada.cabrales@intel.commailto:ada.cabrales@intel.com> Subject: [Starlingx-discuss] [Test] Proposal for Test Repository
Hi Ada and Christopher,
After investigating existing openstack test projects, I think we should have: * separate repo for test cases that are independent from each other. Since they require different instructions for almost everything (install, usage, rules, etc.) * one repo for one auto project. Subrepos are not recommended due to complicity involved in updating common libraries(keywords, fixtures, etc.) and their usages(test cases).
I think having 3 different repositories can become hard to maintain, also could be confusing for people who is contributing to the project.
Maybe we can consolidate all in one repo divided by directories , below an example: -> stx-test - manual-tests - automated-tests - robot-suite - pytets-suite
Thus I suggest 3 different repositories: - repo for manual test cases - Is this section going to have actual scripts or just Test Specifications (preconditions, steps and expected result) in a plain text? - repo for robot test cases - repo for pytest test cases
Inside automated test repository, I would suggest the following structure:
README.rst LICENSE setup.py tox.ini # pep8, py27, etc requirements.txt # project package requirements consts/... # directory for various constants modules keywords/... # directory for helper modules testfixtures/ # directory for commonly used test fixtures modules testcases/cli/mtc/... # directory for mtc test cases that is mainly using cli testcases/cli/heat/... testcases/cli/nova/... testcases/cli/networking/... testcases/cli/security/... testcases/cli/storage/... testcases/cli/sysinv/... testcases/rest/... # directory for restAPI test cases testcases/horizon/... # directory for horizon test cases testcases/system_test/... # directory for complex system test scenarios
Agree with this structure inside of the automated test suites.
Thanks,
Numan
participants (3)
-
Liu, Yang
-
Perez Carranza, Jose
-
Waheed, Numan