[Starlingx-discuss] [docs] Setting up the docs.starlingx.io website
I've started the process of getting the docs.starlingx.io website set up, one of the things required is defining the Zuul publish jobs as this is how the content gets from the build job out to the webserver. I'd like to confirm some things and ask for a bit of information: 1. The stx-docs repo will contain most of the docs.starlingx.io site? 2. The stx-specs repo also needs to be published, should it go to docs.starlingx.io/specs? 3. Release notes also need to be published, should they go in docs.starlingx.io/releasenotes/<project>? 4. Does anyone feel like there is a need to not use a Sphinx-generated page as the root page? Given this is a Friday afternoon before a holiday weekend (for me anyway) I am proceeding with default answers of 1=yes and 4=no to get the process rolling, that can be changed and the rest addressed in follow-ups. dt -- Dean Troyer dtroyer@gmail.com
I've started the process of getting the docs.starlingx.io website set up, one of the things required is defining the Zuul publish jobs as this is how the content gets from the build job out to the webserver.
Thanks Dean
I'd like to confirm some things and ask for a bit of information: 1. The stx-docs repo will contain most of the docs.starlingx.io site?
Also it might include a section to document the existing APIs from OpenStack projects modified by StarlingX.
3. Release notes also need to be published, should they go in docs.starlingx.io/releasenotes/<project>?
Can we please include API Reference documentation? How about under docs.starlingx.io/api-ref/<project>?
4. Does anyone feel like there is a need to not use a Sphinx-generated page as the root page?
Given this is a Friday afternoon before a holiday weekend (for me anyway) I am proceeding with default answers of 1=yes and 4=no to get the process rolling, that can be changed and the rest addressed in follow-ups.
Is this related to generating our documentation using tox -e [releasenotes / api-ref / doc] ?
On Mon, Sep 3, 2018 at 9:43 AM, Arce Moreno, Abraham <abraham.arce.moreno@intel.com> wrote:
I'd like to confirm some things and ask for a bit of information: 1. The stx-docs repo will contain most of the docs.starlingx.io site?
Also it might include a section to document the existing APIs from OpenStack projects modified by StarlingX.
api-ref is generally in-repo with its code....
3. Release notes also need to be published, should they go in docs.starlingx.io/releasenotes/<project>?
Can we please include API Reference documentation? How about under docs.starlingx.io/api-ref/<project>?
Sure.
Is this related to generating our documentation using tox -e [releasenotes / api-ref / doc] ?
Yes, those tox jobs are what generates the documents and the publish jobs are what puts it on the web server. The way to think about this is that it is probably best to consider each repo that has something to be published as a separate sub-tree on the site, with one of them (stx-docs) also being the root of the site. We'll put stx-specs at /specs and do similar with the other things to publish here. dt -- Dean Troyer dtroyer@gmail.com
I'd like to confirm some things and ask for a bit of information: 1. The stx-docs repo will contain most of the docs.starlingx.io site?
Also it might include a section to document the existing APIs from OpenStack projects modified by StarlingX.
api-ref is generally in-repo with its code....
Some of these API modifications fall into one of the existing OpenStack projects [0] and modifications are done at the starlingx-staging repo level [1], that was the reason we were thinking to host them under stx-docs, based in a previous analysis done [2], the latest list is as follows: Dcmanager API v1 - starlingx-staging/stx-distcloud Extensions to Block Storage REST API - starlingx-staging/stx-python-cinderclient Extensions to Compute REST API - starlingx-staging/stx-nova Extensions to Image REST API - starlingx-staging/stx-glance Extensions to Networking REST API - starlingx-staging/stx-neutron Is stx-upstream repository another option to host these OpenStack modified APIs leaving stx-docs repository for doc type information?
Is this related to generating our documentation using tox -e [releasenotes / api-ref / doc] ?
Yes, those tox jobs are what generates the documents and the publish jobs are what puts it on the web server.
The way to think about this is that it is probably best to consider each repo that has something to be published as a separate sub-tree on the site, with one of them (stx-docs) also being the root of the site.
stx-docs being the root of the site and every repo having its own doc/ directory so our working teams can take care of the whole documentation process including release notes and api-ref if applies, is this a correct assumption? stx-docs [StarlingX Landing Page] <project> documentation > docs/ api-ref > api-ref releasenotes > releasenotes/
We'll put stx-specs at /specs and do similar with the other things to publish here.
I will appreciate your help to feedback the patches I need to work on based on this email, the documentation projects [docs / releasenotes / api-ref] and our StarlingX repos [3] Any modification to the existing Yes / No? Once I have approval I will take 1 day to send all the gerrit reviews. [0] https://wiki.openstack.org/wiki/StarlingX/Developer_Guide/API_Documentation#... [1] https://github.com/starlingx-staging [2] https://wiki.openstack.org/wiki/StarlingX/Developer_Guide/API_Documentation#... [3] https://ethercalc.openstack.org/sifnpbvze9lb
On Mon, Sep 3, 2018 at 10:55 AM, Arce Moreno, Abraham <abraham.arce.moreno@intel.com> wrote:
Some of these API modifications fall into one of the existing OpenStack projects [0] and modifications are done at the starlingx-staging repo level [1], that was the reason we were thinking to host them under stx-docs, based in a previous analysis done [2], the latest list is as follows:
Dcmanager API v1 - starlingx-staging/stx-distcloud
This will be in the DC repo which is going to be imported into Gerrit Real Soon Now.
Extensions to Block Storage REST API - starlingx-staging/stx-python-cinderclient Extensions to Compute REST API - starlingx-staging/stx-nova Extensions to Image REST API - starlingx-staging/stx-glance Extensions to Networking REST API - starlingx-staging/stx-neutron
Yup, I forgot about these.
Is stx-upstream repository another option to host these OpenStack modified APIs leaving stx-docs repository for doc type information?
I like this idea, it keeps -docs clean for only the new code.
stx-docs being the root of the site and every repo having its own doc/ directory so our working teams can take care of the whole documentation process including release notes and api-ref if applies, is this a correct assumption?
stx-docs [StarlingX Landing Page] <project> documentation > docs/ api-ref > api-ref releasenotes > releasenotes/
That set of references under <project> can all be done as in-document links, the way redirects are handled makes it a bit clumsy to update. It shouldn't change frequently but I imagine we'll tweak a bit until we're happy at the start.
I will appreciate your help to feedback the patches I need to work on based on this email, the documentation projects [docs / releasenotes / api-ref] and our StarlingX repos [3] Any modification to the existing Yes / No?
I'll come back to this tomorrow, at a glance it looks like it is on-track.
Once I have approval I will take 1 day to send all the gerrit reviews.
You might want to get the first couple put together and merged to make sure everything is sorted out before doing a lot of reviews and having to go back and update them all when something is different that we originally thought. dt -- Dean Troyer dtroyer@gmail.com
Once I have approval I will take 1 day to send all the gerrit reviews.
You might want to get the first couple put together and merged to make sure everything is sorted out before doing a lot of reviews and having to go back and update them all when something is different that we originally thought.
Sure, here you have: stx-metal [Doc] Building docs following Docs Contrib Guide https://review.openstack.org/#/c/590094/ [Doc] Release Notes Management https://review.openstack.org/#/c/599454/ [Doc] OpenStack API Reference Guide https://review.openstack.org/#/c/590097/ stx-ha [Doc] Building docs following Docs Contrib Guide https://review.openstack.org/#/c/599464/ [Doc] Release Notes Management https://review.openstack.org/#/c/599465/ [Doc] OpenStack API Reference Guide https://review.openstack.org/#/c/599466/
On Mon, Sep 3, 2018 at 5:58 PM, Arce Moreno, Abraham <abraham.arce.moreno@intel.com> wrote:
stx-metal [Doc] Building docs following Docs Contrib Guide https://review.openstack.org/#/c/590094/
You can add the docs build job that will actually test the configuration you are setting up. https://review.openstack.org/#/c/599041/3/.zuul.yaml should be all you need, then the results of the job is lined in the Gerrit review so you can see it before it is merged and published. This is really useful since we don't have the publising part set up yet.
[Doc] Release Notes Management https://review.openstack.org/#/c/599454/
I've pushed up https://review.openstack.org/599831 to run the docs and relnotes build jobs to see what they looks like.
[Doc] OpenStack API Reference Guide https://review.openstack.org/#/c/590097/
I don't have any first-hand experience with this and will come back to it...I imagine it will be very similar to the above job-wise. dt -- Dean Troyer dtroyer@gmail.com
On Mon, Sep 3, 2018 at 5:58 PM, Arce Moreno, Abraham <abraham.arce.moreno@intel.com> wrote:
stx-metal [Doc] Building docs following Docs Contrib Guide https://review.openstack.org/#/c/590094/
You can add the docs build job that will actually test the configuration you are setting up. https://review.openstack.org/#/c/599041/3/.zuul.yaml should be all you need, then the results of the job is lined in the Gerrit review so you can see it before it is merged and published. This is really useful since we don't have the publising part set up yet.
What is the difference between .zuul.yaml in each repo and project.yaml from openstack-infra/project-config [0] ?
[Doc] Release Notes Management https://review.openstack.org/#/c/599454/
I've pushed up https://review.openstack.org/599831 to run the docs and relnotes build jobs to see what they looks like.
[Doc] OpenStack API Reference Guide https://review.openstack.org/#/c/590097/
I don't have any first-hand experience with this and will come back to it...I imagine it will be very similar to the above job-wise.
Based on OpenStack API documentation [1]: "Add the api-ref-jobs template..." Working now in taking your changes into stx-ha including api-ref-jobs. [0] https://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul.d/pr... [1] https://docs.openstack.org/doc-contrib-guide/api-guides.html
On Wed, Sep 5, 2018 at 9:30 AM, Arce Moreno, Abraham <abraham.arce.moreno@intel.com> wrote:
What is the difference between .zuul.yaml in each repo and project.yaml from openstack-infra/project-config [0] ?
Primarily that project-config is shared by all of OpenStack Infra and centrally managed, while .zuul.yaml is in the project repo and can be easily maintained within the project team directly.
[Doc] Release Notes Management https://review.openstack.org/#/c/599454/
I've pushed up https://review.openstack.org/599831 to run the docs and relnotes build jobs to see what they looks like.
[Doc] OpenStack API Reference Guide https://review.openstack.org/#/c/590097/
I don't have any first-hand experience with this and will come back to it...I imagine it will be very similar to the above job-wise.
Based on OpenStack API documentation [1]: "Add the api-ref-jobs template..."
Working now in taking your changes into stx-ha including api-ref-jobs.
Like I suggested earlier, let's get one set of these worked out first then fan out to the repos as required. It will kepp all of the discussion in one place and avoid re-working if/when things change before we are happy with the first one. Also, be careful about blindly adopting OpenStack job templates, some of them include the publishing jobs that we are unable to use, we need to define our own, which has started but none are merged yet. This is why I've called build-openstack-releasenotes directly in [0] rather than the upstream template, as I did with build-openstack-sphinx-docs-python3, which does not include any publish jobs. dt [0] https://review.openstack.org/#/c/599831/5/.zuul.yaml -- Dean Troyer dtroyer@gmail.com
Sorry for my comment, Are we considering to include some path in the site in order to upload reports?, should we open a thread for this as well? BR Elio -----Original Message----- From: Arce Moreno, Abraham [mailto:abraham.arce.moreno@intel.com] Sent: Monday, September 3, 2018 10:56 AM To: Dean Troyer <dtroyer@gmail.com> Cc: starlingx-discuss@lists.starlingx.io Subject: Re: [Starlingx-discuss] [docs] Setting up the docs.starlingx.io website
I'd like to confirm some things and ask for a bit of information: 1. The stx-docs repo will contain most of the docs.starlingx.io site?
Also it might include a section to document the existing APIs from OpenStack projects modified by StarlingX.
api-ref is generally in-repo with its code....
Some of these API modifications fall into one of the existing OpenStack projects [0] and modifications are done at the starlingx-staging repo level [1], that was the reason we were thinking to host them under stx-docs, based in a previous analysis done [2], the latest list is as follows: Dcmanager API v1 - starlingx-staging/stx-distcloud Extensions to Block Storage REST API - starlingx-staging/stx-python-cinderclient Extensions to Compute REST API - starlingx-staging/stx-nova Extensions to Image REST API - starlingx-staging/stx-glance Extensions to Networking REST API - starlingx-staging/stx-neutron Is stx-upstream repository another option to host these OpenStack modified APIs leaving stx-docs repository for doc type information?
Is this related to generating our documentation using tox -e [releasenotes / api-ref / doc] ?
Yes, those tox jobs are what generates the documents and the publish jobs are what puts it on the web server.
The way to think about this is that it is probably best to consider each repo that has something to be published as a separate sub-tree on the site, with one of them (stx-docs) also being the root of the site.
stx-docs being the root of the site and every repo having its own doc/ directory so our working teams can take care of the whole documentation process including release notes and api-ref if applies, is this a correct assumption? stx-docs [StarlingX Landing Page] <project> documentation > docs/ api-ref > api-ref releasenotes > releasenotes/
We'll put stx-specs at /specs and do similar with the other things to publish here.
I will appreciate your help to feedback the patches I need to work on based on this email, the documentation projects [docs / releasenotes / api-ref] and our StarlingX repos [3] Any modification to the existing Yes / No? Once I have approval I will take 1 day to send all the gerrit reviews. [0] https://wiki.openstack.org/wiki/StarlingX/Developer_Guide/API_Documentation#... [1] https://github.com/starlingx-staging [2] https://wiki.openstack.org/wiki/StarlingX/Developer_Guide/API_Documentation#... [3] https://ethercalc.openstack.org/sifnpbvze9lb _______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
On Mon, Sep 3, 2018 at 1:28 PM, Martinez Monroy, Elio <elio.martinez.monroy@intel.com> wrote:
Are we considering to include some path in the site in order to upload reports?, should we open a thread for this as well?
This is a static site built from Sphinx docs. What sort of report are you thinking about? dt -- Dean Troyer dtroyer@gmail.com
General reporting porpuses, maybe tars, html and xml robot reports -----Original Message----- From: Dean Troyer [mailto:dtroyer@gmail.com] Sent: Monday, September 3, 2018 1:51 PM To: Martinez Monroy, Elio <elio.martinez.monroy@intel.com> Cc: Arce Moreno, Abraham <abraham.arce.moreno@intel.com>; starlingx-discuss@lists.starlingx.io Subject: Re: [Starlingx-discuss] [docs] Setting up the docs.starlingx.io website On Mon, Sep 3, 2018 at 1:28 PM, Martinez Monroy, Elio <elio.martinez.monroy@intel.com> wrote:
Are we considering to include some path in the site in order to upload reports?, should we open a thread for this as well?
This is a static site built from Sphinx docs. What sort of report are you thinking about? dt -- Dean Troyer dtroyer@gmail.com
Are we considering to include some path in the site in order to upload reports?, should we open a thread for this as well?
This is a static site built from Sphinx docs. What sort of report are you thinking about?
Reports containing results coming from our regular functional tests, an example of the output in xml: -<robot generator="Robot 3.0.4 (Python 2.7.12 on linux2)" generated="20180829 05:18:39.626"> -<suite name="Setup" id="s1" source="/home/elio/secondgerrit/clean/stx-test-suite/Tests/Setup.robot"> +<test name="Test Qemu Libvirt VMs Setup" id="s1-t1"> +<test name="Test Install ISO" id="s1-t2"> +<test name="Test Config Controller" id="s1-t3"> <doc>Tests for Install and Configuration of a controller node. Author(s): - Jose Perez... <status starttime="20180829 05:18:39.627" endtime="20180829 05:57:07.073" status="FAIL"/> </suite> +<statistics> <errors> </errors> </robot> We were thinking where it could make sense to upload / post those results: - Wiki - Launchpad - Repository We also looked at how OpenStack reports results: - For every repo, under Gerrit giving a Pass / Fail - For RefStack, under their homepage page "community results" [0] - Any other that could fit our needs? Some ideas how to share StarlingX test reports: - Test wiki page to show executive summary results parsed from the given xml: - Today this is how we report under our Test wiki page [1] - Partially automated - stx-docs hosting those results parsed from the given xml to rst format: - Fully automated taking advantage of tox infrastructure, how about using "tox -e results"? - Results uploaded to doc.starlingx.io/results? [0] https://refstack.openstack.org/#/community_results [1] https://wiki.openstack.org/wiki/StarlingX/Test#Latest_ISO_image_Sanity_Summa...
On Mon, Sep 3, 2018 at 4:07 PM, Arce Moreno, Abraham <abraham.arce.moreno@intel.com> wrote:
We also looked at how OpenStack reports results: - For every repo, under Gerrit giving a Pass / Fail
This is served from a central log server that Zuul jobs push test logs to. I don't know if infra would let outside systems push here too, I'm leaning toward unlikely. All 3rd party CI systems are required to host their own logs on a public-facing server. I'm not sure why we would be any different here. dt -- Dean Troyer dtroyer@gmail.com
Hi, For third-party CI systems that run on premise we don’t provide a log server but recommend to have a machine to which your job runner can push logs onto (via scp or ftp or rsync or...) with apache with autoindexing turned on to make the log files available. We have a puppet module a lot of third-party CI deployments use which incorporates a log publishing website: https://git.openstack.org/cgit/openstack-infra/puppet-openstackci/tree/manif... It might be a bit more than what you would want as a first run as it does colorization and interactive filtering of common log formats via a WSGI backend, it depends on your preference which route you go down on. In either case you probably also want to have some means of purging logs after they reach a certain age so it doesn't grow out of control (that puppet module I linked above includes a cron-driven script to do that too). Please let me know if you have any questions. Thanks and Best Regards, Ildikó
On 2018. Sep 3., at 15:07, Arce Moreno, Abraham <abraham.arce.moreno@intel.com> wrote:
Are we considering to include some path in the site in order to upload reports?, should we open a thread for this as well?
This is a static site built from Sphinx docs. What sort of report are you thinking about?
Reports containing results coming from our regular functional tests, an example of the output in xml:
-<robot generator="Robot 3.0.4 (Python 2.7.12 on linux2)" generated="20180829 05:18:39.626"> -<suite name="Setup" id="s1" source="/home/elio/secondgerrit/clean/stx-test-suite/Tests/Setup.robot"> +<test name="Test Qemu Libvirt VMs Setup" id="s1-t1"> +<test name="Test Install ISO" id="s1-t2"> +<test name="Test Config Controller" id="s1-t3"> <doc>Tests for Install and Configuration of a controller node. Author(s): - Jose Perez... <status starttime="20180829 05:18:39.627" endtime="20180829 05:57:07.073" status="FAIL"/> </suite> +<statistics> <errors> </errors> </robot>
We were thinking where it could make sense to upload / post those results: - Wiki - Launchpad - Repository
We also looked at how OpenStack reports results: - For every repo, under Gerrit giving a Pass / Fail - For RefStack, under their homepage page "community results" [0] - Any other that could fit our needs?
Some ideas how to share StarlingX test reports:
- Test wiki page to show executive summary results parsed from the given xml: - Today this is how we report under our Test wiki page [1] - Partially automated - stx-docs hosting those results parsed from the given xml to rst format: - Fully automated taking advantage of tox infrastructure, how about using "tox -e results"? - Results uploaded to doc.starlingx.io/results?
[0] https://refstack.openstack.org/#/community_results [1] https://wiki.openstack.org/wiki/StarlingX/Test#Latest_ISO_image_Sanity_Summa...
_______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
participants (4)
-
Arce Moreno, Abraham
-
Dean Troyer
-
Ildiko Vancsa
-
Martinez Monroy, Elio