[Starlingx-discuss] Flock Versioning for packaging

Wed May 15 20:18:49 UTC 2019

I have a little past experience with pbr, etc..

PBR is based off git tags.

The build tools currently do not preserve this git tag information when they prepare the src tarball for the src rpms.
  https://opendev.org/starlingx/root/src/branch/master/build-tools/build-pkg-srpm#L39

To handle this (today), the spec files are able to fake the PBR git tag through the PBR_VERSION env variable
  Ex:  https://opendev.org/starlingx/config/src/branch/master/sysinv/cgts-client/centos/cgts-client.spec#L46

If we update the build tool (and remove those variables from the spec files), then all python components in a particular repo will have the same version.
We have some repos where there are multiple python components in the same repo.

The other (minor) thing to be aware of is the cost of querying the pbr version.
Pbr makes use of pkg_resources, which processes all python files in order to determine entry_points, etc..

You incur this cost during import (only once per process). Typically about half a second in python2.  I don't know about python3. 
I remember explicitly turning it off for cgtsclient, but the savings are begated  if there are any other python imports (like keyring) that are also using pkg_resources. 

Al

-----Original Message-----
From: Jones, Bruce E [mailto:bruce.e.jones at intel.com] 
Sent: Wednesday, May 15, 2019 3:22 PM
To: Saul Wold; starlingx-discuss at lists.starlingx.io
Subject: Re: [Starlingx-discuss] Flock Versioning for packaging

Saul, thank you for sharing this.  I like it a lot.  I'm looking forward to seeing the spec.

Can we / should we extend our support of semantic versioning to the overall release numbering?  We're already on that track - we are doing our 2.0 release now, which is an API level release from stx.2018.10 which we could consider our 1.0 release.  If we call it 2.0.0, we'd be on track to continue following the versioning requirements going forward.

I think clearly communicating API changes to our users would be very helpful.

     brucej

-----Original Message-----
From: Saul Wold [mailto:sgw at linux.intel.com] 
Sent: Wednesday, May 15, 2019 11:29 AM
To: starlingx-discuss at lists.starlingx.io
Subject: [Starlingx-discuss] Flock Versioning for packaging

StarlingX Community:

Currently the versioning of the flock is mostly matching their internal API '1.0-<tis_patch_ver>.tis'. This is historical since the Upgrade process is a complete re-image of the disk from ISO and doesn't actually have to increase the version number between releases (other than maybe the tis_patch_ver).

Moving forward we need to consider using the OpenStack Semantic Versioning [0], and looking at PBR (Python Build Reasonableness) [1] to generate the version that is based on MAJOR.MINOR.PATCH where:

- MAJOR version when you make incompatible API changes
- MINOR version when you add functionality in a backwards-compatible manner
- PATCH version when you make backwards-compatible bug fixes.

Additional labels for pre-release and build metadata are available as extensions to the MAJOR.MINOR.PATCH format.

We understand the requirement that the Flock has across releases to preserve backward compatibility, likely the MAJOR will never change, but we should be considering that we bump the MINOR on each release and reset PATCH and  bump PATCH based on doing a CVE, or required bug fix between releases.

To enable PBR, we would need to change the Python related setup to use setup.cfg with pbr, some of this work has started by Dean with Devstack related work. PBR won't address all packages since some are non-Python.
We would also have to add versoining tags into git for PBR to work correctly. PBR can also help generate information for packaging.

During development, X.Y.Z.dev<N> or a "+g<hash>" is possible for builds. 
This is needed to during the development phase in order to track changes and help debug failures based on what packages are installed, it could also allow for independently updating of packages on a target system using standard packaging tools. This will be particularly important if we install flock packages on a standard OS and not via a custom ISO.
or PBR to work.

For example we are currently at fm-common-1.0-8.tis or sysinv-1.0-317.tis (for the master) as we approach the "stx-2.0" release for each commit to stx-fault a tarball would be generated: 
fm-common-1.0.0.dev<X> ... fm-common-1.0.0.dev<X+1>. When we release
stx-2.0 the release tarball will be fm-common-1.2.0 and a patch will be fm-common-1.2.1.  As we start the next release work will happen on fm-common-1.2.<z>.dev<X> ...

OpenDev has the infrastructure already via Zuul to generate and publish both release and intermediate development artifacts. We will need to make some policy decisions regarding how long we keep devlopment artifacts.

The justification for these changes are to ensure versioning that will work correctly for the various build tools of the different operating systems that maybe required for StarlingX to exist on. Every operating system has its own build recipe format to create packages to distribute the SW project across their users. Each package includes metadata that describes the package’s components, version, release, build and installation instructions among others. Without a proper version from the original source code , the distributions will have a hard time managing the packaging.

Please comment, this is a strawman to get input, after that the next step is to generate a proper Specification.

Thanks
    Sau!

[0] https://docs.openstack.org/pbr/3.1.0/semver.html
[1] https://docs.openstack.org/pbr/latest/

_______________________________________________
Starlingx-discuss mailing list
Starlingx-discuss at lists.starlingx.io
http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
_______________________________________________
Starlingx-discuss mailing list
Starlingx-discuss at lists.starlingx.io
http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss