[Starlingx-discuss] about cherry-pick a patch from OpenStack upstream for testing issue in StarlingX
Penney, Don
Don.Penney at windriver.com
Mon Jul 22 13:59:20 UTC 2019
Hi Zhipeng,
Can you verify that you can run the following in your cloned repo?
[/localdisk/loadbuild/dpenney/nova-update/stx-nova]$ python ./setup.py --version
19.0.1.dev116
-----Original Message-----
From: Liu, ZhipengS [mailto:zhipengs.liu at intel.com]
Sent: Monday, July 22, 2019 3:15 AM
To: Penney, Don; Dean Troyer; starlingx-discuss at lists.starlingx.io
Cc: zhu.boxiang at 99cloud.net
Subject: RE: [Starlingx-discuss] about cherry-pick a patch from OpenStack upstream for testing issue in StarlingX
Hi Don,
I failed to update image. Could you give me some help, thanks!
~/starlingx/cgcs-root/build-tools/build-docker-images$ bash update-stx-image.sh \
> --from starlingx/stx-nova:master-centos-stable-20190715T233000Z.0 \
> --module-src ~/starlingx/workspace/localdisk/loadbuild/zhipengl/nova-update/stx-nova --user zhipengl
~/starlingx/cgcs-root/build-tools/build-docker-images ~/starlingx/cgcs-root/build-tools/build-docker-images
~/starlingx/workspace/localdisk/loadbuild/zhipengl/starlingx/std/update-images/unnamed-update/stx-nova_master-centos-stable-20190715T233000Z.1/pip-packages/wheels ~/starlingx/cgcs-root/build-tools/build-docker-images ~/starlingx/cgcs-root/build-tools/build-docker-images
Running: wget http://mirror.starlingx.cengn.ca/mirror/starlingx/master/centos/latest_docker_image_build/outputs/wheels//stx-centos-stable-wheels.tar
--2019-07-22 02:57:20-- http://mirror.starlingx.cengn.ca/mirror/starlingx/master/centos/latest_docker_image_build/outputs/wheels//stx-centos-stable-wheels.tar
Resolving mirror.starlingx.cengn.ca (mirror.starlingx.cengn.ca)... 135.84.104.40
Connecting to mirror.starlingx.cengn.ca (mirror.starlingx.cengn.ca)|135.84.104.40|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 63354880 (60M) [application/octet-stream]
Saving to: 'stx-centos-stable-wheels.tar'
stx-centos-stable-wheels.tar 100%[===========================================================================>] 60.42M 7.41MB/s in 14s
2019-07-22 02:57:34 (4.42 MB/s) - 'stx-centos-stable-wheels.tar' saved [63354880/63354880]
~/starlingx/workspace/localdisk/loadbuild/zhipengl/starlingx/std/update-images/unnamed-update/stx-nova_master-centos-stable-20190715T233000Z.1/pip-packages/modules/stx-nova ~/starlingx/workspace/localdisk/loadbuild/zhipengl/starlingx/std/update-images/unnamed-update/stx-nova_master-centos-stable-20190715T233000Z.1/pip-packages/wheels ~/starlingx/cgcs-root/build-tools/build-docker-images ~/starlingx/cgcs-root/build-tools/build-docker-images
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'long_description_content_type'
warnings.warn(msg)
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'project_urls'
warnings.warn(msg)
sed: -e expression #1, char 20: unterminated `s' command
~/starlingx/workspace/localdisk/loadbuild/zhipengl/starlingx/std/update-images/unnamed-update/stx-nova_master-centos-stable-20190715T233000Z.1/pip-packages/wheels ~/starlingx/cgcs-root/build-tools/build-docker-images ~/starlingx/cgcs-root/build-tools/build-docker-images
~/starlingx/cgcs-root/build-tools/build-docker-images ~/starlingx/cgcs-root/build-tools/build-docker-images
Running: docker image pull starlingx/stx-nova:master-centos-stable-20190715T233000Z.0
master-centos-stable-20190715T233000Z.0: Pulling from starlingx/stx-nova
5ad559c5ae16: Already exists
3d7ee0b84e39: Pull complete
e1047bfd73cf: Pull complete
16f151ad544f: Pull complete
844126c15d7e: Pull complete
869460797821: Pull complete
0c319da8f09d: Pull complete
2e884702ad07: Pull complete
Digest: sha256:4032358f3ab208e76c2737854fdcf4f7bd582ef8f91e150193ded8feb662fdbe
Status: Downloaded newer image for starlingx/stx-nova:master-centos-stable-20190715T233000Z.0
+ bash -x /image-update/internal-update-stx-image.sh
+ UPDATES_DIR=/image-update
+ PIP_PACKAGES_DIR=/image-update/pip-packages
+ DIST_PACKAGES_DIR=/image-update/dist-packages
+ CUSTOMIZATION_SCRIPT=/image-update/customize.sh
++ source /etc/os-release
+++ NAME='CentOS Linux'
+++ VERSION='7 (Core)'
+++ ID=centos
+++ ID_LIKE='rhel fedora'
+++ VERSION_ID=7
+++ PRETTY_NAME='CentOS Linux 7 (Core)'
+++ ANSI_COLOR='0;31'
+++ CPE_NAME=cpe:/o:centos:centos:7
+++ HOME_URL=https://www.centos.org/
+++ BUG_REPORT_URL=https://bugs.centos.org/
+++ CENTOS_MANTISBT_PROJECT=CentOS-7
+++ CENTOS_MANTISBT_PROJECT_VERSION=7
+++ REDHAT_SUPPORT_PRODUCT=centos
+++ REDHAT_SUPPORT_PRODUCT_VERSION=7
++ echo CentOS Linux
+ OS_NAME='CentOS Linux'
++ getopt -o h -l help: --
+ OPTS=' --'
+ '[' 0 -ne 0 ']'
+ eval set -- ' --'
++ set -- --
+ true
+ case $1 in
+ shift
+ break
+ install_dist_packages
+ local -i file_count=0
++ find /image-update/dist-packages -type f
++ wc -l
+ file_count=0
+ '[' 0 -eq 0 ']'
+ return 0
+ install_pip_packages
+ local modules
+ local wheels
++ find /image-update/pip-packages/modules/stx-nova -maxdepth 0 -type d
+ modules=/image-update/pip-packages/modules/stx-nova
++ find /image-update/pip-packages/wheels/ -type f -name '*.whl'
+ wheels=
+ '[' -z /image-update/pip-packages/modules/stx-nova -a -z '' ']'
+ pip install -vvv --no-deps --no-index --pre --no-cache-dir --only-binary :all: --no-compile --force-reinstall /image-update/pip-packages/modules/stx-nova
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.
Ignoring indexes: https://pypi.org/simple
Created temporary directory: /tmp/pip-ephem-wheel-cache-Pld6sg
Created temporary directory: /tmp/pip-req-tracker-5OQGeX
Created requirements tracker '/tmp/pip-req-tracker-5OQGeX'
Created temporary directory: /tmp/pip-install-8tWna2
Processing /image-update/pip-packages/modules/stx-nova
Created temporary directory: /tmp/pip-req-build-EngJ2Q
Added file:///image-update/pip-packages/modules/stx-nova to build tracker '/tmp/pip-req-tracker-5OQGeX'
Running setup.py (path:/tmp/pip-req-build-EngJ2Q/setup.py) egg_info for package from file:///image-update/pip-packages/modules/stx-nova
Running command python setup.py egg_info
ERROR:root:Error parsing
Traceback (most recent call last):
File "/var/lib/openstack/lib/python2.7/site-packages/pbr/core.py", line 96, in pbr
attrs = util.cfg_to_args(path, dist.script_args)
File "/var/lib/openstack/lib/python2.7/site-packages/pbr/util.py", line 256, in cfg_to_args
pbr.hooks.setup_hook(config)
File "/var/lib/openstack/lib/python2.7/site-packages/pbr/hooks/__init__.py", line 25, in setup_hook
metadata_config.run()
File "/var/lib/openstack/lib/python2.7/site-packages/pbr/hooks/base.py", line 27, in run
self.hook()
File "/var/lib/openstack/lib/python2.7/site-packages/pbr/hooks/metadata.py", line 26, in hook
self.config['name'], self.config.get('version', None))
File "/var/lib/openstack/lib/python2.7/site-packages/pbr/packaging.py", line 849, in get_version
name=package_name))
Exception: Versioning for this project requires either an sdist tarball, or access to an upstream git repository. It's also possible that there is a mismatch between the package name in setup.cfg and the argument given to pbr.version.VersionInfo. Project name nova was given, but was not able to be found.
error in setup command: Error parsing /tmp/pip-req-build-EngJ2Q/setup.cfg: Exception: Versioning for this project requires either an sdist tarball, or access to an upstream git repository. It's also possible that there is a mismatch between the package name in setup.cfg and the argument given to pbr.version.VersionInfo. Project name nova was given, but was not able to be found.
Cleaning up...
Removing source in /tmp/pip-req-build-EngJ2Q
Removed file:///image-update/pip-packages/modules/stx-nova from build tracker '/tmp/pip-req-tracker-5OQGeX'
Removed build tracker '/tmp/pip-req-tracker-5OQGeX'
ERROR: Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-req-build-EngJ2Q/
Exception information:
Traceback (most recent call last):
File "/var/lib/openstack/lib/python2.7/site-packages/pip/_internal/cli/base_command.py", line 178, in main
status = self.run(options, args)
File "/var/lib/openstack/lib/python2.7/site-packages/pip/_internal/commands/install.py", line 352, in run
resolver.resolve(requirement_set)
File "/var/lib/openstack/lib/python2.7/site-packages/pip/_internal/resolve.py", line 131, in resolve
self._resolve_one(requirement_set, req)
File "/var/lib/openstack/lib/python2.7/site-packages/pip/_internal/resolve.py", line 294, in _resolve_one
abstract_dist = self._get_abstract_dist_for(req_to_install)
File "/var/lib/openstack/lib/python2.7/site-packages/pip/_internal/resolve.py", line 242, in _get_abstract_dist_for
self.require_hashes
File "/var/lib/openstack/lib/python2.7/site-packages/pip/_internal/operations/prepare.py", line 362, in prepare_linked_requirement
abstract_dist.prep_for_dist(finder, self.build_isolation)
File "/var/lib/openstack/lib/python2.7/site-packages/pip/_internal/operations/prepare.py", line 171, in prep_for_dist
self.req.prepare_metadata()
File "/var/lib/openstack/lib/python2.7/site-packages/pip/_internal/req/req_install.py", line 537, in prepare_metadata
self.run_egg_info()
File "/var/lib/openstack/lib/python2.7/site-packages/pip/_internal/req/req_install.py", line 615, in run_egg_info
command_desc='python setup.py egg_info')
File "/var/lib/openstack/lib/python2.7/site-packages/pip/_internal/utils/misc.py", line 776, in call_subprocess
% (command_desc, proc.returncode, cwd))
InstallationError: Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-req-build-EngJ2Q/
+ '[' 1 -ne 0 ']'
+ echo 'Failed pip install'
Failed pip install
+ exit 1
Failed to update image: starlingx/stx-nova:master-centos-stable-20190715T233000Z.0
Zhipeng
-----Original Message-----
From: Penney, Don [mailto:Don.Penney at windriver.com]
Sent: 2019年7月20日 2:04
To: Dean Troyer <dtroyer at gmail.com>; starlingx-discuss at lists.starlingx.io
Cc: zhu.boxiang at 99cloud.net
Subject: Re: [Starlingx-discuss] about cherry-pick a patch from OpenStack upstream for testing issue in StarlingX
For updating an image for testing, take a look at:
https://wiki.openstack.org/wiki/StarlingX/Containers/BuildingImages#Incremental_Image_Updates
As Dean notes, clone the repo and cherry-pick the commit, and then do something like:
time bash -x ${MY_REPO}/build-tools/build-docker-images/update-stx-image.sh \
--from starlingx/stx-nova:master-centos-stable-20190715T233000Z.0 \
--module-src ${path_to_cloned_repo}/stx-nova
ie:
# clone stx-nova, get stx/stein.2
cd /localdisk/loadbuild/dpenney/
mkdir nova-update
cd nova-update/
git clone https://github.com/starlingx-staging/stx-nova.git
cd stx-nova/
git fetch https://github.com/starlingx-staging/stx-nova.git stx/stein.2 git checkout FETCH_HEAD
# cherry-pick update
git fetch https://review.opendev.org/openstack/nova refs/changes/69/651969/13 && git cherry-pick FETCH_HEAD
# Fix up conflicts, etc
# Build updated image, from 20190715T233000Z build as base, # specifying cloned/modified repo time bash -x ${MY_REPO}/build-tools/build-docker-images/update-stx-image.sh \
--from starlingx/stx-nova:master-centos-stable-20190715T233000Z.0 \
--module-src /localdisk/loadbuild/dpenney/nova-update/stx-nova \
--user dpenney
This produces an updated image in the local registry:
Updated image: dpenney/stx-nova:master-centos-stable-20190715T233000Z.1
If you also set a registry with --registry and use the --push option, the command will push the updated image to that registry.
This also produces an image record file:
$ cat ${MY_WORKSPACE}/std/update-images/unnamed-update/image-updates.lst
dpenney/stx-nova:master-centos-stable-20190715T233000Z.1
Which you can pass as an argument to build-helm-charts.sh when building your application tarball for testing:
build-helm-charts.sh \
--image-record http://mirror.starlingx.cengn.ca/mirror/starlingx/master/centos/latest_docker_image_build/outputs/docker-images/images-centos-stable-versioned.lst \
--image-record ${MY_WORKSPACE}/std/update-images/unnamed-update/image-updates.lst \
--label centos-stable-versioned
If you look at the yaml file in the tarball, you can see that it now references the updated image:
$ tar xzf ${MY_WORKSPACE}/std/build-helm/stx/stx-openstack-1.0-17-centos-stable-versioned.tgz -O ./stx-openstack.yaml | grep stx-nova:
nova_api: dpenney/stx-nova:master-centos-stable-20190715T233000Z.1
nova_cell_setup: dpenney/stx-nova:master-centos-stable-20190715T233000Z.1
nova_compute: dpenney/stx-nova:master-centos-stable-20190715T233000Z.1
nova_compute_ironic: dpenney/stx-nova:master-centos-stable-20190715T233000Z.1
nova_compute_ssh: dpenney/stx-nova:master-centos-stable-20190715T233000Z.1
nova_conductor: dpenney/stx-nova:master-centos-stable-20190715T233000Z.1
nova_consoleauth: dpenney/stx-nova:master-centos-stable-20190715T233000Z.1
nova_db_sync: dpenney/stx-nova:master-centos-stable-20190715T233000Z.1
nova_novncproxy: dpenney/stx-nova:master-centos-stable-20190715T233000Z.1
nova_placement: dpenney/stx-nova:master-centos-stable-20190715T233000Z.1
nova_scheduler: dpenney/stx-nova:master-centos-stable-20190715T233000Z.1
nova_spiceproxy: dpenney/stx-nova:master-centos-stable-20190715T233000Z.1
nova_spiceproxy_assets: dpenney/stx-nova:master-centos-stable-20190715T233000Z.1
-----Original Message-----
From: Dean Troyer [mailto:dtroyer at gmail.com]
Sent: Friday, July 19, 2019 1:08 PM
To: starlingx-discuss at lists.starlingx.io
Cc: zhu.boxiang at 99cloud.net
Subject: Re: [Starlingx-discuss] about cherry-pick a patch from OpenStack upstream for testing issue in StarlingX
On Fri, Jul 19, 2019 at 10:57 AM Yong Hu <yong.hu at intel.com> wrote:
> For LP[0], there is a patch (https://review.opendev.org/#/c/651969/)
> in Nova upstream, what's the method/process for StarlingX to
> cherry-pick it for testing a LP reported in StarlingX?
>
> [0]:https://bugs.launchpad.net/starlingx/+bug/1820882
At a high level you would clone the stx-nova repo stx/stein.2 branch [0] and cherry-pick/backport the commit you want to test, and rebuild the Nova docker image and test that.
I do have some Zuul jobs in starlingx/tis-repo that will pull that Nova branch and run the unit, functional and pep8 tox jobs to test in OpenStack CI.
Given that the stx/stein branches also carry the NUMA live migration patches you would probably want to do whatever other live migration testing would be done upstream to validate this in the StarlingX context.
dt
[0] https://github.com/starlingx-staging/stx-nova/tree/stx/stein.2
--
Dean Troyer
dtroyer at gmail.com
_______________________________________________
Starlingx-discuss mailing list
Starlingx-discuss at lists.starlingx.io
http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
_______________________________________________
Starlingx-discuss mailing list
Starlingx-discuss at lists.starlingx.io
http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
More information about the Starlingx-discuss
mailing list