Hi Don, Could you help on this python3-rbd dependency issue? We have to fix this issue to finish cinder image build. Thanks a lot! Zhipeng From: YuChengDe <yu.chengde@99cloud.net> Sent: 2020年3月23日 18:55 To: don.penney <Don.Penney@windriver.com> Cc: starlingx-discuss <starlingx-discuss@lists.starlingx.io>; Somerville, Jim <Jim.Somerville@windriver.com>; Liu, ZhipengS <zhipengs.liu@intel.com> Subject: Re:RE: [Starlingx-discuss] [OpenStack Upgrade]Failed when build StarlingX docker images Hi Don: Thank for your suggestion. After exclude the starlingx_dashboard from initial building, we built the stx-horizon successfully. For now, we have the 12/13 openstack image for Ussuri version. The last should be stx-glance. It needs python3-rbd package during building. However, we just have python-rbd package for our enviroment. Do you have any proposal for how to get the required python3-rbd? + yum -y --setopt=skip_missing_names_on_install=False install ceph-common libxml2 python3-rbd postgresql-libs ^[[0mLoaded plugins: fastestmirror, ovl Loading mirror speeds from cached hostfile * base: ftp.sjtu.edu.cn<ftp://ftp.sjtu.edu.cn> * extras: mirrors.ustc.edu.cn * updates: ftp.sjtu.edu.cn<ftp://ftp.sjtu.edu.cn> Package libxml2-2.9.1-6.el7_2.3.x86_64 already installed and latest version No package python3-rbd available. ^[[91mError: Not tolerating missing names on install, stopping. ^[[0mThe command '/bin/sh -c /opt/loci/scripts/install.sh' returned a non-zero code: 1 Command (docker) failed, attempt 1 of 1. [http://mailhz.qiye.163.com/qiyeimage/logo/60511048/1576638602260.jpg] -- ————————————————————————————— 九州云信息科技有限公司 99CLOUD Inc. 于成德 产品开发部 邮箱(Email): yu.chengde@99cloud.net<mailto:yu.chengde@99cloud.net> 手机(Mobile): 13816965096 地址(Addr): 上海市局门路427号1号楼206 Room 206, Bldg 1, No.427 JuMen Road, ShangHai, China 网址(Site): http://www.99cloud.net<http://www.99cloud.net/> 发件人："Liu, ZhipengS" <zhipengs.liu@intel.com<mailto:zhipengs.liu@intel.com>> 发送日期：2020-03-14 10:30:32 收件人："Penney, Don" <Don.Penney@windriver.com<mailto:Don.Penney@windriver.com>>,YuChengDe <yu.chengde@99cloud.net<mailto:yu.chengde@99cloud.net>>,"starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>" <starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>> 抄送人："Somerville, Jim" <Jim.Somerville@windriver.com<mailto:Jim.Somerville@windriver.com>> 主题：RE: [Starlingx-discuss] [OpenStack Upgrade]Failed when build StarlingX docker images Thanks Don! We will try to do it further according to your proposal. Zhipeng From: Penney, Don <Don.Penney@windriver.com<mailto:Don.Penney@windriver.com>> Sent: 2020年3月14日 0:24 To: Liu, ZhipengS <zhipengs.liu@intel.com<mailto:zhipengs.liu@intel.com>>; YuChengDe <yu.chengde@99cloud.net<mailto:yu.chengde@99cloud.net>>; starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Cc: Somerville, Jim <Jim.Somerville@windriver.com<mailto:Jim.Somerville@windriver.com>> Subject: RE: [Starlingx-discuss] [OpenStack Upgrade]Failed when build StarlingX docker images I believe this has dependencies on a variety of StarlingX modules which are currently python2 only. You’ll probably need to exclude starlingx-dashboard for your initial builds, until all the required modules are available for python3. That said, it would be using only small components of these. So an initial effort to get python3 wheels for all the required modules may be small. Smaller than the effort to get these running as python3 on-host, certainly. From: Liu, ZhipengS [mailto:zhipengs.liu@intel.com] Sent: Friday, March 13, 2020 12:20 PM To: Penney, Don; YuChengDe; starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Cc: Somerville, Jim Subject: RE: [Starlingx-discuss] [OpenStack Upgrade]Failed when build StarlingX docker images Hi Don, We have finished 11/13 images build except for horizon and glance. For horizon, it needs starlingx-dashboard python3 whl. However, in our current build, we just build out starlingx-dashboard python2 whl. Do you have any proposal on how to get the required python3 whl. Thanks! Zhipeng From: Liu, ZhipengS Sent: 2020年3月10日 21:00 To: 'Liu, ZhipengS' <zhipengs.liu@intel.com<mailto:zhipengs.liu@intel.com>>; 'Penney, Don' <Don.Penney@windriver.com<mailto:Don.Penney@windriver.com>>; 'YuChengDe' <yu.chengde@99cloud.net<mailto:yu.chengde@99cloud.net>>; 'starlingx-discuss@lists.starlingx.io' <starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>> Cc: 'Somerville, Jim' <Jim.Somerville@windriver.com<mailto:Jim.Somerville@windriver.com>> Subject: RE: [Starlingx-discuss] [OpenStack Upgrade]Failed when build StarlingX docker images Hi Don, Chant and I already fixed this issue and other related dependent packages. We modified docker file and related script for wheel build, and rebuilt out all python3 wheels for the packages in stable-wheels.cfg. As for libvirt-python, we still use 4.7.0 version and rebuilt out python3 wheel currently. Will let you know the status after finishing all image build. Thanks! Zhipeng From: Liu, ZhipengS Sent: 2020年3月10日 0:06 To: 'Liu, ZhipengS' <zhipengs.liu@intel.com<mailto:zhipengs.liu@intel.com>>; Penney, Don <Don.Penney@windriver.com<mailto:Don.Penney@windriver.com>>; YuChengDe <yu.chengde@99cloud.net<mailto:yu.chengde@99cloud.net>>; starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Cc: Somerville, Jim <Jim.Somerville@windriver.com<mailto:Jim.Somerville@windriver.com>> Subject: RE: [Starlingx-discuss] [OpenStack Upgrade]Failed when build StarlingX docker images We also had below issue during build other service image. Looking in links: /tmp/wheels/ Collecting cryptography===2.8 Downloading cryptography-2.8-cp34-abi3-manylinux2010_x86_64.whl (2.3 MB) Collecting PyMySQL===0.9.3 Downloading PyMySQL-0.9.3-py2.py3-none-any.whl (47 kB) Processing /tmp/wheels/python_memcached-1.59-py2.py3-none-any.whl Collecting six===1.14.0 Downloading six-1.14.0-py2.py3-none-any.whl (10 kB) Collecting cffi===1.14.0 Downloading cffi-1.14.0-cp36-cp36m-manylinux1_x86_64.whl (399 kB) ERROR: Could not find a version that satisfies the requirement uWSGI===2.0.17.1 (from -c /tmp/wheels/upper-constraints.txt (line 816)) (from versions: none) ERROR: No matching distribution found for uWSGI===2.0.17.1 (from -c /tmp/wheels/upper-constraints.txt (line 816)) Actually, we get uWSGI-2.0.17.1-cp27-cp27mu-linux_x86_64.whl through the link in stable-config file. So far, not sure how many python3 packages we need. From: Liu, ZhipengS <zhipengs.liu@intel.com<mailto:zhipengs.liu@intel.com>> Sent: 2020年3月9日 23:44 To: Penney, Don <Don.Penney@windriver.com<mailto:Don.Penney@windriver.com>>; YuChengDe <yu.chengde@99cloud.net<mailto:yu.chengde@99cloud.net>>; starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Cc: Somerville, Jim <Jim.Somerville@windriver.com<mailto:Jim.Somerville@windriver.com>> Subject: Re: [Starlingx-discuss] [OpenStack Upgrade]Failed when build StarlingX docker images Thanks Don! See my inline comments. From: Penney, Don <Don.Penney@windriver.com<mailto:Don.Penney@windriver.com>> Sent: 2020年3月9日 23:18 To: Liu, ZhipengS <zhipengs.liu@intel.com<mailto:zhipengs.liu@intel.com>>; YuChengDe <yu.chengde@99cloud.net<mailto:yu.chengde@99cloud.net>>; starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Cc: Somerville, Jim <Jim.Somerville@windriver.com<mailto:Jim.Somerville@windriver.com>> Subject: RE: [Starlingx-discuss] [OpenStack Upgrade]Failed when build StarlingX docker images I think building the openstack master branch will require moving the images to python3. This has implications to the base image, to how we build the base wheels tarball, and possibly to arguments passed to loci (as well as moving to a newer loci SHA).
Yes, we have to set python3 = true to loci and also fix a small known bug. I believe Al started looking at this last week, but got pulled off to look at some other issues. From your reply, it sounds like maybe you’ve also done some of this? Yes When we explicitly add wheels to the tarball, we update the upper-constraints file to align with the module we’ve included. This would be why the build is looking for 4.7.0 here. There is an upper-constraints.txt file in the tarball itself which you can check. Yes, in our script, if we have 4.7.0 in stx repo, it will change the version to 4.7.0 from other versions specified in upper-constraints.txt The libvirt wheel building is python2 only, currently. So it would need to be built for python3 - if that’s possible for 4.7.0. If a newer version of libvirt-python is needed for python3 support, then it will need to be updated. I don’t know how big that effort will be. From code, libvirt-python 4.7.0 can support both python2 and python3, but in our code, we just add patch to let it build out python2 whl. libvirt_python-4.7.0-cp27-cp27mu-linux_x86_64.whl<https://wx.qq.com/cgi-bin/mmwebwx-bin/webwxcheckurl?requrl=http%3A%2F%2Flibvirt_python-4.7.0-cp27-cp27mu-linux_x86_64.whl&skey=%40crypt_616c303a_f399f4eab101b16302fa59340fd7bf35&deviceid=e980571888491758&pass_ticket=USSEyd5y7N2ejqbNx89EA22r2cAuFWiG5M%252BHcswHTXUy7aMIu4fbMpfEtLrLp7Hu&opcode=2&scene=1&username=@72c15f7ccf676b86ee6a89ed6cdd8e41>
From: Liu, ZhipengS [mailto:zhipengs.liu@intel.com] Sent: Thursday, March 05, 2020 1:06 AM To: Penney, Don; YuChengDe; starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Subject: RE: [Starlingx-discuss] [OpenStack Upgrade]Failed when build StarlingX docker images Hi Penney, We just used OpenStack master branch and upper-constraint file below https://raw.githubusercontent.com/openstack/requirements/master/upper-constr... libvirt-python===5.10.0;python_version=='2.7' libvirt-python===6.0.0;python_version=='3.6' libvirt-python===6.0.0;python_version=='3.7' In nova setup.cfg. it adds python require info. python-requires = >=3.6 Then we built python3 related packages to the base image and rebuilt nova image to fix python3 issue. But there is still an error below. collecting cryptography===2.8 Downloading cryptography-2.8-cp34-abi3-manylinux2010_x86_64.whl (2.3 MB) ERROR: Could not find a version that satisfies the requirement libvirt-python===4.7.0 (from -c /tmp/wheels/upper-constraints.txt (line 274)) (from versions: none) ERROR: No matching distribution found for libvirt-python===4.7.0 (from -c /tmp/wheels/upper-constraints.txt (line 274)) The command '/bin/sh -c /opt/loci/scripts/install.sh' returned a non-zero code: 1 From code, I saw that we already built libvirt_python-4.7.0-cp27-cp27mu-linux_x86_64.whl<https://wx.qq.com/cgi-bin/mmwebwx-bin/webwxcheckurl?requrl=http%3A%2F%2Flibvirt_python-4.7.0-cp27-cp27mu-linux_x86_64.whl&skey=%40crypt_616c303a_f399f4eab101b16302fa59340fd7bf35&deviceid=e980571888491758&pass_ticket=USSEyd5y7N2ejqbNx89EA22r2cAuFWiG5M%252BHcswHTXUy7aMIu4fbMpfEtLrLp7Hu&opcode=2&scene=1&username=@72c15f7ccf676b86ee6a89ed6cdd8e41> through a patch which Seems related to python2. cgcs-root/stx/integ/virt/libvirt-python/centos/meta_patches$ 0002-Build-python-wheel.patch It needs libvirt-python whl for python3, right? Do we have plan to upgrade libvirt-python to 6.0.0? Any comment? Thanks a lot! Zhipeng From: Penney, Don <Don.Penney@windriver.com<mailto:Don.Penney@windriver.com>> Sent: 2020年2月27日 0:25 To: YuChengDe <yu.chengde@99cloud.net<mailto:yu.chengde@99cloud.net>>; starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Subject: Re: [Starlingx-discuss] Failed when build StarlingX docker images I was able to successfully build stx-nova image in master with no changes. Do you have any customizations in your environment? From: Penney, Don [mailto:Don.Penney@windriver.com] Sent: Wednesday, February 26, 2020 10:38 AM To: YuChengDe; starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Subject: Re: [Starlingx-discuss] Failed when build StarlingX docker images I guess the upstream nova has updated its requirements.txt to drop support for python2 completely. Al has started digging into the work required to move our image builds to python3, but it’s going to be a multi-day effort. We’re checking to see if there’s a temporary workaround to get this build working in the short term. From: YuChengDe [mailto:yu.chengde@99cloud.net] Sent: Wednesday, February 26, 2020 5:42 AM To: starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Subject: [Starlingx-discuss] Failed when build StarlingX docker images Hi: I have some problem as building the StarlingX docker image. When I built stx-nova through build-stx-base.sh, the process was stuck, and leave the error log as below. Please offer me some suggestions. Many thanks. ... dirname /opt/loci/scripts/install.sh /opt/loci/scripts/pip_install.sh /tmp/heat pycrypto pylint packages='/tmp/heat pycrypto pylint' pip install --no-cache-dir --only-binary :all: --no-compile -c /tmp/wheels/upper-constraints.txt --find-links /tmp/wheels/ /tmp/heat pycrypto pylint DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support ... Ignoring sphinxcontrib-applehelp: markers 'python_version == "3.6"' don't match your environment Ignoring sphinxcontrib-applehelp: markers 'python_version == "3.7"' don't match your environment Ignoring scikit-learn: markers 'python_version == "3.6"' don't match your environment Ignoring scikit-learn: markers 'python_version == "3.7"' don't match your environment Looking in links: /tmp/wheels/ Processing /tmp/nova ERROR: Package 'nova' requires a different Python: 2.7.5 not in '>=3.6' command '/bin/sh -c /opt/loci/scripts/install.sh' returned a non-zero code: 1 [Image removed by sender.] -- —————————————————————————————I 九州云信息科技有限公司 99CLOUD Inc. 于成德 产品开发部 邮箱(Email): yu.chengde@99cloud.net<mailto:yu.chengde@99cloud.net> 手机(Mobile): 13816965096 地址(Addr): 上海市局门路427号1号楼206 Room 206, Bldg 1, No.427 JuMen Road, ShangHai, China 网址(Site): http://www.99cloud.net<http://www.99cloud.net/>
