+ starlingx-discuss@lists.starlingx.io

 

From: Shivashankara Belur, Nidhi <nidhi.shivashankara.belur@intel.com>
Sent: Wednesday, July 13, 2022 4:06 PM
To: Little, Scott <Scott.Little@windriver.com>; Khalil, Ghada <Ghada.Khalil@windriver.com>; Sun, Austin <austin.sun@intel.com>; Waines, Greg <Greg.Waines@windriver.com>; Qi, Mingyuan <mingyuan.qi@intel.com>; Liu, ZhipengS <zhipengs.liu@intel.com>; Burla, Balendu <balendu.burla@intel.com>; Yu, Chengde <chengde.yu@intel.com>; Panech, Davlet <Davlet.Panech@windriver.com>
Cc: Webster, Steven <Steven.Webster@windriver.com>; Thothadri, Ram <Ram.Thothadri@windriver.com>; Melnick, Ruth <Ruth.Melnick@windriver.com>; Jones, Bruce E <bruce.e.jones@intel.com>; Nowell, Anthony <Anthony.Nowell@windriver.com>
Subject: RE: Looking for StarlingX Build Environment help for INTEL Team

 

[Please note: This e-mail is from an EXTERNAL e-mail address]

Hello Davlet,

 

After yesterday’s call, I tried building the sriov-fec-operator package again, the BuildRequires as in the dependency packages chartmuseum, helm and openstack-helm-infra in the application spec file was commented out, therefore it did not show any errors. I rebuilt the package after adding these deps back and still shows the same error. Please refer to the same build logs that I shared in my previous email.

 

Mouli and I discussed after the call yesterday and decided that we want to try bring up build system with centos as the host OS, then try to build packages to see if that makes any difference.

@Panech, Davlet Can you please share any instructions on build system bring up steps for CentOS. Currently we are running this on a Ubuntu 16.04 as per the starlingx official build system documentation. We would like to try this in parallel while working on debugging build issues on ubuntu.

 

Thank you,

Nidhi

 

From: Shivashankara Belur, Nidhi
Sent: Monday, July 11, 2022 1:53 PM
To: Scott Little <scott.little@windriver.com>; Khalil, Ghada <ghada.khalil@windriver.com>; Sun, Austin <austin.sun@intel.com>; Waines, Greg <Greg.waines@windriver.com>; Qi, Mingyuan <mingyuan.qi@intel.com>; Liu, ZhipengS <zhipengs.liu@intel.com>; Burla, Balendu <balendu.burla@intel.com>; Yu, Chengde <chengde.yu@intel.com>; Panech, Davlet <Davlet.Panech@windriver.com>
Cc: Webster, Steven <steven.webster@windriver.com>; Ram.Thothadri <Ram.Thothadri@windriver.com>; ruth.melnick <ruth.melnick@windriver.com>; Jones, Bruce E <bruce.e.jones@intel.com>; Nowell, Anthony <Anthony.Nowell@windriver.com>
Subject: RE: Looking for StarlingX Build Environment help for INTEL Team

 

Hi Davlet,

 

I’m sending build logs from last Friday that you requested.

After adding the missing packages, there are still some errors and build-pkgs fails. I executed the build-pkgs a few times and collected multiple logs, attached in this email.

 

  1. build-pkgs-serial stx-sriov-fec-operator-helm
  2. build-pkgs-serial chartmuseum helm openstack-helm-infra stx-sriov-fec-operator-helm

 

We will definitely need to schedule another debug session for this week, I suggest tomorrow between 9 AM – 11 AM (PST), which is 12 PM – 2 PM (EST). I’ll send a meeting invite, please let me know your availability.

 

Regards,

Nidhi

 

From: Shivashankara Belur, Nidhi
Sent: Wednesday, July 6, 2022 12:14 PM
To: Scott Little <scott.little@windriver.com>; Khalil, Ghada <ghada.khalil@windriver.com>; Sun, Austin <austin.sun@intel.com>; Waines, Greg <Greg.waines@windriver.com>; Qi, Mingyuan <mingyuan.qi@intel.com>; Liu, ZhipengS <zhipengs.liu@intel.com>; Burla, Balendu <balendu.burla@intel.com>; Yu, Chengde <chengde.yu@intel.com>; Panech, Davlet <Davlet.Panech@windriver.com>
Cc: Webster, Steven <steven.webster@windriver.com>; Ram.Thothadri <Ram.Thothadri@windriver.com>; ruth.melnick <ruth.melnick@windriver.com>; Jones, Bruce E <bruce.e.jones@intel.com>; Nowell, Anthony <Anthony.Nowell@windriver.com>
Subject: RE: Looking for StarlingX Build Environment help for INTEL Team

 

I did run just build-pkgs, before attempting with the package name, it also fails building all the pkgs due to same error.

I see the following errors in every single package build.

 

  1. ERROR: Command failed:

19:03:30  # mount -n -t tmpfs -o mode=0755 -o nr_inodes=0 -o size=11g mock_chroot_tmpfs /localdisk/loadbuild/stx_builder/fec-operator/std/mock/b3/root

  1. ERROR: Command failed:

19:03:29  # /usr/bin/yum --installroot /localdisk/loadbuild/stx_builder/fec-operator/std/mock/b0/root/ --releasever 7 install @buildsys-build pigz lbzip2 bash yum python3

 

This 2nd error is followed by below messages, which suggests to me that the mirror is not accessible from my lab maybe due to proxy?

19:03:29 Failed to set locale, defaulting to C

19:03:29 http://127.0.0.1:8088/localdisk/loadbuild/stx_builder/fec-operator/std/results/stx_builder-fec-operator-4.0-std/repodata/$

epomd.xml: [Errno 14] HTTP Error 403 - Forbidden

19:03:29 Trying other mirror.

19:03:29 To address this issue please refer to the below wiki article

19:03:29

19:03:29 https://wiki.centos.org/yum-errors

19:03:29

19:03:29 If above article doesn't help to resolve this issue please use https://bugs.centos.org/.

19:03:29

19:03:29 http://127.0.0.1:8088/localdisk/loadbuild/stx_builder/fec-operator/std/rpmbuild/SRPMS/repodata/repomd.xml: [Errno 14] HTT$

Error 403 - Forbidden

19:03:29 Trying other mirror.

19:03:29 http://127.0.0.1:8088/localdisk/designer/stx_builder/fec-operator/cgcs-root/centos-repo/Binary/repodata/repomd.xml: [Errno

14] HTTP Error 403 - Forbidden

19:03:29 Trying other mirror.

19:03:29  One of the configured repositories failed (Stx-Centos-7-Distro),

19:03:29  and yum doesn't have enough cached data to continue. At this point the only

19:03:29  safe thing yum can do is fail. There are a few ways to work "fix" this:

19:03:29

19:03:29      1. Contact the upstream for the repository and get them to fix the problem.

19:03:29

19:03:29      2. Reconfigure the baseurl/etc. for the repository, to point to a working

19:03:29         upstream. This is most often useful if you are using a newer

19:03:29         distribution release than is supported by the repository (and the

19:03:29         packages for the previous distribution release still work).

19:03:29

19:03:29      3. Run the command with the repository temporarily disabled

19:03:29             yum --disablerepo=StxCentos7Distro ...

19:03:29

19:03:29      4. Disable the repository permanently, so yum won't use it by default. Yum

19:03:29         will then just ignore the repository until you permanently enable it

19:03:29         again or use --enablerepo for temporary usage:

19:03:29

19:03:29             yum-config-manager --disable StxCentos7Distro

19:03:29         or

19:03:29             subscription-manager repos --disable=StxCentos7Distro

19:03:29

19:03:29      5. Configure the failing repository to be skipped, if it is unavailable.

19:03:29         Note that yum will try to contact the repo. when it runs most commands,

19:03:29         so will have to try and fail each time (and thus. yum will be be much

19:03:29         slower). If it is a very temporary problem though, this is often a nice

19:03:29         compromise:

19:03:29

19:03:29             yum-config-manager --save --setopt=StxCentos7Distro.skip_if_unavailable=true

19:03:29

19:03:29 failure: repodata/repomd.xml from StxCentos7Distro: [Errno 256] No more mirrors to try.

19:03:29 http://127.0.0.1:8088/localdisk/designer/stx_builder/fec-operator/cgcs-root/centos-repo/Binary/repodata/repomd.xml: [Errno

14] HTTP Error 403 - Forbidden

19:03:29

 

Regards,

Nidhi

From: Scott Little <scott.little@windriver.com>
Sent: Wednesday, July 6, 2022 7:09 AM
To: Shivashankara Belur, Nidhi <nidhi.shivashankara.belur@intel.com>; Khalil, Ghada <ghada.khalil@windriver.com>; Sun, Austin <austin.sun@intel.com>; Waines, Greg <Greg.waines@windriver.com>; Qi, Mingyuan <mingyuan.qi@intel.com>; Liu, ZhipengS <zhipengs.liu@intel.com>; Burla, Balendu <balendu.burla@intel.com>; Yu, Chengde <chengde.yu@intel.com>; Panech, Davlet <Davlet.Panech@windriver.com>
Cc: Webster, Steven <steven.webster@windriver.com>; Ram.Thothadri <Ram.Thothadri@windriver.com>; ruth.melnick <ruth.melnick@windriver.com>; Jones, Bruce E <bruce.e.jones@intel.com>; Nowell, Anthony <Anthony.Nowell@windriver.com>
Subject: Re: Looking for StarlingX Build Environment help for INTEL Team

 

Did you do a full 'build-pkgs' before attempting 'build-pkgs <pkg-name>'   ?

Your first build should build everything.

Scott

 

On 2022-07-05 8:23 p.m., Shivashankara Belur, Nidhi wrote:

[Please note: This e-mail is from an EXTERNAL e-mail address]

Hi Scott,

 

Quick update from today’s build,

  • Resolved all issues related to “DOS” formatting.
  • build-pkgs moved forward and able to fix some mistakes in our operator source code .spec file

00:05:56 ==== Update repodata complete =====

00:05:56

00:05:56 Successfully built 1 packages:

00:05:56    stx-sriov-fec-operator-helm

00:05:56

00:05:56 Compiled src.rpm's can be found here: /localdisk/loadbuild/stx_builder/fec-operator/std/rpmbuild/SRPMS

 

######## Wed Jul  6 00:05:56 UTC 2022: build-srpm-parallel --std was successful

  • build-pkgs of our fec-operator now fails at the same step as cert-manager.

00:06:16 b0: Mock Version: 1.4.16

00:06:16 b0: INFO: Mock Version: 1.4.16

00:06:16 b0: Start: yum install

00:06:16 b0: ERROR: Command failed:

00:06:16 b0:  # /usr/bin/yum --installroot /localdisk/loadbuild/stx_builder/fec-operator/std/mock/b0/root/ --relea

sever 7 install @buildsys-build pigz lbzip2 bash yum python3

00:06:16 b0: Failed to set locale, defaulting to C

00:06:16 b0: http://127.0.0.1:8088/localdisk/loadbuild/stx_builder/fec-operator/std/results/stx_builder-fec-operat

or-4.0-std/repodata/repomd.xml: [Errno 14] HTTP Error 403 - Forbidden

00:06:16 b0: Trying other mirror.

00:06:16 b0: To address this issue please refer to the below wiki article

00:06:16 b0:

00:06:16 b0: https://wiki.centos.org/yum-errors

00:06:16 b0:

00:06:16 b0: If above article doesn't help to resolve this issue please use https://bugs.centos.org/.

00:06:16 b0:

00:06:16 b0: http://127.0.0.1:8088/localdisk/loadbuild/stx_builder/fec-operator/std/rpmbuild/SRPMS/repodata/repomd

.xml: [Errno 14] HTTP Error 403 - Forbidden

00:06:16 b0: Trying other mirror.

00:06:16 b0: http://127.0.0.1:8088/localdisk/designer/stx_builder/fec-operator/cgcs-root/centos-repo/Binary/repoda

ta/repomd.xml: [Errno 14] HTTP Error 403 - Forbidden

00:06:16 b0: Trying other mirror.

00:06:16 b0: failure: repodata/repomd.xml from StxCentos7Distro: [Errno 256] No more mirrors to try.

00:06:16 b0: http://127.0.0.1:8088/localdisk/designer/stx_builder/fec-operator/cgcs-root/centos-repo/Binary/repoda

ta/repomd.xml: [Errno 14] HTTP Error 403 - Forbidden

00:06:16 b0:

00:06:16 mock_update_or_init: out

00:06:16

 

                             00:06:34 ===== wait for last jobs in iteration 1 to complete =====

00:06:34 End build on 'b0': /localdisk/loadbuild/stx_builder/fec-operator/std/rpmbuild/SRPMS/stx-sriov-fec-operato

r-helm-1.0-2.tis.src.rpm

00:06:34 Error building stx-sriov-fec-operator-helm-1.0-2.tis.src.rpm on 'b0'.

00:06:34 Will try to build again (if some other package will succeed).

00:06:34 ===== iteration 1 complete =====

00:06:34 failed=['/localdisk/loadbuild/stx_builder/fec-operator/std/rpmbuild/SRPMS/build-info-1.0-4.tis.src.rpm',

'/localdisk/loadbuild/stx_builder/fec-operator/std/rpmbuild/SRPMS/stx-sriov-fec-operator-helm-1.0-2.tis.src.rpm']

00:06:34 to_be_built=['/localdisk/loadbuild/stx_builder/fec-operator/std/rpmbuild/SRPMS/build-info-1.0-4.tis.src.r

pm', '/localdisk/loadbuild/stx_builder/fec-operator/std/rpmbuild/SRPMS/stx-sriov-fec-operator-helm-1.0-2.tis.src.r

pm']

00:06:34

00:06:34 *** Build Failed ***

00:06:34 Tried 1 times - following pkgs could not be successfully built:

00:06:34 *** Build Failed ***

00:06:34 /localdisk/loadbuild/stx_builder/fec-operator/std/rpmbuild/SRPMS/build-info-1.0-4.tis.src.rpm

00:06:34 /localdisk/loadbuild/stx_builder/fec-operator/std/rpmbuild/SRPMS/stx-sriov-fec-operator-helm-1.0-2.tis.sr

c.rpm

00:06:34

00:06:34

00:06:34 Results out to: /localdisk/loadbuild/stx_builder/fec-operator/std/results/stx_builder-fec-operator-4.0-st

d

00:06:34

00:06:34 Pkgs built: 0

00:06:34 dirname: missing operand

00:06:34 Try 'dirname --help' for more information.

                             00:06:34 Recreate repodata

00:06:35

00:06:35 Failed to build packages:  stx-sriov-fec-operator-helm-1.0-2.tis.src.rpm  build-info-1.0-4.tis.src.rpm

 

######## Wed Jul  6 00:06:36 UTC 2022: build-rpm-parallel --std failed with rc=1

 

Wed Jul  6 00:06:36 UTC 2022: build-rpm-parallel --std failed with rc=1

 

 

Can we have another live debug session tomorrow to see further into this issue?

 

Thank you,

Nidhi  

 

From: Scott Little <scott.little@windriver.com>
Sent: Tuesday, July 5, 2022 12:52 PM
To: Shivashankara Belur, Nidhi <nidhi.shivashankara.belur@intel.com>; Khalil, Ghada <ghada.khalil@windriver.com>; Sun, Austin <austin.sun@intel.com>; Waines, Greg <Greg.waines@windriver.com>; Qi, Mingyuan <mingyuan.qi@intel.com>; Liu, ZhipengS <zhipengs.liu@intel.com>; Burla, Balendu <balendu.burla@intel.com>; Yu, Chengde <chengde.yu@intel.com>; Panech, Davlet <Davlet.Panech@windriver.com>
Cc: Webster, Steven <steven.webster@windriver.com>; Ram.Thothadri <Ram.Thothadri@windriver.com>; ruth.melnick <ruth.melnick@windriver.com>; Jones, Bruce E <bruce.e.jones@intel.com>; Nowell, Anthony <Anthony.Nowell@windriver.com>
Subject: Re: Looking for StarlingX Build Environment help for INTEL Team

 

Within the build container, the first place it would have been copied is $HOME/buildrc .  That file initializes your environment  variables within the container.  It should remain confined to the environment variables after that as far as I can recall.

 

That file should be reset if you clean and recreate the build container.

 

If that procedure doesn't fix $HOME/buildrc, perhaps deleting and recreating the image from which the container is launched might also be required.  Let me know if you find that to be the case.

 

 

 

 

On 2022-07-05 15:05, Shivashankara Belur, Nidhi wrote:

[Please note: This e-mail is from an EXTERNAL e-mail address]

Hi Scott,

 

Thank you for providing the feedback on remaining issues. It was a federal holiday yesterday and I’m trying the fixes you mentioned right now. I’ll send another update once it completes. If any issues still remain, we would need a follow up call tomorrow.

However, I still see this error pops up, even after applying the ff as unix on localrc

              find: ‘/localdisk/designer/stx_builder/fec-operator/cgcs-root/../stx-tools/centos-mirror-tools/config/centos/flock

\r’: No such file or directory

 

Not quite sure how to find which other file could have been edited on windows.

 

Regards,

Nidhi

 

From: Scott Little <scott.little@windriver.com>
Sent: Tuesday, July 5, 2022 10:20 AM
To: Shivashankara Belur, Nidhi <nidhi.shivashankara.belur@intel.com>; Khalil, Ghada <ghada.khalil@windriver.com>; Sun, Austin <austin.sun@intel.com>; Waines, Greg <Greg.waines@windriver.com>; Qi, Mingyuan <mingyuan.qi@intel.com>; Liu, ZhipengS <zhipengs.liu@intel.com>; Burla, Balendu <balendu.burla@intel.com>; Yu, Chengde <chengde.yu@intel.com>; Panech, Davlet <Davlet.Panech@windriver.com>
Cc: Webster, Steven <steven.webster@windriver.com>; Ram.Thothadri <Ram.Thothadri@windriver.com>; ruth.melnick <ruth.melnick@windriver.com>; Jones, Bruce E <bruce.e.jones@intel.com>; Nowell, Anthony <Anthony.Nowell@windriver.com>
Subject: Re: Looking for StarlingX Build Environment help for INTEL Team

 

Do you need a follow up call ?

 

Scott

 

 

On 2022-07-01 17:39, Shivashankara Belur, Nidhi wrote:

[Please note: This e-mail is from an EXTERNAL e-mail address]

Hi Ghada,

 

Please find below detail description with error logs of all issues along with 1 line “current status” for each issue.

 

Build system errors:

1.       Download source code repositories, step 3: synchronize the repository (repo sync -j`nproc`). This step fails to download yocto linux repository, therefore fails.

Resolved: Replaced “git” with “https” in the manifest default.xml file, another pointer is to run the command “repo sync” without any arguments as “-j” has some known issues.

Resolution was given by WR team on 6/29/22 live debug session (attendees: Davelet, Scott, Steven)

 

2.       Download Packages, download_mirror.sh had multiple failures “permission denied” due to a root privilege docker access issue.

Resolved: Running the following command, adding env variables for sudo as well as proxies.

       cd  $MY_REPO_ROOT_DIR/stx-tools/centos-mirror-tools && sudo bash -c "source env_sudo.sh && bash download_mirror.sh"

       cat env_sudo.sh

export MY_REPO_ROOT_DIR=/localdisk/designer/stx_builder/fec-operator

export MY_REPO=/localdisk/designer/stx_builder/fec-operator/cgcs-root

export MY_WORKSPACE=/localdisk/loadbuild/stx_builder/fec-operator

export http_proxy=http://proxy-chain.intel.com:911

export https_proxy=http://proxy-chain.intel.com:911

Resolution was given by WR team on 6/29/22 live debug session (attendees: Davelet, Scott, Steven)

 

3.       Download packages in the above step had some failures, resolved it by manually downloading and moving the following rpms to “downloads” (Not verified by WR)

[stx_builder@4df2aa3dafa0 centos-mirror-tools]$ cat logs/*_failmoved_*.log

fail_move_srpm:bash-4.2.46-34.el7.x86_64.rpm

fail_move_srpm:linuxptp-2.0-2.el7.src.rpm

fail_move_srpm:net-tools-2.0-0.24.20131004git.el7.src.rpm

fail_move_srpm:openldap-2.4.44-20.el7.tis.9.x86_64.rpm

fail_move_srpm:openldap-2.4.44-20.el7.x86_64.rpm

fail_move_srpm:openssh-7.4p1-21.el7.src.rpm

fail_move_srpm:puppet-firewall-1.8.2-2.2f892e9git.el7.src.rpm

fail_move_srpm:python-2.7.5-89.el7.src.rpm

fail_move_srpm:rabbitmq-server-3.6.5-1.el7.src.rpm

fail_move_srpm:setup-2.8.71-10.el7.src.rpm

fail_move_srpm:sudo-1.8.23-10.el7_9.1.x86_64.rpm

 

Status: Need help from WR to verify that this resolution is valid.

 

4.       Populate downloads step shows an error

[stx_builder@4df2aa3dafa0 cgcs-root]$ ln -s /import/mirrors/CentOS/stx/CentOS/downloads/ $MY_REPO/stx/

ln: failed to create symbolic link ‘/localdisk/designer/stx_builder/fec-operator/cgcs-root/stx/downloads’: File exists

 

[stx_builder@4df2aa3dafa0 cgcs-root]$ populate_downloads.sh /import/mirrors/CentOS/stx/CentOS/

mirror_dir=/import/mirrors/CentOS/stx/CentOS/

config_dir=/localdisk/designer/stx_builder/fec-operator/cgcs-root/../stx-tools/centos-mirror-tools/config

distro=centos

layer=all

find: ‘/localdisk/designer/stx_builder/fec-operator/cgcs-root/../stx-tools/centos-mirror-tools/config/centos/flock

\r’: No such file or directory

find: ‘/localdisk/designer/stx_builder/fec-operator/cgcs-root/../stx-tools/centos-mirror-tools/config/centos/flock

\r’: No such file or directory

 

Status: Need help from WR to resolve this issue.

 

5.       Build packages, updating symbolic links shows similar error:

[stx_builder@4df2aa3dafa0 toCOPY]$ bash generate-centos-repo.sh /import/mirrors/CentOS/stx/CentOS/    

mirror_dir=/import/mirrors/CentOS/stx/CentOS/

config_dir=/localdisk/designer/stx_builder/fec-operator/cgcs-root/../stx-tools/centos-mirror-tools/config

distro=centos

layer=all

 

layer_pkg_urls=

 

layer_image_inc_urls=

 

layer_wheels_inc_urls=

 

find: ‘/localdisk/designer/stx_builder/fec-operator/cgcs-root/../stx-tools/centos-mirror-tools/config/centos/flock\r’: No such file

or directory

find: ‘/localdisk/designer/stx_builder/fec-operator/cgcs-root/../stx-tools/centos-mirror-tools/config/centos/flock\r’: No such file

or directory

find: ‘/localdisk/designer/stx_builder/fec-operator/cgcs-root/../stx-tools/centos-mirror-tools/config/centos/flock\r’: No such file

or directory

find: ‘/localdisk/designer/stx_builder/fec-operator/cgcs-root/../stx-tools/centos-mirror-tools/config/centos/flock\r’: No such file

or directory

rpm_lst_files=rpms_3rdparties.lst rpms_centos3rdparties.lst rpms_centos.lst

rpm_lst_files_rt=

layer_dirs=

 

Status: Need help from WR to resolve this issue.

 

6.       build-pkgs command fails for all packages, probably due to dependency on previous steps.

Error logs:

[stx_builder@4df2aa3dafa0 stx]$ build-pkgs stx-sriov-fec-operator-helm

No changes for stx projects

build-pkgs-parallel stx-sriov-fec-operator-helm

Reading: /localdisk/designer/stx_builder/fec-operator/cgcs-root/build-data/build_avoidance_source

BUILD_AVOIDANCE_DIR=

BUILD_AVOIDANCE_HOST=

BUILD_AVOIDANCE_USR=

BUILD_AVOIDANCE_FLAG=0

CLEAN_FLAG=0

EDIT_FLAG=0

find: ‘/localdisk/designer/stx_builder/fec-operator/cgcs-root/stx/app-sriov-fec-operator/stx-sriov-fec-operator-helm\r/centos/’: No

such file or directory

find: ‘/localdisk/designer/stx_builder/fec-operator/cgcs-root/stx/app-sriov-fec-operator/stx-sriov-fec-operator-helm\r/centos/’: No

such file or directory

TARGETS_STD=

TARGETS_RT=

TARGETS_INSTALLER=

TARGETS_CONTAINERS=

EXTRA_ARGS_COMMON=''

EXTRA_ARGS_SRPM=''

EXTRA_ARGS_RPM=''

TARGETS=' stx-sriov-fec-operator-helm '

TARGETS_STD='  '

TARGETS_RT='  '

TARGETS_INSTALLER='  '

TARGETS_CONTAINERS='  '

TARGETS_MISC='  stx-sriov-fec-operator-helm '

Capture build context

Launching std build, logging to /localdisk/loadbuild/stx_builder/fec-operator/build-std.log

 

######## Fri Jul  1 21:10:15 UTC 2022: Launching build-srpms-parallel --std

 

BUILD_AVOIDANCE_URL=

/localdisk/designer/stx_builder/fec-operator/cgcs-root/build-tools/build-srpms-parallel --std        stx-sriov-fec-operator-helm

MAX_WORKERS=8

CLEAN_FLAG=0

21:10:17 build-srpms-parallel --std stx-sriov-fec-operator-helm

21:10:17 make: stx-sriov-fec-operator-helm

21:10:17 removed ‘/localdisk/loadbuild/stx_builder/fec-operator/std/tmp/SRPM_REBUILT_LIST_6’

' in file '/localdisk/designer/stx_builder/fec-operator/cgcs-root/stx/app-sriov-fec-operator/centos_pkg_dirs'

21:10:29 ==== Update repodata =====

21:10:33 Directory walk started

21:10:33 Directory walk done - 302 packages

21:10:33 Temporary output repo path: /localdisk/loadbuild/stx_builder/fec-operator/std/rpmbuild/SRPMS/.repodata/

21:10:33 Preparing sqlite DBs

21:10:33 Pool started (with 5 workers)

21:10:33 Pool finished

21:10:33 ==== Update repodata complete =====

21:10:33

21:10:33 No packages required a rebuild

21:10:33

21:10:33 ERROR:  (1600): failed to resolve build targets: stx-sriov-fec-operator-helm

 

######## Fri Jul  1 21:10:33 UTC 2022: build-srpm-parallel --std failed with rc=1

 

Fri Jul  1 21:10:33 UTC 2022: build-srpm-parallel --std failed with rc=1

 

Status: Need help from WR to resolve this issue.

 

@Sun, Austin I got your email on proposed changes for build-system issues. I tried a few of them applicable for us.

  1. We have already passed noProxy env variable for docker container
  2. Download_mirror.sh command was executed as “sudo”
  3. "get_sources.sh", is not executable. Fixed this locally by changing the file mode.

 

Question: I didn’t quite understand these two changes,

feat: Modify layer build process (Ibded97db) · Gerrit Code Review (opendev.org) ; didn’t think this was applicable to us since we are not doing layer build. Do we need to do layer build ?

fix: drop the relative gopath (I950184f0) · Gerrit Code Review (opendev.org) ; I applied this fix locally on my system. Is this a mandatory fix ?

 

Regards,

Nidhi

 

From: Khalil, Ghada <Ghada.Khalil@windriver.com>
Sent: Wednesday, June 29, 2022 6:32 AM
To: Sun, Austin <austin.sun@intel.com>; Waines, Greg <Greg.waines@windriver.com>; Qi, Mingyuan <mingyuan.qi@intel.com>; Liu, ZhipengS <zhipengs.liu@intel.com>; Burla, Balendu <balendu.burla@intel.com>; Shivashankara Belur, Nidhi <nidhi.shivashankara.belur@intel.com>; Yu, Chengde <chengde.yu@intel.com>; Little, Scott <Scott.Little@windriver.com>; Panech, Davlet <Davlet.Panech@windriver.com>
Cc: Webster, Steven <steven.webster@windriver.com>; Ram.Thothadri <Ram.Thothadri@windriver.com>; ruth.melnick <ruth.melnick@windriver.com>; Jones, Bruce E <bruce.e.jones@intel.com>; Nowell, Anthony <Anthony.Nowell@windriver.com>
Subject: RE: Looking for StarlingX Build Environment help for INTEL Team

 

There was a live zoom session yesterday with @Shivashankara Belur, Nidhi and @Panech, Davlet / @Little, Scott to investigate the build failures as well.  Can someone provide a readout of the meeting please so that everyone is on the same page?

 

Thanks,

Ghada

 

From: Sun, Austin <austin.sun@intel.com>
Sent: Wednesday, June 29, 2022 3:41 AM
To: Waines, Greg <Greg.Waines@windriver.com>; Qi, Mingyuan <mingyuan.qi@intel.com>; Liu, ZhipengS <zhipengs.liu@intel.com>; Burla, Balendu <balendu.burla@intel.com>; Shivashankara Belur, Nidhi <nidhi.shivashankara.belur@intel.com>; Yu, Chengde <chengde.yu@intel.com>
Cc: Khalil, Ghada <Ghada.Khalil@windriver.com>; Webster, Steven <Steven.Webster@windriver.com>; Thothadri, Ram <Ram.Thothadri@windriver.com>; Melnick, Ruth <Ruth.Melnick@windriver.com>; Jones, Bruce E <bruce.e.jones@intel.com>; Little, Scott <Scott.Little@windriver.com>; Panech, Davlet <Davlet.Panech@windriver.com>; Nowell, Anthony <Anthony.Nowell@windriver.com>
Subject: RE: Looking for StarlingX Build Environment help for INTEL Team

 

[Please note: This e-mail is from an EXTERNAL e-mail address]

Hi Greg and Scott :

Chant has been working on build latest starlingx with layer-build in intel environment and uploaded  several patches to address build issues.

With these patches, Chant can build iso successfully .  would you like review these patches ?

 

https://review.opendev.org/c/starlingx/tools/+/846573

https://review.opendev.org/c/starlingx/tools/+/846846

https://review.opendev.org/c/starlingx/containers/+/847231

https://review.opendev.org/c/starlingx/tools/+/847066

https://review.opendev.org/c/starlingx/tools/+/847061

 

@Burla, Balendu and @Shivashankara Belur, Nidhi: you can cherry-pick those patches and try build in your environment .

 

Thanks.

BR
Austin Sun.

 

From: Waines, Greg <Greg.Waines@windriver.com>
Sent: Friday, June 10, 2022 6:47 PM
To: Sun, Austin <austin.sun@intel.com>; Qi, Mingyuan <mingyuan.qi@intel.com>; Liu, ZhipengS <zhipengs.liu@intel.com>; Burla, Balendu <balendu.burla@intel.com>; Shivashankara Belur, Nidhi <nidhi.shivashankara.belur@intel.com>
Cc: Khalil, Ghada <ghada.khalil@windriver.com>; Webster, Steven <steven.webster@windriver.com>; Ram.Thothadri <Ram.Thothadri@windriver.com>; ruth.melnick <ruth.melnick@windriver.com>; Jones, Bruce E <bruce.e.jones@intel.com>; Little, Scott <Scott.Little@windriver.com>; Panech, Davlet <Davlet.Panech@windriver.com>; Nowell, Anthony <Anthony.Nowell@windriver.com>
Subject: RE: Looking for StarlingX Build Environment help for INTEL Team

 

thanks very much Austin.

Greg.

 

From: Sun, Austin <austin.sun@intel.com>
Sent: Thursday, June 9, 2022 11:39 PM
To: Waines, Greg <Greg.Waines@windriver.com>; Qi, Mingyuan <mingyuan.qi@intel.com>; Liu, ZhipengS <zhipengs.liu@intel.com>; Burla, Balendu <balendu.burla@intel.com>; Shivashankara Belur, Nidhi <nidhi.shivashankara.belur@intel.com>
Cc: Khalil, Ghada <Ghada.Khalil@windriver.com>; Webster, Steven <Steven.Webster@windriver.com>; Thothadri, Ram <Ram.Thothadri@windriver.com>; Melnick, Ruth <Ruth.Melnick@windriver.com>; Jones, Bruce E <bruce.e.jones@intel.com>; Little, Scott <Scott.Little@windriver.com>; Panech, Davlet <Davlet.Panech@windriver.com>; Nowell, Anthony <Anthony.Nowell@windriver.com>
Subject: RE: Looking for StarlingX Build Environment help for INTEL Team

 

[Please note: This e-mail is from an EXTERNAL e-mail address]

Hi Greg:

OK. @Burla, Balendu, and @Shivashankara Belur, Nidhi, please contact us , we can syn-cup internally .

 

Thanks.

BR
Austin Sun.

 

From: Waines, Greg <Greg.Waines@windriver.com>
Sent: Thursday, June 9, 2022 9:41 PM
To: Qi, Mingyuan <mingyuan.qi@intel.com>; Sun, Austin <austin.sun@intel.com>; Liu, ZhipengS <zhipengs.liu@intel.com>
Cc: Burla, Balendu <balendu.burla@intel.com>; Shivashankara Belur, Nidhi <nidhi.shivashankara.belur@intel.com>; Khalil, Ghada <ghada.khalil@windriver.com>; Webster, Steven <steven.webster@windriver.com>; Ram.Thothadri <Ram.Thothadri@windriver.com>; ruth.melnick <ruth.melnick@windriver.com>; Jones, Bruce E <bruce.e.jones@intel.com>; Little, Scott <Scott.Little@windriver.com>; Panech, Davlet <Davlet.Panech@windriver.com>; Nowell, Anthony <Anthony.Nowell@windriver.com>
Subject: Looking for StarlingX Build Environment help for INTEL Team

 

Mingyuan, Austin, Zhipeng,   (Intel StarlingX folks)

 

Balendu (Mouli) Burla and Nidhi Shivashankara Belur are new members of StarlingX from the Intel Networking(?) org who are in the process of contributing a kubernetes operator for configuring FEC in a couple of intel hardware devices.

 

They are having difficulty setting up their StarlingX Build Environment and getting it to work.

 

Is there any chance that you guys (who have working StarlingX Build Environments inside Intel) could help Mouli and Nidhi out in setting up their environment ?

 

let me know,

thanks a lot,

Greg.

 

 

From: Waines, Greg
Sent: Tuesday, June 7, 2022 6:58 AM
To: Burla, Balendu <balendu.burla@intel.com>; Nowell, Anthony <Anthony.Nowell@windriver.com>
Cc: Shivashankara Belur, Nidhi <nidhi.shivashankara.belur@intel.com>; Khalil, Ghada <Ghada.Khalil@windriver.com>; Webster, Steven <Steven.Webster@windriver.com>; Thothadri, Ram <Ram.Thothadri@windriver.com>; Melnick, Ruth <Ruth.Melnick@windriver.com>; Rasa, Nimalini <Nimalini.Rasa@windriver.com>; Peters, Matt <Matt.Peters@windriver.com>
Subject: RE: sriov-fec-operator plan update ?

 

Thanks Mouli.

 

Anthony ?

I think scott or davlet have provided a bit of help to Mouli on his build-pkg issues … any chance we can get some more direct help from them in order to get over this build issue ?

 

Greg.

 

From: Burla, Balendu <balendu.burla@intel.com>
Sent: Monday, June 6, 2022 6:14 PM
To: Waines, Greg <Greg.Waines@windriver.com>; Shivashankara Belur, Nidhi <nidhi.shivashankara.belur@intel.com>
Cc: Khalil, Ghada <Ghada.Khalil@windriver.com>; Webster, Steven <Steven.Webster@windriver.com>; Thothadri, Ram <Ram.Thothadri@windriver.com>; Melnick, Ruth <Ruth.Melnick@windriver.com>; Rasa, Nimalini <Nimalini.Rasa@windriver.com>; Peters, Matt <Matt.Peters@windriver.com>
Subject: RE: sriov-fec-operator plan update ?

 

[Please note: This e-mail is from an EXTERNAL e-mail address]

Hi Greg,

 

We have tested the fec-operator on StarlingX platform manually using the helm chats. And we have packaged the helm chats as system application with the help of documentation and Teresa for integration.  

 

However, To test that, we need a local build environment and we are struck in brining up the local build environment and looking for help from WR/STX to resolve the build-pkg issues since 4 weeks. Without resolving those build issues, we can not submit a working code.  Although, Nidhi will submit the code for review in a draft status.

 

Assuming the local build issues are resolved, from there on we can try to submit the working code, review, tested in two weeks.

 

Can someone help in resolving our local build-pkg issues? Nidhi can setup a debug call, if that work for your team members.

NOTE: these build-pkg issues are not related to fec-operator, trying to build the default iso image with latest master.

 

Best regards,

Mouli.

 

From: Waines, Greg <Greg.Waines@windriver.com>
Sent: Monday, June 6, 2022 6:12 AM
To: Shivashankara Belur, Nidhi <nidhi.shivashankara.belur@intel.com>; Burla, Balendu <balendu.burla@intel.com>
Cc: Khalil, Ghada <ghada.khalil@windriver.com>; Webster, Steven <steven.webster@windriver.com>; Ram.Thothadri <Ram.Thothadri@windriver.com>; ruth.melnick <ruth.melnick@windriver.com>; Rasa, Nimalini <Nimalini.Rasa@windriver.com>; Peters, Matt <matt.peters@windriver.com>
Subject: RE: sriov-fec-operator plan update ?

 

Thanks for the status update.

 

What I’m looking for though is a committed date for
- code submitted, merged and tested, and

- customer documentation updated.

 

Do you have a forecast for that date ?

 

Greg.

 

From: Shivashankara Belur, Nidhi <nidhi.shivashankara.belur@intel.com>
Sent: Friday, June 3, 2022 2:18 PM
To: Waines, Greg <Greg.Waines@windriver.com>; Burla, Balendu <balendu.burla@intel.com>
Cc: Khalil, Ghada <Ghada.Khalil@windriver.com>; Webster, Steven <Steven.Webster@windriver.com>; Thothadri, Ram <Ram.Thothadri@windriver.com>; Melnick, Ruth <Ruth.Melnick@windriver.com>; Rasa, Nimalini <Nimalini.Rasa@windriver.com>; Peters, Matt <Matt.Peters@windriver.com>
Subject: RE: sriov-fec-operator plan update ?

 

[Please note: This e-mail is from an EXTERNAL e-mail address]

Hi Greg,

 

I am currently blocked by 2 issues,

  1. Build-pkgs tool is failing
  2. System application-apply fails to bring up operator pods.

Pending items:

  1. Build fec-operator docker images for STX – WIP:
    1. Imported pre-built docker images for the operator and pushed it with tag registry.local. But system application-apply fails here.
    2. Include build images into build iso – Not yet started.

 

I have 2 active email threads for these 2 issues. Contact person for build issues is Scott Little, there isn’t much progress here yet. Can you please include a contact who can help resolve this issue through a live debug call ?

 

Contact person for system application-apply failure is Teresa Ho, we are  doing 1:1 debug sessions since yesterday. Some reasons for failure identified, we will meet again on Monday to try some solutions.

 

Regards,

Nidhi

 

From: Waines, Greg <Greg.Waines@windriver.com>
Sent: Monday, May 30, 2022 7:39 AM
To: Burla, Balendu <balendu.burla@intel.com>
Cc: Khalil, Ghada <ghada.khalil@windriver.com>; Webster, Steven <steven.webster@windriver.com>; Ram.Thothadri <Ram.Thothadri@windriver.com>; ruth.melnick <ruth.melnick@windriver.com>; Shivashankara Belur, Nidhi <nidhi.shivashankara.belur@intel.com>; Rasa, Nimalini <Nimalini.Rasa@windriver.com>; Peters, Matt <matt.peters@windriver.com>
Subject: sriov-fec-operator plan update ?

 

Hey Mouli,

 

Wrt your ‘FEC Operator Integration to enable FEC Device Configurability’ ( https://storyboard.openstack.org/#!/story/2009749 ),
are you still tracking to your ‘Code Merge’ (5/27) and ‘Feature Tested’ (6/6) dates in the StarlingX STX7.0 Feature Planning Spreadsheet ( ROW 36 of https://docs.google.com/spreadsheets/d/171PJAu9SykXm9h9Ny2IsZ8YMbhvEwOzuTbISWUMQhiE/edit#gid=1107209846 ) ?

 

Although there is no hard commitment in StarlingX for the delivery of this functionality,
there are hard commitments wrt this functionality being available in Wind River’s commercial product based on StarlingX.

 

I’d like to confirm that your feature delivery dates are still accurate and you are getting all the support you need from the StarlingX projects,

so I can make sure your dates align with Wind River commercial product dates.

 

let me know,

Greg.