[Starlingx-discuss] build-pkgs cannot complete std build >> serial builds

Sun, Austin austin.sun at intel.com
Thu Sep 27 13:02:34 UTC 2018


Hi Erich:
     Sorry ,  which variable was missing ?   are you using same patch https://review.openstack.org/#/c/603983/ ? 

Thanks.
BR
Austin Sun. 

-----Original Message-----
From: Cordoba Malibran, Erich 
Sent: Thursday, September 27, 2018 12:28 PM
To: Sun, Austin <austin.sun at intel.com>; Saul Wold <sgw at linux.intel.com>; starlingx-discuss at lists.starlingx.io
Subject: Re: [Starlingx-discuss] build-pkgs cannot complete std build >> serial builds

Austin, 

This worked for me to solve the deleted srpms, just one variable was missing. 

diff --git a/build-tools/build-srpms-serial b/build-tools/build-srpms-serial index fbdd33c..36cf61b 100755
--- a/build-tools/build-srpms-serial
+++ b/build-tools/build-srpms-serial
@@ -116,6 +116,7 @@ create_output_dirs () {

        # create output dirs
        mkdir -p $MY_BUILD_DIR
+       mkdir -p $BUILD_ROOT/tmp
        mkdir -p $MY_SRC_RPM_BUILD_DIR
        mkdir -p $MY_SRC_RPM_BUILD_DIR/SOURCES
        mkdir -p $MY_SRC_RPM_BUILD_DIR/SPECS @@ -1235,6 +1236,7 @@ done  if [ $ALL -eq 1 ]; then
     echo
     echo "Auditing for obsolete srpms"
+    AUDIT_DIR=$(mktemp -d $MY_WORKSPACE/tmp/$USER-$ME-audit-XXXXXX)
     PACKAGES_CONSIDERED=""
     for GIT_ROOT in $GIT_LIST; do
         for p in $(cat $GIT_ROOT/$PKG_DIRS_FILE 2>> /dev/null); do @@ -1245,7 +1247,7 @@ if [ $ALL -eq 1 ]; then
                     for f in $(find $src_dir/${DISTRO} -name '*.spec' | sort -V); do
                         NAME=$(spec_find_tag Name "$f" 2>> /dev/null)
                         if [ $? -eq 0 ]; then
-                            PACKAGES_CONSIDERED="$PACKAGES_CONSIDERED $NAME"
+                            touch "$AUDIT_DIR/$NAME"
                         fi
                     done
                     if [ -f $src_dir/$SRPM_LIST_PATH ]; then @@ -1279,7 +1281,7 @@ if [ $ALL -eq 1 ]; then
                            if [ -f $ORIG_SRPM_PATH ]; then
                                NAME=$(rpm -q --queryformat '%{NAME}\n' -p $ORIG_SRPM_PATH 2>> /dev/null)
                                if [ $? -eq 0 ]; then
-                                   PACKAGES_CONSIDERED="$PACKAGES_CONSIDERED $NAME"
+                                   touch "$AUDIT_DIR/$NAME"
                                fi
                            fi
                         done

I hope this can helps. 

-Erich


On 9/26/18, 10:39 PM, "Cordoba Malibran, Erich" <erich.cordoba.malibran at intel.com> wrote:

    I sent the first fix : https://review.openstack.org/#/c/605578/
    
    This is just for the wrong variable name that was blocking the script. 
    
    On 9/26/18, 10:11 PM, "Sun, Austin" <austin.sun at intel.com> wrote:
    
        Hi Ghada and Saul:
            Thanks.  I'm working for fuzz patch with Shuicheng now,  I will check build serial issue after "fuzz" task. 
        
        BR
        Austin Sun. 
        
        -----Original Message-----
        From: Saul Wold [mailto:sgw at linux.intel.com] 
        Sent: Thursday, September 27, 2018 10:26 AM
        To: starlingx-discuss at lists.starlingx.io
        Subject: Re: [Starlingx-discuss] build-pkgs cannot complete std build >> serial builds
        
        
        
        On 09/26/2018 06:50 PM, Khalil, Ghada wrote:
        > Erich,
        > Given the failures you are seeing, I re-opened 
        > https://bugs.launchpad.net/starlingx/+bug/1790583
        > (it was still in a Fix Released state even though the original commit 
        > was reverted and a new one is currently in review)
        > 
        > Austin, I see that you have an update pending for this: 
        > https://review.openstack.org/#/c/603983/
        > But according to Erich, it didn't address the build issue. Can you please investigate further?
        > 
        OK, I understand Austin's patch now and gave it my +2.
        
        I am going to try again with a scratch build now the libvirt bits have merged (or at least I think they have merged)
        
        Sau!
        > Thanks,
        > Ghada
        > 
        > -----Original Message-----
        > From: Cordoba Malibran, Erich 
        > [mailto:erich.cordoba.malibran at intel.com]
        > Sent: Wednesday, September 26, 2018 7:43 PM
        > To: Little, Scott; starlingx-discuss at lists.starlingx.io
        > Subject: Re: [Starlingx-discuss] build-pkgs cannot complete std build
        > 
        > Trying to get a serial build avoiding the possible mock problems I encounter with this issue:
        > 
        > 
        > BUILD_AVOIDANCE_URL=
        > /localdisk/designer/jenkins-slave/starlingx/cgcs-root/build-tools/buil
        > d-srpms-serial --std
        > /localdisk/designer/jenkins-slave/starlingx/cgcs-root/build-tools/buil
        > d-srpms-serial: line 33: /build-srpms-common.sh: No such file or 
        > directory
        > CLEAN_FLAG=0
        > 22:26:39 build-srpms-serial --std
        > 22:26:39 make: all
        > 22:26:40 ===== Build SRPM for 'remote-clients' =====
        > 22:26:40 
        > PKG_BASE=/localdisk/designer/jenkins-slave/starlingx/cgcs-root/stx/stx
        > -clients/remote-clients
        > 22:26:40 
        > WORK_BASE=/localdisk/loadbuild/jenkins-slave/starlingx/std/inputs/stx/
        > stx-clients/remote-clients
        > 22:26:40 
        > RPMBUILD_BASE=/localdisk/loadbuild/jenkins-slave/starlingx/std/inputs/
        > stx/stx-clients/remote-clients/rpmbuild
        > 22:26:40 
        > /localdisk/designer/jenkins-slave/starlingx/cgcs-root/build-tools/buil
        > d-srpms-serial: line 941: md5sums_from_input_vars: command not found
        > 22:26:40 ERROR: build_dir_spec (943): md5sums_from_input_vars '' '/localdisk/designer/jenkins-slave/starlingx/cgcs-root/stx/stx-clients/remote-clients/centos/remote-clients.spec' '/localdisk/loadbuild/jenkins-slave/starlingx/std/rpmbuild/SOURCES/remote-clients'
        > 
        > Doing sed -i 's/PARALLEL/SERIAL/g' build-tools/build-srpms-serial 
        > fixes this issue. However, after the srpms are created the 
        > build-rpm-serial script is unable to find these srpms. See below
        > 
        > 
        > ######## Wed Sep 26 22:36:34 UTC 2018: build-srpm-serial --std was 
        > successful
        >                                                                                                                                                                                               
        > ######## Wed Sep 26 22:36:34 UTC 2018: Launching build-rpms-serial 
        > --std
        >                                                                                                                                                                                               
        > /localdisk/designer/jenkins-slave/starlingx/cgcs-root/build-tools/buil
        > d-rpms-serial --std
        > FORMAL_BUILD=0
        > modify-build-cfg 
        > /localdisk/loadbuild/jenkins-slave/starlingx/std/jenkins-slave-starlin
        > gx-tis-r5-pike-std.cfg Nothing to build in 
        > '/localdisk/loadbuild/jenkins-slave/starlingx/std/rpmbuild/SRPMS'
        >                                                                                                                                                                                               
        > ######## Wed Sep 26 22:36:35 UTC 2018: build-rpm-serial --std was 
        > successful
        > 
        > This is reported here: 
        > https://bugs.launchpad.net/starlingx/+bug/1790583
        > I tried this but it didn't work for me: 
        > https://review.openstack.org/#/c/603983/
        > 
        > So, for now seems that it's not possible to use serial building.
        > 
        > 
        > On Wed, 2018-09-26 at 12:16 -0400, Scott Little wrote:
        >> aclocal 'too many loops' has been popping up sporadically for a week 
        >> or two now.  Possibly 7.5 related.
        >>
        >> I suspect that there is a build order and/or race condition element
        >> to this.   It often goes away if you just run build-pkgs a second
        >> time.
        >>
        >> The second possible element is that build-pkgs is using flags that 
        >> preserve the mock environment between packages.  The goal was to 
        >> avoid reinstalling required packages that are often common across the
        >> packages we build.   It was a build time speedup that has been in use
        >> for 2-3 years now without incident.
        >>
        >> Google shows that other folks have hit 'aclocal too many loops' as 
        >> well, and there are suggestions that it might be fixed in a recent 
        >> update to automake.  Scanning the changelog for the latest and 
        >> greatest from gnu.org doesn't show any obvious fixes addressing this 
        >> issue.
        >>
        >> Scott
        >>
        >>
        >> On 18-09-26 11:32 AM, Cordoba Malibran, Erich wrote:
        >>> BUILDSTDERR: aclocal: error: too many loops
        >>> BUILDSTDERR: aclocal: Please contact <bug-automake at gnu.org>.
        >>> BUILDSTDERR:  at /usr/share/automake-1.13/Automake/Channels.pm line 
        >>> 662.
        >>> BUILDSTDERR: 	Automake::Channels::msg('automake', '', 'too
        >>> many loops') called at /usr/share/automake- 
        >>> 1.13/Automake/ChannelDefs.pm line 212
        >>> BUILDSTDERR: 	Automake::ChannelDefs::prog_error('too many
        >>> loops') called at /usr/bin/aclocal line 1187
        >>> BUILDSTDERR: error: Bad exit status from /var/tmp/rpm-tmp.Sj0E7c
        >>> (%build)
        >>> BUILDSTDERR:     Macro expanded in comment on line 214: %global
        >>> _libexecdir %{_exec_prefix}/lib
        >>> BUILDSTDERR:     Bad exit status from /var/tmp/rpm-tmp.Sj0E7c
        >>> (%build)
        >>
        >> _______________________________________________
        >> Starlingx-discuss mailing list
        >> Starlingx-discuss at lists.starlingx.io
        >> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
        > _______________________________________________
        > Starlingx-discuss mailing list
        > Starlingx-discuss at lists.starlingx.io
        > http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
        > _______________________________________________
        > Starlingx-discuss mailing list
        > Starlingx-discuss at lists.starlingx.io
        > http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
        > 
        
        _______________________________________________
        Starlingx-discuss mailing list
        Starlingx-discuss at lists.starlingx.io
        http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
        _______________________________________________
        Starlingx-discuss mailing list
        Starlingx-discuss at lists.starlingx.io
        http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
        
    
    _______________________________________________
    Starlingx-discuss mailing list
    Starlingx-discuss at lists.starlingx.io
    http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
    



More information about the Starlingx-discuss mailing list