On 18-09-26 08:05 PM, Cordoba Malibran, Erich wrote:
On Wed, 2018-09-26 at 16:44 -0700, Saul Wold wrote:
On 09/26/2018 02:08 PM, Scott Little wrote:
I have also been investigating another intermittent build error affecting initscripts.
https://bugs.launchpad.net/starlingx/+bug/1794611
So any given package might be built by rpm-4.11.3, or by rpm- 4.14.0. It's a race with many inputs. Ideally it shouldn't matter, but it does. build-pkgs --serial might reduce the odds, but likely won't solve it entirely.
Why is this the case in the first place, can't we ensure we only build rpm-4.14.0?
In this case it is a new option that rpm passes to the patch command, --no-backup-if-mismatch, first introduced by rpm-4.13. This option has the effect of suppressing the creation of '.orig' files.
Creation of .orig files are a consequence of applying patches that are not clean. Unclean patches are ones that require 'fuzzing', i.e. treat the patch line numbers as approximate, rather than a strict requirement, just so long as the before/after context seems to be correct.
Prior to StarlingX, my policy for rebasing patches was that no fuzz is tolerated in our patches.
All the work to upgrade to 7.5 has created a lot of fuzzy patches.
Wow, sorry I was not aware that they were all fuzzy patches.
How did you got this list? There were .orig files in those folder? My query was ...
for f in $(grep -l 'Hunk' $MY_WORKSPACE/std/results/*/*/build.log); do basename $(dirname $f); done As noted in another thread, we might get false positives in the form of CentOS/EL7 patches that are fuzzy, not our own.
audit-2.8.1-3.el7.tis.2 bash-4.2.46-30.el7.tis.3 dhcp-4.2.5-68.el7.centos.1.tis.8 dnsmasq-2.76-5.el7.tis.6 drbd-8.4.3-0.tis.6 facter-2.4.4-4.el7.tis.4 haproxy-1.5.18-7.el7.tis.7 initscripts-9.49.41-1.el7.tis.16 iptables-1.4.21-24.1.el7_5.tis.3 kubernetes-1.10.0-1.tis.1 libevent-2.0.21-4.el7.tis.2 lighttpd-1.4.50-1.el7.tis.6 logrotate-3.8.6-15.el7.tis.3 netpbm-10.79.00-7.el7.tis.2 net-snmp-5.7.2-33.el7_5.2.tis.10 net-tools-2.0-0.22.20131004git.el7.tis.2 nfs-utils-1.3.0-0.54.el7.tis.4 nss-pam-ldapd-0.8.13-16.el7.tis.4 ntp-4.2.6p5-28.el7.centos.tis.3 openldap-2.4.44-15.el7_5.tis.8 openssh-7.4p1-16.el7_4.tis.9 pam-1.1.8-22.el7.tis.4 puppet-4.8.2-1.el7.tis.2 puppet-ceph-2.4.1-1.el7.tis.4 puppet-horizon-11.5.0-1.el7.tis.1 python-2.7.5-69.el7_5.tis.3 python-keyring-5.7.1-1.tis.2 python-wsme-0.9.2-1.el7.tis.3 resource-agents-3.9.5-124.el7.tis.12 rsync-3.1.2-4.el7.tis.2 shadow-utils-4.1.5.1-24.el7.tis.4 sudo-1.8.19p2-14.el7_5.tis.3 watchdog-5.13-11.el7.tis.2
So any of these packages might or might not produce unwanted .orig files. The .orig files might or might not be packaged, or break packaging.
So there are a number of threads to pull at here. 1) Are fuzzy patches tolerated. I vote no. Sooner or later a fuzzy patch will be mis-applied and cause us problems. Agreed, when updating patches should be rebased and de-fuzzed.
2) Can we force rpmbuild within mock to use a consistent policy with respect to creation of orig files? It would be good to investigate that.
Seems we can pass _default_patch_fuzz 0 as a variable to rpmbuild.
What could be the course of action here? - Wire rpmbuild to not allow fuzzy patches. - Then iteratively try to build until all the index are correct
-Erich
I usually start by using a combination of 'build-pkgs --edit', 'git format-patch', and meld (a graphical diff/merge tool). e.g. # If 'build-pkgs --edit' works, it's rather fragile as it's more or less parsing spec files, it produces # two gits, on for working on the tarball content, and a second for the spec file and other files outside the tarball. # Patches are seen as commits, applied with fuzz allowed. STX patches will get a WRS prefix (sorry, I'll make that STX in future) build-pkgs --edit --clean initscripts build-pkgs --edit initscripts ... 11:36:45 b7: ===== 'initscripts' has been extracted for editing. ===== 11:36:45 b7: ===== Metadata can be found at: /localdisk/loadbuild/slittle1/starlingx-4/std/srpm_work/initscripts/rpmbuild 11:36:45 b7: ===== Source code can be found at: /localdisk/loadbuild/slittle1/starlingx-4/std/srpm_work/initscripts/gits/initscripts.spec/initscripts-9.49.41 cd /localdisk/loadbuild/slittle1/starlingx-4/std/srpm_work/initscripts/gits/initscripts.spec/initscripts-9.49.41 git log --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%ci) %C(bold blue)<%an>%Creset' --abbrev-commit * 3aaf0ca - (HEAD, initscripts-9.49.41-1.el7) WRS: Patch15: run-dhclient-as-daemon-for-ipv6.patch (2018-09-27 11:36:45 -0400) <Scott Little> * 126e810 - WRS: Patch14: ifup-eth-stop-waiting-if-link-is-up.patch (2018-09-27 11:36:44 -0400) <Scott Little> * df98ada - WRS: Patch13: ipv6-static-route-support.patch (2018-09-27 11:36:44 -0400) <Scott Little> * cc391dd - WRS: Patch12: sysconfig-unsafe-usage-of-linkdelay-variable.patch (2018-09-27 11:36:44 -0400) <Scott Little> * 398cdf6 - WRS: Patch11: sysconfig-affirmative-check-for-link-carrier.patch (2018-09-27 11:36:43 -0400) <Scott Little> * a3ebe9e - WRS: Patch10: run-ifdown-on-all-interfaces.patch (2018-09-27 11:36:43 -0400) <Scott Little> * 0a6e1fc - WRS: Patch9: 0001-force-delay-check-link-down.patch (2018-09-27 11:36:43 -0400) <Scott Little> * 45075af - WRS: Patch8: 0001-dhclient-remove-1-arg.patch (2018-09-27 11:36:42 -0400) <Scott Little> * 02f8215 - WRS: Patch7: support-interface-promisc.patch (2018-09-27 11:36:42 -0400) <Scott Little> * c5b8eab - WRS: Patch6: dhclient-restrict-interfaces-to-those-on-c.patch (2018-09-27 11:36:41 -0400) <Scott Little> * 6d8bcb5 - WRS: Patch5: relocate-dhclient-leases-to-var-run.patch (2018-09-27 11:36:41 -0400) <Scott Little> * 480e9a5 - WRS: Patch4: support-interface-scriptlets.patch (2018-09-27 11:36:41 -0400) <Scott Little> * 487fdfa - (tag: pre_wrs_initscripts-9.49.41-1.el7) ORIGINAL: extracted archive (2018-09-27 11:36:39 -0400) <Scott Little> * 7ca8368 - (master) ORIGINAL: initial commit (2018-09-27 11:36:39 -0400) <Scott Little> # Create new patches without fuzz. Note: patch comment text is often discarded # git format-patch <SHA_prior_to_first_WRS_patch> git format-patch $(git tag | grep pre_wrs_) # use meld or other diff/merge tool to transfer the revised line numbers. A bit of coding magic is required to translate patch names. cd $MY_REPO/stx/stx-integ/base/initscripts/centos/patches/ for src in $(ls -1 /localdisk/loadbuild/slittle1/starlingx-4/std/srpm_work/initscripts/gits/initscripts.spec/initscripts-9.49.41/00*patch); do dest=$(ls -1 $(basename $src | cut -d '-' -f 4- | sed 's#.patch$#*#')); meld $src $dest; done cd /localdisk/loadbuild/slittle1/starlingx-4/std/srpm_work/initscripts/rpmbuild git log --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%ci) %C(bold blue)<%an>%Creset' --abbrev-commit * 4ecfd77 - (HEAD, initscripts-9.49.41-1.el7) WRS: spec-run-dhclient-as-daemon-for-ipv6.patch (2018-09-27 11:36:40 -0400) <Scott Little> * 69d0bb4 - WRS: spec-ifup-eth-stop-waiting-if-link-is-up.patch (2018-09-27 11:36:40 -0400) <Scott Little> * e2942ca - WRS: ipv6-static-route-support.patch (2018-09-27 11:36:40 -0400) <Scott Little> * 4e0368a - WRS: fix-build-failures-due-to-unwanted-sgid.patch (2018-09-27 11:36:40 -0400) <Scott Little> * bc43676 - WRS: spec-sysconfig-unsafe-usage-of-linkdelay-variable.patch (2018-09-27 11:36:40 -0400) <Scott Little> * 6a38a1d - WRS: spec-sysconfig-affirmative-check-for-link-carrier.patch (2018-09-27 11:36:40 -0400) <Scott Little> * 730fe3c - WRS: 0001-Update-package-versioning-for-TIS-format.patch (2018-09-27 11:36:39 -0400) <Scott Little> * a083a54 - WRS: run-ifdown-on-all-interfaces.patch (2018-09-27 11:36:39 -0400) <Scott Little> * 7f02bc2 - WRS: force-delay-check-link-down.patch (2018-09-27 11:36:39 -0400) <Scott Little> * dbdb232 - WRS: 0001-Fix-Interfaces-intermittenly-do-not-come-up-on-boot.patch (2018-09-27 11:36:39 -0400) <Scott Little> * c51c427 - WRS: spec-add-mountnfs-init-script.patch (2018-09-27 11:36:39 -0400) <Scott Little> * 0a5a833 - WRS: 0001-Support-PROMISC-for-interfaces-config.patch (2018-09-27 11:36:39 -0400) <Scott Little> * e0bbac3 - WRS: 0001-Disable-zeroconf-route.patch (2018-09-27 11:36:39 -0400) <Scott Little> * 6485d13 - WRS: stop-creating-shared-dirs.patch (2018-09-27 11:36:39 -0400) <Scott Little> * 7223930 - WRS: spec-include-TiS-changes.patch (2018-09-27 11:36:39 -0400) <Scott Little> * b63eb3d - (tag: wrs_post_copy_list_initscripts-9.49.41-1.el7) WRS: COPY_LIST content (2018-09-27 11:36:39 -0400) <Scott Little> * 8e1e97a - (tag: pre_wrs_initscripts-9.49.41-1.el7) ORIGINAL: srpm extract (2018-09-27 11:36:38 -0400) <Scott Little> * a6928fd - (master) ORIGINAL: initial commit (2018-09-27 11:36:38 -0400) <Scott Little> # Create new meta-patches without fuzz. Note: patch comment text is often discarded # git format-patch <SHA_prior_to_first_WRS_patch> git format-patch $(git tag | grep wrs_post_copy) # use meld or other diff/merge tool to transfer the revised line numbers. A bit of coding magic is required to translate patch names. cd $MY_REPO/stx/stx-integ/base/initscripts/centos/meta_patches/ for src in $(ls -1 /localdisk/loadbuild/slittle1/starlingx-4/std/srpm_work/initscripts/rpmbuild/00*patch); do dest=$(ls -1 $(basename $src | cut -d '-' -f 3- | sed 's#.patch$#*#')); meld $src $dest; done
3) Can we pre-build rpm, such that all packages build against the same rpm version? Yes, as mentioned above we should use a consistent version.
4) Are we patching any other low level build tools that have similar issues? Possibly explaining ceph? TBD
Yup more info is needed
Sau!
On 18-09-26 12:16 PM, Scott Little wrote:
aclocal 'too many loops' has been popping up sporadically for a week or two now. Possibly 7.5 related.
I suspect that there is a build order and/or race condition element to this. It often goes away if you just run build-pkgs a second time.
The second possible element is that build-pkgs is using flags that preserve the mock environment between packages. The goal was to avoid reinstalling required packages that are often common across the packages we build. It was a build time speedup that has been in use for 2-3 years now without incident.
Google shows that other folks have hit 'aclocal too many loops' as well, and there are suggestions that it might be fixed in a recent update to automake. Scanning the changelog for the latest and greatest from gnu.org doesn't show any obvious fixes addressing this issue.
Scott
On 18-09-26 11:32 AM, Cordoba Malibran, Erich wrote:
BUILDSTDERR: aclocal: error: too many loops BUILDSTDERR: aclocal: Please contact<bug-automake@gnu.org>. BUILDSTDERR: at /usr/share/automake-1.13/Automake/Channels.pm line 662. BUILDSTDERR: Automake::Channels::msg('automake', '', 'too many loops') called at /usr/share/automake- 1.13/Automake/ChannelDefs.pm line 212 BUILDSTDERR: Automake::ChannelDefs::prog_error('too many loops') called at /usr/bin/aclocal line 1187 BUILDSTDERR: error: Bad exit status from /var/tmp/rpm- tmp.Sj0E7c (%build) BUILDSTDERR: Macro expanded in comment on line 214: %global _libexecdir %{_exec_prefix}/lib BUILDSTDERR: Bad exit status from /var/tmp/rpm-tmp.Sj0E7c (%build)
_______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-disc uss
_______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discus s
_______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss