<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif;
color:black;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:12.0pt;
font-family:"Times New Roman",serif;
color:black;}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0in;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";
color:black;}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:Consolas;
color:black;}
span.EmailStyle20
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:#1F497D;}
span.EmailStyle21
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:#1F497D;}
span.EmailStyle22
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:windowtext;}
span.EmailStyle23
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.25in 1.0in 1.25in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body bgcolor="white" lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Agree with Scott<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Brent
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:windowtext">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:windowtext"> Scott Little [mailto:scott.little@windriver.com]
<br>
<b>Sent:</b> Friday, September 28, 2018 9:54 AM<br>
<b>To:</b> Lin, Shuicheng <shuicheng.lin@intel.com>; starlingx-discuss@lists.starlingx.io<br>
<b>Subject:</b> Re: [Starlingx-discuss] de-fuzz centos7.5 upgraded src rpm<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">Certainly that will prevent creation of .orig files, so the risk of build failures in the packaging phase will be eliminated.
<br>
However I have strong distrust of fuzzy patches. It suggests to me that folks are upgrading packages without doing the<br>
due diligence to make sure the patches are still needed, and are being applied correctly.
<br>
<br>
Also I have seen twice in my career, a fuzzy patch get improperly applied to a neighboring bit of code that happened to
<br>
look similar to the proper target. It's rare, but it can occur.<br>
<br>
On the plus side, moving to 4.14 would give us a consistent version of rpm throughout the build. That I like.<br>
<br>
I'm thinking we should do it, but we need a no fuzzy patches policy, and perhaps we create an audit that looks for fuzzy patches.<br>
<br>
Scott<br>
<br>
<br>
<br>
On 18-09-28 02:27 AM, Lin, Shuicheng wrote:<o:p></o:p></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Hi Scott,</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">What do you think about upgrade rpm in mirror lst to 4.14 version? So we could get rid of the fuzz issue.</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">To do this upgrade, rpm will be moved from centos repo lst to tarball lst.</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Here is the release notes of rpm-4.13 to fix this fuzz issue:</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-family:"Arial",sans-serif">Enable –no-backup-if-mismatch by default in %patch macro (<a href="https://bugzilla.redhat.com/show_bug.cgi?id=884755"><span style="color:#9D151A;text-decoration:none">RhBug:884755</span></a>)</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> </span><o:p></o:p></p>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Best Regards</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Shuicheng</span><o:p></o:p></p>
</div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> </span><o:p></o:p></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:windowtext">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:windowtext"> Scott Little [<a href="mailto:scott.little@windriver.com">mailto:scott.little@windriver.com</a>]
<br>
<b>Sent:</b> Thursday, September 27, 2018 11:24 PM<br>
<b>To:</b> Lin, Shuicheng <a href="mailto:shuicheng.lin@intel.com"><shuicheng.lin@intel.com></a>;
<a href="mailto:starlingx-discuss@lists.starlingx.io">starlingx-discuss@lists.starlingx.io</a><br>
<b>Subject:</b> Re: [Starlingx-discuss] de-fuzz centos7.5 upgraded src rpm</span><o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"> <o:p></o:p></p>
<div>
<p class="MsoNormal">Yes, I've seen some upstream packages that ship from CentOS or EL7 with fuzzy patches. Can't do much about that.<br>
<br>
My list might include packages where the only fuzzy patches are those coming from CentOS. We'll have to sort through them by hand I think.<br>
<br>
Scott<br>
<br>
<br>
<br>
On 18-09-27 10:24 AM, Lin, Shuicheng wrote:<o:p></o:p></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Hi all,</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Here is the patch list to de-fuzz the patches in CentOS7.5 upgrade task.
</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><a href="https://review.openstack.org/#/q/topic:patch_fuzz+%28status:open+OR+status:merged%29">https://review.openstack.org/#/q/topic:patch_fuzz+(status:open+OR+status:merged)</a></span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">I notice there are some other src rpms also have the fuzzy issue. Will handle them later.</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Also, I find some SRC RPM itself has the fuzzy issue, such as openldap/rsync/netpbm/openssh etc.</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">So we cannot eliminate all fuzzy issue just by rebase ourselves patches.
</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> </span><o:p></o:p></p>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Best Regards</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Shuicheng</span><o:p></o:p></p>
</div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> </span><o:p></o:p></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:windowtext">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:windowtext"> Scott Little [<a href="mailto:scott.little@windriver.com">mailto:scott.little@windriver.com</a>]
<br>
<b>Sent:</b> Thursday, September 27, 2018 5:08 AM<br>
<b>To:</b> <a href="mailto:starlingx-discuss@lists.starlingx.io">starlingx-discuss@lists.starlingx.io</a><br>
<b>Subject:</b> Re: [Starlingx-discuss] build-pkgs cannot complete std build</span><o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"> <o:p></o:p></p>
<div>
<p class="MsoNormal">I have also been investigating another intermittent build error affecting initscripts.<br>
<br>
<a href="https://bugs.launchpad.net/starlingx/+bug/1794611">https://bugs.launchpad.net/starlingx/+bug/1794611</a><br>
<br>
So any given package might be built by rpm-4.11.3, or by rpm-4.14.0. It's a race with many inputs. Ideally it shouldn't matter, but it does. build-pkgs --serial might reduce the odds, but likely won't solve it entirely.<br>
<br>
In this case it is a new option that rpm passes to the patch command, --no-backup-if-mismatch, first introduced by rpm-4.13.<br>
This option has the effect of suppressing the creation of '.orig' files.<br>
<br>
Creation of .orig files are a consequence of applying patches that are not clean. Unclean patches are ones that require 'fuzzing', i.e. treat the patch line numbers as approximate, rather than a strict requirement, just so long as the before/after context seems
to be correct.<br>
<br>
Prior to StarlingX, my policy for rebasing patches was that no fuzz is tolerated in our patches.<br>
<br>
All the work to upgrade to 7.5 has created a lot of fuzzy patches.<br>
<br>
audit-2.8.1-3.el7.tis.2<br>
bash-4.2.46-30.el7.tis.3<br>
dhcp-4.2.5-68.el7.centos.1.tis.8<br>
dnsmasq-2.76-5.el7.tis.6<br>
drbd-8.4.3-0.tis.6<br>
facter-2.4.4-4.el7.tis.4<br>
haproxy-1.5.18-7.el7.tis.7<br>
initscripts-9.49.41-1.el7.tis.16<br>
iptables-1.4.21-24.1.el7_5.tis.3<br>
kubernetes-1.10.0-1.tis.1<br>
libevent-2.0.21-4.el7.tis.2<br>
lighttpd-1.4.50-1.el7.tis.6<br>
logrotate-3.8.6-15.el7.tis.3<br>
netpbm-10.79.00-7.el7.tis.2<br>
net-snmp-5.7.2-33.el7_5.2.tis.10<br>
net-tools-2.0-0.22.20131004git.el7.tis.2<br>
nfs-utils-1.3.0-0.54.el7.tis.4<br>
nss-pam-ldapd-0.8.13-16.el7.tis.4<br>
ntp-4.2.6p5-28.el7.centos.tis.3<br>
openldap-2.4.44-15.el7_5.tis.8<br>
openssh-7.4p1-16.el7_4.tis.9<br>
pam-1.1.8-22.el7.tis.4<br>
puppet-4.8.2-1.el7.tis.2<br>
puppet-ceph-2.4.1-1.el7.tis.4<br>
puppet-horizon-11.5.0-1.el7.tis.1<br>
python-2.7.5-69.el7_5.tis.3<br>
python-keyring-5.7.1-1.tis.2<br>
python-wsme-0.9.2-1.el7.tis.3<br>
resource-agents-3.9.5-124.el7.tis.12<br>
rsync-3.1.2-4.el7.tis.2<br>
shadow-utils-4.1.5.1-24.el7.tis.4<br>
sudo-1.8.19p2-14.el7_5.tis.3<br>
watchdog-5.13-11.el7.tis.2<br>
<br>
So any of these packages might or might not produce unwanted .orig files.<br>
The .orig files might or might not be packaged, or break packaging.<br>
<br>
So there are a number of threads to pull at here.<br>
1) Are fuzzy patches tolerated. I vote no. Sooner or later a fuzzy patch will be mis-applied and cause us problems.<br>
2) Can we force rpmbuild within mock to use a consistent policy with respect to creation of orig files?<br>
3) Can we pre-build rpm, such that all packages build against the same rpm version?<br>
4) Are we patching any other low level build tools that have similar issues? Possibly explaining ceph? TBD<br>
<br>
<br>
<br>
On 18-09-26 12:16 PM, Scott Little wrote:<o:p></o:p></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal">aclocal 'too many loops' has been popping up sporadically for a week or two now. Possibly 7.5 related.<br>
<br>
I suspect that there is a build order and/or race condition element to this. It often goes away if you just run build-pkgs a second time.<br>
<br>
The second possible element is that build-pkgs is using flags that preserve the mock environment between packages. The goal was to avoid reinstalling required packages that are often common across the packages we build. It was a build time speedup that has
been in use for 2-3 years now without incident.<br>
<br>
Google shows that other folks have hit 'aclocal too many loops' as well, and there are suggestions that it might be fixed in a recent update to automake. Scanning the changelog for the latest and greatest from gnu.org doesn't show any obvious fixes addressing
this issue. <br>
<br>
Scott<br>
<br>
<br>
On 18-09-26 11:32 AM, Cordoba Malibran, Erich wrote:<o:p></o:p></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<pre>BUILDSTDERR: aclocal: error: too many loops<o:p></o:p></pre>
<pre>BUILDSTDERR: aclocal: Please contact <a href="mailto:bug-automake@gnu.org"><bug-automake@gnu.org></a>.<o:p></o:p></pre>
<pre>BUILDSTDERR: at /usr/share/automake-1.13/Automake/Channels.pm line 662.<o:p></o:p></pre>
<pre>BUILDSTDERR: Automake::Channels::msg('automake', '', 'too many loops') called at /usr/share/automake-1.13/Automake/ChannelDefs.pm line 212<o:p></o:p></pre>
<pre>BUILDSTDERR: Automake::ChannelDefs::prog_error('too many loops') called at /usr/bin/aclocal line 1187<o:p></o:p></pre>
<pre>BUILDSTDERR: error: Bad exit status from /var/tmp/rpm-tmp.Sj0E7c (%build)<o:p></o:p></pre>
<pre>BUILDSTDERR: Macro expanded in comment on line 214: %global _libexecdir %{_exec_prefix}/lib<o:p></o:p></pre>
<pre>BUILDSTDERR: Bad exit status from /var/tmp/rpm-tmp.Sj0E7c (%build)<o:p></o:p></pre>
</blockquote>
<p> <o:p></o:p></p>
<p class="MsoNormal"><br>
<br>
<br>
<br>
<br>
<o:p></o:p></p>
<pre>_______________________________________________<o:p></o:p></pre>
<pre>Starlingx-discuss mailing list<o:p></o:p></pre>
<pre><a href="mailto:Starlingx-discuss@lists.starlingx.io">Starlingx-discuss@lists.starlingx.io</a><o:p></o:p></pre>
<pre><a href="http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss">http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss</a><o:p></o:p></pre>
</blockquote>
<p> <o:p></o:p></p>
</blockquote>
<p> <o:p></o:p></p>
</blockquote>
<p><o:p> </o:p></p>
</div>
</body>
</html>