<html><head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
  </head>
  <body>
    <p>I see <a moz-do-not-send="true" href="https://review.opendev.org/#/c/733426/9">https://review.opendev.org/#/c/733426/9</a>
      has been posted</p>
    <p>With this update, layered builds should pass, and would look like
      this ...</p>
    <ul class="com-google-gwtexpui-safehtml-client-SafeHtmlCss-wikiList">
      <li>Flock and iso builds will use 13.2.2.</li>
      <li>All container builds uses 13.2.10.</li>
      <ul>
        <li>Do we want 13.2.10 in ALL containers?</li>
      </ul>
      <li>Any ceph dependent rpms from distro/flock builds that make it
        into a container (if any), will have been compiled against
        13.2.2, but will run against 13.2.10.  I'm more comfortable with
        a increment to the patch level than a decrement.  I think we can
        live with this until we can move to 13.2.10 universally.<br>
      </li>
    </ul>
    <p>Monolithic will continue to build, but will remain confused ...<br>
    </p>
    <p>All lst files, including container layer lsts, are downloaded
      before any package is built.
      Most if not all packages that depend on ceph will build against
      13.2.10 as mock/yum does not understand the 'prefer local'.
      build-iso will use 'prefer local' and ship with 13.2.2. The
      implications of which is unclear. One hopes that the interface is
      stable when the version diff is only at the patch level, but I
      never like to see shipped version LOWER than the complied against
      version.</p>
    <p><br>
    </p>
    <div class="moz-cite-prefix">On 2020-06-03 6:08 p.m., Saul Wold
      wrote:<br>
    </div>
    <blockquote type="cite" cite="mid:b6adc34f-7e0c-9b6f-4c8a-63a0db880030@linux.intel.com">
      <br>
      <br>
      On 6/3/20 2:01 PM, Scott Little wrote:
      <br>
      <blockquote type="cite">No I don't think that would work.  We
        can't have two versions of the same package competing for
        dominance within the mock build environments.  i.e. on time pkg
        X builds against 13.2.2, the next time against 13.2.10.  The
        outcome dependent on the vagaries of job scheduling, build
        speeds, and any other number of factors.  If you compile against
        13.2.10, will you run ok vs 13.2.2.  I wouldn't want to bet on
        it.
        <br>
        <br>
        The build layering solution might be to throw it in it's own
        layer.
        <br>
        <br>
        Until we are 100% committed to build layering, we need to
        converge on ONE version of ceph.
        <br>
        <br>
      </blockquote>
      Ok, so one option is to move to Ceph 13.2.10 or drop the existing
      package list update that brings in the python3 and related Ceph
      packages.
      <br>
      <br>
      Do we need to at least revert that commit in-order to get the
      build working again?
      <br>
      <br>
      We might need to spend a few minutes to hash this out tomorrow
      morning at the PTG.
      <br>
      <br>
      Sau!
      <br>
      <br>
      <blockquote type="cite">Scott
        <br>
        <br>
        <br>
        On 2020-06-03 10:52 a.m., Saul Wold wrote:
        <br>
        <blockquote type="cite">
          <br>
          <br>
          On 6/3/20 1:47 AM, Liu, ZhipengS wrote:
          <br>
          <blockquote type="cite">Hi Scott,
            <br>
            <br>
            For question #1,
            <br>
            <br>
            When we built openstack ussuri image which is python3 only.
            <br>
            <br>
            It needs python3-rbd and related dependency, so we add
            librados2-13.2.10 and related packages.
            <br>
            <br>
            For local built librados2-13.2.2-0.el7.tis.25.x86_64.rpm, it
            is for python2.
            <br>
            <br>
            Shouldn’t  we let the build choose local build first?
            <br>
            <br>
          </blockquote>
          Following up on this we need to be careful about which we
          choose, as I said in the other email is this a one-off issue
          or something that we see more of.  So maybe an audit tool
          would help.
          <br>
          <br>
          <blockquote type="cite">Another option is moving these
            packages to container layer, add rpms_centos.lst in
            config/centos/flock/?
            <br>
            <br>
          </blockquote>
          I understand this option better after chatting with Zhipeng, I
          think this might be the best option adding the Updated Ceph /
          RBD related packages to the container list which will be used
          for the Usurri container builds but not by the platform OS.
          <br>
          <br>
          This would mean that the containers would have Ceph 13.2.10
          related packages and the platform OS would be 13.2.2.  Would
          that cause problems or stability issues?
          <br>
          <br>
          Sau!
          <br>
          <br>
          <blockquote type="cite">Thanks!
            <br>
            <br>
            Zhipeng
            <br>
            <br>
            *From:*Scott Little <a class="moz-txt-link-rfc2396E" href="mailto:scott.little@windriver.com"><scott.little@windriver.com></a>
            <br>
            *Sent:* 2020年6月3日15:57
            <br>
            *To:* <a class="moz-txt-link-abbreviated" href="mailto:starlingx-discuss@lists.starlingx.io">starlingx-discuss@lists.starlingx.io</a>
            <br>
            *Subject:* Re: [Starlingx-discuss] [build-report]
            STX_build_layer_flock_master_master - Build # 132 - Still
            Failing!
            <br>
            <br>
            This was an interesting one.
            <br>
            <br>
            We have been building
            librados2-13.2.2-0.el7.tis.25.x86_64.rpm as part of the
            distro layer for some time.
            <br>
            <br>
            A recent update added librados2-13.2.10-0.el7.x86_64.rpm to
            the lst of the flock layer.
            <br>
            <br>
            Now build-iso preferres locally built packages over
            downloaded ones, even if the downloaded on is of higher
            version.  Now that policy is open for debate, but that is
            what it does.
            <br>
            <br>
            Monolithic build uses the lst files of all layers, but
            having built librados2-13.2.2-0.el7.tis.25.x86_64.rpm, it
            selects librados2-13.2.2-0.el7.tis.25.x86_64.rpm over
            librados2-13.2.10-0.el7.x86_64.rpm when building the iso.
            <br>
            <br>
            Flock layer build, downloads
            librados2-13.2.2-0.el7.tis.25.x86_64.rpm from the distro
            layer build.  It doesn't build it itself.  The downloads
            from the two sources are lumped into a common repo, so it
            has no reason to prefer the lower versioned rpm.  It selects
            librados2-13.2.10-0.el7.x86_64.rpm.
            <br>
            <br>
            The final piece of the puzzle is the transitive list of
            requires for librados2-13.2.10-0.el7.x86_64.rpm.  It has a
            new dependency that pulls in
            lttng-ust-2.10.0-1.el7.x86_64.rpm, which in turn needs
            userspace-rcu-0.10.0-3.el7.x86_64.rpm, which is not present.
            It's wasn't included in the recent lst file changes that
            added librados2-13.2.10-0.el7.x86_64.rpm.
            <br>
            <br>
            A flock layer build-iso should have caught this.  I suspect
            build-iso was only performed on a monolithic build.
            <br>
            <br>
            Open questions.
            <br>
            1) Is there a need to move to librados2-13.2.10 from
            librados2-13.2.2.  If yes, do we still need whatever
            modifications were applied to librados2-13.2.2?  Do they
            need to be ported to librados2-13.2.10 , or can we drop
            librados2 from the set of packages we have patches against?
            <br>
            <br>
            2) For build-iso... should we prefer locally built packages
            even though there is a higher package named in an lst?  If
            yes, then layered build needs apply the local first policy
            accross layers. Alternatively, perhaps drop the local first
            policy, but add an audit tool to detect when a locally built
            package is being masked in this way.
            <br>
            <br>
            Scott
            <br>
            <br>
            On 2020-06-02 10:30 p.m., <a class="moz-txt-link-abbreviated" href="mailto:build.starlingx@gmail.com">build.starlingx@gmail.com</a>
            <a class="moz-txt-link-rfc2396E" href="mailto:build.starlingx@gmail.com"><mailto:build.starlingx@gmail.com></a> wrote:
            <br>
            <br>
                Project: STX_build_layer_flock_master_master
            <br>
            <br>
                Build #: 132
            <br>
            <br>
                Status: Still Failing
            <br>
            <br>
                Timestamp: 20200603T020359Z
            <br>
            <br>
                Check logs at:
            <br>
            <br>
<a class="moz-txt-link-freetext" href="http://mirror.starlingx.cengn.ca/mirror/starlingx/master/centos/flock/20200603T020359Z/logs">http://mirror.starlingx.cengn.ca/mirror/starlingx/master/centos/flock/20200603T020359Z/logs</a>
            <br>
            <br>
--------------------------------------------------------------------------------
            <br>
            <br>
                Parameters
            <br>
            <br>
                FULL_BUILD: false
            <br>
            <br>
                FORCE_BUILD: false
            <br>
            <br>
            <br>
            <br>
                _______________________________________________
            <br>
            <br>
                Starlingx-discuss mailing list
            <br>
            <br>
                <a class="moz-txt-link-abbreviated" href="mailto:Starlingx-discuss@lists.starlingx.io">Starlingx-discuss@lists.starlingx.io</a>
            <a class="moz-txt-link-rfc2396E" href="mailto:Starlingx-discuss@lists.starlingx.io"><mailto:Starlingx-discuss@lists.starlingx.io></a>
            <br>
            <br>
<a class="moz-txt-link-freetext" href="http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss">http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss</a>
            <br>
            <br>
            <br>
            _______________________________________________
            <br>
            Starlingx-discuss mailing list
            <br>
            <a class="moz-txt-link-abbreviated" href="mailto:Starlingx-discuss@lists.starlingx.io">Starlingx-discuss@lists.starlingx.io</a>
            <br>
<a class="moz-txt-link-freetext" href="http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss">http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss</a>
            <br>
            <br>
          </blockquote>
          <br>
          _______________________________________________
          <br>
          Starlingx-discuss mailing list
          <br>
          <a class="moz-txt-link-abbreviated" href="mailto:Starlingx-discuss@lists.starlingx.io">Starlingx-discuss@lists.starlingx.io</a>
          <br>
<a class="moz-txt-link-freetext" href="http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss">http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss</a>
          <br>
        </blockquote>
        <br>
        _______________________________________________
        <br>
        Starlingx-discuss mailing list
        <br>
        <a class="moz-txt-link-abbreviated" href="mailto:Starlingx-discuss@lists.starlingx.io">Starlingx-discuss@lists.starlingx.io</a>
        <br>
<a class="moz-txt-link-freetext" href="http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss">http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss</a>
        <br>
      </blockquote>
      <br>
      _______________________________________________
      <br>
      Starlingx-discuss mailing list
      <br>
      <a class="moz-txt-link-abbreviated" href="mailto:Starlingx-discuss@lists.starlingx.io">Starlingx-discuss@lists.starlingx.io</a>
      <br>
<a class="moz-txt-link-freetext" href="http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss">http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss</a>
      <br>
    </blockquote>
  </body>
</html>