<html><head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
  </head>
  <body text="#000000" bgcolor="#FFFFFF">
    <div class="moz-cite-prefix">The jenkins in the cengn build is
      offline while we repair the file system on the mirror.  The build
      machine both read from, and writes to the mirror, and I don't want
      it to cause further corruptions while the mirror is being
      repaired.<br>
    </div>
    <div class="moz-cite-prefix"><br>
    </div>
    <div class="moz-cite-prefix">The cengn mirror has been cut over to
      the new storage back end.  However any corrupt files from the old
      storage back end will remain corrupted on the new one.<br>
    </div>
    <div class="moz-cite-prefix"><br>
    </div>
    <div class="moz-cite-prefix">Restoration of rpm repos will proceed
      in several phases:</div>
    <div class="moz-cite-prefix"><br>
    </div>
    <div class="moz-cite-prefix">1) An audit of all rpm repos is
      underway.  Rpm's that fail their checksum are being removed and
      repodata will be updated to reflect the missing RPMS.</div>
    <div class="moz-cite-prefix"><br>
    </div>
    <div class="moz-cite-prefix">2) The mirror update job (downloading
      rpms from upstream rpm repos) will be run to restore as many
      missing files as possible. Many rpms may no longer be available as
      we kept old versions of packages that upstream drops when a new
      version is available,<br>
    </div>
    <div class="moz-cite-prefix"><br>
    </div>
    <div class="moz-cite-prefix">3) If rpms that are not available from
      upstream, but are required for our build, we can attempt to
      restore them from one of several places.   e.g. the inputs
      directory of various builds, or the downloads directory on the
      cengn build server (independent storage from the mirror)<br>
    </div>
    <div class="moz-cite-prefix"><br>
    </div>
    <div class="moz-cite-prefix">4) Resume building new loads<br>
    </div>
    <div class="moz-cite-prefix"><br>
    </div>
    <div class="moz-cite-prefix"><br>
    </div>
    <div class="moz-cite-prefix"><br>
    </div>
    <p>All of the release iso's on cengn were corrupt.</p>
    <p><br>
    </p>
    <p>I've been able to restore 4.0.1 and 2.0.1 from the original build
      artifacts.</p>
    <p><br>
    </p>
    <p>Original build artifacts for 3.0.0 are missing.  I do have build
      artifacts for builds that were intended to lead up to a 3.0.1
      release.  If a verifiable copy of 3.0.0 can't be found, we may
      want to consider issuing a 3.0.1.  Alternatively I can rebuild
      3.0.0, but the file checksums wont match the original build due to
      embedded timestamps.<br>
    </p>
    <p><br>
    </p>
    <p><br>
    </p>
    <div class="moz-cite-prefix">Scott</div>
    <div class="moz-cite-prefix"><br>
    </div>
    <div class="moz-cite-prefix"><br>
    </div>
    <div class="moz-cite-prefix"><br>
    </div>
    <div class="moz-cite-prefix">On 2021-01-25 11:08 a.m., Little, Scott
      wrote:<br>
    </div>
    <blockquote type="cite" cite="mid:BY5PR11MB441835A7B47040913D2228B186BD9@BY5PR11MB4418.namprd11.prod.outlook.com">
      
      <div>Work continues at CENGN repair damaged repos and to move us
        to a new backend infrastructure with better redundancy.</div>
      <div><br>
      </div>
      <div>I still recommend that you avoid using download_mirrir.sh
        until the repos are fully repaired.  I don’t have a firm ETA for
        that, perhaps two before we have full confidence in the repairs.</div>
      <div><br>
      </div>
      <div>Scott </div>
      <div><br>
      </div>
      <div><br>
      </div>
      <hr style="display:inline-block;width:98%" tabindex="-1">
      <div id="divRplyFwdMsg" dir="ltr"><font style="font-size:11pt" face="Calibri, sans-serif" color="#000000"><b>From:</b> Scott
          Little <a class="moz-txt-link-rfc2396E" href="mailto:scott.little@windriver.com"><scott.little@windriver.com></a><br>
          <b>Sent:</b> January 23, 2021 2:24 AM<br>
          <b>To:</b> <a class="moz-txt-link-abbreviated" href="mailto:starlingx-discuss@lists.starlingx.io">starlingx-discuss@lists.starlingx.io</a>
          <a class="moz-txt-link-rfc2396E" href="mailto:starlingx-discuss@lists.starlingx.io"><starlingx-discuss@lists.starlingx.io></a><br>
          <b>Subject:</b> Re: [Starlingx-discuss] Cengn Mirror server in
          down?!</font>
        <div> </div>
      </div>
      <div>
        <p>CENGN is up, but there seems to be significant damage to the
          repositories.  <br>
        </p>
        <p><br>
        </p>
        <p>We are investigate options to restore the repositories to
          health.</p>
        <p><br>
        </p>
        <p>I do NOT recommend running download_mirror.sh at this time.</p>
        <p><br>
        </p>
        <p>Scott</p>
        <p><br>
        </p>
        <p><br>
        </p>
        <p><br>
        </p>
        <div class="x_moz-cite-prefix">On 2021-01-22 3:53 p.m., Scott
          Little wrote:<br>
        </div>
        <blockquote type="cite">
          <p>CENGN remains out of service.</p>
          <p><br>
          </p>
          <p>A new problem has been identified with ceph storage.</p>
          <p><br>
          </p>
          <p>They continue to work the outage.  <br>
          </p>
          <p><br>
          </p>
          <p>Scott</p>
          <p><br>
          </p>
          <p><br>
          </p>
          <p><br>
          </p>
          <div class="x_moz-cite-prefix">On 2021-01-22 9:55 a.m., Scott
            Little wrote:<br>
          </div>
          <blockquote type="cite">
            <p>CENGN remains out of service</p>
            <p><br>
            </p>
            <p>AC to the server room has been restored. They are now
              working some firewall issues.</p>
            <p><br>
            </p>
            <p>They hope to have it restored in a few hours.</p>
            <p><br>
            </p>
            <p>Scott</p>
            <p><br>
            </p>
            <p><br>
            </p>
            <div class="x_moz-cite-prefix">On 2021-01-21 10:07 a.m.,
              Panech, Davlet wrote:<br>
            </div>
            <blockquote type="cite">
              <style type="text/css" style="display:none">
<!--
p
        {margin-top:0;
        margin-bottom:0}
-->
</style>
              <div style="font-family:Calibri,Arial,Helvetica,sans-serif;
                font-size:12pt; color:rgb(0,0,0)">
                The hosting company had an issue with their servers,
                they are working on it. They hope to be back online mid
                afternoon (Jan 21, Eastern Standard Time).<br>
              </div>
              <hr tabindex="-1" style="display:inline-block; width:98%">
              <div id="x_divRplyFwdMsg" dir="ltr"><font style="font-size:11pt" face="Calibri, sans-serif" color="#000000"><b>From:</b> Panech, Davlet
                  <a class="x_moz-txt-link-rfc2396E" href="mailto:Davlet.Panech@windriver.com" moz-do-not-send="true"><Davlet.Panech@windriver.com></a><br>
                  <b>Sent:</b> January 21, 2021 9:56 AM<br>
                  <b>To:</b> Dimofte, Alexandru <a class="x_moz-txt-link-rfc2396E" href="mailto:alexandru.dimofte@intel.com" moz-do-not-send="true">
                    <alexandru.dimofte@intel.com></a>; <a class="x_moz-txt-link-abbreviated" href="mailto:starlingx-discuss@lists.starlingx.io" moz-do-not-send="true">
                    starlingx-discuss@lists.starlingx.io</a> <a class="x_moz-txt-link-rfc2396E" href="mailto:starlingx-discuss@lists.starlingx.io" moz-do-not-send="true">
                    <starlingx-discuss@lists.starlingx.io></a><br>
                  <b>Subject:</b> Re: Cengn Mirror server in down?!</font>
                <div> </div>
              </div>
              <style type="text/css" style="display:none">
<!--
p
        {margin-top:0;
        margin-bottom:0}
-->
</style>
              <div dir="ltr">
                <div style="font-family:Calibri,Arial,Helvetica,sans-serif;
                  font-size:12pt; color:rgb(0,0,0)">
                  Thanks Alexandru, we are looking into it.<br>
                </div>
                <div style="font-family:Calibri,Arial,Helvetica,sans-serif;
                  font-size:12pt; color:rgb(0,0,0)">
                  <br>
                </div>
                <hr tabindex="-1" style="display:inline-block;
                  width:98%">
                <div id="x_x_divRplyFwdMsg" dir="ltr"><font style="font-size:11pt" face="Calibri, sans-serif" color="#000000"><b>From:</b> Dimofte, Alexandru
                    <a class="x_moz-txt-link-rfc2396E" href="mailto:alexandru.dimofte@intel.com" moz-do-not-send="true"><alexandru.dimofte@intel.com></a><br>
                    <b>Sent:</b> January 21, 2021 3:24 AM<br>
                    <b>To:</b> <a class="x_moz-txt-link-abbreviated" href="mailto:starlingx-discuss@lists.starlingx.io" moz-do-not-send="true">
                      starlingx-discuss@lists.starlingx.io</a> <a class="x_moz-txt-link-rfc2396E" href="mailto:starlingx-discuss@lists.starlingx.io" moz-do-not-send="true">
                      <starlingx-discuss@lists.starlingx.io></a><br>
                    <b>Subject:</b> [Starlingx-discuss] Cengn Mirror
                    server in down?!</font>
                  <div> </div>
                </div>
                <style>
<!--
@font-face
        {font-family:"Cambria Math"}
@font-face
        {font-family:Calibri}
@font-face
        {font-family:"Intel Clear"}
@font-face
        {font-family:"Intel Clear Pro"}
@font-face
        {font-family:"Lucida Handwriting"}
p.x_x_x_MsoNormal, li.x_x_x_MsoNormal, div.x_x_x_MsoNormal
        {margin:0in;
        margin-bottom:.0001pt;
        font-size:11.0pt;
        font-family:"Calibri",sans-serif}
a:link, span.x_x_x_MsoHyperlink
        {color:#0563C1;
        text-decoration:underline}
span.x_x_x_EmailStyle17
        {font-family:"Calibri",sans-serif;
        color:windowtext}
.x_x_x_MsoChpDefault
        {font-family:"Calibri",sans-serif}
@page WordSection1
        {margin:1.0in 1.0in 1.0in 1.0in}
-->
</style>
                <div lang="EN-US">
                  <div class="x_x_x_WordSection1">
                    <p class="x_x_x_MsoNormal">Hello guys,</p>
                    <p class="x_x_x_MsoNormal"> </p>
                    <p class="x_x_x_MsoNormal">I observed, trying to
                      connect to <a href="http://mirror.starlingx.cengn.ca/mirror/starlingx/master/centos/" moz-do-not-send="true">
http://mirror.starlingx.cengn.ca/mirror/starlingx/master/centos/</a> ,
                      that cengn mirror server is down.</p>
                    <p class="x_x_x_MsoNormal"> </p>
                    <p class="x_x_x_MsoNormal">Best regards,</p>
                    <p class="x_x_x_MsoNormal">Alex<span style="color:#1F497D"></span></p>
                    <p class="x_x_x_MsoNormal"><span style="color:#1F497D"> </span></p>
                    <table class="x_x_x_MsoNormalTable" style="margin-left:.15in;
                      border-collapse:collapse" cellspacing="0" cellpadding="0" border="0">
                      <tbody>
                        <tr>
                          <td style="width:99.3pt; border:none;
                            border-right:solid #0071C5 1.5pt;
                            padding:0in 5.4pt 0in 5.4pt" width="132" valign="top">
                            <p class="x_x_x_MsoNormal"><span style="color:#00B0F0">  </span><span style="color:#00B0F0" lang="PL"><img id="x_x_x_Picture_x0020_1" alt="Logo
                                  Description automatically generated" class="" data-outlook-trace="F:3|T:3" src="cid:part9.C4E36393.AF04423D@windriver.com" width="127" height="55" border="0"></span></p>
                          </td>
                          <td style="width:365.75pt; padding:0in 5.4pt
                            0in 5.4pt" width="488" valign="top">
                            <p class="x_x_x_MsoNormal"><span style="">Dimofte
                                Alexandru</span></p>
                            <p class="x_x_x_MsoNormal"><span style="">Software
                                Engineer</span></p>
                            <p class="x_x_x_MsoNormal"><span style="">STARLINGX
                                TEAM</span></p>
                            <p class="x_x_x_MsoNormal"><span style="">Skype
                                no: +40 336403734</span></p>
                            <p class="x_x_x_MsoNormal"><span style="">Personal
                                Mobile: +40 743167456</span></p>
                            <p class="x_x_x_MsoNormal"><span style="color:#00B0F0"><a href="mailto:alexandru.dimofte@intel.com" moz-do-not-send="true"><span style="color:#00B0F0">alexandru.dimofte</span><span style="">@intel.com</span></a></span><span style=""></span></p>
                            <p class="x_x_x_MsoNormal"><span style="" lang="DE">Intel Romania</span></p>
                          </td>
                        </tr>
                      </tbody>
                    </table>
                    <p class="x_x_x_MsoNormal"><span style="font-size:9.0pt; font-family:"Lucida
                        Handwriting"; color:black"><br>
                        <br>
                      </span></p>
                    <p class="x_x_x_MsoNormal"> </p>
                  </div>
                </div>
              </div>
              <br>
              <fieldset class="x_mimeAttachmentHeader"></fieldset>
              <pre class="x_moz-quote-pre">_______________________________________________
Starlingx-discuss mailing list
<a class="x_moz-txt-link-abbreviated" href="mailto:Starlingx-discuss@lists.starlingx.io" moz-do-not-send="true">Starlingx-discuss@lists.starlingx.io</a>
<a class="x_moz-txt-link-freetext" href="http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss" moz-do-not-send="true">http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss</a>
</pre>
            </blockquote>
            <br>
            <fieldset class="x_mimeAttachmentHeader"></fieldset>
            <pre class="x_moz-quote-pre">_______________________________________________
Starlingx-discuss mailing list
<a class="x_moz-txt-link-abbreviated" href="mailto:Starlingx-discuss@lists.starlingx.io" moz-do-not-send="true">Starlingx-discuss@lists.starlingx.io</a>
<a class="x_moz-txt-link-freetext" href="http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss" moz-do-not-send="true">http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss</a>
</pre>
          </blockquote>
          <br>
          <fieldset class="x_mimeAttachmentHeader"></fieldset>
          <pre class="x_moz-quote-pre">_______________________________________________
Starlingx-discuss mailing list
<a class="x_moz-txt-link-abbreviated" href="mailto:Starlingx-discuss@lists.starlingx.io" moz-do-not-send="true">Starlingx-discuss@lists.starlingx.io</a>
<a class="x_moz-txt-link-freetext" href="http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss" moz-do-not-send="true">http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss</a>
</pre>
        </blockquote>
      </div>
      <br>
      <fieldset class="mimeAttachmentHeader"></fieldset>
      <pre class="moz-quote-pre" wrap="">_______________________________________________
Starlingx-discuss mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Starlingx-discuss@lists.starlingx.io">Starlingx-discuss@lists.starlingx.io</a>
<a class="moz-txt-link-freetext" href="http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss">http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss</a>
</pre>
    </blockquote>
    <p><br>
    </p>
  </body>
</html>