<html><head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
  </head>
  <body>
    <p>Update</p>
    <p><br>
    </p>
    <p>The CENGN builds will now return to the regular daily schedule. 
      <br>
    </p>
    <p><br>
    </p>
    <p>None of the new builds have been sanitized as yet.</p>
    <p><br>
    </p>
    <p>Until those builds are sanitized, it might be advisable to keep
      software updates to a minimum.  I'll defer to the TSC on that
      one.  However from a purely CENGN/mirror/build perspective, I
      think we are ready to allow updates to .lst files again.</p>
    <p><br>
    </p>
    <p>Scott</p>
    <p><br>
    </p>
    <p><br>
    </p>
    <div class="moz-cite-prefix">On 2021-01-31 4:11 p.m., Scott Little
      wrote:<br>
    </div>
    <blockquote type="cite" cite="mid:6d5f08e9-2250-4183-8295-2b9f9375557e@windriver.com">
      
      <p>Update</p>
      <p><br>
      </p>
      <p><br>
      </p>
      <p>CENGN hasn't found any specific network issues as of yet.  They
        have acknowledged that the current network configuration
        implemented in the wake of power outage and the switch to the
        new server is sub-optimal and will revisit it soon.</p>
      <p><br>
      </p>
      <p>We have worked over the last several days to get the firewall
        configuration right to allow all the required services on the
        build server to run.<br>
      </p>
      <p><br>
      </p>
      <p>One particular quirk of the current network setup is that the
        DNS reported address of the mirror is inaccessible.  An
        /etc/hosts entry is required to force the mirror address to
        resolve to an alternate IP address.  I've spent the last several
        sleepless nights learning how to push that change not only into
        our build container, but into all the STX service containers
        that we build. I can finally report successful builds of the
        entire layered build stack.</p>
      <p><br>
      </p>
      <p>On the mirror update side, I have adjusted the mirroring
        algorithm to do more validation steps as it downloads. This
        should prevent further corruptions, but will slow the frequency
        with with we can refresh our repos from upstream.  In stead of
        updating all repos daily, some will be on a ~1/week schedule. <br>
      </p>
      <p><br>
      </p>
      <p>Scott</p>
      <p><br>
      </p>
      <p><br>
      </p>
      <p><br>
      </p>
      <div class="moz-cite-prefix">On 2021-01-29 1:02 a.m., Scott Little
        wrote:<br>
      </div>
      <blockquote type="cite" cite="mid:7428784a-0ee0-5f86-fb4d-bc32213f11a3@windriver.com">
        <p>Update</p>
        <p><br>
        </p>
        <p>Well a few more networking issues have been solved. thanks to
          Davlet, and Moh at CENGN.</p>
        <p><br>
        </p>
        <p>We have had a successful build of the 'compiler' and 'distro'
          layers.  The 'flock' layer built the rpms, but failed to
          update the installer.  It appears to relate to the loopback
          device.  I'll look into that tomorrow. <br>
        </p>
        <p><br>
        </p>
        <p>~350 RPMS failed to restore from upstream.  None are critical
          to any of our current builds.  I don't plan to persu them any
          further<br>
        </p>
        <p><br>
        </p>
        <p>The mirroring job is throwing a lot more error messages than
          I'm used to seeing.  Many rpm download attempts are failing,
          and left 2000 new corrupt rpms in its wake.  I am cleaning up
          the new mess and adjusting the mirroring script to prevent
          download failures from leaving us with corrupt repos. The
          download failures seem to come in batches, so I'll reach out
          to CENGN to see if they had any network issues that might
          explain the problem. <br>
        </p>
        <p><br>
        </p>
        <p>Scott</p>
        <p><br>
        </p>
        <p><br>
        </p>
        <div class="moz-cite-prefix">On 2021-01-28 12:27 a.m., Scott
          Little wrote:<br>
        </div>
        <blockquote type="cite" cite="mid:d368ec67-490a-fb47-1f85-c52739939c7f@windriver.com">
          <div class="moz-cite-prefix">Update</div>
          <div class="moz-cite-prefix"><br>
          </div>
          <div class="moz-cite-prefix">All known firewall issues have
            been fixed.  <br>
          </div>
          <div class="moz-cite-prefix"><br>
          </div>
          <div class="moz-cite-prefix">Jenkins is now up and running on
            the build server once again. Our first manually triggered
            build attempt of the compiler layer is now running<br>
          </div>
          <div class="moz-cite-prefix"><br>
          </div>
          <div class="moz-cite-prefix">Of the original ~13000 corrupted
            rpms, we now have ...</div>
          <div class="moz-cite-prefix"><br>
          </div>
          <div class="moz-cite-prefix">~9250 have been restored,
            including all rpms that are knowm inputs to supported builds<br>
          </div>
          <div class="moz-cite-prefix"><br>
          </div>
          <div class="moz-cite-prefix">~3100 damaged rpms from the 2.0.0
            release will not restored.  TSC and community meetings
            agreed that 2.0.0, being 2 releases out of date, is no
            longer supported.<br>
          </div>
          <div class="moz-cite-prefix"><br>
          </div>
          <div class="moz-cite-prefix">~650 Have yet to be restored from
            Upstream.  The jenkins job that pulls from upstream sources
            continues to run.</div>
          <div class="moz-cite-prefix"><br>
          </div>
          <div class="moz-cite-prefix">I would like to have successful
            CENGN builds of all layers before you all hammer the new
            server with your pent-up download_mirror.sh requests. 
            Please wait a little longer.  We should be back in business
            sometime tomorrow.</div>
          <div class="moz-cite-prefix"><br>
          </div>
          <div class="moz-cite-prefix">Scott<br>
          </div>
          <div class="moz-cite-prefix"><br>
          </div>
          <div class="moz-cite-prefix"><br>
          </div>
          <div class="moz-cite-prefix"><br>
          </div>
          <div class="moz-cite-prefix">On 2021-01-27 9:56 a.m., Scott
            Little wrote:<br>
          </div>
          <blockquote type="cite" cite="mid:b8a3b522-40f5-274b-b3ab-1990640b345f@windriver.com">
            <div class="moz-cite-prefix">Update</div>
            <div class="moz-cite-prefix"><br>
            </div>
            <div class="moz-cite-prefix">Of the original ~13000
              corrupted rpms, we now have ...<br>
            </div>
            <div class="moz-cite-prefix"><br>
            </div>
            <div class="moz-cite-prefix">5700 have been restored</div>
            <div class="moz-cite-prefix"><br>
            </div>
            <div class="moz-cite-prefix">2400 have yet to be assessed</div>
            <div class="moz-cite-prefix"><br>
            </div>
            <div class="moz-cite-prefix">1800 having no backup within
              cengn. Upstream availability has yet to be determined.</div>
            <div class="moz-cite-prefix"><br>
            </div>
            <div class="moz-cite-prefix">3100 I propose to drop as they
              fall within the 2.0.0 release which is in very bad shape,
              whereas 2.0.1 is in fairly good shape and worth saving.<br>
            </div>
            <div class="moz-cite-prefix"><br>
            </div>
            <div class="moz-cite-prefix">Scott</div>
            <div class="moz-cite-prefix"><br>
            </div>
            <div class="moz-cite-prefix"><br>
            </div>
            <div class="moz-cite-prefix"><br>
            </div>
            <div class="moz-cite-prefix">On 2021-01-27 1:37 a.m., Scott
              Little wrote:<br>
            </div>
            <blockquote type="cite" cite="mid:34128b7c-d78d-13a5-8682-99b8615487a5@windriver.com">
              <p>The audit of repos identified ~13000 corrupted rpms</p>
              <p><br>
              </p>
              <p>So far ~5000 have been restored from redundant copies
                within cengn.  <br>
              </p>
              <p><br>
              </p>
              <p>Aprox 7500 have yet to be assessed.  Those remaining
                rpms are inputs or outputs of actual builds.  The rpms
                produced by builds might carry the same files name, but
                may contain different content due to embedded file dates
                if not real changes.  Extra care is required to restore
                from the correct source. If an exact match is
                unavailable, I may be forced to use a source that is
                'near' in time and therefore less likely to contain
                anything other than trivial file timestamp deltas.  I'm
                still pondering the implications of such a substitution.
                <br>
              </p>
              <p><br>
              </p>
              <p> That leaves ~500 having no backup within cengn. 
                Upstream availability has yet to be determined.</p>
              <p><br>
              </p>
              <p>Scott<br>
              </p>
              <p><br>
              </p>
              <p><br>
              </p>
              <div class="moz-cite-prefix">On 2021-01-26 1:28 a.m.,
                Scott Little wrote:<br>
              </div>
              <blockquote type="cite" cite="mid:eb52e4c9-f06d-4683-e068-5aebb51ce7e4@windriver.com">
                <div class="moz-cite-prefix">The jenkins in the cengn
                  build is offline while we repair the file system on
                  the mirror.  The build machine both read from, and
                  writes to the mirror, and I don't want it to cause
                  further corruptions while the mirror is being
                  repaired.<br>
                </div>
                <div class="moz-cite-prefix"><br>
                </div>
                <div class="moz-cite-prefix">The cengn mirror has been
                  cut over to the new storage back end.  However any
                  corrupt files from the old storage back end will
                  remain corrupted on the new one.<br>
                </div>
                <div class="moz-cite-prefix"><br>
                </div>
                <div class="moz-cite-prefix">Restoration of rpm repos
                  will proceed in several phases:</div>
                <div class="moz-cite-prefix"><br>
                </div>
                <div class="moz-cite-prefix">1) An audit of all rpm
                  repos is underway.  Rpm's that fail their checksum are
                  being removed and repodata will be updated to reflect
                  the missing RPMS.</div>
                <div class="moz-cite-prefix"><br>
                </div>
                <div class="moz-cite-prefix">2) The mirror update job
                  (downloading rpms from upstream rpm repos) will be run
                  to restore as many missing files as possible. Many
                  rpms may no longer be available as we kept old
                  versions of packages that upstream drops when a new
                  version is available,<br>
                </div>
                <div class="moz-cite-prefix"><br>
                </div>
                <div class="moz-cite-prefix">3) If rpms that are not
                  available from upstream, but are required for our
                  build, we can attempt to restore them from one of
                  several places.   e.g. the inputs directory of various
                  builds, or the downloads directory on the cengn build
                  server (independent storage from the mirror)<br>
                </div>
                <div class="moz-cite-prefix"><br>
                </div>
                <div class="moz-cite-prefix">4) Resume building new
                  loads<br>
                </div>
                <div class="moz-cite-prefix"><br>
                </div>
                <div class="moz-cite-prefix"><br>
                </div>
                <div class="moz-cite-prefix"><br>
                </div>
                <p>All of the release iso's on cengn were corrupt.</p>
                <p><br>
                </p>
                <p>I've been able to restore 4.0.1 and 2.0.1 from the
                  original build artifacts.</p>
                <p><br>
                </p>
                <p>Original build artifacts for 3.0.0 are missing.  I do
                  have build artifacts for builds that were intended to
                  lead up to a 3.0.1 release.  If a verifiable copy of
                  3.0.0 can't be found, we may want to consider issuing
                  a 3.0.1.  Alternatively I can rebuild 3.0.0, but the
                  file checksums wont match the original build due to
                  embedded timestamps.<br>
                </p>
                <p><br>
                </p>
                <p><br>
                </p>
                <div class="moz-cite-prefix">Scott</div>
                <div class="moz-cite-prefix"><br>
                </div>
                <div class="moz-cite-prefix"><br>
                </div>
                <div class="moz-cite-prefix"><br>
                </div>
                <div class="moz-cite-prefix">On 2021-01-25 11:08 a.m.,
                  Little, Scott wrote:<br>
                </div>
                <blockquote type="cite" cite="mid:BY5PR11MB441835A7B47040913D2228B186BD9@BY5PR11MB4418.namprd11.prod.outlook.com">
                  <div>Work continues at CENGN repair damaged repos and
                    to move us to a new backend infrastructure with
                    better redundancy.</div>
                  <div><br>
                  </div>
                  <div>I still recommend that you avoid using
                    download_mirrir.sh until the repos are fully
                    repaired.  I don’t have a firm ETA for that, perhaps
                    two before we have full confidence in the repairs.</div>
                  <div><br>
                  </div>
                  <div>Scott </div>
                  <div><br>
                  </div>
                  <div><br>
                  </div>
                  <hr style="display:inline-block;width:98%" tabindex="-1">
                  <div id="divRplyFwdMsg" dir="ltr"><font style="font-size:11pt" face="Calibri, sans-serif" color="#000000"><b>From:</b> Scott Little <a class="moz-txt-link-rfc2396E" href="mailto:scott.little@windriver.com" moz-do-not-send="true"><scott.little@windriver.com></a><br>
                      <b>Sent:</b> January 23, 2021 2:24 AM<br>
                      <b>To:</b> <a class="moz-txt-link-abbreviated" href="mailto:starlingx-discuss@lists.starlingx.io" moz-do-not-send="true">starlingx-discuss@lists.starlingx.io</a>
                      <a class="moz-txt-link-rfc2396E" href="mailto:starlingx-discuss@lists.starlingx.io" moz-do-not-send="true"><starlingx-discuss@lists.starlingx.io></a><br>
                      <b>Subject:</b> Re: [Starlingx-discuss] Cengn
                      Mirror server in down?!</font>
                    <div> </div>
                  </div>
                  <div>
                    <p>CENGN is up, but there seems to be significant
                      damage to the repositories.  <br>
                    </p>
                    <p><br>
                    </p>
                    <p>We are investigate options to restore the
                      repositories to health.</p>
                    <p><br>
                    </p>
                    <p>I do NOT recommend running download_mirror.sh at
                      this time.</p>
                    <p><br>
                    </p>
                    <p>Scott</p>
                    <p><br>
                    </p>
                    <p><br>
                    </p>
                    <p><br>
                    </p>
                    <div class="x_moz-cite-prefix">On 2021-01-22 3:53
                      p.m., Scott Little wrote:<br>
                    </div>
                    <blockquote type="cite">
                      <p>CENGN remains out of service.</p>
                      <p><br>
                      </p>
                      <p>A new problem has been identified with ceph
                        storage.</p>
                      <p><br>
                      </p>
                      <p>They continue to work the outage.  <br>
                      </p>
                      <p><br>
                      </p>
                      <p>Scott</p>
                      <p><br>
                      </p>
                      <p><br>
                      </p>
                      <p><br>
                      </p>
                      <div class="x_moz-cite-prefix">On 2021-01-22 9:55
                        a.m., Scott Little wrote:<br>
                      </div>
                      <blockquote type="cite">
                        <p>CENGN remains out of service</p>
                        <p><br>
                        </p>
                        <p>AC to the server room has been restored. They
                          are now working some firewall issues.</p>
                        <p><br>
                        </p>
                        <p>They hope to have it restored in a few hours.</p>
                        <p><br>
                        </p>
                        <p>Scott</p>
                        <p><br>
                        </p>
                        <p><br>
                        </p>
                        <div class="x_moz-cite-prefix">On 2021-01-21
                          10:07 a.m., Panech, Davlet wrote:<br>
                        </div>
                        <blockquote type="cite">
                          <style type="text/css" style="display:none">
<!--
p
        {margin-top:0;
        margin-bottom:0}
-->
</style>
                          <div style="font-family:Calibri,Arial,Helvetica,sans-serif;
                            font-size:12pt; color:rgb(0,0,0)"> The
                            hosting company had an issue with their
                            servers, they are working on it. They hope
                            to be back online mid afternoon (Jan 21,
                            Eastern Standard Time).<br>
                          </div>
                          <hr tabindex="-1" style="display:inline-block;
                            width:98%">
                          <div id="x_divRplyFwdMsg" dir="ltr"><font style="font-size:11pt" face="Calibri,
                              sans-serif" color="#000000"><b>From:</b>
                              Panech, Davlet <a class="x_moz-txt-link-rfc2396E" href="mailto:Davlet.Panech@windriver.com" moz-do-not-send="true"><Davlet.Panech@windriver.com></a><br>
                              <b>Sent:</b> January 21, 2021 9:56 AM<br>
                              <b>To:</b> Dimofte, Alexandru <a class="x_moz-txt-link-rfc2396E" href="mailto:alexandru.dimofte@intel.com" moz-do-not-send="true">
                                <alexandru.dimofte@intel.com></a>;
                              <a class="x_moz-txt-link-abbreviated" href="mailto:starlingx-discuss@lists.starlingx.io" moz-do-not-send="true">
                                starlingx-discuss@lists.starlingx.io</a>
                              <a class="x_moz-txt-link-rfc2396E" href="mailto:starlingx-discuss@lists.starlingx.io" moz-do-not-send="true">
                                <starlingx-discuss@lists.starlingx.io></a><br>
                              <b>Subject:</b> Re: Cengn Mirror server in
                              down?!</font>
                            <div> </div>
                          </div>
                          <style type="text/css" style="display:none">
<!--
p
        {margin-top:0;
        margin-bottom:0}
-->
</style>
                          <div dir="ltr">
                            <div style="font-family:Calibri,Arial,Helvetica,sans-serif;
                              font-size:12pt; color:rgb(0,0,0)"> Thanks
                              Alexandru, we are looking into it.<br>
                            </div>
                            <div style="font-family:Calibri,Arial,Helvetica,sans-serif;
                              font-size:12pt; color:rgb(0,0,0)"> <br>
                            </div>
                            <hr tabindex="-1" style="display:inline-block; width:98%">
                            <div id="x_x_divRplyFwdMsg" dir="ltr"><font style="font-size:11pt" face="Calibri,
                                sans-serif" color="#000000"><b>From:</b>
                                Dimofte, Alexandru <a class="x_moz-txt-link-rfc2396E" href="mailto:alexandru.dimofte@intel.com" moz-do-not-send="true"><alexandru.dimofte@intel.com></a><br>
                                <b>Sent:</b> January 21, 2021 3:24 AM<br>
                                <b>To:</b> <a class="x_moz-txt-link-abbreviated" href="mailto:starlingx-discuss@lists.starlingx.io" moz-do-not-send="true">
                                  starlingx-discuss@lists.starlingx.io</a>
                                <a class="x_moz-txt-link-rfc2396E" href="mailto:starlingx-discuss@lists.starlingx.io" moz-do-not-send="true">
                                  <starlingx-discuss@lists.starlingx.io></a><br>
                                <b>Subject:</b> [Starlingx-discuss]
                                Cengn Mirror server in down?!</font>
                              <div> </div>
                            </div>
                            <style>
<!--
@font-face
        {font-family:"Cambria Math"}
@font-face
        {font-family:Calibri}
@font-face
        {font-family:"Intel Clear"}
@font-face
        {font-family:"Intel Clear Pro"}
@font-face
        {font-family:"Lucida Handwriting"}
p.x_x_x_MsoNormal, li.x_x_x_MsoNormal, div.x_x_x_MsoNormal
        {margin:0in;
        margin-bottom:.0001pt;
        font-size:11.0pt;
        font-family:"Calibri",sans-serif}
a:link, span.x_x_x_MsoHyperlink
        {color:#0563C1;
        text-decoration:underline}
span.x_x_x_EmailStyle17
        {font-family:"Calibri",sans-serif;
        color:windowtext}
.x_x_x_MsoChpDefault
        {font-family:"Calibri",sans-serif}
@page WordSection1
        {margin:1.0in 1.0in 1.0in 1.0in}
-->
</style>
                            <div lang="EN-US">
                              <div class="x_x_x_WordSection1">
                                <p class="x_x_x_MsoNormal">Hello guys,</p>
                                <p class="x_x_x_MsoNormal"> </p>
                                <p class="x_x_x_MsoNormal">I observed,
                                  trying to connect to <a href="http://mirror.starlingx.cengn.ca/mirror/starlingx/master/centos/" moz-do-not-send="true">
http://mirror.starlingx.cengn.ca/mirror/starlingx/master/centos/</a> ,
                                  that cengn mirror server is down.</p>
                                <p class="x_x_x_MsoNormal"> </p>
                                <p class="x_x_x_MsoNormal">Best regards,</p>
                                <p class="x_x_x_MsoNormal">Alex<span style="color:#1F497D"></span></p>
                                <p class="x_x_x_MsoNormal"><span style="color:#1F497D"> </span></p>
                                <table class="x_x_x_MsoNormalTable" style="margin-left:.15in;
                                  border-collapse:collapse" cellspacing="0" cellpadding="0" border="0">
                                  <tbody>
                                    <tr>
                                      <td style="width:99.3pt;
                                        border:none; border-right:solid
                                        #0071C5 1.5pt; padding:0in 5.4pt
                                        0in 5.4pt" width="132" valign="top">
                                        <p class="x_x_x_MsoNormal"><span style="color:#00B0F0">  </span><span style="color:#00B0F0" lang="PL"><img id="x_x_x_Picture_x0020_1" alt="Logo Description
                                              automatically generated" class="" data-outlook-trace="F:3|T:3" src="cid:part12.269E9EF5.83F64821@windriver.com" width="127" height="55" border="0"></span></p>
                                      </td>
                                      <td style="width:365.75pt;
                                        padding:0in 5.4pt 0in 5.4pt" width="488" valign="top">
                                        <p class="x_x_x_MsoNormal"><span style="">Dimofte Alexandru</span></p>
                                        <p class="x_x_x_MsoNormal"><span style="">Software Engineer</span></p>
                                        <p class="x_x_x_MsoNormal"><span style="">STARLINGX TEAM</span></p>
                                        <p class="x_x_x_MsoNormal"><span style="">Skype no: +40
                                            336403734</span></p>
                                        <p class="x_x_x_MsoNormal"><span style="">Personal Mobile:
                                            +40 743167456</span></p>
                                        <p class="x_x_x_MsoNormal"><span style="color:#00B0F0"><a href="mailto:alexandru.dimofte@intel.com" moz-do-not-send="true"><span style="color:#00B0F0">alexandru.dimofte</span><span style="">@intel.com</span></a></span><span style=""></span></p>
                                        <p class="x_x_x_MsoNormal"><span style="" lang="DE">Intel
                                            Romania</span></p>
                                      </td>
                                    </tr>
                                  </tbody>
                                </table>
                                <p class="x_x_x_MsoNormal"><span style="font-size:9.0pt;
                                    font-family:"Lucida
                                    Handwriting"; color:black"><br>
                                    <br>
                                  </span></p>
                                <p class="x_x_x_MsoNormal"> </p>
                              </div>
                            </div>
                          </div>
                          <br>
                          <fieldset class="x_mimeAttachmentHeader"></fieldset>
                          <pre class="x_moz-quote-pre">_______________________________________________
Starlingx-discuss mailing list
<a class="x_moz-txt-link-abbreviated" href="mailto:Starlingx-discuss@lists.starlingx.io" moz-do-not-send="true">Starlingx-discuss@lists.starlingx.io</a>
<a class="x_moz-txt-link-freetext" href="http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss" moz-do-not-send="true">http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss</a>
</pre>
                        </blockquote>
                        <br>
                        <fieldset class="x_mimeAttachmentHeader"></fieldset>
                        <pre class="x_moz-quote-pre">_______________________________________________
Starlingx-discuss mailing list
<a class="x_moz-txt-link-abbreviated" href="mailto:Starlingx-discuss@lists.starlingx.io" moz-do-not-send="true">Starlingx-discuss@lists.starlingx.io</a>
<a class="x_moz-txt-link-freetext" href="http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss" moz-do-not-send="true">http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss</a>
</pre>
                      </blockquote>
                      <br>
                      <fieldset class="x_mimeAttachmentHeader"></fieldset>
                      <pre class="x_moz-quote-pre">_______________________________________________
Starlingx-discuss mailing list
<a class="x_moz-txt-link-abbreviated" href="mailto:Starlingx-discuss@lists.starlingx.io" moz-do-not-send="true">Starlingx-discuss@lists.starlingx.io</a>
<a class="x_moz-txt-link-freetext" href="http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss" moz-do-not-send="true">http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss</a>
</pre>
                    </blockquote>
                  </div>
                  <br>
                  <fieldset class="mimeAttachmentHeader"></fieldset>
                  <pre class="moz-quote-pre" wrap="">_______________________________________________
Starlingx-discuss mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Starlingx-discuss@lists.starlingx.io" moz-do-not-send="true">Starlingx-discuss@lists.starlingx.io</a>
<a class="moz-txt-link-freetext" href="http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss" moz-do-not-send="true">http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss</a>
</pre>
                </blockquote>
                <p><br>
                </p>
                <br>
                <fieldset class="mimeAttachmentHeader"></fieldset>
                <pre class="moz-quote-pre" wrap="">_______________________________________________
Starlingx-discuss mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Starlingx-discuss@lists.starlingx.io" moz-do-not-send="true">Starlingx-discuss@lists.starlingx.io</a>
<a class="moz-txt-link-freetext" href="http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss" moz-do-not-send="true">http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss</a>
</pre>
              </blockquote>
              <br>
              <fieldset class="mimeAttachmentHeader"></fieldset>
              <pre class="moz-quote-pre" wrap="">_______________________________________________
Starlingx-discuss mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Starlingx-discuss@lists.starlingx.io" moz-do-not-send="true">Starlingx-discuss@lists.starlingx.io</a>
<a class="moz-txt-link-freetext" href="http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss" moz-do-not-send="true">http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss</a>
</pre>
            </blockquote>
            <p><br>
            </p>
            <br>
            <fieldset class="mimeAttachmentHeader"></fieldset>
            <pre class="moz-quote-pre" wrap="">_______________________________________________
Starlingx-discuss mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Starlingx-discuss@lists.starlingx.io" moz-do-not-send="true">Starlingx-discuss@lists.starlingx.io</a>
<a class="moz-txt-link-freetext" href="http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss" moz-do-not-send="true">http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss</a>
</pre>
          </blockquote>
          <p><br>
          </p>
          <br>
          <fieldset class="mimeAttachmentHeader"></fieldset>
          <pre class="moz-quote-pre" wrap="">_______________________________________________
Starlingx-discuss mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Starlingx-discuss@lists.starlingx.io" moz-do-not-send="true">Starlingx-discuss@lists.starlingx.io</a>
<a class="moz-txt-link-freetext" href="http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss" moz-do-not-send="true">http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss</a>
</pre>
        </blockquote>
        <br>
        <fieldset class="mimeAttachmentHeader"></fieldset>
        <pre class="moz-quote-pre" wrap="">_______________________________________________
Starlingx-discuss mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Starlingx-discuss@lists.starlingx.io" moz-do-not-send="true">Starlingx-discuss@lists.starlingx.io</a>
<a class="moz-txt-link-freetext" href="http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss" moz-do-not-send="true">http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss</a>
</pre>
      </blockquote>
    </blockquote>
  </body>
</html>