[Starlingx-discuss] [ Test ] Sanity results - summary posted

McKenna, Jason Jason.McKenna at windriver.com
Thu Sep 13 19:50:16 UTC 2018



> -----Original Message-----
> From: Cordoba Malibran, Erich <erich.cordoba.malibran at intel.com>
> Sent: September 13, 2018 2:46 PM
> To: Young, Ken <Ken.Young at windriver.com>; Penney, Don
> <Don.Penney at windriver.com>
> Cc: starlingx-discuss at lists.starlingx.io
> Subject: Re: [Starlingx-discuss] [ Test ] Sanity results - summary posted
> 
> Indeed, those two files had the 600 permissions in our mirror and the
> permissions were transfered into the ISO and then to the /ww/pages/feed
> folder. Changing the permissions now let the compute to be installed.
> 
> I have two questions regarding this:
> 
>   - Which component is responsible to populate the /www/pages/feed folder
> with the rpms. I'm wondering if some permission checking/set needs to be
> done here.

+1

>   - lighttpd runs as non-root user, but the content on /www/pages/feed is
> owned by root. Should this folder be owned by the same user as the one
> that runs lighttpd?

The data needs to be readable by the user running lighttpd, but its okay if it's owned by someone else (including root)

> 
> Thanks
> 
> -Erich
> 
> On Wed, 2018-09-12 at 21:37 +0000, Penney, Don wrote:
> > The problem is one of permissions. In the /www/var/log/lighttpd-
> > access.log file, you can see the http access request is being rejected
> > with a 403 status code, indicating access is forbidden:
> >
> > 192.168.204.85 pxecontroller - [12/Sep/2018:09:30:52 +0000] "GET
> > /feed/rel-18.08/Packages/python2-qpid-proton-0.24.0-1.el7.x86_64.rpm
> > HTTP/1.1" 403 345 "-" "urlgrabber/3.10 yum/3.4.3"
> >
> > You can see this also with a wget command:
> > localhost:/home/wrsroot# wget http://localhost/feed/rel-18.08/Package
> > s/python2-qpid-proton-0.24.0-1.el7.x86_64.rpm
> > --2018-09-12 20:54:19--  http://localhost/feed/rel-18.08/Packages/pyt
> > hon2-qpid-proton-0.24.0-1.el7.x86_64.rpm
> > Resolving localhost (localhost)... ::1, 127.0.0.1 Connecting to
> > localhost (localhost)|::1|:80... connected.
> > HTTP request sent, awaiting response... 403 Forbidden
> > 2018-09-12 20:54:19 ERROR 403: Forbidden.
> >
> >
> > The problem is that two of the RPMs on the ISO have 600 permissions,
> > where all other RPMs are 644. This is carried over when the files are
> > copied to the controller during the post-install phase:
> >
> > controller-0:~$ ls -l /mnt/iso/Packages/ | grep -- ---
> > -rw------- 1 1001 users    258624 Aug 28 12:04 python2-qpid-proton-
> > 0.24.0-1.el7.x86_64.rpm
> > -rw------- 1 1001 users    190128 Aug 28 12:04 qpid-proton-c-0.24.0-
> > 1.el7.x86_64.rpm
> >
> > controller-0:~$ ls -l /www/pages/feed/rel-18.08/Packages/ |grep -- --
> > -
> > -rw-------. 1 root root    258624 Sep 12 20:33 python2-qpid-proton-
> > 0.24.0-1.el7.x86_64.rpm
> > -rw-------. 1 root root    190128 Sep 12 20:33 qpid-proton-c-0.24.0-
> > 1.el7.x86_64.rpm
> >
> >
> > The lighttpd server runs as a non-root user, and cannot access these
> > files. Change the permissions on the files, and the installation of
> > controller-1 should be successful.
> >
> >
> > -----Original Message-----
> > From: Young, Ken [mailto:Ken.Young at windriver.com]
> > Sent: Wednesday, September 12, 2018 4:53 PM
> > To: Cordoba Malibran, Erich
> > Cc: starlingx-discuss at lists.starlingx.io
> > Subject: Re: [Starlingx-discuss] [ Test ] Sanity results - summary
> > posted
> >
> > Erich,
> >
> > One correction below: we are using a Wind River version of the
> > installer (I am confusing this thread with another thread).  This is
> > the installer we shipped Titanium Cloud Release 5.
> >
> > This begs the question: what installer were you using before the
> > changes to the kernel?
> >
> > /KenY
> >
> > On 2018-09-12, 4:36 PM, "Young, Ken" <Ken.Young at windriver.com>
> > wrote:
> >
> >     Erich,
> >
> >     A couple of data points on our side.  For our builds, we are using
> > stock CentOS installer for 7.4.  These are unchanged and we have not
> > seen this issue on our side.
> >
> >     Have you isolated when you made the changes to the installer?  Is
> > it the same time you started seeing this error?  Does it make sense to
> > revert to the stock installer while we get to the bottom of this issue
> > to continue sanity?
> >
> >     Regards,
> >     Ken Y
> >
> >     On 2018-09-11, 8:30 PM, "Cordoba Malibran, Erich" <erich.cordoba.
> > malibran at intel.com> wrote:
> >
> >         I have narrowed down a little bit the issue, these are my
> > findings.
> >
> >           - I compared the bootimage_michel.iso with the stx-2018-09-
> > 11-
> >         111.iso, which is today's ISO.
> >           - The are not significant package differences between them,
> > minor
> >         versions has changed in some packages.
> >           - The issue is that once the kernel has been loaded through
> > PXE,
> >         anaconda is unable to find python2-qpid package. Not sure if
> > this is
> >         the first package that is requested.
> >           - From the anaconda shell I can see the http://pxecontrolle
> > r/ serving
> >         the files and the repodata seems to be updated there. I
> > installed yum
> >         manually and I was able to install python2-qpid package, (and
> > any other
> >         available).
> >           - The content of the pxe-network-installer package is not
> > the same,
> >         WR squashfs.img file has the kernel 3.10.0-693.2.2 but ours
> > has 3.10.0-
> >         862.6.3. We regenerate those files when the kernel was
> > upgraded. (maybe
> >         this matches with the date where the issue started to
> > arise..)
> >
> >         So, my guess now is that the problem could be caused by the
> > pxe-
> >         network-installer package. We create those files with the
> > update-
> >         network-installer script. That's the same used in WR to
> > generate these
> >         files?
> >
> >         I'll keep diging in the generation of these files to see if
> > there is
> >         any problem.
> >
> >         -Erich
> >
> >         On Tue, 2018-09-11 at 20:03 +0000, Lara, Cesar wrote:
> >         > I just wanted to make sure that we are able to reproduce the
> > issue
> >         > regardless of the build environment.
> >         >
> >         > AFAIK CentOS 7.5 upgrade is a different issue/thread, but at
> > the end
> >         > we can fine tune a process in which we are in the same
> > conditions to
> >         > be able to replicate errors in any given build environment.
> >         >
> >         > Regards
> >         > Cesar Lara
> >         >
> >         >
> >         > -----Original Message-----
> >         > From: McKenna, Jason [mailto:Jason.McKenna at windriver.com]
> >         > Sent: Tuesday, September 11, 2018 2:02 PM
> >         > To: Lara, Cesar <cesar.lara at intel.com>; Alonso, Juan Carlos
> > <juan.car
> >         > los.alonso at intel.com>; Khalil, Ghada <Ghada.Khalil at windrive
> > r.com>;
> >         > Martinez Monroy, Elio <elio.martinez.monroy at intel.com>;
> > Jones, Bruce
> >         > E <bruce.e.jones at intel.com>; Cabrales, Ada <ada.cabrales at in
> > tel.com>;
> >         > starlingx-discuss at lists.starlingx.io; Thebeau, Michel
> > <Michel.Thebeau
> >         > @windriver.com>; Young, Ken <Ken.Young at windriver.com>;
> > Little, Scott
> >         > <Scott.Little at windriver.com>
> >         > Subject: RE: [ Test ] Sanity results - summary posted
> >         >
> >         > Hi Cesar,
> >         >
> >         > Would you elaborate on what differences you'd expect to see
> > in an
> >         > empty mirror populated by the download tool, compared to an
> > existing
> >         > mirror which was updated by the download tool?
> >         >
> >         > The mirror tools were written specifically to contain
> > multiple
> >         > versions of packages.  The build environment population
> > tools take
> >         > the .lst file you have checked out, and places that version
> > from the
> >         > mirror into your local environment guaranteeing that your
> > build env
> >         > matches the .lst file.  If your mirror contains versions
> > 1.0.0,
> >         > 1.2.3, and 2.0.0 of a given package, then your environment
> > will only
> >         > use the version called out in the .lst file you have checked
> > out.
> >         >
> >         > This allows the same mirror to be used for multiple
> > branches, for
> >         > multiple releases, for multiple development environments,
> > etc, and
> >         > guarantees that a build can be reproduced down the road even
> > if the
> >         > build inputs disappear from the Internet at some point in
> > the
> >         > future.  As long as all the artifacts named in the .lst file
> > have
> >         > been downloaded at some point in the past, the build
> > environment
> >         > produced from a fresh mirror vs a continually updated mirror
> > will be
> >         > identical.
> >         >
> >         > The issue we are observing with the CentOS 7.5 update is
> > that the
> >         > versions of packages called out in the .lst files for the
> > centos75
> >         > branch of stx-tools do not match the versions of packages
> > called out
> >         > in centos75 branch of the other repos, like stx-integ.  The
> > other
> >         > branches are attempting to patch a (new) version of the
> > source code,
> >         > but the .lst file is only bringing in (or downloading) and
> > older
> >         > version of the source.
> >         >
> >         > -Jason
> >         >
> >         > > -----Original Message-----
> >         > > From: Lara, Cesar <cesar.lara at intel.com>
> >         > > Sent: September 11, 2018 1:44 PM
> >         > > To: Alonso, Juan Carlos <juan.carlos.alonso at intel.com>;
> > Khalil,
> >         > > Ghada
> >         > > <Ghada.Khalil at windriver.com>; Martinez Monroy, Elio
> >         > > <elio.martinez.monroy at intel.com>; Jones, Bruce E
> >         > > <bruce.e.jones at intel.com>; Cabrales, Ada <ada.cabrales at in
> > tel.com>;
> >         > > starlingx-discuss at lists.starlingx.io; Thebeau, Michel
> >         > > <Michel.Thebeau at windriver.com>
> >         > > Subject: Re: [Starlingx-discuss] [ Test ] Sanity results
> > - summary
> >         > > posted
> >         > >
> >         > > Hi all,
> >         > >
> >         > > Just wanted to point out that, to be able to reproduce the
> > issues
> >         > > from
> >         > > Intel's ISO, somebody will need to go and create a build
> >         > > environment
> >         > > from scratch, using external upstream repositories to
> > generate a
> >         > > new
> >         > > mirror, just the way we are doing it in our environment.
> > Failing to
> >         > > do
> >         > > this will result in a apples to oranges comparison.
> >         > >
> >         > > Regards
> >         > > Cesar Lara
> >         > >
> >         > > -----Original Message-----
> >         > > From: Alonso, Juan Carlos [mailto:juan.carlos.alonso at inte
> > l.com]
> >         > > Sent: Tuesday, September 11, 2018 10:54 AM
> >         > > To: Khalil, Ghada <Ghada.Khalil at windriver.com>; Martinez
> > Monroy,
> >         > > Elio
> >         > > <elio.martinez.monroy at intel.com>; Jones, Bruce E
> >         > > <bruce.e.jones at intel.com>; Cabrales, Ada <ada.cabrales at in
> > tel.com>;
> >         > > starlingx-discuss at lists.starlingx.io; Thebeau, Michel
> >         > > <Michel.Thebeau at windriver.com>
> >         > > Subject: Re: [Starlingx-discuss] [ Test ] Sanity results
> > - summary
> >         > > posted
> >         > >
> >         > > Hi Ghada,
> >         > >
> >         > > Yes, before shared our ISOs I followed such steps and got
> > the same
> >         > > issues.
> >         > >
> >         > > As I mention in the issue
> >         > > (https://bugs.launchpad.net/starlingx/+bug/1790213), I
> > could get
> >         > > the
> >         > > ISO from Michel and testes in our infrastructure, issues
> > not found.
> >         > > I
> >         > > guess there are differences between the way an ISO is
> > built by
> >         > > Intel
> >         > > and Wind River since Intel got issues but Wind River does
> > not. I
> >         > > think we should sync about this.
> >         > >
> >         > > Besides, our test infrastructure uses other commands to
> > generate
> >         > > VMs,
> >         > > add/install hosts (controllers and computes), such
> > commands was
> >         > > gotten
> >         > > from "tic_vb" repo, such repo was given by China team, if
> > I am not
> >         > > wrong such repo was based on Wind River documentation; our
> > test
> >         > > framework is based on "tic_vb". Anyway both ways got the
> > same
> >         > > results.
> >         > >
> >         > > Regards.
> >         > > Juan Carlos Alonso
> >         > >
> >         > > -----Original Message-----
> >         > > From: Khalil, Ghada [mailto:Ghada.Khalil at windriver.com]
> >         > > Sent: Tuesday, September 11, 2018 8:53 AM
> >         > > To: Alonso, Juan Carlos <juan.carlos.alonso at intel.com>;
> > Martinez
> >         > > Monroy, Elio <elio.martinez.monroy at intel.com>; Jones,
> > Bruce E
> >         > > <bruce.e.jones at intel.com>; Cabrales, Ada <ada.cabrales at in
> > tel.com>;
> >         > > starlingx-discuss at lists.starlingx.io; Thebeau, Michel
> >         > > <Michel.Thebeau at windriver.com>
> >         > > Subject: RE: [ Test ] Sanity results - summary posted
> >         > >
> >         > > Hello Juan,
> >         > > I read the notes in the Launchpad.
> >         > >
> >         > > And, just to confirm, are you following the instructions
> > under:
> >         > > https://wiki.openstack.org/wiki/StarlingX/Installation_Gu
> > ide_Virtua
> >         > > l_E
> >         > > nviro
> >         > > nment/Controller_Storage
> >         > > and still reproducing the two issues below?
> >         > >
> >         > > I was not able to definitively tell from the Launchpad
> > notes. It's
> >         > > important that both our teams are comparing apples to
> > apples.
> >         > >
> >         > > Thanks,
> >         > > Ghada
> >         > >
> >         > > -----Original Message-----
> >         > > From: Alonso, Juan Carlos [mailto:juan.carlos.alonso at inte
> > l.com]
> >         > > Sent: Monday, September 10, 2018 1:50 PM
> >         > > To: Khalil, Ghada; Martinez Monroy, Elio; Jones, Bruce E;
> >         > > Cabrales,
> >         > > Ada; starlingx-discuss at lists.starlingx.io; Thebeau, Michel
> >         > > Subject: RE: [ Test ] Sanity results - summary posted
> >         > >
> >         > > Hi,
> >         > >
> >         > > Such launchpads were updated.
> >         > >
> >         > > A VM was set to upload the ISO tx-2018-08-28-88.iso, such
> > ISO
> >         > > present
> >         > > two issues. Michael will upload his ISO, then we will re
> > test
> >         > > witch
> >         > > each iso in our environment.
> >         > >
> >         > > An script is being developed to get the differences
> > between ISOs.
> >         > >
> >         > > Regards.
> >         > > Juan Carlos Alonso
> >         > >
> >         > > -----Original Message-----
> >         > > From: Khalil, Ghada [mailto:Ghada.Khalil at windriver.com]
> >         > > Sent: Friday, September 7, 2018 4:43 PM
> >         > > To: Khalil, Ghada <Ghada.Khalil at windriver.com>; Martinez
> > Monroy,
> >         > > Elio
> >         > > <elio.martinez.monroy at intel.com>; Jones, Bruce E
> >         > > <bruce.e.jones at intel.com>; Cabrales, Ada <ada.cabrales at in
> > tel.com>;
> >         > > starlingx-discuss at lists.starlingx.io; Thebeau, Michel
> >         > > <Michel.Thebeau at windriver.com>
> >         > > Subject: Re: [Starlingx-discuss] [ Test ] Sanity results
> > - summary
> >         > > posted
> >         > >
> >         > > Hi Elio/Ada,
> >         > > Are there any updates from your side on these two
> > Launchpads?
> >         > > https://bugs.launchpad.net/starlingx/+bug/1790213
> >         > > https://bugs.launchpad.net/starlingx/+bug/1790716
> >         > >
> >         > > Michel Thebeau (Wind River) has been trying to reproduce
> > these two
> >         > > issues (following the starlingx wikis) without success for
> > two
> >         > > days
> >         > > now. He has been adding his findings to the bugs and
> > communicating
> >         > > with your team.
> >         > > As a result of his investigation, he will be pushing some
> > small
> >         > > fixes
> >         > > to stx- tools shortly (some cleanup).
> >         > >
> >         > > Please let us know if you need any further help from us.
> >         > >
> >         > > Thanks,
> >         > > Ghada
> >         > >
> >         > > -----Original Message-----
> >         > > From: Khalil, Ghada [mailto:Ghada.Khalil at windriver.com]
> >         > > Sent: Wednesday, September 05, 2018 5:39 PM
> >         > > To: Martinez Monroy, Elio; Jones, Bruce E; Cabrales, Ada;
> >         > > starlingx-
> >         > > discuss at lists.starlingx.io
> >         > > Subject: Re: [Starlingx-discuss] [ Test ] Sanity results
> > - summary
> >         > > posted
> >         > >
> >         > > Please see inline. There are notes in all the launchpads
> > as well.
> >         > >
> >         > > I have tagged the last bug for the October release. The
> > rest need
> >         > > more
> >         > > information to determine whether they gate or not: two
> > maybe
> >         > > procedural and one requires more information.
> >         > >
> >         > > -----Original Message-----
> >         > > From: Martinez Monroy, Elio [mailto:elio.martinez.monroy@
> > intel.com]
> >         > > Sent: Wednesday, September 05, 2018 2:23 PM
> >         > > To: Jones, Bruce E; Cabrales, Ada;
> >         > > starlingx-discuss at lists.starlingx.io
> >         > > Subject: Re: [Starlingx-discuss] [ Test ] Sanity results
> > - summary
> >         > > posted
> >         > >
> >         > > My opinion below
> >         > >
> >         > > -----Original Message-----
> >         > > From: Jones, Bruce E [mailto:bruce.e.jones at intel.com]
> >         > > Sent: Wednesday, September 5, 2018 12:56 PM
> >         > > To: Cabrales, Ada <ada.cabrales at intel.com>; starlingx-
> >         > > discuss at lists.starlingx.io
> >         > > Subject: Re: [Starlingx-discuss] [ Test ] Sanity results
> > - summary
> >         > > posted
> >         > >
> >         > > Should the bugs coming out of this be tagged/fixed for the
> > October
> >         > > release?
> >         > >
> >         > > Related bugs:
> >         > > https://bugs.launchpad.net/starlingx/+bug/1790213 < ---
> > Must, this
> >         > > bug
> >         > > is related with 2nd controller provisioning [[GK]] Using
> > the
> >         > > instructions published on the wiki, our designer was not
> > able to
> >         > > reproduce this issue.
> >         > > What instructions are the test team using?
> >         > >
> >         > > https://bugs.launchpad.net/starlingx/+bug/1790514 < ---
> > Can wait
> >         > > until
> >         > > other issues are solved, will fail some test cases related
> > to SSH
> >         > > [[GK]] Requested more information. This cannot be debugged
> > further
> >         > > without this information.
> >         > >
> >         > > https://bugs.launchpad.net/starlingx/+bug/1790716 < ---
> > Must, all
> >         > > instances need at least one compute for multimode [[GK]]
> > Using the
> >         > > instructions published on the wiki, our designer was not
> > able to
> >         > > reproduce this issue. The test team points to internal
> >         > > instructions
> >         > > which we have no access to. The supported install recipes
> > need to
> >         > > be
> >         > > on the wiki. I think it is an invalid bug
> >         > >
> >         > > https://bugs.launchpad.net/starlingx/+bug/1790718 < ---
> > Can wait
> >         > > since
> >         > > is related to FM , test cases will fail [[GK]] As Brent
> > pointed
> >         > > out,
> >         > > this is dependent on the integration of stx-gui which is
> > in
> >         > > progress.
> >         > >
> >         > > -----Original Message-----
> >         > > From: Cabrales, Ada [mailto:ada.cabrales at intel.com]
> >         > > Sent: Wednesday, September 5, 2018 10:50 AM
> >         > > To: starlingx-discuss at lists.starlingx.io
> >         > > Subject: [Starlingx-discuss] [ Test ] Sanity results -
> > summary
> >         > > posted
> >         > >
> >         > > Hello all,
> >         > >
> >         > >     As mentioned on the project meeting today, we're
> > posting the
> >         > > summary of the sanity test runs on the Test wiki [0]. The
> > sanity
> >         > > is
> >         > > triggered each time an ISO is generated (a daily task).
> >         > >
> >         > >    Right now we are posting the results manually, and
> > working on a
> >         > > way
> >         > > for automating it. Logs of the run are not included, if
> > you want
> >         > > to
> >         > > have them, you can contact Elio to get them sent to you.
> >         > >
> >         > >
> >         > > [0]
> >         > > https://wiki.openstack.org/wiki/StarlingX/Test#Latest_ISO
> > _image_San
> >         > > ity
> >         > > _S
> >         > > ummary
> >         > >
> >         > > Regards
> >         > > Ada
> >         > >
> >         > > _______________________________________________
> >         > > Starlingx-discuss mailing list
> >         > > Starlingx-discuss at lists.starlingx.io
> >         > > http://lists.starlingx.io/cgi-bin/mailman/listinfo/starli
> > ngx-discus
> >         > > s
> >         > > _______________________________________________
> >         > > Starlingx-discuss mailing list
> >         > > Starlingx-discuss at lists.starlingx.io
> >         > > http://lists.starlingx.io/cgi-bin/mailman/listinfo/starli
> > ngx-discus
> >         > > s
> >         > > _______________________________________________
> >         > > Starlingx-discuss mailing list
> >         > > Starlingx-discuss at lists.starlingx.io
> >         > > http://lists.starlingx.io/cgi-bin/mailman/listinfo/starli
> > ngx-discus
> >         > > s
> >         > > _______________________________________________
> >         > > Starlingx-discuss mailing list
> >         > > Starlingx-discuss at lists.starlingx.io
> >         > > http://lists.starlingx.io/cgi-bin/mailman/listinfo/starli
> > ngx-discus
> >         > > s
> >         > > _______________________________________________
> >         > > Starlingx-discuss mailing list
> >         > > Starlingx-discuss at lists.starlingx.io
> >         > > http://lists.starlingx.io/cgi-bin/mailman/listinfo/starli
> > ngx-discus
> >         > > s
> >         > > _______________________________________________
> >         > > Starlingx-discuss mailing list
> >         > > Starlingx-discuss at lists.starlingx.io
> >         > > http://lists.starlingx.io/cgi-bin/mailman/listinfo/starli
> > ngx-discus
> >         > > s
> >         > > _______________________________________________
> >         > > Starlingx-discuss mailing list
> >         > > Starlingx-discuss at lists.starlingx.io
> >         > > http://lists.starlingx.io/cgi-bin/mailman/listinfo/starli
> > ngx-discus
> >         > > s
> >         >
> >         > _______________________________________________
> >         > Starlingx-discuss mailing list
> >         > Starlingx-discuss at lists.starlingx.io
> >         > http://lists.starlingx.io/cgi-bin/mailman/listinfo/starling
> > x-discuss
> >
> >     _______________________________________________
> >     Starlingx-discuss mailing list
> >     Starlingx-discuss at lists.starlingx.io
> >     http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-disc
> > uss
> >
> >
> > _______________________________________________
> > Starlingx-discuss mailing list
> > Starlingx-discuss at lists.starlingx.io
> > http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
> _______________________________________________
> Starlingx-discuss mailing list
> Starlingx-discuss at lists.starlingx.io
> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss


More information about the Starlingx-discuss mailing list