[Starlingx-discuss] [ Test ] Sanity results - summary posted
McKenna, Jason
Jason.McKenna at windriver.com
Thu Sep 13 19:50:16 UTC 2018
> -----Original Message-----
> From: Cordoba Malibran, Erich <erich.cordoba.malibran at intel.com>
> Sent: September 13, 2018 2:46 PM
> To: Young, Ken <Ken.Young at windriver.com>; Penney, Don
> <Don.Penney at windriver.com>
> Cc: starlingx-discuss at lists.starlingx.io
> Subject: Re: [Starlingx-discuss] [ Test ] Sanity results - summary posted
>
> Indeed, those two files had the 600 permissions in our mirror and the
> permissions were transfered into the ISO and then to the /ww/pages/feed
> folder. Changing the permissions now let the compute to be installed.
>
> I have two questions regarding this:
>
> - Which component is responsible to populate the /www/pages/feed folder
> with the rpms. I'm wondering if some permission checking/set needs to be
> done here.
+1
> - lighttpd runs as non-root user, but the content on /www/pages/feed is
> owned by root. Should this folder be owned by the same user as the one
> that runs lighttpd?
The data needs to be readable by the user running lighttpd, but its okay if it's owned by someone else (including root)
>
> Thanks
>
> -Erich
>
> On Wed, 2018-09-12 at 21:37 +0000, Penney, Don wrote:
> > The problem is one of permissions. In the /www/var/log/lighttpd-
> > access.log file, you can see the http access request is being rejected
> > with a 403 status code, indicating access is forbidden:
> >
> > 192.168.204.85 pxecontroller - [12/Sep/2018:09:30:52 +0000] "GET
> > /feed/rel-18.08/Packages/python2-qpid-proton-0.24.0-1.el7.x86_64.rpm
> > HTTP/1.1" 403 345 "-" "urlgrabber/3.10 yum/3.4.3"
> >
> > You can see this also with a wget command:
> > localhost:/home/wrsroot# wget http://localhost/feed/rel-18.08/Package
> > s/python2-qpid-proton-0.24.0-1.el7.x86_64.rpm
> > --2018-09-12 20:54:19-- http://localhost/feed/rel-18.08/Packages/pyt
> > hon2-qpid-proton-0.24.0-1.el7.x86_64.rpm
> > Resolving localhost (localhost)... ::1, 127.0.0.1 Connecting to
> > localhost (localhost)|::1|:80... connected.
> > HTTP request sent, awaiting response... 403 Forbidden
> > 2018-09-12 20:54:19 ERROR 403: Forbidden.
> >
> >
> > The problem is that two of the RPMs on the ISO have 600 permissions,
> > where all other RPMs are 644. This is carried over when the files are
> > copied to the controller during the post-install phase:
> >
> > controller-0:~$ ls -l /mnt/iso/Packages/ | grep -- ---
> > -rw------- 1 1001 users 258624 Aug 28 12:04 python2-qpid-proton-
> > 0.24.0-1.el7.x86_64.rpm
> > -rw------- 1 1001 users 190128 Aug 28 12:04 qpid-proton-c-0.24.0-
> > 1.el7.x86_64.rpm
> >
> > controller-0:~$ ls -l /www/pages/feed/rel-18.08/Packages/ |grep -- --
> > -
> > -rw-------. 1 root root 258624 Sep 12 20:33 python2-qpid-proton-
> > 0.24.0-1.el7.x86_64.rpm
> > -rw-------. 1 root root 190128 Sep 12 20:33 qpid-proton-c-0.24.0-
> > 1.el7.x86_64.rpm
> >
> >
> > The lighttpd server runs as a non-root user, and cannot access these
> > files. Change the permissions on the files, and the installation of
> > controller-1 should be successful.
> >
> >
> > -----Original Message-----
> > From: Young, Ken [mailto:Ken.Young at windriver.com]
> > Sent: Wednesday, September 12, 2018 4:53 PM
> > To: Cordoba Malibran, Erich
> > Cc: starlingx-discuss at lists.starlingx.io
> > Subject: Re: [Starlingx-discuss] [ Test ] Sanity results - summary
> > posted
> >
> > Erich,
> >
> > One correction below: we are using a Wind River version of the
> > installer (I am confusing this thread with another thread). This is
> > the installer we shipped Titanium Cloud Release 5.
> >
> > This begs the question: what installer were you using before the
> > changes to the kernel?
> >
> > /KenY
> >
> > On 2018-09-12, 4:36 PM, "Young, Ken" <Ken.Young at windriver.com>
> > wrote:
> >
> > Erich,
> >
> > A couple of data points on our side. For our builds, we are using
> > stock CentOS installer for 7.4. These are unchanged and we have not
> > seen this issue on our side.
> >
> > Have you isolated when you made the changes to the installer? Is
> > it the same time you started seeing this error? Does it make sense to
> > revert to the stock installer while we get to the bottom of this issue
> > to continue sanity?
> >
> > Regards,
> > Ken Y
> >
> > On 2018-09-11, 8:30 PM, "Cordoba Malibran, Erich" <erich.cordoba.
> > malibran at intel.com> wrote:
> >
> > I have narrowed down a little bit the issue, these are my
> > findings.
> >
> > - I compared the bootimage_michel.iso with the stx-2018-09-
> > 11-
> > 111.iso, which is today's ISO.
> > - The are not significant package differences between them,
> > minor
> > versions has changed in some packages.
> > - The issue is that once the kernel has been loaded through
> > PXE,
> > anaconda is unable to find python2-qpid package. Not sure if
> > this is
> > the first package that is requested.
> > - From the anaconda shell I can see the http://pxecontrolle
> > r/ serving
> > the files and the repodata seems to be updated there. I
> > installed yum
> > manually and I was able to install python2-qpid package, (and
> > any other
> > available).
> > - The content of the pxe-network-installer package is not
> > the same,
> > WR squashfs.img file has the kernel 3.10.0-693.2.2 but ours
> > has 3.10.0-
> > 862.6.3. We regenerate those files when the kernel was
> > upgraded. (maybe
> > this matches with the date where the issue started to
> > arise..)
> >
> > So, my guess now is that the problem could be caused by the
> > pxe-
> > network-installer package. We create those files with the
> > update-
> > network-installer script. That's the same used in WR to
> > generate these
> > files?
> >
> > I'll keep diging in the generation of these files to see if
> > there is
> > any problem.
> >
> > -Erich
> >
> > On Tue, 2018-09-11 at 20:03 +0000, Lara, Cesar wrote:
> > > I just wanted to make sure that we are able to reproduce the
> > issue
> > > regardless of the build environment.
> > >
> > > AFAIK CentOS 7.5 upgrade is a different issue/thread, but at
> > the end
> > > we can fine tune a process in which we are in the same
> > conditions to
> > > be able to replicate errors in any given build environment.
> > >
> > > Regards
> > > Cesar Lara
> > >
> > >
> > > -----Original Message-----
> > > From: McKenna, Jason [mailto:Jason.McKenna at windriver.com]
> > > Sent: Tuesday, September 11, 2018 2:02 PM
> > > To: Lara, Cesar <cesar.lara at intel.com>; Alonso, Juan Carlos
> > <juan.car
> > > los.alonso at intel.com>; Khalil, Ghada <Ghada.Khalil at windrive
> > r.com>;
> > > Martinez Monroy, Elio <elio.martinez.monroy at intel.com>;
> > Jones, Bruce
> > > E <bruce.e.jones at intel.com>; Cabrales, Ada <ada.cabrales at in
> > tel.com>;
> > > starlingx-discuss at lists.starlingx.io; Thebeau, Michel
> > <Michel.Thebeau
> > > @windriver.com>; Young, Ken <Ken.Young at windriver.com>;
> > Little, Scott
> > > <Scott.Little at windriver.com>
> > > Subject: RE: [ Test ] Sanity results - summary posted
> > >
> > > Hi Cesar,
> > >
> > > Would you elaborate on what differences you'd expect to see
> > in an
> > > empty mirror populated by the download tool, compared to an
> > existing
> > > mirror which was updated by the download tool?
> > >
> > > The mirror tools were written specifically to contain
> > multiple
> > > versions of packages. The build environment population
> > tools take
> > > the .lst file you have checked out, and places that version
> > from the
> > > mirror into your local environment guaranteeing that your
> > build env
> > > matches the .lst file. If your mirror contains versions
> > 1.0.0,
> > > 1.2.3, and 2.0.0 of a given package, then your environment
> > will only
> > > use the version called out in the .lst file you have checked
> > out.
> > >
> > > This allows the same mirror to be used for multiple
> > branches, for
> > > multiple releases, for multiple development environments,
> > etc, and
> > > guarantees that a build can be reproduced down the road even
> > if the
> > > build inputs disappear from the Internet at some point in
> > the
> > > future. As long as all the artifacts named in the .lst file
> > have
> > > been downloaded at some point in the past, the build
> > environment
> > > produced from a fresh mirror vs a continually updated mirror
> > will be
> > > identical.
> > >
> > > The issue we are observing with the CentOS 7.5 update is
> > that the
> > > versions of packages called out in the .lst files for the
> > centos75
> > > branch of stx-tools do not match the versions of packages
> > called out
> > > in centos75 branch of the other repos, like stx-integ. The
> > other
> > > branches are attempting to patch a (new) version of the
> > source code,
> > > but the .lst file is only bringing in (or downloading) and
> > older
> > > version of the source.
> > >
> > > -Jason
> > >
> > > > -----Original Message-----
> > > > From: Lara, Cesar <cesar.lara at intel.com>
> > > > Sent: September 11, 2018 1:44 PM
> > > > To: Alonso, Juan Carlos <juan.carlos.alonso at intel.com>;
> > Khalil,
> > > > Ghada
> > > > <Ghada.Khalil at windriver.com>; Martinez Monroy, Elio
> > > > <elio.martinez.monroy at intel.com>; Jones, Bruce E
> > > > <bruce.e.jones at intel.com>; Cabrales, Ada <ada.cabrales at in
> > tel.com>;
> > > > starlingx-discuss at lists.starlingx.io; Thebeau, Michel
> > > > <Michel.Thebeau at windriver.com>
> > > > Subject: Re: [Starlingx-discuss] [ Test ] Sanity results
> > - summary
> > > > posted
> > > >
> > > > Hi all,
> > > >
> > > > Just wanted to point out that, to be able to reproduce the
> > issues
> > > > from
> > > > Intel's ISO, somebody will need to go and create a build
> > > > environment
> > > > from scratch, using external upstream repositories to
> > generate a
> > > > new
> > > > mirror, just the way we are doing it in our environment.
> > Failing to
> > > > do
> > > > this will result in a apples to oranges comparison.
> > > >
> > > > Regards
> > > > Cesar Lara
> > > >
> > > > -----Original Message-----
> > > > From: Alonso, Juan Carlos [mailto:juan.carlos.alonso at inte
> > l.com]
> > > > Sent: Tuesday, September 11, 2018 10:54 AM
> > > > To: Khalil, Ghada <Ghada.Khalil at windriver.com>; Martinez
> > Monroy,
> > > > Elio
> > > > <elio.martinez.monroy at intel.com>; Jones, Bruce E
> > > > <bruce.e.jones at intel.com>; Cabrales, Ada <ada.cabrales at in
> > tel.com>;
> > > > starlingx-discuss at lists.starlingx.io; Thebeau, Michel
> > > > <Michel.Thebeau at windriver.com>
> > > > Subject: Re: [Starlingx-discuss] [ Test ] Sanity results
> > - summary
> > > > posted
> > > >
> > > > Hi Ghada,
> > > >
> > > > Yes, before shared our ISOs I followed such steps and got
> > the same
> > > > issues.
> > > >
> > > > As I mention in the issue
> > > > (https://bugs.launchpad.net/starlingx/+bug/1790213), I
> > could get
> > > > the
> > > > ISO from Michel and testes in our infrastructure, issues
> > not found.
> > > > I
> > > > guess there are differences between the way an ISO is
> > built by
> > > > Intel
> > > > and Wind River since Intel got issues but Wind River does
> > not. I
> > > > think we should sync about this.
> > > >
> > > > Besides, our test infrastructure uses other commands to
> > generate
> > > > VMs,
> > > > add/install hosts (controllers and computes), such
> > commands was
> > > > gotten
> > > > from "tic_vb" repo, such repo was given by China team, if
> > I am not
> > > > wrong such repo was based on Wind River documentation; our
> > test
> > > > framework is based on "tic_vb". Anyway both ways got the
> > same
> > > > results.
> > > >
> > > > Regards.
> > > > Juan Carlos Alonso
> > > >
> > > > -----Original Message-----
> > > > From: Khalil, Ghada [mailto:Ghada.Khalil at windriver.com]
> > > > Sent: Tuesday, September 11, 2018 8:53 AM
> > > > To: Alonso, Juan Carlos <juan.carlos.alonso at intel.com>;
> > Martinez
> > > > Monroy, Elio <elio.martinez.monroy at intel.com>; Jones,
> > Bruce E
> > > > <bruce.e.jones at intel.com>; Cabrales, Ada <ada.cabrales at in
> > tel.com>;
> > > > starlingx-discuss at lists.starlingx.io; Thebeau, Michel
> > > > <Michel.Thebeau at windriver.com>
> > > > Subject: RE: [ Test ] Sanity results - summary posted
> > > >
> > > > Hello Juan,
> > > > I read the notes in the Launchpad.
> > > >
> > > > And, just to confirm, are you following the instructions
> > under:
> > > > https://wiki.openstack.org/wiki/StarlingX/Installation_Gu
> > ide_Virtua
> > > > l_E
> > > > nviro
> > > > nment/Controller_Storage
> > > > and still reproducing the two issues below?
> > > >
> > > > I was not able to definitively tell from the Launchpad
> > notes. It's
> > > > important that both our teams are comparing apples to
> > apples.
> > > >
> > > > Thanks,
> > > > Ghada
> > > >
> > > > -----Original Message-----
> > > > From: Alonso, Juan Carlos [mailto:juan.carlos.alonso at inte
> > l.com]
> > > > Sent: Monday, September 10, 2018 1:50 PM
> > > > To: Khalil, Ghada; Martinez Monroy, Elio; Jones, Bruce E;
> > > > Cabrales,
> > > > Ada; starlingx-discuss at lists.starlingx.io; Thebeau, Michel
> > > > Subject: RE: [ Test ] Sanity results - summary posted
> > > >
> > > > Hi,
> > > >
> > > > Such launchpads were updated.
> > > >
> > > > A VM was set to upload the ISO tx-2018-08-28-88.iso, such
> > ISO
> > > > present
> > > > two issues. Michael will upload his ISO, then we will re
> > test
> > > > witch
> > > > each iso in our environment.
> > > >
> > > > An script is being developed to get the differences
> > between ISOs.
> > > >
> > > > Regards.
> > > > Juan Carlos Alonso
> > > >
> > > > -----Original Message-----
> > > > From: Khalil, Ghada [mailto:Ghada.Khalil at windriver.com]
> > > > Sent: Friday, September 7, 2018 4:43 PM
> > > > To: Khalil, Ghada <Ghada.Khalil at windriver.com>; Martinez
> > Monroy,
> > > > Elio
> > > > <elio.martinez.monroy at intel.com>; Jones, Bruce E
> > > > <bruce.e.jones at intel.com>; Cabrales, Ada <ada.cabrales at in
> > tel.com>;
> > > > starlingx-discuss at lists.starlingx.io; Thebeau, Michel
> > > > <Michel.Thebeau at windriver.com>
> > > > Subject: Re: [Starlingx-discuss] [ Test ] Sanity results
> > - summary
> > > > posted
> > > >
> > > > Hi Elio/Ada,
> > > > Are there any updates from your side on these two
> > Launchpads?
> > > > https://bugs.launchpad.net/starlingx/+bug/1790213
> > > > https://bugs.launchpad.net/starlingx/+bug/1790716
> > > >
> > > > Michel Thebeau (Wind River) has been trying to reproduce
> > these two
> > > > issues (following the starlingx wikis) without success for
> > two
> > > > days
> > > > now. He has been adding his findings to the bugs and
> > communicating
> > > > with your team.
> > > > As a result of his investigation, he will be pushing some
> > small
> > > > fixes
> > > > to stx- tools shortly (some cleanup).
> > > >
> > > > Please let us know if you need any further help from us.
> > > >
> > > > Thanks,
> > > > Ghada
> > > >
> > > > -----Original Message-----
> > > > From: Khalil, Ghada [mailto:Ghada.Khalil at windriver.com]
> > > > Sent: Wednesday, September 05, 2018 5:39 PM
> > > > To: Martinez Monroy, Elio; Jones, Bruce E; Cabrales, Ada;
> > > > starlingx-
> > > > discuss at lists.starlingx.io
> > > > Subject: Re: [Starlingx-discuss] [ Test ] Sanity results
> > - summary
> > > > posted
> > > >
> > > > Please see inline. There are notes in all the launchpads
> > as well.
> > > >
> > > > I have tagged the last bug for the October release. The
> > rest need
> > > > more
> > > > information to determine whether they gate or not: two
> > maybe
> > > > procedural and one requires more information.
> > > >
> > > > -----Original Message-----
> > > > From: Martinez Monroy, Elio [mailto:elio.martinez.monroy@
> > intel.com]
> > > > Sent: Wednesday, September 05, 2018 2:23 PM
> > > > To: Jones, Bruce E; Cabrales, Ada;
> > > > starlingx-discuss at lists.starlingx.io
> > > > Subject: Re: [Starlingx-discuss] [ Test ] Sanity results
> > - summary
> > > > posted
> > > >
> > > > My opinion below
> > > >
> > > > -----Original Message-----
> > > > From: Jones, Bruce E [mailto:bruce.e.jones at intel.com]
> > > > Sent: Wednesday, September 5, 2018 12:56 PM
> > > > To: Cabrales, Ada <ada.cabrales at intel.com>; starlingx-
> > > > discuss at lists.starlingx.io
> > > > Subject: Re: [Starlingx-discuss] [ Test ] Sanity results
> > - summary
> > > > posted
> > > >
> > > > Should the bugs coming out of this be tagged/fixed for the
> > October
> > > > release?
> > > >
> > > > Related bugs:
> > > > https://bugs.launchpad.net/starlingx/+bug/1790213 < ---
> > Must, this
> > > > bug
> > > > is related with 2nd controller provisioning [[GK]] Using
> > the
> > > > instructions published on the wiki, our designer was not
> > able to
> > > > reproduce this issue.
> > > > What instructions are the test team using?
> > > >
> > > > https://bugs.launchpad.net/starlingx/+bug/1790514 < ---
> > Can wait
> > > > until
> > > > other issues are solved, will fail some test cases related
> > to SSH
> > > > [[GK]] Requested more information. This cannot be debugged
> > further
> > > > without this information.
> > > >
> > > > https://bugs.launchpad.net/starlingx/+bug/1790716 < ---
> > Must, all
> > > > instances need at least one compute for multimode [[GK]]
> > Using the
> > > > instructions published on the wiki, our designer was not
> > able to
> > > > reproduce this issue. The test team points to internal
> > > > instructions
> > > > which we have no access to. The supported install recipes
> > need to
> > > > be
> > > > on the wiki. I think it is an invalid bug
> > > >
> > > > https://bugs.launchpad.net/starlingx/+bug/1790718 < ---
> > Can wait
> > > > since
> > > > is related to FM , test cases will fail [[GK]] As Brent
> > pointed
> > > > out,
> > > > this is dependent on the integration of stx-gui which is
> > in
> > > > progress.
> > > >
> > > > -----Original Message-----
> > > > From: Cabrales, Ada [mailto:ada.cabrales at intel.com]
> > > > Sent: Wednesday, September 5, 2018 10:50 AM
> > > > To: starlingx-discuss at lists.starlingx.io
> > > > Subject: [Starlingx-discuss] [ Test ] Sanity results -
> > summary
> > > > posted
> > > >
> > > > Hello all,
> > > >
> > > > As mentioned on the project meeting today, we're
> > posting the
> > > > summary of the sanity test runs on the Test wiki [0]. The
> > sanity
> > > > is
> > > > triggered each time an ISO is generated (a daily task).
> > > >
> > > > Right now we are posting the results manually, and
> > working on a
> > > > way
> > > > for automating it. Logs of the run are not included, if
> > you want
> > > > to
> > > > have them, you can contact Elio to get them sent to you.
> > > >
> > > >
> > > > [0]
> > > > https://wiki.openstack.org/wiki/StarlingX/Test#Latest_ISO
> > _image_San
> > > > ity
> > > > _S
> > > > ummary
> > > >
> > > > Regards
> > > > Ada
> > > >
> > > > _______________________________________________
> > > > Starlingx-discuss mailing list
> > > > Starlingx-discuss at lists.starlingx.io
> > > > http://lists.starlingx.io/cgi-bin/mailman/listinfo/starli
> > ngx-discus
> > > > s
> > > > _______________________________________________
> > > > Starlingx-discuss mailing list
> > > > Starlingx-discuss at lists.starlingx.io
> > > > http://lists.starlingx.io/cgi-bin/mailman/listinfo/starli
> > ngx-discus
> > > > s
> > > > _______________________________________________
> > > > Starlingx-discuss mailing list
> > > > Starlingx-discuss at lists.starlingx.io
> > > > http://lists.starlingx.io/cgi-bin/mailman/listinfo/starli
> > ngx-discus
> > > > s
> > > > _______________________________________________
> > > > Starlingx-discuss mailing list
> > > > Starlingx-discuss at lists.starlingx.io
> > > > http://lists.starlingx.io/cgi-bin/mailman/listinfo/starli
> > ngx-discus
> > > > s
> > > > _______________________________________________
> > > > Starlingx-discuss mailing list
> > > > Starlingx-discuss at lists.starlingx.io
> > > > http://lists.starlingx.io/cgi-bin/mailman/listinfo/starli
> > ngx-discus
> > > > s
> > > > _______________________________________________
> > > > Starlingx-discuss mailing list
> > > > Starlingx-discuss at lists.starlingx.io
> > > > http://lists.starlingx.io/cgi-bin/mailman/listinfo/starli
> > ngx-discus
> > > > s
> > > > _______________________________________________
> > > > Starlingx-discuss mailing list
> > > > Starlingx-discuss at lists.starlingx.io
> > > > http://lists.starlingx.io/cgi-bin/mailman/listinfo/starli
> > ngx-discus
> > > > s
> > >
> > > _______________________________________________
> > > Starlingx-discuss mailing list
> > > Starlingx-discuss at lists.starlingx.io
> > > http://lists.starlingx.io/cgi-bin/mailman/listinfo/starling
> > x-discuss
> >
> > _______________________________________________
> > Starlingx-discuss mailing list
> > Starlingx-discuss at lists.starlingx.io
> > http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-disc
> > uss
> >
> >
> > _______________________________________________
> > Starlingx-discuss mailing list
> > Starlingx-discuss at lists.starlingx.io
> > http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
> _______________________________________________
> Starlingx-discuss mailing list
> Starlingx-discuss at lists.starlingx.io
> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
More information about the Starlingx-discuss
mailing list