[Starlingx-discuss] [ Test ] Sanity results - summary posted

Cordoba Malibran, Erich erich.cordoba.malibran at intel.com
Thu Sep 13 18:46:25 UTC 2018


Indeed, those two files had the 600 permissions in our mirror and the
permissions were transfered into the ISO and then to the /ww/pages/feed
folder. Changing the permissions now let the compute to be installed.

I have two questions regarding this: 

  - Which component is responsible to populate the /www/pages/feed
folder with the rpms. I'm wondering if some permission checking/set
needs to be done here.
  - lighttpd runs as non-root user, but the content on /www/pages/feed
is owned by root. Should this folder be owned by the same user as the
one that runs lighttpd?

Thanks

-Erich

On Wed, 2018-09-12 at 21:37 +0000, Penney, Don wrote:
> The problem is one of permissions. In the /www/var/log/lighttpd-
> access.log file, you can see the http access request is being
> rejected with a 403 status code, indicating access is forbidden:
> 
> 192.168.204.85 pxecontroller - [12/Sep/2018:09:30:52 +0000] "GET
> /feed/rel-18.08/Packages/python2-qpid-proton-0.24.0-1.el7.x86_64.rpm
> HTTP/1.1" 403 345 "-" "urlgrabber/3.10 yum/3.4.3"
> 
> You can see this also with a wget command:
> localhost:/home/wrsroot# wget http://localhost/feed/rel-18.08/Package
> s/python2-qpid-proton-0.24.0-1.el7.x86_64.rpm
> --2018-09-12 20:54:19--  http://localhost/feed/rel-18.08/Packages/pyt
> hon2-qpid-proton-0.24.0-1.el7.x86_64.rpm
> Resolving localhost (localhost)... ::1, 127.0.0.1
> Connecting to localhost (localhost)|::1|:80... connected.
> HTTP request sent, awaiting response... 403 Forbidden
> 2018-09-12 20:54:19 ERROR 403: Forbidden.
> 
> 
> The problem is that two of the RPMs on the ISO have 600 permissions,
> where all other RPMs are 644. This is carried over when the files are
> copied to the controller during the post-install phase:
> 
> controller-0:~$ ls -l /mnt/iso/Packages/ | grep -- ---
> -rw------- 1 1001 users    258624 Aug 28 12:04 python2-qpid-proton-
> 0.24.0-1.el7.x86_64.rpm
> -rw------- 1 1001 users    190128 Aug 28 12:04 qpid-proton-c-0.24.0-
> 1.el7.x86_64.rpm
> 
> controller-0:~$ ls -l /www/pages/feed/rel-18.08/Packages/ |grep -- --
> -
> -rw-------. 1 root root    258624 Sep 12 20:33 python2-qpid-proton-
> 0.24.0-1.el7.x86_64.rpm
> -rw-------. 1 root root    190128 Sep 12 20:33 qpid-proton-c-0.24.0-
> 1.el7.x86_64.rpm
> 
> 
> The lighttpd server runs as a non-root user, and cannot access these
> files. Change the permissions on the files, and the installation of
> controller-1 should be successful.
> 
> 
> -----Original Message-----
> From: Young, Ken [mailto:Ken.Young at windriver.com] 
> Sent: Wednesday, September 12, 2018 4:53 PM
> To: Cordoba Malibran, Erich
> Cc: starlingx-discuss at lists.starlingx.io
> Subject: Re: [Starlingx-discuss] [ Test ] Sanity results - summary
> posted
> 
> Erich,
> 
> One correction below: we are using a Wind River version of the
> installer (I am confusing this thread with another thread).  This is
> the installer we shipped Titanium Cloud Release 5.
> 
> This begs the question: what installer were you using before the
> changes to the kernel?
> 
> /KenY
> 
> On 2018-09-12, 4:36 PM, "Young, Ken" <Ken.Young at windriver.com>
> wrote:
> 
>     Erich,
>     
>     A couple of data points on our side.  For our builds, we are
> using stock CentOS installer for 7.4.  These are unchanged and we
> have not seen this issue on our side.
>     
>     Have you isolated when you made the changes to the installer?  Is
> it the same time you started seeing this error?  Does it make sense
> to revert to the stock installer while we get to the bottom of this
> issue to continue sanity?
>     
>     Regards,
>     Ken Y
>     
>     On 2018-09-11, 8:30 PM, "Cordoba Malibran, Erich" <erich.cordoba.
> malibran at intel.com> wrote:
>     
>         I have narrowed down a little bit the issue, these are my
> findings.
>         
>           - I compared the bootimage_michel.iso with the stx-2018-09-
> 11-
>         111.iso, which is today's ISO.
>           - The are not significant package differences between them,
> minor
>         versions has changed in some packages.
>           - The issue is that once the kernel has been loaded through
> PXE,
>         anaconda is unable to find python2-qpid package. Not sure if
> this is
>         the first package that is requested.
>           - From the anaconda shell I can see the http://pxecontrolle
> r/ serving
>         the files and the repodata seems to be updated there. I
> installed yum
>         manually and I was able to install python2-qpid package, (and
> any other
>         available).
>           - The content of the pxe-network-installer package is not
> the same,
>         WR squashfs.img file has the kernel 3.10.0-693.2.2 but ours
> has 3.10.0-
>         862.6.3. We regenerate those files when the kernel was
> upgraded. (maybe
>         this matches with the date where the issue started to
> arise..)
>         
>         So, my guess now is that the problem could be caused by the
> pxe-
>         network-installer package. We create those files with the
> update-
>         network-installer script. That's the same used in WR to
> generate these
>         files?
>         
>         I'll keep diging in the generation of these files to see if
> there is
>         any problem.
>         
>         -Erich
>         
>         On Tue, 2018-09-11 at 20:03 +0000, Lara, Cesar wrote:
>         > I just wanted to make sure that we are able to reproduce
> the issue
>         > regardless of the build environment.
>         > 
>         > AFAIK CentOS 7.5 upgrade is a different issue/thread, but
> at the end
>         > we can fine tune a process in which we are in the same
> conditions to
>         > be able to replicate errors in any given build
> environment. 
>         > 
>         > Regards 
>         > Cesar Lara
>         > 
>         > 
>         > -----Original Message-----
>         > From: McKenna, Jason [mailto:Jason.McKenna at windriver.com] 
>         > Sent: Tuesday, September 11, 2018 2:02 PM
>         > To: Lara, Cesar <cesar.lara at intel.com>; Alonso, Juan Carlos
> <juan.car
>         > los.alonso at intel.com>; Khalil, Ghada <Ghada.Khalil at windrive
> r.com>;
>         > Martinez Monroy, Elio <elio.martinez.monroy at intel.com>;
> Jones, Bruce
>         > E <bruce.e.jones at intel.com>; Cabrales, Ada <ada.cabrales at in
> tel.com>; 
>         > starlingx-discuss at lists.starlingx.io; Thebeau, Michel
> <Michel.Thebeau
>         > @windriver.com>; Young, Ken <Ken.Young at windriver.com>;
> Little, Scott
>         > <Scott.Little at windriver.com>
>         > Subject: RE: [ Test ] Sanity results - summary posted
>         > 
>         > Hi Cesar,
>         > 
>         > Would you elaborate on what differences you'd expect to see
> in an
>         > empty mirror populated by the download tool, compared to an
> existing
>         > mirror which was updated by the download tool?
>         > 
>         > The mirror tools were written specifically to contain
> multiple
>         > versions of packages.  The build environment population
> tools take
>         > the .lst file you have checked out, and places that version
> from the
>         > mirror into your local environment guaranteeing that your
> build env
>         > matches the .lst file.  If your mirror contains versions
> 1.0.0,
>         > 1.2.3, and 2.0.0 of a given package, then your environment
> will only
>         > use the version called out in the .lst file you have
> checked out.
>         > 
>         > This allows the same mirror to be used for multiple
> branches, for
>         > multiple releases, for multiple development environments,
> etc, and
>         > guarantees that a build can be reproduced down the road
> even if the
>         > build inputs disappear from the Internet at some point in
> the
>         > future.  As long as all the artifacts named in the .lst
> file have
>         > been downloaded at some point in the past, the build
> environment
>         > produced from a fresh mirror vs a continually updated
> mirror will be
>         > identical.
>         > 
>         > The issue we are observing with the CentOS 7.5 update is
> that the
>         > versions of packages called out in the .lst files for the
> centos75
>         > branch of stx-tools do not match the versions of packages
> called out
>         > in centos75 branch of the other repos, like stx-integ.  The 
> other
>         > branches are attempting to patch a (new) version of the
> source code,
>         > but the .lst file is only bringing in (or downloading) and
> older
>         > version of the source.
>         > 
>         > -Jason
>         > 
>         > > -----Original Message-----
>         > > From: Lara, Cesar <cesar.lara at intel.com>
>         > > Sent: September 11, 2018 1:44 PM
>         > > To: Alonso, Juan Carlos <juan.carlos.alonso at intel.com>;
> Khalil,
>         > > Ghada 
>         > > <Ghada.Khalil at windriver.com>; Martinez Monroy, Elio 
>         > > <elio.martinez.monroy at intel.com>; Jones, Bruce E 
>         > > <bruce.e.jones at intel.com>; Cabrales, Ada <ada.cabrales at in
> tel.com>; 
>         > > starlingx-discuss at lists.starlingx.io; Thebeau, Michel 
>         > > <Michel.Thebeau at windriver.com>
>         > > Subject: Re: [Starlingx-discuss] [ Test ] Sanity results
> - summary 
>         > > posted
>         > > 
>         > > Hi all,
>         > > 
>         > > Just wanted to point out that, to be able to reproduce
> the issues
>         > > from 
>         > > Intel's ISO, somebody will need to go and create a build
>         > > environment 
>         > > from scratch, using external upstream repositories to
> generate a
>         > > new 
>         > > mirror, just the way we are doing it in our environment.
> Failing to
>         > > do 
>         > > this will result in a apples to oranges comparison.
>         > > 
>         > > Regards
>         > > Cesar Lara
>         > > 
>         > > -----Original Message-----
>         > > From: Alonso, Juan Carlos [mailto:juan.carlos.alonso at inte
> l.com]
>         > > Sent: Tuesday, September 11, 2018 10:54 AM
>         > > To: Khalil, Ghada <Ghada.Khalil at windriver.com>; Martinez
> Monroy,
>         > > Elio 
>         > > <elio.martinez.monroy at intel.com>; Jones, Bruce E 
>         > > <bruce.e.jones at intel.com>; Cabrales, Ada <ada.cabrales at in
> tel.com>; 
>         > > starlingx-discuss at lists.starlingx.io; Thebeau, Michel 
>         > > <Michel.Thebeau at windriver.com>
>         > > Subject: Re: [Starlingx-discuss] [ Test ] Sanity results
> - summary 
>         > > posted
>         > > 
>         > > Hi Ghada,
>         > > 
>         > > Yes, before shared our ISOs I followed such steps and got
> the same
>         > > issues.
>         > > 
>         > > As I mention in the issue
>         > > (https://bugs.launchpad.net/starlingx/+bug/1790213), I
> could get
>         > > the 
>         > > ISO from Michel and testes in our infrastructure, issues
> not found.
>         > > I 
>         > > guess there are differences between the way an ISO is
> built by
>         > > Intel 
>         > > and Wind River since Intel got issues but Wind River does
> not. I
>         > > think we should sync about this.
>         > > 
>         > > Besides, our test infrastructure uses other commands to
> generate
>         > > VMs, 
>         > > add/install hosts (controllers and computes), such
> commands was
>         > > gotten 
>         > > from "tic_vb" repo, such repo was given by China team, if
> I am not 
>         > > wrong such repo was based on Wind River documentation;
> our test 
>         > > framework is based on "tic_vb". Anyway both ways got the
> same
>         > > results.
>         > > 
>         > > Regards.
>         > > Juan Carlos Alonso
>         > > 
>         > > -----Original Message-----
>         > > From: Khalil, Ghada [mailto:Ghada.Khalil at windriver.com]
>         > > Sent: Tuesday, September 11, 2018 8:53 AM
>         > > To: Alonso, Juan Carlos <juan.carlos.alonso at intel.com>;
> Martinez 
>         > > Monroy, Elio <elio.martinez.monroy at intel.com>; Jones,
> Bruce E 
>         > > <bruce.e.jones at intel.com>; Cabrales, Ada <ada.cabrales at in
> tel.com>; 
>         > > starlingx-discuss at lists.starlingx.io; Thebeau, Michel 
>         > > <Michel.Thebeau at windriver.com>
>         > > Subject: RE: [ Test ] Sanity results - summary posted
>         > > 
>         > > Hello Juan,
>         > > I read the notes in the Launchpad.
>         > > 
>         > > And, just to confirm, are you following the instructions
> under:
>         > > https://wiki.openstack.org/wiki/StarlingX/Installation_Gu
> ide_Virtua
>         > > l_E
>         > > nviro
>         > > nment/Controller_Storage
>         > > and still reproducing the two issues below?
>         > > 
>         > > I was not able to definitively tell from the Launchpad
> notes. It's 
>         > > important that both our teams are comparing apples to
> apples.
>         > > 
>         > > Thanks,
>         > > Ghada
>         > > 
>         > > -----Original Message-----
>         > > From: Alonso, Juan Carlos [mailto:juan.carlos.alonso at inte
> l.com]
>         > > Sent: Monday, September 10, 2018 1:50 PM
>         > > To: Khalil, Ghada; Martinez Monroy, Elio; Jones, Bruce E;
>         > > Cabrales, 
>         > > Ada; starlingx-discuss at lists.starlingx.io; Thebeau,
> Michel
>         > > Subject: RE: [ Test ] Sanity results - summary posted
>         > > 
>         > > Hi,
>         > > 
>         > > Such launchpads were updated.
>         > > 
>         > > A VM was set to upload the ISO tx-2018-08-28-88.iso, such
> ISO
>         > > present 
>         > > two issues. Michael will upload his ISO, then we will re
> test
>         > > witch 
>         > > each iso in our environment.
>         > > 
>         > > An script is being developed to get the differences
> between ISOs.
>         > > 
>         > > Regards.
>         > > Juan Carlos Alonso
>         > > 
>         > > -----Original Message-----
>         > > From: Khalil, Ghada [mailto:Ghada.Khalil at windriver.com]
>         > > Sent: Friday, September 7, 2018 4:43 PM
>         > > To: Khalil, Ghada <Ghada.Khalil at windriver.com>; Martinez
> Monroy,
>         > > Elio 
>         > > <elio.martinez.monroy at intel.com>; Jones, Bruce E 
>         > > <bruce.e.jones at intel.com>; Cabrales, Ada <ada.cabrales at in
> tel.com>; 
>         > > starlingx-discuss at lists.starlingx.io; Thebeau, Michel 
>         > > <Michel.Thebeau at windriver.com>
>         > > Subject: Re: [Starlingx-discuss] [ Test ] Sanity results
> - summary 
>         > > posted
>         > > 
>         > > Hi Elio/Ada,
>         > > Are there any updates from your side on these two
> Launchpads?
>         > > https://bugs.launchpad.net/starlingx/+bug/1790213
>         > > https://bugs.launchpad.net/starlingx/+bug/1790716
>         > > 
>         > > Michel Thebeau (Wind River) has been trying to reproduce
> these two 
>         > > issues (following the starlingx wikis) without success
> for two
>         > > days 
>         > > now. He has been adding his findings to the bugs and
> communicating
>         > > with your team.
>         > > As a result of his investigation, he will be pushing some
> small
>         > > fixes 
>         > > to stx- tools shortly (some cleanup).
>         > > 
>         > > Please let us know if you need any further help from us.
>         > > 
>         > > Thanks,
>         > > Ghada
>         > > 
>         > > -----Original Message-----
>         > > From: Khalil, Ghada [mailto:Ghada.Khalil at windriver.com]
>         > > Sent: Wednesday, September 05, 2018 5:39 PM
>         > > To: Martinez Monroy, Elio; Jones, Bruce E; Cabrales, Ada;
>         > > starlingx- 
>         > > discuss at lists.starlingx.io
>         > > Subject: Re: [Starlingx-discuss] [ Test ] Sanity results
> - summary 
>         > > posted
>         > > 
>         > > Please see inline. There are notes in all the launchpads
> as well.
>         > > 
>         > > I have tagged the last bug for the October release. The
> rest need
>         > > more 
>         > > information to determine whether they gate or not: two
> maybe 
>         > > procedural and one requires more information.
>         > > 
>         > > -----Original Message-----
>         > > From: Martinez Monroy, Elio [mailto:elio.martinez.monroy@
> intel.com]
>         > > Sent: Wednesday, September 05, 2018 2:23 PM
>         > > To: Jones, Bruce E; Cabrales, Ada; 
>         > > starlingx-discuss at lists.starlingx.io
>         > > Subject: Re: [Starlingx-discuss] [ Test ] Sanity results
> - summary 
>         > > posted
>         > > 
>         > > My opinion below
>         > > 
>         > > -----Original Message-----
>         > > From: Jones, Bruce E [mailto:bruce.e.jones at intel.com]
>         > > Sent: Wednesday, September 5, 2018 12:56 PM
>         > > To: Cabrales, Ada <ada.cabrales at intel.com>; starlingx- 
>         > > discuss at lists.starlingx.io
>         > > Subject: Re: [Starlingx-discuss] [ Test ] Sanity results
> - summary 
>         > > posted
>         > > 
>         > > Should the bugs coming out of this be tagged/fixed for
> the October
>         > > release?
>         > > 
>         > > Related bugs:
>         > > https://bugs.launchpad.net/starlingx/+bug/1790213 < ---
> Must, this
>         > > bug 
>         > > is related with 2nd controller provisioning [[GK]] Using
> the 
>         > > instructions published on the wiki, our designer was not
> able to
>         > > reproduce this issue.
>         > > What instructions are the test team using?
>         > > 
>         > > https://bugs.launchpad.net/starlingx/+bug/1790514 < ---
> Can wait
>         > > until 
>         > > other issues are solved, will fail some test cases
> related to SSH 
>         > > [[GK]] Requested more information. This cannot be
> debugged further
>         > > without this information.
>         > > 
>         > > https://bugs.launchpad.net/starlingx/+bug/1790716 < ---
> Must, all 
>         > > instances need at least one compute for multimode [[GK]]
> Using the 
>         > > instructions published on the wiki, our designer was not
> able to 
>         > > reproduce this issue. The test team points to internal
>         > > instructions 
>         > > which we have no access to. The supported install recipes
> need to
>         > > be 
>         > > on the wiki. I think it is an invalid bug
>         > > 
>         > > https://bugs.launchpad.net/starlingx/+bug/1790718 < ---
> Can wait
>         > > since 
>         > > is related to FM , test cases will fail [[GK]] As Brent
> pointed
>         > > out, 
>         > > this is dependent on the integration of stx-gui which is
> in
>         > > progress.
>         > > 
>         > > -----Original Message-----
>         > > From: Cabrales, Ada [mailto:ada.cabrales at intel.com]
>         > > Sent: Wednesday, September 5, 2018 10:50 AM
>         > > To: starlingx-discuss at lists.starlingx.io
>         > > Subject: [Starlingx-discuss] [ Test ] Sanity results -
> summary
>         > > posted
>         > > 
>         > > Hello all,
>         > > 
>         > >     As mentioned on the project meeting today, we're
> posting the 
>         > > summary of the sanity test runs on the Test wiki [0]. The
> sanity
>         > > is 
>         > > triggered each time an ISO is generated (a daily task).
>         > > 
>         > >    Right now we are posting the results manually, and
> working on a
>         > > way 
>         > > for automating it. Logs of the run are not included, if
> you want
>         > > to 
>         > > have them, you can contact Elio to get them sent to you.
>         > > 
>         > > 
>         > > [0]
>         > > https://wiki.openstack.org/wiki/StarlingX/Test#Latest_ISO
> _image_San
>         > > ity
>         > > _S
>         > > ummary
>         > > 
>         > > Regards
>         > > Ada
>         > > 
>         > > _______________________________________________
>         > > Starlingx-discuss mailing list
>         > > Starlingx-discuss at lists.starlingx.io
>         > > http://lists.starlingx.io/cgi-bin/mailman/listinfo/starli
> ngx-discus
>         > > s
>         > > _______________________________________________
>         > > Starlingx-discuss mailing list
>         > > Starlingx-discuss at lists.starlingx.io
>         > > http://lists.starlingx.io/cgi-bin/mailman/listinfo/starli
> ngx-discus
>         > > s
>         > > _______________________________________________
>         > > Starlingx-discuss mailing list
>         > > Starlingx-discuss at lists.starlingx.io
>         > > http://lists.starlingx.io/cgi-bin/mailman/listinfo/starli
> ngx-discus
>         > > s
>         > > _______________________________________________
>         > > Starlingx-discuss mailing list
>         > > Starlingx-discuss at lists.starlingx.io
>         > > http://lists.starlingx.io/cgi-bin/mailman/listinfo/starli
> ngx-discus
>         > > s
>         > > _______________________________________________
>         > > Starlingx-discuss mailing list
>         > > Starlingx-discuss at lists.starlingx.io
>         > > http://lists.starlingx.io/cgi-bin/mailman/listinfo/starli
> ngx-discus
>         > > s
>         > > _______________________________________________
>         > > Starlingx-discuss mailing list
>         > > Starlingx-discuss at lists.starlingx.io
>         > > http://lists.starlingx.io/cgi-bin/mailman/listinfo/starli
> ngx-discus
>         > > s
>         > > _______________________________________________
>         > > Starlingx-discuss mailing list
>         > > Starlingx-discuss at lists.starlingx.io
>         > > http://lists.starlingx.io/cgi-bin/mailman/listinfo/starli
> ngx-discus
>         > > s
>         > 
>         > _______________________________________________
>         > Starlingx-discuss mailing list
>         > Starlingx-discuss at lists.starlingx.io
>         > http://lists.starlingx.io/cgi-bin/mailman/listinfo/starling
> x-discuss
>     
>     _______________________________________________
>     Starlingx-discuss mailing list
>     Starlingx-discuss at lists.starlingx.io
>     http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-disc
> uss
>     
> 
> _______________________________________________
> Starlingx-discuss mailing list
> Starlingx-discuss at lists.starlingx.io
> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss


More information about the Starlingx-discuss mailing list