[Starlingx-discuss] Request freeze on commits

Tue May 12 16:36:45 UTC 2020

On 5/12/20 9:35 AM, Jascanu, Nicolae wrote:
> The LAYERED build 20200511T230211Z  used the helm chart: http://mirror.starlingx.cengn.ca/mirror/starlingx/master/centos/containers/20200505T204039Z/outputs/helm-charts/helm-charts-stx-openstack-centos-stable-versioned.tgz
> 
Please reconfirm the link!  It shows 20200505T204039Z, with is from May 
5th, not the May 11th build!

Sau!

> 
> 
> Regards,
> Nicolae Jascanu, Ph.D.
> TSD Engineering Manager
> 
> 
> 
> Internet Of Things Group
> Galati, Romania
> 
> -----Original Message-----
> From: Miller, Frank <Frank.Miller at windriver.com>
> Sent: Tuesday, May 12, 2020 19:27
> To: Jascanu, Nicolae <nicolae.jascanu at intel.com>; Saul Wold <sgw at linux.intel.com>; starlingx-discuss <starlingx-discuss at lists.starlingx.io>
> Subject: RE: [Starlingx-discuss] Request freeze on commits
> 
> Nicolae:
> 
> Where are the helm-charts for sanity coming from? Can you provide the link?
> 
> When one of our developers did their own flock layer build last night and installed their lab they were able to provision stx-openstack.  So the helm-chart provided by a developer build is correctly picking up Sabeel's commit.
> 
> Frank
> 
> 
> -----Original Message-----
> From: Jascanu, Nicolae <nicolae.jascanu at intel.com>
> Sent: Tuesday, May 12, 2020 12:20 PM
> To: Miller, Frank <Frank.Miller at windriver.com>; Saul Wold <sgw at linux.intel.com>; starlingx-discuss <starlingx-discuss at lists.starlingx.io>
> Subject: RE: [Starlingx-discuss] Request freeze on commits
> 
> I will start the build when will become available.
> The layered build 20200511T230211Z failed with the same error. I will send the report in a few minutes
> 
> 
> 2020-05-12 05:55:12.400 36214 ERROR armada.handlers.armada [-] Chart deploy [kube-system-ingress] failed: armada.exceptions.tiller_exceptions.ReleaseException: Failed to Install release: osh-kube-system-ingress - Tiller Message: b'Release "osh-kube-system-ingress" failed: timed out waiting for the condition'
> 2020-05-12 05:55:12.400 36214 ERROR armada.handlers.armada Traceback (most recent call last):
> 2020-05-12 05:55:12.400 36214 ERROR armada.handlers.armada   File "/usr/local/lib/python3.6/dist-packages/armada/handlers/tiller.py", line 473, in install_release
> 2020-05-12 05:55:12.400 36214 ERROR armada.handlers.armada     metadata=self.metadata)
> 2020-05-12 05:55:12.400 36214 ERROR armada.handlers.armada   File "/usr/local/lib/python3.6/dist-packages/grpc/_channel.py", line 533, in __call__
> 2020-05-12 05:55:12.400 36214 ERROR armada.handlers.armada     return _end_unary_response_blocking(state, call, False, None)
> 2020-05-12 05:55:12.400 36214 ERROR armada.handlers.armada   File "/usr/local/lib/python3.6/dist-packages/grpc/_channel.py", line 467, in _end_unary_response_blocking
> 2020-05-12 05:55:12.400 36214 ERROR armada.handlers.armada     raise _Rendezvous(state, None, None, deadline)
> 2020-05-12 05:55:12.400 36214 ERROR armada.handlers.armada grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with:
> 2020-05-12 05:55:12.400 36214 ERROR armada.handlers.armada      status = StatusCode.UNKNOWN
> 2020-05-12 05:55:12.400 36214 ERROR armada.handlers.armada      details = "release osh-kube-system-ingress failed: timed out waiting for the condition"
> 2020-05-12 05:55:12.400 36214 ERROR armada.handlers.armada      debug_error_string = "{"created":"@1589262912.237563359","description":"Error received from peer","file":"src/core/lib/surface/call.cc","file_line":1017,"grpc_message":"release osh-kube-system-ingress failed: timed out waiting for the condition","grpc_status":2}"
> 2020-05-12 05:55:12.400 36214 ERROR armada.handlers.armada >
> 
> Regards,
> Nicolae Jascanu, Ph.D.
> TSD Engineering Manager
> 
> 
> 
> Internet Of Things Group
> Galati, Romania
> 
> -----Original Message-----
> From: Miller, Frank <Frank.Miller at windriver.com>
> Sent: Tuesday, May 12, 2020 18:55
> To: Saul Wold <sgw at linux.intel.com>; starlingx-discuss <starlingx-discuss at lists.starlingx.io>
> Subject: Re: [Starlingx-discuss] Request freeze on commits
> 
> A new build was started 30 minutes ago by Davlet to pick up a fix for the docker image build [1].  Nicolae would it be possible to schedule another sanity today when this build is available?
> 
> Once the sanity results from earlier today are available, we need to determine if the root cause is the same root cause as before [2].  If not we will need additional fixes/plan to address today's sanity error.
> 
> In parallel, Davlet is working on getting the monolithic build to generate helm charts and docker images so we have a backup plan if issues continue to be seen that can be mapped back to build layering.
> 
> Frank
> [1] https://review.opendev.org/#/c/727218/
> [2] https://bugs.launchpad.net/starlingx/+bug/1877548
> 
> 
> -----Original Message-----
> From: Saul Wold <sgw at linux.intel.com>
> Sent: Tuesday, May 12, 2020 11:21 AM
> To: Miller, Frank <Frank.Miller at windriver.com>; starlingx-discuss <starlingx-discuss at lists.starlingx.io>
> Subject: Re: [Starlingx-discuss] Request freeze on commits
> 
> +100!!!!
> 
> I was about to write this same email, Sanity testing is still failing from the last night's load with the Layered build (email will come after Test meeting).
> 
> So the fix that was applied yesterday for the helm chart broke the build?  Or was there some other commit?  I thought we had already declared a full stop last week except for required fixes to address the Sanity failures only.
> 
> Are you seeing the same failure with the helm charts as the Intel test team?
> 
> Is there a way to get the Monolithic helm charts build independent of the ISO? I think this is what Austin was getting at.
> 
> Sau!
> 
> 
> On 5/12/20 8:06 AM, Miller, Frank wrote:
>> StarlingX Community:
>>
>> We have struggled for the past 2+ weeks to get a sane load.  At this
>> point I have no choice but to ask all Cores to freeze commits and not
>> allow any commits to merge unless they address a build or sanity issue.
>>
>> Last night's docker image build failed due to a recent commit and the
>> developer is providing a fix.  But we also have issues prior to this
>> in sanity due to the build layering method pulling in the wrong
>> version of a package.  If the current fixes do not result in sanity
>> passing then I'll ask the build team to revert to the monolithic build
>> until a better solution can be found.
>>
>> **
>>
>> Frank
>>
>> Project Lead for Build
>>
>>
>> _______________________________________________
>> Starlingx-discuss mailing list
>> Starlingx-discuss at lists.starlingx.io
>> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
>>
> 
> _______________________________________________
> Starlingx-discuss mailing list
> Starlingx-discuss at lists.starlingx.io
> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
> _______________________________________________
> Starlingx-discuss mailing list
> Starlingx-discuss at lists.starlingx.io
> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
> 