Adding this thread to public mailing list.
From: Alonso, Juan Carlos
Sent: Tuesday, May 28, 2019 7:08 PM
To: 'Ngo, Tee' <Tee.Ngo@windriver.com>; Perez Carranza, Jose <jose.perez.carranza@intel.com>
Cc: Lemus Contreras, Cristopher J <cristopher.j.lemus.contreras@intel.com>; Church, Robert <Robert.Church@windriver.com>; Wensley, Barton <Barton.Wensley@windriver.com>
Subject: RE: Live debug session
Hi Tee,
After make the workarounds you told, platform-integ-apps is still uploading and applying automatically, applying still failing.
I did not put docker_no_proxy list on localhost.yml since I saw last email after finish test.
As Barton, I got apply-failed, but app uploaded and applied automatically.
manager.py:
After Ansible:
[Service]
Environment="HTTPS_PROXY=http://proxy-chain.intel.com:912"
Environment="NO_PROXY=localhost,127.0.0.1,192.168.204.2,192.168.204.3,10.10.10.3,10.10.10.4"
I did not add docker_no_proxy list on localhost.yml.
Before Ansible NO_PROXY was empty, after Ansible it contains a list of IPs.
After Unlock:
[wrsroot@controller-0 ~(keystone_admin)]$ system application-list
+---------------------+---------+-------------------------------+---------------+--------------+------------------------------------------+
| application | version | manifest name | manifest file | status | progress |
+---------------------+---------+-------------------------------+---------------+--------------+------------------------------------------+
| platform-integ-apps | 1.0-5 | platform-integration-manifest | manifest.yaml | apply-failed | operation aborted, check logs for detail |
+---------------------+---------+-------------------------------+---------------+--------------+------------------------------------------+
Regards.
Juan Carlos Alonso
From: Ngo, Tee [mailto:Tee.Ngo@windriver.com]
Sent: Tuesday, May 28, 2019 6:03 PM
To: Perez Carranza, Jose <jose.perez.carranza@intel.com>; Alonso, Juan Carlos <juan.carlos.alonso@intel.com>
Cc: Lemus Contreras, Cristopher J <cristopher.j.lemus.contreras@intel.com>; Church, Robert <Robert.Church@windriver.com>; Wensley, Barton <Barton.Wensley@windriver.com>
Subject: RE: Live debug session
Just an update on docker_proxy test front:
Bart was able to get passed the upload of platform-integ-apps which entailed successful download of images using May 28th code (I think using May 27th official build is good enough for you
guys) and the following:
a)
Fixed the flakiness of the proxy server
b)
Provided docker_no_proxy list in the localhost.yml before running Ansible bootstrap
Note: Once this commit
https://review.opendev.org/#/c/661825/ gets merged, you would not need to provide docker_no_proxy list unless you want to add more IPs to the derived list
Bart is now having platform-integ-apps apply failure in his environment. Bob is investigating… So if you encounter apply failure, stay tuned.
Thanks,
Tee
P.S. Cristopher, this is regarding
docker_proxy test. Please let me know how far you get with private registry test in the other thread with Ada. Based on the screenshot you showed us today, you also passed upload stage in that config.
From: Ngo, Tee
Sent: May-28-19 4:23 PM
To: 'Perez Carranza, Jose'; Alonso, Juan Carlos
Cc: Lemus Contreras, Cristopher J; Church, Robert (Robert.Church@windriver.com)
Subject: RE: Live debug session
As discussed, JC and Jose, let’s switch to May 27th build.
Here are the instructions to disable automatic upload and apply of platform-integ-apps
Step 1: Switch to “fresh install” snapshot
Step 2: As root, back up the original file
cp /usr/lib64/python2.7/site-packages/sysinv/conductor/manager.py /usr/lib64/python2.7/site-packages/sysinv/conductor/manager.py.orig
Step 3: As root, update the manager.py file (see diff below)
vim /usr/lib64/python2.7/site-packages/sysinv/conductor/manager.py
--- a/sysinv/sysinv/sysinv/sysinv/conductor/manager.py
+++ b/sysinv/sysinv/sysinv/sysinv/conductor/manager.py
@@ -5223,6 +5223,10 @@ class ConductorManager(service.PeriodicService):
LOG.debug("Platform managed application %s: %s" % (app_name, status))
if status == constants.APP_NOT_PRESENT:
+ if not _met_app_apply_prerequisites(app_name):
+ LOG.info("Platform managed application %s: Prerequisites "
+ "not met." % app_name)
+ continue
if _patching_operation_is_occurring():
continue
Step 4: Take a snapshot, name it (fresh install + integ app disabled)
Step 5: Run Ansible and bring up the simplex system
Step 6: Verify that app is not installed after the unlock
system application-list
Step 7: Take a snapshot, name it (host unlocked, integ app disabled)
Step 8: Upload the app manually and check if there are errors
system application-upload /usr/local/share/applications/helm/platform-integ-apps-1.0-5.tgz
Step 9: Apply the app manually
system application-apply platform-integ-apps
Tee
From: Perez Carranza, Jose [mailto:jose.perez.carranza@intel.com]
Sent: May-28-19 2:54 PM
To: Ngo, Tee; Alonso, Juan Carlos
Cc: Lemus Contreras, Cristopher J
Subject: RE: Live debug session
I’m using last Friday code
BUILD_ID="20190524T013000Z"
Regards,
José
From: Ngo, Tee [mailto:Tee.Ngo@windriver.com]
Sent: Tuesday, May 28, 2019 1:51 PM
To: Perez Carranza, Jose <jose.perez.carranza@intel.com>; Alonso, Juan Carlos <juan.carlos.alonso@intel.com>
Cc: Lemus Contreras, Cristopher J <cristopher.j.lemus.contreras@intel.com>
Subject: RE: Live debug session
Are you running the latest code?
From: Perez Carranza, Jose [mailto:jose.perez.carranza@intel.com]
Sent: May-28-19 2:46 PM
To: Ngo, Tee; Alonso, Juan Carlos
Cc: Lemus Contreras, Cristopher J
Subject: RE: Live debug session
Hi Tee
I another machine I used both
docker_http_proxy and docker_https_proxy, but I’m also facing same issue. The only difference that I noted is that using
config_controller ips 192.168.204.4 and 10.10.10.5 listed as part of no_proxy whilst using a ANSIBLE that IP is not listed in no_proxy.
|
|
|
|
CONFIG_CONTROLLER
[Service]
Environment="HTTP_PROXY=http://proxy-chain.intel.com:911"
Environment="HTTPS_PROXY=http://proxy-chain.intel.com:912"
Environment="NO_PROXY=localhost,127.0.0.1,192.168.204.2,192.168.204.3,192.168.204.4,10.10.10.3,10.10.10.4,10.10.10.5"
ANSIBLE
[Service]
Environment="HTTP_PROXY=http://proxy-chain.intel.com:911"
Environment="HTTPS_PROXY=http://proxy-chain.intel.com:912"
Environment="NO_PROXY=localhost,127.0.0.1,192.168.204.2,192.168.204.3,10.10.10.3,10.10.10.4"
Regards,
José
From: Ngo, Tee [mailto:Tee.Ngo@windriver.com]
Sent: Tuesday, May 28, 2019 1:05 PM
To: Alonso, Juan Carlos <juan.carlos.alonso@intel.com>
Cc: Perez Carranza, Jose <jose.perez.carranza@intel.com>; Lemus Contreras, Cristopher J <cristopher.j.lemus.contreras@intel.com>
Subject: RE: Live debug session
A couple of findings:
1.
Your localhost.yml does not have http_proxy specified. Hence, it is missing in both http-proxy.conf and service-parameter-list output.
2.
There is a bug which I have resolved, for now please specify the list of docker_no_proxy in localhost.yml as you would have specified in .ini file for config_controller. I will explain in our meeting this afternoon.
Tee
From: Alonso, Juan Carlos [mailto:juan.carlos.alonso@intel.com]
Sent: May-28-19 1:37 PM
To: Ngo, Tee
Cc: Perez Carranza, Jose; Lemus Contreras, Cristopher J
Subject: RE: Live debug session
Find the information requested attached.
System host-parameter-list command does not exists. I think you are referring system service-parameter-list.
I only use https proxy for Ansible and for config controller both http and https proxy was set.
We did not get any ansible nor docker error during installation.
Regards.
Juan Carlos Alonso
From: Ngo, Tee [mailto:Tee.Ngo@windriver.com]
Sent: Tuesday, May 28, 2019 12:14 PM
To: Alonso, Juan Carlos <juan.carlos.alonso@intel.com>
Cc: Perez Carranza, Jose <jose.perez.carranza@intel.com>; Lemus Contreras, Cristopher J <cristopher.j.lemus.contreras@intel.com>
Subject: RE: Live debug session
By the way, could you send me the following info prior to debug meeting today from both Ansible setup and config_controller setup?
Content of localhost.yml
Content of /etc/systemd/system/docker.service.d/http-proxy.conf
Output of system host-parameter-list
Ansible/docker daemon error logs
Thank you.
Tee
From: Alonso, Juan Carlos [mailto:juan.carlos.alonso@intel.com]
Sent: May-28-19 12:44 PM
To: Ngo, Tee
Subject: Live debug session
Hi Tee,
What time works for you to have the live debug session?
Regards.
Juan Carlos Alonso