Adding this thread to public mailing list.

 

From: Alonso, Juan Carlos
Sent: Tuesday, May 28, 2019 7:08 PM
To: 'Ngo, Tee' <Tee.Ngo@windriver.com>; Perez Carranza, Jose <jose.perez.carranza@intel.com>
Cc: Lemus Contreras, Cristopher J <cristopher.j.lemus.contreras@intel.com>; Church, Robert <Robert.Church@windriver.com>; Wensley, Barton <Barton.Wensley@windriver.com>
Subject: RE: Live debug session

 

Hi Tee,

 

After make the workarounds you told, platform-integ-apps is still uploading and applying automatically, applying still failing.

I did not put docker_no_proxy list on localhost.yml since I saw last email after finish test.

As Barton, I got apply-failed, but app uploaded and applied automatically.

 

 

manager.py:

 

 

After Ansible:

[Service]

Environment="HTTPS_PROXY=http://proxy-chain.intel.com:912"

Environment="NO_PROXY=localhost,127.0.0.1,192.168.204.2,192.168.204.3,10.10.10.3,10.10.10.4"

 

I did not add docker_no_proxy list on localhost.yml.

Before Ansible NO_PROXY was empty, after Ansible it contains a list of IPs.

 

After Unlock:

[wrsroot@controller-0 ~(keystone_admin)]$ system application-list

+---------------------+---------+-------------------------------+---------------+--------------+------------------------------------------+

| application         | version | manifest name                 | manifest file | status       | progress                                 |

+---------------------+---------+-------------------------------+---------------+--------------+------------------------------------------+

| platform-integ-apps | 1.0-5   | platform-integration-manifest | manifest.yaml | apply-failed | operation aborted, check logs for detail |

+---------------------+---------+-------------------------------+---------------+--------------+------------------------------------------+

 

Regards.

Juan Carlos Alonso

 

 

From: Ngo, Tee [mailto:Tee.Ngo@windriver.com]
Sent: Tuesday, May 28, 2019 6:03 PM
To: Perez Carranza, Jose <jose.perez.carranza@intel.com>; Alonso, Juan Carlos <juan.carlos.alonso@intel.com>
Cc: Lemus Contreras, Cristopher J <cristopher.j.lemus.contreras@intel.com>; Church, Robert <Robert.Church@windriver.com>; Wensley, Barton <Barton.Wensley@windriver.com>
Subject: RE: Live debug session

 

Just an update on docker_proxy test front:

 

Bart was able to get passed the upload of platform-integ-apps which entailed successful download of images using May 28th code (I think using May 27th official build is good enough for you guys) and the following:

a)      Fixed the flakiness of the proxy server

b)      Provided docker_no_proxy list in the localhost.yml before running Ansible bootstrap

Note: Once this commit https://review.opendev.org/#/c/661825/ gets merged, you would not need to provide docker_no_proxy list unless you want to add more IPs to the derived list

 

Bart is now having platform-integ-apps apply failure in his environment. Bob is investigating… So if you encounter apply failure, stay tuned.

 

Thanks,

Tee

P.S. Cristopher, this is regarding docker_proxy test. Please let me know how far you get with private registry test in the other thread with Ada. Based on the screenshot you showed us today, you also passed upload stage in that config.

 

From: Ngo, Tee
Sent: May-28-19 4:23 PM
To: 'Perez Carranza, Jose'; Alonso, Juan Carlos
Cc: Lemus Contreras, Cristopher J; Church, Robert (Robert.Church@windriver.com)
Subject: RE: Live debug session

 

As discussed, JC and Jose, let’s switch to May 27th build.

 

Here are the instructions to disable automatic upload and apply of platform-integ-apps

 

Step 1: Switch to “fresh install” snapshot

 

Step 2: As root, back up the original file

cp /usr/lib64/python2.7/site-packages/sysinv/conductor/manager.py /usr/lib64/python2.7/site-packages/sysinv/conductor/manager.py.orig

 

Step 3: As root, update the manager.py file (see diff below)

vim /usr/lib64/python2.7/site-packages/sysinv/conductor/manager.py

 

 

--- a/sysinv/sysinv/sysinv/sysinv/conductor/manager.py

+++ b/sysinv/sysinv/sysinv/sysinv/conductor/manager.py

@@ -5223,6 +5223,10 @@ class ConductorManager(service.PeriodicService):

 

             LOG.debug("Platform managed application %s: %s" % (app_name, status))

             if status == constants.APP_NOT_PRESENT:

+                if not _met_app_apply_prerequisites(app_name):

+                    LOG.info("Platform managed application %s: Prerequisites "

+                             "not met." % app_name)

+                    continue

 

                 if _patching_operation_is_occurring():

                     continue

 

Step 4: Take a snapshot, name it (fresh install + integ app disabled)

 

Step 5: Run Ansible and bring up the simplex system

 

Step 6: Verify that app is not installed after the unlock

system application-list

 

Step 7: Take a snapshot, name it (host unlocked, integ app disabled)

 

Step 8: Upload the app manually and check if there are errors

system application-upload /usr/local/share/applications/helm/platform-integ-apps-1.0-5.tgz

 

Step 9: Apply the app manually

system application-apply platform-integ-apps

 

Tee

 

From: Perez Carranza, Jose [mailto:jose.perez.carranza@intel.com]
Sent: May-28-19 2:54 PM
To: Ngo, Tee; Alonso, Juan Carlos
Cc: Lemus Contreras, Cristopher J
Subject: RE: Live debug session

 

I’m using last Friday code

 

BUILD_ID="20190524T013000Z"

 

Regards,

José

 

From: Ngo, Tee [mailto:Tee.Ngo@windriver.com]
Sent: Tuesday, May 28, 2019 1:51 PM
To: Perez Carranza, Jose <jose.perez.carranza@intel.com>; Alonso, Juan Carlos <juan.carlos.alonso@intel.com>
Cc: Lemus Contreras, Cristopher J <cristopher.j.lemus.contreras@intel.com>
Subject: RE: Live debug session

 

Are you running the latest code?

 

From: Perez Carranza, Jose [mailto:jose.perez.carranza@intel.com]
Sent: May-28-19 2:46 PM
To: Ngo, Tee; Alonso, Juan Carlos
Cc: Lemus Contreras, Cristopher J
Subject: RE: Live debug session

 

Hi Tee

 

I another machine I used both docker_http_proxy and docker_https_proxy, but I’m also facing same issue. The only difference that I noted is that using config_controller ips 192.168.204.4 and  10.10.10.5 listed as part of no_proxy  whilst  using a ANSIBLE that IP is not listed in no_proxy.

 

 

 

CONFIG_CONTROLLER

[Service]

Environment="HTTP_PROXY=http://proxy-chain.intel.com:911"

Environment="HTTPS_PROXY=http://proxy-chain.intel.com:912"

Environment="NO_PROXY=localhost,127.0.0.1,192.168.204.2,192.168.204.3,192.168.204.4,10.10.10.3,10.10.10.4,10.10.10.5"

 

ANSIBLE

[Service]

Environment="HTTP_PROXY=http://proxy-chain.intel.com:911"

Environment="HTTPS_PROXY=http://proxy-chain.intel.com:912"

Environment="NO_PROXY=localhost,127.0.0.1,192.168.204.2,192.168.204.3,10.10.10.3,10.10.10.4"

 

Regards,

José

 

From: Ngo, Tee [mailto:Tee.Ngo@windriver.com]
Sent: Tuesday, May 28, 2019 1:05 PM
To: Alonso, Juan Carlos <juan.carlos.alonso@intel.com>
Cc: Perez Carranza, Jose <jose.perez.carranza@intel.com>; Lemus Contreras, Cristopher J <cristopher.j.lemus.contreras@intel.com>
Subject: RE: Live debug session

 

A couple of findings:

1.       Your localhost.yml does not have http_proxy specified. Hence, it is missing in both http-proxy.conf and service-parameter-list output.

2.       There is a bug which I have resolved, for now please specify the list of docker_no_proxy in localhost.yml as you would have specified in .ini file for config_controller. I will explain in our meeting this afternoon.

 

Tee

 

From: Alonso, Juan Carlos [mailto:juan.carlos.alonso@intel.com]
Sent: May-28-19 1:37 PM
To: Ngo, Tee
Cc: Perez Carranza, Jose; Lemus Contreras, Cristopher J
Subject: RE: Live debug session

 

Find the information requested attached.

 

System host-parameter-list command does not exists. I think you are referring system service-parameter-list.

I only use https proxy for Ansible and for config controller both http and https proxy was set.

 

We did not get any ansible nor docker error during installation.

 

Regards.

Juan Carlos Alonso

 

From: Ngo, Tee [mailto:Tee.Ngo@windriver.com]
Sent: Tuesday, May 28, 2019 12:14 PM
To: Alonso, Juan Carlos <juan.carlos.alonso@intel.com>
Cc: Perez Carranza, Jose <jose.perez.carranza@intel.com>; Lemus Contreras, Cristopher J <cristopher.j.lemus.contreras@intel.com>
Subject: RE: Live debug session

 

By the way, could you send me the following info prior to debug meeting today from both Ansible setup and config_controller setup?

 

Content of localhost.yml

Content of /etc/systemd/system/docker.service.d/http-proxy.conf

Output of system host-parameter-list

 

Ansible/docker daemon error logs

 

Thank you.

Tee

 

From: Alonso, Juan Carlos [mailto:juan.carlos.alonso@intel.com]
Sent: May-28-19 12:44 PM
To: Ngo, Tee
Subject: Live debug session

 

Hi Tee,

 

What time works for you to have the live debug session?

 

Regards.

Juan Carlos Alonso