I raised the bug for this

https://bugs.launchpad.net/starlingx/+bug/1840292

My attempted change (dropping nova/neutron) did not work, so I have removed devstack as a zuul job from config.

If approved, this will almost definitely need to be cherry picked to R2.0, since I suspect any attempted cherry-picks for already committed bugs back to the R2 config branch will also fail zuul.

From: Bailey, Henry Albert (Al) [mailto:Al.Bailey@windriver.com]
Sent: Thursday, August 15, 2019 11:39 AM
To: Rowsell, Brent; starlingx-discuss
Subject: Re: [Starlingx-discuss] StarlingX devstack jobs randomly failing

So I think that because the devstack job for config is requiring nova and neutron, and nova pulls in glance, that is why.

I think those jobs were started before containerized openstack support, so I don’t think we need nova and neutron anymore.

I’m trying this review

https://review.opendev.org/#/c/676719/

If that does not work, we likely just need to turn off devstack job for config.

Our env (sysinv) uses postgres, and perhaps postgres support has been dropped in train.

From: Rowsell, Brent
Sent: Thursday, August 15, 2019 11:31 AM
To: Bailey, Henry Albert (Al); starlingx-discuss
Subject: RE: StarlingX devstack jobs randomly failing

Why would config be setting up a glance db ?

Brent

From: Bailey, Henry Albert (Al) [mailto:Al.Bailey@windriver.com]
Sent: Thursday, August 15, 2019 11:12 AM
To: starlingx-discuss <starlingx-discuss@lists.starlingx.io>
Subject: [Starlingx-discuss] StarlingX devstack jobs randomly failing

The devstack zuul jobs for config are failing.

The error looks to be a database migration failure when initializing the glance DB

These are the DB components being installed

psycopg2 2.8.3

SQLAlchemy 1.3.6

Sqlalchemy_migrate 0.12.0

I think this may be due to this upstream commit which merged this morning

https://review.opendev.org/#/c/665606/

I don’t have any idea on how to fix this. We can disable the jobs, or remove glance from our devstack jobs, but we are blocked until this is resolved.

It may affect other flock repos as well, if they also have devstack jobs that are setting up glance.

Example stacktrace:

File "/opt/stack/glance/glance/db/sqlalchemy/alembic_migrations/data_migrations/train_migrate01_backend_to_store.py", line 28, in has_migrations
    metadata_backend = con.execute(sql_query)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 982, in execute
    return self._execute_text(object_, multiparams, params)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1155, in _execute_text
    parameters,
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1248, in _execute_context
    e, statement, parameters, cursor, context
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1464, in _handle_dbapi_exception
    util.raise_from_cause(newraise, exc_info)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 398, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1244, in _execute_context
    cursor, statement, parameters, context
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 552, in do_execute
    cursor.execute(statement, parameters)
DBError: (psycopg2.errors.UndefinedFunction) function instr(text, unknown) does not exist
LINE 1: select meta_data from image_locations where INSTR(meta_data,...
                                                    ^
HINT: No function matches the given name and argument types. You might need to add explicit type casts.

[SQL: select meta_data from image_locations where INSTR(meta_data, '"backend":') > 0]
(Background on this error at: http://sqlalche.me/e/f405)
Upgraded database to: train_expand01, current revision(s): train_expand01