[Starlingx-discuss] [Build] CENGN unable to build debian master since March 3
scott.little at eng.windriver.com
Wed Mar 8 15:56:50 UTC 2023
CENGN has been unable to complete a build since March 3.
Since then, four builds were attempted, and all builds hung within the
post-build unit tests of python3.9_3.9.2-1.stx.1.
One build was hung for nearly 48 hours.
The logs are not specific on which test is hanging.
Killing the hung unit test results in the overall build of the python3.9
package build failing, but a retry loop attempts to rebuild, and it
again hangs in the unit tests.
There is nothing in the change logs that directly affect this package.
The only build system changes relate to secure boot. signing and as the
python package is not one that requires signing, I'm currently
discounting those as a cause.
An equivalent build using an internal WindRiver build machine has so far
not hit this issue. The main difference being that CENGN uses minikube,
and the internal server is using kubernetes directly.
One theory was that something change upstream that affects the content
within the build containers. However, both CENGN and the internal build
server rebuild the build containers each time. If there was a change
upstream, I would expect both builds to see it.
Most designers are likely using minikube, and so far I've seen no
complaints from designers on this topic. Perhaps designers are using a
build environment created on or before March 3, and haven't seen it
yet. Have you encountered hung builds in the last week? Please report
the issue, and we would like to know when the build containers were last
rebuilt. We'll be trying to setup a fresh minikube build later today.
At some point we may be forced to disable the python unit tests as a
work around, until a better solution presents itself.
More information about the Starlingx-discuss