On Fri, Aug 3, 2018 at 9:31 AM, Scott Little <scott.little@windriver.com> wrote:
So it seems there can be multiple queues per git. Withing a queue there are dependencies, but different queues can make progress independently? Or is there a queue of queues for the git?
[0] is the Zuul concept page, but basically it has a set of pipelines for different job types, the two we care the most about are check and gate. Check jobs run on every Gerrit submission, gate jobs run after Workflow +1 is set. There is overlap between those job sets (check may have run last week) but some things like non-voting jobs don't run in the gate. The other pipeline we'll be using is experimental, for on-demand runs, that's where I plan to put the initial py3 jobs for example, so we can see where we are but not waste resource running them all the time.
On top of that, it seems like there are a finite number of execution engines. If no execution engines are available, none of your queues will progress. Is that about right?
Not just about, that is it exactly. An older Zuul status showed the VM allocation graphs at the bottom, I'm not sure where those went after the Zuul v3 upgrade... All of OpenStack CI is run on donated cloud resources (mostly single-use VMs) from places like Rackspace, Vexxhost, OVH, Dreamcloud and about 7 more that I don't remember offhand. We have a quota on each cloud and the load during North American working hours usually puts us way over. This is a big part of why Zuul was born, to dynamically manage that ever-changing pool of test resources (VMs). It turns out that hosting OpenStack CI is an excellent cloud load test, we've found more than a few scaling problems this way. dt [0] https://zuul-ci.org/docs/zuul/user/concepts.html -- Dean Troyer dtroyer@gmail.com