[Starlingx-discuss] Migrating git repos to OpenDev

[Clark Boylan]

On Thu, Mar 7, 2019, at 1:47 PM, James E. Blair wrote:
> Hi,
> 
> As discussed in November[1], the OpenStack project infrastructure is
> being rebranded as "OpenDev" to better support a wider community of
> projects.
> 
> We are nearly ready to perform the part of this transition with the
> largest impact: moving the authoritative git repositories for existing
> projects.
> 
> In this email, I'd like to introduce the new hosting system we are
> preparing, discuss the transition, and invite projects to work with us
> on the logistics of the change.
> 
> Gerrit
> ======
> 
> Gerrit is the core of our system and it will remain so in OpenDev.  As
> part of this move, we will rename the gerrit server from
> review.openstack.org to review.opendev.org.  As part of the transition,
> we will automatically merge appropriate changes to all branches of all
> repositories updating .gitreview and Zuul configuration files.  Any
> further changes (README files, etc.) we expect to be made by individual
> project contributors.
> 
> Repository Browsing
> ===================
> 
> Currently our canonical *public* repository system is the cgit server at
> https://git.openstack.org/ (and git.airshipit.org, git.starlingx.io, and
> git.zuul-ci.org).  This is a load balanced cluster of several servers
> which is designed to handle all the public git repository traffic, as it
> scales much better than Gerrit (and has a more friendly domain name).
> From a technical standpoint, it's excellent, but its usability could be
> improved.
> 
> Therefore, as part of this transition, we will replace the cgit servers
> with a new system based on Gitea.  Gitea is a complete development
> collaboration system, but it's very flexible and will allow us to
> disable components which we aren't using.  We will operate it in a
> read-only configuration where it will act as the public mirror for
> Gerrit.  The advantages it has over the current system are:
> 
> * Shorter domain name in project URLs:
>   https://git.openstack.org/openstack/nova   vs
>   https://opendev.org/openstack/nova
> * Clone and browsing URLs are the same (with cgit, the browsing URL has
>   an extra path component)
> * More visually pleasing code browsing
> * Integrated code searching
> * Ability to highlight multiple lines in links
> 
> When we perform the transition we will install redirects from
> git.openstack.org (and the other git sites) to opendev.org, and will
> maintain those redirects for the foreseeable future.  We will construct
> them so that even existing deep links to individual files in individual
> commits to cgit will redirect to the correct location on opendev.org.
> 
> This system is up and running now with a live mirror of data from
> Gerrit, and you can start testing it out today at https://opendev.org/
> 
> Please let us know if you encounter any problems.
> 
> If you would like to read more about the design of this system and the
> transition, see the infra-spec[2].
> 
> GitHub
> ======
> 
> Currently all OpenStack projects are replicated to GitHub.  We do not
> plan on changing that during the transition, however, any projects
> outside of the openstack*/ namespaces will not automatically be
> replicated to GitHub, and we do not plan on adding that in the future.
> We do, however, support projects using Zuul to run post-merge jobs to
> push updates to GitHub or any other third-party mirrors with their own
> credentials.  We will be happy to work with anyone interested in that to
> help set up jobs to do so.
> 
> We are adopting this approach so that individual projects can have more
> control over how they are represented in social media, and to give us
> more flexibility in supporting our own organizational namespaces on
> OpenDev without assuming they map directly to GitHub.
> 
> Eventually we plan on moving the OpenStack project to that system as
> well and retiring direct replication from Gerrit to GitHub completely.
> But we will defer that work until after this transition.
> 
> Logistics
> =========
> 
> We can prepare much of the system in advance (as we have for the hosting
> system on opendev.org), but the actual transition and renaming of the
> Gerrit server will need to happen at once during an outage window.  We
> need to schedule that outage and begin preparing for it.
> 
> Since all of the project git URLs are going to change (to replace
> git.openstack.org with opendev.org and review.openstack.org with
> review.opendev.org), we can additionally take the opportunity to
> reorganize projects into different organizations.
> 
> For example, during the transition we will rename Zuul, and it's
> associated projects, from the "openstack-infra" org to "zuul".  So their
> new names will be "zuul/zuul", "zuul/nodepool", etc.
> 
> This is an excellent time for the rest of the OpenStack Foundation
> pilot projects to do the same.
> 
> If the OpenStack project desires this, it would also be a good time to
> move unofficial projects out of the openstack/ namespace.
> 
> Therefore, we need your help:
> 
> Action Items
> ============
> 
> We need each of the following projects:
> 
>   * OpenStack
>   * Airship
>   * StarlingX
>   * Zuul
> 
> To nominate a single point of contact to work with us on the transition.
> It would be helpful for that person to attend the next (and possibly
> next several) openstack infra team meetings in IRC [3].  We will work
> with those people on scheduling the transition, as well as finalizing
> the list of projects which should be renamed as part of the transition.
> 
> If you manage an unofficial project and would like to take the
> opportunity to move or rename your project, please add it to this
> ethercalc[4].
> 
> [1] 
> http://lists.openstack.org/pipermail/openstack-dev/2018-November/136403.html
> [2] 
> http://specs.openstack.org/openstack-infra/infra-specs/specs/opendev-gerrit.html
> [3] http://eavesdrop.openstack.org/#Project_Infrastructure_Team_Meeting
> [4] https://ethercalc.openstack.org/opendev-transition

We've made good progress in preparing this change and are still on track to do this Friday April 19, 2019.  Our project liasons have been drawing up lists of projects to rename during the outage. One big thing to keep in mind is unofficial OpenStack projects will no longer be in the "openstack" namespace.  They will be placed in the 'x/' namespace instead which intends to indicate no endorsement or special ownership.

One side effect of this change (as noted in Jim's earlier email) is that we will stop replicating projects that move out of the openstack namespace. Projects that wish to be replicated to Github or anywhere else they like can do so following the steps that David Moreau-Simard put together for us here [5].

This transition is likely to be a bit bumpy particularly at the start. We'll be around after the transition to help fix unexpected errors and are likely to spend a fair bit of time at the PTG improving things as well.

Finally, to be extra clear, we intend to put http redirects in place so that all your old http(s) urls continue to work.  Fungi has set this up for testing with details here [6] if you would like to ensure your urls redirect properly.

[5] http://lists.openstack.org/pipermail/openstack-discuss/2019-April/005007.html
[6] http://lists.openstack.org/pipermail/openstack-discuss/2019-April/004921.html

Thank you for your patience and feel free to reach out with any questions you might have.

Clark