[Starlingx-discuss] [Containers] Approach for adding a local mirror of docker images

Sat Jan 26 15:54:44 UTC 2019

Hi Brent,

Did you get the chance to update more info to SB 2004711?

Mingyuan

-----Original Message-----
From: Rowsell, Brent [mailto:Brent.Rowsell at windriver.com] 
Sent: Wednesday, January 23, 2019 10:51
To: Qi, Mingyuan <mingyuan.qi at intel.com>; Ponce Castaneda, Guillermo A <guillermo.a.ponce.castaneda at intel.com>; Saul Wold <sgw at linux.intel.com>; Miller, Frank <Frank.Miller at windriver.com>
Cc: starlingx-discuss at lists.starlingx.io
Subject: RE: [Starlingx-discuss] [Containers] Approach for adding a local mirror of docker images

We will need to provide the ability to update the address for all images in question if the standard external registries are not used. 
As mentioned below, I will add some additional detail to the story.

Brent 

-----Original Message-----
From: Qi, Mingyuan [mailto:mingyuan.qi at intel.com] 
Sent: Tuesday, January 22, 2019 9:48 PM
To: Rowsell, Brent <Brent.Rowsell at windriver.com>; Ponce Castaneda, Guillermo A <guillermo.a.ponce.castaneda at intel.com>; Saul Wold <sgw at linux.intel.com>; Miller, Frank <Frank.Miller at windriver.com>
Cc: starlingx-discuss at lists.starlingx.io
Subject: RE: [Starlingx-discuss] [Containers] Approach for adding a local mirror of docker images

Memo,

The issue is that during config_controller to bootstrap kubernetes and apply stx-openstack-helm, timeout occurred if pulling images from public docker registry are too slow.

Regarding to the "local mirror", it means (1)setting up a local mirror registry server within the LAN that controller-0 can access, or (2)redirect default docker registry to a public docker registry mirror located in user's local region. Both of them are not related to the "on-host" registry on controller-0.

One obstacle is that docker native mirror support is not enough for registries like gcr.io/quay.io. The project you mentioned[0] is a good approach for users willing/be able to setup a self-controlled local registry mirror. It won't introduce much change in starlingx and leverages proxy setting as the redirection to registry mirror. But for users not willing to hold a local registry mirror and relies on regional public registry mirror, it's not an option.

These 2 cases seems can't be resolved by one approach. My thinking for the later one is to enhance docker's native mirror pulling mechanism, or to add/change the override of the image address.

[0] https://github.com/rpardini/docker-registry-proxy

Mingyuan

-----Original Message-----
From: Rowsell, Brent [mailto:Brent.Rowsell at windriver.com] 
Sent: Wednesday, January 23, 2019 4:28
To: Ponce Castaneda, Guillermo A <guillermo.a.ponce.castaneda at intel.com>; Saul Wold <sgw at linux.intel.com>; Miller, Frank <Frank.Miller at windriver.com>
Cc: starlingx-discuss at lists.starlingx.io
Subject: Re: [Starlingx-discuss] [Containers] Approach for adding a local mirror of docker images

The controller will already host a local docker registry after the system has been boot strapped. 
What is required is a private external registry that a system call pull from.
I will be updating the story with additional info

Brent  

-----Original Message-----
From: Ponce Castaneda, Guillermo A [mailto:guillermo.a.ponce.castaneda at intel.com] 
Sent: Tuesday, January 22, 2019 3:20 PM
To: Saul Wold <sgw at linux.intel.com>; Miller, Frank <Frank.Miller at windriver.com>
Cc: starlingx-discuss at lists.starlingx.io
Subject: Re: [Starlingx-discuss] [Containers] Approach for adding a local mirror of docker images

Hello Frank and team,
After having some ideas exchanged with Saul, we reach to the idea that the Docker registry/proxy could be located on the controller-0, and that way all the other nodes will point to it to pull the docker images they require.
This approach will require to have extra disk space on controller-0 to host the images that all the other want to pull, and we also need to change the configs on every node so they can see the controller-0 as docker registry.

What do you all think?
- Memo

On 1/21/19, 4:29 PM, "Ponce Castaneda, Guillermo A" <guillermo.a.ponce.castaneda at intel.com> wrote:

    On 1/21/19, 3:39 PM, "Saul Wold" <sgw at linux.intel.com> wrote:

        On 1/21/19 11:54 AM, Ponce Castaneda, Guillermo A wrote:
        > Hello Frank,
        > 
        > Thanks for initiating the conversation, my proposed solution is to bring 
        > up a docker registry that will have to be in the local network of each 
        > office, so the speed of the pulls will be the faster.
        > 
        > The problem with this approach might be that the references of the 
        > docker pull have to change so it points to the local docker registry, I 
        > have already implemented this approach locally at GDC and can provide 
        > documentation on how to do this.
        > 
        > Another approach that I am researching is to use this project: 
        > https://github.com/rpardini/docker-registry-proxy, so far this option 
        > seems much better but I need to explore it a little bit further, I will 
        > provide more details on it as soon as possible.
        > 
        Do we need access to more than the standard docker hub?  It also seems 
        that this approach will require modifications to the images wanting to 
        use the proxy.

    We do not really need access to more than the standard docker hub, but this one
    way to solve the problem of the people having troubles with slow networks, the
    docker registry proxy method promises to be transparent for the user, the user
    will have to modify their docker daemon file to add the registry as proxy and just
    pull images normally, I am working to set that up and do a test on our network
    right now, once it is done I will be able to tell if it is really transparent.

        I am sure this is true in most proxy setups.

        Sau!

        > All the feedback and other ideas are welcome.
        > 
        > Thanks and Regards.
        > 
        > Guillermo (Memo) Ponce
        > 
        > *From: *"Miller, Frank" <Frank.Miller at windriver.com>
        > *Date: *Monday, January 21, 2019 at 12:54 PM
        > *To: *"Martin, Guillermo Oscar" <Guillermo-Oscar.Martin at windriver.com>
        > *Cc: *"'starlingx-discuss at lists.starlingx.io'" 
        > <starlingx-discuss at lists.starlingx.io>
        > *Subject: *[Starlingx-discuss] [Containers] Approach for adding a local 
        > mirror of docker images
        > 
        > Guillermo:
        > 
        > As discussed at today’s containerization meeting, please reply with your 
        > initial thoughts on how to address 
        > https://storyboard.openstack.org/#!/story/2004711 .  If you first need 
        > to ask a member of the containerization subteam a few questions to 
        > understand what is needed then try reaching out to Bob Church and Angie 
        > Wang.
        > 
        > Frank
        > 
        > 
        > _______________________________________________
        > Starlingx-discuss mailing list
        > Starlingx-discuss at lists.starlingx.io
        > http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
        > 

        _______________________________________________
        Starlingx-discuss mailing list
        Starlingx-discuss at lists.starlingx.io
        http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss

    _______________________________________________
    Starlingx-discuss mailing list
    Starlingx-discuss at lists.starlingx.io
    http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss

_______________________________________________
Starlingx-discuss mailing list
Starlingx-discuss at lists.starlingx.io
http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
_______________________________________________
Starlingx-discuss mailing list
Starlingx-discuss at lists.starlingx.io
http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss