[Starlingx-discuss] [build] go packages, version 2

[Victor Rodriguez]

Hi Jason


On Tue, Jan 15, 2019 at 11:57 AM McKenna, Jason
<Jason.McKenna at windriver.com> wrote:
>
> Hi build team,
>
>
>
> At the previous build meeting I had identified an issue with the way some go based packages were being built be built (they required internet access), and promised I’d update the mailing list on a potential way forward that we were prototyping.
>
>
>
> Some preliminary points:
>
> -          go usually attempts to resolve dependencies at build time, by going out to the internet and fetching stuff (like dependency source code) using the “go get” command
>
> -          Sometimes the stuff fetched by “go get” isn’t appropriate (i.e. “go get” fetches the latest version, but deprecated APIs may have been removed, etc)
>
> -          Different versions of go packages may require different versions of dependencies
>
> -          We want builds to be reproducible without unexpected code changes (i.e. we want to know what we’re compiling in)
>
> -          Some people build in environments where they don’t have Internet access
>
>
>
> The initial solution (which didn’t take into account the Internet access problem) was to use “dep”.  “dep” is an external tool which was an “official experiment” of the go project.  Rather than fetch the latest dependencies from the internet (like “go get”), it allowed specific revisions of dependencies to be captured.  “dep” fetched those versions from the Internet.  This solved the deprecated API issue, the reproducible build issue, and the issue of not using rpms for the dependencies.  However, if someone was attempting to build in an internet-less context, the system would fail.
>
>
>
> Enter this second revision.
>
>
>
> The dependency packages are now downloaded at download-mirrors.sh time as tarballs.  The tarballs are produced as of a specific commit for each dependency.

I agree taht this is a good solution , in terms of make it work , my
concern is that we are mantaining a lot of tar balls , and that scares
me a bit

 This allows us to hit all our bullet points – code is snapshotted,
reproducible, we don’t end up having to create a bunch of new rpms
with dependency source code and potential version conflicts, and it
requires no internet access (other than at download-mirrors.sh time)

If it works for you is fine , my concern will be to mantain changes of
broken links in the future ( that sure go dep will fix )

+1 from my part as long as it does not geneerate a lot of mantainance
does not create a problem for ourselvs

How hard to aloud internet during build ?

Regards

>
>
>
> Jerry has posted a preview code review showing his work using this mechanism.  I’ve marked the reviews as workflow -1 to give the build team a chance to see the mechanism.
>
> https://review.openstack.org/#/c/631001/
>
> https://review.openstack.org/#/c/631002/
>
>
>
> -Jason
>
> _______________________________________________
> Starlingx-discuss mailing list
> Starlingx-discuss at lists.starlingx.io
> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss