[Starlingx-discuss] Trying a "mirrorless" build
Hi all,
I tried an experiment on building StarlingX without downloading a mirror and use mock/yum to download and resolve depenencies on the fly.
I want to share with you this report[0] with the findings.
I hope you can find this interesting :)
-Erich
[0] https://gist.github.com/ericho/049d1908f5d80485541e918515996702
I think there is a fundamental tension between between two views of were StarlingX should go.
Vision A, StarlingX's is about a usable iso) + If software context X (tag, branch what ever) compiles today, it should compile tomorrow. + Software context X (tag, branch what ever) should always produce the same iso when compiled. - Requires that the set of packages that we compile against be frozen in some fashion.
Vision B, StarlingX is just software) + Always use the latest and greatest packages. I don't want to hand manage which external packages are used. - Builds may break at any time due to external factors, even on release branches.
Folks who want to make a functioning product likely prefer A.
Folks that want to work with the latest features, or prototyping new features might prefer B.
You experiment is solidly in the B camp.
I did experiment with yum priorities recently, but it didn't deliver the desired result. An older StarlingX package with higher priority failed to override a newer upstream package. Instead yum threw an error because newest package was ... 'eclipsed'... I think the word was. Couldn't find a quick way to suppress that error, so I abandoned the experiment.
Scott
On 18-07-27 07:02 PM, Cordoba Malibran, Erich wrote:
Hi all,
I tried an experiment on building StarlingX without downloading a mirror and use mock/yum to download and resolve depenencies on the fly.
I want to share with you this report[0] with the findings.
I hope you can find this interesting :)
-Erich
[0] https://gist.github.com/ericho/049d1908f5d80485541e918515996702 _______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
Scott, you raised a very interesting point.
I think that the tension between A and B is more about wanting to do both at the same time. We want to provide high levels of performance and stability, pushing us toward A. But the pain of maintaining the patches necessary and managing older package versions pushes us toward B.
I think we need to find the balancing point between the two - where performance is high and the pain as low.
brucej
-----Original Message----- From: Scott Little [mailto:scott.little@windriver.com] Sent: Monday, July 30, 2018 7:40 AM To: starlingx-discuss@lists.starlingx.io Subject: Re: [Starlingx-discuss] Trying a "mirrorless" build
I think there is a fundamental tension between between two views of were StarlingX should go.
Vision A, StarlingX's is about a usable iso) + If software context X (tag, branch what ever) compiles today, it should compile tomorrow. + Software context X (tag, branch what ever) should always produce the same iso when compiled. - Requires that the set of packages that we compile against be frozen in some fashion.
Vision B, StarlingX is just software) + Always use the latest and greatest packages. I don't want to hand manage which external packages are used. - Builds may break at any time due to external factors, even on release branches.
Folks who want to make a functioning product likely prefer A.
Folks that want to work with the latest features, or prototyping new features might prefer B.
You experiment is solidly in the B camp.
I did experiment with yum priorities recently, but it didn't deliver the desired result. An older StarlingX package with higher priority failed to override a newer upstream package. Instead yum threw an error because newest package was ... 'eclipsed'... I think the word was. Couldn't find a quick way to suppress that error, so I abandoned the experiment.
Scott
On 18-07-27 07:02 PM, Cordoba Malibran, Erich wrote:
Hi all,
I tried an experiment on building StarlingX without downloading a mirror and use mock/yum to download and resolve depenencies on the fly.
I want to share with you this report[0] with the findings.
I hope you can find this interesting :)
-Erich
[0] https://gist.github.com/ericho/049d1908f5d80485541e918515996702 _______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
_______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
On 07/30/2018 11:18 AM, Jones, Bruce E wrote:
Scott, you raised a very interesting point.
I think that the tension between A and B is more about wanting to do both at the same time. We want to provide high levels of performance and stability, pushing us toward A. But the pain of maintaining the patches necessary and managing older package versions pushes us toward B.
I think we need to find the balancing point between the two - where performance is high and the pain as low.
I think Scott's point about A is not so much about keeping the same set of packages forever, but rather that if I build a given release today and then build that same release a month (or a year) from now it should result in the same package versions in the ISO.
Theoretically I think you could have both behaviours as long as:
1) Each build using the latest upstream package versions gets an auto-generated identifier and all package versions used in that build get dumped into a mirror somewhere. 2) If you build with an explicit identifier you would use the package versions that were originally used for the identified build.
This would let you run with the latest and greatest during a development cycle, and then lock down the versions on a release branch.
Chris
Chris wrote:
I think Scott's point about A is not so much about keeping the same set of packages forever, but rather that if I build a given release today and then build that same release a month (or a year) from now it should result in the same package versions in the ISO.
I agree that repeatable release builds should be the goal. That's why we branch, of course. I don't think we need (or want) to get to completely repeatable releases (e.g. at the toolchain level) but just at the package level.
For package dependencies, I know of three ways to handle them - 1) create a big mirror that includes the union of everything 2) version the mirror in parallel with the release branches 3) let koji handle dependencies.
I think we're at 1) now, we should start moving to 2) with a long term goal of moving to 3).
brucej
-----Original Message----- From: Chris Friesen [mailto:chris.friesen@windriver.com] Sent: Monday, July 30, 2018 10:51 AM To: starlingx-discuss@lists.starlingx.io Subject: Re: [Starlingx-discuss] Trying a "mirrorless" build
On 07/30/2018 11:18 AM, Jones, Bruce E wrote:
Scott, you raised a very interesting point.
I think that the tension between A and B is more about wanting to do both at the same time. We want to provide high levels of performance and stability, pushing us toward A. But the pain of maintaining the patches necessary and managing older package versions pushes us toward B.
I think we need to find the balancing point between the two - where performance is high and the pain as low.
I think Scott's point about A is not so much about keeping the same set of packages forever, but rather that if I build a given release today and then build that same release a month (or a year) from now it should result in the same package versions in the ISO.
Theoretically I think you could have both behaviours as long as:
1) Each build using the latest upstream package versions gets an auto-generated identifier and all package versions used in that build get dumped into a mirror somewhere. 2) If you build with an explicit identifier you would use the package versions that were originally used for the identified build.
This would let you run with the latest and greatest during a development cycle, and then lock down the versions on a release branch.
Chris
_______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
participants (4)
-
Chris Friesen
-
Cordoba Malibran, Erich
-
Jones, Bruce E
-
Scott Little