[Starlingx-discuss] StarlingX.repo using CentOS 7.3 binary repos instead of 7.4?
Hi all, The StarlingX.repo file in stx-tools has all the CentOS 7.4.1708 binary repos disabled and is instead using 7.3.1611. This would likely explain why some of the specified RPMs cannot be downloaded via yum, as they were upversioned in 7.4. Was this done for a particular reason, or is it a configuration error? Don Penney, Developer, Wind River
StarlingX should be based on Centos 7.4 for the vast majority of packages. I think there are a dozen that have made the move to 7.5. There are also a few that we blacklisted and are pulling from earlier releases. I'll attach my black_list notes as they exist today. I'll have to audit them to make sure my notes are up to date. Scott On 18-07-18 10:47 AM, Penney, Don wrote:
Hi all,
The StarlingX.repo file in stx-tools has all the CentOS 7.4.1708 binary repos disabled and is instead using 7.3.1611. This would likely explain why some of the specified RPMs cannot be downloaded via yum, as they were upversioned in 7.4.
Was this done for a particular reason, or is it a configuration error?
*Don Penney*, Developer, *Wind River*
_______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
What if we move to the latest 7.5 1804? Do you see any drawbacks or hard dependencies still on old packages? Regards, Yong From: Scott Little <scott.little@windriver.com> Date: Wednesday, 18 July 2018 at 11:17 PM To: "starlingx-discuss@lists.starlingx.io" <starlingx-discuss@lists.starlingx.io> Subject: Re: [Starlingx-discuss] StarlingX.repo using CentOS 7.3 binary repos instead of 7.4? StarlingX should be based on Centos 7.4 for the vast majority of packages. I think there are a dozen that have made the move to 7.5. There are also a few that we blacklisted and are pulling from earlier releases. I'll attach my black_list notes as they exist today. I'll have to audit them to make sure my notes are up to date. Scott On 18-07-18 10:47 AM, Penney, Don wrote: Hi all, The StarlingX.repo file in stx-tools has all the CentOS 7.4.1708 binary repos disabled and is instead using 7.3.1611. This would likely explain why some of the specified RPMs cannot be downloaded via yum, as they were upversioned in 7.4. Was this done for a particular reason, or is it a configuration error? Don Penney, Developer, Wind River _______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io<mailto:Starlingx-discuss@lists.starlingx.io> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
Moving to 7.5 should be considered as part of the upcoming release, but retirement of existing patches needs to be part of that plan. Brent From: Hu, Yong [mailto:yong.hu@intel.com] Sent: Wednesday, July 18, 2018 8:48 PM To: Little, Scott <Scott.Little@windriver.com>; starlingx-discuss@lists.starlingx.io Subject: Re: [Starlingx-discuss] StarlingX.repo using CentOS 7.3 binary repos instead of 7.4? What if we move to the latest 7.5 1804? Do you see any drawbacks or hard dependencies still on old packages? Regards, Yong From: Scott Little <scott.little@windriver.com<mailto:scott.little@windriver.com>> Date: Wednesday, 18 July 2018 at 11:17 PM To: "starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>" <starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>> Subject: Re: [Starlingx-discuss] StarlingX.repo using CentOS 7.3 binary repos instead of 7.4? StarlingX should be based on Centos 7.4 for the vast majority of packages. I think there are a dozen that have made the move to 7.5. There are also a few that we blacklisted and are pulling from earlier releases. I'll attach my black_list notes as they exist today. I'll have to audit them to make sure my notes are up to date. Scott On 18-07-18 10:47 AM, Penney, Don wrote: Hi all, The StarlingX.repo file in stx-tools has all the CentOS 7.4.1708 binary repos disabled and is instead using 7.3.1611. This would likely explain why some of the specified RPMs cannot be downloaded via yum, as they were upversioned in 7.4. Was this done for a particular reason, or is it a configuration error? Don Penney, Developer, Wind River _______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io<mailto:Starlingx-discuss@lists.starlingx.io> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
Brent, Scott, This is great information! It turns out the way of handling the retired RPM packages are different from WR internal mirror and Intel scripts (to generate mirror): - WR will move the retired RPM and its dependency RPM from 7.4 repo to 7.5 repo, and this is where the blacklist from Scott coming from; - Intel turn to other 3rd party repo (hosted by somebody else other than RHS) and download the exact same version (the missing ones from RHS repo due to retirement). To me, if WR has gone through the RPM dependency map analysis and correctly migrate the retired RPMs from 7.4 to 7.5, then I think it’s a more reliable way. Also this is prefer way from security point of view. @ Scott, I think this worthy a discussion in Bruce’s meeting for how to setup external mirror. And I wish this could be a short-term solution before Cesar’s long-term solution up & running. Then we can ensure external build can work for WR. Thanks. - cindy From: Rowsell, Brent [mailto:Brent.Rowsell@windriver.com] Sent: Thursday, July 19, 2018 9:04 AM To: Hu, Yong <yong.hu@intel.com>; Little, Scott <Scott.Little@windriver.com>; starlingx-discuss@lists.starlingx.io Subject: Re: [Starlingx-discuss] StarlingX.repo using CentOS 7.3 binary repos instead of 7.4? Moving to 7.5 should be considered as part of the upcoming release, but retirement of existing patches needs to be part of that plan. Brent From: Hu, Yong [mailto:yong.hu@intel.com] Sent: Wednesday, July 18, 2018 8:48 PM To: Little, Scott <Scott.Little@windriver.com<mailto:Scott.Little@windriver.com>>; starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Subject: Re: [Starlingx-discuss] StarlingX.repo using CentOS 7.3 binary repos instead of 7.4? What if we move to the latest 7.5 1804? Do you see any drawbacks or hard dependencies still on old packages? Regards, Yong From: Scott Little <scott.little@windriver.com<mailto:scott.little@windriver.com>> Date: Wednesday, 18 July 2018 at 11:17 PM To: "starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>" <starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>> Subject: Re: [Starlingx-discuss] StarlingX.repo using CentOS 7.3 binary repos instead of 7.4? StarlingX should be based on Centos 7.4 for the vast majority of packages. I think there are a dozen that have made the move to 7.5. There are also a few that we blacklisted and are pulling from earlier releases. I'll attach my black_list notes as they exist today. I'll have to audit them to make sure my notes are up to date. Scott On 18-07-18 10:47 AM, Penney, Don wrote: Hi all, The StarlingX.repo file in stx-tools has all the CentOS 7.4.1708 binary repos disabled and is instead using 7.3.1611. This would likely explain why some of the specified RPMs cannot be downloaded via yum, as they were upversioned in 7.4. Was this done for a particular reason, or is it a configuration error? Don Penney, Developer, Wind River _______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io<mailto:Starlingx-discuss@lists.starlingx.io> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
Hi Cindy, Just as a heads’ up, we have not done the analysis to move from CentOS 7.4 to 7.5. The blacklist we generated was as we migrated from 7.2, through 7.3, and to 7.4. We’ve uprev’d a few packages since then to the match 7.5, but this has been on an as-required basis. -Jason From: Xie, Cindy <cindy.xie@intel.com> Sent: July 19, 2018 2:57 AM To: Rowsell, Brent <Brent.Rowsell@windriver.com>; Hu, Yong <yong.hu@intel.com>; Little, Scott <Scott.Little@windriver.com>; starlingx-discuss@lists.starlingx.io Subject: Re: [Starlingx-discuss] StarlingX.repo using CentOS 7.3 binary repos instead of 7.4? Brent, Scott, This is great information! It turns out the way of handling the retired RPM packages are different from WR internal mirror and Intel scripts (to generate mirror): - WR will move the retired RPM and its dependency RPM from 7.4 repo to 7.5 repo, and this is where the blacklist from Scott coming from; - Intel turn to other 3rd party repo (hosted by somebody else other than RHS) and download the exact same version (the missing ones from RHS repo due to retirement). To me, if WR has gone through the RPM dependency map analysis and correctly migrate the retired RPMs from 7.4 to 7.5, then I think it’s a more reliable way. Also this is prefer way from security point of view. @ Scott, I think this worthy a discussion in Bruce’s meeting for how to setup external mirror. And I wish this could be a short-term solution before Cesar’s long-term solution up & running. Then we can ensure external build can work for WR. Thanks. - cindy From: Rowsell, Brent [mailto:Brent.Rowsell@windriver.com] Sent: Thursday, July 19, 2018 9:04 AM To: Hu, Yong <yong.hu@intel.com<mailto:yong.hu@intel.com>>; Little, Scott <Scott.Little@windriver.com<mailto:Scott.Little@windriver.com>>; starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Subject: Re: [Starlingx-discuss] StarlingX.repo using CentOS 7.3 binary repos instead of 7.4? Moving to 7.5 should be considered as part of the upcoming release, but retirement of existing patches needs to be part of that plan. Brent From: Hu, Yong [mailto:yong.hu@intel.com] Sent: Wednesday, July 18, 2018 8:48 PM To: Little, Scott <Scott.Little@windriver.com<mailto:Scott.Little@windriver.com>>; starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Subject: Re: [Starlingx-discuss] StarlingX.repo using CentOS 7.3 binary repos instead of 7.4? What if we move to the latest 7.5 1804? Do you see any drawbacks or hard dependencies still on old packages? Regards, Yong From: Scott Little <scott.little@windriver.com<mailto:scott.little@windriver.com>> Date: Wednesday, 18 July 2018 at 11:17 PM To: "starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>" <starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>> Subject: Re: [Starlingx-discuss] StarlingX.repo using CentOS 7.3 binary repos instead of 7.4? StarlingX should be based on Centos 7.4 for the vast majority of packages. I think there are a dozen that have made the move to 7.5. There are also a few that we blacklisted and are pulling from earlier releases. I'll attach my black_list notes as they exist today. I'll have to audit them to make sure my notes are up to date. Scott On 18-07-18 10:47 AM, Penney, Don wrote: Hi all, The StarlingX.repo file in stx-tools has all the CentOS 7.4.1708 binary repos disabled and is instead using 7.3.1611. This would likely explain why some of the specified RPMs cannot be downloaded via yum, as they were upversioned in 7.4. Was this done for a particular reason, or is it a configuration error? Don Penney, Developer, Wind River _______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io<mailto:Starlingx-discuss@lists.starlingx.io> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
Jason, Thanks for letting me know that the list was used for evaluating 7.2 to 7.3 as well as to 7.4. If we all think this is a quicker way to address the retired RPM issues, then we can start looking into using those retired packages from 7.5. Thanks. - cindy From: McKenna, Jason [mailto:Jason.McKenna@windriver.com] Sent: Thursday, July 19, 2018 10:36 PM To: Xie, Cindy <cindy.xie@intel.com>; Rowsell, Brent <Brent.Rowsell@windriver.com>; Hu, Yong <yong.hu@intel.com>; Little, Scott <Scott.Little@windriver.com>; starlingx-discuss@lists.starlingx.io Subject: RE: [Starlingx-discuss] StarlingX.repo using CentOS 7.3 binary repos instead of 7.4? Hi Cindy, Just as a heads’ up, we have not done the analysis to move from CentOS 7.4 to 7.5. The blacklist we generated was as we migrated from 7.2, through 7.3, and to 7.4. We’ve uprev’d a few packages since then to the match 7.5, but this has been on an as-required basis. -Jason From: Xie, Cindy <cindy.xie@intel.com<mailto:cindy.xie@intel.com>> Sent: July 19, 2018 2:57 AM To: Rowsell, Brent <Brent.Rowsell@windriver.com<mailto:Brent.Rowsell@windriver.com>>; Hu, Yong <yong.hu@intel.com<mailto:yong.hu@intel.com>>; Little, Scott <Scott.Little@windriver.com<mailto:Scott.Little@windriver.com>>; starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Subject: Re: [Starlingx-discuss] StarlingX.repo using CentOS 7.3 binary repos instead of 7.4? Brent, Scott, This is great information! It turns out the way of handling the retired RPM packages are different from WR internal mirror and Intel scripts (to generate mirror): - WR will move the retired RPM and its dependency RPM from 7.4 repo to 7.5 repo, and this is where the blacklist from Scott coming from; - Intel turn to other 3rd party repo (hosted by somebody else other than RHS) and download the exact same version (the missing ones from RHS repo due to retirement). To me, if WR has gone through the RPM dependency map analysis and correctly migrate the retired RPMs from 7.4 to 7.5, then I think it’s a more reliable way. Also this is prefer way from security point of view. @ Scott, I think this worthy a discussion in Bruce’s meeting for how to setup external mirror. And I wish this could be a short-term solution before Cesar’s long-term solution up & running. Then we can ensure external build can work for WR. Thanks. - cindy From: Rowsell, Brent [mailto:Brent.Rowsell@windriver.com] Sent: Thursday, July 19, 2018 9:04 AM To: Hu, Yong <yong.hu@intel.com<mailto:yong.hu@intel.com>>; Little, Scott <Scott.Little@windriver.com<mailto:Scott.Little@windriver.com>>; starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Subject: Re: [Starlingx-discuss] StarlingX.repo using CentOS 7.3 binary repos instead of 7.4? Moving to 7.5 should be considered as part of the upcoming release, but retirement of existing patches needs to be part of that plan. Brent From: Hu, Yong [mailto:yong.hu@intel.com] Sent: Wednesday, July 18, 2018 8:48 PM To: Little, Scott <Scott.Little@windriver.com<mailto:Scott.Little@windriver.com>>; starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Subject: Re: [Starlingx-discuss] StarlingX.repo using CentOS 7.3 binary repos instead of 7.4? What if we move to the latest 7.5 1804? Do you see any drawbacks or hard dependencies still on old packages? Regards, Yong From: Scott Little <scott.little@windriver.com<mailto:scott.little@windriver.com>> Date: Wednesday, 18 July 2018 at 11:17 PM To: "starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>" <starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>> Subject: Re: [Starlingx-discuss] StarlingX.repo using CentOS 7.3 binary repos instead of 7.4? StarlingX should be based on Centos 7.4 for the vast majority of packages. I think there are a dozen that have made the move to 7.5. There are also a few that we blacklisted and are pulling from earlier releases. I'll attach my black_list notes as they exist today. I'll have to audit them to make sure my notes are up to date. Scott On 18-07-18 10:47 AM, Penney, Don wrote: Hi all, The StarlingX.repo file in stx-tools has all the CentOS 7.4.1708 binary repos disabled and is instead using 7.3.1611. This would likely explain why some of the specified RPMs cannot be downloaded via yum, as they were upversioned in 7.4. Was this done for a particular reason, or is it a configuration error? Don Penney, Developer, Wind River _______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io<mailto:Starlingx-discuss@lists.starlingx.io> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
Hi Scott, I noticed in the black_list, there are a couple of RPMs (such as erlang*-R16B*.rpm”), which were in tis-r4-CentOS mirror but were NOT used since tis-r5-CentOS. As well, we are certain they are not needed for building R5 or current StarlingX codebase. Could we just drop them? BTW: in tis-r5-CentOS mirror, there are over 30,000 RPMs, while actually among those only ~1900 RPMs are necessary. regards, Yong From: Scott Little <scott.little@windriver.com> Date: Wednesday, 18 July 2018 at 11:17 PM To: "starlingx-discuss@lists.starlingx.io" <starlingx-discuss@lists.starlingx.io> Subject: Re: [Starlingx-discuss] StarlingX.repo using CentOS 7.3 binary repos instead of 7.4? StarlingX should be based on Centos 7.4 for the vast majority of packages. I think there are a dozen that have made the move to 7.5. There are also a few that we blacklisted and are pulling from earlier releases. I'll attach my black_list notes as they exist today. I'll have to audit them to make sure my notes are up to date. Scott On 18-07-18 10:47 AM, Penney, Don wrote: Hi all, The StarlingX.repo file in stx-tools has all the CentOS 7.4.1708 binary repos disabled and is instead using 7.3.1611. This would likely explain why some of the specified RPMs cannot be downloaded via yum, as they were upversioned in 7.4. Was this done for a particular reason, or is it a configuration error? Don Penney, Developer, Wind River _______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io<mailto:Starlingx-discuss@lists.starlingx.io> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
erlang*18.3.4.4*rpm are the ones to use. The R16B version should have been excluded. Yes, our mirror was of all ~30000 rpms. In this way, when a designer adds a new package or dependency, it and it's recursive dependencies were already present and ready for use. The method that has been concocted for StarlingX requires adding packages to .lst files every time there is a change. Further there doesn't seem to be a tool to assist in the search for missing dependencies. The result is likely to be several iterations of ... add package, compile, fail due to missing dependency, add package, compile, fail due to missing dependency ... very time consuming and frustrating. Scott On 18-07-19 09:00 PM, Hu, Yong wrote:
Hi Scott,
I noticed in the black_list, there are a couple of RPMs (such as erlang*-R16B*.rpm”), which were in tis-r4-CentOS mirror but were NOT used since tis-r5-CentOS.
As well, we are certain they are not needed for building R5 or current StarlingX codebase.
Could we just drop them?
BTW: in tis-r5-CentOS mirror, there are over 30,000 RPMs, while actually among those only ~1900 RPMs are necessary.
regards,
Yong
*From: *Scott Little <scott.little@windriver.com> *Date: *Wednesday, 18 July 2018 at 11:17 PM *To: *"starlingx-discuss@lists.starlingx.io" <starlingx-discuss@lists.starlingx.io> *Subject: *Re: [Starlingx-discuss] StarlingX.repo using CentOS 7.3 binary repos instead of 7.4?
StarlingX should be based on Centos 7.4 for the vast majority of packages. I think there are a dozen that have made the move to 7.5. There are also a few that we blacklisted and are pulling from earlier releases.
I'll attach my black_list notes as they exist today. I'll have to audit them to make sure my notes are up to date.
Scott
On 18-07-18 10:47 AM, Penney, Don wrote:
Hi all,
The StarlingX.repo file in stx-tools has all the CentOS 7.4.1708 binary repos disabled and is instead using 7.3.1611. This would likely explain why some of the specified RPMs cannot be downloaded via yum, as they were upversioned in 7.4.
Was this done for a particular reason, or is it a configuration error?
*Don Penney*, Developer, *Wind River*
_______________________________________________
Starlingx-discuss mailing list
Starlingx-discuss@lists.starlingx.io<mailto:Starlingx-discuss@lists.starlingx.io>
http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
Managing the rpm dependency tree in an automated way is one of the key things I’m hoping we can learn from (leverage) from the Clear Linux project. They seem to have solved this problem. Several ex-Clear team members are on our team now and thinking about this… brucej From: Scott Little [mailto:scott.little@windriver.com] Sent: Friday, July 20, 2018 6:48 AM To: Hu, Yong <yong.hu@intel.com>; starlingx-discuss@lists.starlingx.io Subject: Re: [Starlingx-discuss] StarlingX.repo using CentOS 7.3 binary repos instead of 7.4? erlang*18.3.4.4*rpm are the ones to use. The R16B version should have been excluded. Yes, our mirror was of all ~30000 rpms. In this way, when a designer adds a new package or dependency, it and it's recursive dependencies were already present and ready for use. The method that has been concocted for StarlingX requires adding packages to .lst files every time there is a change. Further there doesn't seem to be a tool to assist in the search for missing dependencies. The result is likely to be several iterations of ... add package, compile, fail due to missing dependency, add package, compile, fail due to missing dependency ... very time consuming and frustrating. Scott On 18-07-19 09:00 PM, Hu, Yong wrote: Hi Scott, I noticed in the black_list, there are a couple of RPMs (such as erlang*-R16B*.rpm”), which were in tis-r4-CentOS mirror but were NOT used since tis-r5-CentOS. As well, we are certain they are not needed for building R5 or current StarlingX codebase. Could we just drop them? BTW: in tis-r5-CentOS mirror, there are over 30,000 RPMs, while actually among those only ~1900 RPMs are necessary. regards, Yong From: Scott Little <scott.little@windriver.com><mailto:scott.little@windriver.com> Date: Wednesday, 18 July 2018 at 11:17 PM To: "starlingx-discuss@lists.starlingx.io"<mailto:starlingx-discuss@lists.starlingx.io> <starlingx-discuss@lists.starlingx.io><mailto:starlingx-discuss@lists.starlingx.io> Subject: Re: [Starlingx-discuss] StarlingX.repo using CentOS 7.3 binary repos instead of 7.4? StarlingX should be based on Centos 7.4 for the vast majority of packages. I think there are a dozen that have made the move to 7.5. There are also a few that we blacklisted and are pulling from earlier releases. I'll attach my black_list notes as they exist today. I'll have to audit them to make sure my notes are up to date. Scott On 18-07-18 10:47 AM, Penney, Don wrote: Hi all, The StarlingX.repo file in stx-tools has all the CentOS 7.4.1708 binary repos disabled and is instead using 7.3.1611. This would likely explain why some of the specified RPMs cannot be downloaded via yum, as they were upversioned in 7.4. Was this done for a particular reason, or is it a configuration error? Don Penney, Developer, Wind River _______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io<mailto:Starlingx-discuss@lists.starlingx.io> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
We have 2 types of dependencies to handle: 1). Build dependencies (indicated by “build requires”) and 2) deployment (installation) dependencies (indicated by “requires”). If we are aware of a new package is needed, this package name could be added into *.lst, and accordingly the package itself can be managed to download and added into the mirror. But the *indirect* dependencies brought into by this new package couldn’t be explicitly figured out until the build failures, going along with the building iterations. It would be helpful to have a script to analyze the dependency chain, BEFORE actually trying the build iterations painfully. Would love to see how Clear Linux project efficiently resolve this problem. Regards, Yong From: "Jones, Bruce E" <bruce.e.jones@intel.com> Date: Friday, 20 July 2018 at 11:59 PM To: Scott Little <scott.little@windriver.com>, "Hu, Yong" <yong.hu@intel.com>, "starlingx-discuss@lists.starlingx.io" <starlingx-discuss@lists.starlingx.io> Subject: RE: [Starlingx-discuss] StarlingX.repo using CentOS 7.3 binary repos instead of 7.4? Managing the rpm dependency tree in an automated way is one of the key things I’m hoping we can learn from (leverage) from the Clear Linux project. They seem to have solved this problem. Several ex-Clear team members are on our team now and thinking about this… brucej From: Scott Little [mailto:scott.little@windriver.com] Sent: Friday, July 20, 2018 6:48 AM To: Hu, Yong <yong.hu@intel.com>; starlingx-discuss@lists.starlingx.io Subject: Re: [Starlingx-discuss] StarlingX.repo using CentOS 7.3 binary repos instead of 7.4? erlang*18.3.4.4*rpm are the ones to use. The R16B version should have been excluded. Yes, our mirror was of all ~30000 rpms. In this way, when a designer adds a new package or dependency, it and it's recursive dependencies were already present and ready for use. The method that has been concocted for StarlingX requires adding packages to .lst files every time there is a change. Further there doesn't seem to be a tool to assist in the search for missing dependencies. The result is likely to be several iterations of ... add package, compile, fail due to missing dependency, add package, compile, fail due to missing dependency ... very time consuming and frustrating. Scott On 18-07-19 09:00 PM, Hu, Yong wrote: Hi Scott, I noticed in the black_list, there are a couple of RPMs (such as erlang*-R16B*.rpm”), which were in tis-r4-CentOS mirror but were NOT used since tis-r5-CentOS. As well, we are certain they are not needed for building R5 or current StarlingX codebase. Could we just drop them? BTW: in tis-r5-CentOS mirror, there are over 30,000 RPMs, while actually among those only ~1900 RPMs are necessary. regards, Yong From: Scott Little <scott.little@windriver.com><mailto:scott.little@windriver.com> Date: Wednesday, 18 July 2018 at 11:17 PM To: "starlingx-discuss@lists.starlingx.io"<mailto:starlingx-discuss@lists.starlingx.io> <starlingx-discuss@lists.starlingx.io><mailto:starlingx-discuss@lists.starlingx.io> Subject: Re: [Starlingx-discuss] StarlingX.repo using CentOS 7.3 binary repos instead of 7.4? StarlingX should be based on Centos 7.4 for the vast majority of packages. I think there are a dozen that have made the move to 7.5. There are also a few that we blacklisted and are pulling from earlier releases. I'll attach my black_list notes as they exist today. I'll have to audit them to make sure my notes are up to date. Scott On 18-07-18 10:47 AM, Penney, Don wrote: Hi all, The StarlingX.repo file in stx-tools has all the CentOS 7.4.1708 binary repos disabled and is instead using 7.3.1611. This would likely explain why some of the specified RPMs cannot be downloaded via yum, as they were upversioned in 7.4. Was this done for a particular reason, or is it a configuration error? Don Penney, Developer, Wind River _______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io<mailto:Starlingx-discuss@lists.starlingx.io> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
So I guess Chen Yan’s scripts can solve the dependencies of 2), right? From: Hu, Yong [mailto:yong.hu@intel.com] Sent: Monday, July 23, 2018 9:07 AM To: Jones, Bruce E <bruce.e.jones@intel.com>; Scott Little <scott.little@windriver.com>; starlingx-discuss@lists.starlingx.io Subject: Re: [Starlingx-discuss] StarlingX.repo using CentOS 7.3 binary repos instead of 7.4? We have 2 types of dependencies to handle: 1). Build dependencies (indicated by “build requires”) and 2) deployment (installation) dependencies (indicated by “requires”). If we are aware of a new package is needed, this package name could be added into *.lst, and accordingly the package itself can be managed to download and added into the mirror. But the *indirect* dependencies brought into by this new package couldn’t be explicitly figured out until the build failures, going along with the building iterations. It would be helpful to have a script to analyze the dependency chain, BEFORE actually trying the build iterations painfully. Would love to see how Clear Linux project efficiently resolve this problem. Regards, Yong From: "Jones, Bruce E" <bruce.e.jones@intel.com<mailto:bruce.e.jones@intel.com>> Date: Friday, 20 July 2018 at 11:59 PM To: Scott Little <scott.little@windriver.com<mailto:scott.little@windriver.com>>, "Hu, Yong" <yong.hu@intel.com<mailto:yong.hu@intel.com>>, "starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>" <starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>> Subject: RE: [Starlingx-discuss] StarlingX.repo using CentOS 7.3 binary repos instead of 7.4? Managing the rpm dependency tree in an automated way is one of the key things I’m hoping we can learn from (leverage) from the Clear Linux project. They seem to have solved this problem. Several ex-Clear team members are on our team now and thinking about this… brucej From: Scott Little [mailto:scott.little@windriver.com] Sent: Friday, July 20, 2018 6:48 AM To: Hu, Yong <yong.hu@intel.com<mailto:yong.hu@intel.com>>; starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Subject: Re: [Starlingx-discuss] StarlingX.repo using CentOS 7.3 binary repos instead of 7.4? erlang*18.3.4.4*rpm are the ones to use. The R16B version should have been excluded. Yes, our mirror was of all ~30000 rpms. In this way, when a designer adds a new package or dependency, it and it's recursive dependencies were already present and ready for use. The method that has been concocted for StarlingX requires adding packages to .lst files every time there is a change. Further there doesn't seem to be a tool to assist in the search for missing dependencies. The result is likely to be several iterations of ... add package, compile, fail due to missing dependency, add package, compile, fail due to missing dependency ... very time consuming and frustrating. Scott On 18-07-19 09:00 PM, Hu, Yong wrote: Hi Scott, I noticed in the black_list, there are a couple of RPMs (such as erlang*-R16B*.rpm”), which were in tis-r4-CentOS mirror but were NOT used since tis-r5-CentOS. As well, we are certain they are not needed for building R5 or current StarlingX codebase. Could we just drop them? BTW: in tis-r5-CentOS mirror, there are over 30,000 RPMs, while actually among those only ~1900 RPMs are necessary. regards, Yong From: Scott Little <scott.little@windriver.com><mailto:scott.little@windriver.com> Date: Wednesday, 18 July 2018 at 11:17 PM To: "starlingx-discuss@lists.starlingx.io"<mailto:starlingx-discuss@lists.starlingx.io> <starlingx-discuss@lists.starlingx.io><mailto:starlingx-discuss@lists.starlingx.io> Subject: Re: [Starlingx-discuss] StarlingX.repo using CentOS 7.3 binary repos instead of 7.4? StarlingX should be based on Centos 7.4 for the vast majority of packages. I think there are a dozen that have made the move to 7.5. There are also a few that we blacklisted and are pulling from earlier releases. I'll attach my black_list notes as they exist today. I'll have to audit them to make sure my notes are up to date. Scott On 18-07-18 10:47 AM, Penney, Don wrote: Hi all, The StarlingX.repo file in stx-tools has all the CentOS 7.4.1708 binary repos disabled and is instead using 7.3.1611. This would likely explain why some of the specified RPMs cannot be downloaded via yum, as they were upversioned in 7.4. Was this done for a particular reason, or is it a configuration error? Don Penney, Developer, Wind River _______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io<mailto:Starlingx-discuss@lists.starlingx.io> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
I guess so, currently we already get some information from the rpms. We will review the data first and I will improve the tool to get more. Yan From: Xie, Cindy [mailto:cindy.xie@intel.com] Sent: Monday, July 23, 2018 11:55 To: Hu, Yong <yong.hu@intel.com>; Jones, Bruce E <bruce.e.jones@intel.com>; Scott Little <scott.little@windriver.com>; starlingx-discuss@lists.starlingx.io Subject: Re: [Starlingx-discuss] StarlingX.repo using CentOS 7.3 binary repos instead of 7.4? So I guess Chen Yan’s scripts can solve the dependencies of 2), right? From: Hu, Yong [mailto:yong.hu@intel.com] Sent: Monday, July 23, 2018 9:07 AM To: Jones, Bruce E <bruce.e.jones@intel.com<mailto:bruce.e.jones@intel.com>>; Scott Little <scott.little@windriver.com<mailto:scott.little@windriver.com>>; starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Subject: Re: [Starlingx-discuss] StarlingX.repo using CentOS 7.3 binary repos instead of 7.4? We have 2 types of dependencies to handle: 1). Build dependencies (indicated by “build requires”) and 2) deployment (installation) dependencies (indicated by “requires”). If we are aware of a new package is needed, this package name could be added into *.lst, and accordingly the package itself can be managed to download and added into the mirror. But the *indirect* dependencies brought into by this new package couldn’t be explicitly figured out until the build failures, going along with the building iterations. It would be helpful to have a script to analyze the dependency chain, BEFORE actually trying the build iterations painfully. Would love to see how Clear Linux project efficiently resolve this problem. Regards, Yong From: "Jones, Bruce E" <bruce.e.jones@intel.com<mailto:bruce.e.jones@intel.com>> Date: Friday, 20 July 2018 at 11:59 PM To: Scott Little <scott.little@windriver.com<mailto:scott.little@windriver.com>>, "Hu, Yong" <yong.hu@intel.com<mailto:yong.hu@intel.com>>, "starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>" <starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>> Subject: RE: [Starlingx-discuss] StarlingX.repo using CentOS 7.3 binary repos instead of 7.4? Managing the rpm dependency tree in an automated way is one of the key things I’m hoping we can learn from (leverage) from the Clear Linux project. They seem to have solved this problem. Several ex-Clear team members are on our team now and thinking about this… brucej From: Scott Little [mailto:scott.little@windriver.com] Sent: Friday, July 20, 2018 6:48 AM To: Hu, Yong <yong.hu@intel.com<mailto:yong.hu@intel.com>>; starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Subject: Re: [Starlingx-discuss] StarlingX.repo using CentOS 7.3 binary repos instead of 7.4? erlang*18.3.4.4*rpm are the ones to use. The R16B version should have been excluded. Yes, our mirror was of all ~30000 rpms. In this way, when a designer adds a new package or dependency, it and it's recursive dependencies were already present and ready for use. The method that has been concocted for StarlingX requires adding packages to .lst files every time there is a change. Further there doesn't seem to be a tool to assist in the search for missing dependencies. The result is likely to be several iterations of ... add package, compile, fail due to missing dependency, add package, compile, fail due to missing dependency ... very time consuming and frustrating. Scott On 18-07-19 09:00 PM, Hu, Yong wrote: Hi Scott, I noticed in the black_list, there are a couple of RPMs (such as erlang*-R16B*.rpm”), which were in tis-r4-CentOS mirror but were NOT used since tis-r5-CentOS. As well, we are certain they are not needed for building R5 or current StarlingX codebase. Could we just drop them? BTW: in tis-r5-CentOS mirror, there are over 30,000 RPMs, while actually among those only ~1900 RPMs are necessary. regards, Yong From: Scott Little <scott.little@windriver.com><mailto:scott.little@windriver.com> Date: Wednesday, 18 July 2018 at 11:17 PM To: "starlingx-discuss@lists.starlingx.io"<mailto:starlingx-discuss@lists.starlingx.io> <starlingx-discuss@lists.starlingx.io><mailto:starlingx-discuss@lists.starlingx.io> Subject: Re: [Starlingx-discuss] StarlingX.repo using CentOS 7.3 binary repos instead of 7.4? StarlingX should be based on Centos 7.4 for the vast majority of packages. I think there are a dozen that have made the move to 7.5. There are also a few that we blacklisted and are pulling from earlier releases. I'll attach my black_list notes as they exist today. I'll have to audit them to make sure my notes are up to date. Scott On 18-07-18 10:47 AM, Penney, Don wrote: Hi all, The StarlingX.repo file in stx-tools has all the CentOS 7.4.1708 binary repos disabled and is instead using 7.3.1611. This would likely explain why some of the specified RPMs cannot be downloaded via yum, as they were upversioned in 7.4. Was this done for a particular reason, or is it a configuration error? Don Penney, Developer, Wind River _______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io<mailto:Starlingx-discuss@lists.starlingx.io> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
The script I sent in another thread (Adding packages to build .lst files) uses repoquery to recursively determine dependencies of packages, with commands like: repoquery --quiet -c $REPOCFG --resolve --requires --recursive From: Chen, Yan [mailto:yan.chen@intel.com] Sent: Monday, July 23, 2018 1:54 AM To: Xie, Cindy; Hu, Yong; Jones, Bruce E; Little, Scott; starlingx-discuss@lists.starlingx.io Subject: Re: [Starlingx-discuss] StarlingX.repo using CentOS 7.3 binary repos instead of 7.4? I guess so, currently we already get some information from the rpms. We will review the data first and I will improve the tool to get more. Yan From: Xie, Cindy [mailto:cindy.xie@intel.com] Sent: Monday, July 23, 2018 11:55 To: Hu, Yong <yong.hu@intel.com>; Jones, Bruce E <bruce.e.jones@intel.com>; Scott Little <scott.little@windriver.com>; starlingx-discuss@lists.starlingx.io Subject: Re: [Starlingx-discuss] StarlingX.repo using CentOS 7.3 binary repos instead of 7.4? So I guess Chen Yan’s scripts can solve the dependencies of 2), right? From: Hu, Yong [mailto:yong.hu@intel.com] Sent: Monday, July 23, 2018 9:07 AM To: Jones, Bruce E <bruce.e.jones@intel.com<mailto:bruce.e.jones@intel.com>>; Scott Little <scott.little@windriver.com<mailto:scott.little@windriver.com>>; starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Subject: Re: [Starlingx-discuss] StarlingX.repo using CentOS 7.3 binary repos instead of 7.4? We have 2 types of dependencies to handle: 1). Build dependencies (indicated by “build requires”) and 2) deployment (installation) dependencies (indicated by “requires”). If we are aware of a new package is needed, this package name could be added into *.lst, and accordingly the package itself can be managed to download and added into the mirror. But the *indirect* dependencies brought into by this new package couldn’t be explicitly figured out until the build failures, going along with the building iterations. It would be helpful to have a script to analyze the dependency chain, BEFORE actually trying the build iterations painfully. Would love to see how Clear Linux project efficiently resolve this problem. Regards, Yong From: "Jones, Bruce E" <bruce.e.jones@intel.com<mailto:bruce.e.jones@intel.com>> Date: Friday, 20 July 2018 at 11:59 PM To: Scott Little <scott.little@windriver.com<mailto:scott.little@windriver.com>>, "Hu, Yong" <yong.hu@intel.com<mailto:yong.hu@intel.com>>, "starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>" <starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>> Subject: RE: [Starlingx-discuss] StarlingX.repo using CentOS 7.3 binary repos instead of 7.4? Managing the rpm dependency tree in an automated way is one of the key things I’m hoping we can learn from (leverage) from the Clear Linux project. They seem to have solved this problem. Several ex-Clear team members are on our team now and thinking about this… brucej From: Scott Little [mailto:scott.little@windriver.com] Sent: Friday, July 20, 2018 6:48 AM To: Hu, Yong <yong.hu@intel.com<mailto:yong.hu@intel.com>>; starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Subject: Re: [Starlingx-discuss] StarlingX.repo using CentOS 7.3 binary repos instead of 7.4? erlang*18.3.4.4*rpm are the ones to use. The R16B version should have been excluded. Yes, our mirror was of all ~30000 rpms. In this way, when a designer adds a new package or dependency, it and it's recursive dependencies were already present and ready for use. The method that has been concocted for StarlingX requires adding packages to .lst files every time there is a change. Further there doesn't seem to be a tool to assist in the search for missing dependencies. The result is likely to be several iterations of ... add package, compile, fail due to missing dependency, add package, compile, fail due to missing dependency ... very time consuming and frustrating. Scott On 18-07-19 09:00 PM, Hu, Yong wrote: Hi Scott, I noticed in the black_list, there are a couple of RPMs (such as erlang*-R16B*.rpm”), which were in tis-r4-CentOS mirror but were NOT used since tis-r5-CentOS. As well, we are certain they are not needed for building R5 or current StarlingX codebase. Could we just drop them? BTW: in tis-r5-CentOS mirror, there are over 30,000 RPMs, while actually among those only ~1900 RPMs are necessary. regards, Yong From: Scott Little <scott.little@windriver.com><mailto:scott.little@windriver.com> Date: Wednesday, 18 July 2018 at 11:17 PM To: "starlingx-discuss@lists.starlingx.io"<mailto:starlingx-discuss@lists.starlingx.io> <starlingx-discuss@lists.starlingx.io><mailto:starlingx-discuss@lists.starlingx.io> Subject: Re: [Starlingx-discuss] StarlingX.repo using CentOS 7.3 binary repos instead of 7.4? StarlingX should be based on Centos 7.4 for the vast majority of packages. I think there are a dozen that have made the move to 7.5. There are also a few that we blacklisted and are pulling from earlier releases. I'll attach my black_list notes as they exist today. I'll have to audit them to make sure my notes are up to date. Scott On 18-07-18 10:47 AM, Penney, Don wrote: Hi all, The StarlingX.repo file in stx-tools has all the CentOS 7.4.1708 binary repos disabled and is instead using 7.3.1611. This would likely explain why some of the specified RPMs cannot be downloaded via yum, as they were upversioned in 7.4. Was this done for a particular reason, or is it a configuration error? Don Penney, Developer, Wind River _______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io<mailto:Starlingx-discuss@lists.starlingx.io> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
Marcela got this script merges that is looking for dependencies on added packages , it is now merged in stx-tools https://git.openstack.org/cgit/openstack/stx-tools/commit/?id=b2a81a46128c06... Regards Cesar Lara From: Penney, Don [mailto:Don.Penney@windriver.com] Sent: Monday, July 23, 2018 9:05 AM To: Chen, Yan <yan.chen@intel.com>; Xie, Cindy <cindy.xie@intel.com>; Hu, Yong <yong.hu@intel.com>; Jones, Bruce E <bruce.e.jones@intel.com>; Little, Scott <Scott.Little@windriver.com>; starlingx-discuss@lists.starlingx.io Subject: Re: [Starlingx-discuss] StarlingX.repo using CentOS 7.3 binary repos instead of 7.4? The script I sent in another thread (Adding packages to build .lst files) uses repoquery to recursively determine dependencies of packages, with commands like: repoquery --quiet -c $REPOCFG --resolve --requires --recursive From: Chen, Yan [mailto:yan.chen@intel.com] Sent: Monday, July 23, 2018 1:54 AM To: Xie, Cindy; Hu, Yong; Jones, Bruce E; Little, Scott; starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Subject: Re: [Starlingx-discuss] StarlingX.repo using CentOS 7.3 binary repos instead of 7.4? I guess so, currently we already get some information from the rpms. We will review the data first and I will improve the tool to get more. Yan From: Xie, Cindy [mailto:cindy.xie@intel.com] Sent: Monday, July 23, 2018 11:55 To: Hu, Yong <yong.hu@intel.com<mailto:yong.hu@intel.com>>; Jones, Bruce E <bruce.e.jones@intel.com<mailto:bruce.e.jones@intel.com>>; Scott Little <scott.little@windriver.com<mailto:scott.little@windriver.com>>; starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Subject: Re: [Starlingx-discuss] StarlingX.repo using CentOS 7.3 binary repos instead of 7.4? So I guess Chen Yan’s scripts can solve the dependencies of 2), right? From: Hu, Yong [mailto:yong.hu@intel.com] Sent: Monday, July 23, 2018 9:07 AM To: Jones, Bruce E <bruce.e.jones@intel.com<mailto:bruce.e.jones@intel.com>>; Scott Little <scott.little@windriver.com<mailto:scott.little@windriver.com>>; starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Subject: Re: [Starlingx-discuss] StarlingX.repo using CentOS 7.3 binary repos instead of 7.4? We have 2 types of dependencies to handle: 1). Build dependencies (indicated by “build requires”) and 2) deployment (installation) dependencies (indicated by “requires”). If we are aware of a new package is needed, this package name could be added into *.lst, and accordingly the package itself can be managed to download and added into the mirror. But the *indirect* dependencies brought into by this new package couldn’t be explicitly figured out until the build failures, going along with the building iterations. It would be helpful to have a script to analyze the dependency chain, BEFORE actually trying the build iterations painfully. Would love to see how Clear Linux project efficiently resolve this problem. Regards, Yong From: "Jones, Bruce E" <bruce.e.jones@intel.com<mailto:bruce.e.jones@intel.com>> Date: Friday, 20 July 2018 at 11:59 PM To: Scott Little <scott.little@windriver.com<mailto:scott.little@windriver.com>>, "Hu, Yong" <yong.hu@intel.com<mailto:yong.hu@intel.com>>, "starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>" <starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>> Subject: RE: [Starlingx-discuss] StarlingX.repo using CentOS 7.3 binary repos instead of 7.4? Managing the rpm dependency tree in an automated way is one of the key things I’m hoping we can learn from (leverage) from the Clear Linux project. They seem to have solved this problem. Several ex-Clear team members are on our team now and thinking about this… brucej From: Scott Little [mailto:scott.little@windriver.com] Sent: Friday, July 20, 2018 6:48 AM To: Hu, Yong <yong.hu@intel.com<mailto:yong.hu@intel.com>>; starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Subject: Re: [Starlingx-discuss] StarlingX.repo using CentOS 7.3 binary repos instead of 7.4? erlang*18.3.4.4*rpm are the ones to use. The R16B version should have been excluded. Yes, our mirror was of all ~30000 rpms. In this way, when a designer adds a new package or dependency, it and it's recursive dependencies were already present and ready for use. The method that has been concocted for StarlingX requires adding packages to .lst files every time there is a change. Further there doesn't seem to be a tool to assist in the search for missing dependencies. The result is likely to be several iterations of ... add package, compile, fail due to missing dependency, add package, compile, fail due to missing dependency ... very time consuming and frustrating. Scott On 18-07-19 09:00 PM, Hu, Yong wrote: Hi Scott, I noticed in the black_list, there are a couple of RPMs (such as erlang*-R16B*.rpm”), which were in tis-r4-CentOS mirror but were NOT used since tis-r5-CentOS. As well, we are certain they are not needed for building R5 or current StarlingX codebase. Could we just drop them? BTW: in tis-r5-CentOS mirror, there are over 30,000 RPMs, while actually among those only ~1900 RPMs are necessary. regards, Yong From: Scott Little <scott.little@windriver.com><mailto:scott.little@windriver.com> Date: Wednesday, 18 July 2018 at 11:17 PM To: "starlingx-discuss@lists.starlingx.io"<mailto:starlingx-discuss@lists.starlingx.io> <starlingx-discuss@lists.starlingx.io><mailto:starlingx-discuss@lists.starlingx.io> Subject: Re: [Starlingx-discuss] StarlingX.repo using CentOS 7.3 binary repos instead of 7.4? StarlingX should be based on Centos 7.4 for the vast majority of packages. I think there are a dozen that have made the move to 7.5. There are also a few that we blacklisted and are pulling from earlier releases. I'll attach my black_list notes as they exist today. I'll have to audit them to make sure my notes are up to date. Scott On 18-07-18 10:47 AM, Penney, Don wrote: Hi all, The StarlingX.repo file in stx-tools has all the CentOS 7.4.1708 binary repos disabled and is instead using 7.3.1611. This would likely explain why some of the specified RPMs cannot be downloaded via yum, as they were upversioned in 7.4. Was this done for a particular reason, or is it a configuration error? Don Penney, Developer, Wind River _______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io<mailto:Starlingx-discuss@lists.starlingx.io> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
This could help to start an analysis on the rpms, I created a spreadsheet detailing the package, version and from which repository is being downloaded. There's a lot that comes from centos 7.2 and that could be because that some old package from StarlingX requires these versions. Here is the spreadsheet: https://ethercalc.openstack.org/48zp05mejrly -Erich ________________________________________ From: Lara, Cesar [cesar.lara@intel.com] Sent: Monday, July 23, 2018 11:14 AM To: Penney, Don; Chen, Yan; Xie, Cindy; Hu, Yong; Jones, Bruce E; Little, Scott; starlingx-discuss@lists.starlingx.io Subject: Re: [Starlingx-discuss] StarlingX.repo using CentOS 7.3 binary repos instead of 7.4? Marcela got this script merges that is looking for dependencies on added packages , it is now merged in stx-tools https://git.openstack.org/cgit/openstack/stx-tools/commit/?id=b2a81a46128c06... Regards Cesar Lara From: Penney, Don [mailto:Don.Penney@windriver.com] Sent: Monday, July 23, 2018 9:05 AM To: Chen, Yan <yan.chen@intel.com>; Xie, Cindy <cindy.xie@intel.com>; Hu, Yong <yong.hu@intel.com>; Jones, Bruce E <bruce.e.jones@intel.com>; Little, Scott <Scott.Little@windriver.com>; starlingx-discuss@lists.starlingx.io Subject: Re: [Starlingx-discuss] StarlingX.repo using CentOS 7.3 binary repos instead of 7.4? The script I sent in another thread (Adding packages to build .lst files) uses repoquery to recursively determine dependencies of packages, with commands like: repoquery --quiet -c $REPOCFG --resolve --requires --recursive From: Chen, Yan [mailto:yan.chen@intel.com] Sent: Monday, July 23, 2018 1:54 AM To: Xie, Cindy; Hu, Yong; Jones, Bruce E; Little, Scott; starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Subject: Re: [Starlingx-discuss] StarlingX.repo using CentOS 7.3 binary repos instead of 7.4? I guess so, currently we already get some information from the rpms. We will review the data first and I will improve the tool to get more. Yan From: Xie, Cindy [mailto:cindy.xie@intel.com] Sent: Monday, July 23, 2018 11:55 To: Hu, Yong <yong.hu@intel.com<mailto:yong.hu@intel.com>>; Jones, Bruce E <bruce.e.jones@intel.com<mailto:bruce.e.jones@intel.com>>; Scott Little <scott.little@windriver.com<mailto:scott.little@windriver.com>>; starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Subject: Re: [Starlingx-discuss] StarlingX.repo using CentOS 7.3 binary repos instead of 7.4? So I guess Chen Yan’s scripts can solve the dependencies of 2), right? From: Hu, Yong [mailto:yong.hu@intel.com] Sent: Monday, July 23, 2018 9:07 AM To: Jones, Bruce E <bruce.e.jones@intel.com<mailto:bruce.e.jones@intel.com>>; Scott Little <scott.little@windriver.com<mailto:scott.little@windriver.com>>; starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Subject: Re: [Starlingx-discuss] StarlingX.repo using CentOS 7.3 binary repos instead of 7.4? We have 2 types of dependencies to handle: 1). Build dependencies (indicated by “build requires”) and 2) deployment (installation) dependencies (indicated by “requires”). If we are aware of a new package is needed, this package name could be added into *.lst, and accordingly the package itself can be managed to download and added into the mirror. But the *indirect* dependencies brought into by this new package couldn’t be explicitly figured out until the build failures, going along with the building iterations. It would be helpful to have a script to analyze the dependency chain, BEFORE actually trying the build iterations painfully. Would love to see how Clear Linux project efficiently resolve this problem. Regards, Yong From: "Jones, Bruce E" <bruce.e.jones@intel.com<mailto:bruce.e.jones@intel.com>> Date: Friday, 20 July 2018 at 11:59 PM To: Scott Little <scott.little@windriver.com<mailto:scott.little@windriver.com>>, "Hu, Yong" <yong.hu@intel.com<mailto:yong.hu@intel.com>>, "starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>" <starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io>> Subject: RE: [Starlingx-discuss] StarlingX.repo using CentOS 7.3 binary repos instead of 7.4? Managing the rpm dependency tree in an automated way is one of the key things I’m hoping we can learn from (leverage) from the Clear Linux project. They seem to have solved this problem. Several ex-Clear team members are on our team now and thinking about this… brucej From: Scott Little [mailto:scott.little@windriver.com] Sent: Friday, July 20, 2018 6:48 AM To: Hu, Yong <yong.hu@intel.com<mailto:yong.hu@intel.com>>; starlingx-discuss@lists.starlingx.io<mailto:starlingx-discuss@lists.starlingx.io> Subject: Re: [Starlingx-discuss] StarlingX.repo using CentOS 7.3 binary repos instead of 7.4? erlang*18.3.4.4*rpm are the ones to use. The R16B version should have been excluded. Yes, our mirror was of all ~30000 rpms. In this way, when a designer adds a new package or dependency, it and it's recursive dependencies were already present and ready for use. The method that has been concocted for StarlingX requires adding packages to .lst files every time there is a change. Further there doesn't seem to be a tool to assist in the search for missing dependencies. The result is likely to be several iterations of ... add package, compile, fail due to missing dependency, add package, compile, fail due to missing dependency ... very time consuming and frustrating. Scott On 18-07-19 09:00 PM, Hu, Yong wrote: Hi Scott, I noticed in the black_list, there are a couple of RPMs (such as erlang*-R16B*.rpm”), which were in tis-r4-CentOS mirror but were NOT used since tis-r5-CentOS. As well, we are certain they are not needed for building R5 or current StarlingX codebase. Could we just drop them? BTW: in tis-r5-CentOS mirror, there are over 30,000 RPMs, while actually among those only ~1900 RPMs are necessary. regards, Yong From: Scott Little <scott.little@windriver.com><mailto:scott.little@windriver.com> Date: Wednesday, 18 July 2018 at 11:17 PM To: "starlingx-discuss@lists.starlingx.io"<mailto:starlingx-discuss@lists.starlingx.io> <starlingx-discuss@lists.starlingx.io><mailto:starlingx-discuss@lists.starlingx.io> Subject: Re: [Starlingx-discuss] StarlingX.repo using CentOS 7.3 binary repos instead of 7.4? StarlingX should be based on Centos 7.4 for the vast majority of packages. I think there are a dozen that have made the move to 7.5. There are also a few that we blacklisted and are pulling from earlier releases. I'll attach my black_list notes as they exist today. I'll have to audit them to make sure my notes are up to date. Scott On 18-07-18 10:47 AM, Penney, Don wrote: Hi all, The StarlingX.repo file in stx-tools has all the CentOS 7.4.1708 binary repos disabled and is instead using 7.3.1611. This would likely explain why some of the specified RPMs cannot be downloaded via yum, as they were upversioned in 7.4. Was this done for a particular reason, or is it a configuration error? Don Penney, Developer, Wind River _______________________________________________ Starlingx-discuss mailing list Starlingx-discuss@lists.starlingx.io<mailto:Starlingx-discuss@lists.starlingx.io> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
participants (10)
-
Chen, Yan
-
Cordoba Malibran, Erich
-
Hu, Yong
-
Jones, Bruce E
-
Lara, Cesar
-
McKenna, Jason
-
Penney, Don
-
Rowsell, Brent
-
Scott Little
-
Xie, Cindy