[Starlingx-discuss] how to deal with loop dependent in building stage

[Zhang, Xiao]

Comments inline:

...

> For centos, we found that loops almost always depend on one of a 
> handful of low level packages, e.g. bash, python, gcc, rpm
>>>
>>> We had two ways to try and deal with this.
>>>
>>> 1) Allow the use of a pre-compiled binary from upstream to satisfy 
>>> the dependency when the StarlingX modification are unlikely to 
>>> affect how dependent packages compile.  Packages in the 'mock' lst 
>>> files could satisfy this type of requirement.
>>> e.g. A=bash
>>> So in your example, compile order would be:
>>>     F (vs upstream A), D (vs upstream A), C, B, A
>>>
>>> 2) Use a different 'build-type'... other than 'std' or 'rt' ... when 
>>> the modified StarlingX package was likely to affect the output of 
>>> dependent packages.  Compile packages in that build type first.
>>> e.g. A=rpm
>>>  = build-type 'installer' compile order would be:
>>>         A (vs upstream binaries) B and C ... call this A-intermediate
>> ? So, the 'installer' type of A doesn't depend on B and C, right? 
>> Just like the official method to break the loop manually?
>
> [SL] I wouldn't say that A doesn't depend on B or C.   Rather the 
> 'installer' package set does not build B or C, so they do not factor 
> into build order calculations. This does require that upstream B and C 
> be available to satisfy build dependencies when compiling within the 
> 'installer' package set. When compiling the 'std' set, but the 
> packages from the 'installer' package must be available to satisfy 
> dependencies. The 'installer' compiled versions of B and C must be of 
> higher version that those from upstream... and thus installed instead 
> of the upstream ones.
>
So? In this kind of case, we also needn't break the loop manually but 
just separate the into different layers/sets and compile them in 
different stage, right?

In the bottom logic(or, with the compiler's point of view), it is the 
same with the first one. The key difference comes from the special 
package A. It's so basic that it has to be compiled tice

>>>  = build-type 'std' compile order would be:
>>>         F (vs A-intermediate), D (vs A-intermediate), C, B, A
>>>
>> Seems the first one is more fit for automatic build. Then, any 
>> special cases thus we have to use the second method? Or we can always 
>> use the first one?
>>
> [SL] You'll need to support both methods of resolving dependency cycles.
>
> Note that the layered build concept of the CentOS build was another 
> way to address partitioning of packages into sets that might help with 
> breaking dependency loops by not having A,B,C in the same compile set. 
> Look to the 'compiler' layer.
>
> I think you need to support at least one of the 'build-type' or 'build 
> layer' concepts to begin with. Bonus points if you support both.  
> Supporting both was needed for the 'rpm' package which we wanted to 
> patch, and the patch does alter the format of the rpms generated, and 
> the rpm package itself had to be in that format.  So rpm was compiled 
> for build-type 'installer' of layer 'compiler' (new rpm code in old 
> rpm package), then recompiled in build-type 'std' of layer 'compiler' 
> (new rpm code in new rpm package).  Finally the std build of rpm was 
> published for use by higher layer builds.
>
> Only designers working on packages in the 'installer' set, or the 
> 'compiler' layer, need to worry about the multi pass build.  For most 
> packages, a single pass 'flock' layer, 'std' build-type, build is fine.
>
So, in build process, we need to build "compiler" layer firstly, then 
use the new build "compilers" build other layers. Some special packages 
like "rpm" may exist in more than one layers/sets.


BTW: I suppose the upstream rpm is already powerful enough, why we have 
to modify it? Just bug fix or new features? If the later one, StarlingX 
need some special features it hasn't? It maybe very useful/important for 
us in porting StarlingX onto other Linux releases.


Thanks

Xiao


> Scott
>
>
>>
>> Thanks
>>
>> Xiao
>>
>>> Scott
>>>
>>>
>>> On 2021-09-18 2:50 a.m., Zhang, Xiao wrote:
>>>>  Hi，
>>>>
>>>>
>>>> We are trying to construct basic environment for porting starlingX 
>>>> on Debian. While when I dealing with the build order of user space 
>>>> packages, the loop dependent problem blocked me.
>>>>
>>>> The easiest example: source package A build depend on B while B is 
>>>> also build depend on A. We can just build A, B, A, B and only use 
>>>> the later result.
>>>>
>>>> A fairly complex example: A depends on B and C, B depends on D, C 
>>>> depends on F, D depends on A and F. In this case there will be 
>>>> three cycles as below:
>>>>
>>>> A->B->D->A, A->B->D->F->A, A->C->F->A . Even more, if in some cases 
>>>> we needn't B or D, then we have only one cycle: ACFA
>>>>
>>>> I tried to find a method to deal with it but failed.
>>>>
>>>> So I wonder how did we deal with such loop dependent before, on 
>>>> CentOS. Any advises about it?
>>>>
>>>>
>>>> Thanks a lot
>>>>
>>>> Xiao
>>>>
>>>>
>>>> _______________________________________________
>>>> Starlingx-discuss mailing list
>>>> Starlingx-discuss at lists.starlingx.io
>>>> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
>>>
>>>
>>>
>>> _______________________________________________
>>> Starlingx-discuss mailing list
>>> Starlingx-discuss at lists.starlingx.io
>>> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
>>
>> _______________________________________________
>> Starlingx-discuss mailing list
>> Starlingx-discuss at lists.starlingx.io
>> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
>
>
>
> _______________________________________________
> Starlingx-discuss mailing list
> Starlingx-discuss at lists.starlingx.io
> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.starlingx.io/pipermail/starlingx-discuss/attachments/20210923/0d517aa1/attachment.htm>