[Starlingx-discuss] how to deal with loop dependent in building stage

[Scott Little]

Inline comments in blue

Scott

On 2021-09-21 9:07 a.m., Zhang, Xiao wrote:
>
> On 9/20/2021 9:39 PM, Scott Little wrote:
>> For centos, we found that loops almost always depend on one of a 
>> handful of low level packages, e.g. bash, python, gcc, rpm
>>
>> We had two ways to try and deal with this.
>>
>> 1) Allow the use of a pre-compiled binary from upstream to satisfy 
>> the dependency when the StarlingX modification are unlikely to affect 
>> how dependent packages compile.  Packages in the 'mock' lst files 
>> could satisfy this type of requirement.
>> e.g. A=bash
>> So in your example, compile order would be:
>>     F (vs upstream A), D (vs upstream A), C, B, A
>>
>> 2) Use a different 'build-type'... other than 'std' or 'rt' ... when 
>> the modified StarlingX package was likely to affect the output of 
>> dependent packages.  Compile packages in that build type first.
>> e.g. A=rpm
>>  = build-type 'installer' compile order would be:
>>         A (vs upstream binaries) B and C ... call this A-intermediate
> ? So, the 'installer' type of A doesn't depend on B and C, right? Just 
> like the official method to break the loop manually?

[SL] I wouldn't say that A doesn't depend on B or C.   Rather the 
'installer' package set does not build B or C, so they do not factor 
into build order calculations.  This does require that upstream B and C 
be available to satisfy build dependencies when compiling within the 
'installer' package set. When compiling the 'std' set, but the packages 
from the 'installer' package must be available to satisfy dependencies. 
The 'installer' compiled versions of B and C must be of higher version 
that those from upstream... and thus installed instead of the upstream ones.

>>  = build-type 'std' compile order would be:
>>         F (vs A-intermediate), D (vs A-intermediate), C, B, A
>>
> Seems the first one is more fit for automatic build. Then, any special 
> cases thus we have to use the second method? Or we can always use the 
> first one?
>
[SL] You'll need to support both methods of resolving dependency cycles.

Note that the layered build concept of the CentOS build was another way 
to address partitioning of packages into sets that might help with 
breaking dependency loops by not having A,B,C in the same compile set. 
Look to the 'compiler' layer.

I think you need to support at least one of the 'build-type' or 'build 
layer' concepts to begin with.  Bonus points if you support both.  
Supporting both was needed for the 'rpm' package which we wanted to 
patch, and the patch does alter the format of the rpms generated, and 
the rpm package itself had to be in that format.  So rpm was compiled 
for build-type 'installer' of layer 'compiler' (new rpm code in old rpm 
package), then recompiled in build-type 'std' of layer 'compiler' (new 
rpm code in new rpm package). Finally the std build of rpm was published 
for use by higher layer builds.

Only designers working on packages in the 'installer' set, or the 
'compiler' layer, need to worry about the multi pass build.  For most 
packages, a single pass 'flock' layer, 'std' build-type, build is fine.

Scott


>
> Thanks
>
> Xiao
>
>> Scott
>>
>>
>> On 2021-09-18 2:50 a.m., Zhang, Xiao wrote:
>>>  Hi，
>>>
>>>
>>> We are trying to construct basic environment for porting starlingX 
>>> on Debian. While when I dealing with the build order of user space 
>>> packages, the loop dependent problem blocked me.
>>>
>>> The easiest example: source package A build depend on B while B is 
>>> also build depend on A. We can just build A, B, A, B and only use 
>>> the later result.
>>>
>>> A fairly complex example: A depends on B and C, B depends on D, C 
>>> depends on F, D depends on A and F. In this case there will be three 
>>> cycles as below:
>>>
>>> A->B->D->A, A->B->D->F->A, A->C->F->A . Even more, if in some cases 
>>> we needn't B or D, then we have only one cycle: ACFA
>>>
>>> I tried to find a method to deal with it but failed.
>>>
>>> So I wonder how did we deal with such loop dependent before, on 
>>> CentOS. Any advises about it?
>>>
>>>
>>> Thanks a lot
>>>
>>> Xiao
>>>
>>>
>>> _______________________________________________
>>> Starlingx-discuss mailing list
>>> Starlingx-discuss at lists.starlingx.io
>>> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
>>
>>
>>
>> _______________________________________________
>> Starlingx-discuss mailing list
>> Starlingx-discuss at lists.starlingx.io
>> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
>
> _______________________________________________
> Starlingx-discuss mailing list
> Starlingx-discuss at lists.starlingx.io
> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.starlingx.io/pipermail/starlingx-discuss/attachments/20210921/5255d62a/attachment.htm>