[Starlingx-discuss] how to deal with loop dependent in building stage

[Scott Little]

There was a security feature (not sure it's name or if it's still 
supported) that required a new rpm feature not available in the standard 
CentOS rpm (file signatures).  The behavior of file signing was further 
modified by 0001-sign-files-only.patch

Greg, can you comment on this?  Do the Debian packaging tools need to 
support file signatures?

Scott


On 2021-09-23 4:56 a.m., Zhang, Xiao wrote:
>
> Comments inline:
>
> ...
>
>> For centos, we found that loops almost always depend on one of a 
>> handful of low level packages, e.g. bash, python, gcc, rpm
>>>>
>>>> We had two ways to try and deal with this.
>>>>
>>>> 1) Allow the use of a pre-compiled binary from upstream to satisfy 
>>>> the dependency when the StarlingX modification are unlikely to 
>>>> affect how dependent packages compile.  Packages in the 'mock' lst 
>>>> files could satisfy this type of requirement.
>>>> e.g. A=bash
>>>> So in your example, compile order would be:
>>>>     F (vs upstream A), D (vs upstream A), C, B, A
>>>>
>>>> 2) Use a different 'build-type'... other than 'std' or 'rt' ... 
>>>> when the modified StarlingX package was likely to affect the output 
>>>> of dependent packages.  Compile packages in that build type first.
>>>> e.g. A=rpm
>>>>  = build-type 'installer' compile order would be:
>>>>         A (vs upstream binaries) B and C ... call this A-intermediate
>>> ? So, the 'installer' type of A doesn't depend on B and C, right? 
>>> Just like the official method to break the loop manually?
>>
>> [SL] I wouldn't say that A doesn't depend on B or C.   Rather the 
>> 'installer' package set does not build B or C, so they do not factor 
>> into build order calculations.  This does require that upstream B and 
>> C be available to satisfy build dependencies when compiling within 
>> the 'installer' package set. When compiling the 'std' set, but the 
>> packages from the 'installer' package must be available to satisfy 
>> dependencies. The 'installer' compiled versions of B and C must be of 
>> higher version that those from upstream... and thus installed instead 
>> of the upstream ones.
>>
> So? In this kind of case, we also needn't break the loop manually but 
> just separate the into different layers/sets and compile them in 
> different stage, right?
>
> In the bottom logic(or, with the compiler's point of view), it is the 
> same with the first one. The key difference comes from the special 
> package A. It's so basic that it has to be compiled tice
>
>>>>  = build-type 'std' compile order would be:
>>>>         F (vs A-intermediate), D (vs A-intermediate), C, B, A
>>>>
>>> Seems the first one is more fit for automatic build. Then, any 
>>> special cases thus we have to use the second method? Or we can 
>>> always use the first one?
>>>
>> [SL] You'll need to support both methods of resolving dependency cycles.
>>
>> Note that the layered build concept of the CentOS build was another 
>> way to address partitioning of packages into sets that might help 
>> with breaking dependency loops by not having A,B,C in the same 
>> compile set. Look to the 'compiler' layer.
>>
>> I think you need to support at least one of the 'build-type' or 
>> 'build layer' concepts to begin with.  Bonus points if you support 
>> both.  Supporting both was needed for the 'rpm' package which we 
>> wanted to patch, and the patch does alter the format of the rpms 
>> generated, and the rpm package itself had to be in that format.  So 
>> rpm was compiled for build-type 'installer' of layer 'compiler' (new 
>> rpm code in old rpm package), then recompiled in build-type 'std' of 
>> layer 'compiler' (new rpm code in new rpm package).  Finally the std 
>> build of rpm was published for use by higher layer builds.
>>
>> Only designers working on packages in the 'installer' set, or the 
>> 'compiler' layer, need to worry about the multi pass build.  For most 
>> packages, a single pass 'flock' layer, 'std' build-type, build is fine.
>>
> So, in build process, we need to build "compiler" layer firstly, then 
> use the new build "compilers" build other layers. Some special 
> packages like "rpm" may exist in more than one layers/sets.
>
>
> BTW: I suppose the upstream rpm is already powerful enough, why we 
> have to modify it? Just bug fix or new features? If the later one, 
> StarlingX need some special features it hasn't? It maybe very 
> useful/important for us in porting StarlingX onto other Linux releases.
>
>
> Thanks
>
> Xiao
>
>
>> Scott
>>
>>
>>>
>>> Thanks
>>>
>>> Xiao
>>>
>>>> Scott
>>>>
>>>>
>>>> On 2021-09-18 2:50 a.m., Zhang, Xiao wrote:
>>>>>  Hi，
>>>>>
>>>>>
>>>>> We are trying to construct basic environment for porting starlingX 
>>>>> on Debian. While when I dealing with the build order of user space 
>>>>> packages, the loop dependent problem blocked me.
>>>>>
>>>>> The easiest example: source package A build depend on B while B is 
>>>>> also build depend on A. We can just build A, B, A, B and only use 
>>>>> the later result.
>>>>>
>>>>> A fairly complex example: A depends on B and C, B depends on D, C 
>>>>> depends on F, D depends on A and F. In this case there will be 
>>>>> three cycles as below:
>>>>>
>>>>> A->B->D->A, A->B->D->F->A, A->C->F->A . Even more, if in some 
>>>>> cases we needn't B or D, then we have only one cycle: ACFA
>>>>>
>>>>> I tried to find a method to deal with it but failed.
>>>>>
>>>>> So I wonder how did we deal with such loop dependent before, on 
>>>>> CentOS. Any advises about it?
>>>>>
>>>>>
>>>>> Thanks a lot
>>>>>
>>>>> Xiao
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Starlingx-discuss mailing list
>>>>> Starlingx-discuss at lists.starlingx.io
>>>>> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Starlingx-discuss mailing list
>>>> Starlingx-discuss at lists.starlingx.io
>>>> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
>>>
>>> _______________________________________________
>>> Starlingx-discuss mailing list
>>> Starlingx-discuss at lists.starlingx.io
>>> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
>>
>>
>>
>> _______________________________________________
>> Starlingx-discuss mailing list
>> Starlingx-discuss at lists.starlingx.io
>> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
>
> _______________________________________________
> Starlingx-discuss mailing list
> Starlingx-discuss at lists.starlingx.io
> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.starlingx.io/pipermail/starlingx-discuss/attachments/20210923/6681d091/attachment-0001.htm>