[Starlingx-discuss] Slow repo sync times and the yocto kernel

Tue Jun 29 18:59:11 UTC 2021

Concerns have been raised with the security implications of the git 
protocol.  We really should return to https if at all possible.

Options:

1)  Use no-clone-bundle, as in

     repo sync -j20 --no-clone-bundle

However this hurts performance and transfers a significant load onto the 
git servers.

Slightly better would be ...

     repo sync -j20 || repo sync -j20 --no-clone-bundle

Where the first commands is efficient, but fails on yocto, followed by 
the second that handles yocto.

2) Return to downloading the full history of linux-yocto.  It eats an 
hour on the initial clone, but after that it's not too bad.

3) Set up a mirror of linux-yocto on a more powerful hosting site.   git 
hub?

4) Switch to tarball snapshots of linux-yocto.

Scott

On 2021-06-28 10:54 a.m., Scott Little wrote:
> I have posted a better fix for review ...
>
> https://review.opendev.org/c/starlingx/manifest/+/798340
>
> Scott
>
>
> On 2021-06-28 10:51 a.m., Scott Little wrote:
>> The manifest change to implement option 2 has been merged.  However a 
>> lot of folks are now having 'repo sync' issues with linux-yocto. 
>> Strangely it doesn't hit the first repo sync, but all subsequent repo 
>> sync's are affected.
>>
>> If you are seeing an error from repo sync, please use this work 
>> around ...
>>
>> repo init ...
>> rm -rf .repo/project-objects/linux-yocto.git* 
>> .repo/projects/cgcs-root/stx/git/linux-yocto-*
>> repo sync ...
>>
>> I've reported the bug upsteam... 
>> https://bugs.chromium.org/p/gerrit/issues/detail?id=14700&q=tyranscooter&can=2 
>> ... if you have a gmail account and are willing to do so, add a star 
>> to increase visibility of this strange error.
>>
>> I'll also be looking for a better fix on our end.
>>
>> Scott
>>
>>
>>
>> On 2021-06-21 4:10 p.m., Scott Little wrote:
>>> Hi all
>>>
>>> The yocto kernel git was added to the StarlingX manifests late last 
>>> week.  Since then I've heard a lot of grumbling about slow 'repo 
>>> sync' times.  It affects folks setting up a new distro or monolithic 
>>> workspace for the first time.  The repo-sync time can exceed an hour 
>>> as the entire history of the linux kernel is downloaded.  You will 
>>> also notice an additional 5.5 GB of storage consumed to hold all 
>>> this history.  Subsequent repo sync's should be fast.
>>>
>>> So the question is... what if anything do we do about it?
>>>
>>> Our options...
>>>
>>> 1) Leave it as is.
>>>
>>> Hope that folks are mostly working in the 'flock' or 'container' 
>>> layers, and NOT using monolithic builds, and so the number of folk 
>>> impacted is low.   Folk working at the distro layer or using 
>>> monolithic can work on something else, or go to lunch, while they 
>>> wait for the initial repo sync to complete.
>>>
>>>
>>> 2) Try to minimize the amount of kernel history we download through 
>>> a manifest change.  Limiting the git history depth does the trick ...
>>>
>>>    <!-- linux-yocto Upstream -->
>>>   <project remote="linux-yocto"    sync-c="true" sync-tags="false" 
>>> clone-depth="100" upstream="v5.10/standard/intel-x86" 
>>> revision="refs/tags/v5.10.30" name="linux-yocto" 
>>> path="cgcs-root/stx/git/linux-yocto-std"/>
>>>   <project remote="linux-yocto"    sync-c="true" sync-tags="false" 
>>> clone-depth="100" upstream="v5.10/standard/preempt-rt/intel-x86" 
>>> revision="2112f10d3d0b558c9ece3ab562c41b7f6d89cff4" 
>>> name="linux-yocto.git" path="cgcs-root/stx/git/linux-yocto-rt"/>
>>>
>>> The good ...
>>>
>>>  - repo sync time drops from ~1 hr to ~5 min
>>>
>>>  - storage drops from ~5.5 GB to ~ 5GB
>>>
>>> The bad ...
>>>
>>> - This is fragile.  It assumes that the desired rt sha can be 
>>> reached from 100 commits from head of branch.  However, the 
>>> connection to the upstream git server drops if we ask for much more 
>>> than that. e.g. depth=500 is a guaranteed fail.  So upstream adds a 
>>> few patches and we might start failing our repo-sync.
>>>
>>> - The history is incomplete, this may hinder kernel developers. A 
>>> 'git fetch linux-yocto' should pull in the rest of the history, so 
>>> probably not a blocker.
>>>
>>>
>>> 3) We could double the number of manifests at each layer. One would 
>>> only pull in the minimal kernel history, and other the full history.
>>>
>>>
>>> 4) Create a mirror an a larger git server, like github, and hope 
>>> that significantly improves the download speed.
>>>
>>>
>>> 5) Download a tarball of the yocto kernel, rather than pulling in 
>>> it's git tree.  Yocto's git server doesn't seem to be set up to 
>>> serve custom tarballs based on a requested sha.  We would have to 
>>> set it all up manually, and it's not remotely convenient to kernel 
>>> developers.
>>>
>>>
>>> Of these options, I'm leaning toward option 2, but look forward to 
>>> hearing from the community.
>>>
>>>
>>> Scott
>>>
>>>
>>> _______________________________________________
>>> Starlingx-discuss mailing list
>>> Starlingx-discuss at lists.starlingx.io
>>> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
>>
>>
>>
>> _______________________________________________
>> Starlingx-discuss mailing list
>> Starlingx-discuss at lists.starlingx.io
>> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss
>
>
>
> _______________________________________________
> Starlingx-discuss mailing list
> Starlingx-discuss at lists.starlingx.io
> http://lists.starlingx.io/cgi-bin/mailman/listinfo/starlingx-discuss