On 2021-07-31 10:04 a.m., Tao, Yue wrote:
Hi
Exactly, --download-version specify a version to download, but the restriction of uscan is the url in debian/watch must be a regex expression
$cat isc-dhcp-4.4.1/debian/watch
version=3
opts="uversionmangle=s/(rc|a|b|c)/~$1/,pgpsigurlmangle=s/$/.sha512.asc/" \
http://ftp.isc.org/isc/dhcp/(\d.\d.\d*)/dhcp-(\d.*)\.(?:tgz|tbz2|txz|tar\.(?:gz|bz2|xz)) <http://ftp.isc.org/isc/dhcp/(/d./d./d*)/dhcp-(/d.*)/.(?:tgz|tbz2|txz|tar\.(?:gz|bz2|xz))>
As I mentioned before, some urls of 3^rd part packages can’t be converted to a regex expression, so I think uscan may not suitable for downloading 3^rd part packages. But uscan is a very powerful tool, probably we use it in other places.
Let's keep it in mind as we refine things. If it is possible to detect when it may not function as expected, or only allow its use when we are 100% sure it will function as expected it might help be a good assist to have in place. MarkA
FYI the special urls.
http://git.yoctoproject.org/cgit/cgit.cgi/linux-yocto/snapshot/linux-yocto-b... <http://git.yoctoproject.org/cgit/cgit.cgi/linux-yocto/snapshot/linux-yocto-b44437fb32fe50b5664afd12098d928e1aaee111.tar.bz2>
https://api.github.com/repos/ceph/jerasure/tarball/96c76b89d661c163f65a014b8... <https://api.github.com/repos/ceph/jerasure/tarball/96c76b89d661c163f65a014b8042c9354ccf7f31>
BTW, I attach a revision layout, which updates md5 checksum in dl_patch.
Thanks,
Ytao
*From:*Short, Charles <Charles.Short@windriver.com> *Sent:* Friday, July 30, 2021 9:07 PM *To:* Tao, Yue <Yue.Tao@windriver.com>; starlingx-discuss@lists.starlingx.io; Asselstine, Mark <Mark.Asselstine@windriver.com>; Bai, Haiqing <Haiqing.Bai@windriver.com>; Panech, Davlet <Davlet.Panech@windriver.com> *Subject:* RE: [Starlingx-discuss] [Debian Build]: Layout of the debian folder
Hi,
You can specify the version you wish to download, I did the following:
* apt-get source isc-dhcp * cd isc-dhcp-4.4.1 * uscan –download-version 4.3.5 *
Regards
chuck
*From:*Tao, Yue <Yue.Tao@windriver.com <mailto:Yue.Tao@windriver.com>> *Sent:* Friday, July 30, 2021 6:50 AM *To:* Tao, Yue <Yue.Tao@windriver.com <mailto:Yue.Tao@windriver.com>>; Short, Charles <Charles.Short@windriver.com <mailto:Charles.Short@windriver.com>>; starlingx-discuss@lists.starlingx.io <mailto:starlingx-discuss@lists.starlingx.io>; Asselstine, Mark <Mark.Asselstine@windriver.com <mailto:Mark.Asselstine@windriver.com>>; Bai, Haiqing <Haiqing.Bai@windriver.com <mailto:Haiqing.Bai@windriver.com>>; Panech, Davlet <Davlet.Panech@windriver.com <mailto:Davlet.Panech@windriver.com>> *Subject:* RE: [Starlingx-discuss] [Debian Build]: Layout of the debian folder
Hi Charles:
I did some investigation of debian/watch file, which is the input file for uscan command. It’s purpose is to detect if upstream has a newer release. I tried the uscan, which is limited to run with a full debian folder, not only a dedbian/watch. I also try to the option “—watchfile” to specify a watch file, but it still checks the debian/changelog file.
$uscan –watchfile path/watch
uscan die: Are you in the source code tree?
Cannot find readable debian/changelog anywhere! at
And the url in watch file must be a regex expression, for example,
Allow
http://ftp.isc.org/isc/dhcp/(\d.\d.\d*)/dhcp-(\d.*)\.(?:tgz|tbz2|txz|tar\.(?:gz|bz2|xz)) <http://ftp.isc.org/isc/dhcp/(/d./d./d*)/dhcp-(/d.*)/.(?:tgz|tbz2|txz|tar\.(?:gz|bz2|xz))>
Not allow
http://ftp.isc.org/isc/dhcp/4.4.2/dhcp-4.4.2.tar.gz <http://ftp.isc.org/isc/dhcp/4.4.2/dhcp-4.4.2.tar.gz>
A developer only care about a special version, but he has to build a regex expression, of cause uscan has –download-version to specify a version, but watch file can’t meet a unregular url, like https://api.github.com/repos/ceph/jerasure/tarball/96c76b89d661c163f65a014b8... <https://api.github.com/repos/ceph/jerasure/tarball/96c76b89d661c163f65a014b8042c9354ccf7f31>, which is a downloading url in Starlingx repositories. I think it’s better not to use the uscan+watch to download 3^rd part packages.
Thanks,
ytao
*From:*Tao, Yue <yue.tao@windriver.com <mailto:yue.tao@windriver.com>> *Sent:* Tuesday, July 27, 2021 10:53 AM *To:* Short, Charles <Charles.Short@windriver.com <mailto:Charles.Short@windriver.com>>; starlingx-discuss@lists.starlingx.io <mailto:starlingx-discuss@lists.starlingx.io>; Asselstine, Mark <Mark.Asselstine@windriver.com <mailto:Mark.Asselstine@windriver.com>>; Bai, Haiqing <Haiqing.Bai@windriver.com <mailto:Haiqing.Bai@windriver.com>>; Panech, Davlet <Davlet.Panech@windriver.com <mailto:Davlet.Panech@windriver.com>> *Subject:* Re: [Starlingx-discuss] [Debian Build]: Layout of the debian folder
On 7/27/21 7:05 AM, Short, Charles wrote:
Hi,
First of all, I'm appreciated for your input.
I had a look through the document and it looks okay. I just have a couple of comments/questions about the folder content:
* **debver** - This is the package version of the debian package? This is generated from the debian/changelog when the package is built. I guess the problem is that you want to differentiate from a starlingx package and native package? My suggestion would be what the Ubuntu developers do. For example, if we have patches to apply for systemd, the debian version would be 247.3-3, while the StarlingX package would be 247.3-3stx0. Then a user or developer can quickly see that they are working with a native debian package or a starlingx modified package.
For a Debian native package, the *debver* is the debian version. We use it to download the src via "apt-source packagname=debver", for a starlingx package, it is up to the developer. I observed in CentOS version, they added a patch to change the package version, for example integ/base/lighttpd/centos/meta_patches/Update-package-versioning-for-TIS-format.patch, which appends a 'tis' to package version, that requests developer to update the version manually. After transiting to debian, OBS can achieve it automatically. It can append 'tis' or 'stx' to a package, so don't need developers to update it manually.
* **deb_patches** - As a developer, if the package has its debian/patches I would rather use the debian/patches directory. If the directory doesnt exist I am more likely to create a debian/patches directory and then send a patch to debian to fix a problem.
Exactly, we should send a fix to debian community and then uprev the package to get the fix, but we also need to consider local changes. The *patches" folder contains the local change for source codes, that will be copied to debian/patches (or create it if doesn't exist), and update the debian/patches/series. The *deb_patches* contains the patches to change debian folder, for example, we want to customize the installation of a package, we need to update override_dh_install stage in debian/rules. The patches will be applied to package/debian folder directly.
The *deb_patches* is similar with centos/meta_patches
The *patches* is similar with centos/patches
* * **dl_path** - Debian already does this for you already, the debian/watch file keeps track of where to download a tarball, or a zip file, or a python wheel for you. You can use uscan(1), which is a part of devscripts package. Debian/Ubuntu developers have been doing this for years. (https://wiki.debian.org/debian/watch <https://wiki.debian.org/debian/watch>)
Thanks you for pointing me the debian/watch. I will take some time to learn about debian/watch and go back soon. It looks a debian file for monitoring the upstream version change. I am not sure if it is suitable for a no debian package. In our internal discussion, Davlet told me a special case, that the url of a tar ball likes this https://api.github.com/repos/ceph/jerasure/tarball/96c76b89d661c163f65a014b8... <https://api.github.com/repos/ceph/jerasure/tarball/96c76b89d661c163f65a014b8042c9354ccf7f31>
As Davlet's suggestion, the *dl_path* refers to example file <https://opendev.org/starlingx/integ/src/branch/master/centos_tarball-dl.lst> and the script that parses it <https://opendev.org/starlingx/tools/src/branch/master/centos-mirror-tools/dl_tarball.sh#L215>. I will do assessment for debian/watch.
Thanks,
ytao
If you have any questions please let me know.
Regards
chuck
debian/watch - Debian Wiki <https://wiki.debian.org/debian/watch>
debian/watch. The file named watch in the debian directory is used to check for newer versions of upstream software is available and to download it if necessary. The download itself will be performed with the uscan program from the devscripts package. It takes the path to the debian directory that uses the watch file as an argument or searches the directories underneath the current working ...
wiki.debian.org
------------------------------------------------------------------------
*From:* ytao <yue.tao@windriver.com> <mailto:yue.tao@windriver.com> *Sent:* July 26, 2021 3:33 AM *To:* starlingx-discuss@lists.starlingx.io <mailto:starlingx-discuss@lists.starlingx.io> <starlingx-discuss@lists.starlingx.io> <mailto:starlingx-discuss@lists.starlingx.io>; Asselstine, Mark <Mark.Asselstine@windriver.com> <mailto:Mark.Asselstine@windriver.com> *Subject:* [Starlingx-discuss] [Debian Build]: Layout of the debian folder
Hello Everyone:
You may have known Wind River team is working on the transition to Debian OS, more details of the project can be found at https://docs.starlingx.io/specs/specs/stx-6.0/ <https://docs.starlingx.io/specs/specs/stx-6.0/>. My purpose is launching a discussion about the userspace packages transition to Debian. The spec can be found at https://docs.starlingx.io/specs/specs/stx-6.0/approved/starlingx_2008704_deb... <https://docs.starlingx.io/specs/specs/stx-6.0/approved/starlingx_2008704_debian_transition.html>
In order to inherit the existing userspace construction as much as possible, our proposal is creating a 'debian' folder in same directory with 'centos' folder for each package. For example, the dhcp package in integ repo, it's centos folder is at:
https://opendev.org/starlingx/integ/src/branch/master/base/dhcp/centos <https://opendev.org/starlingx/integ/src/branch/master/base/dhcp/centos>
We will create a debian folder in same location
https://opendev.org/starlingx/integ/src/branch/master/base/dhcp/debian <https://opendev.org/starlingx/integ/src/branch/master/base/dhcp/debian>.
All materials in this folder control building dhcp under Debian host. The "stx_deb_folder_layout.rst" is the layout of debian folder. I also attach a couple of samples to demonstrate how to fill the debian folder for a debian package and a 3rd package.
This layout is not the final version, I'm appreciated for any suggestion from you.
Thanks,
ytao