From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6816610675337461760 Date: Wed, 22 Apr 2020 03:11:11 -0700 (PDT) From: vijai kumar To: isar-users Message-Id: In-Reply-To: <20200422091539.76f52aaf@md1za8fc.ad001.siemens.net> References: <20200417093040.15130-1-Vijaikumar_Kanagarajan@mentor.com> <20200417093040.15130-5-Vijaikumar_Kanagarajan@mentor.com> <20200422091539.76f52aaf@md1za8fc.ad001.siemens.net> Subject: Re: [PATCH v5 04/13] deb-dl-dir: Make debsrc_download faster MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_Part_2875_1240019294.1587550271505" X-Google-Token: EL-wgPUFngHXH85jSD40 X-Google-IP: 192.94.34.34 X-TUID: RRUZXP7VzV+R ------=_Part_2875_1240019294.1587550271505 Content-Type: multipart/alternative; boundary="----=_Part_2876_709519840.1587550271506" ------=_Part_2876_709519840.1587550271506 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit On Wednesday, April 22, 2020 at 12:45:42 PM UTC+5:30, Henning Schild wrote: > > On Fri, 17 Apr 2020 15:00:31 +0530 > Vijai Kumar K > wrote: > > > Eventhough apt-get source skips redownloading of files, it is still > > slow and takes a lot of time. Instead, lookup if the dsc file is > > already present in the cache and skip based on it. > > > > Signed-off-by: Vijai Kumar K > > > --- > > meta/classes/deb-dl-dir.bbclass | 3 +++ > > 1 file changed, 3 insertions(+) > > > > diff --git a/meta/classes/deb-dl-dir.bbclass > > b/meta/classes/deb-dl-dir.bbclass index 472b9fe..9399741 100644 > > --- a/meta/classes/deb-dl-dir.bbclass > > +++ b/meta/classes/deb-dl-dir.bbclass > > @@ -32,6 +32,9 @@ EOSUDO > > if [ -z "${src}" ];then > > src="$( dpkg-deb --show --showformat '${Package}' > > "${package}" )" fi > > + # Strip epoch, if any, from version. > > + local dscfile=$(find "${DEBSRCDIR}"/"${rootfs_distro}" -name > > "${src}_${version#*:}.dsc") > > Is that reliable, or might you end up exiting too early. You process > that "version" quite a bit, it needs to remain a unique identifier. So > you do not skip "foo-1.2.3_r1@deb~42" only because "foo-1.2.3" was > there. I made that up but such a version is allowed to have all sorts > of funny chars. > As per the Debian policy manual, version format is [epoch:]upstream_version[-debian_revision]. While downloading using apt-get source =, I observed that epoch is needed for downloading the source correctly. However the downloaded dsc file does not contain epoch. Hence this stripping while checking for the presence of the corresponding dsc file. Also, Debian policy manual[1] states that . + - ~ (full stop, plus, hyphen, tilde) are the valid values for use in upstream-version part and + . ~ (plus, full stop, tilde) for debian-revision part. So, this epoch stripping should be fine. [1] https://www.debian.org/doc/debian-policy/ch-controlfields.html#s-f-version > > + [ -z "$dscfile" ] || continue > > I personally would say "-n" && continue, might be a matter of taste. > Henning > > > sudo -E chroot --userspec=$( id -u ):$( id -g ) ${rootfs} \ > > sh -c ' mkdir -p "/deb-src/${1}/${2}" && cd > > "/deb-src/${1}/${2}" && apt-get -y --download-only --only-source > > source "$2"="$3" ' download-src "${rootfs_distro}" "${src}" > > "${version}" > > ------=_Part_2876_709519840.1587550271506 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable


On Wednesday, April 22, 2020 at 12:45:42 PM UTC+5:= 30, Henning Schild wrote:
On Fr= i, 17 Apr 2020 15:00:31 +0530
Vijai Kumar K <vijaikumar...@gmail.com> wrote:

> Eventhough apt-get source skips redownloading of files, it is stil= l
> slow and takes a lot of time. Instead, lookup if the dsc file is
> already present in the cache and skip based on it.
>=20
> Signed-off-by: Vijai Kumar K <Vijaikumar_...@mentor.com&g= t;
> ---
> =C2=A0meta/classes/deb-dl-dir.bbclass | 3 +++
> =C2=A01 file changed, 3 insertions(+)
>=20
> diff --git a/meta/classes/deb-dl-dir.bbclass
> b/meta/classes/deb-dl-dir.bbclass index 472b9fe..9399741 1006= 44
> --- a/meta/classes/deb-dl-dir.bbclass
> +++ b/meta/classes/deb-dl-dir.bbclass
> @@ -32,6 +32,9 @@ EOSUDO
> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if [ -z "${src}" ];the= n
> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0src=3D"$( dpk= g-deb --show --showformat '${Package}'
> "${package}" )" fi
> + =C2=A0 =C2=A0 =C2=A0 =C2=A0# Strip epoch, if any, from version.
> + =C2=A0 =C2=A0 =C2=A0 =C2=A0local dscfile=3D$(find "${DEBSRC= DIR}"/"${rootfs_distro}" -name
> "${src}_${version#*:}.dsc")

Is that reliable, or might you end up exiting too early. You process
that "version" quite a bit, it needs to remain a unique ident= ifier. So
you do not skip "foo-1.2.3_r1@deb~42" only because "foo-= 1.2.3" was
there. I made that up but such a version is allowed to have all sorts
of funny chars.

As per the Debian policy manual,=C2=A0= version format is=C2=A0[epoch:]upstream_version[-debian_revision].
While downloading using apt-= get source <source pkg>=3D<version>, I observed that epoch is n= eeded
for downloading the source correctly. However the downloade= d dsc file does not contain epoch. Hence
this stripping while che= cking for the presence of the corresponding dsc file.

<= div>Also, Debian policy manual[1] states that .=C2=A0<= span class=3D"pre" style=3D"hyphens: none;">+= =C2=A0-=C2=A0~=C2=A0(full stop, plus, hyphen, tilde) are the va= lid values
for use in upstream-version par= t and=C2=A0+=C2=A0.= =C2=A0~=C2=A0(plus, full stop, tilde) for debian-revision part.
<= div>
So, this epoch strippi= ng should be fine.



> + =C2=A0 =C2=A0 =C2=A0 =C2=A0[ -z "$dscfile" ] || contin= ue

I personally would say "-n" && continue, might be a m= atter of taste.=C2=A0
=C2=A0
H= enning

> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0sudo -E chroot --userspec=3D$( i= d -u ):$( id -g ) ${rootfs} \
> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0sh -c ' mkdir = -p "/deb-src/${1}/${2}" && cd
> "/deb-src/${1}/${2}" && apt-get -y --download-on= ly --only-source
> source "$2"=3D"$3" ' download-src "${= rootfs_distro}" "${src}"
> "${version}"

------=_Part_2876_709519840.1587550271506-- ------=_Part_2875_1240019294.1587550271505--