* [PATCH] sbuild: Don't update apt database on every package build @ 2022-10-24 13:10 Uladzimir Bely 2022-10-24 14:14 ` Jan Kiszka 0 siblings, 1 reply; 6+ messages in thread From: Uladzimir Bely @ 2022-10-24 13:10 UTC (permalink / raw) To: isar-users By default, sbuild updates apt database beforethe build. It may cause an inconsistency between apt databases for different packages built at the different time if external (Debian) mirrors has updated recently. With new changes, sbuild will rely on global apt state that was got during debootstrap stage. Local `isar-apt` repo will still be automatically updated while it is passed via `--extra-repository=` argument. Signed-off-by: Uladzimir Bely <ubely@ilbers.de> --- meta/classes/dpkg.bbclass | 1 + 1 file changed, 1 insertion(+) diff --git a/meta/classes/dpkg.bbclass b/meta/classes/dpkg.bbclass index c92ea7db..a55b5776 100644 --- a/meta/classes/dpkg.bbclass +++ b/meta/classes/dpkg.bbclass @@ -90,6 +90,7 @@ dpkg_runbuild() { sbuild -A -n -c ${SBUILD_CHROOT} --extra-repository="${ISAR_APT_REPO}" \ --host=${PACKAGE_ARCH} --build=${SBUILD_HOST_ARCH} ${profiles} \ --no-run-lintian --no-run-piuparts --no-run-autopkgtest --resolve-alternatives \ + --no-apt-update --no-apt-upgrade --no-apt-distupgrade \ --chroot-setup-commands="rm -f /var/log/dpkg.log" \ --chroot-setup-commands="cp -n --no-preserve=owner ${ext_deb_dir}/*.deb -t ${deb_dir}/ || :" \ --finished-build-commands="rm -f ${deb_dir}/sbuild-build-depends-main-dummy_*.deb" \ -- 2.20.1 ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] sbuild: Don't update apt database on every package build 2022-10-24 13:10 [PATCH] sbuild: Don't update apt database on every package build Uladzimir Bely @ 2022-10-24 14:14 ` Jan Kiszka 2022-10-25 9:00 ` Uladzimir Bely 0 siblings, 1 reply; 6+ messages in thread From: Jan Kiszka @ 2022-10-24 14:14 UTC (permalink / raw) To: Uladzimir Bely, isar-users On 24.10.22 15:10, Uladzimir Bely wrote: > By default, sbuild updates apt database beforethe build. It may cause > an inconsistency between apt databases for different packages built at > the different time if external (Debian) mirrors has updated recently. > > With new changes, sbuild will rely on global apt state that was got > during debootstrap stage. > > Local `isar-apt` repo will still be automatically updated while it is > passed via `--extra-repository=` argument. Will that also ensure upgrades, e.g. from essential debian packages to self-built ones hosted in isar-apt? We had a few of such cases in the past already in downstream layers, and I recall permitting upgrades to the old buildchroot for that. Jan > > Signed-off-by: Uladzimir Bely <ubely@ilbers.de> > --- > meta/classes/dpkg.bbclass | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/meta/classes/dpkg.bbclass b/meta/classes/dpkg.bbclass > index c92ea7db..a55b5776 100644 > --- a/meta/classes/dpkg.bbclass > +++ b/meta/classes/dpkg.bbclass > @@ -90,6 +90,7 @@ dpkg_runbuild() { > sbuild -A -n -c ${SBUILD_CHROOT} --extra-repository="${ISAR_APT_REPO}" \ > --host=${PACKAGE_ARCH} --build=${SBUILD_HOST_ARCH} ${profiles} \ > --no-run-lintian --no-run-piuparts --no-run-autopkgtest --resolve-alternatives \ > + --no-apt-update --no-apt-upgrade --no-apt-distupgrade \ > --chroot-setup-commands="rm -f /var/log/dpkg.log" \ > --chroot-setup-commands="cp -n --no-preserve=owner ${ext_deb_dir}/*.deb -t ${deb_dir}/ || :" \ > --finished-build-commands="rm -f ${deb_dir}/sbuild-build-depends-main-dummy_*.deb" \ -- Siemens AG, Technology Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] sbuild: Don't update apt database on every package build 2022-10-24 14:14 ` Jan Kiszka @ 2022-10-25 9:00 ` Uladzimir Bely 2022-10-25 13:37 ` Jan Kiszka 0 siblings, 1 reply; 6+ messages in thread From: Uladzimir Bely @ 2022-10-25 9:00 UTC (permalink / raw) To: isar-users, Jan Kiszka In mail from понедельник, 24 октября 2022 г. 17:14:52 +03 user Jan Kiszka wrote: > On 24.10.22 15:10, Uladzimir Bely wrote: > > By default, sbuild updates apt database beforethe build. It may cause > > an inconsistency between apt databases for different packages built at > > the different time if external (Debian) mirrors has updated recently. > > > > With new changes, sbuild will rely on global apt state that was got > > during debootstrap stage. > > > > Local `isar-apt` repo will still be automatically updated while it is > > passed via `--extra-repository=` argument. > > Will that also ensure upgrades, e.g. from essential debian packages to > self-built ones hosted in isar-apt? We had a few of such cases in the > past already in downstream layers, and I recall permitting upgrades to > the old buildchroot for that. > I'm afraid I don't exactly get you question... Do you mean, it may happen that Isay may update 'isar-apt' repository, but won't even try to install "newer" (custom) package from isar-apt as an upgrade of some Debian package installed at debootstrap stage? Yes, probably it may happen. The case I tried to cover is just to make sbuild know about custom packages in 'isar-apt' so it could install them as depends. But if the package already installed (e.g. at debootstrap or sbuild-chroot stage) it may happen it won't be upgraded. I guess, there should be someting like "--no-apt-update --apt-distupgrade" then. So, apt database won't be updated, but if 'isar-apt' provides new version of already installed (Debian) package, it will be used. Or, even "--no-apt-update" should be enough, while "$apt_distupgrade" is already set by default in sbuild... Additinally, it would be good to have some test for this case. For example, we could have "hello" (from Debian) preinstalled before build, but upgrade it to custom "patched" version during sbuild. > Jan > > > Signed-off-by: Uladzimir Bely <ubely@ilbers.de> > > --- > > > > meta/classes/dpkg.bbclass | 1 + > > 1 file changed, 1 insertion(+) > > > > diff --git a/meta/classes/dpkg.bbclass b/meta/classes/dpkg.bbclass > > index c92ea7db..a55b5776 100644 > > --- a/meta/classes/dpkg.bbclass > > +++ b/meta/classes/dpkg.bbclass > > @@ -90,6 +90,7 @@ dpkg_runbuild() { > > > > sbuild -A -n -c ${SBUILD_CHROOT} > > --extra-repository="${ISAR_APT_REPO}" \ > > > > --host=${PACKAGE_ARCH} --build=${SBUILD_HOST_ARCH} ${profiles} \ > > --no-run-lintian --no-run-piuparts --no-run-autopkgtest > > --resolve-alternatives \> > > + --no-apt-update --no-apt-upgrade --no-apt-distupgrade \ > > > > --chroot-setup-commands="rm -f /var/log/dpkg.log" \ > > --chroot-setup-commands="cp -n --no-preserve=owner > > ${ext_deb_dir}/*.deb -t ${deb_dir}/ || :" \ > > --finished-build-commands="rm -f > > ${deb_dir}/sbuild-build-depends-main-dummy_*.deb" \ ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] sbuild: Don't update apt database on every package build 2022-10-25 9:00 ` Uladzimir Bely @ 2022-10-25 13:37 ` Jan Kiszka 2022-10-26 7:06 ` Uladzimir Bely 0 siblings, 1 reply; 6+ messages in thread From: Jan Kiszka @ 2022-10-25 13:37 UTC (permalink / raw) To: Uladzimir Bely, isar-users On 25.10.22 11:00, Uladzimir Bely wrote: > In mail from понедельник, 24 октября 2022 г. 17:14:52 +03 user Jan Kiszka > wrote: >> On 24.10.22 15:10, Uladzimir Bely wrote: >>> By default, sbuild updates apt database beforethe build. It may cause >>> an inconsistency between apt databases for different packages built at >>> the different time if external (Debian) mirrors has updated recently. >>> >>> With new changes, sbuild will rely on global apt state that was got >>> during debootstrap stage. >>> >>> Local `isar-apt` repo will still be automatically updated while it is >>> passed via `--extra-repository=` argument. >> >> Will that also ensure upgrades, e.g. from essential debian packages to >> self-built ones hosted in isar-apt? We had a few of such cases in the >> past already in downstream layers, and I recall permitting upgrades to >> the old buildchroot for that. >> > > I'm afraid I don't exactly get you question... > > Do you mean, it may happen that Isay may update 'isar-apt' repository, but > won't even try to install "newer" (custom) package from isar-apt as an upgrade > of some Debian package installed at debootstrap stage? Actually, part of my former problems were downgrades, not upgrades, see [1] and [2]. Those are possible as well because anything coming from isar-apt has higher prior than other repo. Now I was only wondering if that behavior is still given, with all those switches applied and schroot in between us and the actual apt calls. > > Yes, probably it may happen. The case I tried to cover is just to make sbuild > know about custom packages in 'isar-apt' so it could install them as depends. > But if the package already installed (e.g. at debootstrap or sbuild-chroot > stage) it may happen it won't be upgraded. > > I guess, there should be someting like "--no-apt-update --apt-distupgrade" > then. So, apt database won't be updated, but if 'isar-apt' provides new > version of already installed (Debian) package, it will be used. > > Or, even "--no-apt-update" should be enough, while "$apt_distupgrade" is > already set by default in sbuild... > > > Additinally, it would be good to have some test for this case. For example, we > could have "hello" (from Debian) preinstalled before build, but upgrade it to > custom "patched" version during sbuild. > Yes, that would indeed be good. Jan [1] https://github.com/ilbers/isar/commit/5367ba74d9f2f0b82560bab6c146f0521e13049f [2] https://github.com/ilbers/isar/commit/77e62c1e9bbda107a10136d1aa5f26f5f6660371 -- Siemens AG, Technology Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] sbuild: Don't update apt database on every package build 2022-10-25 13:37 ` Jan Kiszka @ 2022-10-26 7:06 ` Uladzimir Bely 2022-10-26 7:17 ` Jan Kiszka 0 siblings, 1 reply; 6+ messages in thread From: Uladzimir Bely @ 2022-10-26 7:06 UTC (permalink / raw) To: isar-users, Jan Kiszka In mail from вторник, 25 октября 2022 г. 16:37:51 +03 user Jan Kiszka wrote: > On 25.10.22 11:00, Uladzimir Bely wrote: > > In mail from понедельник, 24 октября 2022 г. 17:14:52 +03 user Jan Kiszka > > > > wrote: > >> On 24.10.22 15:10, Uladzimir Bely wrote: > >>> By default, sbuild updates apt database beforethe build. It may cause > >>> an inconsistency between apt databases for different packages built at > >>> the different time if external (Debian) mirrors has updated recently. > >>> > >>> With new changes, sbuild will rely on global apt state that was got > >>> during debootstrap stage. > >>> > >>> Local `isar-apt` repo will still be automatically updated while it is > >>> passed via `--extra-repository=` argument. > >> > >> Will that also ensure upgrades, e.g. from essential debian packages to > >> self-built ones hosted in isar-apt? We had a few of such cases in the > >> past already in downstream layers, and I recall permitting upgrades to > >> the old buildchroot for that. > > > > I'm afraid I don't exactly get you question... > > > > Do you mean, it may happen that Isay may update 'isar-apt' repository, but > > won't even try to install "newer" (custom) package from isar-apt as an > > upgrade of some Debian package installed at debootstrap stage? > > Actually, part of my former problems were downgrades, not upgrades, see > [1] and [2]. Those are possible as well because anything coming from > isar-apt has higher prior than other repo. Now I was only wondering if > that behavior is still given, with all those switches applied and > schroot in between us and the actual apt calls. It seems sbuild man pages say nothing about "--allow-downgrades". So, I'm not sure if it's used. Anyway, now I guess that setting `--no-apt-upgrade` and `--no-apt-distupgrade` in the v1 of patch is wrong. Most probably, such configuration won't allow even upgrades from debian packages to isar-apt ones. Also, you are saying, "isar-apt has higher prior than other repo". Where is it configured? I thought, `isar-apt` is just one of sources from "apt" and if Debian has newer package version than we compiled by hand, these newer dependencies are to be installed... > > > Yes, probably it may happen. The case I tried to cover is just to make > > sbuild know about custom packages in 'isar-apt' so it could install them > > as depends. But if the package already installed (e.g. at debootstrap or > > sbuild-chroot stage) it may happen it won't be upgraded. > > > > I guess, there should be someting like "--no-apt-update --apt-distupgrade" > > then. So, apt database won't be updated, but if 'isar-apt' provides new > > version of already installed (Debian) package, it will be used. > > > > Or, even "--no-apt-update" should be enough, while "$apt_distupgrade" is > > already set by default in sbuild... > > > > > > Additinally, it would be good to have some test for this case. For > > example, we could have "hello" (from Debian) preinstalled before build, > > but upgrade it to custom "patched" version during sbuild. > > Yes, that would indeed be good. > > Jan > > [1] > https://github.com/ilbers/isar/commit/5367ba74d9f2f0b82560bab6c146f0521e1304 > 9f [2] > https://github.com/ilbers/isar/commit/77e62c1e9bbda107a10136d1aa5f26f5f66603 > 71 ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] sbuild: Don't update apt database on every package build 2022-10-26 7:06 ` Uladzimir Bely @ 2022-10-26 7:17 ` Jan Kiszka 0 siblings, 0 replies; 6+ messages in thread From: Jan Kiszka @ 2022-10-26 7:17 UTC (permalink / raw) To: Uladzimir Bely, isar-users On 26.10.22 09:06, Uladzimir Bely wrote: > In mail from вторник, 25 октября 2022 г. 16:37:51 +03 user Jan Kiszka wrote: >> On 25.10.22 11:00, Uladzimir Bely wrote: >>> In mail from понедельник, 24 октября 2022 г. 17:14:52 +03 user Jan Kiszka >>> >>> wrote: >>>> On 24.10.22 15:10, Uladzimir Bely wrote: >>>>> By default, sbuild updates apt database beforethe build. It may cause >>>>> an inconsistency between apt databases for different packages built at >>>>> the different time if external (Debian) mirrors has updated recently. >>>>> >>>>> With new changes, sbuild will rely on global apt state that was got >>>>> during debootstrap stage. >>>>> >>>>> Local `isar-apt` repo will still be automatically updated while it is >>>>> passed via `--extra-repository=` argument. >>>> >>>> Will that also ensure upgrades, e.g. from essential debian packages to >>>> self-built ones hosted in isar-apt? We had a few of such cases in the >>>> past already in downstream layers, and I recall permitting upgrades to >>>> the old buildchroot for that. >>> >>> I'm afraid I don't exactly get you question... >>> >>> Do you mean, it may happen that Isay may update 'isar-apt' repository, but >>> won't even try to install "newer" (custom) package from isar-apt as an >>> upgrade of some Debian package installed at debootstrap stage? >> >> Actually, part of my former problems were downgrades, not upgrades, see >> [1] and [2]. Those are possible as well because anything coming from >> isar-apt has higher prior than other repo. Now I was only wondering if >> that behavior is still given, with all those switches applied and >> schroot in between us and the actual apt calls. > > It seems sbuild man pages say nothing about "--allow-downgrades". So, I'm not > sure if it's used. > > Anyway, now I guess that setting `--no-apt-upgrade` and `--no-apt-distupgrade` > in the v1 of patch is wrong. Most probably, such configuration won't allow > even upgrades from debian packages to isar-apt ones. > > Also, you are saying, "isar-apt has higher prior than other repo". Where is it > configured? I thought, `isar-apt` is just one of sources from "apt" and if > Debian has newer package version than we compiled by hand, these newer > dependencies are to be installed... https://github.com/ilbers/isar/blob/18db2a59f22ec3c376a1518627250d14683338fa/meta/classes/rootfs.bbclass#L93 Jan -- Siemens AG, Technology Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2022-10-26 7:17 UTC | newest] Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2022-10-24 13:10 [PATCH] sbuild: Don't update apt database on every package build Uladzimir Bely 2022-10-24 14:14 ` Jan Kiszka 2022-10-25 9:00 ` Uladzimir Bely 2022-10-25 13:37 ` Jan Kiszka 2022-10-26 7:06 ` Uladzimir Bely 2022-10-26 7:17 ` Jan Kiszka
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox