* Offline build broken with sbuild? @ 2022-11-21 6:49 Jan Kiszka 2022-11-21 7:36 ` Uladzimir Bely 0 siblings, 1 reply; 8+ messages in thread From: Jan Kiszka @ 2022-11-21 6:49 UTC (permalink / raw) To: isar-users Hi, seems like at least sbuild-chroot-host is not properly cached /wrt the additional Debian packages is pulls. Therefore, offline build fails. I've just confirmed this bug report: https://github.com/siemens/meta-iot2050/issues/382 What could be missing? Jan -- Siemens AG, Technology Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Offline build broken with sbuild? 2022-11-21 6:49 Offline build broken with sbuild? Jan Kiszka @ 2022-11-21 7:36 ` Uladzimir Bely 2022-11-22 12:15 ` Uladzimir Bely 0 siblings, 1 reply; 8+ messages in thread From: Uladzimir Bely @ 2022-11-21 7:36 UTC (permalink / raw) To: isar-users; +Cc: Jan Kiszka In the email from Monday, 21 November 2022 09:49:00 +03 user Jan Kiszka wrote: > Hi, > > seems like at least sbuild-chroot-host is not properly cached /wrt the > additional Debian packages is pulls. Therefore, offline build fails. > I've just confirmed this bug report: > > https://github.com/siemens/meta-iot2050/issues/382 > > What could be missing? > > Jan Just performed a quick test with the latest Isar 'next': 1. Run first build ``` bitbake mc:qemuarm64-bullseye:sbuild-chroot-host ``` 2. Cleaned "tmp" directory 3. Added ISAR_USE_CACHED_BASE_REPO = "1" and BB_NO_NETWORK = "1" 4. To be sure, stopped network on the machine and run build again: ``` bitbake mc:qemuarm64-bullseye:sbuild-chroot-host ``` Repo "base-apt" was created and the build was OK. I'll try to check more things (e.g. Isar revision mentioned in the bug; complete image build, etc), but at the first glance sbuild-chroot-host looks to be cached OK, at least with the latest Isar. -- Uladzimir Bely ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Offline build broken with sbuild? 2022-11-21 7:36 ` Uladzimir Bely @ 2022-11-22 12:15 ` Uladzimir Bely 2022-11-22 13:11 ` Jan Kiszka 0 siblings, 1 reply; 8+ messages in thread From: Uladzimir Bely @ 2022-11-22 12:15 UTC (permalink / raw) To: isar-users; +Cc: Jan Kiszka In the email from Monday, 21 November 2022 10:36:43 +03 user Uladzimir Bely wrote: > > I'll try to check more things (e.g. Isar revision mentioned in the bug; > complete image build, etc), but at the first glance sbuild-chroot-host looks > to be cached OK, at least with the latest Isar. > So, I've got the following result for now: - In Isar itself cached cross-build works well (while both HOST_DISTRO and DISTRO are equal). The only exception: raspberry targets, while DISTRO and HOST_DISTRO are different by nature. - In meta-iot2050, cached cross-build doesn't work for both "sbuild" and "pre- sbuild" cases. Evend "isar-bootstrap-host" task doesn't work for cached build. I've applied the following patch for now and it at least fixed sbuild-chroot- host task for cached build. Waiting for full build finished and will update you with the result. diff --git a/meta/recipes-devtools/base-apt/base-apt.bb b/meta/recipes- devtools/base-apt/base-apt.bb index 506a28f..316960d 100644 --- a/meta/recipes-devtools/base-apt/base-apt.bb +++ b/meta/recipes-devtools/base-apt/base-apt.bb @@ -13,7 +13,9 @@ KEYFILES ?= "" BASE_REPO_FEATURES ?= "" populate_base_apt() { - find "${DEBDIR}"/"${DISTRO}" -name '*\.deb' | while read package; do + distro="${BASE_DISTRO}-${BASE_DISTRO_CODENAME}" + + find "${DEBDIR}"/"${distro}" -name '*\.deb' | while read package; do # NOTE: due to packages stored by reprepro are not modified, we can # use search by filename to check if package is already in repo. In # addition, md5sums are compared to ensure that the package is the @@ -38,7 +40,7 @@ populate_base_apt() { "${package}" done - find "${DEBSRCDIR}"/"${DISTRO}" -name '*\.dsc' | while read package; do + find "${DEBSRCDIR}"/"${distro}" -name '*\.dsc' | while read package; do repo_add_srcpackage "${REPO_BASE_DIR}"/"${BASE_DISTRO}" \ "${REPO_BASE_DB_DIR}"/"${BASE_DISTRO}" \ "${BASE_DISTRO_CODENAME}" \ -- Uladzimir Bely ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Offline build broken with sbuild? 2022-11-22 12:15 ` Uladzimir Bely @ 2022-11-22 13:11 ` Jan Kiszka 2022-11-23 15:49 ` Jan Kiszka 0 siblings, 1 reply; 8+ messages in thread From: Jan Kiszka @ 2022-11-22 13:11 UTC (permalink / raw) To: Uladzimir Bely, isar-users On 22.11.22 13:15, Uladzimir Bely wrote: > In the email from Monday, 21 November 2022 10:36:43 +03 user Uladzimir Bely > wrote: >> >> I'll try to check more things (e.g. Isar revision mentioned in the bug; >> complete image build, etc), but at the first glance sbuild-chroot-host looks >> to be cached OK, at least with the latest Isar. >> > > So, I've got the following result for now: > > - In Isar itself cached cross-build works well (while both HOST_DISTRO and > DISTRO are equal). The only exception: raspberry targets, while DISTRO and > HOST_DISTRO are different by nature. > > - In meta-iot2050, cached cross-build doesn't work for both "sbuild" and "pre- > sbuild" cases. Evend "isar-bootstrap-host" task doesn't work for cached build. > > I've applied the following patch for now and it at least fixed sbuild-chroot- > host task for cached build. Waiting for full build finished and will update > you with the result. > > diff --git a/meta/recipes-devtools/base-apt/base-apt.bb b/meta/recipes- > devtools/base-apt/base-apt.bb > index 506a28f..316960d 100644 > --- a/meta/recipes-devtools/base-apt/base-apt.bb > +++ b/meta/recipes-devtools/base-apt/base-apt.bb > @@ -13,7 +13,9 @@ KEYFILES ?= "" > BASE_REPO_FEATURES ?= "" > > populate_base_apt() { > - find "${DEBDIR}"/"${DISTRO}" -name '*\.deb' | while read package; do > + distro="${BASE_DISTRO}-${BASE_DISTRO_CODENAME}" > + > + find "${DEBDIR}"/"${distro}" -name '*\.deb' | while read package; do > # NOTE: due to packages stored by reprepro are not modified, we can > # use search by filename to check if package is already in repo. In > # addition, md5sums are compared to ensure that the package is the > @@ -38,7 +40,7 @@ populate_base_apt() { > "${package}" > done > > - find "${DEBSRCDIR}"/"${DISTRO}" -name '*\.dsc' | while read package; do > + find "${DEBSRCDIR}"/"${distro}" -name '*\.dsc' | while read package; do > repo_add_srcpackage "${REPO_BASE_DIR}"/"${BASE_DISTRO}" \ > "${REPO_BASE_DB_DIR}"/"${BASE_DISTRO}" \ > "${BASE_DISTRO_CODENAME}" \ > > Ah, another DISTRO-override issue! Hope we don't have more of that type in the core. Maybe worth to scan again and check. Thanks for analyzing and (likely) fixing! Jan -- Siemens AG, Technology Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Offline build broken with sbuild? 2022-11-22 13:11 ` Jan Kiszka @ 2022-11-23 15:49 ` Jan Kiszka 2022-11-23 16:17 ` Jan Kiszka 0 siblings, 1 reply; 8+ messages in thread From: Jan Kiszka @ 2022-11-23 15:49 UTC (permalink / raw) To: Uladzimir Bely, isar-users On 22.11.22 14:11, Jan Kiszka wrote: > On 22.11.22 13:15, Uladzimir Bely wrote: >> In the email from Monday, 21 November 2022 10:36:43 +03 user Uladzimir Bely >> wrote: >>> >>> I'll try to check more things (e.g. Isar revision mentioned in the bug; >>> complete image build, etc), but at the first glance sbuild-chroot-host looks >>> to be cached OK, at least with the latest Isar. >>> >> >> So, I've got the following result for now: >> >> - In Isar itself cached cross-build works well (while both HOST_DISTRO and >> DISTRO are equal). The only exception: raspberry targets, while DISTRO and >> HOST_DISTRO are different by nature. >> >> - In meta-iot2050, cached cross-build doesn't work for both "sbuild" and "pre- >> sbuild" cases. Evend "isar-bootstrap-host" task doesn't work for cached build. >> >> I've applied the following patch for now and it at least fixed sbuild-chroot- >> host task for cached build. Waiting for full build finished and will update >> you with the result. >> >> diff --git a/meta/recipes-devtools/base-apt/base-apt.bb b/meta/recipes- >> devtools/base-apt/base-apt.bb >> index 506a28f..316960d 100644 >> --- a/meta/recipes-devtools/base-apt/base-apt.bb >> +++ b/meta/recipes-devtools/base-apt/base-apt.bb >> @@ -13,7 +13,9 @@ KEYFILES ?= "" >> BASE_REPO_FEATURES ?= "" >> >> populate_base_apt() { >> - find "${DEBDIR}"/"${DISTRO}" -name '*\.deb' | while read package; do >> + distro="${BASE_DISTRO}-${BASE_DISTRO_CODENAME}" >> + >> + find "${DEBDIR}"/"${distro}" -name '*\.deb' | while read package; do >> # NOTE: due to packages stored by reprepro are not modified, we can >> # use search by filename to check if package is already in repo. In >> # addition, md5sums are compared to ensure that the package is the >> @@ -38,7 +40,7 @@ populate_base_apt() { >> "${package}" >> done >> >> - find "${DEBSRCDIR}"/"${DISTRO}" -name '*\.dsc' | while read package; do >> + find "${DEBSRCDIR}"/"${distro}" -name '*\.dsc' | while read package; do >> repo_add_srcpackage "${REPO_BASE_DIR}"/"${BASE_DISTRO}" \ >> "${REPO_BASE_DB_DIR}"/"${BASE_DISTRO}" \ >> "${BASE_DISTRO_CODENAME}" \ >> >> > > Ah, another DISTRO-override issue! Hope we don't have more of that type > in the core. Maybe worth to scan again and check. > > Thanks for analyzing and (likely) fixing! > sbuild-chroot-host is indeed working now, but there are likely more cases with (custom) DISTRO vs. BASE_DISTRO: ERROR: isar-bootstrap-target-1.0-r0 do_bootstrap: ExecutionError('/build/tmp/work/iot2050-debian-arm64/isar-bootstrap-target/1.0-r0/temp/run.do_bootstrap.21764', 1, None, None) ERROR: Logfile of failure stored in: /build/tmp/work/iot2050-debian-arm64/isar-bootstrap-target/1.0-r0/temp/log.do_bootstrap.21764 Log data follows: | DEBUG: Executing python function sstate_task_prefunc | DEBUG: Python function sstate_task_prefunc finished | DEBUG: Executing shell function do_bootstrap | W: qemu-debootstrap is deprecated. Please use regular debootstrap directly | I: Running command: debootstrap --verbose --variant=minbase --include=locales --no-check-gpg --arch=arm64 --components=main,contrib,non-free bullseye /build/tmp/work/iot2050-debian-arm64/isar-bootstrap-target/1.0-r0/rootfs file:///build/tmp/deploy/base-apt/iot2050-debian/apt/debian | I: Retrieving InRelease | I: Retrieving Release | I: Retrieving Packages | I: Validating Packages | I: Resolving dependencies of required packages... | I: Resolving dependencies of base packages... | I: Checking component main on file:///build/tmp/deploy/base-apt/iot2050-debian/apt/debian... | E: Couldn't find these debs: apt | WARNING: exit code 1 from a shell command. | ERROR: ExecutionError('/build/tmp/work/iot2050-debian-arm64/isar-bootstrap-target/1.0-r0/temp/run.do_bootstrap.21764', 1, None, None) ERROR: Task (/build/../work/isar/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb:do_bootstrap) failed with exit code '1' Jan -- Siemens AG, Technology Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Offline build broken with sbuild? 2022-11-23 15:49 ` Jan Kiszka @ 2022-11-23 16:17 ` Jan Kiszka 2022-11-24 6:57 ` Uladzimir Bely 0 siblings, 1 reply; 8+ messages in thread From: Jan Kiszka @ 2022-11-23 16:17 UTC (permalink / raw) To: Uladzimir Bely, isar-users On 23.11.22 16:49, Jan Kiszka wrote: > On 22.11.22 14:11, Jan Kiszka wrote: >> On 22.11.22 13:15, Uladzimir Bely wrote: >>> In the email from Monday, 21 November 2022 10:36:43 +03 user Uladzimir Bely >>> wrote: >>>> >>>> I'll try to check more things (e.g. Isar revision mentioned in the bug; >>>> complete image build, etc), but at the first glance sbuild-chroot-host looks >>>> to be cached OK, at least with the latest Isar. >>>> >>> >>> So, I've got the following result for now: >>> >>> - In Isar itself cached cross-build works well (while both HOST_DISTRO and >>> DISTRO are equal). The only exception: raspberry targets, while DISTRO and >>> HOST_DISTRO are different by nature. >>> >>> - In meta-iot2050, cached cross-build doesn't work for both "sbuild" and "pre- >>> sbuild" cases. Evend "isar-bootstrap-host" task doesn't work for cached build. >>> >>> I've applied the following patch for now and it at least fixed sbuild-chroot- >>> host task for cached build. Waiting for full build finished and will update >>> you with the result. >>> >>> diff --git a/meta/recipes-devtools/base-apt/base-apt.bb b/meta/recipes- >>> devtools/base-apt/base-apt.bb >>> index 506a28f..316960d 100644 >>> --- a/meta/recipes-devtools/base-apt/base-apt.bb >>> +++ b/meta/recipes-devtools/base-apt/base-apt.bb >>> @@ -13,7 +13,9 @@ KEYFILES ?= "" >>> BASE_REPO_FEATURES ?= "" >>> >>> populate_base_apt() { >>> - find "${DEBDIR}"/"${DISTRO}" -name '*\.deb' | while read package; do >>> + distro="${BASE_DISTRO}-${BASE_DISTRO_CODENAME}" >>> + >>> + find "${DEBDIR}"/"${distro}" -name '*\.deb' | while read package; do >>> # NOTE: due to packages stored by reprepro are not modified, we can >>> # use search by filename to check if package is already in repo. In >>> # addition, md5sums are compared to ensure that the package is the >>> @@ -38,7 +40,7 @@ populate_base_apt() { >>> "${package}" >>> done >>> >>> - find "${DEBSRCDIR}"/"${DISTRO}" -name '*\.dsc' | while read package; do >>> + find "${DEBSRCDIR}"/"${distro}" -name '*\.dsc' | while read package; do >>> repo_add_srcpackage "${REPO_BASE_DIR}"/"${BASE_DISTRO}" \ >>> "${REPO_BASE_DB_DIR}"/"${BASE_DISTRO}" \ >>> "${BASE_DISTRO_CODENAME}" \ >>> >>> >> >> Ah, another DISTRO-override issue! Hope we don't have more of that type >> in the core. Maybe worth to scan again and check. >> >> Thanks for analyzing and (likely) fixing! >> > > sbuild-chroot-host is indeed working now, but there are likely more > cases with (custom) DISTRO vs. BASE_DISTRO: > > ERROR: isar-bootstrap-target-1.0-r0 do_bootstrap: ExecutionError('/build/tmp/work/iot2050-debian-arm64/isar-bootstrap-target/1.0-r0/temp/run.do_bootstrap.21764', 1, None, None) > ERROR: Logfile of failure stored in: /build/tmp/work/iot2050-debian-arm64/isar-bootstrap-target/1.0-r0/temp/log.do_bootstrap.21764 > Log data follows: > | DEBUG: Executing python function sstate_task_prefunc > | DEBUG: Python function sstate_task_prefunc finished > | DEBUG: Executing shell function do_bootstrap > | W: qemu-debootstrap is deprecated. Please use regular debootstrap directly > | I: Running command: debootstrap --verbose --variant=minbase --include=locales --no-check-gpg --arch=arm64 --components=main,contrib,non-free bullseye /build/tmp/work/iot2050-debian-arm64/isar-bootstrap-target/1.0-r0/rootfs file:///build/tmp/deploy/base-apt/iot2050-debian/apt/debian > | I: Retrieving InRelease > | I: Retrieving Release > | I: Retrieving Packages > | I: Validating Packages > | I: Resolving dependencies of required packages... > | I: Resolving dependencies of base packages... > | I: Checking component main on file:///build/tmp/deploy/base-apt/iot2050-debian/apt/debian... > | E: Couldn't find these debs: apt > | WARNING: exit code 1 from a shell command. > | ERROR: ExecutionError('/build/tmp/work/iot2050-debian-arm64/isar-bootstrap-target/1.0-r0/temp/run.do_bootstrap.21764', 1, None, None) > ERROR: Task (/build/../work/isar/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb:do_bootstrap) failed with exit code '1' > I think we have two problems here: 1. base-apt is set up in sources-list using the BASE_DISTRO, but we are creating it using DISTRO, not only in bast-apt, also dpkg-base 2. the whole base-apt caching does not yet consider BASE_DISTRO(DISTRO) != BASE_DISTRO(HOST_DISTRO), and that maybe even before all the sbuild changes The latter should break the Raspian case. Jan -- Siemens AG, Technology Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Offline build broken with sbuild? 2022-11-23 16:17 ` Jan Kiszka @ 2022-11-24 6:57 ` Uladzimir Bely 2022-11-24 7:32 ` Jan Kiszka 0 siblings, 1 reply; 8+ messages in thread From: Uladzimir Bely @ 2022-11-24 6:57 UTC (permalink / raw) To: isar-users, Jan Kiszka In the email from Wednesday, 23 November 2022 19:17:31 +03 user Jan Kiszka wrote: > On 23.11.22 16:49, Jan Kiszka wrote: > > On 22.11.22 14:11, Jan Kiszka wrote: > >> On 22.11.22 13:15, Uladzimir Bely wrote: > >>> In the email from Monday, 21 November 2022 10:36:43 +03 user Uladzimir > >>> Bely > >>> > >>> wrote: > >>>> I'll try to check more things (e.g. Isar revision mentioned in the bug; > >>>> complete image build, etc), but at the first glance sbuild-chroot-host > >>>> looks to be cached OK, at least with the latest Isar. > >>> > >>> So, I've got the following result for now: > >>> > >>> - In Isar itself cached cross-build works well (while both HOST_DISTRO > >>> and > >>> DISTRO are equal). The only exception: raspberry targets, while DISTRO > >>> and > >>> HOST_DISTRO are different by nature. > >>> > >>> - In meta-iot2050, cached cross-build doesn't work for both "sbuild" and > >>> "pre- sbuild" cases. Evend "isar-bootstrap-host" task doesn't work for > >>> cached build. > >>> > >>> I've applied the following patch for now and it at least fixed > >>> sbuild-chroot- host task for cached build. Waiting for full build > >>> finished and will update you with the result. > >>> > >>> diff --git a/meta/recipes-devtools/base-apt/base-apt.bb b/meta/recipes- > >>> devtools/base-apt/base-apt.bb > >>> index 506a28f..316960d 100644 > >>> --- a/meta/recipes-devtools/base-apt/base-apt.bb > >>> +++ b/meta/recipes-devtools/base-apt/base-apt.bb > >>> @@ -13,7 +13,9 @@ KEYFILES ?= "" > >>> > >>> BASE_REPO_FEATURES ?= "" > >>> > >>> populate_base_apt() { > >>> > >>> - find "${DEBDIR}"/"${DISTRO}" -name '*\.deb' | while read package; > >>> do > >>> + distro="${BASE_DISTRO}-${BASE_DISTRO_CODENAME}" > >>> + > >>> + find "${DEBDIR}"/"${distro}" -name '*\.deb' | while read package; > >>> do > >>> > >>> # NOTE: due to packages stored by reprepro are not modified, we > >>> can > >>> # use search by filename to check if package is already in > >>> repo. In > >>> # addition, md5sums are compared to ensure that the package is > >>> the > >>> > >>> @@ -38,7 +40,7 @@ populate_base_apt() { > >>> > >>> "${package}" > >>> > >>> done > >>> > >>> - find "${DEBSRCDIR}"/"${DISTRO}" -name '*\.dsc' | while read > >>> package; do + find "${DEBSRCDIR}"/"${distro}" -name '*\.dsc' | while > >>> read package; do>>> > >>> repo_add_srcpackage "${REPO_BASE_DIR}"/"${BASE_DISTRO}" \ > >>> > >>> "${REPO_BASE_DB_DIR}"/"${BASE_DISTRO}" \ > >>> "${BASE_DISTRO_CODENAME}" \ > >> > >> Ah, another DISTRO-override issue! Hope we don't have more of that type > >> in the core. Maybe worth to scan again and check. > >> > >> Thanks for analyzing and (likely) fixing! > > > > sbuild-chroot-host is indeed working now, but there are likely more > > cases with (custom) DISTRO vs. BASE_DISTRO: > > > > ERROR: isar-bootstrap-target-1.0-r0 do_bootstrap: > > ExecutionError('/build/tmp/work/iot2050-debian-arm64/isar-bootstrap-targe > > t/1.0-r0/temp/run.do_bootstrap.21764', 1, None, None) ERROR: Logfile of > > failure stored in: > > /build/tmp/work/iot2050-debian-arm64/isar-bootstrap-target/1.0-r0/temp/lo > > g.do_bootstrap.21764> > > Log data follows: > > | DEBUG: Executing python function sstate_task_prefunc > > | DEBUG: Python function sstate_task_prefunc finished > > | DEBUG: Executing shell function do_bootstrap > > | W: qemu-debootstrap is deprecated. Please use regular debootstrap > > | directly > > | I: Running command: debootstrap --verbose --variant=minbase > > | --include=locales --no-check-gpg --arch=arm64 > > | --components=main,contrib,non-free bullseye > > | /build/tmp/work/iot2050-debian-arm64/isar-bootstrap-target/1.0-r0/rootf > > | s file:///build/tmp/deploy/base-apt/iot2050-debian/apt/debian I: > > | Retrieving InRelease > > | I: Retrieving Release > > | I: Retrieving Packages > > | I: Validating Packages > > | I: Resolving dependencies of required packages... > > | I: Resolving dependencies of base packages... > > | I: Checking component main on > > | file:///build/tmp/deploy/base-apt/iot2050-debian/apt/debian... E: > > | Couldn't find these debs: apt > > | WARNING: exit code 1 from a shell command. > > | ERROR: > > | ExecutionError('/build/tmp/work/iot2050-debian-arm64/isar-bootstrap-tar > > | get/1.0-r0/temp/run.do_bootstrap.21764', 1, None, None)> > > ERROR: Task > > (/build/../work/isar/meta/recipes-core/isar-bootstrap/isar-bootstrap-targ > > et.bb:do_bootstrap) failed with exit code '1' > I think we have two problems here: > > 1. base-apt is set up in sources-list using the BASE_DISTRO, but we are > creating it using DISTRO, not only in bast-apt, also dpkg-base > > 2. the whole base-apt caching does not yet consider > BASE_DISTRO(DISTRO) != BASE_DISTRO(HOST_DISTRO), and that maybe even > before all the sbuild changes > > The latter should break the Raspian case. > > Jan I've just send patchset that fixes case when DISTRO is changed from default value (but still in fact remains "debian"). I set up the following configuration in local.conf: > require conf/distro/debian-bullseye.conf > DISTRO_NAME = "My Debian System" > DISTRO = "my-debian" And got the same problem with sbuild-chroot-host installation at 2nd (cached) build. The patches solved the issue. The case with Raspberry (target) / Debian (host) is more complex and still doesn't work. To make cached build work in this case we at least need to rework `base-apt` - split it onto 2 repos that are populated only from their own original repos. What concerns `meta-iot2050`, it still fails to do cached build, but due to some local problems. For example, `python-absl` recipe from `meta-coral` layer conflicts with BB_NO_NETWORK = "1" option... -- Uladzimir Bely ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Offline build broken with sbuild? 2022-11-24 6:57 ` Uladzimir Bely @ 2022-11-24 7:32 ` Jan Kiszka 0 siblings, 0 replies; 8+ messages in thread From: Jan Kiszka @ 2022-11-24 7:32 UTC (permalink / raw) To: Uladzimir Bely, isar-users On 24.11.22 07:57, Uladzimir Bely wrote: > In the email from Wednesday, 23 November 2022 19:17:31 +03 user Jan Kiszka > wrote: >> On 23.11.22 16:49, Jan Kiszka wrote: >>> On 22.11.22 14:11, Jan Kiszka wrote: >>>> On 22.11.22 13:15, Uladzimir Bely wrote: >>>>> In the email from Monday, 21 November 2022 10:36:43 +03 user Uladzimir >>>>> Bely >>>>> >>>>> wrote: >>>>>> I'll try to check more things (e.g. Isar revision mentioned in the bug; >>>>>> complete image build, etc), but at the first glance sbuild-chroot-host >>>>>> looks to be cached OK, at least with the latest Isar. >>>>> >>>>> So, I've got the following result for now: >>>>> >>>>> - In Isar itself cached cross-build works well (while both HOST_DISTRO >>>>> and >>>>> DISTRO are equal). The only exception: raspberry targets, while DISTRO >>>>> and >>>>> HOST_DISTRO are different by nature. >>>>> >>>>> - In meta-iot2050, cached cross-build doesn't work for both "sbuild" and >>>>> "pre- sbuild" cases. Evend "isar-bootstrap-host" task doesn't work for >>>>> cached build. >>>>> >>>>> I've applied the following patch for now and it at least fixed >>>>> sbuild-chroot- host task for cached build. Waiting for full build >>>>> finished and will update you with the result. >>>>> >>>>> diff --git a/meta/recipes-devtools/base-apt/base-apt.bb b/meta/recipes- >>>>> devtools/base-apt/base-apt.bb >>>>> index 506a28f..316960d 100644 >>>>> --- a/meta/recipes-devtools/base-apt/base-apt.bb >>>>> +++ b/meta/recipes-devtools/base-apt/base-apt.bb >>>>> @@ -13,7 +13,9 @@ KEYFILES ?= "" >>>>> >>>>> BASE_REPO_FEATURES ?= "" >>>>> >>>>> populate_base_apt() { >>>>> >>>>> - find "${DEBDIR}"/"${DISTRO}" -name '*\.deb' | while read package; >>>>> do >>>>> + distro="${BASE_DISTRO}-${BASE_DISTRO_CODENAME}" >>>>> + >>>>> + find "${DEBDIR}"/"${distro}" -name '*\.deb' | while read package; >>>>> do >>>>> >>>>> # NOTE: due to packages stored by reprepro are not modified, we >>>>> can >>>>> # use search by filename to check if package is already in >>>>> repo. In >>>>> # addition, md5sums are compared to ensure that the package is >>>>> the >>>>> >>>>> @@ -38,7 +40,7 @@ populate_base_apt() { >>>>> >>>>> "${package}" >>>>> >>>>> done >>>>> >>>>> - find "${DEBSRCDIR}"/"${DISTRO}" -name '*\.dsc' | while read >>>>> package; do + find "${DEBSRCDIR}"/"${distro}" -name '*\.dsc' | while >>>>> read package; do>>> >>>>> repo_add_srcpackage "${REPO_BASE_DIR}"/"${BASE_DISTRO}" \ >>>>> >>>>> "${REPO_BASE_DB_DIR}"/"${BASE_DISTRO}" \ >>>>> "${BASE_DISTRO_CODENAME}" \ >>>> >>>> Ah, another DISTRO-override issue! Hope we don't have more of that type >>>> in the core. Maybe worth to scan again and check. >>>> >>>> Thanks for analyzing and (likely) fixing! >>> >>> sbuild-chroot-host is indeed working now, but there are likely more >>> cases with (custom) DISTRO vs. BASE_DISTRO: >>> >>> ERROR: isar-bootstrap-target-1.0-r0 do_bootstrap: >>> ExecutionError('/build/tmp/work/iot2050-debian-arm64/isar-bootstrap-targe >>> t/1.0-r0/temp/run.do_bootstrap.21764', 1, None, None) ERROR: Logfile of >>> failure stored in: >>> /build/tmp/work/iot2050-debian-arm64/isar-bootstrap-target/1.0-r0/temp/lo >>> g.do_bootstrap.21764> >>> Log data follows: >>> | DEBUG: Executing python function sstate_task_prefunc >>> | DEBUG: Python function sstate_task_prefunc finished >>> | DEBUG: Executing shell function do_bootstrap >>> | W: qemu-debootstrap is deprecated. Please use regular debootstrap >>> | directly >>> | I: Running command: debootstrap --verbose --variant=minbase >>> | --include=locales --no-check-gpg --arch=arm64 >>> | --components=main,contrib,non-free bullseye >>> | /build/tmp/work/iot2050-debian-arm64/isar-bootstrap-target/1.0-r0/rootf >>> | s file:///build/tmp/deploy/base-apt/iot2050-debian/apt/debian I: >>> | Retrieving InRelease >>> | I: Retrieving Release >>> | I: Retrieving Packages >>> | I: Validating Packages >>> | I: Resolving dependencies of required packages... >>> | I: Resolving dependencies of base packages... >>> | I: Checking component main on >>> | file:///build/tmp/deploy/base-apt/iot2050-debian/apt/debian... E: >>> | Couldn't find these debs: apt >>> | WARNING: exit code 1 from a shell command. >>> | ERROR: >>> | ExecutionError('/build/tmp/work/iot2050-debian-arm64/isar-bootstrap-tar >>> | get/1.0-r0/temp/run.do_bootstrap.21764', 1, None, None)> >>> ERROR: Task >>> (/build/../work/isar/meta/recipes-core/isar-bootstrap/isar-bootstrap-targ >>> et.bb:do_bootstrap) failed with exit code '1' >> I think we have two problems here: >> >> 1. base-apt is set up in sources-list using the BASE_DISTRO, but we are >> creating it using DISTRO, not only in bast-apt, also dpkg-base >> >> 2. the whole base-apt caching does not yet consider >> BASE_DISTRO(DISTRO) != BASE_DISTRO(HOST_DISTRO), and that maybe even >> before all the sbuild changes >> >> The latter should break the Raspian case. >> >> Jan > > I've just send patchset that fixes case when DISTRO is changed from default > value (but still in fact remains "debian"). > > I set up the following configuration in local.conf: > >> require conf/distro/debian-bullseye.conf >> DISTRO_NAME = "My Debian System" >> DISTRO = "my-debian" > > And got the same problem with sbuild-chroot-host installation at 2nd (cached) > build. The patches solved the issue. > > The case with Raspberry (target) / Debian (host) is more complex and still > doesn't work. To make cached build work in this case we at least need to > rework `base-apt` - split it onto 2 repos that are populated only from their > own original repos. Left a comment > > What concerns `meta-iot2050`, it still fails to do cached build, but due to > some local problems. For example, `python-absl` recipe from `meta-coral` layer > conflicts with BB_NO_NETWORK = "1" option... > Yes, this is understood already, and a fix for meta-coral is pending. Thanks, Jan -- Siemens AG, Technology Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2022-11-24 7:32 UTC | newest] Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2022-11-21 6:49 Offline build broken with sbuild? Jan Kiszka 2022-11-21 7:36 ` Uladzimir Bely 2022-11-22 12:15 ` Uladzimir Bely 2022-11-22 13:11 ` Jan Kiszka 2022-11-23 15:49 ` Jan Kiszka 2022-11-23 16:17 ` Jan Kiszka 2022-11-24 6:57 ` Uladzimir Bely 2022-11-24 7:32 ` Jan Kiszka
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox