* Offline build broken with sbuild?
@ 2022-11-21 6:49 Jan Kiszka
2022-11-21 7:36 ` Uladzimir Bely
0 siblings, 1 reply; 8+ messages in thread
From: Jan Kiszka @ 2022-11-21 6:49 UTC (permalink / raw)
To: isar-users
Hi,
seems like at least sbuild-chroot-host is not properly cached /wrt the
additional Debian packages is pulls. Therefore, offline build fails.
I've just confirmed this bug report:
https://github.com/siemens/meta-iot2050/issues/382
What could be missing?
Jan
--
Siemens AG, Technology
Competence Center Embedded Linux
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Offline build broken with sbuild?
2022-11-21 6:49 Offline build broken with sbuild? Jan Kiszka
@ 2022-11-21 7:36 ` Uladzimir Bely
2022-11-22 12:15 ` Uladzimir Bely
0 siblings, 1 reply; 8+ messages in thread
From: Uladzimir Bely @ 2022-11-21 7:36 UTC (permalink / raw)
To: isar-users; +Cc: Jan Kiszka
In the email from Monday, 21 November 2022 09:49:00 +03 user Jan Kiszka wrote:
> Hi,
>
> seems like at least sbuild-chroot-host is not properly cached /wrt the
> additional Debian packages is pulls. Therefore, offline build fails.
> I've just confirmed this bug report:
>
> https://github.com/siemens/meta-iot2050/issues/382
>
> What could be missing?
>
> Jan
Just performed a quick test with the latest Isar 'next':
1. Run first build
```
bitbake mc:qemuarm64-bullseye:sbuild-chroot-host
```
2. Cleaned "tmp" directory
3. Added ISAR_USE_CACHED_BASE_REPO = "1" and BB_NO_NETWORK = "1"
4. To be sure, stopped network on the machine and run build again:
```
bitbake mc:qemuarm64-bullseye:sbuild-chroot-host
```
Repo "base-apt" was created and the build was OK.
I'll try to check more things (e.g. Isar revision mentioned in the bug;
complete image build, etc), but at the first glance sbuild-chroot-host looks
to be cached OK, at least with the latest Isar.
--
Uladzimir Bely
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Offline build broken with sbuild?
2022-11-21 7:36 ` Uladzimir Bely
@ 2022-11-22 12:15 ` Uladzimir Bely
2022-11-22 13:11 ` Jan Kiszka
0 siblings, 1 reply; 8+ messages in thread
From: Uladzimir Bely @ 2022-11-22 12:15 UTC (permalink / raw)
To: isar-users; +Cc: Jan Kiszka
In the email from Monday, 21 November 2022 10:36:43 +03 user Uladzimir Bely
wrote:
>
> I'll try to check more things (e.g. Isar revision mentioned in the bug;
> complete image build, etc), but at the first glance sbuild-chroot-host looks
> to be cached OK, at least with the latest Isar.
>
So, I've got the following result for now:
- In Isar itself cached cross-build works well (while both HOST_DISTRO and
DISTRO are equal). The only exception: raspberry targets, while DISTRO and
HOST_DISTRO are different by nature.
- In meta-iot2050, cached cross-build doesn't work for both "sbuild" and "pre-
sbuild" cases. Evend "isar-bootstrap-host" task doesn't work for cached build.
I've applied the following patch for now and it at least fixed sbuild-chroot-
host task for cached build. Waiting for full build finished and will update
you with the result.
diff --git a/meta/recipes-devtools/base-apt/base-apt.bb b/meta/recipes-
devtools/base-apt/base-apt.bb
index 506a28f..316960d 100644
--- a/meta/recipes-devtools/base-apt/base-apt.bb
+++ b/meta/recipes-devtools/base-apt/base-apt.bb
@@ -13,7 +13,9 @@ KEYFILES ?= ""
BASE_REPO_FEATURES ?= ""
populate_base_apt() {
- find "${DEBDIR}"/"${DISTRO}" -name '*\.deb' | while read package; do
+ distro="${BASE_DISTRO}-${BASE_DISTRO_CODENAME}"
+
+ find "${DEBDIR}"/"${distro}" -name '*\.deb' | while read package; do
# NOTE: due to packages stored by reprepro are not modified, we can
# use search by filename to check if package is already in repo. In
# addition, md5sums are compared to ensure that the package is the
@@ -38,7 +40,7 @@ populate_base_apt() {
"${package}"
done
- find "${DEBSRCDIR}"/"${DISTRO}" -name '*\.dsc' | while read package; do
+ find "${DEBSRCDIR}"/"${distro}" -name '*\.dsc' | while read package; do
repo_add_srcpackage "${REPO_BASE_DIR}"/"${BASE_DISTRO}" \
"${REPO_BASE_DB_DIR}"/"${BASE_DISTRO}" \
"${BASE_DISTRO_CODENAME}" \
--
Uladzimir Bely
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Offline build broken with sbuild?
2022-11-22 12:15 ` Uladzimir Bely
@ 2022-11-22 13:11 ` Jan Kiszka
2022-11-23 15:49 ` Jan Kiszka
0 siblings, 1 reply; 8+ messages in thread
From: Jan Kiszka @ 2022-11-22 13:11 UTC (permalink / raw)
To: Uladzimir Bely, isar-users
On 22.11.22 13:15, Uladzimir Bely wrote:
> In the email from Monday, 21 November 2022 10:36:43 +03 user Uladzimir Bely
> wrote:
>>
>> I'll try to check more things (e.g. Isar revision mentioned in the bug;
>> complete image build, etc), but at the first glance sbuild-chroot-host looks
>> to be cached OK, at least with the latest Isar.
>>
>
> So, I've got the following result for now:
>
> - In Isar itself cached cross-build works well (while both HOST_DISTRO and
> DISTRO are equal). The only exception: raspberry targets, while DISTRO and
> HOST_DISTRO are different by nature.
>
> - In meta-iot2050, cached cross-build doesn't work for both "sbuild" and "pre-
> sbuild" cases. Evend "isar-bootstrap-host" task doesn't work for cached build.
>
> I've applied the following patch for now and it at least fixed sbuild-chroot-
> host task for cached build. Waiting for full build finished and will update
> you with the result.
>
> diff --git a/meta/recipes-devtools/base-apt/base-apt.bb b/meta/recipes-
> devtools/base-apt/base-apt.bb
> index 506a28f..316960d 100644
> --- a/meta/recipes-devtools/base-apt/base-apt.bb
> +++ b/meta/recipes-devtools/base-apt/base-apt.bb
> @@ -13,7 +13,9 @@ KEYFILES ?= ""
> BASE_REPO_FEATURES ?= ""
>
> populate_base_apt() {
> - find "${DEBDIR}"/"${DISTRO}" -name '*\.deb' | while read package; do
> + distro="${BASE_DISTRO}-${BASE_DISTRO_CODENAME}"
> +
> + find "${DEBDIR}"/"${distro}" -name '*\.deb' | while read package; do
> # NOTE: due to packages stored by reprepro are not modified, we can
> # use search by filename to check if package is already in repo. In
> # addition, md5sums are compared to ensure that the package is the
> @@ -38,7 +40,7 @@ populate_base_apt() {
> "${package}"
> done
>
> - find "${DEBSRCDIR}"/"${DISTRO}" -name '*\.dsc' | while read package; do
> + find "${DEBSRCDIR}"/"${distro}" -name '*\.dsc' | while read package; do
> repo_add_srcpackage "${REPO_BASE_DIR}"/"${BASE_DISTRO}" \
> "${REPO_BASE_DB_DIR}"/"${BASE_DISTRO}" \
> "${BASE_DISTRO_CODENAME}" \
>
>
Ah, another DISTRO-override issue! Hope we don't have more of that type
in the core. Maybe worth to scan again and check.
Thanks for analyzing and (likely) fixing!
Jan
--
Siemens AG, Technology
Competence Center Embedded Linux
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Offline build broken with sbuild?
2022-11-22 13:11 ` Jan Kiszka
@ 2022-11-23 15:49 ` Jan Kiszka
2022-11-23 16:17 ` Jan Kiszka
0 siblings, 1 reply; 8+ messages in thread
From: Jan Kiszka @ 2022-11-23 15:49 UTC (permalink / raw)
To: Uladzimir Bely, isar-users
On 22.11.22 14:11, Jan Kiszka wrote:
> On 22.11.22 13:15, Uladzimir Bely wrote:
>> In the email from Monday, 21 November 2022 10:36:43 +03 user Uladzimir Bely
>> wrote:
>>>
>>> I'll try to check more things (e.g. Isar revision mentioned in the bug;
>>> complete image build, etc), but at the first glance sbuild-chroot-host looks
>>> to be cached OK, at least with the latest Isar.
>>>
>>
>> So, I've got the following result for now:
>>
>> - In Isar itself cached cross-build works well (while both HOST_DISTRO and
>> DISTRO are equal). The only exception: raspberry targets, while DISTRO and
>> HOST_DISTRO are different by nature.
>>
>> - In meta-iot2050, cached cross-build doesn't work for both "sbuild" and "pre-
>> sbuild" cases. Evend "isar-bootstrap-host" task doesn't work for cached build.
>>
>> I've applied the following patch for now and it at least fixed sbuild-chroot-
>> host task for cached build. Waiting for full build finished and will update
>> you with the result.
>>
>> diff --git a/meta/recipes-devtools/base-apt/base-apt.bb b/meta/recipes-
>> devtools/base-apt/base-apt.bb
>> index 506a28f..316960d 100644
>> --- a/meta/recipes-devtools/base-apt/base-apt.bb
>> +++ b/meta/recipes-devtools/base-apt/base-apt.bb
>> @@ -13,7 +13,9 @@ KEYFILES ?= ""
>> BASE_REPO_FEATURES ?= ""
>>
>> populate_base_apt() {
>> - find "${DEBDIR}"/"${DISTRO}" -name '*\.deb' | while read package; do
>> + distro="${BASE_DISTRO}-${BASE_DISTRO_CODENAME}"
>> +
>> + find "${DEBDIR}"/"${distro}" -name '*\.deb' | while read package; do
>> # NOTE: due to packages stored by reprepro are not modified, we can
>> # use search by filename to check if package is already in repo. In
>> # addition, md5sums are compared to ensure that the package is the
>> @@ -38,7 +40,7 @@ populate_base_apt() {
>> "${package}"
>> done
>>
>> - find "${DEBSRCDIR}"/"${DISTRO}" -name '*\.dsc' | while read package; do
>> + find "${DEBSRCDIR}"/"${distro}" -name '*\.dsc' | while read package; do
>> repo_add_srcpackage "${REPO_BASE_DIR}"/"${BASE_DISTRO}" \
>> "${REPO_BASE_DB_DIR}"/"${BASE_DISTRO}" \
>> "${BASE_DISTRO_CODENAME}" \
>>
>>
>
> Ah, another DISTRO-override issue! Hope we don't have more of that type
> in the core. Maybe worth to scan again and check.
>
> Thanks for analyzing and (likely) fixing!
>
sbuild-chroot-host is indeed working now, but there are likely more
cases with (custom) DISTRO vs. BASE_DISTRO:
ERROR: isar-bootstrap-target-1.0-r0 do_bootstrap: ExecutionError('/build/tmp/work/iot2050-debian-arm64/isar-bootstrap-target/1.0-r0/temp/run.do_bootstrap.21764', 1, None, None)
ERROR: Logfile of failure stored in: /build/tmp/work/iot2050-debian-arm64/isar-bootstrap-target/1.0-r0/temp/log.do_bootstrap.21764
Log data follows:
| DEBUG: Executing python function sstate_task_prefunc
| DEBUG: Python function sstate_task_prefunc finished
| DEBUG: Executing shell function do_bootstrap
| W: qemu-debootstrap is deprecated. Please use regular debootstrap directly
| I: Running command: debootstrap --verbose --variant=minbase --include=locales --no-check-gpg --arch=arm64 --components=main,contrib,non-free bullseye /build/tmp/work/iot2050-debian-arm64/isar-bootstrap-target/1.0-r0/rootfs file:///build/tmp/deploy/base-apt/iot2050-debian/apt/debian
| I: Retrieving InRelease
| I: Retrieving Release
| I: Retrieving Packages
| I: Validating Packages
| I: Resolving dependencies of required packages...
| I: Resolving dependencies of base packages...
| I: Checking component main on file:///build/tmp/deploy/base-apt/iot2050-debian/apt/debian...
| E: Couldn't find these debs: apt
| WARNING: exit code 1 from a shell command.
| ERROR: ExecutionError('/build/tmp/work/iot2050-debian-arm64/isar-bootstrap-target/1.0-r0/temp/run.do_bootstrap.21764', 1, None, None)
ERROR: Task (/build/../work/isar/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb:do_bootstrap) failed with exit code '1'
Jan
--
Siemens AG, Technology
Competence Center Embedded Linux
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Offline build broken with sbuild?
2022-11-23 15:49 ` Jan Kiszka
@ 2022-11-23 16:17 ` Jan Kiszka
2022-11-24 6:57 ` Uladzimir Bely
0 siblings, 1 reply; 8+ messages in thread
From: Jan Kiszka @ 2022-11-23 16:17 UTC (permalink / raw)
To: Uladzimir Bely, isar-users
On 23.11.22 16:49, Jan Kiszka wrote:
> On 22.11.22 14:11, Jan Kiszka wrote:
>> On 22.11.22 13:15, Uladzimir Bely wrote:
>>> In the email from Monday, 21 November 2022 10:36:43 +03 user Uladzimir Bely
>>> wrote:
>>>>
>>>> I'll try to check more things (e.g. Isar revision mentioned in the bug;
>>>> complete image build, etc), but at the first glance sbuild-chroot-host looks
>>>> to be cached OK, at least with the latest Isar.
>>>>
>>>
>>> So, I've got the following result for now:
>>>
>>> - In Isar itself cached cross-build works well (while both HOST_DISTRO and
>>> DISTRO are equal). The only exception: raspberry targets, while DISTRO and
>>> HOST_DISTRO are different by nature.
>>>
>>> - In meta-iot2050, cached cross-build doesn't work for both "sbuild" and "pre-
>>> sbuild" cases. Evend "isar-bootstrap-host" task doesn't work for cached build.
>>>
>>> I've applied the following patch for now and it at least fixed sbuild-chroot-
>>> host task for cached build. Waiting for full build finished and will update
>>> you with the result.
>>>
>>> diff --git a/meta/recipes-devtools/base-apt/base-apt.bb b/meta/recipes-
>>> devtools/base-apt/base-apt.bb
>>> index 506a28f..316960d 100644
>>> --- a/meta/recipes-devtools/base-apt/base-apt.bb
>>> +++ b/meta/recipes-devtools/base-apt/base-apt.bb
>>> @@ -13,7 +13,9 @@ KEYFILES ?= ""
>>> BASE_REPO_FEATURES ?= ""
>>>
>>> populate_base_apt() {
>>> - find "${DEBDIR}"/"${DISTRO}" -name '*\.deb' | while read package; do
>>> + distro="${BASE_DISTRO}-${BASE_DISTRO_CODENAME}"
>>> +
>>> + find "${DEBDIR}"/"${distro}" -name '*\.deb' | while read package; do
>>> # NOTE: due to packages stored by reprepro are not modified, we can
>>> # use search by filename to check if package is already in repo. In
>>> # addition, md5sums are compared to ensure that the package is the
>>> @@ -38,7 +40,7 @@ populate_base_apt() {
>>> "${package}"
>>> done
>>>
>>> - find "${DEBSRCDIR}"/"${DISTRO}" -name '*\.dsc' | while read package; do
>>> + find "${DEBSRCDIR}"/"${distro}" -name '*\.dsc' | while read package; do
>>> repo_add_srcpackage "${REPO_BASE_DIR}"/"${BASE_DISTRO}" \
>>> "${REPO_BASE_DB_DIR}"/"${BASE_DISTRO}" \
>>> "${BASE_DISTRO_CODENAME}" \
>>>
>>>
>>
>> Ah, another DISTRO-override issue! Hope we don't have more of that type
>> in the core. Maybe worth to scan again and check.
>>
>> Thanks for analyzing and (likely) fixing!
>>
>
> sbuild-chroot-host is indeed working now, but there are likely more
> cases with (custom) DISTRO vs. BASE_DISTRO:
>
> ERROR: isar-bootstrap-target-1.0-r0 do_bootstrap: ExecutionError('/build/tmp/work/iot2050-debian-arm64/isar-bootstrap-target/1.0-r0/temp/run.do_bootstrap.21764', 1, None, None)
> ERROR: Logfile of failure stored in: /build/tmp/work/iot2050-debian-arm64/isar-bootstrap-target/1.0-r0/temp/log.do_bootstrap.21764
> Log data follows:
> | DEBUG: Executing python function sstate_task_prefunc
> | DEBUG: Python function sstate_task_prefunc finished
> | DEBUG: Executing shell function do_bootstrap
> | W: qemu-debootstrap is deprecated. Please use regular debootstrap directly
> | I: Running command: debootstrap --verbose --variant=minbase --include=locales --no-check-gpg --arch=arm64 --components=main,contrib,non-free bullseye /build/tmp/work/iot2050-debian-arm64/isar-bootstrap-target/1.0-r0/rootfs file:///build/tmp/deploy/base-apt/iot2050-debian/apt/debian
> | I: Retrieving InRelease
> | I: Retrieving Release
> | I: Retrieving Packages
> | I: Validating Packages
> | I: Resolving dependencies of required packages...
> | I: Resolving dependencies of base packages...
> | I: Checking component main on file:///build/tmp/deploy/base-apt/iot2050-debian/apt/debian...
> | E: Couldn't find these debs: apt
> | WARNING: exit code 1 from a shell command.
> | ERROR: ExecutionError('/build/tmp/work/iot2050-debian-arm64/isar-bootstrap-target/1.0-r0/temp/run.do_bootstrap.21764', 1, None, None)
> ERROR: Task (/build/../work/isar/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb:do_bootstrap) failed with exit code '1'
>
I think we have two problems here:
1. base-apt is set up in sources-list using the BASE_DISTRO, but we are
creating it using DISTRO, not only in bast-apt, also dpkg-base
2. the whole base-apt caching does not yet consider
BASE_DISTRO(DISTRO) != BASE_DISTRO(HOST_DISTRO), and that maybe even
before all the sbuild changes
The latter should break the Raspian case.
Jan
--
Siemens AG, Technology
Competence Center Embedded Linux
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Offline build broken with sbuild?
2022-11-23 16:17 ` Jan Kiszka
@ 2022-11-24 6:57 ` Uladzimir Bely
2022-11-24 7:32 ` Jan Kiszka
0 siblings, 1 reply; 8+ messages in thread
From: Uladzimir Bely @ 2022-11-24 6:57 UTC (permalink / raw)
To: isar-users, Jan Kiszka
In the email from Wednesday, 23 November 2022 19:17:31 +03 user Jan Kiszka
wrote:
> On 23.11.22 16:49, Jan Kiszka wrote:
> > On 22.11.22 14:11, Jan Kiszka wrote:
> >> On 22.11.22 13:15, Uladzimir Bely wrote:
> >>> In the email from Monday, 21 November 2022 10:36:43 +03 user Uladzimir
> >>> Bely
> >>>
> >>> wrote:
> >>>> I'll try to check more things (e.g. Isar revision mentioned in the bug;
> >>>> complete image build, etc), but at the first glance sbuild-chroot-host
> >>>> looks to be cached OK, at least with the latest Isar.
> >>>
> >>> So, I've got the following result for now:
> >>>
> >>> - In Isar itself cached cross-build works well (while both HOST_DISTRO
> >>> and
> >>> DISTRO are equal). The only exception: raspberry targets, while DISTRO
> >>> and
> >>> HOST_DISTRO are different by nature.
> >>>
> >>> - In meta-iot2050, cached cross-build doesn't work for both "sbuild" and
> >>> "pre- sbuild" cases. Evend "isar-bootstrap-host" task doesn't work for
> >>> cached build.
> >>>
> >>> I've applied the following patch for now and it at least fixed
> >>> sbuild-chroot- host task for cached build. Waiting for full build
> >>> finished and will update you with the result.
> >>>
> >>> diff --git a/meta/recipes-devtools/base-apt/base-apt.bb b/meta/recipes-
> >>> devtools/base-apt/base-apt.bb
> >>> index 506a28f..316960d 100644
> >>> --- a/meta/recipes-devtools/base-apt/base-apt.bb
> >>> +++ b/meta/recipes-devtools/base-apt/base-apt.bb
> >>> @@ -13,7 +13,9 @@ KEYFILES ?= ""
> >>>
> >>> BASE_REPO_FEATURES ?= ""
> >>>
> >>> populate_base_apt() {
> >>>
> >>> - find "${DEBDIR}"/"${DISTRO}" -name '*\.deb' | while read package;
> >>> do
> >>> + distro="${BASE_DISTRO}-${BASE_DISTRO_CODENAME}"
> >>> +
> >>> + find "${DEBDIR}"/"${distro}" -name '*\.deb' | while read package;
> >>> do
> >>>
> >>> # NOTE: due to packages stored by reprepro are not modified, we
> >>> can
> >>> # use search by filename to check if package is already in
> >>> repo. In
> >>> # addition, md5sums are compared to ensure that the package is
> >>> the
> >>>
> >>> @@ -38,7 +40,7 @@ populate_base_apt() {
> >>>
> >>> "${package}"
> >>>
> >>> done
> >>>
> >>> - find "${DEBSRCDIR}"/"${DISTRO}" -name '*\.dsc' | while read
> >>> package; do + find "${DEBSRCDIR}"/"${distro}" -name '*\.dsc' | while
> >>> read package; do>>>
> >>> repo_add_srcpackage "${REPO_BASE_DIR}"/"${BASE_DISTRO}" \
> >>>
> >>> "${REPO_BASE_DB_DIR}"/"${BASE_DISTRO}" \
> >>> "${BASE_DISTRO_CODENAME}" \
> >>
> >> Ah, another DISTRO-override issue! Hope we don't have more of that type
> >> in the core. Maybe worth to scan again and check.
> >>
> >> Thanks for analyzing and (likely) fixing!
> >
> > sbuild-chroot-host is indeed working now, but there are likely more
> > cases with (custom) DISTRO vs. BASE_DISTRO:
> >
> > ERROR: isar-bootstrap-target-1.0-r0 do_bootstrap:
> > ExecutionError('/build/tmp/work/iot2050-debian-arm64/isar-bootstrap-targe
> > t/1.0-r0/temp/run.do_bootstrap.21764', 1, None, None) ERROR: Logfile of
> > failure stored in:
> > /build/tmp/work/iot2050-debian-arm64/isar-bootstrap-target/1.0-r0/temp/lo
> > g.do_bootstrap.21764>
> > Log data follows:
> > | DEBUG: Executing python function sstate_task_prefunc
> > | DEBUG: Python function sstate_task_prefunc finished
> > | DEBUG: Executing shell function do_bootstrap
> > | W: qemu-debootstrap is deprecated. Please use regular debootstrap
> > | directly
> > | I: Running command: debootstrap --verbose --variant=minbase
> > | --include=locales --no-check-gpg --arch=arm64
> > | --components=main,contrib,non-free bullseye
> > | /build/tmp/work/iot2050-debian-arm64/isar-bootstrap-target/1.0-r0/rootf
> > | s file:///build/tmp/deploy/base-apt/iot2050-debian/apt/debian I:
> > | Retrieving InRelease
> > | I: Retrieving Release
> > | I: Retrieving Packages
> > | I: Validating Packages
> > | I: Resolving dependencies of required packages...
> > | I: Resolving dependencies of base packages...
> > | I: Checking component main on
> > | file:///build/tmp/deploy/base-apt/iot2050-debian/apt/debian... E:
> > | Couldn't find these debs: apt
> > | WARNING: exit code 1 from a shell command.
> > | ERROR:
> > | ExecutionError('/build/tmp/work/iot2050-debian-arm64/isar-bootstrap-tar
> > | get/1.0-r0/temp/run.do_bootstrap.21764', 1, None, None)>
> > ERROR: Task
> > (/build/../work/isar/meta/recipes-core/isar-bootstrap/isar-bootstrap-targ
> > et.bb:do_bootstrap) failed with exit code '1'
> I think we have two problems here:
>
> 1. base-apt is set up in sources-list using the BASE_DISTRO, but we are
> creating it using DISTRO, not only in bast-apt, also dpkg-base
>
> 2. the whole base-apt caching does not yet consider
> BASE_DISTRO(DISTRO) != BASE_DISTRO(HOST_DISTRO), and that maybe even
> before all the sbuild changes
>
> The latter should break the Raspian case.
>
> Jan
I've just send patchset that fixes case when DISTRO is changed from default
value (but still in fact remains "debian").
I set up the following configuration in local.conf:
> require conf/distro/debian-bullseye.conf
> DISTRO_NAME = "My Debian System"
> DISTRO = "my-debian"
And got the same problem with sbuild-chroot-host installation at 2nd (cached)
build. The patches solved the issue.
The case with Raspberry (target) / Debian (host) is more complex and still
doesn't work. To make cached build work in this case we at least need to
rework `base-apt` - split it onto 2 repos that are populated only from their
own original repos.
What concerns `meta-iot2050`, it still fails to do cached build, but due to
some local problems. For example, `python-absl` recipe from `meta-coral` layer
conflicts with BB_NO_NETWORK = "1" option...
--
Uladzimir Bely
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Offline build broken with sbuild?
2022-11-24 6:57 ` Uladzimir Bely
@ 2022-11-24 7:32 ` Jan Kiszka
0 siblings, 0 replies; 8+ messages in thread
From: Jan Kiszka @ 2022-11-24 7:32 UTC (permalink / raw)
To: Uladzimir Bely, isar-users
On 24.11.22 07:57, Uladzimir Bely wrote:
> In the email from Wednesday, 23 November 2022 19:17:31 +03 user Jan Kiszka
> wrote:
>> On 23.11.22 16:49, Jan Kiszka wrote:
>>> On 22.11.22 14:11, Jan Kiszka wrote:
>>>> On 22.11.22 13:15, Uladzimir Bely wrote:
>>>>> In the email from Monday, 21 November 2022 10:36:43 +03 user Uladzimir
>>>>> Bely
>>>>>
>>>>> wrote:
>>>>>> I'll try to check more things (e.g. Isar revision mentioned in the bug;
>>>>>> complete image build, etc), but at the first glance sbuild-chroot-host
>>>>>> looks to be cached OK, at least with the latest Isar.
>>>>>
>>>>> So, I've got the following result for now:
>>>>>
>>>>> - In Isar itself cached cross-build works well (while both HOST_DISTRO
>>>>> and
>>>>> DISTRO are equal). The only exception: raspberry targets, while DISTRO
>>>>> and
>>>>> HOST_DISTRO are different by nature.
>>>>>
>>>>> - In meta-iot2050, cached cross-build doesn't work for both "sbuild" and
>>>>> "pre- sbuild" cases. Evend "isar-bootstrap-host" task doesn't work for
>>>>> cached build.
>>>>>
>>>>> I've applied the following patch for now and it at least fixed
>>>>> sbuild-chroot- host task for cached build. Waiting for full build
>>>>> finished and will update you with the result.
>>>>>
>>>>> diff --git a/meta/recipes-devtools/base-apt/base-apt.bb b/meta/recipes-
>>>>> devtools/base-apt/base-apt.bb
>>>>> index 506a28f..316960d 100644
>>>>> --- a/meta/recipes-devtools/base-apt/base-apt.bb
>>>>> +++ b/meta/recipes-devtools/base-apt/base-apt.bb
>>>>> @@ -13,7 +13,9 @@ KEYFILES ?= ""
>>>>>
>>>>> BASE_REPO_FEATURES ?= ""
>>>>>
>>>>> populate_base_apt() {
>>>>>
>>>>> - find "${DEBDIR}"/"${DISTRO}" -name '*\.deb' | while read package;
>>>>> do
>>>>> + distro="${BASE_DISTRO}-${BASE_DISTRO_CODENAME}"
>>>>> +
>>>>> + find "${DEBDIR}"/"${distro}" -name '*\.deb' | while read package;
>>>>> do
>>>>>
>>>>> # NOTE: due to packages stored by reprepro are not modified, we
>>>>> can
>>>>> # use search by filename to check if package is already in
>>>>> repo. In
>>>>> # addition, md5sums are compared to ensure that the package is
>>>>> the
>>>>>
>>>>> @@ -38,7 +40,7 @@ populate_base_apt() {
>>>>>
>>>>> "${package}"
>>>>>
>>>>> done
>>>>>
>>>>> - find "${DEBSRCDIR}"/"${DISTRO}" -name '*\.dsc' | while read
>>>>> package; do + find "${DEBSRCDIR}"/"${distro}" -name '*\.dsc' | while
>>>>> read package; do>>>
>>>>> repo_add_srcpackage "${REPO_BASE_DIR}"/"${BASE_DISTRO}" \
>>>>>
>>>>> "${REPO_BASE_DB_DIR}"/"${BASE_DISTRO}" \
>>>>> "${BASE_DISTRO_CODENAME}" \
>>>>
>>>> Ah, another DISTRO-override issue! Hope we don't have more of that type
>>>> in the core. Maybe worth to scan again and check.
>>>>
>>>> Thanks for analyzing and (likely) fixing!
>>>
>>> sbuild-chroot-host is indeed working now, but there are likely more
>>> cases with (custom) DISTRO vs. BASE_DISTRO:
>>>
>>> ERROR: isar-bootstrap-target-1.0-r0 do_bootstrap:
>>> ExecutionError('/build/tmp/work/iot2050-debian-arm64/isar-bootstrap-targe
>>> t/1.0-r0/temp/run.do_bootstrap.21764', 1, None, None) ERROR: Logfile of
>>> failure stored in:
>>> /build/tmp/work/iot2050-debian-arm64/isar-bootstrap-target/1.0-r0/temp/lo
>>> g.do_bootstrap.21764>
>>> Log data follows:
>>> | DEBUG: Executing python function sstate_task_prefunc
>>> | DEBUG: Python function sstate_task_prefunc finished
>>> | DEBUG: Executing shell function do_bootstrap
>>> | W: qemu-debootstrap is deprecated. Please use regular debootstrap
>>> | directly
>>> | I: Running command: debootstrap --verbose --variant=minbase
>>> | --include=locales --no-check-gpg --arch=arm64
>>> | --components=main,contrib,non-free bullseye
>>> | /build/tmp/work/iot2050-debian-arm64/isar-bootstrap-target/1.0-r0/rootf
>>> | s file:///build/tmp/deploy/base-apt/iot2050-debian/apt/debian I:
>>> | Retrieving InRelease
>>> | I: Retrieving Release
>>> | I: Retrieving Packages
>>> | I: Validating Packages
>>> | I: Resolving dependencies of required packages...
>>> | I: Resolving dependencies of base packages...
>>> | I: Checking component main on
>>> | file:///build/tmp/deploy/base-apt/iot2050-debian/apt/debian... E:
>>> | Couldn't find these debs: apt
>>> | WARNING: exit code 1 from a shell command.
>>> | ERROR:
>>> | ExecutionError('/build/tmp/work/iot2050-debian-arm64/isar-bootstrap-tar
>>> | get/1.0-r0/temp/run.do_bootstrap.21764', 1, None, None)>
>>> ERROR: Task
>>> (/build/../work/isar/meta/recipes-core/isar-bootstrap/isar-bootstrap-targ
>>> et.bb:do_bootstrap) failed with exit code '1'
>> I think we have two problems here:
>>
>> 1. base-apt is set up in sources-list using the BASE_DISTRO, but we are
>> creating it using DISTRO, not only in bast-apt, also dpkg-base
>>
>> 2. the whole base-apt caching does not yet consider
>> BASE_DISTRO(DISTRO) != BASE_DISTRO(HOST_DISTRO), and that maybe even
>> before all the sbuild changes
>>
>> The latter should break the Raspian case.
>>
>> Jan
>
> I've just send patchset that fixes case when DISTRO is changed from default
> value (but still in fact remains "debian").
>
> I set up the following configuration in local.conf:
>
>> require conf/distro/debian-bullseye.conf
>> DISTRO_NAME = "My Debian System"
>> DISTRO = "my-debian"
>
> And got the same problem with sbuild-chroot-host installation at 2nd (cached)
> build. The patches solved the issue.
>
> The case with Raspberry (target) / Debian (host) is more complex and still
> doesn't work. To make cached build work in this case we at least need to
> rework `base-apt` - split it onto 2 repos that are populated only from their
> own original repos.
Left a comment
>
> What concerns `meta-iot2050`, it still fails to do cached build, but due to
> some local problems. For example, `python-absl` recipe from `meta-coral` layer
> conflicts with BB_NO_NETWORK = "1" option...
>
Yes, this is understood already, and a fix for meta-coral is pending.
Thanks,
Jan
--
Siemens AG, Technology
Competence Center Embedded Linux
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2022-11-24 7:32 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-11-21 6:49 Offline build broken with sbuild? Jan Kiszka
2022-11-21 7:36 ` Uladzimir Bely
2022-11-22 12:15 ` Uladzimir Bely
2022-11-22 13:11 ` Jan Kiszka
2022-11-23 15:49 ` Jan Kiszka
2022-11-23 16:17 ` Jan Kiszka
2022-11-24 6:57 ` Uladzimir Bely
2022-11-24 7:32 ` Jan Kiszka
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox