[PATCH] Adjust do_apt_fetch dependency to crossbuild

[PATCH] Adjust do_apt_fetch dependency to crossbuild

[Jan Kiszka]

From: Jan Kiszka <jan.kiszka@siemens.com>

So far, do_apt_fetch has a hard-coded dependency on buildchroot-target.
But that is not accurate. In a crossbuild scenario, it rather depends on
buildchroot-host. Adjust that by updating the automatic dependency
setting of buildchroot.bbclass from do_prepare_build to do_apt_fetch.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 meta/classes/buildchroot.bbclass | 2 +-
 meta/classes/dpkg-base.bbclass   | 2 --
 2 files changed, 1 insertion(+), 3 deletions(-)

diff --git a/meta/classes/buildchroot.bbclass b/meta/classes/buildchroot.bbclass
index a87e144..64ac450 100644
--- a/meta/classes/buildchroot.bbclass
+++ b/meta/classes/buildchroot.bbclass
@@ -17,7 +17,7 @@ python __anonymous() {
         dep = "buildchroot-host:do_build"
         rootfs = d.getVar('BUILDCHROOT_HOST_DIR', True)

-    d.setVarFlag('do_prepare_build', 'depends', dep)
+    d.setVarFlag('do_apt_fetch', 'depends', dep)
     d.setVar('BUILDCHROOT_DIR', rootfs)
 }

diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass
index d50d565..4c6e02a 100644
--- a/meta/classes/dpkg-base.bbclass
+++ b/meta/classes/dpkg-base.bbclass
@@ -22,8 +22,6 @@ addtask patch after do_adjust_git before do_dpkg_build

 SRC_APT ?= ""

-do_apt_fetch[depends] = "buildchroot-target:do_build"
-
 do_apt_fetch() {
 	if [ -z "${@d.getVar("SRC_APT", True).strip()}" ]; then
 		exit
--

2.16.4

Re: [PATCH] Adjust do_apt_fetch dependency to crossbuild

[Baurzhan Ismagulov]

On Sat, Aug 31, 2019 at 12:38:22PM +0200, Jan Kiszka wrote:
> So far, do_apt_fetch has a hard-coded dependency on buildchroot-target.
> But that is not accurate. In a crossbuild scenario, it rather depends on
> buildchroot-host. Adjust that by updating the automatic dependency
> setting of buildchroot.bbclass from do_prepare_build to do_apt_fetch.

Thanks, applied to next.

SRC_APT should ideally be covered by a test case.

With kind regards,
Baurzhan.

Re: [PATCH] Adjust do_apt_fetch dependency to crossbuild

[Jan Kiszka]

On 12.09.19 15:49, Baurzhan Ismagulov wrote:
> On Sat, Aug 31, 2019 at 12:38:22PM +0200, Jan Kiszka wrote:
>> So far, do_apt_fetch has a hard-coded dependency on buildchroot-target.
>> But that is not accurate. In a crossbuild scenario, it rather depends on
>> buildchroot-host. Adjust that by updating the automatic dependency
>> setting of buildchroot.bbclass from do_prepare_build to do_apt_fetch.
> 
> Thanks, applied to next.
> 
> SRC_APT should ideally be covered by a test case.

meta-isar/recipes-app/hello/

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

Re: [PATCH] Adjust do_apt_fetch dependency to crossbuild

[Baurzhan Ismagulov]

On Fri, Sep 13, 2019 at 08:22:55AM +0200, Jan Kiszka wrote:
> > SRC_APT should ideally be covered by a test case.
> 
> meta-isar/recipes-app/hello/

Thanks. Turns out it is not enabled in local.conf.sample. I've tried to quickly
enable it, but the build failed with:

E: Can not find version '2.10-99+isar' of package 'hello'
E: Unable to find a source package for hello

Not sure why it tries to fetch 2.10-99+isar while run.do_apt_fetch specifies
apt-get -y source hello=2.10.

With kind regards,
Baurzhan.

Re: [PATCH] Adjust do_apt_fetch dependency to crossbuild

[Jan Kiszka]

On 16.09.19 22:46, Baurzhan Ismagulov wrote:
> On Fri, Sep 13, 2019 at 08:22:55AM +0200, Jan Kiszka wrote:
>>> SRC_APT should ideally be covered by a test case.
>>
>> meta-isar/recipes-app/hello/
> 
> Thanks. Turns out it is not enabled in local.conf.sample. I've tried to quickly
> enable it, but the build failed with:
> 
> E: Can not find version '2.10-99+isar' of package 'hello'
> E: Unable to find a source package for hello
> 
> Not sure why it tries to fetch 2.10-99+isar while run.do_apt_fetch specifies
> apt-get -y source hello=2.10.

Which target exactly fails? Just tried one successfully, and we are also using 
the very same pattern that hello_2.10.bb and hello.inc implement in production 
for other packages.

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

Re: [PATCH] Adjust do_apt_fetch dependency to crossbuild

[Baurzhan Ismagulov]

On Mon, Sep 16, 2019 at 10:58:42PM +0200, Jan Kiszka wrote:
> Which target exactly fails? Just tried one successfully, and we are also
> using the very same pattern that hello_2.10.bb and hello.inc implement in
> production for other packages.

It succeeds for amd64 and arm64, and fails for de0-nano-soc-stretch:

egrep -n " bitbake |hello-2.10-r0: task do_apt_fetch|Running task.*hello-isar.bb:do_apt_fetch" log
134:+ bitbake multiconfig:qemuarm-stretch:isar-image-base multiconfig:qemuarm-buster:isar-image-base multiconfig:qemuarm-bullseye:isar-image-base multiconfig:qemuarm64-stretch:isar-image-base multiconfig:qemuamd64-stretch:isar-image-base multiconfig:de0-nano-soc-stretch:isar-image-base multiconfig:rpi-stretch:isar-image-base
826:NOTE: recipe hello-2.10-r0: task do_apt_fetch: Started
838:NOTE: Running task 314 of 896 (multiconfig:qemuamd64-stretch:/workspace/build/isar_ibr_devel_fast/10/meta-isar/recipes-app/hello-isar/hello-isar.bb:do_apt_fetch)
840:NOTE: recipe hello-2.10-r0: task do_apt_fetch: Succeeded
1295:NOTE: Running task 483 of 896 (multiconfig:qemuarm64-stretch:/workspace/build/isar_ibr_devel_fast/10/meta-isar/recipes-app/hello-isar/hello-isar.bb:do_apt_fetch)
1308:NOTE: recipe hello-2.10-r0: task do_apt_fetch: Started
1344:NOTE: recipe hello-2.10-r0: task do_apt_fetch: Succeeded
1420:NOTE: recipe hello-2.10-r0: task do_apt_fetch: Started
1439:NOTE: recipe hello-2.10-r0: task do_apt_fetch: Failed
1441:NOTE: Running task 532 of 896 (multiconfig:de0-nano-soc-stretch:/workspace/build/isar_ibr_devel_fast/10/meta-isar/recipes-app/hello-isar/hello-isar.bb:do_apt_fetch)

With kind regards,
Baurzhan.

Re: [PATCH] Adjust do_apt_fetch dependency to crossbuild

[Jan Kiszka]

On 16.09.19 23:19, Baurzhan Ismagulov wrote:
> On Mon, Sep 16, 2019 at 10:58:42PM +0200, Jan Kiszka wrote:
>> Which target exactly fails? Just tried one successfully, and we are also
>> using the very same pattern that hello_2.10.bb and hello.inc implement in
>> production for other packages.
> 
> It succeeds for amd64 and arm64, and fails for de0-nano-soc-stretch:
> 
> egrep -n " bitbake |hello-2.10-r0: task do_apt_fetch|Running task.*hello-isar.bb:do_apt_fetch" log
> 134:+ bitbake multiconfig:qemuarm-stretch:isar-image-base multiconfig:qemuarm-buster:isar-image-base multiconfig:qemuarm-bullseye:isar-image-base multiconfig:qemuarm64-stretch:isar-image-base multiconfig:qemuamd64-stretch:isar-image-base multiconfig:de0-nano-soc-stretch:isar-image-base multiconfig:rpi-stretch:isar-image-base
> 826:NOTE: recipe hello-2.10-r0: task do_apt_fetch: Started
> 838:NOTE: Running task 314 of 896 (multiconfig:qemuamd64-stretch:/workspace/build/isar_ibr_devel_fast/10/meta-isar/recipes-app/hello-isar/hello-isar.bb:do_apt_fetch)
> 840:NOTE: recipe hello-2.10-r0: task do_apt_fetch: Succeeded
> 1295:NOTE: Running task 483 of 896 (multiconfig:qemuarm64-stretch:/workspace/build/isar_ibr_devel_fast/10/meta-isar/recipes-app/hello-isar/hello-isar.bb:do_apt_fetch)
> 1308:NOTE: recipe hello-2.10-r0: task do_apt_fetch: Started
> 1344:NOTE: recipe hello-2.10-r0: task do_apt_fetch: Succeeded
> 1420:NOTE: recipe hello-2.10-r0: task do_apt_fetch: Started
> 1439:NOTE: recipe hello-2.10-r0: task do_apt_fetch: Failed
> 1441:NOTE: Running task 532 of 896 (multiconfig:de0-nano-soc-stretch:/workspace/build/isar_ibr_devel_fast/10/meta-isar/recipes-app/hello-isar/hello-isar.bb:do_apt_fetch)
> 

Sharing issue, similar to linux-libc-dev: Once you deployed a self-built version 
of hello that matches this build, apt will try to pull the sources of that 
higher prioritized isar-apt version, rather than from upstream. Not sure if we 
can easily define that "apt://" excludes isar-apt, but that would solve this. It 
probably also makes sense.

Consequently, building only de0-nano-soc-stretch:hello in a clean built env 
works fine.

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

Re: [PATCH] Adjust do_apt_fetch dependency to crossbuild

[Henning Schild]

Am Thu, 12 Sep 2019 15:49:52 +0200
schrieb Baurzhan Ismagulov <ibr@radix50.net>:

> On Sat, Aug 31, 2019 at 12:38:22PM +0200, Jan Kiszka wrote:
> > So far, do_apt_fetch has a hard-coded dependency on
> > buildchroot-target. But that is not accurate. In a crossbuild
> > scenario, it rather depends on buildchroot-host. Adjust that by
> > updating the automatic dependency setting of buildchroot.bbclass
> > from do_prepare_build to do_apt_fetch.  
> 
> Thanks, applied to next.

I have patches for that since i introduced the feature. But "apt-get
source" does not work for offline builds and CI never successfully ran.
I did not investigate why but the patch can be found here:

https://github.com/henning-schild-work/isar/commit/879fe27596c7bf77e10623c750d0829379779a61

Henning

> SRC_APT should ideally be covered by a test case.
> 
> With kind regards,
> Baurzhan.
>