From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7168357740651216896 X-Received: by 2002:a92:c8c9:0:b0:2ff:e4b1:5436 with SMTP id c9-20020a92c8c9000000b002ffe4b15436mr13387371ilq.305.1669273047580; Wed, 23 Nov 2022 22:57:27 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:6638:cc8:b0:358:3055:3faa with SMTP id e8-20020a0566380cc800b0035830553faals144551jak.6.-pod-prod-gmail; Wed, 23 Nov 2022 22:57:27 -0800 (PST) X-Google-Smtp-Source: AA0mqf6mEbe4fsTWNstW5+/qmeV3JDA1XU7uvSQrOVGLVjzZJxQiP9GqthgY6rA4UYIdBvs6E5Cm X-Received: by 2002:a02:a695:0:b0:376:2353:5805 with SMTP id j21-20020a02a695000000b0037623535805mr14703684jam.4.1669273047021; Wed, 23 Nov 2022 22:57:27 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669273047; cv=none; d=google.com; s=arc-20160816; b=sna8KoGhUKI5VNFYZQeM3zJvX5h3FlNEc0yv9cHu+8hSBc3cYKMgmXjeyj2KYK9njq I5jKQEhP8YFLKvXnUjlT+jmEmK5/Qn88vcQ3myKpLzJpBprA9Lsv2dlNUPQ1EWxLXaZX DGo4Fbuinyn04yh4dalizaJSuKV3LB7moaEaDzYKhDqdpq179ekhyshs7Dx9zHQ/OP74 RrS7wg/WBPop/3D/x/l/YSgHcrsu1UXkiJ2qbwNn62xpOfaGZBQoFHOtoA4C1wWqeyru 4yBaZlm8Dxjnu+4D30jNom7YuUTIV+y34JDlMn/uxggNdaZv0SzDbuLtZHvohOX/OTWJ eqFQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from; bh=CRdFqbUJpf7qWKK+tPsJm1sFnkuqHtYDOqaupRMS/aM=; b=JEfNQMkRGCk5eGaCHIOgwlddIK+CKPHHzRxCv+5qnydBcNQ96/1qZZRj8qjTgCWHgd 06d0KAslMWH8lj9C0X3M1ZMzFdBGLLKqt5hH+Od1FW8A3LqS9sEEOozn+DCsTDDRzIyx 7DvEBLGS3iFbQJH6pOtBy3t1JRZrvpAR2RE9UqUCz/yfHnrQZUyQes1inmiNSrVUIus5 6bEg76FJW3/xTHTVkqASilcA7/GmxKBNAD8I1dK2/KTpTWAdfbM62vhCLbiAAqFq+rFP iV+HTcEqsQO5cUw9AfM0ch3PJTaGRSZlCzlEtPFXhRqYU1hfVMdXO5EY0vnk0cLojxPE I3Ng== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Return-Path: Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id bp7-20020a056e02348700b00302df8abfa9si21040ilb.1.2022.11.23.22.57.26 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 23 Nov 2022 22:57:26 -0800 (PST) Received-SPF: pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Received: from home.localnet (44-208-124-178-static.mgts.by [178.124.208.44] (may be forged)) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 2AO6vOqh010060 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 24 Nov 2022 07:57:25 +0100 From: Uladzimir Bely To: isar-users , Jan Kiszka Subject: Re: Offline build broken with sbuild? Date: Thu, 24 Nov 2022 09:57:19 +0300 Message-ID: <2600055.fDdHjke4Dd@home> In-Reply-To: References: <9d111e65-99ac-52fe-eb30-1605facce23c@siemens.com> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-TUID: Yg18Che9teX4 In the email from Wednesday, 23 November 2022 19:17:31 +03 user Jan Kiszka wrote: > On 23.11.22 16:49, Jan Kiszka wrote: > > On 22.11.22 14:11, Jan Kiszka wrote: > >> On 22.11.22 13:15, Uladzimir Bely wrote: > >>> In the email from Monday, 21 November 2022 10:36:43 +03 user Uladzimir > >>> Bely > >>> > >>> wrote: > >>>> I'll try to check more things (e.g. Isar revision mentioned in the bug; > >>>> complete image build, etc), but at the first glance sbuild-chroot-host > >>>> looks to be cached OK, at least with the latest Isar. > >>> > >>> So, I've got the following result for now: > >>> > >>> - In Isar itself cached cross-build works well (while both HOST_DISTRO > >>> and > >>> DISTRO are equal). The only exception: raspberry targets, while DISTRO > >>> and > >>> HOST_DISTRO are different by nature. > >>> > >>> - In meta-iot2050, cached cross-build doesn't work for both "sbuild" and > >>> "pre- sbuild" cases. Evend "isar-bootstrap-host" task doesn't work for > >>> cached build. > >>> > >>> I've applied the following patch for now and it at least fixed > >>> sbuild-chroot- host task for cached build. Waiting for full build > >>> finished and will update you with the result. > >>> > >>> diff --git a/meta/recipes-devtools/base-apt/base-apt.bb b/meta/recipes- > >>> devtools/base-apt/base-apt.bb > >>> index 506a28f..316960d 100644 > >>> --- a/meta/recipes-devtools/base-apt/base-apt.bb > >>> +++ b/meta/recipes-devtools/base-apt/base-apt.bb > >>> @@ -13,7 +13,9 @@ KEYFILES ?= "" > >>> > >>> BASE_REPO_FEATURES ?= "" > >>> > >>> populate_base_apt() { > >>> > >>> - find "${DEBDIR}"/"${DISTRO}" -name '*\.deb' | while read package; > >>> do > >>> + distro="${BASE_DISTRO}-${BASE_DISTRO_CODENAME}" > >>> + > >>> + find "${DEBDIR}"/"${distro}" -name '*\.deb' | while read package; > >>> do > >>> > >>> # NOTE: due to packages stored by reprepro are not modified, we > >>> can > >>> # use search by filename to check if package is already in > >>> repo. In > >>> # addition, md5sums are compared to ensure that the package is > >>> the > >>> > >>> @@ -38,7 +40,7 @@ populate_base_apt() { > >>> > >>> "${package}" > >>> > >>> done > >>> > >>> - find "${DEBSRCDIR}"/"${DISTRO}" -name '*\.dsc' | while read > >>> package; do + find "${DEBSRCDIR}"/"${distro}" -name '*\.dsc' | while > >>> read package; do>>> > >>> repo_add_srcpackage "${REPO_BASE_DIR}"/"${BASE_DISTRO}" \ > >>> > >>> "${REPO_BASE_DB_DIR}"/"${BASE_DISTRO}" \ > >>> "${BASE_DISTRO_CODENAME}" \ > >> > >> Ah, another DISTRO-override issue! Hope we don't have more of that type > >> in the core. Maybe worth to scan again and check. > >> > >> Thanks for analyzing and (likely) fixing! > > > > sbuild-chroot-host is indeed working now, but there are likely more > > cases with (custom) DISTRO vs. BASE_DISTRO: > > > > ERROR: isar-bootstrap-target-1.0-r0 do_bootstrap: > > ExecutionError('/build/tmp/work/iot2050-debian-arm64/isar-bootstrap-targe > > t/1.0-r0/temp/run.do_bootstrap.21764', 1, None, None) ERROR: Logfile of > > failure stored in: > > /build/tmp/work/iot2050-debian-arm64/isar-bootstrap-target/1.0-r0/temp/lo > > g.do_bootstrap.21764> > > Log data follows: > > | DEBUG: Executing python function sstate_task_prefunc > > | DEBUG: Python function sstate_task_prefunc finished > > | DEBUG: Executing shell function do_bootstrap > > | W: qemu-debootstrap is deprecated. Please use regular debootstrap > > | directly > > | I: Running command: debootstrap --verbose --variant=minbase > > | --include=locales --no-check-gpg --arch=arm64 > > | --components=main,contrib,non-free bullseye > > | /build/tmp/work/iot2050-debian-arm64/isar-bootstrap-target/1.0-r0/rootf > > | s file:///build/tmp/deploy/base-apt/iot2050-debian/apt/debian I: > > | Retrieving InRelease > > | I: Retrieving Release > > | I: Retrieving Packages > > | I: Validating Packages > > | I: Resolving dependencies of required packages... > > | I: Resolving dependencies of base packages... > > | I: Checking component main on > > | file:///build/tmp/deploy/base-apt/iot2050-debian/apt/debian... E: > > | Couldn't find these debs: apt > > | WARNING: exit code 1 from a shell command. > > | ERROR: > > | ExecutionError('/build/tmp/work/iot2050-debian-arm64/isar-bootstrap-tar > > | get/1.0-r0/temp/run.do_bootstrap.21764', 1, None, None)> > > ERROR: Task > > (/build/../work/isar/meta/recipes-core/isar-bootstrap/isar-bootstrap-targ > > et.bb:do_bootstrap) failed with exit code '1' > I think we have two problems here: > > 1. base-apt is set up in sources-list using the BASE_DISTRO, but we are > creating it using DISTRO, not only in bast-apt, also dpkg-base > > 2. the whole base-apt caching does not yet consider > BASE_DISTRO(DISTRO) != BASE_DISTRO(HOST_DISTRO), and that maybe even > before all the sbuild changes > > The latter should break the Raspian case. > > Jan I've just send patchset that fixes case when DISTRO is changed from default value (but still in fact remains "debian"). I set up the following configuration in local.conf: > require conf/distro/debian-bullseye.conf > DISTRO_NAME = "My Debian System" > DISTRO = "my-debian" And got the same problem with sbuild-chroot-host installation at 2nd (cached) build. The patches solved the issue. The case with Raspberry (target) / Debian (host) is more complex and still doesn't work. To make cached build work in this case we at least need to rework `base-apt` - split it onto 2 repos that are populated only from their own original repos. What concerns `meta-iot2050`, it still fails to do cached build, but due to some local problems. For example, `python-absl` recipe from `meta-coral` layer conflicts with BB_NO_NETWORK = "1" option... -- Uladzimir Bely