From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7319406651323187200 X-Received: by 2002:a05:600c:14b:b0:40e:6227:175e with SMTP id w11-20020a05600c014b00b0040e6227175emr269082wmm.61.1705563599051; Wed, 17 Jan 2024 23:39:59 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:600c:1395:b0:40d:5b7e:eada with SMTP id u21-20020a05600c139500b0040d5b7eeadals516662wmf.0.-pod-prod-07-eu; Wed, 17 Jan 2024 23:39:57 -0800 (PST) X-Google-Smtp-Source: AGHT+IH5gZauW9s8sPTLnpKiEBb2UCOvvxFKDdghFHwokZvp1bd5J7SiHZVYzhVufOWfDYlfucjJ X-Received: by 2002:a05:600c:164a:b0:40d:3112:a2d7 with SMTP id o10-20020a05600c164a00b0040d3112a2d7mr228283wmn.187.1705563596776; Wed, 17 Jan 2024 23:39:56 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1705563596; cv=none; d=google.com; s=arc-20160816; b=MkioB4IvK6AsLaXegsCG7VmJcUjQgtq1Ut4DglOoyTqiPR6icY81oioNs8seLlZPf7 xJaamboBYzSIzc4UdLL38thbPPQgFEWraDwj9+I2vUSbbOcbP6q8uQRQ7IDR+YF2V5cI ssYGtVY+6e7mLJAwewUzjuWFCf/qtTKuLBi3QkRd7SFWvh+0bhAAZeOckoG+doIvd/MO 6/3znlKUwXXhhXYsmeCIQlZ/omaTl57btbcRxJrJDc0BjMCqBFAYH5mHAjcINQnPKKk1 z4JI2usexpSgEOdy9LtSjGWqVGpEiEGMxEeqRO9V/k4k9GA1CetA8PByWuwiYlvOpLzB QIhw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:user-agent:content-transfer-encoding:references :in-reply-to:date:to:from:subject:message-id; bh=s3KFc9FnMqLaNyiaOW/9YbEVPMqw+OvFKZ08c8lWTwE=; fh=IxbeHM4vWxalf1XNf2rnGcQL+r1v0Lr0C6ICsGNNkYw=; b=OL2191NZ8JU6C4LKjmxCIpQ77XnyZTBKukn79tzafFBhbo87TrlU/SujHgkcfY6Kdq 1v1kgmx0zOiFAob5wuGzcdsl/Q7kb7fEzmtwZu8o0S9DeS8BLsXp0zKeECvMyRr+GpUj VzROCdp2Y5WIhHKiIjBGdqwV1ijLvBocOT8HnY3ocgMpSDz2FGIWOSgjuQfdqm0occap HP+elifCBtfwvJEwX5lk4aGNYsp77y1urAgBq/oBtJaZfY0aMlccQr0p5j+fGy3cX6vV eIuzLqRmih0U7iYd7kVeL5q9Y8Pw2sZfgEgAl7nS6O0rMjFxggqE6Z1nItf/kxGLC+Ig scVw== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Return-Path: Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id c6-20020a05600c0a4600b0040e657c8b72si432411wmq.2.2024.01.17.23.39.56 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 17 Jan 2024 23:39:56 -0800 (PST) Received-SPF: pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Received: from [127.0.0.1] (host-80-81-17-52.static.customer.m-online.net [80.81.17.52]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 40I7ds5O005294 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 18 Jan 2024 08:39:55 +0100 Message-ID: <4e906c530ad336deeb58623179bebacd024b34df.camel@ilbers.de> Subject: Re: [PATCH v5] dpkg: Restore support for replacing pre-installed packages in sbuild-chroot From: Uladzimir Bely To: Jan Kiszka , isar-users Date: Thu, 18 Jan 2024 10:39:53 +0300 In-Reply-To: References: <028f69ac16ec3769875a03d6ec5d062271e3752c.camel@ilbers.de> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable User-Agent: Evolution 3.50.3 (by Flathub.org) MIME-Version: 1.0 X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-TUID: H9TjsGnaAT2S On Thu, 2024-01-18 at 07:59 +0100, Jan Kiszka wrote: > On 18.01.24 07:34, Uladzimir Bely wrote: > > On Thu, 2024-01-18 at 09:28 +0300, Uladzimir Bely wrote: > > > On Tue, 2024-01-16 at 09:22 +0100, Jan Kiszka wrote: > > > > From: Jan Kiszka > > > >=20 > > > > During the migration to sbuild, support for using self-built > > > > packages > > > > in > > > > the build environment was lost if those were already part of > > > > the > > > > sbuild-chroot. This restores it by adding --apt-distupgrade to > > > > the > > > > sbuild call. But that is not enough because sbuild will only > > > > upgrade > > > > packages from already configured sources, not those specified > > > > via > > > > --extra-repository. We therefore have to switch back to > > > > configuring > > > > isar-apt during sbuild-chroot creation. > > > >=20 > > > > As rootfs_configure_isar_apt configures the isar repo under > > > > /isar-apt, we bind-mount the one in /home/builder/${PN} to that > > > > folder. > > > > Another difference is that we now need to run apt-get update > > > > explicitly, > > > > but only for isar-apt. > > > >=20 > > > > And because the sbuild-chroots are now left behind with isar- > > > > apt > > > > configured, the configuration in imager_run can be removed > > > > (credits > > > > to > > > > Srinuvasan Arjunan). > > > >=20 > > > > Signed-off-by: Jan Kiszka > > > > --- > > > >=20 > > > > Changes in v5: > > > > =C2=A0- also use dpkg_schroot_create_configs for apt_fetch and > > > > apt_unpack > > > > =C2=A0=C2=A0 (while I wasn't able to reproduce the issue, I think t= his > > > > should > > > > =C2=A0=C2=A0 resolve the CI problems) > > > >=20 > > >=20 > > > Unfortunately, the picture in full CI didn't change. Even worse, > > > one > > > more job (NoCrossTest.test_nocross has failed with the similar > > > error. > > >=20 > > > tmp/work/debian-bullseye-amd64/hello/1.0- > > > r0/temp/log.do_apt_unpack.1785250: > > > ``` > > > DEBUG: Executing shell function do_apt_unpack > > > cp: cannot stat '/downloads/deb-src/debian- > > > bullseye/hello/hello_2.10- > > > 2+isar.debian.tar.xz': No such file or directory > > > cp: cannot stat '/downloads/deb-src/debian- > > > bullseye/hello/hello_2.10- > > > 2+isar.dsc': No such file or directory > > > ``` > > >=20 > > > I'm continuing debugging the issue. > > >=20 > >=20 > > Just one addition: previously (in v4) builds were failing in > > hello:apt_fetch, not they fail in hello:apt_unpack. > >=20 >=20 > Logs? The pattern above looks rather different from the previous > issue. >=20 > > Also, now repro tests failed earlier - during base_apt:do_cache. > > E.g: > >=20 > > tmp_repro_signed/work/debian-bullseye-arm64/base-apt/1.0- > > r0/temp/log.do_cache.2901916: > >=20 > > ``` > > ... > > Exporting indices... > > Error opening '/build/isar_ub_devel/359/downloads/deb-src/debian- > > bullseye/hello/hello_2.10-2+isar.dsc': No such file or directory > > There have been errors! > > WARNING: exit code 254 from a shell command. > > DEBUG: Python function do_cache finished > > ``` > >=20 >=20 > How can this be reproduced locally in kas? That would be essential in > order to debug. >=20 > Jan >=20 It's hard to reproduce the issue in some "short" form, but at least I found one quite fast way - steps int it are similar to what is do in unsigned repro test. In this test we do thre builds: normal build, cached build (after tmp cleanup), one more normal build (without cleanup). The steps below do not fail with current 'next' branch of Isar, but produce hello:apt_fetch error with 'next + patch v5': ``` ./kas/kas-container menu # Just save and exit ./kas/kas-container shell # The rest done in kas shell # 1st build: bitbake hello # Build is OK # 2nd cached build: sudo rm -rf sstate-cache/ tmp/ echo 'ISAR_USE_CACHED_BASE_REPO =3D "1"' >> conf/local.conf echo 'BB_NO_NETWORK =3D "1"' >> conf/local.conf bitbake hello # Build is OK # 3rd build with no cleanup echo 'ISAR_USE_CACHED_BASE_REPO =3D "0"' >> conf/local.conf echo 'BB_NO_NETWORK =3D "0"' >> conf/local.conf bitbake hello # Build fails in hello.bb:do_apt_fetch ``` I guess, other errors we observe in CI (e.g., in "do_apt_unpack") also have the similar nature, coming from sharing working directories between different tests that use the same targets but with different local.conf settings.