From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Fri, 28 Feb 2025 14:13:06 +0100 X-Sieve: CMU Sieve 2.4 Received: from mail-lj1-f183.google.com (mail-lj1-f183.google.com [209.85.208.183]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 51SDD5Y5022145 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 28 Feb 2025 14:13:05 +0100 Received: by mail-lj1-f183.google.com with SMTP id 38308e7fff4ca-30a4506714fsf14195451fa.0 for ; Fri, 28 Feb 2025 05:13:05 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1740748380; cv=pass; d=google.com; s=arc-20240605; b=H0ALkCkXf85dhW19vnXGA2tDyozvKpJu95LBRmH0W/Y508xEAsn7X5H2VDkijdmywH wpA42PSsqFfCu04l6FCt1vbVE6iJYWNwrVh+G5L/PgjZzMxH90p1C0TZVcyh82wNXhRu arfnCAXqF0n28YkkoRwMzDBDTfTXCkkbitCGykkEgcszwN+5z8x4D5g92+IVTAYuU1sk bsWqHHJi0f8AnKEwNzpCcaHvF3eRB2+1baBq4stbyY8kb2/OwSDd+RPf/Qf0RYDp5tCb l1z/L5i5uyugOhc8oy1sLkfRyrdLnW4BGX3irvNEB16TDdstOCL+7bf6T2TV7oPLYSxS HDww== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:user-agent :content-transfer-encoding:autocrypt:references:in-reply-to:date:cc :to:from:subject:message-id:sender:dkim-signature; bh=a0RjL9OSTHhmWWOeio3YCPryshPGI5hn5ki0asKT/68=; fh=k/l04axsUzCWjdasi9biG7i+yK1oUQVb0Rk6FUJ3s7Y=; b=i533X4673T1SmTYlC2F10AojZ/TgeDTIzpGj1aGEUpVbr+II+DxIrVE8Lw/mqCP8zO AAh6MnApYlp62vjpQ/bORRydiwqHlRkIRtQVJ3t4bbuF6pJBpY7M+rRnRmI8ihK00wBG FUY4Ou1eSp51m8EYJSdykjR560HiClGJAWVSZhbMCLi8VG7J7HQ7l2MV1oqkucNVUdGa ZZvabfHDLBOinCGh1zTB5rP18a6xIPyzpT6qjcsROlUF7T2pVsm7esHgvGEKi+ABhVxu lVng2LebKZOpdQD4swUF9ZGLS0eex/IfwhwRWOcvxlPCUtlENVJ+5wxmKBGv+vFZrI07 bCXQ==; darn=ilbers.de ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1740748380; x=1741353180; darn=ilbers.de; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:mime-version:user-agent:content-transfer-encoding :autocrypt:references:in-reply-to:date:cc:to:from:subject:message-id :sender:from:to:cc:subject:date:message-id:reply-to; bh=a0RjL9OSTHhmWWOeio3YCPryshPGI5hn5ki0asKT/68=; b=jQm1xVHExTq+HSoQ/rH0WV0LaToE8nBmKPC+Y8VmKMQHxufsF2pY77tXZLH649wbqE 8LcnQk/7exdcpcxH72OwJlpUzZs6BUw3VqyBbFm9gvBge2crL67jHxxgi91E4RxiCShg D3A0MeXfDx8/+fxI6FGHYBPa6N6kv/cflIiQHUjvKPlGLSbjs5UR9lTJ8deurlLsEe9d MHjp0ezOoamBnG15z9qJYgMlMKdeNdoFxRSD4vpLojmc7/5kNko32bl7b0X6VkY4IL0i KZA3RJ1kqGW7aY1TFCDyP1WQ4wHfH3Rrl69/Z7GCVsrjbi+9BAIVIHT5zvrm+rG602Rd eQOw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740748380; x=1741353180; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender:mime-version :user-agent:content-transfer-encoding:autocrypt:references :in-reply-to:date:cc:to:from:subject:message-id:x-beenthere :x-gm-message-state:sender:from:to:cc:subject:date:message-id :reply-to; bh=a0RjL9OSTHhmWWOeio3YCPryshPGI5hn5ki0asKT/68=; b=Qq633HKjM+QqGUwSlXD+TbO47vXpa342acUz3ajB/ZHT0c+ExJwdz0CAguLzutlanl iuVteH4raqV7SCp/9ZqUMz7SguZDQy0Z2vNWX5M2JHyoA05W9eO0E5V+hQU68X1YTpjf wbAEK7m6ipW3FfyUS2cyfe7DN2xrsuccCm+EiLDzGWapzpYEjKmRC7awMJOmcUviDFns eNSApQxJXedOy3eBHmoehBq+99XNWwJ3NO5V13BlSlHf6M3xm2IwKdbP65fF+0Ih0hwl Fypi/rlmGsx2F2LB40VBeMTCNOBJb0UeqXAzPVW87BUL6m+fpQCXZjiAuZShgTieebnQ U/gw== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AJvYcCW5hqKXsIr0/pcdHkEoNjiloAak0DacAyiKshczJSdMKgHfs413RhdY9jaK6svZA5+Lot4a@ilbers.de X-Gm-Message-State: AOJu0YzU2LVaj04ZeuyvzH494mgxC0sIr+t3emN4xUjss8yf3Hh+gwux hNej9QpRAL/76m2zdsjgr4nFoAejDDK1YCOsW6AZL8cwJLarkZ01 X-Google-Smtp-Source: AGHT+IHXvoBv7a9WqamndiDF3Ry4Twew3FGRr1RvnIou8r3+Ev2R1R1FRMxKeXqZ2YPLwOr522qwaA== X-Received: by 2002:a2e:b604:0:b0:309:22dc:6917 with SMTP id 38308e7fff4ca-30b90a825a6mr10961471fa.19.1740748379156; Fri, 28 Feb 2025 05:12:59 -0800 (PST) X-BeenThere: isar-users@googlegroups.com; h=Adn5yVFa4/hWyd0cod5ONabzoMofPpBfi6kvtl5ABeFfuShaDg== Received: by 2002:a05:651c:515:b0:30b:78cd:df7c with SMTP id 38308e7fff4ca-30b846e3e2cls63511fa.0.-pod-prod-00-eu; Fri, 28 Feb 2025 05:12:56 -0800 (PST) X-Forwarded-Encrypted: i=2; AJvYcCX08DYyMn+7NGgWSPuVvY4kYgTJWPt5G77j+bJ2u3ollCWcThQydlyFNXgas+qEu6OGs1EAEMZQHuuC@googlegroups.com X-Received: by 2002:a2e:ad0b:0:b0:309:2999:77d4 with SMTP id 38308e7fff4ca-30b84650d3dmr38046941fa.6.1740748376413; Fri, 28 Feb 2025 05:12:56 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1740748376; cv=none; d=google.com; s=arc-20240605; b=DIjV9BwIrvNdxRi2trEVoJ9KLTD4qTofglNy2SKaZ3XFKYtIL7PBTSl7GfXXhJV/va xTEr+iMhl2K2LqvdoiokqWWv7Zyx9zQ1tqY5Y/UYqcEiWaJY+pey78AUak76kE1OhEQ0 nEF3A7e7dvEbYo3Z+gHd4tgZEPARQ0C7zqODZlNNUSgKL8ahRKgh1A7JhT9nPfZoaR4a XddPdfh/gpDrwj8HLO9mtXrzdYRxsrK0SJkYWGEIHNqc5omIwziQCZf8K7ydoAJouwzM C7NcjbL/fWZYrChM7R0nCN+4L/hmaPu/mgbQtRsCn9eBZkJE52YxzniGA8jmL40mjc7T fLFA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=mime-version:user-agent:content-transfer-encoding:autocrypt :references:in-reply-to:date:cc:to:from:subject:message-id; bh=iaiRUIwFPi4Isi7P+sw4l8dz4Z9d7BkX9KcNekyRmqs=; fh=cxID2ovG8s3L83udlHclRO4+u6DmLfKj9+ApObJOhFc=; b=f1jbqna0nj9SuLM4A47M/4MLykvo9EdfgPbjOqS1jWc5kKCZkk6sUSnH6ZRvxmi5Qa 1Bfg253ttQtX6EYI0vofw5FYRm0blH+J12wfraC1IG3aeQnV9Uy04BSy5payQgdXDLYp uGS5QH8XN80YIMneBqQfFoak282hsGIctb/fDLZ4JM1AEs5xSx+6Mqc/V8SqEjYhmEJE 6sOp3ObIn+wMYac+L/WInADYjB+Hlhrw4yXloEUFQTJbbZDkhgg4etawVfjhZqKAaBOM YzdnIfYgpf0oA6JfjO7NWr3HNcUCzkZZnUdRJUpAOzJW76Y05fC31ZV27XosukLzG3NJ Sj9w==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 38308e7fff4ca-30b868323f8si2142231fa.6.2025.02.28.05.12.56 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 28 Feb 2025 05:12:56 -0800 (PST) Received-SPF: pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from [IPv6:::1] (host-80-81-17-52.static.customer.m-online.net [80.81.17.52]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 51SDCrK2022137 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 28 Feb 2025 14:12:54 +0100 Message-ID: <71c74cfd2792b317849a12f5d55185e94e263467.camel@ilbers.de> Subject: Re: [PATCH v2 0/1] deb-dl-dir: fix package source download From: Uladzimir Bely To: Benedikt Niedermayr , isar-users@googlegroups.com Cc: Jan Kiszka Date: Fri, 28 Feb 2025 16:12:53 +0300 In-Reply-To: <20250124091219.3489707-1-benedikt.niedermayr@siemens.com> References: <20250124091219.3489707-1-benedikt.niedermayr@siemens.com> Autocrypt: addr=ubely@ilbers.de; prefer-encrypt=mutual; keydata=mQENBGO2eUkBCACtT+T3OrPVSExBmqfgXT3lp9XcdxRzjYp26wezkgYjjBXaf36bxtaAf S471VoQtpar0RVeFfW7WDDdfX9ZclSj36zBQe+RVSJzoNoNQfjOXWuSHb5Z+cpAFtqBY4muxK4+ia IlLJd6CN3ejOsLHATtCeHHq8wi0z2T+KdLQO+wQRgo2hjj0Lp9pGTrKJry50HP/o7Vbdu14dOx2xq r8+wPc6SQbBIrcqaa4MqCQC00vQG7eXvo+k2MOw59FDdpMH0KR9mHgp3u/s4I+4YRBArukt9G9xz/ rsEFmxAIBC6N/a6Hzwg4puc91n7ABDsPg8Vp+X3MDraujN0dvR6OKVNtABEBAAG0IFVsYWR6aW1pc iBCZWx5IDx1YmVseUBpbGJlcnMuZGU+iQFOBBMBCAA4FiEEJqPNVhVGyk12Eh+PAUQYBM/2FkoFAm O2eUkCGwMFCwkIBwIGFQoJCAsCBBYCAwECHgECF4AACgkQAUQYBM/2FkprlAgAmuna8Hm9EyoEtTl DBGDg6Zm4ZLp5ffvZBE946h92jepDrteoxsJ7pSzJVC2HmDLa4iZUao7lLLbDsUj5x45/iLJcqBZK k3YnAxP2r6a+kI+1VVQY1pxdG1nlJAbdNzoojm/qmezNPSrqni61KVMQKsXBCWhIjSXDSM9CsBj21 a+9qaVqfxovJGTn9lgrZO+xzKQNMKZeOouJlscVuFj21P0ww3/YENiU/nMeTSuYypO76mDtAd08Jo nc3yuHa9MJGei5ixN3wT+IrGR2aL2hdw2M6NgH7sYbL2Zi4ugD6RXHJai1Bh2yvFSVqSQ+M6QOInT 4ud7wslm1XRB065dXtA== Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable User-Agent: Evolution 3.54.3 (by Flathub.org) MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2, RCVD_IN_RP_CERTIFIED,RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: ubely@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-TUID: SDT9u8pcAN1X On Fri, 2025-01-24 at 10:12 +0100, 'Benedikt Niedermayr' via isar-users wrote: > I recently wanted to download all source packages for a specific > image and encountered > that some of the source packages were missing (e.g. adduser). >=20 > I tracked the issue down to the deb-dl-dir.bbclass and found that the > "is_not_part_of_current_build" function skipped the source package > download > because the "adduser" package was not listed in the > "/var/log/dpkg.log" file. >=20 > I assume that this package has been installed during early > debootstrap > and therefore the package is not listed in the dpkg.log file. >=20 > **As discussed in V1, the issue has been introduced with mmdebstrap, > which > either doesn't create or deletes the dpkg.log file.** >=20 > I'm not sure if I got the comment in the function right: >=20 > "Since we are parsing all the debs in DEBDIR, we can to some extend > try to eliminate some debs that are not part of the current > multiconfig > build using the below method" >=20 > AFAIK we could also achieve this by running dpkg-query commands on > the target > filesystem and also catch the packages that have been installed > during bootstrap. >=20 > I'm not sure if this patch will interfere in any way with multiconfig > builds, > as mentioned in the comment above, but I think it's worth a try. >=20 >=20 > Changes since v1: > - Adapt the commit message: Make it clear that the problem > =C2=A0 has been introduced with mmdebstrap >=20 >=20 > Benedikt Niedermayr (1): > =C2=A0 deb-dl-dir: fix package source download >=20 > =C2=A0meta/classes/deb-dl-dir.bbclass | 39 ++++++++++--------------------= - > -- > =C2=A01 file changed, 11 insertions(+), 28 deletions(-) >=20 > -- > 2.34.1 >=20 Hello. Sorry for delayed response on this patchset. It was initially tested a couple of weeks ago and caused our CI to fail with. So the patchset merging was postponed for detailed debugging. With this patch, internal (full) CI fails in SstateTest.test_sstate_populate with the following error: ``` [stdlog] 2025-02-27 20:06:15,150 avocado.test INFO | NOTE: recipe hello-1.0-r0: task do_unpack: Started [stdlog] 2025-02-27 20:06:17,363 avocado.app ERROR| ERROR: mc:qemuamd64-bullseye:hello-1.0-r0 do_unpack: Fetcher failure: Fetch command export PSEUDO_DISABLED=3D1; export PATH=3D"/workspace/build/isar_ub_devel/115/isar- sstate/scripts:/workspace/build/isar_ub_devel/115/isar- sstate/bitbake/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/g ames:/sbin"; export HOME=3D"/workspace"; [stdlog] 2025-02-27 20:06:17,363 avocado.test INFO | NOTE: recipe hello-1.0-r0: task do_unpack: Failed [stdlog] 2025-02-27 20:06:17,363 avocado.app ERROR| set -e [stdlog] 2025-02-27 20:06:17,363 avocado.app ERROR| =20 schroot -r -c isar-jenkins-ecfca206-95e9-4d1e-844d-e2033288e6a6- 2125943-83ed2888-a8fa-4485-94ed-2b16dc2cbea8 -d / -u root -- =20 rm /etc/apt/sources.list.d/isar-apt.list /etc/apt/preferences.d/isar- apt [stdlog] 2025-02-27 20:06:17,363 avocado.app ERROR| =20 schroot -r -c isar-jenkins-ecfca206-95e9-4d1e-844d-e2033288e6a6- 2125943-83ed2888-a8fa-4485-94ed-2b16dc2cbea8 -d / -- =20 sh -c ' [stdlog] 2025-02-27 20:06:17,363 avocado.app ERROR| =20 set -e [stdlog] 2025-02-27 20:06:17,363 avocado.app ERROR| =20 dscfile=3D$(apt-get -y -qq --print-uris --only-source source hello | =20 cut -d " " -f2 | grep -E "\.dsc") [stdlog] 2025-02-27 20:06:17,364 avocado.app ERROR| =20 cp /downloads/deb-src/debian-bullseye/hello/* /home/builder/hello [stdlog] 2025-02-27 20:06:17,364 avocado.app ERROR| =20 cd /home/builder/hello [stdlog] 2025-02-27 20:06:17,364 avocado.app ERROR| =20 mv -f hello-1.0 hello-1.0.prev [stdlog] 2025-02-27 20:06:17,364 avocado.app ERROR| =20 dpkg-source -x "$dscfile" hello-1.0 [stdlog] 2025-02-27 20:06:17,364 avocado.app ERROR| =20 find hello-1.0.prev -mindepth 1 -maxdepth 1 -exec mv {} hello-1.0/ \; [stdlog] 2025-02-27 20:06:17,364 avocado.app ERROR| =20 rmdir hello-1.0.prev [stdlog] 2025-02-27 20:06:17,364 avocado.app ERROR| =20 ' [stdlog] 2025-02-27 20:06:17,364 avocado.app ERROR| =20 failed with exit code 1, output: [stdlog] 2025-02-27 20:06:17,364 avocado.app ERROR| cp: cannot stat '/downloads/deb-src/debian-bullseye/hello/hello_2.10- 2+isar.debian.tar.xz': No such file or directory [stdlog] 2025-02-27 20:06:17,364 avocado.app ERROR| cp: cannot stat '/downloads/deb-src/debian-bullseye/hello/hello_2.10-2+isar.dsc': No such file or directory [stdlog] 2025-02-27 20:06:17,364 avocado.app ERROR|=20 [stdlog] 2025-02-27 20:06:17,364 avocado.app ERROR| ERROR: Logfile of failure stored in: /workspace/build/isar_ub_devel/115/build- sstate/tmp/work/debian-bullseye-amd64/hello/1.0- r0/temp/log.do_unpack.2125943 [stdlog] 2025-02-27 20:06:17,370 avocado.app ERROR| ERROR: Task (mc:qemuamd64-bullseye:/workspace/build/isar_ub_devel/115/isar- sstate/meta-isar/recipes-app/hello/hello.bb:do_unpack) failed with exit code '1' ``` Recently I've got some free time to debug it and the issue is a bit confusing: - If run only SstateTest.test_sstate_populate, it passes OK - If run it after ReproTest.test_repro_unsigned - it passes OK - If run it after CcacheTest.test_ccache_rebuild - it also passes OK But when it run with more tests ("full" CI chain) - it fails. For me, it looks like some repo conflict between self-built "hello_2.10-2+isar.dsc" and Debian's "hello_2.10-2.dsc", but I still haven't figured out the exact issue. Continue debugging... --=20 Best regards, Uladzimir. --=20 You received this message because you are subscribed to the Google Groups "= isar-users" group. To unsubscribe from this group and stop receiving emails from it, send an e= mail to isar-users+unsubscribe@googlegroups.com. To view this discussion visit https://groups.google.com/d/msgid/isar-users/= 71c74cfd2792b317849a12f5d55185e94e263467.camel%40ilbers.de.