From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Wed, 16 Apr 2025 16:13:13 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-il1-f187.google.com (mail-il1-f187.google.com [209.85.166.187]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 53GEDBnF010478 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 16 Apr 2025 16:13:12 +0200 Received: by mail-il1-f187.google.com with SMTP id e9e14a558f8ab-3d5da4fb5e0sf65194225ab.2 for ; Wed, 16 Apr 2025 07:13:12 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1744812786; cv=pass; d=google.com; s=arc-20240605; b=kXkO21Y6pQqKh77OM3dS3uIToHxp17K9V7YkXcq0G9dv0zH8QyjVdOrQAdjF68gKRZ NuXQjJwJDEHdEOubx/i2QbeYMDXkaU/DEgeua2Bgk471XShO+cXwh4pZ0Z1ViqJcMx8E vfXE7d61WMdEvmu54DuHtPIsFemvlP1tJ+0zRLHElq6za7A34s7khcv1VUjqfqST7SY3 jTmDRP79icAOMAm8VW873TqHxHCrM/dgj3QTZQRIeYaDunnQGJcrcEXFWD6jAz026Fef oQgBOz0rVU6fl+K0UzPNrtrEqvufXESDjM8lCd9dtwSyC8o/iHcnD0JD/x5CLAiWcLrx Oxpg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:in-reply-to:content-disposition :mime-version:references:mail-followup-to:message-id:subject:cc:to :from:date:sender:dkim-signature; bh=JBkletLkkzVyqYzOSM0T8cFwmTcAJXZf/dVM03J50qo=; fh=QcyXJaA3SnArEMntYBXMWbjGZdRarU3/g43PMGhpEg0=; b=Q0lwuvgj9Jksbm35S5WoNfLAczXSK4lDFAuaZ7pMgNqDQCeyqTEPB90/elAnrfYh41 OaUfU6zZUwDVXFAa34klU3JQzq2p3ELvHYtuKe2R2iBIMk/AXP1KtXN8Yb4VpKJQYn1j eJi4SjStxCR5CCoMfO1dEF4B+fNj2G4GSbmd6kMT4BTfgGJBqvBsv2snIwTVB6xEg13N HuRfTG7QVr3axisK4MVCemC7+qM+iB6b8mhT4+RHhMCw9vIgAz7OLw0YTWs3TwVlxIEA V2Ujg9fFwPa5zMNKJBWkAcaeRB4N1ItVM2Rqu6s+hz5fOaNqdiIhFt2a4sI08fKnIQ2b XHeg==; darn=ilbers.de ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of ibr@radix50.net designates 85.214.156.166 as permitted sender) smtp.mailfrom=ibr@radix50.net DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1744812786; x=1745417586; darn=ilbers.de; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:in-reply-to:content-disposition:mime-version :references:mail-followup-to:message-id:subject:cc:to:from:date :sender:from:to:cc:subject:date:message-id:reply-to; bh=JBkletLkkzVyqYzOSM0T8cFwmTcAJXZf/dVM03J50qo=; b=VbQjQjEUN0C6lXxNcEQr8tb5Ky88nUMp4/MWwPZoKF57MPT8fG2UA92BQfnunHivv/ 8pIc7+Ov9fHqjq9BhKWg4wBEHKH+/nAu4fHMqjXQs2CnLwPEHHdyik7WJoWEFu2hSuuy 4WE/vmtTs68n+s6SRTzyK/w4lJpjTJ3GvMo8bYOBZv9/wZ+okczKDwyI57UZk4nBUCkL 7ykFYVjd8miZjHV143y7glC3Vx3mBGHKnK+Km07P1BIETRm554AiKiz3QN8XMpfE6rZl qdaEjN7RSjfTC+K7toDwIOtJ4SQErw2ZTIPSX9LEqDQn6dpMMasiuLYxfay47DdZpB3a M2VA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1744812786; x=1745417586; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender:in-reply-to :content-disposition:mime-version:references:mail-followup-to :message-id:subject:cc:to:from:date:x-beenthere:x-gm-message-state :sender:from:to:cc:subject:date:message-id:reply-to; bh=JBkletLkkzVyqYzOSM0T8cFwmTcAJXZf/dVM03J50qo=; b=A7DevCFFehy1KedOCox22VJ6mIwFkxLJOJllxK2wO5dpsfBdY0m+tt95oADCLP1O2J TQdHUKjJmfbSLBW4dxWjwUUGR6CreQtiJO6ZqGYM7ZuSq9lOnREKBJr3gD0DZZDTx+uZ 2Zwf70lIrL86fB8r4/RUltVlwuq9s8eCbHsLTSThtb1LyMoxnOm7LjKErm+35amirHSy UZ7PBuebkuagpoD96e9LpKP2ntOzCeAHakxo9+IiTypfNKXKoTLiL88lDioHZUfPOMv4 Gs2IFt1Kspodlrf7PqdZrZRaLR8Wwv+yeTGuju+o0IarCnAVt2rsHCmmuMfk8Vb78w8t S3zQ== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AJvYcCUcoPF39CPW2dSbPLQORwwk3D6a/IgW7beUcruYMuBvKckxTS/F/+HsXInxoTNAme5lVw+i@ilbers.de X-Gm-Message-State: AOJu0YysKX8p5ayDwywIf8RGmZZ5223e5SsJ0CISezN1kpfUuVxq0uaK YDnjxLZ139bNvhzPph+/Y5aGFEYxnXdzP90Pk+RvfFXvRCiNBwAw X-Google-Smtp-Source: AGHT+IGeaMGIioCi+0Vbvg4YfNZGM1LxBiJ/65ar2pjyBr+ekI5I8lDa5htXxy3CPE/LMNGnX885HA== X-Received: by 2002:a05:6e02:1fe4:b0:3d2:bac3:b45f with SMTP id e9e14a558f8ab-3d815afd969mr19462505ab.4.1744812785474; Wed, 16 Apr 2025 07:13:05 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com; h=ARLLPAIrCOrdpK9PbNMoqm3mQEMD6NtV42yQaXqoR76Bx7NNnA== Received: by 2002:a05:6e02:3604:b0:3d6:d838:8f38 with SMTP id e9e14a558f8ab-3d7e3d14630ls2266045ab.1.-pod-prod-06-us; Wed, 16 Apr 2025 07:13:04 -0700 (PDT) X-Received: by 2002:a05:6602:4c0c:b0:85b:505a:7e01 with SMTP id ca18e2360f4ac-861c5082d9amr197886639f.5.1744812783843; Wed, 16 Apr 2025 07:13:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1744812783; cv=none; d=google.com; s=arc-20240605; b=KN9XyghE21C5JNYodfGUYkWAqxyobKlnEAgNl8Q39didLTFxDNNKnZL+II+IzeymL3 kWdIjxAiwVPelQLM/pbJhuLTIXar0BwEx6bPUC7YA0v/XRaIhbN5ssx6Osf/+g5fHtXi rAEJSVhJUKX5L5AoyyO/gPvaLHom/mE4bKJDatLZGhao88Fe1qNZu03uMbhSNSFCbFQR T44Z+t0hGrfw8j8KmF2Yjv+n+4hOMrdCdWvGxAtPAd5qYTOHhFgN6iwL1QpXv8sgvH96 z8HX5nFkR98Pp7ClSd29cpRg0nri01WzQDQDC25tpWfrO+V1mAcIswZjZCukdAhoXKYh eniA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=in-reply-to:content-disposition:mime-version:references :mail-followup-to:message-id:subject:cc:to:from:date; bh=e/cchWMCFFqYb4WjV5XE/LcomttN7LBxeWF+uytUlMk=; fh=wTfLAPiT8iy63hccwzcZuLf6IB/ehBDfYt5BOmR3yPE=; b=YgDskJ5oUf+SbUJ+2morqQ3Pqg5ju+w1jbvh0GRXyF4aN2m7Wkds9gJiYTXxciWmAx HDLBFk370A0q0poygVPLPDR9r8HU0R69d0/XXyLT1dMTiOBlyBE8sMvGHAl+O86P1Rh7 UU7Le/k9Pi6mh1FH3OLMwcLCEH9nVY9HFPDr6Cvjrr4Al9Y4CZ1C7mr/rDoER8tzENyH FJlRzn5OFLOBGwZr0XO8m06knOpwzx9L4Xff1XGmNVMTB9fWOn38ozlHyrv52/DzJrKQ IpPC4+irIUcMi+NXNHRYCm9X9EOJ/erL2KnCMIXxI+g/AJJ+HlmN6dmS9tGi1fk5/4Uy XQmQ==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of ibr@radix50.net designates 85.214.156.166 as permitted sender) smtp.mailfrom=ibr@radix50.net Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 8926c6da1cb9f-4f505dd8147si143816173.6.2025.04.16.07.13.03 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 16 Apr 2025 07:13:03 -0700 (PDT) Received-SPF: pass (google.com: domain of ibr@radix50.net designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from abai.de ([88.130.203.42]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 53GECxtj010466 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 16 Apr 2025 16:13:01 +0200 Date: Wed, 16 Apr 2025 16:12:59 +0200 From: Baurzhan Ismagulov To: isar-users@googlegroups.com Cc: Adriaan Schmidt Subject: Re: [PATCH] fix: add currenttask-guards to SSTATE(CREATE|POSTINST)FUNCS Message-ID: Mail-Followup-To: isar-users@googlegroups.com, Adriaan Schmidt References: <20250411143304.384348-1-adriaan.schmidt@siemens.com> MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Disposition: inline In-Reply-To: <20250411143304.384348-1-adriaan.schmidt@siemens.com> X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2, RCVD_IN_RP_CERTIFIED,RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: ibr@radix50.net X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of ibr@radix50.net designates 85.214.156.166 as permitted sender) smtp.mailfrom=ibr@radix50.net Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-TUID: /cVWApVEwIMT On 2025-04-11 16:33, 'Adriaan Schmidt' via isar-users wrote: > We use SSTATECREATEFUNCS and SSTATEPOSTINSTFUNCS to insert > code for preparation/postprocessing of sstate artifacts. > As it turns out, those are not per-job, but global, so if > a recipe has multiple sstate-cacheable tasks, all of those > functions are executed, and we need to make sure to bail out > of all unintended calls. Thanks Adriaan for the patch, we've had a look and have a few comments (numbering for easier referencing): 1. For this use case, I'd like to have a testcase. Would you like to submit one or provide quick hints what to add to trigger the behavior? We've started checking some simple cases but haven't reproduced it so far. > --- a/meta/classes/base.bbclass > +++ b/meta/classes/base.bbclass > @@ -204,6 +204,15 @@ def isar_export_ccache(d): > else: > os.environ['CCACHE_DISABLE'] = '1' > > +isar_check_current_task() { > + if [ "do_${BB_CURRENTTASK}" = "$1" ]; then > + return 0 > + else > + echo "skipping because current task is not $1" > + return 1 > + fi > +} 2. Could we name "isar_check_current_task" to reflect when it returns what? E.g., "fail_if_task_not" or something. (Minor but easy and helpful) 3. Technically, this function itself doesn't skip, so the message might be misleading if the caller makes a mistake. (Just a comment; I don't have a better wording on the spot) > --- a/meta/classes/dpkg-base.bbclass > +++ b/meta/classes/dpkg-base.bbclass > @@ -184,6 +184,7 @@ SSTATECREATEFUNCS += "dpkg_build_sstate_prepare" > SSTATEPOSTINSTFUNCS += "dpkg_build_sstate_finalize" > > dpkg_build_sstate_prepare() { > + (isar_check_current_task "do_dpkg_build" || return) || true 4. If isar_check_current_task succeeds, the execution should continue. If it fails, we return. Why is || true needed? (Just double-checking) Coincidentally, many || true occurrences needed fixing in the past, so I'm not impartial towards those :) , although it's just a handier form of "if". 5. I wonder if there are ways to remove hard-coding of "do_dpkg_build" into the function (something similar to overrides, flags, etc.). (Just brainstorming) (I've always wanted to separate hard-coded cases like "i386", "stretch", etc. from the code = separate generic mechanism from the specific machine, distro, etc. -- haven't come to that so far) 6. Just for my understanding, what are some examples where *CREATE* and *POSTINST* are useful? 7. Given that this touches central classes and sstate bugs could silently produce incorrect deliverables, what are the opinions regarding applying this after the 0.11 release? The use case doesn't sound (to me, in any case) like it would affect the most downstreams, and they are using next anyway. With kind regards, Baurzhan > > Signed-off-by: Adriaan Schmidt > --- > meta/classes/base.bbclass | 9 +++++++++ > meta/classes/dpkg-base.bbclass | 2 ++ > meta/classes/rootfs.bbclass | 2 ++ > meta/recipes-core/isar-bootstrap/isar-bootstrap.inc | 2 ++ > meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc | 2 ++ > 5 files changed, 17 insertions(+) > > diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass > index c730eec9..d5a3142b 100644 > --- a/meta/classes/base.bbclass > +++ b/meta/classes/base.bbclass > @@ -204,6 +204,15 @@ def isar_export_ccache(d): > else: > os.environ['CCACHE_DISABLE'] = '1' > > +isar_check_current_task() { > + if [ "do_${BB_CURRENTTASK}" = "$1" ]; then > + return 0 > + else > + echo "skipping because current task is not $1" > + return 1 > + fi > +} > + > do_fetch[dirs] = "${DL_DIR}" > do_fetch[file-checksums] = "${@bb.fetch.get_checksum_file_list(d)}" > do_fetch[vardeps] += "SRCREV" > diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass > index 4468a49a..7f2dcd3f 100644 > --- a/meta/classes/dpkg-base.bbclass > +++ b/meta/classes/dpkg-base.bbclass > @@ -184,6 +184,7 @@ SSTATECREATEFUNCS += "dpkg_build_sstate_prepare" > SSTATEPOSTINSTFUNCS += "dpkg_build_sstate_finalize" > > dpkg_build_sstate_prepare() { > + (isar_check_current_task "do_dpkg_build" || return) || true > # this runs in SSTATE_BUILDDIR, which will be deleted automatically > if [ -n "$(find ${WORKDIR} -maxdepth 1 -name '*.deb' -print -quit)" ]; then > cp -f ${WORKDIR}/*.deb -t . > @@ -191,6 +192,7 @@ dpkg_build_sstate_prepare() { > } > > dpkg_build_sstate_finalize() { > + (isar_check_current_task "do_dpkg_build_setscene" || return) || true > # this runs in SSTATE_INSTDIR > if [ -n "$(find . -maxdepth 1 -name '*.deb' -print -quit)" ]; then > mv -f ./*.deb -t ${WORKDIR}/ > diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass > index cd8fd118..aed50a91 100644 > --- a/meta/classes/rootfs.bbclass > +++ b/meta/classes/rootfs.bbclass > @@ -437,6 +437,7 @@ SSTATE_TAR_ATTR_FLAGS ?= "--xattrs --xattrs-include='*'" > > # the rootfs is owned by root, so we need some sudoing to pack and unpack > rootfs_install_sstate_prepare() { > + (isar_check_current_task "do_rootfs_install" || return) || true > # this runs in SSTATE_BUILDDIR, which will be deleted automatically > # tar --one-file-system will cross bind-mounts to the same filesystem, > # so we use some mount magic to prevent that > @@ -450,6 +451,7 @@ rootfs_install_sstate_prepare() { > do_rootfs_install_sstate_prepare[lockfiles] = "${REPO_ISAR_DIR}/isar.lock" > > rootfs_install_sstate_finalize() { > + (isar_check_current_task "do_rootfs_install_setscene" || return) || true > # this runs in SSTATE_INSTDIR > # - after building the rootfs, the tar won't be there, but we also don't need to unpack > # - after restoring from cache, there will be a tar which we unpack and then delete > diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc > index 08d61a84..a3ab28ca 100644 > --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc > +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc > @@ -219,12 +219,14 @@ SSTATECREATEFUNCS += "bootstrap_sstate_prepare" > SSTATEPOSTINSTFUNCS += "bootstrap_sstate_finalize" > > bootstrap_sstate_prepare() { > + (isar_check_current_task "do_bootstrap" || return) || true > # this runs in SSTATE_BUILDDIR, which will be deleted automatically > sudo cp -a "${WORKDIR}/rootfs.tar.zst" ./bootstrap.tar.zst > sudo chown $(id -u):$(id -g) bootstrap.tar.zst > } > > bootstrap_sstate_finalize() { > + (isar_check_current_task "do_bootstrap_setscene" || return) || true > # this runs in SSTATE_INSTDIR > # we should restore symlinks after using tar > if [ -f bootstrap.tar.zst ]; then > diff --git a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc > index 931f6f13..44775bc5 100644 > --- a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc > +++ b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc > @@ -222,12 +222,14 @@ SSTATECREATEFUNCS += "bootstrap_sstate_prepare" > SSTATEPOSTINSTFUNCS += "bootstrap_sstate_finalize" > > bootstrap_sstate_prepare() { > + (isar_check_current_task "do_bootstrap" || return) || true > # this runs in SSTATE_BUILDDIR, which will be deleted automatically > sudo cp -a "${WORKDIR}/rootfs.tar.zst" ./bootstrap.tar.zst > sudo chown $(id -u):$(id -g) bootstrap.tar.zst > } > > bootstrap_sstate_finalize() { > + (isar_check_current_task "do_bootstrap_setscene" || return) || true > # this runs in SSTATE_INSTDIR > # we should restore symlinks after using tar > if [ -f bootstrap.tar.zst ]; then > -- > 2.39.5 -- You received this message because you are subscribed to the Google Groups "isar-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com. To view this discussion visit https://groups.google.com/d/msgid/isar-users/Z_-663VgehHkoczP%40abai.de.