From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Fri, 17 Jul 2026 16:06:19 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-wr1-f55.google.com (mail-wr1-f55.google.com [209.85.221.55]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 66HE6JAt005066 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 17 Jul 2026 16:06:19 +0200 Received: by mail-wr1-f55.google.com with SMTP id ffacd0b85a97d-4729d2a64efsf989801f8f.0 for ; Fri, 17 Jul 2026 07:06:19 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1784297174; cv=pass; d=google.com; s=arc-20260327; b=Z8nH2AgOjPYlNFHp+6zo7L147UKiYAShgqtrkvVz3e28h1kFVaqkiSafQZhwoZM6+3 qlQ6szQhhhlgLtytNkj+NEAvsWNKBVRPxH5zNbmr6LsTue+/8aW3pVxVyHACMODLzXK+ w/G8NWMVsRPwKzDs8zBsGTqDD8WpEuow07RJu2q79yTWf4RTB71gVLUsl8u36+hOvAyE m9fL+PWP6JUavQbnkEFAVWemuYgdWTkZ8un5RZ1JMcLfl96GD4FEUPGlO6d7aWLYFnUy 2c7xVYBk03PHC8UaL9EQrLgVAY3hmNbZZEhhI1T6vDepR9tGIOey8DyGD46DBQhZzLW5 9gUA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:content-transfer-encoding :in-reply-to:from:content-language:references:cc:to:subject :user-agent:mime-version:date:message-id:sender:dkim-signature; bh=NXKBMr4bimg5GNNXB8AA8X+QpUsPyR8/QsFl0XovGM0=; fh=lYDHQ09eJl27ixSiGj8QxfgDUS7SwQWVv2pDnnWsLqc=; b=oWOR75oBQrL8era9Zp0ULlapA2vtjLeu8Fy5bBdboyn4dPyJfICDjwTiT+lasRCK73 2ZDEWcLbUTzN6ku6772qRJ3q2qtRlvNGDxDh+2pl78FGuhVj2dMq+tZEtDmFZr9J1nAY 5+JeYzDvvvHtUq1wNJ2rej9/nHx/XEUrN9xfDqVSgLBjtWYDtnvf6lv7u7IPeBw/CMeR rE4TyeFrshpjK+wTK6Ukocu0MoxnzyCm3zqzwn91H3zzH3Ir1YpSw3X6jK4xNMvaaolm GcZYYBuSS2nszWvtFDuJDaUk2aT76JoRUPZAIkhRIZJYnLsMIIEb3sQVmgXdl+cNNB/h DODw==; darn=ilbers.de ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of wzh@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=wzh@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20251104; t=1784297174; x=1784901974; darn=ilbers.de; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:content-transfer-encoding:content-type :in-reply-to:from:content-language:references:cc:to:subject :user-agent:mime-version:date:message-id:sender:from:to:cc:subject :date:message-id:reply-to:content-type; bh=NXKBMr4bimg5GNNXB8AA8X+QpUsPyR8/QsFl0XovGM0=; b=d7HWE9I9tQnoGojFkjYueSohePvjOcTfcl5iKNRKKHcQMQX6SyvH1rCq7urZF7y4F0 UD7DPaIfoufnlo215QlRooAMMZ48kiOUS9/xzVeO6EAA8Hl5OQ5miKvUzYVy4tbNUyBD BYtXhxZjL5J6wXuBeKM7OlwqoMZaME9etWk3AnfJ9VyhTX4HRbv585irUZsdoVuwDnGW Wg4+HSLm0NeIOH68XqkPHBkQxf4Jtur+eAiLQAs5PUmU0a5ud6ocfc+zZAw2GgIen/th MEwVzBKjwSSZCNJlUMGCs74uqIacAYBPeOXJpUUuoBvLTP4libwBT/q3KHxlJZMOzARN iRrQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784297174; x=1784901974; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender :content-transfer-encoding:content-type:in-reply-to:from :content-language:references:cc:to:subject:user-agent:mime-version :date:message-id:x-beenthere:x-gm-message-state:sender:from:to:cc :subject:date:message-id:reply-to:content-type; bh=NXKBMr4bimg5GNNXB8AA8X+QpUsPyR8/QsFl0XovGM0=; b=JaM38NjTCVRzXYeor2SnwSMs5O201tYs02fL/Oi4jSIPGi278kE1bont8mN6OMtZrP OydjWBR2qfYGkhuPy97qiyiSpUPldcgxa631lQt6YvlN3VAtXrKDQLgdLmuV9RXYzRoY ysmvNsCXTo0Mo4/ZTGeE2hcSFBSZGPJoW4UWpGErJpwZ5TEohnVyXrTyMfeXijm53+nq O/U0n0QD09CNR6/OGKNA86co6pyG4poOzoA1TzMCLShqrEGzZDcqfLVjAhqB+/E9B6SW FKdo7u56CFQk8lIjtLC6ZZ6sfsaug5/LG+LKO0hKX06vpE0bMhLnTlUPWE97FqxOiLGa Amtw== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AHgh+RqHw/miSS3cr+zJ93inuc0gwV5Xsa/aqchEUFu7mz6vVDTcfwXNhGF700S+ZX9Dt430QqLV@ilbers.de X-Gm-Message-State: AOJu0YyaMesfCT57z8G5P1FsEMZcL3vUybozNFbFfSx/awAfh1S7haPB 0LriT4TeLpQc8Ar8lXfYfi+khJ8L2YZaSbxWEIZtEIdBZDItQ+YUhZc0 X-Received: by 2002:a05:6000:40c8:b0:478:7019:e5ef with SMTP id ffacd0b85a97d-47f5a556612mr10999725f8f.21.1784297173773; Fri, 17 Jul 2026 07:06:13 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com; h="Aa7YSPQGSdhWvjXgiuuBLBN41Ok5P7240R1Ndc9Ra0yP2C/Z4A==" Received: by 2002:a5d:5702:0:b0:470:5711:4074 with SMTP id ffacd0b85a97d-47f5b86fb29ls561943f8f.0.-pod-prod-00-eu; Fri, 17 Jul 2026 07:06:11 -0700 (PDT) X-Forwarded-Encrypted: i=2; AHgh+RrXDiGN9ibRFB2aZS6qeIG/o9rcYo18DosNR5jFnBSWZXwvSD0l2bgc66jc/Z28Fgjq9l+aE27oVJdB@googlegroups.com X-Received: by 2002:a05:6000:186c:b0:475:f0c2:75b0 with SMTP id ffacd0b85a97d-47f623f19cdmr3463445f8f.27.1784297170977; Fri, 17 Jul 2026 07:06:10 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1784297170; cv=none; d=google.com; s=arc-20260327; b=Gkqa0E5rrXRMUy3SzaJvpBzyeCH6w7BwEeUJxbVh4chh1axCQ7j+R7RjD0gmf3xkPO KdnnWyHyx/ownWgty/JQXiwKDhXBhhketxD3+opl90VFi2ynKdPHu9B8TW+2Rf45RZn+ Ku3JeGqS0d3AJ2GONIjM/sIUOBmiL0XGTMlHcNdJ3+J+vM0ZPjcA82vNZVuh4DCYjtqa pyufsJFylDgnTEmKw5uKGNwrf/Y+K93QG9OhTwAVNg2VNaOIca1gG/oQfW7O8Yk1gox7 CSdgTpaxfIDP8Ta+VrMNdKN+yn/FchWVsA4McRRemJ6RFYas3w5GTSp/eVwouSJYdPa8 P3ww== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id; bh=eWl3O58A0HVtJ6EXH8Po+AB7D80mR3Qg8BsmXpblnU4=; fh=9FyJCdcsdoHCrd4GY44BTUFx7IdqKInKDDLgfnGyrJQ=; b=JdwxtzCzG2LaI5SYA4xyBVyRuwtVQFeX3pMj3GCjA7Q1TV8LRJVyc4Q0sFcAPLYm6J OuZKrIlP8bJQYytJmulsqP12xDojE/lf5tkUczU+lCKqFbxjtHgESk7Yh9PSXrh2pCCg Aa8OH/HcgI3VvbBIVRuUbUQPSEgpgRosS43QKQQOT2HDa46RYWSkYR9H0DQ/jjEM02JL 3XXrQDMKm+BjFUXg399gNaN85ANRB9pWTnvuUWjvmA9CdmZBd39JKxnrFKa6RvDAHcVJ yCO8uYhCNgGncIZaK/Yniccv7FjRHXDbbhR0ErlFblT+6jIa+ZYK2vOfc//Od3RYlou7 ncvQ==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of wzh@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=wzh@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id ffacd0b85a97d-47f63e93e3dsi28891f8f.3.2026.07.17.07.06.10 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 17 Jul 2026 07:06:10 -0700 (PDT) Received-SPF: pass (google.com: domain of wzh@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from [192.168.178.148] ([88.130.203.42]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 66HE69KH005060 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 17 Jul 2026 16:06:10 +0200 Message-ID: <7bee0933-5da1-4197-ba7d-f94365e62cdb@ilbers.de> Date: Fri, 17 Jul 2026 16:06:09 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH 1/1] meta: Update sstate.bbclass to v5.0.3 To: Felix Moessbauer , isar-users@googlegroups.com Cc: adriaan.schmidt@siemens.com References: <20260713142154.1547339-1-felix.moessbauer@siemens.com> Content-Language: en-US From: Zhihang Wei In-Reply-To: <20260713142154.1547339-1-felix.moessbauer@siemens.com> Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: quoted-printable X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2, RCVD_IN_RP_CERTIFIED,RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: wzh@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of wzh@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=wzh@ilbers.de Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-TUID: pmruvszNbIgs Applied to next, thanks. Zhihang On 7/13/26 16:21, 'Felix Moessbauer' via isar-users wrote: > OE-core Revision: 236ac1b43308df722a78d3aa20aef065dfae5b2b. > > Note, that this has to be done in a lockstep with the oe-libs update. > > Fixes: 50254447 ("meta: Update OE-libs to v5.0.3") > Signed-off-by: Felix Moessbauer > --- > PS: I tested this in a CI environment that failed previously. > However, I did not check if the sstate.bbclass had changes that > were isar specific (a git log did not indicate that, though). > I further did not check if other classes also need to be copied in for > v5.0.3 update. > > Best regards, > Felix > > meta/classes-global/sstate.bbclass | 256 +++++++++++++++++------------ > 1 file changed, 147 insertions(+), 109 deletions(-) > > diff --git a/meta/classes-global/sstate.bbclass b/meta/classes-global/sst= ate.bbclass > index 3513269b..76a7b596 100644 > --- a/meta/classes-global/sstate.bbclass > +++ b/meta/classes-global/sstate.bbclass > @@ -1,4 +1,10 @@ > -SSTATE_VERSION =3D "10" > +# > +# Copyright OpenEmbedded Contributors > +# > +# SPDX-License-Identifier: MIT > +# > + > +SSTATE_VERSION =3D "12" > =20 > SSTATE_ZSTD_CLEVEL ??=3D "8" > =20 > @@ -49,8 +55,6 @@ PV[vardepvalue] =3D "${PV}" > SSTATE_EXTRAPATH[vardepvalue] =3D "" > SSTATE_EXTRAPATHWILDCARD[vardepvalue] =3D "" > =20 > -# For multilib rpm the allarch packagegroup files can overwrite (in theo= ry they're identical) > -SSTATE_ALLOW_OVERLAP_FILES =3D "${DEPLOY_DIR}/licenses/" > # Avoid docbook/sgml catalog warnings for now > SSTATE_ALLOW_OVERLAP_FILES +=3D "${STAGING_ETCDIR_NATIVE}/sgml ${STAGIN= G_DATADIR_NATIVE}/sgml" > # sdk-provides-dummy-nativesdk and nativesdk-buildtools-perl-dummy over= lap for different SDKMACHINE > @@ -79,14 +83,15 @@ SSTATE_HASHEQUIV_FILEMAP ?=3D " \ > =20 > BB_HASHFILENAME =3D "False ${SSTATE_PKGSPEC} ${SSTATE_SWSPEC}" > =20 > +SSTATE_ARCHS_TUNEPKG ??=3D "${TUNE_PKGARCH}" > SSTATE_ARCHS =3D " \ > ${BUILD_ARCH} \ > ${BUILD_ARCH}_${ORIGNATIVELSBSTRING} \ > ${BUILD_ARCH}_${SDK_ARCH}_${SDK_OS} \ > ${SDK_ARCH}_${SDK_OS} \ > - ${SDK_ARCH}_${PACKAGE_ARCH} \ > + ${SDK_ARCH}_${SDK_ARCH}-${SDKPKGSUFFIX} \ > allarch \ > - ${PACKAGE_ARCH} \ > + ${SSTATE_ARCHS_TUNEPKG} \ > ${PACKAGE_EXTRA_ARCHS} \ > ${MACHINE_ARCH}" > SSTATE_ARCHS[vardepsexclude] =3D "ORIGNATIVELSBSTRING" > @@ -262,7 +267,7 @@ def sstate_install(ss, d): > overlap_allowed =3D (d.getVar("SSTATE_ALLOW_OVERLAP_FILES") or "").= split() > match =3D [] > for f in sharedfiles: > - if os.path.exists(f) and not os.path.islink(f): > + if os.path.exists(f): > f =3D os.path.normpath(f) > realmatch =3D True > for w in overlap_allowed: > @@ -272,36 +277,18 @@ def sstate_install(ss, d): > break > if realmatch: > match.append(f) > - sstate_search_cmd =3D "grep -rlF '%s' %s --exclude=3Dmas= ter.list | sed -e 's:^.*/::'" % (f, d.expand("${SSTATE_MANIFESTS}")) > + sstate_search_cmd =3D "grep -rlF '%s' %s --exclude=3Dind= ex-* | sed -e 's:^.*/::'" % (f, d.expand("${SSTATE_MANIFESTS}")) > search_output =3D subprocess.Popen(sstate_search_cmd, s= hell=3DTrue, stdout=3Dsubprocess.PIPE).communicate()[0] > if search_output: > match.append(" (matched in %s)" % search_output.de= code('utf-8').rstrip()) > else: > match.append(" (not matched to any task)") > if match: > - bb.error("The recipe %s is trying to install files into a shared= " \ > - "area when those files already exist. Those files and their ma= nifest " \ > - "location are:\n %s\nPlease verify which recipe should provid= e the " \ > - "above files.\n\nThe build has stopped, as continuing in this = scenario WILL " \ > - "break things - if not now, possibly in the future (we've seen= builds fail " \ > - "several months later). If the system knew how to recover from= this " \ > - "automatically it would, however there are several different s= cenarios " \ > - "which can result in this and we don't know which one this is.= It may be " \ > - "you have switched providers of something like virtual/kernel = (e.g. from " \ > - "linux-yocto to linux-yocto-dev), in that case you need to exe= cute the " \ > - "clean task for both recipes and it will resolve this error. I= t may be " \ > - "you changed DISTRO_FEATURES from systemd to udev or vice vers= a. Cleaning " \ > - "those recipes should again resolve this error, however switch= ing " \ > - "DISTRO_FEATURES on an existing build directory is not support= ed - you " \ > - "should really clean out tmp and rebuild (reusing sstate shoul= d be safe). " \ > - "It could be the overlapping files detected are harmless in wh= ich case " \ > - "adding them to SSTATE_ALLOW_OVERLAP_FILES may be the correct = solution. It could " \ > - "also be your build is including two different conflicting ver= sions of " \ > - "things (e.g. bluez 4 and bluez 5 and the correct solution for= that would " \ > - "be to resolve the conflict. If in doubt, please ask on the ma= iling list, " \ > - "sharing the error and filelist above." % \ > + bb.fatal("Recipe %s is trying to install files into a shared " \ > + "area when those files already exist. The files and the manife= sts listing " \ > + "them are:\n %s\n" > + "Please adjust the recipes so only one recipe provides a given= file. " % \ > (d.getVar('PN'), "\n ".join(match))) > - bb.fatal("If the above message is too much, the simpler version = is you're advised to wipe out tmp and rebuild (reusing sstate is fine). Tha= t will likely fix things in most (but not all) cases.") > =20 > if ss['fixmedir'] and os.path.exists(ss['fixmedir'] + "/fixmepath.c= md"): > sharedfiles.append(ss['fixmedir'] + "/fixmepath.cmd") > @@ -349,7 +336,7 @@ def sstate_install(ss, d): > for lock in locks: > bb.utils.unlockfile(lock) > =20 > -sstate_install[vardepsexclude] +=3D "SSTATE_ALLOW_OVERLAP_FILES STATE_MA= NMACH SSTATE_MANFILEPREFIX" > +sstate_install[vardepsexclude] +=3D "SSTATE_ALLOW_OVERLAP_FILES SSTATE_M= ANMACH SSTATE_MANFILEPREFIX" > sstate_install[vardeps] +=3D "${SSTATEPOSTINSTFUNCS}" > =20 > def sstate_installpkg(ss, d): > @@ -359,8 +346,9 @@ def sstate_installpkg(ss, d): > d.setVar("SSTATE_CURRTASK", ss['task']) > sstatefetch =3D d.getVar('SSTATE_PKGNAME') > sstatepkg =3D d.getVar('SSTATE_PKG') > + verify_sig =3D bb.utils.to_boolean(d.getVar("SSTATE_VERIFY_SIG"), Fa= lse) > =20 > - if not os.path.exists(sstatepkg): > + if not os.path.exists(sstatepkg) or (verify_sig and not os.path.exis= ts(sstatepkg + '.sig')): > pstaging_fetch(sstatefetch, d) > =20 > if not os.path.isfile(sstatepkg): > @@ -371,7 +359,7 @@ def sstate_installpkg(ss, d): > =20 > d.setVar('SSTATE_INSTDIR', sstateinst) > =20 > - if bb.utils.to_boolean(d.getVar("SSTATE_VERIFY_SIG"), False): > + if verify_sig: > if not os.path.isfile(sstatepkg + '.sig'): > bb.warn("No signature file for sstate package %s, skipping = acceleration..." % sstatepkg) > return False > @@ -715,9 +703,7 @@ def sstate_package(ss, d): > if d.getVar('SSTATE_SKIP_CREATION') =3D=3D '1': > return > =20 > - sstate_create_package =3D ['sstate_report_unihash', 'sstate_create_p= ackage'] > - if d.getVar('SSTATE_SIG_KEY'): > - sstate_create_package.append('sstate_sign_package') > + sstate_create_package =3D ['sstate_report_unihash', 'sstate_create_a= nd_sign_package'] > =20 > for f in (d.getVar('SSTATECREATEFUNCS') or '').split() + \ > sstate_create_package + \ > @@ -762,7 +748,6 @@ def pstaging_fetch(sstatefetch, d): > localdata.setVar('FILESPATH', dldir) > localdata.setVar('DL_DIR', dldir) > localdata.setVar('PREMIRRORS', mirrors) > - localdata.setVar('SRCPV', d.getVar('SRCPV')) > =20 > # if BB_NO_NETWORK is set but we also have SSTATE_MIRROR_ALLOW_NETW= ORK, > # we'll want to allow network access for the current set of fetches= . > @@ -778,6 +763,7 @@ def pstaging_fetch(sstatefetch, d): > uris +=3D ['file://{0}.sig;downloadfilename=3D{0}.sig'.format(s= statefetch)] > =20 > for srcuri in uris: > + localdata.delVar('SRC_URI') > localdata.setVar('SRC_URI', srcuri) > try: > fetcher =3D bb.fetch2.Fetch([srcuri], localdata, cache=3DFa= lse) > @@ -787,9 +773,6 @@ def pstaging_fetch(sstatefetch, d): > except bb.fetch2.BBFetchException: > pass > =20 > -pstaging_fetch[vardepsexclude] +=3D "SRCPV" > - > - > def sstate_setscene(d): > shared_state =3D sstate_state_fromvars(d) > accelerate =3D sstate_installpkg(shared_state, d) > @@ -825,21 +808,100 @@ python sstate_task_postfunc () { > } > sstate_task_postfunc[dirs] =3D "${WORKDIR}" > =20 > +# Create a sstate package > +# If enabled, sign the package. > +# Package and signature are created in a sub-directory > +# and renamed in place once created. > +python sstate_create_and_sign_package () { > + from pathlib import Path > =20 > -# > -# Shell function to generate a sstate package from a directory > -# set as SSTATE_BUILDDIR. Will be run from within SSTATE_BUILDDIR. > -# > -sstate_create_package () { > - # Exit early if it already exists > - if [ -e ${SSTATE_PKG} ]; then > - touch ${SSTATE_PKG} 2>/dev/null || true > - return > - fi > + # Best effort touch > + def touch(file): > + try: > + file.touch() > + except: > + pass > + > + def update_file(src, dst, force=3DFalse): > + if dst.is_symlink() and not dst.exists(): > + force=3DTrue > + try: > + # This relies on that src is a temporary file that can be re= named > + # or left as is. > + if force: > + src.rename(dst) > + else: > + os.link(src, dst) > + return True > + except: > + pass > + > + if dst.exists(): > + touch(dst) > + > + return False > =20 > - mkdir --mode=3D0775 -p `dirname ${SSTATE_PKG}` > - TFILE=3D`mktemp ${SSTATE_PKG}.XXXXXXXX` > + sign_pkg =3D ( > + bb.utils.to_boolean(d.getVar("SSTATE_VERIFY_SIG")) and > + bool(d.getVar("SSTATE_SIG_KEY")) > + ) > + > + sstate_pkg =3D Path(d.getVar("SSTATE_PKG")) > + sstate_pkg_sig =3D Path(str(sstate_pkg) + ".sig") > + if sign_pkg: > + if sstate_pkg.exists() and sstate_pkg_sig.exists(): > + touch(sstate_pkg) > + touch(sstate_pkg_sig) > + return > + else: > + if sstate_pkg.exists(): > + touch(sstate_pkg) > + return > =20 > + # Create the required sstate directory if it is not present. > + if not sstate_pkg.parent.is_dir(): > + with bb.utils.umask(0o002): > + bb.utils.mkdirhier(str(sstate_pkg.parent)) > + > + if sign_pkg: > + from tempfile import TemporaryDirectory > + with TemporaryDirectory(dir=3Dsstate_pkg.parent) as tmp_dir: > + tmp_pkg =3D Path(tmp_dir) / sstate_pkg.name > + d.setVar("TMP_SSTATE_PKG", str(tmp_pkg)) > + bb.build.exec_func('sstate_archive_package', d) > + > + from oe.gpg_sign import get_signer > + signer =3D get_signer(d, 'local') > + signer.detach_sign(str(tmp_pkg), d.getVar('SSTATE_SIG_KEY'),= None, > + d.getVar('SSTATE_SIG_PASSPHRASE'), armor= =3DFalse) > + > + tmp_pkg_sig =3D Path(tmp_dir) / sstate_pkg_sig.name > + if not update_file(tmp_pkg_sig, sstate_pkg_sig): > + # If the created signature file could not be copied into= place, > + # then we should not use the sstate package either. > + return > + > + # If the .sig file was updated, then the sstate package must= also > + # be updated. > + update_file(tmp_pkg, sstate_pkg, force=3DTrue) > + else: > + from tempfile import NamedTemporaryFile > + with NamedTemporaryFile(prefix=3Dsstate_pkg.name, dir=3Dsstate_p= kg.parent) as tmp_pkg_fd: > + tmp_pkg =3D tmp_pkg_fd.name > + d.setVar("TMP_SSTATE_PKG", str(tmp_pkg)) > + bb.build.exec_func('sstate_archive_package',d) > + update_file(tmp_pkg, sstate_pkg) > + # update_file() may have renamed tmp_pkg, which must exist w= hen the > + # NamedTemporaryFile() context handler ends. > + touch(Path(tmp_pkg)) > + > +} > + > +# Shell function to generate a sstate package from a directory > +# set as SSTATE_BUILDDIR. Will be run from within SSTATE_BUILDDIR. > +# The calling function handles moving the sstate package into the final > +# destination. > +sstate_archive_package () { > OPT=3D"-cS" > ZSTD=3D"zstd -${SSTATE_ZSTD_CLEVEL} -T${ZSTD_THREADS}" > # Use pzstd if available > @@ -850,42 +912,18 @@ sstate_create_package () { > # Need to handle empty directories > if [ "$(ls -A)" ]; then > set +e > - tar -I "$ZSTD" $OPT -f $TFILE * > + tar -I "$ZSTD" $OPT -f ${TMP_SSTATE_PKG} * > ret=3D$? > if [ $ret -ne 0 ] && [ $ret -ne 1 ]; then > exit 1 > fi > set -e > else > - tar -I "$ZSTD" $OPT --file=3D$TFILE --files-from=3D/dev/null > - fi > - chmod 0664 $TFILE > - # Skip if it was already created by some other process > - if [ -h ${SSTATE_PKG} ] && [ ! -e ${SSTATE_PKG} ]; then > - # There is a symbolic link, but it links to nothing. > - # Forcefully replace it with the new file. > - ln -f $TFILE ${SSTATE_PKG} || true > - elif [ ! -e ${SSTATE_PKG} ]; then > - # Move into place using ln to attempt an atomic op. > - # Abort if it already exists > - ln $TFILE ${SSTATE_PKG} || true > - else > - touch ${SSTATE_PKG} 2>/dev/null || true > + tar -I "$ZSTD" $OPT --file=3D${TMP_SSTATE_PKG} --files-from=3D/dev/nul= l > fi > - rm $TFILE > + chmod 0664 ${TMP_SSTATE_PKG} > } > =20 > -python sstate_sign_package () { > - from oe.gpg_sign import get_signer > - > - > - signer =3D get_signer(d, 'local') > - sstate_pkg =3D d.getVar('SSTATE_PKG') > - if os.path.exists(sstate_pkg + '.sig'): > - os.unlink(sstate_pkg + '.sig') > - signer.detach_sign(sstate_pkg, d.getVar('SSTATE_SIG_KEY', False), No= ne, > - d.getVar('SSTATE_SIG_PASSPHRASE'), armor=3DFalse) > -} > =20 > python sstate_report_unihash() { > report_unihash =3D getattr(bb.parse.siggen, 'report_unihash', None) > @@ -918,6 +956,8 @@ sstate_unpack_package () { > BB_HASHCHECK_FUNCTION =3D "sstate_checkhashes" > =20 > def sstate_checkhashes(sq_data, d, siginfo=3DFalse, currentcount=3D0, s= ummary=3DTrue, **kwargs): > + import itertools > + > found =3D set() > missed =3D set() > =20 > @@ -950,6 +990,7 @@ def sstate_checkhashes(sq_data, d, siginfo=3DFalse, c= urrentcount=3D0, summary=3DTrue, > sstatefile =3D d.expand("${SSTATE_DIR}/" + getsstatefile(tid, s= iginfo, d)) > =20 > if os.path.exists(sstatefile): > + oe.utils.touch(sstatefile) > found.add(tid) > bb.debug(2, "SState: Found valid sstate file %s" % sstatefi= le) > else: > @@ -977,15 +1018,19 @@ def sstate_checkhashes(sq_data, d, siginfo=3DFalse= , currentcount=3D0, summary=3DTrue, > localdata.delVar('BB_NO_NETWORK') > =20 > from bb.fetch2 import FetchConnectionCache > - def checkstatus_init(thread_worker): > - thread_worker.connection_cache =3D FetchConnectionCache() > + def checkstatus_init(): > + while not connection_cache_pool.full(): > + connection_cache_pool.put(FetchConnectionCache()) > =20 > - def checkstatus_end(thread_worker): > - thread_worker.connection_cache.close_connections() > + def checkstatus_end(): > + while not connection_cache_pool.empty(): > + connection_cache =3D connection_cache_pool.get() > + connection_cache.close_connections() > =20 > - def checkstatus(thread_worker, arg): > + def checkstatus(arg): > (tid, sstatefile) =3D arg > =20 > + connection_cache =3D connection_cache_pool.get() > localdata2 =3D bb.data.createCopy(localdata) > srcuri =3D "file://" + sstatefile > localdata2.setVar('SRC_URI', srcuri) > @@ -995,7 +1040,7 @@ def sstate_checkhashes(sq_data, d, siginfo=3DFalse, = currentcount=3D0, summary=3DTrue, > =20 > try: > fetcher =3D bb.fetch2.Fetch(srcuri.split(), localdata2, > - connection_cache=3Dthread_worker.connection_= cache) > + connection_cache=3Dconnection_cache) > fetcher.checkstatus() > bb.debug(2, "SState: Successful fetch test for %s" % sr= curi) > found.add(tid) > @@ -1005,8 +1050,11 @@ def sstate_checkhashes(sq_data, d, siginfo=3DFalse= , currentcount=3D0, summary=3DTrue, > except Exception as e: > bb.error("SState: cannot test %s: %s\n%s" % (srcuri, re= pr(e), traceback.format_exc())) > =20 > + connection_cache_pool.put(connection_cache) > + > if progress: > - bb.event.fire(bb.event.ProcessProgress(msg, len(tasklist= ) - thread_worker.tasks.qsize()), d) > + bb.event.fire(bb.event.ProcessProgress(msg, next(cnt_tas= ks_done)), d) > + bb.event.check_for_interrupts(d) > =20 > tasklist =3D [] > for tid in missed: > @@ -1016,6 +1064,8 @@ def sstate_checkhashes(sq_data, d, siginfo=3DFalse,= currentcount=3D0, summary=3DTrue, > if tasklist: > nproc =3D min(int(d.getVar("BB_NUMBER_THREADS")), len(taskl= ist)) > =20 > + ## thread-safe counter > + cnt_tasks_done =3D itertools.count(start =3D 1) > progress =3D len(tasklist) >=3D 100 > if progress: > msg =3D "Checking sstate mirror object availability" > @@ -1025,13 +1075,13 @@ def sstate_checkhashes(sq_data, d, siginfo=3DFals= e, currentcount=3D0, summary=3DTrue, > fetcherenv =3D bb.fetch2.get_fetcher_environment(d) > with bb.utils.environment(**fetcherenv): > bb.event.enable_threadlock() > - pool =3D oe.utils.ThreadedPool(nproc, len(tasklist), > - worker_init=3Dcheckstatus_init, worker_end=3Dche= ckstatus_end, > - name=3D"sstate_checkhashes-") > - for t in tasklist: > - pool.add_task(checkstatus, t) > - pool.start() > - pool.wait_completion() > + import concurrent.futures > + from queue import Queue > + connection_cache_pool =3D Queue(nproc) > + checkstatus_init() > + with concurrent.futures.ThreadPoolExecutor(max_workers= =3Dnproc) as executor: > + executor.map(checkstatus, tasklist.copy()) > + checkstatus_end() > bb.event.disable_threadlock() > =20 > if progress: > @@ -1065,7 +1115,7 @@ def sstate_checkhashes(sq_data, d, siginfo=3DFalse,= currentcount=3D0, summary=3DTrue, > bb.parse.siggen.checkhashes(sq_data, missed, found, d) > =20 > return found > -setscene_depvalid[vardepsexclude] =3D "SSTATE_EXCLUDEDEPS_SYSROOT" > +setscene_depvalid[vardepsexclude] =3D "SSTATE_EXCLUDEDEPS_SYSROOT _SSTAT= E_EXCLUDEDEPS_SYSROOT" > =20 > BB_SETSCENE_DEPVALID =3D "setscene_depvalid" > =20 > @@ -1084,7 +1134,7 @@ def setscene_depvalid(task, taskdependees, notneede= d, d, log=3DNone): > =20 > logit("Considering setscene task: %s" % (str(taskdependees[task])),= log) > =20 > - directtasks =3D ["do_populate_lic", "do_deploy_source_date_epoch", "= do_shared_workdir", "do_stash_locale", "do_gcc_stash_builddir", "do_create_= spdx"] > + directtasks =3D ["do_populate_lic", "do_deploy_source_date_epoch", "= do_shared_workdir", "do_stash_locale", "do_gcc_stash_builddir", "do_create_= spdx", "do_deploy_archives"] > =20 > def isNativeCross(x): > return x.endswith("-native") or "-cross-" in x or "-crosssdk" i= n x or x.endswith("-cross") > @@ -1120,11 +1170,6 @@ def setscene_depvalid(task, taskdependees, notneed= ed, d, log=3DNone): > if isNativeCross(taskdependees[dep][0]) and taskdependees[dep][= 1] in ['do_package_write_deb', 'do_package_write_ipk', 'do_package_write_rp= m', 'do_packagedata', 'do_package', 'do_package_qa']: > continue > =20 > - # This is due to the [depends] in useradd.bbclass complicating m= atters > - # The logic *is* reversed here due to the way hard setscene depe= ndencies are injected > - if (taskdependees[task][1] =3D=3D 'do_package' or taskdependees[= task][1] =3D=3D 'do_populate_sysroot') and taskdependees[dep][0].endswith((= 'shadow-native', 'shadow-sysroot', 'base-passwd', 'pseudo-native')) and tas= kdependees[dep][1] =3D=3D 'do_populate_sysroot': > - continue > - > # Consider sysroot depending on sysroot tasks > if taskdependees[task][1] =3D=3D 'do_populate_sysroot' and task= dependees[dep][1] =3D=3D 'do_populate_sysroot': > # Allow excluding certain recursive dependencies. If a reci= pe needs it should add a > @@ -1187,16 +1232,7 @@ python sstate_eventhandler() { > if not os.path.exists(siginfo): > bb.siggen.dump_this_task(siginfo, d) > else: > - try: > - os.utime(siginfo, None) > - except PermissionError: > - pass > - except OSError as e: > - # Handle read-only file systems gracefully > - import errno > - if e.errno !=3D errno.EROFS: > - raise e > - > + oe.utils.touch(siginfo) > } > =20 > SSTATE_PRUNE_OBSOLETEWORKDIR ?=3D "1" > @@ -1278,6 +1314,7 @@ python sstate_eventhandler_reachablestamps() { > lines.remove(r) > removed =3D removed + 1 > bb.event.fire(bb.event.ProcessProgress(msg, removed), d= ) > + bb.event.check_for_interrupts(d) > =20 > bb.event.fire(bb.event.ProcessFinished(msg), d) > =20 > @@ -1347,6 +1384,7 @@ python sstate_eventhandler_stalesstate() { > bb.utils.remove(stamp) > removed =3D removed + 1 > bb.event.fire(bb.event.ProcessProgress(msg, removed), d= ) > + bb.event.check_for_interrupts(d) > =20 > bb.event.fire(bb.event.ProcessFinished(msg), d) > } --=20 You received this message because you are subscribed to the Google Groups "= isar-users" group. To unsubscribe from this group and stop receiving emails from it, send an e= mail to isar-users+unsubscribe@googlegroups.com. To view this discussion visit https://groups.google.com/d/msgid/isar-users/= 7bee0933-5da1-4197-ba7d-f94365e62cdb%40ilbers.de.