From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7016230395466219520 X-Received: by 2002:a17:906:1856:: with SMTP id w22mr3873294eje.393.1633593439640; Thu, 07 Oct 2021 00:57:19 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:6402:b34:: with SMTP id bo20ls2783009edb.0.gmail; Thu, 07 Oct 2021 00:57:18 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzj7Anbp8QBx+59JSzr3DCvsGXesQHktnkU6t/CXDYI4fPueU67gfLL5hW3CF7uXAlcmk/1 X-Received: by 2002:a05:6402:202a:: with SMTP id ay10mr4178811edb.270.1633593438697; Thu, 07 Oct 2021 00:57:18 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1633593438; cv=none; d=google.com; s=arc-20160816; b=NPRbQ5beFEMZ1hrHgFZn+VLclHWQKSQdLXMwzg+S5VjUvE2am3Xsre3ZgeTgmgSrCM QZq7g0ti5ep36uSpNhxuqbuwuae0ei2gfAz45g5CNy+OGsSI2j0DEyKF2DC8NlF44Et8 IMkiS1FFdfs7CJM2UMxd7naxO1vnM6VkR4mkeR6+7VlnrhYdC3N4P8ccJksMjExS//Cq pIqUF7DsYdYrTWoCeP9clzj/2w+2GLj7maWbQnMXK2bxMcuZEytad80XSasPNIfrbUgt jUJx797vSLmWllf4jJJHWSSPuAdddN6ULrE+hvauBBBWrrkYnTz6Tl+LITwkLiww44Iu fF1g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from; bh=Mz0EXr2fbYQcoKy9o3XfR+XStR4P84ubM82A/jg6FwQ=; b=iLz08o+6XCjaHrUPPY2XuCJpoo72CQKzhG7Ej79dKMzNCNcrhH3MLJapg0XBNsjp9e wFS+gL9Mv/V9ogVCXLEw4oktjnZe3JBVfSgQ+6lXvjjbR3QJv0DmL2ql/r1p3NK89M+P uChgul+WP6HjIEMFjsKGPftdzMq5M4U4z5/NVXLtdZVy5SHNa9IYfYaB6JYpfacolbeO Vq/JknwgHYmG3vEEz54OoimpyO5sRtKoRsXyC4pBkQNkEMWCuvx06l0BM5DtmaVLak/i jd/FwJq3jdyTjDWlxg8ux47iKv2cbdEQ0pE7b54UWC26zjDPUpBTpZFrysS2DQJTHv+u L37Q== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of adriaan.schmidt@siemens.com designates 192.35.17.2 as permitted sender) smtp.mailfrom=adriaan.schmidt@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from thoth.sbs.de (thoth.sbs.de. [192.35.17.2]) by gmr-mx.google.com with ESMTPS id r21si998058edq.2.2021.10.07.00.57.18 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 07 Oct 2021 00:57:18 -0700 (PDT) Received-SPF: pass (google.com: domain of adriaan.schmidt@siemens.com designates 192.35.17.2 as permitted sender) client-ip=192.35.17.2; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of adriaan.schmidt@siemens.com designates 192.35.17.2 as permitted sender) smtp.mailfrom=adriaan.schmidt@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: from mail1.sbs.de (mail1.sbs.de [192.129.41.35]) by thoth.sbs.de (8.15.2/8.15.2) with ESMTPS id 1977vIWN012520 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Thu, 7 Oct 2021 09:57:18 +0200 Received: from random.ppmd.siemens.net (random.ppmd.siemens.net [139.25.68.25]) by mail1.sbs.de (8.15.2/8.15.2) with ESMTP id 1977vIkU004086; Thu, 7 Oct 2021 09:57:18 +0200 From: Adriaan Schmidt To: isar-users@googlegroups.com Cc: Adriaan Schmidt Subject: [PATCH 00/10] Add sstate-cache Date: Thu, 7 Oct 2021 09:56:27 +0200 Message-Id: <20211007075637.590860-1-adriaan.schmidt@siemens.com> X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-TUID: rgqZl0BztacI This series adds the sstate-cache feature from OE to Isar. The cache holds the results of bootstrapping, rootfs generation (buildchroot, image rootfs), and deb package generation. To use the cache, the only configuration neccessary is setting SSTATE_DIR. The contents of that directory need to be preserved across bitbake invocations. One known weakness is that the package lists of cached rootfs's can run out of sync with upstream ("apt-get update" only happens at bootstrap time). But this also happens with an "old" local build dir, and is something that may be addressed elsewhere. For now, the recommendation is to frequently clear the cache (in one of our projects we run a nightly "clear&populate cache" CI job). Patches 1..5 fix (unrelated) issues that would otherwise block sstate caching, patch 6 copies files from OE, and patches 7..10 add caching to Isar. Adriaan Schmidt (10): oe imports in central location wic-img: create deploy dir rootfs: recursively depend on packages base: remove unneeded "before do_build" task dependencies dpkg: add explicit dependency to isar-apt meta: add sstate feature from oe sstate: configure sstate: add caching to isar-bootstrap sstate: add caching to rootfs sstate: add caching to debian packages meta/classes/base.bbclass | 32 +- meta/classes/dpkg-base.bbclass | 33 +- meta/classes/dpkg.bbclass | 1 + meta/classes/patch.bbclass | 5 - meta/classes/rootfs.bbclass | 26 +- meta/classes/sstate.bbclass | 1311 +++++++++++++++++ meta/classes/wic-img.bbclass | 1 + meta/conf/bitbake.conf | 10 +- meta/lib/oe/gpg_sign.py | 130 ++ meta/lib/oe/sstatesig.py | 603 ++++++++ .../isar-bootstrap/isar-bootstrap.inc | 39 +- 11 files changed, 2167 insertions(+), 24 deletions(-) create mode 100644 meta/classes/sstate.bbclass create mode 100644 meta/lib/oe/gpg_sign.py create mode 100644 meta/lib/oe/sstatesig.py -- 2.30.2