From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7016230395466219520 X-Received: by 2002:a5d:6d01:: with SMTP id e1mr30827972wrq.410.1634042195206; Tue, 12 Oct 2021 05:36:35 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a5d:6141:: with SMTP id y1ls1923718wrt.3.gmail; Tue, 12 Oct 2021 05:36:34 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyQ+pt4GlNs/kcrjP/GxQZhdUahMi1DdK0J0ZAtA+HYCSk0qUluB9Bo3H5UsbK9ooSqumqy X-Received: by 2002:adf:a18f:: with SMTP id u15mr30413372wru.183.1634042194290; Tue, 12 Oct 2021 05:36:34 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1634042194; cv=none; d=google.com; s=arc-20160816; b=JNeNWFOINIjzcc6e5BFNM6d8v3/yrscu3iFGnNqhH5LC9+efebARMLxUiFh9EwgYiL dIdzw2xF8ORLMjmF6mvRASQR3hLl5AW0aN6Fe7dAOFyY55QQwwg1pm9xcLwDnuIYFT8+ HotPjE0jLEsBfuZB0P1Q2MMMVQwL4k7txR4MjTllF5JT0oPx7amHBTTGUMrp2DyV+5kJ mqUWfKj5R/r/0L1mbpXdi8yBBekutVrn/YDD7WzGKAAB01zyWiqBpdWxrjoJidXkaj8y F7XwgPOcD+ugMhU37XWxx3TmwbsY+QHAne0GiciWkhGnD5qNuFQprXBgN0Q3klHqg1Ik 5oiA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from; bh=gEr59S+pVYQbCgIAfeoutVUSixSLxmVBO/o80No+FGY=; b=bw1X2FH3RQgfwfSBpG3astvqGLD7OyXTdAAlvjlWDoU5FXunmk/YcMaDyIisrZuxY5 KVlzdkj30CP/yXZkFcz/jTbmR7qQ0j2t0u/pnKIW4nzy+7NGtzqU/s1K/ooStAZIx7ig OGXpPsOSwWCOC00oNfg7KQlDgT8He5zcRHdCtzsLLGYBbCkWnxADx4CN9gMNfEuEstJr 3UiAHuTfry1hD+7eO1qsQ9dpZj2R9BD0RPvyGlygj5PkW8qGZiFtDturws+nrof4Z1Qt 1rZHVGw5AXidl60lsNkkb/Z2S4tImlpTFSRPGHP7L6amP9MoImYEwa2/sZKicJxBw+pa 173g== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of adriaan.schmidt@siemens.com designates 194.138.37.40 as permitted sender) smtp.mailfrom=adriaan.schmidt@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from gecko.sbs.de (gecko.sbs.de. [194.138.37.40]) by gmr-mx.google.com with ESMTPS id l3si174219wml.2.2021.10.12.05.36.34 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 12 Oct 2021 05:36:34 -0700 (PDT) Received-SPF: pass (google.com: domain of adriaan.schmidt@siemens.com designates 194.138.37.40 as permitted sender) client-ip=194.138.37.40; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of adriaan.schmidt@siemens.com designates 194.138.37.40 as permitted sender) smtp.mailfrom=adriaan.schmidt@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: from mail1.sbs.de (mail1.sbs.de [192.129.41.35]) by gecko.sbs.de (8.15.2/8.15.2) with ESMTPS id 19CCaXiX025566 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Tue, 12 Oct 2021 14:36:33 +0200 Received: from random.ppmd.siemens.net (random.ppmd.siemens.net [139.25.68.25]) by mail1.sbs.de (8.15.2/8.15.2) with ESMTP id 19CCaXOc017377; Tue, 12 Oct 2021 14:36:33 +0200 From: Adriaan Schmidt To: isar-users@googlegroups.com Cc: Adriaan Schmidt Subject: [PATCH v2 00/10] Add sstate-cache Date: Tue, 12 Oct 2021 14:36:15 +0200 Message-Id: <20211012123625.1703627-1-adriaan.schmidt@siemens.com> X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-TUID: oVDy3C/7Z/fT This series adds the sstate-cache feature from OE to Isar. The cache holds the results of bootstrapping, rootfs generation (buildchroot, image rootfs), and deb package generation. To use the cache, the only configuration neccessary is setting SSTATE_DIR. The contents of that directory need to be preserved across bitbake invocations. One known weakness is that the package lists of cached rootfs's can run out of sync with upstream ("apt-get update" only happens at bootstrap time). But this also happens with an "old" local build dir, and is something that may be addressed elsewhere. For now, the recommendation is to frequently clear the cache (in one of our projects we run a nightly "clear&populate cache" CI job). Patches 1..5 fix (unrelated) issues that would otherwise block sstate caching, patch 6 copies files from OE, and patches 7..10 add caching to Isar. --- Changes since v1: * fix copy/paste typo in rootfs.bbclass * add mounting trick to tar rootfs (because --one-file-system does not stop at bind mounts) * have install_imager_deps also depend explicitly on isar-apt Adriaan Schmidt (10): oe imports in central location wic-img: create deploy dir rootfs: recursively depend on packages base: remove unneeded "before do_build" task dependencies dpkg: add explicit dependency to isar-apt meta: add sstate feature from oe sstate: configure sstate: add caching to isar-bootstrap sstate: add caching to rootfs sstate: add caching to debian packages meta/classes/base.bbclass | 32 +- meta/classes/dpkg-base.bbclass | 33 +- meta/classes/dpkg.bbclass | 1 + meta/classes/image-tools-extension.bbclass | 2 +- meta/classes/patch.bbclass | 5 - meta/classes/rootfs.bbclass | 29 +- meta/classes/sstate.bbclass | 1311 +++++++++++++++++ meta/classes/wic-img.bbclass | 1 + meta/conf/bitbake.conf | 10 +- meta/lib/oe/gpg_sign.py | 130 ++ meta/lib/oe/sstatesig.py | 603 ++++++++ .../isar-bootstrap/isar-bootstrap.inc | 39 +- 12 files changed, 2171 insertions(+), 25 deletions(-) create mode 100644 meta/classes/sstate.bbclass create mode 100644 meta/lib/oe/gpg_sign.py create mode 100644 meta/lib/oe/sstatesig.py -- 2.30.2