From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6771003121820762112 X-Received: by 2002:a05:651c:321:: with SMTP id b1mr19437176ljp.62.1576497015423; Mon, 16 Dec 2019 03:50:15 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:ac2:554a:: with SMTP id l10ls1421764lfk.14.gmail; Mon, 16 Dec 2019 03:50:14 -0800 (PST) X-Google-Smtp-Source: APXvYqyIF0qI/xUoXgVnz3FL9tbwnb8wBBUGVhYdLodHHrBkbAj3hWMYyEHTcxLQS6kP0sr69Co4 X-Received: by 2002:a19:4a12:: with SMTP id x18mr16344732lfa.158.1576497014679; Mon, 16 Dec 2019 03:50:14 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1576497014; cv=none; d=google.com; s=arc-20160816; b=J19mD8n6jhBteKE85E8gIfSF951dgPwtRM+qSzb1IrWSwzxWmf20HJpOF1+mW3GDYa WUcV/14tGa1BYt/QewF5ML8ZJCBluT81wj9xGbGAvTb3CvRne8/qvk5gVW7iLXz6vdeq qmHKZRMGybJGXYANoNDPNEt0o5vb1Qf2ZGK1JExiM7aQojquZ3dJmrRBVLW1TUI2oeb9 MPlr00qo80MO7ImUUFWCasviOBw0w7jqiUtEhPMgd70Ln/W9zz68WgmnmTfveB8Ejepf cIpKRxVugrts9w+HB2oMno97clJGr0iU2njSjL6CnrzoQMe+sohoUd6Qv388jVFE4qWv lGtQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from; bh=4H1ftNKnC/Wgv3wc2HXQRn9AYNYmwISE0ecT9N5TW0I=; b=ZCDu73KT0JV3fjisDEhrLTd1Zc/sMOU15KsC2scqZIZcYin5JpKjdUPYfty9EQRZJ8 ysWN36kjOjjwWI6Q9IlO4iLb1ZSMRxbDzBumv9dw9N55zfOQsavYxBFSV7xKyaw+vfcm FbjUg2oFTHwMyUJGBsrE3uhY8XsZdcMJ15WrsTgvvPJFZfgsdgnerTi8gKYxEIRYnMNT baHcZIoXNxNIVdfsRwY6Th5THEh+cKFCV5KpbQivsnLVjVX7FPGRRq2Heb6wAekRBjlZ J8HDToyR3+EJzTjyYfIRb3LW+v2+O8JKa9CDDlnxrD9xrcfqOdEs8CTgXCJ/zItNlOe2 V47Q== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.14 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from david.siemens.de (david.siemens.de. [192.35.17.14]) by gmr-mx.google.com with ESMTPS id o193si701431lff.4.2019.12.16.03.50.14 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 16 Dec 2019 03:50:14 -0800 (PST) Received-SPF: pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.14 as permitted sender) client-ip=192.35.17.14; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.14 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: from mail2.sbs.de (mail2.sbs.de [192.129.41.66]) by david.siemens.de (8.15.2/8.15.2) with ESMTPS id xBGBoDCl024808 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 16 Dec 2019 12:50:13 +0100 Received: from md1za8fc.ad001.siemens.net ([139.25.69.101]) by mail2.sbs.de (8.15.2/8.15.2) with ESMTP id xBGBoDWL009084; Mon, 16 Dec 2019 12:50:13 +0100 From: Henning Schild To: isar-users@googlegroups.com Cc: Jan Kiszka , Baurzhan Ismagulov , chombourger@gmail.com, Henning Schild Subject: [PATCHv2 00/25] base-apt-rework Date: Mon, 16 Dec 2019 12:49:46 +0100 Message-Id: <20191216115011.17664-1-henning.schild@siemens.com> X-Mailer: git-send-email 2.24.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-TUID: b35IfV9bnw56 From: Henning Schild That one had to wait for some time. I finally managed to have a look at it again and came up with a v2. Sorry that one is really big and a lot of things happened since v1. The most significant change to last time it p23, which switches the way the offline cache works. Unfortunately this is turned out pretty big, splitting it into readably and working chunks was not reasonable. Everything up to p19 was already sent once, at least roughly ... I hope a v3 will come faster and we will have better tracking of changes between 2 and 3. changes since v1: - pull the base-apt building to the front of the build chain - performance optimze all that copying - add import/export of bootstrap step - rebased on current next - exclude isar-apt in deb_dl_dir_export - use find exec + for deb_dl_dir_import - fixed "==" operator in deps.sh in p8 - fixed typo in p2 message - fixed variable scope to local in repository functions Henning Schild (25): repository: new class to deal with repos dpkg-base: add download caching of apt:// downloads meta: dpkg-base: convert "exit" into "return 0" base-apt: change the sources.list to also offer deb-src base-apt: add deb-src packages as well base-apt: do not skip gpg check when it is signed ci: conf: add "hello" to the sample config and every build meta: split all apt-get invocations into download and execution meta: create DL_DIR support for all apt-get downloaded .debs meta: import DL_DIR debs before apt-get download steps meta: include DL_DIR deb import/export into bootstrap base-apt: populate from DEBDIR as well base-apt: drop the "apt_cache" feature base-apt: do not copy debs directly out of rootfss anymore base-apt: rework base-apt population base-apt: move class "base-apt-helper" into only user CI: include "isar-disable-apt-cache" into all CI images CI: include "cowsay" into default build to test dpkg-gbp CI: set BB_NO_NETWORK for cached rebuild meta: repository: implement repo_contains_package and use it in base_apt meta: repository: simplify the matching of packages meta: deb-dl-dir: do not cache debs from isar-apt base-apt: pull base-apt population to the front of the build chain meta: base-apt: remove isar-apt check from population meta: deb-dl-dir: only export newly downloaded files doc/user_manual.md | 9 +- meta-isar/conf/local.conf.sample | 10 +- meta/classes/base-apt-helper.bbclass | 57 ---------- meta/classes/deb-dl-dir.bbclass | 38 +++++++ meta/classes/dpkg-base.bbclass | 47 ++++---- meta/classes/dpkg-gbp.bbclass | 5 + meta/classes/dpkg.bbclass | 7 +- meta/classes/image-cache-extension.bbclass | 36 ------- meta/classes/image-locales-extension.bbclass | 2 +- meta/classes/image-tools-extension.bbclass | 7 ++ meta/classes/image.bbclass | 3 +- meta/classes/repository.bbclass | 100 ++++++++++++++++++ meta/classes/rootfs.bbclass | 30 +++--- meta/conf/bitbake.conf | 6 +- .../isar-bootstrap/files/base-apt-sources | 1 - .../isar-bootstrap/isar-bootstrap.inc | 21 ++-- meta/recipes-devtools/base-apt/base-apt.bb | 92 ++++++++++------ .../base-apt/files/distributions.in | 3 - .../buildchroot/files/deps.sh | 37 ++++--- .../isar-apt/files/distributions.in | 3 - meta/recipes-devtools/isar-apt/isar-apt.bb | 26 ++--- scripts/ci_build.sh | 13 ++- 22 files changed, 321 insertions(+), 232 deletions(-) delete mode 100644 meta/classes/base-apt-helper.bbclass create mode 100644 meta/classes/deb-dl-dir.bbclass delete mode 100644 meta/classes/image-cache-extension.bbclass create mode 100644 meta/classes/repository.bbclass delete mode 100644 meta/recipes-core/isar-bootstrap/files/base-apt-sources delete mode 100644 meta/recipes-devtools/base-apt/files/distributions.in delete mode 100644 meta/recipes-devtools/isar-apt/files/distributions.in -- 2.24.1