From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6788114222392803328 X-Received: by 2002:a05:6402:7c7:: with SMTP id u7mr656362edy.214.1584779518722; Sat, 21 Mar 2020 01:31:58 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:6402:17f6:: with SMTP id t22ls4896644edy.6.gmail; Sat, 21 Mar 2020 01:31:58 -0700 (PDT) X-Google-Smtp-Source: ADFU+vtCYQjS90H3AnLOzi9654K7PwLBaWVsva5D3t5XpuWQjJjoaaXjqq8LDq+BleAah9C5/T8w X-Received: by 2002:a50:f743:: with SMTP id j3mr12113930edn.22.1584779518132; Sat, 21 Mar 2020 01:31:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1584779518; cv=none; d=google.com; s=arc-20160816; b=v4PqZdPDrnE6nzx70zlyLMGqSuuxK2FwzRza81I6y5OM7wuRS4YS4kI/JhoRx75R2i Mp38+xSmmzVAN+jEuu7rDFr29WwrcW718hUCc4jijD1sSLhb7bxL+rO3NhuZJD4vLZa/ 6htmX0ud833JzlMmgQ8wCusjCrFe/zDpK5078ikXq2rOvmorYE8OQlWn12nPc/5rp+LE kKmx4spzpmJYXacWLLZDYpXLzu53e5C2LP3NHRQxi5UPSV4PFenlSuhCZ7LusQg6m4w2 wSGw6EgCx93RP+ZPh3l7F0TMQeb5M3O+zwIdYW/2VNlBLBRlMmsR4Iy1W6pRQ/rhZe98 kXEA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from; bh=lPGoaWCP0CGFb3jpfrtKXgvY91ob1O1hTpTqxgvCRhg=; b=KXENaEVLVGdtEbv3xQBejwHFE5lCdyyG8bgV82dyFMq084fYS/C/PZRoxoboaL7sdh PST9cdRCcjJqcwY5IZI+/aVgnqWIcpxZCZ5ivUwKQFC/06xe5yT8CTZaOLEgLnDbevWN 5wEjXgSUk9cbFjPFvJRO57+Wj+sAssKZhtsJGMtro2Mq1QvaOiH3vBqbsdNlwcC/lWRz qMW8hGaJIsPFtsH1eyD1ZoP6pP50+X1KIMezBXO1OkFwPXtl0HKc5c0GThkJn4egScat 16kFwrJHSZpbM9tmZpb3Prd8NROjxGUW/6LwcZ27iHY4kd0dxPdWaBFDIIk5YRqzbwba JZMA== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.14 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from david.siemens.de (david.siemens.de. [192.35.17.14]) by gmr-mx.google.com with ESMTPS id f2si173455edm.1.2020.03.21.01.31.58 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 21 Mar 2020 01:31:58 -0700 (PDT) Received-SPF: pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.14 as permitted sender) client-ip=192.35.17.14; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.14 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: from mail2.sbs.de (mail2.sbs.de [192.129.41.66]) by david.siemens.de (8.15.2/8.15.2) with ESMTPS id 02L8VvrI026293 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 21 Mar 2020 09:31:57 +0100 Received: from localhost.localdomain ([139.22.32.253]) by mail2.sbs.de (8.15.2/8.15.2) with ESMTP id 02L8VuGE021533; Sat, 21 Mar 2020 09:31:56 +0100 From: Henning Schild To: isar-users@googlegroups.com Cc: Vijai Kumar K , Baurzhan Ismagulov , Jan Kiszka , Henning Schild Subject: [PATCHv7 00/29] base-apt-rework Date: Sat, 21 Mar 2020 09:31:19 +0100 Message-Id: <20200321083148.26160-1-henning.schild@siemens.com> X-Mailer: git-send-email 2.24.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-TUID: JLPqk+xMBXeF From: Henning Schild changes since v5: - took whitespace and RECIPE-API-CHANGELOG modifications from Baurzhan - fixed p25 to use another mechanism to speed up exports - fixed p11 to actually not re-fetch on bootstrap, saves time and bandwidth - calling it v7 because some patches appeared as v6 on the list changes since v4: - add missing - in p8 download-only arg matching, found by Baurzhan - adding patch 27 and 28 to deal with shell code reuse issues pointed out by Su, Bao Cheng ... "command not found" - bump copyright headers from 2019 to 2020 - adding p29, RECIPE-API-CHANGELOG p8 and p1 appeared on the list as v5, but not from me, the p8 thing is in here the p1 change will probably be added by Baurzhan on a merge, not calling this v6 because of that repost changes since v2: Rebased/merged onto current next and added p26. The custom kernel caching that was missing in v2 is not required on next anymore. This one was sent as v3 by Vijai Kumar K . That just contained a fix in case the patches would not be rebased, can be ignored. Henning Schild (29): repository: new class to deal with repos dpkg-base: add download caching of apt:// downloads meta: dpkg-base: convert "exit" into "return 0" base-apt: change the sources.list to also offer deb-src base-apt: add deb-src packages as well base-apt: do not skip gpg check when it is signed ci: conf: add "hello" to the sample config and every build meta: split all apt-get invocations into download and execution meta: create DL_DIR support for all apt-get downloaded .debs meta: import DL_DIR debs before apt-get download steps meta: include DL_DIR deb import/export into bootstrap base-apt: populate from DEBDIR as well base-apt: drop the "apt_cache" feature base-apt: do not copy debs directly out of rootfss anymore base-apt: rework base-apt population base-apt: move class "base-apt-helper" into only user CI: include "isar-disable-apt-cache" into all CI images CI: include "cowsay" into default build to test dpkg-gbp CI: set BB_NO_NETWORK for cached rebuild meta: repository: implement repo_contains_package and use it in base_apt meta: repository: simplify the matching of packages meta: deb-dl-dir: do not cache debs from isar-apt base-apt: pull base-apt population to the front of the build chain meta: base-apt: remove isar-apt check from population meta: deb-dl-dir: only export newly downloaded files CI: test a custom kernel build in the base-apt offline rebuild meta/deb-dl-dir: make subshell verbose and fail on error meta: deb-dl-dir: fix "command not found" error when excluding pkgs RECIPE-API-CHANGELOG: add a few bits on the base-apt caching changes RECIPE-API-CHANGELOG.md | 7 ++ doc/user_manual.md | 9 +- meta-isar/conf/local.conf.sample | 10 +- meta/classes/base-apt-helper.bbclass | 57 ---------- meta/classes/deb-dl-dir.bbclass | 45 ++++++++ meta/classes/dpkg-base.bbclass | 47 ++++---- meta/classes/dpkg-gbp.bbclass | 5 + meta/classes/dpkg.bbclass | 7 +- meta/classes/image-cache-extension.bbclass | 36 ------- meta/classes/image-locales-extension.bbclass | 2 +- meta/classes/image-tools-extension.bbclass | 7 ++ meta/classes/image.bbclass | 3 +- meta/classes/repository.bbclass | 100 ++++++++++++++++++ meta/classes/rootfs.bbclass | 32 +++--- meta/conf/bitbake.conf | 6 +- .../isar-bootstrap/files/base-apt-sources | 1 - .../isar-bootstrap/isar-bootstrap.inc | 31 +++--- meta/recipes-devtools/base-apt/base-apt.bb | 92 ++++++++++------ .../base-apt/files/distributions.in | 3 - .../buildchroot/files/deps.sh | 37 ++++--- .../isar-apt/files/distributions.in | 3 - meta/recipes-devtools/isar-apt/isar-apt.bb | 26 ++--- scripts/ci_build.sh | 15 ++- 23 files changed, 342 insertions(+), 239 deletions(-) delete mode 100644 meta/classes/base-apt-helper.bbclass create mode 100644 meta/classes/deb-dl-dir.bbclass delete mode 100644 meta/classes/image-cache-extension.bbclass create mode 100644 meta/classes/repository.bbclass delete mode 100644 meta/recipes-core/isar-bootstrap/files/base-apt-sources delete mode 100644 meta/recipes-devtools/base-apt/files/distributions.in delete mode 100644 meta/recipes-devtools/isar-apt/files/distributions.in -- 2.24.1