From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6788114222392803328 X-Received: by 2002:a19:5e06:: with SMTP id s6mr5944575lfb.154.1585230211682; Thu, 26 Mar 2020 06:43:31 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:6512:3188:: with SMTP id i8ls1193736lfe.4.gmail; Thu, 26 Mar 2020 06:43:31 -0700 (PDT) X-Google-Smtp-Source: ADFU+vtE3ISj7MZgY50mG4Rzqna5k2dEYibWbyV2ycNAdMwipYm1k0QofduUOmykS/2xsrKzGro/ X-Received: by 2002:ac2:4307:: with SMTP id l7mr5702012lfh.37.1585230210919; Thu, 26 Mar 2020 06:43:30 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1585230210; cv=none; d=google.com; s=arc-20160816; b=ZtNXrewR5l+ZtpCDtYvPdEpvzks7jO+1bllCgTtLWhoMOCc7oz2pN9pp1987U6F3fL T25+mvOOgLlSc3DwI5ch+re/tPPXeetQx6dciSyeiLMXXw7pbADi0jYI0SMwyA998eo7 HiOpIXhQ/H1Dp5lbVQvRacIeq0ud1NhJ9X1DhMCabqDop3WUivVmP+70OBS8KBbRm0mr ielXvdClOG9kYHfJJde/WZBYTKYlIAt4UfICgPItbkp0RaECb/ffbcsg6rryqHCAV2GG Z99ErZs5kJI0+zs8ZKgPj3hAaha6PGQuRthTpajTYgm2+41NswutFXHNvYBQ84rjxfHk AwcQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from; bh=Zs/C9mJxSzbqb8J/aeILBNvKJfNurjcHwRT/4QXl6QE=; b=aHsGSdgLEij2OQ8T4NFpKPwuz7EZ/eWy8kh+KDhV8wkDUptaMwjdg4Pu4WrK0ybfoU KDrlnLaC6TDMu7hBB1cNLqAWw7Ifm0utsZ2d8bYDFfApO361s0tVkxga9A+S6Yp/6Lxm AtMJ/+EzduofApSSeP4sp4GZnFm8d+VvL/PWn4L6vW3n2P4i2tLDFtLQ7iv1Wu6W8gkl gF5QLHXA80be+CouPYNKRX1lp6nKKbrVVag42iXYKPE+F6q1pvy/MEWbI6aEDIy+OmTH 4o8pevXNJIhXfV0YNasSqoxLPIZfHKSazTWykmrUVQVJ7iniA73/qhbdMv9T0hQgugkT Zqig== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 194.138.37.39 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from lizzard.sbs.de (lizzard.sbs.de. [194.138.37.39]) by gmr-mx.google.com with ESMTPS id s4si72611ljj.2.2020.03.26.06.43.30 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 26 Mar 2020 06:43:30 -0700 (PDT) Received-SPF: pass (google.com: domain of henning.schild@siemens.com designates 194.138.37.39 as permitted sender) client-ip=194.138.37.39; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 194.138.37.39 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: from mail2.sbs.de (mail2.sbs.de [192.129.41.66]) by lizzard.sbs.de (8.15.2/8.15.2) with ESMTPS id 02QDhTpS002363 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 26 Mar 2020 14:43:29 +0100 Received: from localhost.localdomain ([167.87.11.29]) by mail2.sbs.de (8.15.2/8.15.2) with ESMTP id 02QDhSBW031897; Thu, 26 Mar 2020 14:43:28 +0100 From: Henning Schild To: isar-users@googlegroups.com Cc: Vijai Kumar K , Baurzhan Ismagulov , Jan Kiszka , Henning Schild Subject: [PATCHv8 00/29] base-apt-rework Date: Thu, 26 Mar 2020 14:42:56 +0100 Message-Id: <20200326134325.8672-1-henning.schild@siemens.com> X-Mailer: git-send-email 2.24.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-TUID: w9JQ/eF3vJIb From: Henning Schild changes since v7: - fix directory permissions of download cache import in bootstrap (p11) changes since v5: - took whitespace and RECIPE-API-CHANGELOG modifications from Baurzhan - fixed p25 to use another mechanism to speed up exports - fixed p11 to actually not re-fetch on bootstrap, saves time and bandwidth - calling it v7 because some patches appeared as v6 on the list changes since v4: - add missing - in p8 download-only arg matching, found by Baurzhan - adding patch 27 and 28 to deal with shell code reuse issues pointed out by Su, Bao Cheng ... "command not found" - bump copyright headers from 2019 to 2020 - adding p29, RECIPE-API-CHANGELOG p8 and p1 appeared on the list as v5, but not from me, the p8 thing is in here the p1 change will probably be added by Baurzhan on a merge, not calling this v6 because of that repost changes since v2: Rebased/merged onto current next and added p26. The custom kernel caching that was missing in v2 is not required on next anymore. This one was sent as v3 by Vijai Kumar K . That just contained a fix in case the patches would not be rebased, can be ignored. Henning Schild (29): repository: new class to deal with repos dpkg-base: add download caching of apt:// downloads meta: dpkg-base: convert "exit" into "return 0" base-apt: change the sources.list to also offer deb-src base-apt: add deb-src packages as well base-apt: do not skip gpg check when it is signed ci: conf: add "hello" to the sample config and every build meta: split all apt-get invocations into download and execution meta: create DL_DIR support for all apt-get downloaded .debs meta: import DL_DIR debs before apt-get download steps meta: include DL_DIR deb import/export into bootstrap base-apt: populate from DEBDIR as well base-apt: drop the "apt_cache" feature base-apt: do not copy debs directly out of rootfss anymore base-apt: rework base-apt population base-apt: move class "base-apt-helper" into only user CI: include "isar-disable-apt-cache" into all CI images CI: include "cowsay" into default build to test dpkg-gbp CI: set BB_NO_NETWORK for cached rebuild meta: repository: implement repo_contains_package and use it in base_apt meta: repository: simplify the matching of packages meta: deb-dl-dir: do not cache debs from isar-apt base-apt: pull base-apt population to the front of the build chain meta: base-apt: remove isar-apt check from population meta: deb-dl-dir: only export newly downloaded files CI: test a custom kernel build in the base-apt offline rebuild meta/deb-dl-dir: make subshell verbose and fail on error meta: deb-dl-dir: fix "command not found" error when excluding pkgs RECIPE-API-CHANGELOG: add a few bits on the base-apt caching changes RECIPE-API-CHANGELOG.md | 7 ++ doc/user_manual.md | 9 +- meta-isar/conf/local.conf.sample | 10 +- meta/classes/base-apt-helper.bbclass | 57 ---------- meta/classes/deb-dl-dir.bbclass | 45 ++++++++ meta/classes/dpkg-base.bbclass | 47 ++++---- meta/classes/dpkg-gbp.bbclass | 5 + meta/classes/dpkg.bbclass | 7 +- meta/classes/image-cache-extension.bbclass | 36 ------- meta/classes/image-locales-extension.bbclass | 2 +- meta/classes/image-tools-extension.bbclass | 7 ++ meta/classes/image.bbclass | 3 +- meta/classes/repository.bbclass | 100 ++++++++++++++++++ meta/classes/rootfs.bbclass | 32 +++--- meta/conf/bitbake.conf | 6 +- .../isar-bootstrap/files/base-apt-sources | 1 - .../isar-bootstrap/isar-bootstrap.inc | 31 +++--- meta/recipes-devtools/base-apt/base-apt.bb | 92 ++++++++++------ .../base-apt/files/distributions.in | 3 - .../buildchroot/files/deps.sh | 37 ++++--- .../isar-apt/files/distributions.in | 3 - meta/recipes-devtools/isar-apt/isar-apt.bb | 26 ++--- scripts/ci_build.sh | 15 ++- 23 files changed, 342 insertions(+), 239 deletions(-) delete mode 100644 meta/classes/base-apt-helper.bbclass create mode 100644 meta/classes/deb-dl-dir.bbclass delete mode 100644 meta/classes/image-cache-extension.bbclass create mode 100644 meta/classes/repository.bbclass delete mode 100644 meta/recipes-core/isar-bootstrap/files/base-apt-sources delete mode 100644 meta/recipes-devtools/base-apt/files/distributions.in delete mode 100644 meta/recipes-devtools/isar-apt/files/distributions.in -- 2.24.1