From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6995249996219023360 X-Received: by 2002:a7b:c316:: with SMTP id k22mr11134436wmj.56.1628708559891; Wed, 11 Aug 2021 12:02:39 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:600c:1:: with SMTP id g1ls3615920wmc.3.canary-gmail; Wed, 11 Aug 2021 12:02:38 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzJhgXDCGt+CGv7y87fKlHU0vXLzOChN2mHg4mREMN+7+zOhHLig9NhvN33wXUmPs6OsrB1 X-Received: by 2002:a1c:20c5:: with SMTP id g188mr9793684wmg.142.1628708558816; Wed, 11 Aug 2021 12:02:38 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1628708558; cv=none; d=google.com; s=arc-20160816; b=QIK8yYMNJKXwUaeFqvioXdgaIrPJEgGcHkMU7XQt91o6VfqYHSdkDu3nIZgyI3fNE9 8A7QESRQEfod72h+63BetZ/7cueIwoZsXYgHGSsVaH5P7+7UkO4FiNDyqut77ubINEWT YFRQVFxPbe7LgqFLIstBXTP91gowE/C+NG8kwJnq9+tqJCMgSxKGlAnXmshFFJQy/fHq El1GwNOUbhKdbnAt8/u0y+xr3L+cUBA8yN1MZG1H+aHqAtBz809HRL3Oo7/4+YJKdJup bA3kFnmVvCkZrrwtqgqrvdliAEwIuiXYfhyRuKvB8pdyu9CXWJ6nF1I5sBDo8iBxuIwo gTQw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from; bh=MwbrxHQlzEc7wv71U1D3/cX/By1j2DA/9Ln9bIZnbfA=; b=Av+dtcE7LRzyBUnYV2pz/S54MF2DeVv3JZp2VHQujHAfZscWggWaHnArK8w+e4hUcA yzowId2Iej8ruKMJfjEhUru2u7T9QaPhrLXkU0ZNIpA44MHX0uYVMGdR45wuOGPO80Nl +JQz68xevsh8VF6dkfnF8CUVPZYTEiL1JR4dSlnORTln/HdthbZ1qeaSKE+Lq5gcELxh X8v+r9JNKRhPpuUqV0bW4BskKYJ7EkNKCPcooxR6gqjfilDFXKRBaefcwu7N/WTtZ2wL RrP2iyzG9diPadP/aanOI/P8Hp1kke5cW35LEP0pLg+4T+9EqFhxg8Tv4e7fvGJkOga6 uztw== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.14 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from david.siemens.de (david.siemens.de. [192.35.17.14]) by gmr-mx.google.com with ESMTPS id z5si440381wml.4.2021.08.11.12.02.38 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 11 Aug 2021 12:02:38 -0700 (PDT) Received-SPF: pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.14 as permitted sender) client-ip=192.35.17.14; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.14 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: from mail2.sbs.de (mail2.sbs.de [192.129.41.66]) by david.siemens.de (8.15.2/8.15.2) with ESMTPS id 17BJ2cxQ007235 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Wed, 11 Aug 2021 21:02:38 +0200 Received: from md1f2u6c.ad001.siemens.net ([167.87.241.87]) by mail2.sbs.de (8.15.2/8.15.2) with ESMTP id 17BJ2bHC028609 for ; Wed, 11 Aug 2021 21:02:38 +0200 From: Jan Kiszka To: isar-users Subject: [PATCH v2 1/4] isar-bootstrap: Remove obsolete and incorrect skipping of rebuild Date: Wed, 11 Aug 2021 21:02:34 +0200 Message-Id: X-Mailer: git-send-email 2.31.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-TUID: NoCbxk8P9QPN From: Jan Kiszka This check dates back to the time isar-bootstrap could have run multiple times for the same distro-arch and was lock-protected to avoid this. Long history, locks were removed in 1cae951259b3. Now this check only prevented proper rebuilding when apt sources or preferences changed. Signed-off-by: Jan Kiszka --- .../isar-bootstrap/isar-bootstrap.inc | 202 +++++++++--------- 1 file changed, 100 insertions(+), 102 deletions(-) diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc index 0edefc5..b858c8c 100644 --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc @@ -256,125 +256,123 @@ isar_bootstrap() { fi E="${@ isar_export_proxies(d)}" export IS_HOST debootstrap_args E - if [ ! -e "${DEPLOY_ISAR_BOOTSTRAP}" ]; then - sudo rm -rf --one-file-system "${ROOTFSDIR}" - if [ "${IS_HOST}" ];then - deb_dl_dir_import "${ROOTFSDIR}" "${HOST_DISTRO}" + + sudo rm -rf --one-file-system "${ROOTFSDIR}" + if [ "${IS_HOST}" ];then + deb_dl_dir_import "${ROOTFSDIR}" "${HOST_DISTRO}" + else + deb_dl_dir_import "${ROOTFSDIR}" "${DISTRO}" + fi + + sudo -E -s <<'EOSUDO' + set -e + if [ ${IS_HOST} ]; then + ${DEBOOTSTRAP} $debootstrap_args \ + ${@get_distro_components_argument(d, True)} \ + "${@get_distro_suite(d, True)}" \ + "${ROOTFSDIR}" \ + "${@get_distro_source(d, True)}" \ + ${DISTRO_DEBOOTSTRAP_SCRIPT} else - deb_dl_dir_import "${ROOTFSDIR}" "${DISTRO}" + ${DEBOOTSTRAP} $debootstrap_args \ + --arch="${DISTRO_ARCH}" \ + ${@get_distro_components_argument(d, False)} \ + "${@get_distro_suite(d, False)}" \ + "${ROOTFSDIR}" \ + "${@get_distro_source(d, False)}" \ + ${DISTRO_DEBOOTSTRAP_SCRIPT} fi - sudo -E -s <<'EOSUDO' - set -e - if [ ${IS_HOST} ]; then - ${DEBOOTSTRAP} $debootstrap_args \ - ${@get_distro_components_argument(d, True)} \ - "${@get_distro_suite(d, True)}" \ - "${ROOTFSDIR}" \ - "${@get_distro_source(d, True)}" \ - ${DISTRO_DEBOOTSTRAP_SCRIPT} - else - ${DEBOOTSTRAP} $debootstrap_args \ - --arch="${DISTRO_ARCH}" \ - ${@get_distro_components_argument(d, False)} \ - "${@get_distro_suite(d, False)}" \ - "${ROOTFSDIR}" \ - "${@get_distro_source(d, False)}" \ - ${DISTRO_DEBOOTSTRAP_SCRIPT} + # Install apt config + mkdir -p "${ROOTFSDIR}/etc/apt/preferences.d" + install -v -m644 "${APTPREFS}" \ + "${ROOTFSDIR}/etc/apt/preferences.d/bootstrap" + mkdir -p "${ROOTFSDIR}/etc/apt/sources.list.d" + if [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" ]; then + line="file:///base-apt/${BASE_DISTRO} ${BASE_DISTRO_CODENAME} main" + if [ -z "${BASE_REPO_KEY}" ]; then + line="[trusted=yes] ${line}" fi + echo "deb ${line}" > "${ROOTFSDIR}/etc/apt/sources.list.d/base-apt.list" + echo "deb-src ${line}" >> "${ROOTFSDIR}/etc/apt/sources.list.d/base-apt.list" - # Install apt config - mkdir -p "${ROOTFSDIR}/etc/apt/preferences.d" - install -v -m644 "${APTPREFS}" \ - "${ROOTFSDIR}/etc/apt/preferences.d/bootstrap" - mkdir -p "${ROOTFSDIR}/etc/apt/sources.list.d" - if [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" ]; then - line="file:///base-apt/${BASE_DISTRO} ${BASE_DISTRO_CODENAME} main" - if [ -z "${BASE_REPO_KEY}" ]; then - line="[trusted=yes] ${line}" - fi - echo "deb ${line}" > "${ROOTFSDIR}/etc/apt/sources.list.d/base-apt.list" - echo "deb-src ${line}" >> "${ROOTFSDIR}/etc/apt/sources.list.d/base-apt.list" - - mkdir -p ${ROOTFSDIR}/base-apt - mount --bind ${REPO_BASE_DIR} ${ROOTFSDIR}/base-apt - else - install -v -m644 "${APTSRCS}" \ - "${ROOTFSDIR}/etc/apt/sources.list.d/bootstrap.list" - fi - install -v -m644 "${APTSRCS_INIT}" \ - "${ROOTFSDIR}/etc/apt/sources-list" - rm -f "${ROOTFSDIR}/etc/apt/sources.list" - rm -rf "${ROOTFSDIR}/var/lib/apt/lists/"* - mkdir -p "${ROOTFSDIR}/etc/apt/apt.conf.d" - install -v -m644 "${WORKDIR}/isar-apt.conf" \ - "${ROOTFSDIR}/etc/apt/apt.conf.d/50isar.conf" - if [ "${@get_distro_needs_gpg_support(d)}" = "gnupg" ]; then - MY_GPGHOME="$(chroot "${ROOTFSDIR}" mktemp -d /tmp/gpghomeXXXXXXXXXX)" - echo "Created temporary directory ${MY_GPGHOME} for gpg-agent" - export GNUPGHOME="${MY_GPGHOME}" - chroot "${ROOTFSDIR}" gpg-agent --daemon - APT_KEY_APPEND="--homedir ${MY_GPGHOME}" - fi - find ${APT_KEYS_DIR}/ -type f | while read keyfile - do - kfn="$(basename $keyfile)" - cp $keyfile "${ROOTFSDIR}/tmp/$kfn" - chroot "${ROOTFSDIR}" /usr/bin/apt-key \ - --keyring ${THIRD_PARTY_APT_KEYRING} ${APT_KEY_APPEND} add "/tmp/$kfn" - rm "${ROOTFSDIR}/tmp/$kfn" - done - if [ -d "${MY_GPGHOME}" ]; then - echo "Killing gpg-agent for ${MY_GPGHOME}" - chroot "${ROOTFSDIR}" gpgconf --kill gpg-agent && /bin/rm -rf "${MY_GPGHOME}" - fi + mkdir -p ${ROOTFSDIR}/base-apt + mount --bind ${REPO_BASE_DIR} ${ROOTFSDIR}/base-apt + else + install -v -m644 "${APTSRCS}" \ + "${ROOTFSDIR}/etc/apt/sources.list.d/bootstrap.list" + fi + install -v -m644 "${APTSRCS_INIT}" "${ROOTFSDIR}/etc/apt/sources-list" + rm -f "${ROOTFSDIR}/etc/apt/sources.list" + rm -rf "${ROOTFSDIR}/var/lib/apt/lists/"* + mkdir -p "${ROOTFSDIR}/etc/apt/apt.conf.d" + install -v -m644 "${WORKDIR}/isar-apt.conf" \ + "${ROOTFSDIR}/etc/apt/apt.conf.d/50isar.conf" + if [ "${@get_distro_needs_gpg_support(d)}" = "gnupg" ]; then + MY_GPGHOME="$(chroot "${ROOTFSDIR}" mktemp -d /tmp/gpghomeXXXXXXXXXX)" + echo "Created temporary directory ${MY_GPGHOME} for gpg-agent" + export GNUPGHOME="${MY_GPGHOME}" + chroot "${ROOTFSDIR}" gpg-agent --daemon + APT_KEY_APPEND="--homedir ${MY_GPGHOME}" + fi + find ${APT_KEYS_DIR}/ -type f | while read keyfile + do + kfn="$(basename $keyfile)" + cp $keyfile "${ROOTFSDIR}/tmp/$kfn" + chroot "${ROOTFSDIR}" /usr/bin/apt-key \ + --keyring ${THIRD_PARTY_APT_KEYRING} ${APT_KEY_APPEND} add "/tmp/$kfn" + rm "${ROOTFSDIR}/tmp/$kfn" + done + if [ -d "${MY_GPGHOME}" ]; then + echo "Killing gpg-agent for ${MY_GPGHOME}" + chroot "${ROOTFSDIR}" gpgconf --kill gpg-agent && /bin/rm -rf "${MY_GPGHOME}" + fi - if [ "${@get_distro_suite(d, True)}" = "stretch" ] && [ "${@get_host_release().split('.')[0]}" -lt "4" ]; then - install -v -m644 "${WORKDIR}/isar-apt-fallback.conf" \ - "${ROOTFSDIR}/etc/apt/apt.conf.d/55isar-fallback.conf" - fi + if [ "${@get_distro_suite(d, True)}" = "stretch" ] && [ "${@get_host_release().split('.')[0]}" -lt "4" ]; then + install -v -m644 "${WORKDIR}/isar-apt-fallback.conf" \ + "${ROOTFSDIR}/etc/apt/apt.conf.d/55isar-fallback.conf" + fi - # Set locale - install -v -m644 "${WORKDIR}/locale" "${ROOTFSDIR}/etc/locale" + # Set locale + install -v -m644 "${WORKDIR}/locale" "${ROOTFSDIR}/etc/locale" - sed -i '/en_US.UTF-8 UTF-8/s/^#//g' "${ROOTFSDIR}/etc/locale.gen" - chroot "${ROOTFSDIR}" /usr/sbin/locale-gen + sed -i '/en_US.UTF-8 UTF-8/s/^#//g' "${ROOTFSDIR}/etc/locale.gen" + chroot "${ROOTFSDIR}" /usr/sbin/locale-gen - # setup chroot - install -v -m755 "${WORKDIR}/chroot-setup.sh" "${ROOTFSDIR}/chroot-setup.sh" - "${ROOTFSDIR}/chroot-setup.sh" "setup" "${ROOTFSDIR}" + # setup chroot + install -v -m755 "${WORKDIR}/chroot-setup.sh" "${ROOTFSDIR}/chroot-setup.sh" + "${ROOTFSDIR}/chroot-setup.sh" "setup" "${ROOTFSDIR}" - # update APT - mount --rbind /dev ${ROOTFSDIR}/dev - mount --make-rslave ${ROOTFSDIR}/dev - mount -t proc none ${ROOTFSDIR}/proc - mount --rbind /sys ${ROOTFSDIR}/sys - mount --make-rslave ${ROOTFSDIR}/sys + # update APT + mount --rbind /dev ${ROOTFSDIR}/dev + mount --make-rslave ${ROOTFSDIR}/dev + mount -t proc none ${ROOTFSDIR}/proc + mount --rbind /sys ${ROOTFSDIR}/sys + mount --make-rslave ${ROOTFSDIR}/sys - export DEBIAN_FRONTEND=noninteractive + export DEBIAN_FRONTEND=noninteractive - if [ ${IS_HOST} ]; then - chroot "${ROOTFSDIR}" /usr/bin/dpkg --add-architecture ${DISTRO_ARCH} - fi + if [ ${IS_HOST} ]; then + chroot "${ROOTFSDIR}" /usr/bin/dpkg --add-architecture ${DISTRO_ARCH} + fi - if [ "${ISAR_ENABLE_COMPAT_ARCH}" = "1" ]; then - chroot "${ROOTFSDIR}" /usr/bin/dpkg --add-architecture ${COMPAT_DISTRO_ARCH} - fi + if [ "${ISAR_ENABLE_COMPAT_ARCH}" = "1" ]; then + chroot "${ROOTFSDIR}" /usr/bin/dpkg --add-architecture ${COMPAT_DISTRO_ARCH} + fi - chroot "${ROOTFSDIR}" /usr/bin/apt-get update -y - chroot "${ROOTFSDIR}" /usr/bin/apt-get install -y -f - chroot "${ROOTFSDIR}" /usr/bin/apt-get dist-upgrade -y \ - -o Debug::pkgProblemResolver=yes + chroot "${ROOTFSDIR}" /usr/bin/apt-get update -y + chroot "${ROOTFSDIR}" /usr/bin/apt-get install -y -f + chroot "${ROOTFSDIR}" /usr/bin/apt-get dist-upgrade -y \ + -o Debug::pkgProblemResolver=yes - umount -l "${ROOTFSDIR}/dev" - umount -l "${ROOTFSDIR}/proc" - umount -l "${ROOTFSDIR}/sys" - umount -l "${ROOTFSDIR}/base-apt" || true + umount -l "${ROOTFSDIR}/dev" + umount -l "${ROOTFSDIR}/proc" + umount -l "${ROOTFSDIR}/sys" + umount -l "${ROOTFSDIR}/base-apt" || true - # Finalize debootstrap by setting the link in deploy - ln -Tfsr "${ROOTFSDIR}" "${DEPLOY_ISAR_BOOTSTRAP}" + # Finalize debootstrap by setting the link in deploy + ln -Tfsr "${ROOTFSDIR}" "${DEPLOY_ISAR_BOOTSTRAP}" EOSUDO - fi if [ "${IS_HOST}" ];then deb_dl_dir_export "${ROOTFSDIR}" "${HOST_DISTRO}" else -- 2.31.1