From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6752900179667451904 X-Received: by 2002:a1c:7311:: with SMTP id d17mr231075wmb.49.1572282095452; Mon, 28 Oct 2019 10:01:35 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:adf:8b1d:: with SMTP id n29ls12443400wra.8.gmail; Mon, 28 Oct 2019 10:01:34 -0700 (PDT) X-Google-Smtp-Source: APXvYqwPWRfaVny+xSoup1u/wHDLP+Ka2Qnibr0m92HWa/4jgxTCus317hB0xGTiSLRrpEMwdHSt X-Received: by 2002:adf:cd87:: with SMTP id q7mr16729261wrj.216.1572282094821; Mon, 28 Oct 2019 10:01:34 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1572282094; cv=none; d=google.com; s=arc-20160816; b=WGdIEwiwgp3eEeqD5Jsy7kKP9K/V3XSUiagLY/DiG9jyRMFcpfl3ytDmzWIDDTAMsg QjFT8N1XYJCr6O3+uihX/qPbNArhkUN31Kqu3YqLVx00EWcBGzERyqZo6/qAFqi/Q3Nr G4lYAswKOWwbY5xE47Vr8XwdxpaZKpAaNODXeO0SA3J5/Sb5FDrDKn7nGYBLWN6dzNq1 4uPeYecPLYdRx9f4VFrqJnO5ORZxTIs0So3Dvb9CUigfD9ZPmI8MpeW9sfDtqi/Av/ov 1luFCe9kZ+2jRvMle9g/zvjLKKnINu8Mib31s0iPQP74lF5BS4Ci4qUYFG7KMFKnqwPT DSBQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=9Q59b7CVVdYjwfyMrt2SAWSqk6GpH1wBYSmUG65NP5Q=; b=ld0CdH9TxQc8l5KeTkCUz/q80mWVRXT9FRfFWq5HUIYmSoGVxVOoo0lrNHc50RpAyd kl1tXZJATy/r6jhUn+MaGhEZXo+hqZVBDKSaw7l87WK0OWKw+f1exMZqK0jNmUaLYpHW HDBvSqbi3830nh8n1jx//j24JfFFMb++6Mef0RNz1DIbYzKBfQ8GZ5aHs8SeEV7UApxw 5939vC2JQnrHBtvVLsk2gdMBYWdq0doE7Pxm7AwqsJnvPA4Xdplsznq1njzkSTDr/JxY OrhWCVA0eBBGR8bzY9YNWUaYZRzXz++LaNIT7pgeXzDnu0lbO1Q+aOFuw+0LIk17boRs 4Vbw== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.28 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from goliath.siemens.de (goliath.siemens.de. [192.35.17.28]) by gmr-mx.google.com with ESMTPS id y188si8079wmc.0.2019.10.28.10.01.34 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 28 Oct 2019 10:01:34 -0700 (PDT) Received-SPF: pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.28 as permitted sender) client-ip=192.35.17.28; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.28 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: from mail1.sbs.de (mail1.sbs.de [192.129.41.35]) by goliath.siemens.de (8.15.2/8.15.2) with ESMTPS id x9SH1Yft007970 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Mon, 28 Oct 2019 18:01:34 +0100 Received: from md1za8fc.ad001.siemens.net ([139.25.69.244]) by mail1.sbs.de (8.15.2/8.15.2) with ESMTP id x9SH1YP7025390; Mon, 28 Oct 2019 18:01:34 +0100 From: Henning Schild To: isar-users@googlegroups.com Cc: Henning Schild Subject: [PATCH 01/17] repository: new class to deal with repos Date: Mon, 28 Oct 2019 18:01:16 +0100 Message-Id: <20191028170132.28072-2-henning.schild@siemens.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191028170132.28072-1-henning.schild@siemens.com> References: <20191028170132.28072-1-henning.schild@siemens.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-TUID: U5chQLnzzPvj From: Henning Schild Factor out all the "reprepro" code into a common class. There are slight functional changes since the two copies of the code got out of sync. Signed-off-by: Henning Schild --- meta/classes/base-apt-helper.bbclass | 28 ++++---- meta/classes/dpkg-base.bbclass | 24 ++----- meta/classes/repository.bbclass | 64 +++++++++++++++++++ meta/recipes-devtools/base-apt/base-apt.bb | 31 +++------ .../base-apt/files/distributions.in | 3 - .../isar-apt/files/distributions.in | 3 - meta/recipes-devtools/isar-apt/isar-apt.bb | 26 ++------ 7 files changed, 97 insertions(+), 82 deletions(-) create mode 100644 meta/classes/repository.bbclass delete mode 100644 meta/recipes-devtools/base-apt/files/distributions.in delete mode 100644 meta/recipes-devtools/isar-apt/files/distributions.in diff --git a/meta/classes/base-apt-helper.bbclass b/meta/classes/base-apt-helper.bbclass index d5674c9..567eaea 100644 --- a/meta/classes/base-apt-helper.bbclass +++ b/meta/classes/base-apt-helper.bbclass @@ -1,5 +1,10 @@ # This software is a part of ISAR. # Copyright (C) 2018 ilbers GmbH +# Copyright (C) 2019 Siemens AG +# +# SPDX-License-Identifier: MIT + +inherit repository compare_pkg_md5sums() { pkg1=$1 @@ -14,10 +19,6 @@ compare_pkg_md5sums() { populate_base_apt() { search_dir=$1 - if [ -n "${GNUPGHOME}" ]; then - export GNUPGHOME="${GNUPGHOME}" - fi - find $search_dir -name '*.deb' | while read package; do # NOTE: due to packages stored by reprepro are not modified, we can # use search by filename to check if package is already in repo. In @@ -40,18 +41,15 @@ populate_base_apt() { compare_pkg_md5sums "$package" "$base_apt_p" && continue # md5sum differs, so remove the package from base-apt - name=$(echo $base_name | cut -d '_' -f 1) - reprepro -b ${REPO_BASE_DIR}/${BASE_DISTRO} \ - --dbdir ${REPO_BASE_DB_DIR}/${BASE_DISTRO} \ - -C main -A ${DISTRO_ARCH} \ - remove ${BASE_DISTRO_CODENAME} \ - $name + repo_del_package "${REPO_BASE_DIR}"/"${BASE_DISTRO}" \ + "${REPO_BASE_DB_DIR}"/"${BASE_DISTRO}" \ + "${BASE_DISTRO_CODENAME}" \ + "${base_apt_p}" fi - reprepro -b ${REPO_BASE_DIR}/${BASE_DISTRO} \ - --dbdir ${REPO_BASE_DB_DIR}/${BASE_DISTRO} \ - -C main \ - includedeb ${BASE_DISTRO_CODENAME} \ - $package + repo_add_packages "${REPO_BASE_DIR}"/"${BASE_DISTRO}" \ + "${REPO_BASE_DB_DIR}"/"${BASE_DISTRO}" \ + "${BASE_DISTRO_CODENAME}" \ + "${package}" done } diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass index fae0a2b..3f1e99f 100644 --- a/meta/classes/dpkg-base.bbclass +++ b/meta/classes/dpkg-base.bbclass @@ -7,6 +7,7 @@ inherit buildchroot inherit debianize inherit terminal +inherit repository DEPENDS ?= "" @@ -114,31 +115,16 @@ repo_clean() { DEBS=$( find ${S}/.. -maxdepth 1 -name "*.deb" || [ ! -d ${S} ] ) if [ -n "${DEBS}" ]; then for d in ${DEBS}; do - p=$( dpkg-deb --show --showformat '${Package}' ${d} ) - a=$( dpkg-deb --show --showformat '${Architecture}' ${d} ) - # removing "all" means no arch - aarg="-A ${a}" - [ "${a}" = "all" ] && aarg="" - reprepro -b ${REPO_ISAR_DIR}/${DISTRO} \ - --dbdir ${REPO_ISAR_DB_DIR}/${DISTRO} \ - -C main ${aarg} \ - remove ${DEBDISTRONAME} \ - ${p} + repo_del_package "${REPO_ISAR_DIR}"/"${DISTRO}" \ + "${REPO_ISAR_DB_DIR}"/"${DISTRO}" "${DEBDISTRONAME}" "${d}" done fi } -# Install package to Isar-apt do_deploy_deb() { - if [ -n "${GNUPGHOME}" ]; then - export GNUPGHOME="${GNUPGHOME}" - fi repo_clean - reprepro -b ${REPO_ISAR_DIR}/${DISTRO} \ - --dbdir ${REPO_ISAR_DB_DIR}/${DISTRO} \ - -C main \ - includedeb ${DEBDISTRONAME} \ - ${S}/../*.deb + repo_add_packages "${REPO_ISAR_DIR}"/"${DISTRO}" \ + "${REPO_ISAR_DB_DIR}"/"${DISTRO}" "${DEBDISTRONAME}" ${S}/../*.deb } addtask deploy_deb after do_dpkg_build before do_build diff --git a/meta/classes/repository.bbclass b/meta/classes/repository.bbclass new file mode 100644 index 0000000..afd3aed --- /dev/null +++ b/meta/classes/repository.bbclass @@ -0,0 +1,64 @@ +# This software is a part of ISAR. +# Copyright (C) 2017-2019 Siemens AG +# Copyright (C) 2019 ilbers GmbH +# +# SPDX-License-Identifier: MIT + +repo_create() { + dir="$1" + dbdir="$2" + codename="$3" + gpgkey="$4" + + if [ -n "${GNUPGHOME}" ]; then + export GNUPGHOME="${GNUPGHOME}" + fi + + if [ ! -f "${dir}"/conf/distributions ]; then + mkdir -p "${dir}"/conf/ + cat < "${dir}"/conf/distributions +Codename: ${codename} +Architectures: i386 armhf arm64 amd64 mipsel source +Components: main +EOF + if [ -n "${gpgkey}" ] ; then + echo "SignWith: yes" >> "${dir}"/conf/distributions + fi + fi + if [ ! -d "${dbdir}" ]; then + reprepro -b "${dir}" --dbdir "${dbdir}" export "${codename}" + fi +} + +repo_add_packages() { + dir="$1" + dbdir="$2" + codename="$3" + shift; shift; shift + + if [ -n "${GNUPGHOME}" ]; then + export GNUPGHOME="${GNUPGHOME}" + fi + reprepro -b "${dir}" --dbdir "${dbdir}" -C main \ + includedeb "${codename}" \ + "$@" +} + +repo_del_package() { + dir="$1" + dbdir="$2" + codename="$3" + file="$4" + + if [ -n "${GNUPGHOME}" ]; then + export GNUPGHOME="${GNUPGHOME}" + fi + p=$( dpkg-deb --show --showformat '${Package}' "${file}" ) + a=$( dpkg-deb --show --showformat '${Architecture}' "${file}" ) + # removing "all" means no arch + aarg="-A ${a}" + [ "${a}" = "all" ] && aarg="" + reprepro -b "${dir}" --dbdir "${dbdir}" -C main ${aarg} \ + remove "${codename}" \ + "${p}" +} diff --git a/meta/recipes-devtools/base-apt/base-apt.bb b/meta/recipes-devtools/base-apt/base-apt.bb index 6acd6e7..765bb1b 100644 --- a/meta/recipes-devtools/base-apt/base-apt.bb +++ b/meta/recipes-devtools/base-apt/base-apt.bb @@ -1,38 +1,23 @@ # This software is a part of ISAR. # Copyright (C) 2018 ilbers GmbH +# Copyright (C) 2019 Siemens AG +# +# SPDX-License-Identifier: MIT -SRC_URI = "file://distributions.in" +inherit repository BASE_REPO_KEY ?= "" -CACHE_CONF_DIR = "${REPO_BASE_DIR}/${BASE_DISTRO}/conf" -do_cache_config[dirs] = "${CACHE_CONF_DIR}" do_cache_config[stamp-extra-info] = "${DISTRO}" do_cache_config[lockfiles] = "${REPO_BASE_DIR}/isar.lock" # Generate reprepro config for current distro if it doesn't exist. Once it's # generated, this task should do nothing. do_cache_config() { - if [ ! -e "${CACHE_CONF_DIR}/distributions" ]; then - sed -e "s#{CODENAME}#"${BASE_DISTRO_CODENAME}"#g" \ - ${WORKDIR}/distributions.in > ${CACHE_CONF_DIR}/distributions - if [ "${BASE_REPO_KEY}" ] ; then - # To generate Release.gpg - echo "SignWith: yes" >> ${CACHE_CONF_DIR}/distributions - fi - fi - - path_cache="${REPO_BASE_DIR}/${BASE_DISTRO}" - path_databases="${REPO_BASE_DB_DIR}/${BASE_DISTRO}" - - if [ ! -d "${path_databases}" ]; then - if [ -n "${GNUPGHOME}" ]; then - export GNUPGHOME="${GNUPGHOME}" - fi - reprepro -b ${path_cache} \ - --dbdir ${path_databases} \ - export ${BASE_DISTRO_CODENAME} - fi + repo_create "${REPO_BASE_DIR}"/"${BASE_DISTRO}" \ + "${REPO_BASE_DB_DIR}"/"${BASE_DISTRO}" \ + "${BASE_DISTRO_CODENAME}" \ + "${BASE_REPO_KEY}" } addtask cache_config after do_unpack before do_build diff --git a/meta/recipes-devtools/base-apt/files/distributions.in b/meta/recipes-devtools/base-apt/files/distributions.in deleted file mode 100644 index fd7de0b..0000000 --- a/meta/recipes-devtools/base-apt/files/distributions.in +++ /dev/null @@ -1,3 +0,0 @@ -Codename: {CODENAME} -Architectures: i386 armhf arm64 amd64 mipsel source -Components: main diff --git a/meta/recipes-devtools/isar-apt/files/distributions.in b/meta/recipes-devtools/isar-apt/files/distributions.in deleted file mode 100644 index a24a700..0000000 --- a/meta/recipes-devtools/isar-apt/files/distributions.in +++ /dev/null @@ -1,3 +0,0 @@ -Codename: {DISTRO_NAME} -Architectures: i386 armhf arm64 amd64 mipsel source -Components: main diff --git a/meta/recipes-devtools/isar-apt/isar-apt.bb b/meta/recipes-devtools/isar-apt/isar-apt.bb index e6f1753..0227c1b 100644 --- a/meta/recipes-devtools/isar-apt/isar-apt.bb +++ b/meta/recipes-devtools/isar-apt/isar-apt.bb @@ -1,32 +1,20 @@ # This software is a part of ISAR. # Copyright (C) 2015-2017 ilbers GmbH +# Copyright (C) 2019 Siemens AG +# +# SPDX-License-Identifier: MIT -SRC_URI = "file://distributions.in" +inherit repository -CACHE_CONF_DIR = "${REPO_ISAR_DIR}/${DISTRO}/conf" -do_cache_config[dirs] = "${CACHE_CONF_DIR}" do_cache_config[stamp-extra-info] = "${DISTRO}" do_cache_config[lockfiles] = "${REPO_ISAR_DIR}/isar.lock" # Generate reprepro config for current distro if it doesn't exist. Once it's # generated, this task should do nothing. do_cache_config() { - if [ ! -e "${CACHE_CONF_DIR}/distributions" ]; then - sed -e "s#{DISTRO_NAME}#"${DEBDISTRONAME}"#g" \ - ${WORKDIR}/distributions.in > ${CACHE_CONF_DIR}/distributions - fi - - path_cache="${REPO_ISAR_DIR}/${DISTRO}" - path_databases="${REPO_ISAR_DB_DIR}/${DISTRO}" - - if [ ! -d "${path_databases}" ]; then - if [ -n "${GNUPGHOME}" ]; then - export GNUPGHOME="${GNUPGHOME}" - fi - reprepro -b ${path_cache} \ - --dbdir ${path_databases} \ - export ${DEBDISTRONAME} - fi + repo_create "${REPO_ISAR_DIR}"/"${DISTRO}" \ + "${REPO_ISAR_DB_DIR}"/"${DISTRO}" \ + "${DEBDISTRONAME}" } addtask cache_config after do_unpack before do_build -- 2.23.0