From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6788114222392803328 X-Received: by 2002:a50:d506:: with SMTP id u6mr5102722edi.225.1583364513498; Wed, 04 Mar 2020 15:28:33 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a17:906:584f:: with SMTP id h15ls321382ejs.9.gmail; Wed, 04 Mar 2020 15:28:30 -0800 (PST) X-Google-Smtp-Source: ADFU+vuAllDWH4nmrOsqZ72jHIVK+FZuIPanxunyrYDjITUE3KzMrwm3Cs7PN93dmw/E2Z3FHe8v X-Received: by 2002:a17:906:e5b:: with SMTP id q27mr3441403eji.126.1583364510293; Wed, 04 Mar 2020 15:28:30 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1583364510; cv=none; d=google.com; s=arc-20160816; b=aGRYttui3DqNqQ3dQnE8OCcpI5j2GHRAuemChMuqXOmGJsc92RidDPkENw27iRacPc 7v/tlhXbIhVaiMTPl39m0Mbih3PG+bo7wtDOF7b2FadnQUn5GKt9QAwyt+32imV25kXH tLO9tM0uoXIu6XX0gTGrUXnbna5SXuMNe7qQkjvvy2fmrnyt9a+osU2eTOD/Prke5pSz 84CjrAO0kSNvDIW+/5LOidt3iN3H6A3g7g7Y0UAvcdPBfDobVj8AlDQwzXxAbsZuaPRd +xfn5dqfy1iXDUqkai8a499x3u+66GXHHg8NnktTRRDJNFXZAMhUU8sHp08Bh4upDA3v Webg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from; bh=Ta+408H86DjgRYz+F3rK29TZinG5fNehrVUNsAzL1FI=; b=yUroGo7uWzjFDo5CzkhUR0vThukghF5ywzM0k0bKWhU/uEgahiqP1SWiCnHhczuQlf 3P4frq4sLVsKar3T/O/R1iQnhq7Tvgc70kZaDDi5w5+goaUVL+5RUscvfbHSr3i+LYmo uPdwrnfpU4sHrkExrLy53jobgLoYJm/Uc5cm3e7Ob71f4eYHCe2Z/M1vOtSLGXS8nY09 tdQG5nZrUI+UnQCD9gmoZGkAfSpb/M5ejELzDmFGkRNzHekbS8iYGovij9Bl43CZPdSt 1fb+xwcwoN4IQZd+pRilEkFgBlmKR+pOGhAl6XsNuJ58PCtYwMZD35UpeVRnBtGLY6CZ jQZQ== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=neutral (google.com: 85.214.156.166 is neither permitted nor denied by best guess record for domain of ibr@radix50.net) smtp.mailfrom=ibr@radix50.net Return-Path: Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id h10si201895edn.1.2020.03.04.15.28.30 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 04 Mar 2020 15:28:30 -0800 (PST) Received-SPF: neutral (google.com: 85.214.156.166 is neither permitted nor denied by best guess record for domain of ibr@radix50.net) client-ip=85.214.156.166; Authentication-Results: gmr-mx.google.com; spf=neutral (google.com: 85.214.156.166 is neither permitted nor denied by best guess record for domain of ibr@radix50.net) smtp.mailfrom=ibr@radix50.net Received: from yssyq.m.ilbers.de (dslb-084-061-174-236.084.061.pools.vodafone-ip.de [84.61.174.236]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8) with ESMTPSA id 024NSTVm004572 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 5 Mar 2020 00:28:29 +0100 Received: from yssyq.m.ilbers.de (localhost [127.0.0.1]) by yssyq.m.ilbers.de (8.15.2/8.15.2/Debian-14~deb10u1) with ESMTPS id 024NSOi4016563 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NOT) for ; Thu, 5 Mar 2020 00:28:24 +0100 Received: (from ibr@localhost) by yssyq.m.ilbers.de (8.15.2/8.15.2/Submit) id 024NSNXR016562 for isar-users@googlegroups.com; Thu, 5 Mar 2020 00:28:23 +0100 From: Baurzhan Ismagulov To: isar-users@googlegroups.com Subject: [PATCH v5 01/26] repository: new class to deal with repos Date: Thu, 5 Mar 2020 00:28:23 +0100 Message-Id: <20200304232823.16520-1-ibr@radix50.net> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200304232537.rdhqt4udjhyadz25@yssyq.m.ilbers.de> References: <20200304232537.rdhqt4udjhyadz25@yssyq.m.ilbers.de> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-TUID: FmoloRQrA/Mq From: Henning Schild Factor out all the "reprepro" code into a common class. There are slight functional changes since the two copies of the code got out of sync. Signed-off-by: Henning Schild --- meta/classes/base-apt-helper.bbclass | 28 ++++---- meta/classes/dpkg-base.bbclass | 24 ++----- meta/classes/repository.bbclass | 67 +++++++++++++++++++ meta/recipes-devtools/base-apt/base-apt.bb | 38 +++-------- .../isar-apt/files/distributions.in | 2 +- meta/recipes-devtools/isar-apt/isar-apt.bb | 27 +++----- 6 files changed, 105 insertions(+), 81 deletions(-) create mode 100644 meta/classes/repository.bbclass diff --git a/meta/classes/base-apt-helper.bbclass b/meta/classes/base-apt-helper.bbclass index 90b2cfc..1f08c70 100644 --- a/meta/classes/base-apt-helper.bbclass +++ b/meta/classes/base-apt-helper.bbclass @@ -1,5 +1,10 @@ # This software is a part of ISAR. # Copyright (C) 2018 ilbers GmbH +# Copyright (C) 2019 Siemens AG +# +# SPDX-License-Identifier: MIT + +inherit repository compare_pkg_md5sums() { pkg1=$1 @@ -14,10 +19,6 @@ compare_pkg_md5sums() { populate_base_apt() { search_dir=$1 - if [ -n "${GNUPGHOME}" ]; then - export GNUPGHOME="${GNUPGHOME}" - fi - find $search_dir -name '*.deb' | while read package; do # NOTE: due to packages stored by reprepro are not modified, we can # use search by filename to check if package is already in repo. In @@ -40,18 +41,15 @@ populate_base_apt() { compare_pkg_md5sums "$package" "$base_apt_p" && continue # md5sum differs, so remove the package from base-apt - name=$(echo $base_name | cut -d '_' -f 1) - reprepro -b ${REPO_BASE_DIR}/${BASE_DISTRO} \ - --dbdir ${REPO_BASE_DB_DIR}/${BASE_DISTRO} \ - -C main -A ${DISTRO_ARCH} \ - remove ${BASE_DISTRO_CODENAME} \ - $name + repo_del_package "${REPO_BASE_DIR}"/"${BASE_DISTRO}" \ + "${REPO_BASE_DB_DIR}"/"${BASE_DISTRO}" \ + "${BASE_DISTRO_CODENAME}" \ + "${base_apt_p}" fi - reprepro -b ${REPO_BASE_DIR}/${BASE_DISTRO} \ - --dbdir ${REPO_BASE_DB_DIR}/${BASE_DISTRO} \ - -C main \ - includedeb ${BASE_DISTRO_CODENAME} \ - $package + repo_add_packages "${REPO_BASE_DIR}"/"${BASE_DISTRO}" \ + "${REPO_BASE_DB_DIR}"/"${BASE_DISTRO}" \ + "${BASE_DISTRO_CODENAME}" \ + "${package}" done } diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass index 1decf58..3d0a905 100644 --- a/meta/classes/dpkg-base.bbclass +++ b/meta/classes/dpkg-base.bbclass @@ -7,6 +7,7 @@ inherit buildchroot inherit debianize inherit terminal +inherit repository DEPENDS ?= "" @@ -142,31 +143,16 @@ repo_clean() { DEBS=$( find ${S}/.. -maxdepth 1 -name "*.deb" || [ ! -d ${S} ] ) if [ -n "${DEBS}" ]; then for d in ${DEBS}; do - p=$( dpkg-deb --show --showformat '${Package}' ${d} ) - a=$( dpkg-deb --show --showformat '${Architecture}' ${d} ) - # removing "all" means no arch - aarg="-A ${a}" - [ "${a}" = "all" ] && aarg="" - reprepro -b ${REPO_ISAR_DIR}/${DISTRO} \ - --dbdir ${REPO_ISAR_DB_DIR}/${DISTRO} \ - -C main ${aarg} \ - remove ${DEBDISTRONAME} \ - ${p} + repo_del_package "${REPO_ISAR_DIR}"/"${DISTRO}" \ + "${REPO_ISAR_DB_DIR}"/"${DISTRO}" "${DEBDISTRONAME}" "${d}" done fi } -# Install package to Isar-apt do_deploy_deb() { - if [ -n "${GNUPGHOME}" ]; then - export GNUPGHOME="${GNUPGHOME}" - fi repo_clean - reprepro -b ${REPO_ISAR_DIR}/${DISTRO} \ - --dbdir ${REPO_ISAR_DB_DIR}/${DISTRO} \ - -C main \ - includedeb ${DEBDISTRONAME} \ - ${S}/../*.deb + repo_add_packages "${REPO_ISAR_DIR}"/"${DISTRO}" \ + "${REPO_ISAR_DB_DIR}"/"${DISTRO}" "${DEBDISTRONAME}" ${S}/../*.deb } addtask deploy_deb after do_dpkg_build before do_build diff --git a/meta/classes/repository.bbclass b/meta/classes/repository.bbclass new file mode 100644 index 0000000..04a6454 --- /dev/null +++ b/meta/classes/repository.bbclass @@ -0,0 +1,67 @@ +# This software is a part of ISAR. +# Copyright (C) 2017-2019 Siemens AG +# Copyright (C) 2019 ilbers GmbH +# +# SPDX-License-Identifier: MIT + +repo_create() { + local dir="$1" + local dbdir="$2" + local codename="$3" + local reprepro_in="$4" + local keyfiles="$5" + + if [ -n "${GNUPGHOME}" ]; then + export GNUPGHOME="${GNUPGHOME}" + fi + + if [ ! -f "${dir}"/conf/distributions ]; then + mkdir -p "${dir}"/conf/ + sed -e "s#{CODENAME}#${codename}#g" ${reprepro_in} \ + >"${dir}"/conf/distributions + if [ -n "${keyfiles}" ] ; then + local option="" + for key in ${keyfiles}; do + keyid=$(gpg --keyid-format 0xlong --with-colons ${key} 2>/dev/null | grep "^pub:" | awk -F':' '{print $5;}') + option="${option}${keyid} " + done + echo "SignWith: ${option}" >> "${dir}"/conf/distributions + fi + fi + if [ ! -d "${dbdir}" ]; then + reprepro -b "${dir}" --dbdir "${dbdir}" export "${codename}" + fi +} + +repo_add_packages() { + local dir="$1" + local dbdir="$2" + local codename="$3" + shift; shift; shift + + if [ -n "${GNUPGHOME}" ]; then + export GNUPGHOME="${GNUPGHOME}" + fi + reprepro -b "${dir}" --dbdir "${dbdir}" -C main \ + includedeb "${codename}" \ + "$@" +} + +repo_del_package() { + local dir="$1" + local dbdir="$2" + local codename="$3" + local file="$4" + + if [ -n "${GNUPGHOME}" ]; then + export GNUPGHOME="${GNUPGHOME}" + fi + local p=$( dpkg-deb --show --showformat '${Package}' "${file}" ) + local a=$( dpkg-deb --show --showformat '${Architecture}' "${file}" ) + # removing "all" means no arch + local aarg="-A ${a}" + [ "${a}" = "all" ] && aarg="" + reprepro -b "${dir}" --dbdir "${dbdir}" -C main ${aarg} \ + remove "${codename}" \ + "${p}" +} diff --git a/meta/recipes-devtools/base-apt/base-apt.bb b/meta/recipes-devtools/base-apt/base-apt.bb index 9a0f7c8..8bb169c 100644 --- a/meta/recipes-devtools/base-apt/base-apt.bb +++ b/meta/recipes-devtools/base-apt/base-apt.bb @@ -1,45 +1,27 @@ # This software is a part of ISAR. # Copyright (C) 2018 ilbers GmbH +# Copyright (C) 2019 Siemens AG +# +# SPDX-License-Identifier: MIT + +inherit repository SRC_URI = "file://distributions.in" BASE_REPO_KEY ?= "" KEYFILES ?= "" -CACHE_CONF_DIR = "${REPO_BASE_DIR}/${BASE_DISTRO}/conf" -do_cache_config[dirs] = "${CACHE_CONF_DIR}" do_cache_config[stamp-extra-info] = "${DISTRO}" do_cache_config[lockfiles] = "${REPO_BASE_DIR}/isar.lock" # Generate reprepro config for current distro if it doesn't exist. Once it's # generated, this task should do nothing. repo_config() { - if [ -n "${GNUPGHOME}" ]; then - export GNUPGHOME="${GNUPGHOME}" - fi - - if [ ! -e "${CACHE_CONF_DIR}/distributions" ]; then - sed -e "s#{CODENAME}#"${BASE_DISTRO_CODENAME}"#g" \ - ${WORKDIR}/distributions.in > ${CACHE_CONF_DIR}/distributions - if [ -n "${KEYFILES}" ]; then - option="" - for key in ${KEYFILES}; do - keyid=$(gpg --keyid-format 0xlong --with-colons ${key} 2>/dev/null | grep "^pub:" | awk -F':' '{print $5;}') - option="${option}${keyid} " - done - # To generate Release.gpg - echo "SignWith: ${option}" >> ${CACHE_CONF_DIR}/distributions - fi - fi - - path_cache="${REPO_BASE_DIR}/${BASE_DISTRO}" - path_databases="${REPO_BASE_DB_DIR}/${BASE_DISTRO}" - - if [ ! -d "${path_databases}" ]; then - reprepro -b ${path_cache} \ - --dbdir ${path_databases} \ - export ${BASE_DISTRO_CODENAME} - fi + repo_create "${REPO_BASE_DIR}"/"${BASE_DISTRO}" \ + "${REPO_BASE_DB_DIR}"/"${BASE_DISTRO}" \ + "${BASE_DISTRO_CODENAME}" \ + "${WORKDIR}/distributions.in" \ + "${KEYFILES}" } python do_cache_config() { diff --git a/meta/recipes-devtools/isar-apt/files/distributions.in b/meta/recipes-devtools/isar-apt/files/distributions.in index 6471736..3cf7ea5 100644 --- a/meta/recipes-devtools/isar-apt/files/distributions.in +++ b/meta/recipes-devtools/isar-apt/files/distributions.in @@ -1,3 +1,3 @@ -Codename: {DISTRO_NAME} +Codename: {CODENAME} Architectures: i386 armhf arm64 amd64 mipsel riscv64 source Components: main diff --git a/meta/recipes-devtools/isar-apt/isar-apt.bb b/meta/recipes-devtools/isar-apt/isar-apt.bb index e6f1753..c052934 100644 --- a/meta/recipes-devtools/isar-apt/isar-apt.bb +++ b/meta/recipes-devtools/isar-apt/isar-apt.bb @@ -1,32 +1,23 @@ # This software is a part of ISAR. # Copyright (C) 2015-2017 ilbers GmbH +# Copyright (C) 2019 Siemens AG +# +# SPDX-License-Identifier: MIT + +inherit repository SRC_URI = "file://distributions.in" -CACHE_CONF_DIR = "${REPO_ISAR_DIR}/${DISTRO}/conf" -do_cache_config[dirs] = "${CACHE_CONF_DIR}" do_cache_config[stamp-extra-info] = "${DISTRO}" do_cache_config[lockfiles] = "${REPO_ISAR_DIR}/isar.lock" # Generate reprepro config for current distro if it doesn't exist. Once it's # generated, this task should do nothing. do_cache_config() { - if [ ! -e "${CACHE_CONF_DIR}/distributions" ]; then - sed -e "s#{DISTRO_NAME}#"${DEBDISTRONAME}"#g" \ - ${WORKDIR}/distributions.in > ${CACHE_CONF_DIR}/distributions - fi - - path_cache="${REPO_ISAR_DIR}/${DISTRO}" - path_databases="${REPO_ISAR_DB_DIR}/${DISTRO}" - - if [ ! -d "${path_databases}" ]; then - if [ -n "${GNUPGHOME}" ]; then - export GNUPGHOME="${GNUPGHOME}" - fi - reprepro -b ${path_cache} \ - --dbdir ${path_databases} \ - export ${DEBDISTRONAME} - fi + repo_create "${REPO_ISAR_DIR}"/"${DISTRO}" \ + "${REPO_ISAR_DB_DIR}"/"${DISTRO}" \ + "${DEBDISTRONAME}" \ + "${WORKDIR}/distributions.in" } addtask cache_config after do_unpack before do_build -- 2.20.1