From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6788114222392803328 X-Received: by 2002:adf:a30a:: with SMTP id c10mr28814390wrb.175.1583846079620; Tue, 10 Mar 2020 06:14:39 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a1c:208f:: with SMTP id g137ls129629wmg.1.gmail; Tue, 10 Mar 2020 06:14:39 -0700 (PDT) X-Google-Smtp-Source: ADFU+vswy5XJJqhLebyIL7JJhdF9OzleesBMPgQ8R6PUiMw7w7EdTTv4LdqmbeNyssVRAB5qbErS X-Received: by 2002:a05:600c:2188:: with SMTP id e8mr2259012wme.83.1583846079025; Tue, 10 Mar 2020 06:14:39 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1583846079; cv=none; d=google.com; s=arc-20160816; b=qV6CrunuoJqo63RCPQPikBH7LvIV+rzU2+KCrXHBM0KjOhKb1jz0y7WMrfwnxtM/nc nvub92lhLeAPu4SxN5Sn3SVcrFlR/bx4eijM9MOyviTcKaM67Lr80OKKYqgCBYBXkPA7 fMVMYQdHEipLRWK2WRwGfhiZzlEdP+ErsvuJ6WM3qkGPgcC/s4A9bFv4J0bslbkbVQiA 4gH3dwUBqBOD04CG16oIGv0DwIGx1Pptkd0t+EUHAabujE5M/VsL66MCWKFuytN+RoYf DAX/gCdyS6Bv56kb1R1qI6jX+2azJP2okn/Nvke+ntf+erP94sBHdaRWVWkLNxaKRiYO Efhg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date; bh=YPxrOvEGHSErywQ8jWI5iJBzlIIaOmNWVjKdS5Na4Bg=; b=bsqQSAa1rguuxrMnqIVLzoHD9c6900+3x7UFyAfMPkJJbO9id8IxMjlGDPa1S4ci6o 8P7MCWDrg7YZk/9ZIMur3PaY30x5aC1xrAeUKAv6hZgOPX/9xJBs1XG4Lt8pNheXpKGP KZUjAwAo2+TnDu9nYvAEH1c07BBWopzdOaOdHhbxkWj+1GK3i7GYQQ4cNoaTLT4wHp+P 9DAzWeMNB9MH86lk82yscVBR8816hggzwduExFivu7dA/puI2Wa00ZDXbZOr04kPnmmp M5NBLB2V10kQCj5FY0kZqTs8ck/8ysKeIY78huiHYJ3d3RF6RQrV3KPZARHIqTx77Vo9 bDVQ== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 194.138.37.39 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from lizzard.sbs.de (lizzard.sbs.de. [194.138.37.39]) by gmr-mx.google.com with ESMTPS id q202si106630wme.2.2020.03.10.06.14.38 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 10 Mar 2020 06:14:39 -0700 (PDT) Received-SPF: pass (google.com: domain of henning.schild@siemens.com designates 194.138.37.39 as permitted sender) client-ip=194.138.37.39; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 194.138.37.39 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: from mail2.sbs.de (mail2.sbs.de [192.129.41.66]) by lizzard.sbs.de (8.15.2/8.15.2) with ESMTPS id 02ADEcho030073 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 10 Mar 2020 14:14:38 +0100 Received: from md1za8fc.ad001.siemens.net ([139.25.68.205]) by mail2.sbs.de (8.15.2/8.15.2) with ESMTP id 02ADEcSh005307; Tue, 10 Mar 2020 14:14:38 +0100 Date: Tue, 10 Mar 2020 14:14:37 +0100 From: Henning Schild To: Baurzhan Ismagulov Cc: Subject: Re: [PATCH v5 01/26] repository: new class to deal with repos Message-ID: <20200310141437.705cd61c@md1za8fc.ad001.siemens.net> In-Reply-To: <20200304232823.16520-1-ibr@radix50.net> References: <20200304232537.rdhqt4udjhyadz25@yssyq.m.ilbers.de> <20200304232823.16520-1-ibr@radix50.net> X-Mailer: Claws Mail 3.17.4 (GTK+ 2.24.32; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-TUID: VsaseGBgOsyp I will in fact ignore this one again. Since this is not my code anymore and i would need to compare the patch to my original patch. My suggestion would be to restore the split of code and config on top again, if that is ok with you. Maybe directly by you, with the merge. If you want to fold it, please become a co-author of the patch, with a signed-off. My next version will have 2020 copyright headers. Henning On Thu, 5 Mar 2020 00:28:23 +0100 Baurzhan Ismagulov wrote: > From: Henning Schild > > Factor out all the "reprepro" code into a common class. There are > slight functional changes since the two copies of the code got out of > sync. > > Signed-off-by: Henning Schild > --- > meta/classes/base-apt-helper.bbclass | 28 ++++---- > meta/classes/dpkg-base.bbclass | 24 ++----- > meta/classes/repository.bbclass | 67 > +++++++++++++++++++ meta/recipes-devtools/base-apt/base-apt.bb | > 38 +++-------- .../isar-apt/files/distributions.in | 2 +- > meta/recipes-devtools/isar-apt/isar-apt.bb | 27 +++----- > 6 files changed, 105 insertions(+), 81 deletions(-) > create mode 100644 meta/classes/repository.bbclass > > diff --git a/meta/classes/base-apt-helper.bbclass > b/meta/classes/base-apt-helper.bbclass index 90b2cfc..1f08c70 100644 > --- a/meta/classes/base-apt-helper.bbclass > +++ b/meta/classes/base-apt-helper.bbclass > @@ -1,5 +1,10 @@ > # This software is a part of ISAR. > # Copyright (C) 2018 ilbers GmbH > +# Copyright (C) 2019 Siemens AG > +# > +# SPDX-License-Identifier: MIT > + > +inherit repository > > compare_pkg_md5sums() { > pkg1=$1 > @@ -14,10 +19,6 @@ compare_pkg_md5sums() { > populate_base_apt() { > search_dir=$1 > > - if [ -n "${GNUPGHOME}" ]; then > - export GNUPGHOME="${GNUPGHOME}" > - fi > - > find $search_dir -name '*.deb' | while read package; do > # NOTE: due to packages stored by reprepro are not modified, > we can # use search by filename to check if package is already in > repo. In @@ -40,18 +41,15 @@ populate_base_apt() { > compare_pkg_md5sums "$package" "$base_apt_p" && continue > > # md5sum differs, so remove the package from base-apt > - name=$(echo $base_name | cut -d '_' -f 1) > - reprepro -b ${REPO_BASE_DIR}/${BASE_DISTRO} \ > - --dbdir ${REPO_BASE_DB_DIR}/${BASE_DISTRO} \ > - -C main -A ${DISTRO_ARCH} \ > - remove ${BASE_DISTRO_CODENAME} \ > - $name > + repo_del_package "${REPO_BASE_DIR}"/"${BASE_DISTRO}" \ > + "${REPO_BASE_DB_DIR}"/"${BASE_DISTRO}" \ > + "${BASE_DISTRO_CODENAME}" \ > + "${base_apt_p}" > fi > > - reprepro -b ${REPO_BASE_DIR}/${BASE_DISTRO} \ > - --dbdir ${REPO_BASE_DB_DIR}/${BASE_DISTRO} \ > - -C main \ > - includedeb ${BASE_DISTRO_CODENAME} \ > - $package > + repo_add_packages "${REPO_BASE_DIR}"/"${BASE_DISTRO}" \ > + "${REPO_BASE_DB_DIR}"/"${BASE_DISTRO}" \ > + "${BASE_DISTRO_CODENAME}" \ > + "${package}" > done > } > diff --git a/meta/classes/dpkg-base.bbclass > b/meta/classes/dpkg-base.bbclass index 1decf58..3d0a905 100644 > --- a/meta/classes/dpkg-base.bbclass > +++ b/meta/classes/dpkg-base.bbclass > @@ -7,6 +7,7 @@ > inherit buildchroot > inherit debianize > inherit terminal > +inherit repository > > DEPENDS ?= "" > > @@ -142,31 +143,16 @@ repo_clean() { > DEBS=$( find ${S}/.. -maxdepth 1 -name "*.deb" || [ ! -d ${S} ] ) > if [ -n "${DEBS}" ]; then > for d in ${DEBS}; do > - p=$( dpkg-deb --show --showformat '${Package}' ${d} ) > - a=$( dpkg-deb --show --showformat '${Architecture}' ${d} > ) > - # removing "all" means no arch > - aarg="-A ${a}" > - [ "${a}" = "all" ] && aarg="" > - reprepro -b ${REPO_ISAR_DIR}/${DISTRO} \ > - --dbdir ${REPO_ISAR_DB_DIR}/${DISTRO} \ > - -C main ${aarg} \ > - remove ${DEBDISTRONAME} \ > - ${p} > + repo_del_package "${REPO_ISAR_DIR}"/"${DISTRO}" \ > + "${REPO_ISAR_DB_DIR}"/"${DISTRO}" "${DEBDISTRONAME}" > "${d}" done > fi > } > > -# Install package to Isar-apt > do_deploy_deb() { > - if [ -n "${GNUPGHOME}" ]; then > - export GNUPGHOME="${GNUPGHOME}" > - fi > repo_clean > - reprepro -b ${REPO_ISAR_DIR}/${DISTRO} \ > - --dbdir ${REPO_ISAR_DB_DIR}/${DISTRO} \ > - -C main \ > - includedeb ${DEBDISTRONAME} \ > - ${S}/../*.deb > + repo_add_packages "${REPO_ISAR_DIR}"/"${DISTRO}" \ > + "${REPO_ISAR_DB_DIR}"/"${DISTRO}" "${DEBDISTRONAME}" > ${S}/../*.deb } > > addtask deploy_deb after do_dpkg_build before do_build > diff --git a/meta/classes/repository.bbclass > b/meta/classes/repository.bbclass new file mode 100644 > index 0000000..04a6454 > --- /dev/null > +++ b/meta/classes/repository.bbclass > @@ -0,0 +1,67 @@ > +# This software is a part of ISAR. > +# Copyright (C) 2017-2019 Siemens AG > +# Copyright (C) 2019 ilbers GmbH > +# > +# SPDX-License-Identifier: MIT > + > +repo_create() { > + local dir="$1" > + local dbdir="$2" > + local codename="$3" > + local reprepro_in="$4" > + local keyfiles="$5" > + > + if [ -n "${GNUPGHOME}" ]; then > + export GNUPGHOME="${GNUPGHOME}" > + fi > + > + if [ ! -f "${dir}"/conf/distributions ]; then > + mkdir -p "${dir}"/conf/ > + sed -e "s#{CODENAME}#${codename}#g" ${reprepro_in} \ > + >"${dir}"/conf/distributions > + if [ -n "${keyfiles}" ] ; then > + local option="" > + for key in ${keyfiles}; do > + keyid=$(gpg --keyid-format 0xlong --with-colons ${key} > 2>/dev/null | grep "^pub:" | awk -F':' '{print $5;}') > + option="${option}${keyid} " > + done > + echo "SignWith: ${option}" >> "${dir}"/conf/distributions > + fi > + fi > + if [ ! -d "${dbdir}" ]; then > + reprepro -b "${dir}" --dbdir "${dbdir}" export "${codename}" > + fi > +} > + > +repo_add_packages() { > + local dir="$1" > + local dbdir="$2" > + local codename="$3" > + shift; shift; shift > + > + if [ -n "${GNUPGHOME}" ]; then > + export GNUPGHOME="${GNUPGHOME}" > + fi > + reprepro -b "${dir}" --dbdir "${dbdir}" -C main \ > + includedeb "${codename}" \ > + "$@" > +} > + > +repo_del_package() { > + local dir="$1" > + local dbdir="$2" > + local codename="$3" > + local file="$4" > + > + if [ -n "${GNUPGHOME}" ]; then > + export GNUPGHOME="${GNUPGHOME}" > + fi > + local p=$( dpkg-deb --show --showformat '${Package}' "${file}" ) > + local a=$( dpkg-deb --show --showformat '${Architecture}' > "${file}" ) > + # removing "all" means no arch > + local aarg="-A ${a}" > + [ "${a}" = "all" ] && aarg="" > + reprepro -b "${dir}" --dbdir "${dbdir}" -C main ${aarg} \ > + remove "${codename}" \ > + "${p}" > +} > diff --git a/meta/recipes-devtools/base-apt/base-apt.bb > b/meta/recipes-devtools/base-apt/base-apt.bb index 9a0f7c8..8bb169c > 100644 --- a/meta/recipes-devtools/base-apt/base-apt.bb > +++ b/meta/recipes-devtools/base-apt/base-apt.bb > @@ -1,45 +1,27 @@ > # This software is a part of ISAR. > # Copyright (C) 2018 ilbers GmbH > +# Copyright (C) 2019 Siemens AG > +# > +# SPDX-License-Identifier: MIT > + > +inherit repository > > SRC_URI = "file://distributions.in" > > BASE_REPO_KEY ?= "" > KEYFILES ?= "" > > -CACHE_CONF_DIR = "${REPO_BASE_DIR}/${BASE_DISTRO}/conf" > -do_cache_config[dirs] = "${CACHE_CONF_DIR}" > do_cache_config[stamp-extra-info] = "${DISTRO}" > do_cache_config[lockfiles] = "${REPO_BASE_DIR}/isar.lock" > > # Generate reprepro config for current distro if it doesn't exist. > Once it's # generated, this task should do nothing. > repo_config() { > - if [ -n "${GNUPGHOME}" ]; then > - export GNUPGHOME="${GNUPGHOME}" > - fi > - > - if [ ! -e "${CACHE_CONF_DIR}/distributions" ]; then > - sed -e "s#{CODENAME}#"${BASE_DISTRO_CODENAME}"#g" \ > - ${WORKDIR}/distributions.in > > ${CACHE_CONF_DIR}/distributions > - if [ -n "${KEYFILES}" ]; then > - option="" > - for key in ${KEYFILES}; do > - keyid=$(gpg --keyid-format 0xlong --with-colons > ${key} 2>/dev/null | grep "^pub:" | awk -F':' '{print $5;}') > - option="${option}${keyid} " > - done > - # To generate Release.gpg > - echo "SignWith: ${option}" >> > ${CACHE_CONF_DIR}/distributions > - fi > - fi > - > - path_cache="${REPO_BASE_DIR}/${BASE_DISTRO}" > - path_databases="${REPO_BASE_DB_DIR}/${BASE_DISTRO}" > - > - if [ ! -d "${path_databases}" ]; then > - reprepro -b ${path_cache} \ > - --dbdir ${path_databases} \ > - export ${BASE_DISTRO_CODENAME} > - fi > + repo_create "${REPO_BASE_DIR}"/"${BASE_DISTRO}" \ > + "${REPO_BASE_DB_DIR}"/"${BASE_DISTRO}" \ > + "${BASE_DISTRO_CODENAME}" \ > + "${WORKDIR}/distributions.in" \ > + "${KEYFILES}" > } > > python do_cache_config() { > diff --git a/meta/recipes-devtools/isar-apt/files/distributions.in > b/meta/recipes-devtools/isar-apt/files/distributions.in index > 6471736..3cf7ea5 100644 --- > a/meta/recipes-devtools/isar-apt/files/distributions.in +++ > b/meta/recipes-devtools/isar-apt/files/distributions.in @@ -1,3 +1,3 > @@ -Codename: {DISTRO_NAME} > +Codename: {CODENAME} > Architectures: i386 armhf arm64 amd64 mipsel riscv64 source > Components: main > diff --git a/meta/recipes-devtools/isar-apt/isar-apt.bb > b/meta/recipes-devtools/isar-apt/isar-apt.bb index e6f1753..c052934 > 100644 --- a/meta/recipes-devtools/isar-apt/isar-apt.bb > +++ b/meta/recipes-devtools/isar-apt/isar-apt.bb > @@ -1,32 +1,23 @@ > # This software is a part of ISAR. > # Copyright (C) 2015-2017 ilbers GmbH > +# Copyright (C) 2019 Siemens AG > +# > +# SPDX-License-Identifier: MIT > + > +inherit repository > > SRC_URI = "file://distributions.in" > > -CACHE_CONF_DIR = "${REPO_ISAR_DIR}/${DISTRO}/conf" > -do_cache_config[dirs] = "${CACHE_CONF_DIR}" > do_cache_config[stamp-extra-info] = "${DISTRO}" > do_cache_config[lockfiles] = "${REPO_ISAR_DIR}/isar.lock" > > # Generate reprepro config for current distro if it doesn't exist. > Once it's # generated, this task should do nothing. > do_cache_config() { > - if [ ! -e "${CACHE_CONF_DIR}/distributions" ]; then > - sed -e "s#{DISTRO_NAME}#"${DEBDISTRONAME}"#g" \ > - ${WORKDIR}/distributions.in > > ${CACHE_CONF_DIR}/distributions > - fi > - > - path_cache="${REPO_ISAR_DIR}/${DISTRO}" > - path_databases="${REPO_ISAR_DB_DIR}/${DISTRO}" > - > - if [ ! -d "${path_databases}" ]; then > - if [ -n "${GNUPGHOME}" ]; then > - export GNUPGHOME="${GNUPGHOME}" > - fi > - reprepro -b ${path_cache} \ > - --dbdir ${path_databases} \ > - export ${DEBDISTRONAME} > - fi > + repo_create "${REPO_ISAR_DIR}"/"${DISTRO}" \ > + "${REPO_ISAR_DB_DIR}"/"${DISTRO}" \ > + "${DEBDISTRONAME}" \ > + "${WORKDIR}/distributions.in" > } > > addtask cache_config after do_unpack before do_build