From: Henning Schild <henning.schild@siemens.com>
To: Baurzhan Ismagulov <ibr@radix50.net>
Cc: <isar-users@googlegroups.com>
Subject: Re: [PATCH v5 01/26] repository: new class to deal with repos
Date: Tue, 10 Mar 2020 14:14:37 +0100 [thread overview]
Message-ID: <20200310141437.705cd61c@md1za8fc.ad001.siemens.net> (raw)
In-Reply-To: <20200304232823.16520-1-ibr@radix50.net>
I will in fact ignore this one again. Since this is not my code anymore
and i would need to compare the patch to my original patch.
My suggestion would be to restore the split of code and config on top
again, if that is ok with you. Maybe directly by you, with the merge.
If you want to fold it, please become a co-author of the patch, with a
signed-off.
My next version will have 2020 copyright headers.
Henning
On Thu, 5 Mar 2020 00:28:23 +0100
Baurzhan Ismagulov <ibr@radix50.net> wrote:
> From: Henning Schild <henning.schild@siemens.com>
>
> Factor out all the "reprepro" code into a common class. There are
> slight functional changes since the two copies of the code got out of
> sync.
>
> Signed-off-by: Henning Schild <henning.schild@siemens.com>
> ---
> meta/classes/base-apt-helper.bbclass | 28 ++++----
> meta/classes/dpkg-base.bbclass | 24 ++-----
> meta/classes/repository.bbclass | 67
> +++++++++++++++++++ meta/recipes-devtools/base-apt/base-apt.bb |
> 38 +++-------- .../isar-apt/files/distributions.in | 2 +-
> meta/recipes-devtools/isar-apt/isar-apt.bb | 27 +++-----
> 6 files changed, 105 insertions(+), 81 deletions(-)
> create mode 100644 meta/classes/repository.bbclass
>
> diff --git a/meta/classes/base-apt-helper.bbclass
> b/meta/classes/base-apt-helper.bbclass index 90b2cfc..1f08c70 100644
> --- a/meta/classes/base-apt-helper.bbclass
> +++ b/meta/classes/base-apt-helper.bbclass
> @@ -1,5 +1,10 @@
> # This software is a part of ISAR.
> # Copyright (C) 2018 ilbers GmbH
> +# Copyright (C) 2019 Siemens AG
> +#
> +# SPDX-License-Identifier: MIT
> +
> +inherit repository
>
> compare_pkg_md5sums() {
> pkg1=$1
> @@ -14,10 +19,6 @@ compare_pkg_md5sums() {
> populate_base_apt() {
> search_dir=$1
>
> - if [ -n "${GNUPGHOME}" ]; then
> - export GNUPGHOME="${GNUPGHOME}"
> - fi
> -
> find $search_dir -name '*.deb' | while read package; do
> # NOTE: due to packages stored by reprepro are not modified,
> we can # use search by filename to check if package is already in
> repo. In @@ -40,18 +41,15 @@ populate_base_apt() {
> compare_pkg_md5sums "$package" "$base_apt_p" && continue
>
> # md5sum differs, so remove the package from base-apt
> - name=$(echo $base_name | cut -d '_' -f 1)
> - reprepro -b ${REPO_BASE_DIR}/${BASE_DISTRO} \
> - --dbdir ${REPO_BASE_DB_DIR}/${BASE_DISTRO} \
> - -C main -A ${DISTRO_ARCH} \
> - remove ${BASE_DISTRO_CODENAME} \
> - $name
> + repo_del_package "${REPO_BASE_DIR}"/"${BASE_DISTRO}" \
> + "${REPO_BASE_DB_DIR}"/"${BASE_DISTRO}" \
> + "${BASE_DISTRO_CODENAME}" \
> + "${base_apt_p}"
> fi
>
> - reprepro -b ${REPO_BASE_DIR}/${BASE_DISTRO} \
> - --dbdir ${REPO_BASE_DB_DIR}/${BASE_DISTRO} \
> - -C main \
> - includedeb ${BASE_DISTRO_CODENAME} \
> - $package
> + repo_add_packages "${REPO_BASE_DIR}"/"${BASE_DISTRO}" \
> + "${REPO_BASE_DB_DIR}"/"${BASE_DISTRO}" \
> + "${BASE_DISTRO_CODENAME}" \
> + "${package}"
> done
> }
> diff --git a/meta/classes/dpkg-base.bbclass
> b/meta/classes/dpkg-base.bbclass index 1decf58..3d0a905 100644
> --- a/meta/classes/dpkg-base.bbclass
> +++ b/meta/classes/dpkg-base.bbclass
> @@ -7,6 +7,7 @@
> inherit buildchroot
> inherit debianize
> inherit terminal
> +inherit repository
>
> DEPENDS ?= ""
>
> @@ -142,31 +143,16 @@ repo_clean() {
> DEBS=$( find ${S}/.. -maxdepth 1 -name "*.deb" || [ ! -d ${S} ] )
> if [ -n "${DEBS}" ]; then
> for d in ${DEBS}; do
> - p=$( dpkg-deb --show --showformat '${Package}' ${d} )
> - a=$( dpkg-deb --show --showformat '${Architecture}' ${d}
> )
> - # removing "all" means no arch
> - aarg="-A ${a}"
> - [ "${a}" = "all" ] && aarg=""
> - reprepro -b ${REPO_ISAR_DIR}/${DISTRO} \
> - --dbdir ${REPO_ISAR_DB_DIR}/${DISTRO} \
> - -C main ${aarg} \
> - remove ${DEBDISTRONAME} \
> - ${p}
> + repo_del_package "${REPO_ISAR_DIR}"/"${DISTRO}" \
> + "${REPO_ISAR_DB_DIR}"/"${DISTRO}" "${DEBDISTRONAME}"
> "${d}" done
> fi
> }
>
> -# Install package to Isar-apt
> do_deploy_deb() {
> - if [ -n "${GNUPGHOME}" ]; then
> - export GNUPGHOME="${GNUPGHOME}"
> - fi
> repo_clean
> - reprepro -b ${REPO_ISAR_DIR}/${DISTRO} \
> - --dbdir ${REPO_ISAR_DB_DIR}/${DISTRO} \
> - -C main \
> - includedeb ${DEBDISTRONAME} \
> - ${S}/../*.deb
> + repo_add_packages "${REPO_ISAR_DIR}"/"${DISTRO}" \
> + "${REPO_ISAR_DB_DIR}"/"${DISTRO}" "${DEBDISTRONAME}"
> ${S}/../*.deb }
>
> addtask deploy_deb after do_dpkg_build before do_build
> diff --git a/meta/classes/repository.bbclass
> b/meta/classes/repository.bbclass new file mode 100644
> index 0000000..04a6454
> --- /dev/null
> +++ b/meta/classes/repository.bbclass
> @@ -0,0 +1,67 @@
> +# This software is a part of ISAR.
> +# Copyright (C) 2017-2019 Siemens AG
> +# Copyright (C) 2019 ilbers GmbH
> +#
> +# SPDX-License-Identifier: MIT
> +
> +repo_create() {
> + local dir="$1"
> + local dbdir="$2"
> + local codename="$3"
> + local reprepro_in="$4"
> + local keyfiles="$5"
> +
> + if [ -n "${GNUPGHOME}" ]; then
> + export GNUPGHOME="${GNUPGHOME}"
> + fi
> +
> + if [ ! -f "${dir}"/conf/distributions ]; then
> + mkdir -p "${dir}"/conf/
> + sed -e "s#{CODENAME}#${codename}#g" ${reprepro_in} \
> + >"${dir}"/conf/distributions
> + if [ -n "${keyfiles}" ] ; then
> + local option=""
> + for key in ${keyfiles}; do
> + keyid=$(gpg --keyid-format 0xlong --with-colons ${key}
> 2>/dev/null | grep "^pub:" | awk -F':' '{print $5;}')
> + option="${option}${keyid} "
> + done
> + echo "SignWith: ${option}" >> "${dir}"/conf/distributions
> + fi
> + fi
> + if [ ! -d "${dbdir}" ]; then
> + reprepro -b "${dir}" --dbdir "${dbdir}" export "${codename}"
> + fi
> +}
> +
> +repo_add_packages() {
> + local dir="$1"
> + local dbdir="$2"
> + local codename="$3"
> + shift; shift; shift
> +
> + if [ -n "${GNUPGHOME}" ]; then
> + export GNUPGHOME="${GNUPGHOME}"
> + fi
> + reprepro -b "${dir}" --dbdir "${dbdir}" -C main \
> + includedeb "${codename}" \
> + "$@"
> +}
> +
> +repo_del_package() {
> + local dir="$1"
> + local dbdir="$2"
> + local codename="$3"
> + local file="$4"
> +
> + if [ -n "${GNUPGHOME}" ]; then
> + export GNUPGHOME="${GNUPGHOME}"
> + fi
> + local p=$( dpkg-deb --show --showformat '${Package}' "${file}" )
> + local a=$( dpkg-deb --show --showformat '${Architecture}'
> "${file}" )
> + # removing "all" means no arch
> + local aarg="-A ${a}"
> + [ "${a}" = "all" ] && aarg=""
> + reprepro -b "${dir}" --dbdir "${dbdir}" -C main ${aarg} \
> + remove "${codename}" \
> + "${p}"
> +}
> diff --git a/meta/recipes-devtools/base-apt/base-apt.bb
> b/meta/recipes-devtools/base-apt/base-apt.bb index 9a0f7c8..8bb169c
> 100644 --- a/meta/recipes-devtools/base-apt/base-apt.bb
> +++ b/meta/recipes-devtools/base-apt/base-apt.bb
> @@ -1,45 +1,27 @@
> # This software is a part of ISAR.
> # Copyright (C) 2018 ilbers GmbH
> +# Copyright (C) 2019 Siemens AG
> +#
> +# SPDX-License-Identifier: MIT
> +
> +inherit repository
>
> SRC_URI = "file://distributions.in"
>
> BASE_REPO_KEY ?= ""
> KEYFILES ?= ""
>
> -CACHE_CONF_DIR = "${REPO_BASE_DIR}/${BASE_DISTRO}/conf"
> -do_cache_config[dirs] = "${CACHE_CONF_DIR}"
> do_cache_config[stamp-extra-info] = "${DISTRO}"
> do_cache_config[lockfiles] = "${REPO_BASE_DIR}/isar.lock"
>
> # Generate reprepro config for current distro if it doesn't exist.
> Once it's # generated, this task should do nothing.
> repo_config() {
> - if [ -n "${GNUPGHOME}" ]; then
> - export GNUPGHOME="${GNUPGHOME}"
> - fi
> -
> - if [ ! -e "${CACHE_CONF_DIR}/distributions" ]; then
> - sed -e "s#{CODENAME}#"${BASE_DISTRO_CODENAME}"#g" \
> - ${WORKDIR}/distributions.in >
> ${CACHE_CONF_DIR}/distributions
> - if [ -n "${KEYFILES}" ]; then
> - option=""
> - for key in ${KEYFILES}; do
> - keyid=$(gpg --keyid-format 0xlong --with-colons
> ${key} 2>/dev/null | grep "^pub:" | awk -F':' '{print $5;}')
> - option="${option}${keyid} "
> - done
> - # To generate Release.gpg
> - echo "SignWith: ${option}" >>
> ${CACHE_CONF_DIR}/distributions
> - fi
> - fi
> -
> - path_cache="${REPO_BASE_DIR}/${BASE_DISTRO}"
> - path_databases="${REPO_BASE_DB_DIR}/${BASE_DISTRO}"
> -
> - if [ ! -d "${path_databases}" ]; then
> - reprepro -b ${path_cache} \
> - --dbdir ${path_databases} \
> - export ${BASE_DISTRO_CODENAME}
> - fi
> + repo_create "${REPO_BASE_DIR}"/"${BASE_DISTRO}" \
> + "${REPO_BASE_DB_DIR}"/"${BASE_DISTRO}" \
> + "${BASE_DISTRO_CODENAME}" \
> + "${WORKDIR}/distributions.in" \
> + "${KEYFILES}"
> }
>
> python do_cache_config() {
> diff --git a/meta/recipes-devtools/isar-apt/files/distributions.in
> b/meta/recipes-devtools/isar-apt/files/distributions.in index
> 6471736..3cf7ea5 100644 ---
> a/meta/recipes-devtools/isar-apt/files/distributions.in +++
> b/meta/recipes-devtools/isar-apt/files/distributions.in @@ -1,3 +1,3
> @@ -Codename: {DISTRO_NAME}
> +Codename: {CODENAME}
> Architectures: i386 armhf arm64 amd64 mipsel riscv64 source
> Components: main
> diff --git a/meta/recipes-devtools/isar-apt/isar-apt.bb
> b/meta/recipes-devtools/isar-apt/isar-apt.bb index e6f1753..c052934
> 100644 --- a/meta/recipes-devtools/isar-apt/isar-apt.bb
> +++ b/meta/recipes-devtools/isar-apt/isar-apt.bb
> @@ -1,32 +1,23 @@
> # This software is a part of ISAR.
> # Copyright (C) 2015-2017 ilbers GmbH
> +# Copyright (C) 2019 Siemens AG
> +#
> +# SPDX-License-Identifier: MIT
> +
> +inherit repository
>
> SRC_URI = "file://distributions.in"
>
> -CACHE_CONF_DIR = "${REPO_ISAR_DIR}/${DISTRO}/conf"
> -do_cache_config[dirs] = "${CACHE_CONF_DIR}"
> do_cache_config[stamp-extra-info] = "${DISTRO}"
> do_cache_config[lockfiles] = "${REPO_ISAR_DIR}/isar.lock"
>
> # Generate reprepro config for current distro if it doesn't exist.
> Once it's # generated, this task should do nothing.
> do_cache_config() {
> - if [ ! -e "${CACHE_CONF_DIR}/distributions" ]; then
> - sed -e "s#{DISTRO_NAME}#"${DEBDISTRONAME}"#g" \
> - ${WORKDIR}/distributions.in >
> ${CACHE_CONF_DIR}/distributions
> - fi
> -
> - path_cache="${REPO_ISAR_DIR}/${DISTRO}"
> - path_databases="${REPO_ISAR_DB_DIR}/${DISTRO}"
> -
> - if [ ! -d "${path_databases}" ]; then
> - if [ -n "${GNUPGHOME}" ]; then
> - export GNUPGHOME="${GNUPGHOME}"
> - fi
> - reprepro -b ${path_cache} \
> - --dbdir ${path_databases} \
> - export ${DEBDISTRONAME}
> - fi
> + repo_create "${REPO_ISAR_DIR}"/"${DISTRO}" \
> + "${REPO_ISAR_DB_DIR}"/"${DISTRO}" \
> + "${DEBDISTRONAME}" \
> + "${WORKDIR}/distributions.in"
> }
>
> addtask cache_config after do_unpack before do_build
next prev parent reply other threads:[~2020-03-10 13:14 UTC|newest]
Thread overview: 45+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-01-31 14:29 [PATCHv4 00/26] base-apt-rework Henning Schild
2020-01-31 14:29 ` [PATCHv4 01/26] repository: new class to deal with repos Henning Schild
2020-01-31 14:29 ` [PATCHv4 02/26] dpkg-base: add download caching of apt:// downloads Henning Schild
2020-01-31 14:29 ` [PATCHv4 03/26] meta: dpkg-base: convert "exit" into "return 0" Henning Schild
2020-01-31 14:29 ` [PATCHv4 04/26] base-apt: change the sources.list to also offer deb-src Henning Schild
2020-01-31 14:29 ` [PATCHv4 05/26] base-apt: add deb-src packages as well Henning Schild
2020-01-31 14:29 ` [PATCHv4 06/26] base-apt: do not skip gpg check when it is signed Henning Schild
2020-01-31 14:29 ` [PATCHv4 07/26] ci: conf: add "hello" to the sample config and every build Henning Schild
2020-01-31 14:29 ` [PATCHv4 08/26] meta: split all apt-get invocations into download and execution Henning Schild
2020-01-31 14:29 ` [PATCHv4 09/26] meta: create DL_DIR support for all apt-get downloaded .debs Henning Schild
2020-01-31 14:29 ` [PATCHv4 10/26] meta: import DL_DIR debs before apt-get download steps Henning Schild
2020-01-31 14:29 ` [PATCHv4 11/26] meta: include DL_DIR deb import/export into bootstrap Henning Schild
2020-01-31 14:29 ` [PATCHv4 12/26] base-apt: populate from DEBDIR as well Henning Schild
2020-01-31 14:29 ` [PATCHv4 13/26] base-apt: drop the "apt_cache" feature Henning Schild
2020-01-31 14:29 ` [PATCHv4 14/26] base-apt: do not copy debs directly out of rootfss anymore Henning Schild
2020-01-31 14:29 ` [PATCHv4 15/26] base-apt: rework base-apt population Henning Schild
2020-01-31 14:29 ` [PATCHv4 16/26] base-apt: move class "base-apt-helper" into only user Henning Schild
2020-01-31 14:29 ` [PATCHv4 17/26] CI: include "isar-disable-apt-cache" into all CI images Henning Schild
2020-01-31 14:29 ` [PATCHv4 18/26] CI: include "cowsay" into default build to test dpkg-gbp Henning Schild
2020-01-31 14:29 ` [PATCHv4 19/26] CI: set BB_NO_NETWORK for cached rebuild Henning Schild
2020-01-31 14:29 ` [PATCHv4 20/26] meta: repository: implement repo_contains_package and use it in base_apt Henning Schild
2020-01-31 14:29 ` [PATCHv4 21/26] meta: repository: simplify the matching of packages Henning Schild
2020-01-31 14:29 ` [PATCHv4 22/26] meta: deb-dl-dir: do not cache debs from isar-apt Henning Schild
2020-02-03 17:20 ` Jan Kiszka
2020-02-03 18:27 ` Henning Schild
2020-02-03 18:59 ` Henning Schild
2020-02-04 2:11 ` Su, Bao Cheng
2020-02-04 5:10 ` Su, Bao Cheng
2020-02-04 5:14 ` Jan Kiszka
2020-02-04 5:31 ` Su, Bao Cheng
2020-02-04 16:26 ` Henning Schild
2020-02-04 9:33 ` Henning Schild
2020-02-05 12:11 ` Henning Schild
2020-02-05 14:21 ` Su, Bao Cheng
2020-01-31 14:29 ` [PATCHv4 23/26] base-apt: pull base-apt population to the front of the build chain Henning Schild
2020-01-31 14:29 ` [PATCHv4 24/26] meta: base-apt: remove isar-apt check from population Henning Schild
2020-01-31 14:29 ` [PATCHv4 25/26] meta: deb-dl-dir: only export newly downloaded files Henning Schild
2020-01-31 14:30 ` [PATCHv4 26/26] CI: test a custom kernel build in the base-apt offline rebuild Henning Schild
2020-01-31 14:30 ` [PATCHv4 00/26] base-apt-rework Henning Schild
2020-03-04 23:25 ` Baurzhan Ismagulov
2020-03-04 23:28 ` [PATCH v5 01/26] repository: new class to deal with repos Baurzhan Ismagulov
2020-03-10 13:14 ` Henning Schild [this message]
2020-03-10 14:13 ` Baurzhan Ismagulov
2020-03-04 23:28 ` [PATCH v5 08/26] meta: split all apt-get invocations into download and execution Baurzhan Ismagulov
2020-03-10 13:01 ` [PATCHv4 00/26] base-apt-rework Henning Schild
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200310141437.705cd61c@md1za8fc.ad001.siemens.net \
--to=henning.schild@siemens.com \
--cc=ibr@radix50.net \
--cc=isar-users@googlegroups.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox