From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6662700470316826624 X-Received: by 2002:a1c:2d86:: with SMTP id t128mr758019wmt.2.1551889886374; Wed, 06 Mar 2019 08:31:26 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a1c:6c01:: with SMTP id h1ls297553wmc.13.gmail; Wed, 06 Mar 2019 08:31:25 -0800 (PST) X-Google-Smtp-Source: APXvYqyP1zLgrIuts8qHxnnGrMlM+VJ2rYVPXx50o0dFAiLsWHfcci3vjWF/3zRqq0vk4njkfPem X-Received: by 2002:a1c:1a4a:: with SMTP id a71mr690438wma.21.1551889885950; Wed, 06 Mar 2019 08:31:25 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1551889885; cv=none; d=google.com; s=arc-20160816; b=D2CnsthNwnbpWQrR9R1O4HMCGxNRKiELa1vlbJaQzIzYqVoZZ8fLkAvzewInFZtWT2 I8fRtRKsHsCkqNbtwBGpOm6e9FZBPN3++zNGu3NMDeAbRbZnjm4IY3ktrlDhzpvHN0ss kuEFBTfHwsmnWrJ8OQid9TlNFFiH5TmHAsZCp1wwR3xCtjXO7FXptcUmG43zFtAG/92f N990vFNZEqW48cZFIJEhqywt0O4Aq6t+83d4FjhC8C1VKju1Ah6eECNh7DFeYN6V3JQh 4WHHIrEQAO4kug2iSk5K3lFWwqj7oGkbD3g6zUtYm88/jtQf0VtjMZXfjALIIGuqRXvf rQUg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=user-agent:in-reply-to:content-disposition:content-description :mime-version:references:message-id:subject:cc:to:from:date; bh=3JHp+w7he4MTsZ42zSZikFWMBrAeyrBifUvaliOTM4A=; b=yv/P+Xs8GRFcwZ7M2yy89LKQ3YYWbBvFc8U8jDgODKTuHbUdBghMM839KAAmxotDAT k6PX90bujM/0EkNQ7D47U1tpVTZkpwk/TZTz4IYzW9VQsnSHH7InKGSuWYNp0Y2chQdy 7Q7CG7Q776+SyaQMTzDS7/VR1uPxfFdiB12mAMM8V/f2v35daHxti4rpXVEDXaLGx7Uf YkNWsnTUqBazkFX/34LFpP01DMRsQ8BGZfRutUQoNz6NLzJKzolwEeGOKHzrypfzeRZs yeg/hsj4NHKYPdmXF+dFKXlPldMg1vjSwDEw/kGc5n76S3sJ7k6LhYWTAlBkHmsGFN3o OnMA== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of andreas.reichel.ext@siemens.com designates 192.35.17.14 as permitted sender) smtp.mailfrom=andreas.reichel.ext@siemens.com Return-Path: Received: from david.siemens.de (david.siemens.de. [192.35.17.14]) by gmr-mx.google.com with ESMTPS id b8si101617wru.1.2019.03.06.08.31.25 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 06 Mar 2019 08:31:25 -0800 (PST) Received-SPF: pass (google.com: domain of andreas.reichel.ext@siemens.com designates 192.35.17.14 as permitted sender) client-ip=192.35.17.14; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of andreas.reichel.ext@siemens.com designates 192.35.17.14 as permitted sender) smtp.mailfrom=andreas.reichel.ext@siemens.com Received: from mail1.sbs.de (mail1.sbs.de [192.129.41.35]) by david.siemens.de (8.15.2/8.15.2) with ESMTPS id x26GVPO3006721 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Wed, 6 Mar 2019 17:31:25 +0100 Received: from iiotirae (golem.ppmd.siemens.net [139.25.69.17]) by mail1.sbs.de (8.15.2/8.15.2) with SMTP id x26GVPEi032755; Wed, 6 Mar 2019 17:31:25 +0100 Date: Wed, 6 Mar 2019 17:29:38 +0100 From: Andreas Reichel To: Henning Schild Cc: isar-users@googlegroups.com Subject: Re: [PATCH v2 1/3] Fix and simplify apt keyring generation Message-ID: <20190306162938.GB2005@iiotirae> References: <20190227151856.11594-1-andreas.reichel.ext@siemens.com> <20190227151856.11594-2-andreas.reichel.ext@siemens.com> <20190227171242.5c2c73b4@md1za8fc.ad001.siemens.net> <20190301090949.GB10350@iiotirae> <20190301111848.25e60eee@md1za8fc.ad001.siemens.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Description: message Content-Disposition: inline In-Reply-To: <20190301111848.25e60eee@md1za8fc.ad001.siemens.net> User-Agent: Mutt/1.11.3 (2019-02-01) X-TUID: PYHU06fS7oSF On Fri, Mar 01, 2019 at 11:18:48AM +0100, Henning Schild wrote: > Am Fri, 1 Mar 2019 10:09:49 +0100 > schrieb Andreas Reichel : > > > On Wed, Feb 27, 2019 at 05:12:42PM +0100, Henning Schild wrote: > > > Am Wed, 27 Feb 2019 16:18:54 +0100 > > > schrieb "[ext] Andreas J. Reichel" > > > : > > > > From: Andreas Reichel > > > > > > > > Different fetcher stored keys in different sub dirs, and we can > > > > never be sure about where downloaded files go. > > > > To avoid this without making any assumptions, we ask the fetcher > > > > where the file will be after it is downloaded. This way we also > > > > don't need to parse the URL manually. > > > > > > > > The code is simplified by removing duplicate code and using > > > > apt-key instead of manually calling gpg. > > > > > > > > Signed-off-by: Andreas Reichel > > > > --- > > > > meta/classes/isar-bootstrap-helper.bbclass | 11 ++++++ > > > > .../isar-bootstrap/isar-bootstrap.inc | 39 > > > > +++++++++---------- 2 files changed, 29 insertions(+), 21 > > > > deletions(-) > > > > > > > > diff --git a/meta/classes/isar-bootstrap-helper.bbclass > > > > b/meta/classes/isar-bootstrap-helper.bbclass index > > > > d780b85..b8c41f9 100644 --- > > > > a/meta/classes/isar-bootstrap-helper.bbclass +++ > > > > b/meta/classes/isar-bootstrap-helper.bbclass @@ -119,6 +119,16 @@ > > > > setup_root_file_system() { export LANG=C > > > > export LANGUAGE=C > > > > export LC_ALL=C > > > > + > > > > + if [ -d ${TMPDIR}/aptkeys ]; then > > > > + for keyfile in ${TMPDIR}/aptkeys/* > > > > + do > > > > + kfn="tmp/$(basename $keyfile)" > > > > + cp $keyfile "$ROOTFSDIR/$kfn" > > > > + sudo -E chroot "$ROOTFSDIR" /usr/bin/apt-key add > > > > "/$kfn" > > > > > > Would probably be a good idea to work in /tmp/ as well and in a > > > subdir. Try naming your key "root" or "usr" ... > > > > > > > + rm "$ROOTFSDIR/$kfn" > > > > + done > > > > + fi > > > > sudo -E chroot "$ROOTFSDIR" /usr/bin/apt-get update \ > > > > -o Dir::Etc::sourcelist="sources.list.d/isar-apt.list" \ > > > > -o Dir::Etc::sourceparts="-" \ > > > > @@ -128,6 +138,7 @@ setup_root_file_system() { > > > > sudo -E chroot "$ROOTFSDIR" /usr/bin/dpkg > > > > --add-architecture ${DISTRO_ARCH} sudo -E chroot > > > > "$ROOTFSDIR" /usr/bin/apt-get update fi > > > > + sudo -E chroot "$ROOTFSDIR" /usr/bin/apt-key update > > > > sudo -E chroot "$ROOTFSDIR" \ > > > > /usr/bin/apt-get ${APT_ARGS} --download-only $PACKAGES \ > > > > ${IMAGE_TRANSIENT_PACKAGES} > > > > diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc > > > > b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc index > > > > 234d339..2ef3b1e 100644 --- > > > > a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc +++ > > > > b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc @@ -23,35 > > > > +23,30 @@ APTSRCS = "${WORKDIR}/apt-sources" APTSRCS_INIT = > > > > "${WORKDIR}/apt-sources-init" BASEAPTSRCS = > > > > "${WORKDIR}/base-apt-sources" APTKEYFILES = "" > > > > -APTKEYRING = "${WORKDIR}/apt-keyring.gpg" > > > > -DEBOOTSTRAP_KEYRING = "" > > > > DEPLOY_ISAR_BOOTSTRAP ?= "" > > > > -DISTRO_BOOTSTRAP_BASE_PACKAGES = "locales" > > > > +DISTRO_BOOTSTRAP_BASE_PACKAGES = "locales gnupg2 > > > > apt-transport-https ca-certificates" > > > > > > That looks more like leftovers from an experiment. Are you sure > > > that is needed? > > > > This is needed because debootstrap can switch to https by itself if it > > is unsure about the sources (apt-transport-https, ca-certificates), > > gnupg2 we need for apt-key. > > Adding the packages should probably depend on whether DISTRO_APT_KEYS > is non-empty. And adding the ones that the https-detector adds should > be tested, not that dual mention makes debootstrap crap out. > > I guess the OVERRIDES method from 9196b820636a9 should be used for gpg > support as well. > Done in v3. > Henning > > > > > > > > > > In general i think that patch should be splittet into two or > > > even more patches. Because it solves at least two problems. > > > > > > Henning > > > > > > > DISTRO_APT_PREMIRRORS ?= "${@ "http://ftp\.(\S+\.)?debian.org > > > > file:///${REPO_BASE_DIR} \n" if > > > > bb.utils.to_boolean(d.getVar('ISAR_USE_CACHED_BASE_REPO')) else > > > > "" }" inherit base-apt-helper > > > > python () { > > > > - from urllib.parse import urlparse > > > > distro_apt_keys = d.getVar("DISTRO_APT_KEYS", False) > > > > - wd = d.getVar("WORKDIR", True) > > > > + aptkeys = [] > > > > + > > > > if distro_apt_keys: > > > > - d.setVar("DEBOOTSTRAP_KEYRING", "--keyring > > > > ${APTKEYRING}") > > > > - for key in distro_apt_keys.split(): > > > > - url = urlparse(key) > > > > - filename = ''.join([wd, url.path]) > > > > - d.appendVar("SRC_URI", " %s" % key) > > > > - d.appendVar("APTKEYFILES", " %s" % filename) > > > > + aptkeys += distro_apt_keys.split() > > > > + > > > > if > > > > bb.utils.to_boolean(d.getVar('ISAR_USE_CACHED_BASE_REPO')): > > > > own_pub_key = d.getVar("BASE_REPO_KEY", False) if own_pub_key: > > > > - d.setVar("DEBOOTSTRAP_KEYRING", "--keyring > > > > ${APTKEYRING}") > > > > - for key in own_pub_key.split(): > > > > - url = urlparse(key) > > > > - filename = ''.join([wd, url.path]) > > > > - d.appendVar("SRC_URI", " %s" % key) > > > > - d.appendVar("APTKEYFILES", " %s" % filename) > > > > + aptkeys += own_pub_keys.split() > > > > + > > > > + for key in aptkeys: > > > > + d.appendVar("SRC_URI", " %s" % key) > > > > + fetcher = bb.fetch2.Fetch([key], d) > > > > + filename = fetcher.localpath(key) > > > > + d.appendVar("APTKEYFILES", " %s" % filename) > > > > } > > > > > > > > def aggregate_files(d, file_list, file_out): > > > > @@ -174,13 +169,17 @@ def get_distro_components_argument(d, > > > > is_host): else: > > > > return "" > > > > > > > > +APTKEYTMPDIR := "${TMPDIR}/aptkeys" > > > > + > > > > +do_generate_keyring[cleandirs] = "${APTKEYTMPDIR}" > > > > do_generate_keyring[dirs] = "${DL_DIR}" > > > > do_generate_keyring[vardeps] += "DISTRO_APT_KEYS" > > > > do_generate_keyring() { > > > > if [ -n "${@d.getVar("APTKEYFILES", True) or ""}" ]; then > > > > + chmod 777 "${APTKEYTMPDIR}" > > > > for keyfile in ${@d.getVar("APTKEYFILES", True)}; do > > > > - gpg --no-default-keyring --keyring "${APTKEYRING}" \ > > > > - --no-tty --homedir "${DL_DIR}" --import > > > > "$keyfile" > > > > + cp "$keyfile" "${APTKEYTMPDIR}"/"$(basename > > > > "$keyfile")" > > > > + sudo apt-key add "$keyfile" > > > > done > > > > fi > > > > } > > > > @@ -222,7 +221,6 @@ isar_bootstrap() { > > > > if [ ${IS_HOST} ]; then > > > > ${DEBOOTSTRAP} $debootstrap_args \ > > > > ${@get_distro_components_argument(d, > > > > True)} \ > > > > - ${DEBOOTSTRAP_KEYRING} \ > > > > "${@get_distro_suite(d, True)}" \ > > > > "${ROOTFSDIR}" \ > > > > "${@get_distro_source(d, True)}" > > > > @@ -231,7 +229,6 @@ isar_bootstrap() { > > > > "${DEBOOTSTRAP}" $debootstrap_args \ > > > > --arch="${DISTRO_ARCH}" \ > > > > ${@get_distro_components_argument(d, > > > > False)} \ > > > > - ${DEBOOTSTRAP_KEYRING} \ > > > > "${@get_distro_suite(d, > > > > False)}" \ "${ROOTFSDIR}" \ > > > > "${@get_distro_source(d, > > > > False)}" > > > > > > -- Andreas Reichel Dipl.-Phys. (Univ.) Software Consultant Andreas.Reichel@tngtech.com, +49-174-3180074 TNG Technology Consulting GmbH, Betastr. 13a, 85774 Unterfoehring Geschaeftsfuehrer: Henrik Klagges, Dr. Robert Dahlke, Gerhard Mueller Sitz: Unterfoehring * Amtsgericht Muenchen * HRB 135082