From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6635926855683670016 X-Received: by 2002:a24:24e:: with SMTP id 75mr4843089itu.0.1545202362024; Tue, 18 Dec 2018 22:52:42 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a5d:818d:: with SMTP id u13ls3882419ion.3.gmail; Tue, 18 Dec 2018 22:52:41 -0800 (PST) X-Google-Smtp-Source: AFSGD/X/38lrGPoXxqmuIetJerjkyWHzmjvr63QXQwgMPMKfM/6hGrt3mp/17IBbC5bk+CjxMJ7J X-Received: by 2002:a5d:9a0b:: with SMTP id s11mr16880093iol.5.1545202361700; Tue, 18 Dec 2018 22:52:41 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1545202361; cv=none; d=google.com; s=arc-20160816; b=oHrEooRnULz06q1xdrnAgQHz1jPHkJKFqC2EeDkIazEvDf1u+k7FV/I91rNwnHiZwU ROjDTDfu7yIE05JAPETZ2zyyNrZVene0aHuANK3hGmqjFxbMqMbrT6n0C0Qk1IqQWTzo cMU6uwqEVw8XACwCWRA97Tz9erLQ4I7s/lb4mAauYYVsYZgakMknN5eS3BgrZGmo8OmH XLYtPouGF6dME6dFtPFWrpqHWtLsrCXEt9b7UWX0cKUxnYCWBVK5T6FRNyWLTePe6WrF wMzK4vdWlzYezcycam6pqrrjxIM8usSQpNEkRw8eY9bIFbhG3J1F8zKcJ4cC3B3r14zh H1Yw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:dkim-signature; bh=PcjWFiS3nDgQyR01Bu5SiRFZ73iudVmKs1pEfUuEA2U=; b=bLq3xUvyvfivxOkNTvq6k4LUli/Synuo5hUKSnztrw55E4RnlMORNJMtBjBVtKOiw0 OWw6L0MVS+5hZMPCpwoIJ2JXh5vIPQB3ySfCjnD5aUFYkl5oQB7W7MWJ10HE1o++l+vq f2zZFMeS9+wnj8ZPNcHAURCJgZ4m9s8Cf6WH+QZu32Yqpyge945a7+SKdAqdnjZbz2W0 VjoVjIyf+BviU2TOyemmDqSGLZuzDGC6FKv0JDJh4p/lq5Rgb8Dncspymvdq8BqUzneQ R3JmpthvibZzfMHGvdyAT0dfLZCAgNBkrWlExl4mVE42RAde2/RO7dcHTT/2hf3sYNee P4Aw== ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=ptyhePJn; spf=pass (google.com: domain of fasdfasdas@gmail.com designates 2607:f8b0:4864:20::d32 as permitted sender) smtp.mailfrom=fasdfasdas@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from mail-io1-xd32.google.com (mail-io1-xd32.google.com. [2607:f8b0:4864:20::d32]) by gmr-mx.google.com with ESMTPS id g1si9222iom.2.2018.12.18.22.52.41 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 18 Dec 2018 22:52:41 -0800 (PST) Received-SPF: pass (google.com: domain of fasdfasdas@gmail.com designates 2607:f8b0:4864:20::d32 as permitted sender) client-ip=2607:f8b0:4864:20::d32; Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=ptyhePJn; spf=pass (google.com: domain of fasdfasdas@gmail.com designates 2607:f8b0:4864:20::d32 as permitted sender) smtp.mailfrom=fasdfasdas@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: by mail-io1-xd32.google.com with SMTP id p7so7148025iog.12 for ; Tue, 18 Dec 2018 22:52:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=PcjWFiS3nDgQyR01Bu5SiRFZ73iudVmKs1pEfUuEA2U=; b=ptyhePJn1DYD3HLUtPqdnGZvF+Ff7ABkyCMnblee6FqCDJya2WPRY7GY5U34QbY3kn OEevYd+6+kkovN3EDYFP7WG7OFCl8VJTkkU1XnN3X9vW9Kdizjfa4ho4EOrMcBkNDPTp olTQXQzNMDvavRKMY/iwahcIaZNaSwIEvp4JEkxzKYZJwHsYjjualpn+kx9fxNrAQ9K6 9+APhfdGUfiHEo/jNrx+0Ls3tvW3IaErZJGyFcdBjNZxvW8fmj4NeoPPrrLsIwf5wdl2 gx3/0QWcHMaX4Bk64NQ4RZ87oKtrls3KqXBDQ/qdXXqEHimMdm2iW8dJVj+wNtws82Yi HJAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=PcjWFiS3nDgQyR01Bu5SiRFZ73iudVmKs1pEfUuEA2U=; b=pcIzImwhTrVS+yxGIfTd3inzxbfoR/TDB06rdd9EeGcH0P6powsb8xWzGdgekjHE0T NglWPGLpV/V1tvzP6YxwRFCHWTWlzTOPlwdSBJ7NZ61iU8hQTeLAal1wz/U/Bi4Kd8TI M+Fw9qWIszOF6m/UzuR/KVb2t/kjiRYrkCFLZbUHijLFogXc6UIrDh7x0bccSDAJKS5m akPKFQbBAKaMGQ/3o5+YphgoFk5PYqf/6ksIv5r94stpV9cwxLZX3eZIWXAfvCsnF7I4 Ic9cm7aOrq4+3hJEG5Ez1fJRY0XmI1TwTsoVMYETYrDNS0LUoShDSbjWLW3+adieCMt3 NmtQ== X-Gm-Message-State: AJcUukfH7cAXjCZ7FdHPfue/5lnl8XBZrUo6DPVnTEfNmBtgzIbaO2Bu +/ZRhlyQ7Z/OJXgLCKrYOhHVqZAkqXUolDBRY3PFy5Wn X-Received: by 2002:a5d:9696:: with SMTP id m22mr41105ion.243.1545202361204; Tue, 18 Dec 2018 22:52:41 -0800 (PST) MIME-Version: 1.0 References: <20181217114518.17995-1-tolga.hosgor@siemens.com> <20181217114518.17995-2-tolga.hosgor@siemens.com> In-Reply-To: From: =?UTF-8?B?VG9sZ2EgSG/Fn2fDtnI=?= Date: Wed, 19 Dec 2018 09:52:28 +0300 Message-ID: Subject: Re: [PATCH v2 1/2] isar-bootstrap: debootstrap https support To: Claudius Heine Cc: =?UTF-8?B?VG9sZ2EgSG/Fn2fDtnI=?= , isar-users@googlegroups.com Content-Type: text/plain; charset="UTF-8" X-TUID: Ee/p+RkGiG14 Hello, On Tue, Dec 18, 2018 at 4:20 PM Claudius Heine wrote: > > Hi again, > > On 17/12/2018 12.45, Hosgor, Tolga (CT RDA DS EU TR MTS) wrote: > > Building 'isar-bootstrap-target' fails when there are HTTPS URIs in > > the distro APT sources and the first URI is not HTTPS. > > The first URI in the APT sources is passed to 'debootstrap' and > > the distro APT sources file is written to the isar bootstrap rootfs. > > Then, following 'apt-get update' fails due to apt-transport-https, > > ca-certificates being missing. > > > > This patch allows a distro to specify 'DISTRO_BOOTSTRAP_BASE_PACKAGES' > > and introduces 'https-support' concept using bitbake OVERRIDES. > > An example usage is specifying the necessary packages for http support > > via 'DISTRO_BOOTSTRAP_BASE_PACKAGES_append_https-support' in distro > > configuration. > > > > Signed-off-by: Hosgor, Tolga (CT RDA DS EU TR MTS) > > --- > > meta-isar/conf/distro/debian-buster.list | 1 + > > meta-isar/conf/distro/debian-jessie.list | 1 + > > .../conf/multiconfig/qemuamd64-buster.conf | 2 ++ > > .../conf/multiconfig/qemuamd64-jessie.conf | 2 ++ > > .../isar-bootstrap/isar-bootstrap-host.bb | 2 ++ > > .../isar-bootstrap/isar-bootstrap-target.bb | 2 ++ > > .../isar-bootstrap/isar-bootstrap.inc | 29 +++++++++++++++++-- > > 7 files changed, 36 insertions(+), 3 deletions(-) > > > > diff --git a/meta-isar/conf/distro/debian-buster.list b/meta-isar/conf/distro/debian-buster.list > > index 18311d8..92d20e1 100644 > > --- a/meta-isar/conf/distro/debian-buster.list > > +++ b/meta-isar/conf/distro/debian-buster.list > > @@ -1,3 +1,4 @@ > > deb http://ftp.de.debian.org/debian buster main contrib non-free > > +deb https://debian.inf.tu-dresden.de/debian buster main contrib non-free > > deb http://ftp.de.debian.org/debian buster-updates main contrib non-free > > deb http://security.debian.org buster/updates main contrib non-free > > diff --git a/meta-isar/conf/distro/debian-jessie.list b/meta-isar/conf/distro/debian-jessie.list > > index be46a57..2471402 100644 > > --- a/meta-isar/conf/distro/debian-jessie.list > > +++ b/meta-isar/conf/distro/debian-jessie.list > > @@ -1,3 +1,4 @@ > > +deb https://debian.inf.tu-dresden.de/debian jessie main contrib non-free > > deb http://ftp.de.debian.org/debian jessie main contrib non-free > > deb http://ftp.de.debian.org/debian jessie-updates main contrib non-free > > deb http://security.debian.org jessie/updates main contrib non-free > > diff --git a/meta-isar/conf/multiconfig/qemuamd64-buster.conf b/meta-isar/conf/multiconfig/qemuamd64-buster.conf > > index 059ea00..394d76b 100644 > > --- a/meta-isar/conf/multiconfig/qemuamd64-buster.conf > > +++ b/meta-isar/conf/multiconfig/qemuamd64-buster.conf > > @@ -15,3 +15,5 @@ QEMU_ARCH ?= "x86_64" > > QEMU_MACHINE ?= "q35" > > QEMU_CPU ?= "" > > QEMU_DISK_ARGS ?= "-hda ##ROOTFS_IMAGE## -bios /usr/local/share/ovmf/OVMF.fd" > > + > > +DISTRO_BOOTSTRAP_BASE_PACKAGES_append_https-support = " apt-transport-https ca-certificates" > > diff --git a/meta-isar/conf/multiconfig/qemuamd64-jessie.conf b/meta-isar/conf/multiconfig/qemuamd64-jessie.conf > > index ca00e15..d1335ff 100644 > > --- a/meta-isar/conf/multiconfig/qemuamd64-jessie.conf > > +++ b/meta-isar/conf/multiconfig/qemuamd64-jessie.conf > > @@ -14,3 +14,5 @@ QEMU_ARCH ?= "x86_64" > > QEMU_MACHINE ?= "pc" > > QEMU_CPU ?= "" > > QEMU_DISK_ARGS ?= "-hda ##ROOTFS_IMAGE##" > > + > > +DISTRO_BOOTSTRAP_BASE_PACKAGES_append_https-support = " apt-transport-https ca-certificates" > > diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb b/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb > > index 55696ea..47cff63 100644 > > --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb > > +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb > > @@ -40,6 +40,8 @@ python do_apt_config_prepare() { > > } > > addtask apt_config_prepare before do_bootstrap after do_unpack > > > > +OVERRIDES_append = ":${@get_distro_needs_https_support(d, True)}" > > + > > do_bootstrap[stamp-extra-info] = "${HOST_DISTRO}-${HOST_ARCH}" > > do_bootstrap[vardeps] += "HOST_DISTRO_APT_SOURCES" > > do_bootstrap() { > > diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb b/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb > > index 5752b14..57b607f 100644 > > --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb > > +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb > > @@ -39,6 +39,8 @@ python do_apt_config_prepare() { > > } > > addtask apt_config_prepare before do_bootstrap after do_unpack > > > > +OVERRIDES_append = ":${@get_distro_needs_https_support(d, False)}" > > + > > do_bootstrap[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" > > do_bootstrap[vardeps] += "DISTRO_APT_SOURCES" > > do_bootstrap() { > > diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc > > index cfad136..d868bb6 100644 > > --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc > > +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc > > @@ -23,6 +23,7 @@ APTKEYFILES = "" > > APTKEYRING = "${WORKDIR}/apt-keyring.gpg" > > DEBOOTSTRAP_KEYRING = "" > > DEPLOY_ISAR_BOOTSTRAP ?= "" > > +DISTRO_BOOTSTRAP_BASE_PACKAGES = "locales" > > > > python () { > > from urllib.parse import urlparse > > @@ -109,11 +110,15 @@ def aggregate_aptsources_list(d, file_list, file_out): > > out_fd.write("\n".encode()) > > out_fd.write("\n".encode()) > > > > -def get_distro_primary_source_entry(d, is_host=False): > > +def get_aptsources_list(d, is_host=False): > > if is_host: > > apt_sources_list = (d.getVar("HOST_DISTRO_APT_SOURCES", True) or "").split() > > else: > > apt_sources_list = (d.getVar("DISTRO_APT_SOURCES", True) or "").split() > > + return apt_sources_list > > After some thought really don't like this `is_host` parameter... > > For me that looks like some kind of backwards approach. First you have > the difference between target and host, when you use in the different > files `isar-bootstrap-target.bb` and `isar-bootstrap-host.bb` and then, > because you use common functionality from `isar-bootstrap.inc` you need > to put this difference into a boolean and pass that to all of those > functions. Yeah it is really counter-intuitive. I would expect a variable to be filled with the distro source decision at some point and go on with it. This also applies to the rest of the code (the two "debootstrap" calls with True/False parameters to functions). > > IMO the right solutions would have been, to do it like this: > > isar-bootstrap.inc: > OVERRIDES_append = ":${@get_distro_needs_https_support(d)}" > def get_aptsources_list(d): > return (d.getVar("BOOTSTRAP_DISTRO_APT_SOURCES", True) or > "").split() > > isar-bootstrap-target.bb: > BOOTSTRAP_DISTRO_APT_SOURCE = "${DISTRO_APT_SOURCES}" > > isar-bootstrap-host.bb: > BOOTSTRAP_DISTRO_APT_SOURCE = "${HOST_DISTRO_APT_SOURCES}" Exactly. > > Of course that is not in scope of your patch, but your patch > demonstrates that weakness nicely by splitting up that function. > > Claudius > > -- > DENX Software Engineering GmbH, Managing Director: Wolfgang Denk > HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany > Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de