From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6635926855683670016 X-Received: by 2002:a17:906:2352:: with SMTP id m18-v6mr2499445eja.9.1545139203984; Tue, 18 Dec 2018 05:20:03 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a50:b656:: with SMTP id c22ls4806522ede.0.gmail; Tue, 18 Dec 2018 05:20:03 -0800 (PST) X-Google-Smtp-Source: AFSGD/XP4auPYslaEbdsEJpsKB56DSSVEGlRJsIJjkHM7a+Dftw2zy7Hyp8tHpjqKEbVsHS5NqzX X-Received: by 2002:a50:ac53:: with SMTP id w19mr3255040edc.3.1545139203530; Tue, 18 Dec 2018 05:20:03 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1545139203; cv=none; d=google.com; s=arc-20160816; b=N3ozEfl8tPiminS0n/rwdjuFDnY6IxU1uDEt+2NqIPOc4CYy+8+/r6ZoIFUXKg3bZw k3exwSKXayRDXcYAVRDotYmrczu2JXriOC3Ab9X6AbE3k7xxPm6jfkXYX3lsuHAdY3/2 g5Ca859UCP4gvghWBGTKoXIa7ZVeapCfTay3t5EZi3zjrDqmteePuQhi2E8nWITmqaqe 44+/qd6JsFiV9pqYEhpESrsyVTxa616khHALH0V7u1IrAMqBNgPkoN5IRhmVX4/RN0DX D82Oa9d9aypANrOi7Nvbb9UulZo8wAk2zDxYnzAV7u7XCS5kjaI5bP7jcahxAKF1YZKX EZAA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:to:subject; bh=YX1Lsnz6L794sD/rJgi7WAvjLoNYMuKnPBjiJz1u0Kk=; b=nkXVdLnj1oZVUFS3OmAk1ohE5MldTGBaNj6EU89/7iJyJCbgz3WtlehygMHVrCu8Ti RyMOKZIXGglf6L8GokcjCf2WPU0IkaapJzwvEPnFdDPD+KrXi00P4rNCNjzcbZsWpTVq QIALESre1Te/SXMBDqQsvLfsDOMJASRW2C87g/4/mwqimuePm0A2+j+S2IEnjZhwvWJq c8v+6VPzhBLFeh/fws0fPSGfib8E9eGCEWg5mYJUVkFrEi08zqyLLY6s9SnQW9EfO11E AJwK0MdqkaAHCaD9I+kbv+1kY4pdlQUC0OUNhGzxeP3drAHieIFI9H0fuCHawnUP/A+/ MJHw== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of claudius.heine.ext@siemens.com designates 192.35.17.14 as permitted sender) smtp.mailfrom=claudius.heine.ext@siemens.com Return-Path: Received: from david.siemens.de (david.siemens.de. [192.35.17.14]) by gmr-mx.google.com with ESMTPS id k25si813580edd.1.2018.12.18.05.20.03 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 18 Dec 2018 05:20:03 -0800 (PST) Received-SPF: pass (google.com: domain of claudius.heine.ext@siemens.com designates 192.35.17.14 as permitted sender) client-ip=192.35.17.14; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of claudius.heine.ext@siemens.com designates 192.35.17.14 as permitted sender) smtp.mailfrom=claudius.heine.ext@siemens.com Received: from mail1.sbs.de (mail1.sbs.de [192.129.41.35]) by david.siemens.de (8.15.2/8.15.2) with ESMTPS id wBIDK2o1032043 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 18 Dec 2018 14:20:02 +0100 Received: from [139.25.69.181] (linux-ses-ext02.ppmd.siemens.net [139.25.69.181]) by mail1.sbs.de (8.15.2/8.15.2) with ESMTP id wBIDK2LZ009939; Tue, 18 Dec 2018 14:20:02 +0100 Subject: Re: [PATCH v2 1/2] isar-bootstrap: debootstrap https support To: "Hosgor, Tolga (CT RDA DS EU TR MTS)" , isar-users@googlegroups.com References: <20181217114518.17995-1-tolga.hosgor@siemens.com> <20181217114518.17995-2-tolga.hosgor@siemens.com> From: Claudius Heine Message-ID: Date: Tue, 18 Dec 2018 14:20:02 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.3.0 MIME-Version: 1.0 In-Reply-To: <20181217114518.17995-2-tolga.hosgor@siemens.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-TUID: /PbDT9l1rcZI Hi again, On 17/12/2018 12.45, Hosgor, Tolga (CT RDA DS EU TR MTS) wrote: > Building 'isar-bootstrap-target' fails when there are HTTPS URIs in > the distro APT sources and the first URI is not HTTPS. > The first URI in the APT sources is passed to 'debootstrap' and > the distro APT sources file is written to the isar bootstrap rootfs. > Then, following 'apt-get update' fails due to apt-transport-https, > ca-certificates being missing. > > This patch allows a distro to specify 'DISTRO_BOOTSTRAP_BASE_PACKAGES' > and introduces 'https-support' concept using bitbake OVERRIDES. > An example usage is specifying the necessary packages for http support > via 'DISTRO_BOOTSTRAP_BASE_PACKAGES_append_https-support' in distro > configuration. > > Signed-off-by: Hosgor, Tolga (CT RDA DS EU TR MTS) > --- > meta-isar/conf/distro/debian-buster.list | 1 + > meta-isar/conf/distro/debian-jessie.list | 1 + > .../conf/multiconfig/qemuamd64-buster.conf | 2 ++ > .../conf/multiconfig/qemuamd64-jessie.conf | 2 ++ > .../isar-bootstrap/isar-bootstrap-host.bb | 2 ++ > .../isar-bootstrap/isar-bootstrap-target.bb | 2 ++ > .../isar-bootstrap/isar-bootstrap.inc | 29 +++++++++++++++++-- > 7 files changed, 36 insertions(+), 3 deletions(-) > > diff --git a/meta-isar/conf/distro/debian-buster.list b/meta-isar/conf/distro/debian-buster.list > index 18311d8..92d20e1 100644 > --- a/meta-isar/conf/distro/debian-buster.list > +++ b/meta-isar/conf/distro/debian-buster.list > @@ -1,3 +1,4 @@ > deb http://ftp.de.debian.org/debian buster main contrib non-free > +deb https://debian.inf.tu-dresden.de/debian buster main contrib non-free > deb http://ftp.de.debian.org/debian buster-updates main contrib non-free > deb http://security.debian.org buster/updates main contrib non-free > diff --git a/meta-isar/conf/distro/debian-jessie.list b/meta-isar/conf/distro/debian-jessie.list > index be46a57..2471402 100644 > --- a/meta-isar/conf/distro/debian-jessie.list > +++ b/meta-isar/conf/distro/debian-jessie.list > @@ -1,3 +1,4 @@ > +deb https://debian.inf.tu-dresden.de/debian jessie main contrib non-free > deb http://ftp.de.debian.org/debian jessie main contrib non-free > deb http://ftp.de.debian.org/debian jessie-updates main contrib non-free > deb http://security.debian.org jessie/updates main contrib non-free > diff --git a/meta-isar/conf/multiconfig/qemuamd64-buster.conf b/meta-isar/conf/multiconfig/qemuamd64-buster.conf > index 059ea00..394d76b 100644 > --- a/meta-isar/conf/multiconfig/qemuamd64-buster.conf > +++ b/meta-isar/conf/multiconfig/qemuamd64-buster.conf > @@ -15,3 +15,5 @@ QEMU_ARCH ?= "x86_64" > QEMU_MACHINE ?= "q35" > QEMU_CPU ?= "" > QEMU_DISK_ARGS ?= "-hda ##ROOTFS_IMAGE## -bios /usr/local/share/ovmf/OVMF.fd" > + > +DISTRO_BOOTSTRAP_BASE_PACKAGES_append_https-support = " apt-transport-https ca-certificates" > diff --git a/meta-isar/conf/multiconfig/qemuamd64-jessie.conf b/meta-isar/conf/multiconfig/qemuamd64-jessie.conf > index ca00e15..d1335ff 100644 > --- a/meta-isar/conf/multiconfig/qemuamd64-jessie.conf > +++ b/meta-isar/conf/multiconfig/qemuamd64-jessie.conf > @@ -14,3 +14,5 @@ QEMU_ARCH ?= "x86_64" > QEMU_MACHINE ?= "pc" > QEMU_CPU ?= "" > QEMU_DISK_ARGS ?= "-hda ##ROOTFS_IMAGE##" > + > +DISTRO_BOOTSTRAP_BASE_PACKAGES_append_https-support = " apt-transport-https ca-certificates" > diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb b/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb > index 55696ea..47cff63 100644 > --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb > +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb > @@ -40,6 +40,8 @@ python do_apt_config_prepare() { > } > addtask apt_config_prepare before do_bootstrap after do_unpack > > +OVERRIDES_append = ":${@get_distro_needs_https_support(d, True)}" > + > do_bootstrap[stamp-extra-info] = "${HOST_DISTRO}-${HOST_ARCH}" > do_bootstrap[vardeps] += "HOST_DISTRO_APT_SOURCES" > do_bootstrap() { > diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb b/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb > index 5752b14..57b607f 100644 > --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb > +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb > @@ -39,6 +39,8 @@ python do_apt_config_prepare() { > } > addtask apt_config_prepare before do_bootstrap after do_unpack > > +OVERRIDES_append = ":${@get_distro_needs_https_support(d, False)}" > + > do_bootstrap[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" > do_bootstrap[vardeps] += "DISTRO_APT_SOURCES" > do_bootstrap() { > diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc > index cfad136..d868bb6 100644 > --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc > +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc > @@ -23,6 +23,7 @@ APTKEYFILES = "" > APTKEYRING = "${WORKDIR}/apt-keyring.gpg" > DEBOOTSTRAP_KEYRING = "" > DEPLOY_ISAR_BOOTSTRAP ?= "" > +DISTRO_BOOTSTRAP_BASE_PACKAGES = "locales" > > python () { > from urllib.parse import urlparse > @@ -109,11 +110,15 @@ def aggregate_aptsources_list(d, file_list, file_out): > out_fd.write("\n".encode()) > out_fd.write("\n".encode()) > > -def get_distro_primary_source_entry(d, is_host=False): > +def get_aptsources_list(d, is_host=False): > if is_host: > apt_sources_list = (d.getVar("HOST_DISTRO_APT_SOURCES", True) or "").split() > else: > apt_sources_list = (d.getVar("DISTRO_APT_SOURCES", True) or "").split() > + return apt_sources_list After some thought really don't like this `is_host` parameter... For me that looks like some kind of backwards approach. First you have the difference between target and host, when you use in the different files `isar-bootstrap-target.bb` and `isar-bootstrap-host.bb` and then, because you use common functionality from `isar-bootstrap.inc` you need to put this difference into a boolean and pass that to all of those functions. IMO the right solutions would have been, to do it like this: isar-bootstrap.inc: OVERRIDES_append = ":${@get_distro_needs_https_support(d)}" def get_aptsources_list(d): return (d.getVar("BOOTSTRAP_DISTRO_APT_SOURCES", True) or "").split() isar-bootstrap-target.bb: BOOTSTRAP_DISTRO_APT_SOURCE = "${DISTRO_APT_SOURCES}" isar-bootstrap-host.bb: BOOTSTRAP_DISTRO_APT_SOURCE = "${HOST_DISTRO_APT_SOURCES}" Of course that is not in scope of your patch, but your patch demonstrates that weakness nicely by splitting up that function. Claudius -- DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de