From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6631131420328722432 X-Received: by 2002:a2e:9053:: with SMTP id n19-v6mr1888465ljg.12.1543933390448; Tue, 04 Dec 2018 06:23:10 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a19:fc1e:: with SMTP id a30ls1626527lfi.12.gmail; Tue, 04 Dec 2018 06:23:09 -0800 (PST) X-Google-Smtp-Source: AFSGD/VVrpALg+AcqzK2eW2jhkLDUIPxFzY0jby8WHh64JCMlIguGxVb0bEUblxnZ3b5KY9d9X0D X-Received: by 2002:a19:a245:: with SMTP id l66mr1607928lfe.0.1543933389904; Tue, 04 Dec 2018 06:23:09 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1543933389; cv=none; d=google.com; s=arc-20160816; b=J2QbjO4wD2K77SWcZqcDDM+T5EE6yOYJ0w1DUfkCsX2Khmr5m05UdxqKb0ZiLX7In7 +w+rkRpRvaCNxV2Zojq95FZxlZFLhPEA/Y/yC67tvimrS2zoDztAPkLphcbjiNij4kI+ o/JrG0A1le6VvwMUnAWoYVsYF8G14FBLnde7BxdTLgfzSptzvRI+woRkxVc+wRjeCESc zepBDN/tYXUHISHJql8m9K50urbXhJNr+UpJadYw6T3GAnJ12NOfk0qW3wMK+NN43H3e OKpNmoJwA8/ijPZ3okG0xm+3C0K0Jr5QY+5FQI8uiiNgoUhK2y3+QA+TIWpgP/I0TLtQ s/og== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:to:subject; bh=Y/vauGP8N4EhMg+/d3GlXMrpxOjh/PchOH4KeT16F8Y=; b=j9oRwU8QDZDqQp5G19bxoNli4WOWNhyhv+d5F6UYhumhh4a/PjyE1wxg2UEF2dEqWH bOkOoUDZxweB7j/7NLdxaLAchrItk+FYik56ffhweICSLweMbDVG3mmUAwzkm2n7v38q SaEg5X8Hk9/XtZzL2ckatu7VwR36ltgY9RFRt60gA1JiOKTRSJIveI7WnDqOem0BP5NE M8/ZSnUIsL1u0GnNUlwu1aHLontTi24v96BfPtGXVf0iambVNw2EIcIqqqJzvz/+WOQ5 8gDQDM5yJi90mZ31/y4pwfpG5OMOSeo45rjIrrCrVJfSzz17UaDfQQjZ4LtNeDAizGZS eEkA== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of claudius.heine.ext@siemens.com designates 192.35.17.14 as permitted sender) smtp.mailfrom=claudius.heine.ext@siemens.com Return-Path: Received: from david.siemens.de (david.siemens.de. [192.35.17.14]) by gmr-mx.google.com with ESMTPS id y18si698601lfe.3.2018.12.04.06.23.09 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 04 Dec 2018 06:23:09 -0800 (PST) Received-SPF: pass (google.com: domain of claudius.heine.ext@siemens.com designates 192.35.17.14 as permitted sender) client-ip=192.35.17.14; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of claudius.heine.ext@siemens.com designates 192.35.17.14 as permitted sender) smtp.mailfrom=claudius.heine.ext@siemens.com Received: from mail1.sbs.de (mail1.sbs.de [192.129.41.35]) by david.siemens.de (8.15.2/8.15.2) with ESMTPS id wB4EN8XU012691 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 4 Dec 2018 15:23:08 +0100 Received: from [139.25.69.181] (linux-ses-ext02.ppmd.siemens.net [139.25.69.181]) by mail1.sbs.de (8.15.2/8.15.2) with ESMTP id wB4EN8Lj000975; Tue, 4 Dec 2018 15:23:08 +0100 Subject: Re: [PATCH] isar-bootstrap: debootstrap --include flag to support https:// sources To: "Hosgor, Tolga (CT RDA DS EU TR MTS)" , isar-users@googlegroups.com References: <20181204133544.50621-1-tolga.hosgor@siemens.com> From: Claudius Heine Message-ID: Date: Tue, 4 Dec 2018 15:23:08 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.3.0 MIME-Version: 1.0 In-Reply-To: <20181204133544.50621-1-tolga.hosgor@siemens.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-TUID: 4mTPJuzRwFxz Hi, On 04/12/2018 14.35, Hosgor, Tolga (CT RDA DS EU TR MTS) wrote: > Building 'isar-bootstrap-target' fails when there are HTTPS URIs in > the distro APT sources and the first URI is not HTTPS. > The first URI in the APT sources is passed to 'debootstrap' and > the distro APT sources file is written to the isar bootstrap rootfs. > Then, following 'apt-get update' fails due to apt-transport-https, > ca-certificates being missing. In a normal workflow, 'debootstrap' > automatically includes them if the MIRROR is HTTPS. > > Signed-off-by: Hosgor, Tolga (CT RDA DS EU TR MTS) > --- > .../isar-bootstrap/isar-bootstrap.inc | 28 +++++++++++++++++-- > 1 file changed, 25 insertions(+), 3 deletions(-) > > diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc > index cfad136..33ad8e8 100644 > --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc > +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc > @@ -109,11 +109,15 @@ def aggregate_aptsources_list(d, file_list, file_out): > out_fd.write("\n".encode()) > out_fd.write("\n".encode()) > > -def get_distro_primary_source_entry(d, is_host=False): > +def get_aptsources_list(d, is_host=False): > if is_host: > apt_sources_list = (d.getVar("HOST_DISTRO_APT_SOURCES", True) or "").split() > else: > apt_sources_list = (d.getVar("DISTRO_APT_SOURCES", True) or "").split() > + return apt_sources_list > + > +def get_distro_primary_source_entry(d, is_host=False): > + apt_sources_list = get_aptsources_list(d, is_host) > for entry in apt_sources_list: > entry_real = bb.parse.resolve_file(entry, d) > with open(entry_real, "r") as in_fd: > @@ -125,6 +129,18 @@ def get_distro_primary_source_entry(d, is_host=False): > return parsed[2:] > return ["", "", ""] > > +def get_distro_have_https_source(d, is_host=False): > + for entry in get_aptsources_list(d, is_host): > + entry_real = bb.parse.resolve_file(entry, d) > + with open(entry_real, "r") as in_fd: > + for line in in_fd: > + parsed = parse_aptsources_list_line(line) > + if parsed: > + parsed = get_apt_source_mirror(d, parsed) > + if parsed[2].startswith("https://"): > + return True > + return False Maybe you can use the any [1] statement with a generator, to simplify the code here. [1] https://docs.python.org/3/library/functions.html#any > + > def get_distro_source(d, is_host): > return get_distro_primary_source_entry(d, is_host)[0] > > @@ -138,6 +154,12 @@ def get_distro_components_argument(d, is_host): > else: > return "" > > +def get_debootstrap_includes(d, is_host=False): > + if get_distro_have_https_source(d, is_host): > + return "locales,apt-transport-https,ca-certificates" Those package names should be configured in the distro config. Claudius > + else: > + return "locales" > + > do_generate_keyring[dirs] = "${DL_DIR}" > do_generate_keyring[vardeps] += "DISTRO_APT_KEYS" > do_generate_keyring() { > @@ -183,7 +205,7 @@ isar_bootstrap() { > if [ ${IS_HOST} ]; then > ${DEBOOTSTRAP} --verbose \ > --variant=minbase \ > - --include=locales \ > + --include="${@get_debootstrap_includes(d, True)}" \ > ${@get_distro_components_argument(d, True)} \ > ${DEBOOTSTRAP_KEYRING} \ > "${@get_distro_suite(d, True)}" \ > @@ -194,7 +216,7 @@ isar_bootstrap() { > "${DEBOOTSTRAP}" --verbose \ > --variant=minbase \ > --arch="${DISTRO_ARCH}" \ > - --include=locales \ > + --include="${@get_debootstrap_includes(d, False)}" \ > ${@get_distro_components_argument(d, False)} \ > ${DEBOOTSTRAP_KEYRING} \ > "${@get_distro_suite(d, False)}" \ > -- DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de