From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6631131420328722432 X-Received: by 2002:a17:906:3582:: with SMTP id o2-v6mr3338892ejb.13.1543932704078; Tue, 04 Dec 2018 06:11:44 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a17:906:1304:: with SMTP id w4-v6ls3594400ejb.10.gmail; Tue, 04 Dec 2018 06:11:43 -0800 (PST) X-Google-Smtp-Source: AFSGD/X6rTTd8KzRURAW7XO2dpgwM7743o5pqLStK1qFG7PVUvFXfTt3yk5M1E9BVAN8Dzw/w/ad X-Received: by 2002:a17:906:b7d9:: with SMTP id fy25-v6mr3362946ejb.2.1543932703500; Tue, 04 Dec 2018 06:11:43 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1543932703; cv=none; d=google.com; s=arc-20160816; b=MZMgU1QlfPwmDiUTZOXRH1jsxUM6WyxYzRd195gkCeW9uXk5lR13t6gxcvlzW2zsN5 3BpgGS0yRIZ04WxS6rv13i86eEhB6j/5ERdE9P0fRKG3NmEMQIv9jm8cQnZbm4/X/No5 FrEmWn9Ha+7F26ooGqR0FsvYAqY/OLi3muJ1oDZjVvg/nzAfwfNnFRsifNu86s3qkl4w 7w45/QbDtZPaVrqxGygWh7M1pE+wv5prtsFZVclnayyUpt7u2KD07OpQ9hs1SjbkNrCf IXLKA3W/MGlfK87hkIYlM3raP8f5rTlLBpDOVkKHUOjBKpRUNub06gZ37Hika1jaY3Z6 Va6A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:to:subject; bh=NrOti458zj+Rt5KPaHmBNiaewM0RZmuJVnaujeoS4PQ=; b=0y7qR6wUY4NqeN79youz1GzHHCW0bfQ5Cp6Vd0ZhkF/KAhPhx77BAmqSTeYRmsy01u IbOWgXxF0TEs72iPcMYXpCku4JtJtnCOxo8Lh7CTXXjLhX5byikEfQhVPuC0TF0zN2BW n7FDlhZRUcuDr0oK0acRHbKg26zduQpmA69qreKdIaqYE6R+Lm2bIsyj+yVhjGGQYxFO 3FYpnOxCKrvb1DA3JrOyg7jXnM4/gIFK1Mn4JOIfpsNGO8npii2OLNF8qbkp73iWci7e IBzLgnVruqOjoLpdMLHs61gieB8COsf0u0RNYitoyjz8bKqtY1d5nASvWV4qRVKE8vXW 67SQ== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 194.138.37.40 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com Return-Path: Received: from gecko.sbs.de (gecko.sbs.de. [194.138.37.40]) by gmr-mx.google.com with ESMTPS id h28si633212edh.4.2018.12.04.06.11.43 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 04 Dec 2018 06:11:43 -0800 (PST) Received-SPF: pass (google.com: domain of jan.kiszka@siemens.com designates 194.138.37.40 as permitted sender) client-ip=194.138.37.40; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 194.138.37.40 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com Received: from mail2.sbs.de (mail2.sbs.de [192.129.41.66]) by gecko.sbs.de (8.15.2/8.15.2) with ESMTPS id wB4EBgG4001489 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 4 Dec 2018 15:11:42 +0100 Received: from [167.87.41.97] ([167.87.41.97]) by mail2.sbs.de (8.15.2/8.15.2) with ESMTP id wB4EBea4012315; Tue, 4 Dec 2018 15:11:42 +0100 Subject: Re: [PATCH] isar-bootstrap: debootstrap --include flag to support https:// sources To: "Hosgor, Tolga (CT RDA DS EU TR MTS)" , isar-users@googlegroups.com References: <20181204133544.50621-1-tolga.hosgor@siemens.com> From: Jan Kiszka Message-ID: Date: Tue, 4 Dec 2018 15:11:40 +0100 User-Agent: Mozilla/5.0 (X11; U; Linux i686 (x86_64); de; rv:1.8.1.12) Gecko/20080226 SUSE/2.0.0.12-1.1 Thunderbird/2.0.0.12 Mnenhy/0.7.5.666 MIME-Version: 1.0 In-Reply-To: <20181204133544.50621-1-tolga.hosgor@siemens.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-TUID: HWhU23+C06xm You should also add a From: ... line here when you are using a different transport email account. On 04.12.18 14:35, Hosgor, Tolga (CT RDA DS EU TR MTS) wrote: > Building 'isar-bootstrap-target' fails when there are HTTPS URIs in > the distro APT sources and the first URI is not HTTPS. > The first URI in the APT sources is passed to 'debootstrap' and > the distro APT sources file is written to the isar bootstrap rootfs. > Then, following 'apt-get update' fails due to apt-transport-https, > ca-certificates being missing. In a normal workflow, 'debootstrap' > automatically includes them if the MIRROR is HTTPS. > > Signed-off-by: Hosgor, Tolga (CT RDA DS EU TR MTS) > --- > .../isar-bootstrap/isar-bootstrap.inc | 28 +++++++++++++++++-- > 1 file changed, 25 insertions(+), 3 deletions(-) > > diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc > index cfad136..33ad8e8 100644 > --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc > +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc > @@ -109,11 +109,15 @@ def aggregate_aptsources_list(d, file_list, file_out): > out_fd.write("\n".encode()) > out_fd.write("\n".encode()) > > -def get_distro_primary_source_entry(d, is_host=False): > +def get_aptsources_list(d, is_host=False): > if is_host: > apt_sources_list = (d.getVar("HOST_DISTRO_APT_SOURCES", True) or "").split() > else: > apt_sources_list = (d.getVar("DISTRO_APT_SOURCES", True) or "").split() > + return apt_sources_list > + > +def get_distro_primary_source_entry(d, is_host=False): > + apt_sources_list = get_aptsources_list(d, is_host) > for entry in apt_sources_list: > entry_real = bb.parse.resolve_file(entry, d) > with open(entry_real, "r") as in_fd: > @@ -125,6 +129,18 @@ def get_distro_primary_source_entry(d, is_host=False): > return parsed[2:] > return ["", "", ""] > > +def get_distro_have_https_source(d, is_host=False): > + for entry in get_aptsources_list(d, is_host): > + entry_real = bb.parse.resolve_file(entry, d) > + with open(entry_real, "r") as in_fd: > + for line in in_fd: > + parsed = parse_aptsources_list_line(line) > + if parsed: > + parsed = get_apt_source_mirror(d, parsed) Minor, but I would use different variable names here to make things more readable. > + if parsed[2].startswith("https://"): > + return True > + return False > + > def get_distro_source(d, is_host): > return get_distro_primary_source_entry(d, is_host)[0] > > @@ -138,6 +154,12 @@ def get_distro_components_argument(d, is_host): > else: > return "" > > +def get_debootstrap_includes(d, is_host=False): > + if get_distro_have_https_source(d, is_host): > + return "locales,apt-transport-https,ca-certificates" > + else: > + return "locales" > + > do_generate_keyring[dirs] = "${DL_DIR}" > do_generate_keyring[vardeps] += "DISTRO_APT_KEYS" > do_generate_keyring() { > @@ -183,7 +205,7 @@ isar_bootstrap() { > if [ ${IS_HOST} ]; then > ${DEBOOTSTRAP} --verbose \ > --variant=minbase \ > - --include=locales \ > + --include="${@get_debootstrap_includes(d, True)}" \ > ${@get_distro_components_argument(d, True)} \ > ${DEBOOTSTRAP_KEYRING} \ > "${@get_distro_suite(d, True)}" \ > @@ -194,7 +216,7 @@ isar_bootstrap() { > "${DEBOOTSTRAP}" --verbose \ > --variant=minbase \ > --arch="${DISTRO_ARCH}" \ > - --include=locales \ > + --include="${@get_debootstrap_includes(d, False)}" \ > ${@get_distro_components_argument(d, False)} \ > ${DEBOOTSTRAP_KEYRING} \ > "${@get_distro_suite(d, False)}" \ > Jan -- Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux