From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6631131420328722432 X-Received: by 2002:a1c:860f:: with SMTP id i15mr1657411wmd.24.1544029964473; Wed, 05 Dec 2018 09:12:44 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:adf:8274:: with SMTP id 107ls6442241wrb.10.gmail; Wed, 05 Dec 2018 09:12:44 -0800 (PST) X-Google-Smtp-Source: AFSGD/VhYSfqHqVqfHmRKG2jEJrouXBuVODrR396mrqWk/K7JsKVInQ0zYuwyduUrZWOy/Eel36e X-Received: by 2002:a5d:6543:: with SMTP id z3mr2843968wrv.7.1544029964063; Wed, 05 Dec 2018 09:12:44 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1544029964; cv=none; d=google.com; s=arc-20160816; b=DOlOCFldOlEKPmIZMKcRdUf5nhH8y7p4zVJ55CTq8jtEyH3gsVzoiahGRg74KQWEFg d1opW+dRnS3lvSrOpfjnEh7breu8eO0GK8fxHjcjeej2xyyFNv6ahKgedWZMSyKtlEho WAg2YtmYVQhIov4n0fNjAQHn11/DfFpgqAAJGK4CXasQQP8FJzRN8Tjq2uOPk2424Wng pYF5//e0GedUT/zvdlPuPkXOp67uYu0nDEDAoBiPXMFuXAYmh1S8wygEizjerhc61ofh +n6CpEOrGe7QkiqwRs3CyVEnd3goXUca+F1Bxn+mrzdy61Wb5MtZTdRFgfCl+kXhC2Mk skng== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date; bh=elp94FUxTkAJKZpZJbdbN+xxCPD0KAaFsS7nBcnLWyk=; b=Lm77QXoLLL5ZnEz4gRDcA8n0wvLcHyRertKzkjlUNuAY/417R+dQMeXG1Am6sKa/Xq p7j5QniwjjE4c00a0ZYt7rSIAcbA949MsZxf6BvxZo443wVoXO8+8vsR/tKaw8dhYX3i y26TuR6Qk3Pw9QPVPpByPbL6qL+nNAb6juyDq/kyDI7yUCSEi/ElRYfvA/y3gZkMrtRw OOr6VU76b8qM+m83vaRiEY5DnWU1Qs8VpZuo2MO+S9ZbyWZR+EuRBVvDNbmFUuNjlYJf /yJvvTTChiMAuECiLaY3cJYEIvKdgjiHASmrTPx7cqDPWQbIIyIfqq+8nALFH0NR1e+9 nkNw== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 194.138.37.39 as permitted sender) smtp.mailfrom=henning.schild@siemens.com Return-Path: Received: from lizzard.sbs.de (lizzard.sbs.de. [194.138.37.39]) by gmr-mx.google.com with ESMTPS id p15si662226wrm.5.2018.12.05.09.12.43 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 05 Dec 2018 09:12:44 -0800 (PST) Received-SPF: pass (google.com: domain of henning.schild@siemens.com designates 194.138.37.39 as permitted sender) client-ip=194.138.37.39; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 194.138.37.39 as permitted sender) smtp.mailfrom=henning.schild@siemens.com Received: from mail1.sbs.de (mail1.sbs.de [192.129.41.35]) by lizzard.sbs.de (8.15.2/8.15.2) with ESMTPS id wB5HChG2031210 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 5 Dec 2018 18:12:43 +0100 Received: from md1za8fc.ad001.siemens.net ([139.25.69.6]) by mail1.sbs.de (8.15.2/8.15.2) with ESMTP id wB5HChGu007650; Wed, 5 Dec 2018 18:12:43 +0100 Date: Wed, 5 Dec 2018 18:12:42 +0100 From: Henning Schild To: "Hosgor, Tolga (CT RDA DS EU TR MTS)" Cc: Subject: Re: [PATCH] isar-bootstrap: debootstrap --include flag to support https:// sources Message-ID: <20181205181242.37c507fa@md1za8fc.ad001.siemens.net> In-Reply-To: <20181204133544.50621-1-tolga.hosgor@siemens.com> References: <20181204133544.50621-1-tolga.hosgor@siemens.com> X-Mailer: Claws Mail 3.15.0-dirty (GTK+ 2.24.32; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-TUID: 3Ts2vVuiJQYE Very good idea and thanks for looking into that. Please make sure to add a CI test. Would be simple as changing one of our .list files to have http in the first line and at least one https in another. And maybe comment on the weird mix inside the file. I would suggest the buster one. Henning Am Tue, 4 Dec 2018 16:35:44 +0300 schrieb "Hosgor, Tolga (CT RDA DS EU TR MTS)" : > Building 'isar-bootstrap-target' fails when there are HTTPS URIs in > the distro APT sources and the first URI is not HTTPS. > The first URI in the APT sources is passed to 'debootstrap' and > the distro APT sources file is written to the isar bootstrap rootfs. > Then, following 'apt-get update' fails due to apt-transport-https, > ca-certificates being missing. In a normal workflow, 'debootstrap' > automatically includes them if the MIRROR is HTTPS. > > Signed-off-by: Hosgor, Tolga (CT RDA DS EU TR MTS) > --- > .../isar-bootstrap/isar-bootstrap.inc | 28 > +++++++++++++++++-- 1 file changed, 25 insertions(+), 3 deletions(-) > > diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc > b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc index > cfad136..33ad8e8 100644 --- > a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc +++ > b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc @@ -109,11 > +109,15 @@ def aggregate_aptsources_list(d, file_list, file_out): > out_fd.write("\n".encode()) out_fd.write("\n".encode()) > > -def get_distro_primary_source_entry(d, is_host=False): > +def get_aptsources_list(d, is_host=False): > if is_host: > apt_sources_list = (d.getVar("HOST_DISTRO_APT_SOURCES", > True) or "").split() else: > apt_sources_list = (d.getVar("DISTRO_APT_SOURCES", True) or > "").split() > + return apt_sources_list > + > +def get_distro_primary_source_entry(d, is_host=False): > + apt_sources_list = get_aptsources_list(d, is_host) > for entry in apt_sources_list: > entry_real = bb.parse.resolve_file(entry, d) > with open(entry_real, "r") as in_fd: > @@ -125,6 +129,18 @@ def get_distro_primary_source_entry(d, > is_host=False): return parsed[2:] > return ["", "", ""] > > +def get_distro_have_https_source(d, is_host=False): > + for entry in get_aptsources_list(d, is_host): > + entry_real = bb.parse.resolve_file(entry, d) > + with open(entry_real, "r") as in_fd: > + for line in in_fd: > + parsed = parse_aptsources_list_line(line) > + if parsed: > + parsed = get_apt_source_mirror(d, parsed) > + if parsed[2].startswith("https://"): > + return True > + return False > + > def get_distro_source(d, is_host): > return get_distro_primary_source_entry(d, is_host)[0] > > @@ -138,6 +154,12 @@ def get_distro_components_argument(d, is_host): > else: > return "" > > +def get_debootstrap_includes(d, is_host=False): > + if get_distro_have_https_source(d, is_host): > + return "locales,apt-transport-https,ca-certificates" > + else: > + return "locales" > + > do_generate_keyring[dirs] = "${DL_DIR}" > do_generate_keyring[vardeps] += "DISTRO_APT_KEYS" > do_generate_keyring() { > @@ -183,7 +205,7 @@ isar_bootstrap() { > if [ ${IS_HOST} ]; then > ${DEBOOTSTRAP} --verbose \ > --variant=minbase \ > - --include=locales \ > + > --include="${@get_debootstrap_includes(d, True)}" \ > ${@get_distro_components_argument(d, True)} \ ${DEBOOTSTRAP_KEYRING} \ > "${@get_distro_suite(d, True)}" \ > @@ -194,7 +216,7 @@ isar_bootstrap() { > "${DEBOOTSTRAP}" --verbose \ > --variant=minbase \ > --arch="${DISTRO_ARCH}" \ > - --include=locales \ > + > --include="${@get_debootstrap_includes(d, False)}" \ > ${@get_distro_components_argument(d, False)} \ ${DEBOOTSTRAP_KEYRING} > \ "${@get_distro_suite(d, False)}" \