From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6631131420328722432 X-Received: by 2002:a24:248d:: with SMTP id f135mr2422150ita.36.1544548105624; Tue, 11 Dec 2018 09:08:25 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a02:5f48:: with SMTP id r69ls4844543jab.2.gmail; Tue, 11 Dec 2018 09:08:25 -0800 (PST) X-Google-Smtp-Source: AFSGD/VY01fw51HNvBHkxAeaPF4rQCa4cCNJTHx+ZV/VRekj9+Lp6mt3CnmZZVNvRBa1J66lZO/R X-Received: by 2002:a02:77c2:: with SMTP id g185mr13904270jac.9.1544548105210; Tue, 11 Dec 2018 09:08:25 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1544548105; cv=none; d=google.com; s=arc-20160816; b=wcPmRnGKCXzF+8W6OfCfVZe71mthRfjQzdJw3G4GfVgDnec+dyBlbBECCw1L4NR8jg etwigTQnEF5ZmvxafzPWhzrePItpDOvlXCwSN7LPznrlDG2ctziV97ewGAqrJ5MQt7Wg Sv1/YYe4VKoqMgp+u9HPzjvyjqMGg1oJlGIgwUBgRKQF+XjLfAwmVet07YnwJ1IjbamR JXAZojExZMOm1wWgE7HmkiTVzqrdm0fMnfny4hf2UB8ZQp1hfPhKqH9FNF0MYnDgNpqh v/3eKfXUMOlDWUge0nM7MUGKQORqmT4ygAmr1ow8RkGADCdd+be7gDFLTLFdS8prYC1A BLNw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:dkim-signature; bh=NDqQUGSF9IVlsxMhD3y+PNBGJ06pxVBg1gpkot0Mv8Q=; b=qgDuaCLWExQ1Cc3se8s0BbWWwANbMTMybrR4P2U4A8EqiJz39aorpjaiNewGX86teT rD/eKtr/V524GZBSHzpAXYknoFZgYTLCgO9yy46Isz+MOOzcoox0F457cxrG73eGtpxB ++fls4VQpfNIS/u8FCFEkLuAWubii7HW2ZRfPQQoN46bYKvPWeLH7l2zu+q8uld2HDYr FRQnb1SjZu2c3cDMHX3S/RpQ1OiaVb+/UMXtItg+Svs6/bXCFgBs/vfyRWKRwUzjcXoF Flb4OHFZCAnmyDHCV+YApTtoNi7Iydf1iJkp0jpmGBBuemjH+YCFq3MWLzGi/AwIyi9t aL2w== ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=WZLsN3M2; spf=pass (google.com: domain of fasdfasdas@gmail.com designates 2607:f8b0:4864:20::135 as permitted sender) smtp.mailfrom=fasdfasdas@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from mail-it1-x135.google.com (mail-it1-x135.google.com. [2607:f8b0:4864:20::135]) by gmr-mx.google.com with ESMTPS id l71si49201itb.1.2018.12.11.09.08.25 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 11 Dec 2018 09:08:25 -0800 (PST) Received-SPF: pass (google.com: domain of fasdfasdas@gmail.com designates 2607:f8b0:4864:20::135 as permitted sender) client-ip=2607:f8b0:4864:20::135; Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=WZLsN3M2; spf=pass (google.com: domain of fasdfasdas@gmail.com designates 2607:f8b0:4864:20::135 as permitted sender) smtp.mailfrom=fasdfasdas@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: by mail-it1-x135.google.com with SMTP id a6so4986409itl.4 for ; Tue, 11 Dec 2018 09:08:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=NDqQUGSF9IVlsxMhD3y+PNBGJ06pxVBg1gpkot0Mv8Q=; b=WZLsN3M26+4EbQjm+avce75RvR6I26p6wBYEu8+/PrsDVd8PeccO+0vVKMJZr1GU/5 xfeKr6P9wRx8gkOMWaupYRyAdGYrh6blBohR7V4BfQ06+qGEotNKgugUuMfu/194ZRnq iWU/qzgHarjZ0dq/xfpv71iNzQWv7b081CGqfXNBuoK6vcflK3SR8OnnSHZbKgQqlp7t 2Nnd75uTUaaFPYIDDLTS4IgdqKBrdtfYYSFplu0x8eL6HAg40xeE3J3hPbhnA+koPtf6 QZwImNf80tWhNZenubGgmrbrI5GcIks5Zzf9IcXa7Q65kz7zntWaU2v8Qxu8xOoLuNxF to7w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=NDqQUGSF9IVlsxMhD3y+PNBGJ06pxVBg1gpkot0Mv8Q=; b=r6RWmMoKackw7Dxv81lFS0Wcu4wj2q4YiOMPm5BQzYlLUYuHQr6XiBkvNVCGO3pcFk fAK/wHAHNP+0f7jv6dPOUizaGyUnHjIyg4qvEigkf9VdcF1vPXJKunxJf4jd8S7GCS8G X4jOabdyr5xhdkyo+rB28CGVPrtZ7OVJVV7yiqVSu1eRl9mcmPfwpfxCQIGNylhCrS5N Ywqs/hRN0/sz/p/YSG9ee/YyXwMquPr1Sj43vtfi+pLXwaFK8PxwAvJBWrIrGgsLiWrb SaDmrHlo4KNVhoCdCGA/toimxSFOLNaAuUIIRbpJZ10mFi9E57l/PoVmu/sQqUI0itTa xjdQ== X-Gm-Message-State: AA+aEWbUJ1EmxAkpBRmR4581FzNxivF0O4itQVxRa7vuS9MRG1S78Md7 M16Xtkrztv59FQiB9MXjHdhXKZ9Dt75ykvbK5L36oJ5G X-Received: by 2002:a24:6e88:: with SMTP id w130mr2762280itc.103.1544548104890; Tue, 11 Dec 2018 09:08:24 -0800 (PST) MIME-Version: 1.0 References: <20181204133544.50621-1-tolga.hosgor@siemens.com> <20181211081751.073ed373@md1za8fc.ad001.siemens.net> In-Reply-To: <20181211081751.073ed373@md1za8fc.ad001.siemens.net> From: =?UTF-8?B?VG9sZ2EgSG/Fn2fDtnI=?= Date: Tue, 11 Dec 2018 18:08:14 +0100 Message-ID: Subject: Re: [PATCH] isar-bootstrap: debootstrap --include flag to support https:// sources To: Henning Schild Cc: =?UTF-8?B?VG9sZ2EgSG/Fn2fDtnI=?= , isar-users@googlegroups.com Content-Type: multipart/alternative; boundary="000000000000216c71057cc221f7" X-TUID: yR7Se1VJL7nN --000000000000216c71057cc221f7 Content-Type: text/plain; charset="UTF-8" Hi, I would like to when I have the time. :) I am convinced to the tidyness of the "https-support" with overrides idea. On Tue, Dec 11, 2018, 08:20 Henning Schild Hi, > > will you send a v2? > > Henning > > Am Tue, 4 Dec 2018 16:35:44 +0300 > schrieb "Hosgor, Tolga (CT RDA DS EU TR MTS)" : > > > Building 'isar-bootstrap-target' fails when there are HTTPS URIs in > > the distro APT sources and the first URI is not HTTPS. > > The first URI in the APT sources is passed to 'debootstrap' and > > the distro APT sources file is written to the isar bootstrap rootfs. > > Then, following 'apt-get update' fails due to apt-transport-https, > > ca-certificates being missing. In a normal workflow, 'debootstrap' > > automatically includes them if the MIRROR is HTTPS. > > > > Signed-off-by: Hosgor, Tolga (CT RDA DS EU TR MTS) > > --- > > .../isar-bootstrap/isar-bootstrap.inc | 28 > > +++++++++++++++++-- 1 file changed, 25 insertions(+), 3 deletions(-) > > > > diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc > > b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc index > > cfad136..33ad8e8 100644 --- > > a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc +++ > > b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc @@ -109,11 > > +109,15 @@ def aggregate_aptsources_list(d, file_list, file_out): > > out_fd.write("\n".encode()) out_fd.write("\n".encode()) > > > > -def get_distro_primary_source_entry(d, is_host=False): > > +def get_aptsources_list(d, is_host=False): > > if is_host: > > apt_sources_list = (d.getVar("HOST_DISTRO_APT_SOURCES", > > True) or "").split() else: > > apt_sources_list = (d.getVar("DISTRO_APT_SOURCES", True) or > > "").split() > > + return apt_sources_list > > + > > +def get_distro_primary_source_entry(d, is_host=False): > > + apt_sources_list = get_aptsources_list(d, is_host) > > for entry in apt_sources_list: > > entry_real = bb.parse.resolve_file(entry, d) > > with open(entry_real, "r") as in_fd: > > @@ -125,6 +129,18 @@ def get_distro_primary_source_entry(d, > > is_host=False): return parsed[2:] > > return ["", "", ""] > > > > +def get_distro_have_https_source(d, is_host=False): > > + for entry in get_aptsources_list(d, is_host): > > + entry_real = bb.parse.resolve_file(entry, d) > > + with open(entry_real, "r") as in_fd: > > + for line in in_fd: > > + parsed = parse_aptsources_list_line(line) > > + if parsed: > > + parsed = get_apt_source_mirror(d, parsed) > > + if parsed[2].startswith("https://"): > > + return True > > + return False > > + > > def get_distro_source(d, is_host): > > return get_distro_primary_source_entry(d, is_host)[0] > > > > @@ -138,6 +154,12 @@ def get_distro_components_argument(d, is_host): > > else: > > return "" > > > > +def get_debootstrap_includes(d, is_host=False): > > + if get_distro_have_https_source(d, is_host): > > + return "locales,apt-transport-https,ca-certificates" > > + else: > > + return "locales" > > + > > do_generate_keyring[dirs] = "${DL_DIR}" > > do_generate_keyring[vardeps] += "DISTRO_APT_KEYS" > > do_generate_keyring() { > > @@ -183,7 +205,7 @@ isar_bootstrap() { > > if [ ${IS_HOST} ]; then > > ${DEBOOTSTRAP} --verbose \ > > --variant=minbase \ > > - --include=locales \ > > + > > --include="${@get_debootstrap_includes(d, True)}" \ > > ${@get_distro_components_argument(d, True)} \ ${DEBOOTSTRAP_KEYRING} \ > > "${@get_distro_suite(d, True)}" \ > > @@ -194,7 +216,7 @@ isar_bootstrap() { > > "${DEBOOTSTRAP}" --verbose \ > > --variant=minbase \ > > --arch="${DISTRO_ARCH}" \ > > - --include=locales \ > > + > > --include="${@get_debootstrap_includes(d, False)}" \ > > ${@get_distro_components_argument(d, False)} \ ${DEBOOTSTRAP_KEYRING} > > \ "${@get_distro_suite(d, False)}" \ > > --000000000000216c71057cc221f7 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi,

I w= ould like to when I have the time. :)

I am convinced to the tidyness of the "https-support&quo= t; with overrides idea.


On Tue, Dec 11, 2018, 08:20 Henning Schild <hen= ning.schild@siemens.com wrote:
= Hi,

will you send a v2?

Henning

Am Tue, 4 Dec 2018 16:35:44 +0300
schrieb "Hosgor, Tolga (CT RDA DS EU TR MTS)" <tlghosgor@gmail.com>:

> Building 'isar-bootstrap-target' fails when there are HTTPS UR= Is in
> the distro APT sources and the first URI is not HTTPS.
> The first URI in the APT sources is passed to 'debootstrap' an= d
> the distro APT sources file is written to the isar bootstrap rootfs. > Then, following 'apt-get update' fails due to apt-transport-ht= tps,
> ca-certificates being missing. In a normal workflow, 'debootstrap&= #39;
> automatically includes them if the MIRROR is HTTPS.
>
> Signed-off-by: Hosgor, Tolga (CT RDA DS EU TR MTS)
> <tolga.hosgor@siemens.com> --- >=C2=A0 .../isar-bootstrap/isar-bootstrap.inc=C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0| 28
> +++++++++++++++++-- 1 file changed, 25 insertions(+), 3 deletions(-) >
> diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
> b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc index
> cfad136..33ad8e8 100644 ---
> a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc +++
> b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc @@ -109,11
> +109,15 @@ def aggregate_aptsources_list(d, file_list, file_out):
> out_fd.write("\n".encode()) out_fd.write("\n".enco= de())
>=C2=A0
> -def get_distro_primary_source_entry(d, is_host=3DFalse):
> +def get_aptsources_list(d, is_host=3DFalse):
>=C2=A0 =C2=A0 =C2=A0 if is_host:
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 apt_sources_list =3D (d.getVar("= ;HOST_DISTRO_APT_SOURCES",
> True) or "").split() else:
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 apt_sources_list =3D (d.getVar("= ;DISTRO_APT_SOURCES", True) or
> "").split()
> +=C2=A0 =C2=A0 return apt_sources_list
> +
> +def get_distro_primary_source_entry(d, is_host=3DFalse):
> +=C2=A0 =C2=A0 apt_sources_list =3D get_aptsources_list(d, is_host) >=C2=A0 =C2=A0 =C2=A0 for entry in apt_sources_list:
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 entry_real =3D bb.parse.resolve_file= (entry, d)
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 with open(entry_real, "r")= as in_fd:
> @@ -125,6 +129,18 @@ def get_distro_primary_source_entry(d,
> is_host=3DFalse): return parsed[2:]
>=C2=A0 =C2=A0 =C2=A0 return ["", "", ""]<= br> >=C2=A0
> +def get_distro_have_https_source(d, is_host=3DFalse):
> +=C2=A0 =C2=A0 for entry in get_aptsources_list(d, is_host):
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 entry_real =3D bb.parse.resolve_file(entr= y, d)
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 with open(entry_real, "r") as i= n_fd:
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 for line in in_fd:
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 parsed =3D pa= rse_aptsources_list_line(line)
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 if parsed: > +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= parsed =3D get_apt_source_mirror(d, parsed)
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= if parsed[2].startswith("https://"):
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 return True
> +=C2=A0 =C2=A0 return False
> +
>=C2=A0 def get_distro_source(d, is_host):
>=C2=A0 =C2=A0 =C2=A0 return get_distro_primary_source_entry(d, is_host)= [0]
>=C2=A0
> @@ -138,6 +154,12 @@ def get_distro_components_argument(d, is_host): >=C2=A0 =C2=A0 =C2=A0 else:
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 return ""
>=C2=A0
> +def get_debootstrap_includes(d, is_host=3DFalse):
> +=C2=A0 =C2=A0 if get_distro_have_https_source(d, is_host):
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 return "locales,apt-transport-https,= ca-certificates"
> +=C2=A0 =C2=A0 else:
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 return "locales"
> +
>=C2=A0 do_generate_keyring[dirs] =3D "${DL_DIR}"
>=C2=A0 do_generate_keyring[vardeps] +=3D "DISTRO_APT_KEYS" >=C2=A0 do_generate_keyring() {
> @@ -183,7 +205,7 @@ isar_bootstrap() {
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 if [ ${IS_HOST} ]; the= n
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 ${DEBOOT= STRAP} --verbose \
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0--variant=3Dminbase \
> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0--include=3Dlocales \
> +
> --include=3D"${@get_debootstrap_includes(d, True)}" \
> ${@get_distro_components_argument(d, True)} \ ${DEBOOTSTRAP_KEYRING} \=
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0"${@get_distro_suite(d= , True)}" \
> @@ -194,7 +216,7 @@ isar_bootstrap() {
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0&q= uot;${DEBOOTSTRAP}" --verbose \
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 --variant=3Dminbase= \
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 --arch=3D"${DI= STRO_ARCH}" \
> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 --include=3Dlocales \
> +
> --include=3D"${@get_debootstrap_includes(d, False)}" \
> ${@get_distro_components_argument(d, False)} \ ${DEBOOTSTRAP_KEYRING}<= br> > \ "${@get_distro_suite(d, False)}" \

--000000000000216c71057cc221f7--