From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6631131420328722432 X-Received: by 2002:a1c:e308:: with SMTP id a8mr2143488wmh.14.1543930596368; Tue, 04 Dec 2018 05:36:36 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a1c:80c4:: with SMTP id b187ls3127656wmd.0.canary-gmail; Tue, 04 Dec 2018 05:36:35 -0800 (PST) X-Google-Smtp-Source: AFSGD/WcS/KVicZUIlRTjHnNVKtdsPfn32N3GhFNZxUW16X4v+iw6KHua5VzNgLvYwgeC8p2LCwc X-Received: by 2002:a1c:8083:: with SMTP id b125mr2282572wmd.11.1543930595977; Tue, 04 Dec 2018 05:36:35 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1543930595; cv=none; d=google.com; s=arc-20160816; b=U91TmKjr0iTpskUrbEoDhbLpZ+eqsaEw5TXvfl9cMXQKiBKIRxFGinQ2zyCffqRvkp K9nzrdZTijMpKJweH3FeawyYBFjYvpRpFWg+5vVNWlsjhvbBfHGu/gGo0np9HpThIhBw jcjZRjYtEjHXEwvQrc3j8DeV+r+gtSNXOKFqgghWg4gAcOaqIdnmSNB2ncIrNA4W81KC 1mwFp4QEum7lAhnhCRk17YvJ8rTZuuByP7IgrdZyzRQrKR9LH5YWEZUTAXBGnVs1t4Wf VEVQRe/oOdl/3GOm8Ca5oO9zfmsA2PNZbAOVuj+ziy9pjUbL7krXGGPuz0KuXvl8TFok sO0A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:dkim-signature; bh=+Gsr5Y1Cw8QJrqcqp1v3nMcjRaYsDKFjQG4qI1c2IBY=; b=Ac1ScPPhKtBVbfNsYF8wW28x8+9E27Pe4uZtnDf0l2IjGWaD0DMVqDBmfEkcU4BaLd 4xBRz8qy04nzrpTd2hJi/CxZcaMCqs0xE4GYL9UjAVnOj2lH9cwBCWd2vi4cVvPDLxBK wiaXO2V980lwX58CEANKUdcBFsy0sqdhZY9I61Mt25jvz3Ez+l+7leEGOabwXqaVjI5f Lnx8nf0P0tPZpmD+4AaVecXrhWE3OfzmTkkY/eO6+6lm1Ezhws/AZjWsf51ra7uneaP7 XvKDZl/7qOjNSp+UDm4zVh/rFNdDFjKNvx+NAW33tJa4ezW8BD/nso5ciUfazOq7QlMu DogQ== ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=BBDZUByN; spf=pass (google.com: domain of tlghosgor@gmail.com designates 2a00:1450:4864:20::444 as permitted sender) smtp.mailfrom=tlghosgor@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from mail-wr1-x444.google.com (mail-wr1-x444.google.com. [2a00:1450:4864:20::444]) by gmr-mx.google.com with ESMTPS id k140si380779wmd.2.2018.12.04.05.36.35 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 04 Dec 2018 05:36:35 -0800 (PST) Received-SPF: pass (google.com: domain of tlghosgor@gmail.com designates 2a00:1450:4864:20::444 as permitted sender) client-ip=2a00:1450:4864:20::444; Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=BBDZUByN; spf=pass (google.com: domain of tlghosgor@gmail.com designates 2a00:1450:4864:20::444 as permitted sender) smtp.mailfrom=tlghosgor@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: by mail-wr1-x444.google.com with SMTP id x10so15979143wrs.8 for ; Tue, 04 Dec 2018 05:36:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:mime-version :content-transfer-encoding; bh=+Gsr5Y1Cw8QJrqcqp1v3nMcjRaYsDKFjQG4qI1c2IBY=; b=BBDZUByN1UyWhML1bk2mw4OqYvrh1hqkRrjKBvvaahFkhAtlpaF77RR/LcmXclSQf6 12CgG636NDnqaZdN9RFELkb7uLPuahzqM1uoHivzvVw6F6ly7XVFNiWbCP76ML1YvngE naVDfdqNn3EDTyrdRRgZ9gkLWNNyi6lltDNdSGG8nNJcWD1GtmVelR4Vhwv9n1Ix8Svq FmKv4hWn2TlFB5plzUhjSGqVCrdz5UDauODcLuwPz3/Eu4l/kB6WvdI4NIfZ8doqE7xr yQO3fIObNtYHZHvXE8RT434ElvYYZlFRqu3kfOpeQhcRMJyjyUYuxlqWfFI1Vz6hIWva NdDg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:mime-version :content-transfer-encoding; bh=+Gsr5Y1Cw8QJrqcqp1v3nMcjRaYsDKFjQG4qI1c2IBY=; b=KAQNEE9Vmv27+AVdePExDZJVUKtX//yPcMRjWJ11B+3t/V+5sESqPRovarVNTuukm1 X2f7WLY+pJiJU0qfsQ4zZwMqRDqm7VHk/0xcApvN2A3iX9FPTgb2AKf6NXK+HgVIz5js nhMiwVN9SShr9get2/40aMwX4Z/1s1RvhgQLzQNx4t6r7rdIxTQDVbeeRqHjv9HM2UDI LaCnekc2Ju8+XtLCSueQNalfq66yj6UiHaPQt15EB/ot0p4riI1eTXqsW+1x8N9ujiy/ 5cjEX/41MECImaWa3pm6XQDSsuJCSRAm2HJnH96GwJL6+vIIU/RDGStN4SrKrw38HPnc Abhg== X-Gm-Message-State: AA+aEWbFP+xu9UbR0m+AhFhLa4cMIp6DD0bby8ycLehGG4nbCk0a3pnh fPJB3WQ6wwUo8KDWS4yAq5ANT9/WdZ+6kzUQ X-Received: by 2002:adf:ee89:: with SMTP id b9mr18930436wro.246.1543930595352; Tue, 04 Dec 2018 05:36:35 -0800 (PST) Return-Path: Received: from localhost.localdomain ([212.133.129.9]) by smtp.gmail.com with ESMTPSA id o8sm13289181wrx.15.2018.12.04.05.36.34 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 04 Dec 2018 05:36:34 -0800 (PST) From: "Hosgor, Tolga (CT RDA DS EU TR MTS)" X-Google-Original-From: "Hosgor, Tolga (CT RDA DS EU TR MTS)" To: isar-users@googlegroups.com Subject: [PATCH] isar-bootstrap: debootstrap --include flag to support https:// sources Date: Tue, 4 Dec 2018 16:35:44 +0300 Message-Id: <20181204133544.50621-1-tolga.hosgor@siemens.com> X-Mailer: git-send-email 2.19.2 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-TUID: IYhTKpy6dwz7 Building 'isar-bootstrap-target' fails when there are HTTPS URIs in the distro APT sources and the first URI is not HTTPS. The first URI in the APT sources is passed to 'debootstrap' and the distro APT sources file is written to the isar bootstrap rootfs. Then, following 'apt-get update' fails due to apt-transport-https, ca-certificates being missing. In a normal workflow, 'debootstrap' automatically includes them if the MIRROR is HTTPS. Signed-off-by: Hosgor, Tolga (CT RDA DS EU TR MTS) --- .../isar-bootstrap/isar-bootstrap.inc | 28 +++++++++++++++++-- 1 file changed, 25 insertions(+), 3 deletions(-) diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc index cfad136..33ad8e8 100644 --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc @@ -109,11 +109,15 @@ def aggregate_aptsources_list(d, file_list, file_out): out_fd.write("\n".encode()) out_fd.write("\n".encode()) -def get_distro_primary_source_entry(d, is_host=False): +def get_aptsources_list(d, is_host=False): if is_host: apt_sources_list = (d.getVar("HOST_DISTRO_APT_SOURCES", True) or "").split() else: apt_sources_list = (d.getVar("DISTRO_APT_SOURCES", True) or "").split() + return apt_sources_list + +def get_distro_primary_source_entry(d, is_host=False): + apt_sources_list = get_aptsources_list(d, is_host) for entry in apt_sources_list: entry_real = bb.parse.resolve_file(entry, d) with open(entry_real, "r") as in_fd: @@ -125,6 +129,18 @@ def get_distro_primary_source_entry(d, is_host=False): return parsed[2:] return ["", "", ""] +def get_distro_have_https_source(d, is_host=False): + for entry in get_aptsources_list(d, is_host): + entry_real = bb.parse.resolve_file(entry, d) + with open(entry_real, "r") as in_fd: + for line in in_fd: + parsed = parse_aptsources_list_line(line) + if parsed: + parsed = get_apt_source_mirror(d, parsed) + if parsed[2].startswith("https://"): + return True + return False + def get_distro_source(d, is_host): return get_distro_primary_source_entry(d, is_host)[0] @@ -138,6 +154,12 @@ def get_distro_components_argument(d, is_host): else: return "" +def get_debootstrap_includes(d, is_host=False): + if get_distro_have_https_source(d, is_host): + return "locales,apt-transport-https,ca-certificates" + else: + return "locales" + do_generate_keyring[dirs] = "${DL_DIR}" do_generate_keyring[vardeps] += "DISTRO_APT_KEYS" do_generate_keyring() { @@ -183,7 +205,7 @@ isar_bootstrap() { if [ ${IS_HOST} ]; then ${DEBOOTSTRAP} --verbose \ --variant=minbase \ - --include=locales \ + --include="${@get_debootstrap_includes(d, True)}" \ ${@get_distro_components_argument(d, True)} \ ${DEBOOTSTRAP_KEYRING} \ "${@get_distro_suite(d, True)}" \ @@ -194,7 +216,7 @@ isar_bootstrap() { "${DEBOOTSTRAP}" --verbose \ --variant=minbase \ --arch="${DISTRO_ARCH}" \ - --include=locales \ + --include="${@get_debootstrap_includes(d, False)}" \ ${@get_distro_components_argument(d, False)} \ ${DEBOOTSTRAP_KEYRING} \ "${@get_distro_suite(d, False)}" \ -- 2.19.2