From: "Hosgor, Tolga (CT RDA DS EU TR MTS)" <tlghosgor@gmail.com>
To: isar-users@googlegroups.com
Subject: [PATCH v2 1/2] isar-bootstrap: debootstrap https support
Date: Mon, 17 Dec 2018 14:45:17 +0300 [thread overview]
Message-ID: <20181217114518.17995-2-tolga.hosgor@siemens.com> (raw)
In-Reply-To: <20181217114518.17995-1-tolga.hosgor@siemens.com>
Building 'isar-bootstrap-target' fails when there are HTTPS URIs in
the distro APT sources and the first URI is not HTTPS.
The first URI in the APT sources is passed to 'debootstrap' and
the distro APT sources file is written to the isar bootstrap rootfs.
Then, following 'apt-get update' fails due to apt-transport-https,
ca-certificates being missing.
This patch allows a distro to specify 'DISTRO_BOOTSTRAP_BASE_PACKAGES'
and introduces 'https-support' concept using bitbake OVERRIDES.
An example usage is specifying the necessary packages for http support
via 'DISTRO_BOOTSTRAP_BASE_PACKAGES_append_https-support' in distro
configuration.
Signed-off-by: Hosgor, Tolga (CT RDA DS EU TR MTS) <tolga.hosgor@siemens.com>
---
meta-isar/conf/distro/debian-buster.list | 1 +
meta-isar/conf/distro/debian-jessie.list | 1 +
.../conf/multiconfig/qemuamd64-buster.conf | 2 ++
.../conf/multiconfig/qemuamd64-jessie.conf | 2 ++
.../isar-bootstrap/isar-bootstrap-host.bb | 2 ++
.../isar-bootstrap/isar-bootstrap-target.bb | 2 ++
.../isar-bootstrap/isar-bootstrap.inc | 29 +++++++++++++++++--
7 files changed, 36 insertions(+), 3 deletions(-)
diff --git a/meta-isar/conf/distro/debian-buster.list b/meta-isar/conf/distro/debian-buster.list
index 18311d8..92d20e1 100644
--- a/meta-isar/conf/distro/debian-buster.list
+++ b/meta-isar/conf/distro/debian-buster.list
@@ -1,3 +1,4 @@
deb http://ftp.de.debian.org/debian buster main contrib non-free
+deb https://debian.inf.tu-dresden.de/debian buster main contrib non-free
deb http://ftp.de.debian.org/debian buster-updates main contrib non-free
deb http://security.debian.org buster/updates main contrib non-free
diff --git a/meta-isar/conf/distro/debian-jessie.list b/meta-isar/conf/distro/debian-jessie.list
index be46a57..2471402 100644
--- a/meta-isar/conf/distro/debian-jessie.list
+++ b/meta-isar/conf/distro/debian-jessie.list
@@ -1,3 +1,4 @@
+deb https://debian.inf.tu-dresden.de/debian jessie main contrib non-free
deb http://ftp.de.debian.org/debian jessie main contrib non-free
deb http://ftp.de.debian.org/debian jessie-updates main contrib non-free
deb http://security.debian.org jessie/updates main contrib non-free
diff --git a/meta-isar/conf/multiconfig/qemuamd64-buster.conf b/meta-isar/conf/multiconfig/qemuamd64-buster.conf
index 059ea00..394d76b 100644
--- a/meta-isar/conf/multiconfig/qemuamd64-buster.conf
+++ b/meta-isar/conf/multiconfig/qemuamd64-buster.conf
@@ -15,3 +15,5 @@ QEMU_ARCH ?= "x86_64"
QEMU_MACHINE ?= "q35"
QEMU_CPU ?= ""
QEMU_DISK_ARGS ?= "-hda ##ROOTFS_IMAGE## -bios /usr/local/share/ovmf/OVMF.fd"
+
+DISTRO_BOOTSTRAP_BASE_PACKAGES_append_https-support = " apt-transport-https ca-certificates"
diff --git a/meta-isar/conf/multiconfig/qemuamd64-jessie.conf b/meta-isar/conf/multiconfig/qemuamd64-jessie.conf
index ca00e15..d1335ff 100644
--- a/meta-isar/conf/multiconfig/qemuamd64-jessie.conf
+++ b/meta-isar/conf/multiconfig/qemuamd64-jessie.conf
@@ -14,3 +14,5 @@ QEMU_ARCH ?= "x86_64"
QEMU_MACHINE ?= "pc"
QEMU_CPU ?= ""
QEMU_DISK_ARGS ?= "-hda ##ROOTFS_IMAGE##"
+
+DISTRO_BOOTSTRAP_BASE_PACKAGES_append_https-support = " apt-transport-https ca-certificates"
diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb b/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb
index 55696ea..47cff63 100644
--- a/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb
+++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb
@@ -40,6 +40,8 @@ python do_apt_config_prepare() {
}
addtask apt_config_prepare before do_bootstrap after do_unpack
+OVERRIDES_append = ":${@get_distro_needs_https_support(d, True)}"
+
do_bootstrap[stamp-extra-info] = "${HOST_DISTRO}-${HOST_ARCH}"
do_bootstrap[vardeps] += "HOST_DISTRO_APT_SOURCES"
do_bootstrap() {
diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb b/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb
index 5752b14..57b607f 100644
--- a/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb
+++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb
@@ -39,6 +39,8 @@ python do_apt_config_prepare() {
}
addtask apt_config_prepare before do_bootstrap after do_unpack
+OVERRIDES_append = ":${@get_distro_needs_https_support(d, False)}"
+
do_bootstrap[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
do_bootstrap[vardeps] += "DISTRO_APT_SOURCES"
do_bootstrap() {
diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
index cfad136..d868bb6 100644
--- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
+++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
@@ -23,6 +23,7 @@ APTKEYFILES = ""
APTKEYRING = "${WORKDIR}/apt-keyring.gpg"
DEBOOTSTRAP_KEYRING = ""
DEPLOY_ISAR_BOOTSTRAP ?= ""
+DISTRO_BOOTSTRAP_BASE_PACKAGES = "locales"
python () {
from urllib.parse import urlparse
@@ -109,11 +110,15 @@ def aggregate_aptsources_list(d, file_list, file_out):
out_fd.write("\n".encode())
out_fd.write("\n".encode())
-def get_distro_primary_source_entry(d, is_host=False):
+def get_aptsources_list(d, is_host=False):
if is_host:
apt_sources_list = (d.getVar("HOST_DISTRO_APT_SOURCES", True) or "").split()
else:
apt_sources_list = (d.getVar("DISTRO_APT_SOURCES", True) or "").split()
+ return apt_sources_list
+
+def get_distro_primary_source_entry(d, is_host=False):
+ apt_sources_list = get_aptsources_list(d, is_host)
for entry in apt_sources_list:
entry_real = bb.parse.resolve_file(entry, d)
with open(entry_real, "r") as in_fd:
@@ -125,6 +130,24 @@ def get_distro_primary_source_entry(d, is_host=False):
return parsed[2:]
return ["", "", ""]
+def get_distro_have_https_source(d, is_host=False):
+ for entry in get_aptsources_list(d, is_host):
+ entry_real = bb.parse.resolve_file(entry, d)
+ with open(entry_real, "r") as in_fd:
+ for line in in_fd:
+ parsed = parse_aptsources_list_line(line)
+ if parsed:
+ parsed = get_apt_source_mirror(d, parsed)
+ if parsed[2].startswith("https://"):
+ return True
+ return False
+
+def get_distro_needs_https_support(d, is_host=False):
+ if get_distro_have_https_source(d, is_host):
+ return "https-support"
+ else:
+ return ""
+
def get_distro_source(d, is_host):
return get_distro_primary_source_entry(d, is_host)[0]
@@ -183,7 +206,7 @@ isar_bootstrap() {
if [ ${IS_HOST} ]; then
${DEBOOTSTRAP} --verbose \
--variant=minbase \
- --include=locales \
+ --include='${DISTRO_BOOTSTRAP_BASE_PACKAGES}' \
${@get_distro_components_argument(d, True)} \
${DEBOOTSTRAP_KEYRING} \
"${@get_distro_suite(d, True)}" \
@@ -194,7 +217,7 @@ isar_bootstrap() {
"${DEBOOTSTRAP}" --verbose \
--variant=minbase \
--arch="${DISTRO_ARCH}" \
- --include=locales \
+ --include='${DISTRO_BOOTSTRAP_BASE_PACKAGES}' \
${@get_distro_components_argument(d, False)} \
${DEBOOTSTRAP_KEYRING} \
"${@get_distro_suite(d, False)}" \
--
2.19.2
next prev parent reply other threads:[~2018-12-17 11:45 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-12-17 11:45 [PATCH v2 0/2] " Hosgor, Tolga (CT RDA DS EU TR MTS)
2018-12-17 11:45 ` Hosgor, Tolga (CT RDA DS EU TR MTS) [this message]
2018-12-18 13:20 ` [PATCH v2 1/2] " Claudius Heine
2018-12-19 6:52 ` Tolga Hoşgör
2018-12-17 11:45 ` [PATCH v2 2/2] isar-bootstrap: simplified distro source functions Hosgor, Tolga (CT RDA DS EU TR MTS)
2018-12-17 11:55 ` Claudius Heine
2018-12-17 18:11 ` Tolga Hoşgör
2018-12-18 8:20 ` Claudius Heine
2019-01-17 14:08 ` [PATCH v2 0/2] isar-bootstrap: debootstrap https support Henning Schild
2019-01-17 14:21 ` Claudius Heine
2019-01-23 13:39 ` Maxim Yu. Osipov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20181217114518.17995-2-tolga.hosgor@siemens.com \
--to=tlghosgor@gmail.com \
--cc=isar-users@googlegroups.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox