From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6522397978051739648 X-Received: by 10.46.77.79 with SMTP id a76mr104640ljb.12.1518614119958; Wed, 14 Feb 2018 05:15:19 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 10.25.202.87 with SMTP id h23ls1871394lfj.0.gmail; Wed, 14 Feb 2018 05:15:19 -0800 (PST) X-Google-Smtp-Source: AH8x226+bnssAPhT5OXZdPWom263gxIs6RPcKmI/GN5mgkDyAkhiYk7GbkKpVtNtijngN0q5Y5Em X-Received: by 10.25.208.132 with SMTP id h126mr372592lfg.33.1518614119142; Wed, 14 Feb 2018 05:15:19 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1518614119; cv=none; d=google.com; s=arc-20160816; b=e4QNYP9sOkvPLbqqkQHbzYZqaTVdVWCCVdIHB3beypYZ1daCItMzww9gISRSz0Ty7h kAAG4KfVC/E+WBy2VlnxaqXYoSShQeW5q+p60QhptOk7ff6u9Lqs+Oso6OVpRxpUnFZE vWT03daosBmojvoFp5bZI7VPW5+hz6EjGgCPoGkzJuEIrZ0fu1yTmxJOs4A88Io8Xiqc /jiSR0wcuWMOOwpo2nvdQ8UuW1frHuniDHTgUFnO/Wm5xjlog3hQDMduGoRy/+ThzKV0 72ph1n+xNtCfeoOHFcTJv59J82D3J//Jp96pz9LAAUKFOvv8+Ao3Gv384BGlsZ98bfUg JH6A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :arc-authentication-results; bh=otUQ4/gvtB6XwwEvEGipn954LyKbMY2nKQEjrRfSsHA=; b=O6kJjoaKc0+FRQPFaoQe+VmbLR7kxC4myjXjC/dY1P6psJP1e3snkE1e8D1c8t5p+D nLUCoovxIA9g+d03U8XgbmeScr+YPEvObWN6dqVJJIVBQmY6Lxq7ll3XOf9KHDWRxq0z nWHKhLDHKa1TK2OkwGlYOG5hvZAAVXi2DrXePb3GqMTDWA/TI+0UOTtUN+vS8pXxRwKj yiaJasRFA5YpqJKxE/lYX+lZ3nNJHs3XffRt54ZWd+CwhV64SKFw72ApyLloF/f65sp6 HKGHNUE4Kv/TFwG0NhKb5YwCus4h9s2z/ftdfOZ1uID0u6PdidaUD0re4TlvILSSnVV0 rsKA== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of claudius.heine.ext@siemens.com designates 192.35.17.2 as permitted sender) smtp.mailfrom=claudius.heine.ext@siemens.com Return-Path: Received: from thoth.sbs.de (thoth.sbs.de. [192.35.17.2]) by gmr-mx.google.com with ESMTPS id z10si763234lfj.3.2018.02.14.05.15.18 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 14 Feb 2018 05:15:19 -0800 (PST) Received-SPF: pass (google.com: domain of claudius.heine.ext@siemens.com designates 192.35.17.2 as permitted sender) client-ip=192.35.17.2; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of claudius.heine.ext@siemens.com designates 192.35.17.2 as permitted sender) smtp.mailfrom=claudius.heine.ext@siemens.com Received: from mail2.sbs.de (mail2.sbs.de [192.129.41.66]) by thoth.sbs.de (8.15.2/8.15.2) with ESMTPS id w1EDFHgo000986 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 14 Feb 2018 14:15:18 +0100 Received: from ring.ppmd.siemens.net (linux-ses-ext02.ppmd.siemens.net [139.25.68.223]) by mail2.sbs.de (8.15.2/8.15.2) with ESMTP id w1EDFHOn022563; Wed, 14 Feb 2018 14:15:17 +0100 From: claudius.heine.ext@siemens.com To: isar-users@googlegroups.com Cc: Claudius Heine Subject: [PATCH 2/2] centralize multistrap configuration generation Date: Wed, 14 Feb 2018 14:15:09 +0100 Message-Id: <20180214131509.16361-3-claudius.heine.ext@siemens.com> X-Mailer: git-send-email 2.15.1 In-Reply-To: <20180214131509.16361-1-claudius.heine.ext@siemens.com> References: <20180214131509.16361-1-claudius.heine.ext@siemens.com> X-TUID: UiAHo2sxuMay From: Claudius Heine Previous there was redundant code between the buildchroot and isar-image-base recipe. This patch moves the generation of the multistrap configuration to a central python script that can be parametrized by both recipes. Signed-off-by: Claudius Heine --- .../recipes-core/images/files/multistrap.conf.in | 38 ---------- meta-isar/recipes-core/images/isar-image-base.bb | 33 ++++----- meta/classes/base.bbclass | 2 +- meta/lib/isar/__init__.py | 8 +++ meta/lib/isar/multistrap.py | 80 ++++++++++++++++++++++ meta/recipes-devtools/buildchroot/buildchroot.bb | 28 +++----- .../buildchroot/files/multistrap.conf.in | 37 ---------- 7 files changed, 111 insertions(+), 115 deletions(-) delete mode 100644 meta-isar/recipes-core/images/files/multistrap.conf.in create mode 100644 meta/lib/isar/__init__.py create mode 100644 meta/lib/isar/multistrap.py delete mode 100644 meta/recipes-devtools/buildchroot/files/multistrap.conf.in diff --git a/meta-isar/recipes-core/images/files/multistrap.conf.in b/meta-isar/recipes-core/images/files/multistrap.conf.in deleted file mode 100644 index 432b6af..0000000 --- a/meta-isar/recipes-core/images/files/multistrap.conf.in +++ /dev/null @@ -1,38 +0,0 @@ -# This software is a part of ISAR. -# Copyright (C) 2015-2017 ilbers GmbH - -[General] -noauth=true -unpack=true -ignorenativearch=true -bootstrap=##DISTRO_MULTICONF_BOOTSTRAP## Isar -aptsources=##DISTRO_MULTICONF_APTSOURCES## -configscript=##CONFIG_SCRIPT## -setupscript=##SETUP_SCRIPT## -hookdir=##DIR_HOOKS## - -[base] -source=##DISTRO_APT_SOURCE## -suite=##DISTRO_SUITE## -components=##DISTRO_COMPONENTS## -packages=##IMAGE_PREINSTALL## -omitdebsrc=true - -[updates] -source=##DISTRO_APT_SOURCE## -suite=##DISTRO_SUITE##-updates -components=##DISTRO_COMPONENTS## -omitdebsrc=true - -[security] -source=##DISTRO_APT_SOURCE_SEC## -suite=##DISTRO_SUITE##/updates -components=##DISTRO_COMPONENTS## -omitdebsrc=true - -[Isar] -packages=##IMAGE_INSTALL## -source=##DEPLOY_DIR_APT## -suite=##ISAR_DISTRO_SUITE## -components=main -omitdebsrc=true diff --git a/meta-isar/recipes-core/images/isar-image-base.bb b/meta-isar/recipes-core/images/isar-image-base.bb index 8ddbabb..9913977 100644 --- a/meta-isar/recipes-core/images/isar-image-base.bb +++ b/meta-isar/recipes-core/images/isar-image-base.bb @@ -9,8 +9,7 @@ LICENSE = "gpl-2.0" LIC_FILES_CHKSUM = "file://${LAYERDIR_isar}/licenses/COPYING.GPLv2;md5=751419260aa954499f7abaabaa882bbe" FILESPATH =. "${LAYERDIR_isar}/recipes-core/images/files:" -SRC_URI = "file://multistrap.conf.in \ - file://${DISTRO_CONFIG_SCRIPT} \ +SRC_URI = "file://${DISTRO_CONFIG_SCRIPT} \ file://setup.sh \ file://download_dev-random" @@ -28,6 +27,14 @@ WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}" do_rootfs[dirs] = "${WORKDIR}/hooks_multistrap" +MULTISTRAP_CONFIG_SCRIPT = "${DISTRO_CONFIG_SCRIPT}" +MULTISTRAP_APTSOURCES = "${DISTRO_MULTICONF_APTSOURCES}" +MULTISTRAP_BOOTSTRAP = "${DISTRO_MULTICONF_BOOTSTRAP} isar" +MULTISTRAP_PREINSTALL = "${IMAGE_PREINSTALL}" +MULTISTRAP_ISAR_SOURCE = "copy:///${DEPLOY_DIR_APT}/${DISTRO}" +MULTISTRAP_ISAR_INSTALL = "${IMAGE_INSTALL}" +MULTISTRAP_ISAR_SUITE = "${DEBDISTRONAME}" + do_rootfs() { E="${@ bb.utils.export_proxies(d)}" @@ -35,25 +42,8 @@ do_rootfs() { chmod +x "${WORKDIR}/setup.sh" install -m 755 "${WORKDIR}/download_dev-random" "${WORKDIR}/hooks_multistrap/" - # Multistrap accepts only relative path in configuration files, so get it: - cd ${TOPDIR} - WORKDIR_REL=${@ os.path.relpath(d.getVar("WORKDIR", True))} - - # Adjust multistrap config - sed -e 's|##IMAGE_PREINSTALL##|${IMAGE_PREINSTALL}|g' \ - -e 's|##DISTRO_MULTICONF_BOOTSTRAP##|${DISTRO_MULTICONF_BOOTSTRAP}|g' \ - -e 's|##DISTRO_MULTICONF_APTSOURCES##|${DISTRO_MULTICONF_APTSOURCES}|g' \ - -e 's|##DISTRO_APT_SOURCE##|${DISTRO_APT_SOURCE}|g' \ - -e 's|##DISTRO_APT_SOURCE_SEC##|${DISTRO_APT_SOURCE_SEC}|g' \ - -e 's|##DISTRO_SUITE##|${DISTRO_SUITE}|g' \ - -e 's|##DISTRO_COMPONENTS##|${DISTRO_COMPONENTS}|g' \ - -e 's|##CONFIG_SCRIPT##|./'"$WORKDIR_REL"'/${DISTRO_CONFIG_SCRIPT}|g' \ - -e 's|##SETUP_SCRIPT##|./'"$WORKDIR_REL"'/setup.sh|g' \ - -e 's|##DIR_HOOKS##|./'"$WORKDIR_REL"'/hooks_multistrap|g' \ - -e 's|##IMAGE_INSTALL##|${IMAGE_INSTALL}|g' \ - -e 's|##DEPLOY_DIR_APT##|copy:///${DEPLOY_DIR_APT}/${DISTRO}|g' \ - -e 's|##ISAR_DISTRO_SUITE##|${DEBDISTRONAME}|g' \ - "${WORKDIR}/multistrap.conf.in" > "${WORKDIR}/multistrap.conf" + echo "${@ isar.multistrap.generate_multistrap_config(d) }" > \ + "${WORKDIR}/multistrap.conf" # Do not use bitbake flag [dirs] here because this folder should have # specific ownership. @@ -62,6 +52,7 @@ do_rootfs() { # Create root filesystem. We must use sudo -E here to preserve the environment # because of proxy settings + cd "${TOPDIR}" sudo -E multistrap -a ${DISTRO_ARCH} -d "${IMAGE_ROOTFS}" -f "${WORKDIR}/multistrap.conf" # Configure root filesystem diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass index 3334789..2a45a1e 100644 --- a/meta/classes/base.bbclass +++ b/meta/classes/base.bbclass @@ -38,7 +38,7 @@ bbfatal() { } # Derived from openembedded-core/meta/classes/base.bbclass -ISAR_IMPORTS += "os sys time" +ISAR_IMPORTS += "os sys time isar" def isar_imports(d): import sys diff --git a/meta/lib/isar/__init__.py b/meta/lib/isar/__init__.py new file mode 100644 index 0000000..e6aeeaa --- /dev/null +++ b/meta/lib/isar/__init__.py @@ -0,0 +1,8 @@ +# Custom kernel recipe include +# +# This software is a part of ISAR. +# Copyright (c) Siemens AG, 2018 +# +# SPDX-License-Identifier: MIT + +from . import multistrap diff --git a/meta/lib/isar/multistrap.py b/meta/lib/isar/multistrap.py new file mode 100644 index 0000000..7022f91 --- /dev/null +++ b/meta/lib/isar/multistrap.py @@ -0,0 +1,80 @@ +# Custom kernel recipe include +# +# This software is a part of ISAR. +# Copyright (c) Siemens AG, 2018 +# +# SPDX-License-Identifier: MIT + +def generate_multistrap_config(d): + import configparser as cp + import io + + workdir = d.getVar("WORKDIR", True) or "" + topdir = d.getVar("TOPDIR", True) or "" + workdir_rel = os.path.relpath(workdir, start=topdir) + + msconf = cp.RawConfigParser() + + # Set general multistrap configuration: + configscript = d.getVar("MULTISTRAP_CONFIG_SCRIPT", True) or "" + setupscript = d.getVar("MULTISTRAP_SETUP_SCRIPT", True) or "setup.sh" + hookdir = d.getVar("MULTISTRAP_HOOK_DIR", True) or "hooks_multistrap" + aptsources = (d.getVar("MULTISTRAP_APTSOURCES", True) or "").split() + bootstrap = (d.getVar("MULTISTRAP_BOOTSTRAP", True) or "").split() + + msconf.add_section("General") + msconf.set("General", "noauth", "true") + msconf.set("General", "unpack", "true") + msconf.set("General", "ignorenativearch", "true") + msconf.set("General", "configscript", "./" + + os.path.join(workdir_rel, configscript)) + msconf.set("General", "setupscript", "./" + + os.path.join(workdir_rel, setupscript)) + msconf.set("General", "hookdir", "./" + + os.path.join(workdir_rel, hookdir)) + msconf.set("General", "aptsources", " ".join(aptsources)) + msconf.set("General", "bootstrap", " ".join(bootstrap)) + + # Setup upstream repositories in multistrap configuration: + apt_source = d.getVar("DISTRO_APT_SOURCE", True) or "" + apt_source_sec = d.getVar("DISTRO_APT_SOURCE_SEC", True) or "" + apt_suite = d.getVar("DISTRO_SUITE", True) or "" + apt_components = (d.getVar("DISTRO_COMPONENTS", True) or "").split() + deb_packages = (d.getVar("MULTISTRAP_PREINSTALL", True) or "").split() + isar_source = d.getVar("MULTISTRAP_ISAR_SOURCE", True) or "" + isar_packages = (d.getVar("MULTISTRAP_ISAR_INSTALL", True) or "").split() + isar_suite = d.getVar("MULTISTRAP_ISAR_SUITE", True) or "isar" + + msconf.add_section("base") + msconf.set("base", "source", apt_source) + msconf.set("base", "suite", apt_suite) + msconf.set("base", "components", " ".join(apt_components)) + msconf.set("base", "omitdebsrc", "true") + msconf.set("base", "packages", " ".join(deb_packages)) + + msconf.add_section("updates") + msconf.set("updates", "source", apt_source) + msconf.set("updates", "suite", apt_suite + "-updates") + msconf.set("updates", "components", " ".join(apt_components)) + msconf.set("updates", "omitdebsrc", "true") + + msconf.add_section("security") + msconf.set("security", "source", apt_source_sec) + msconf.set("security", "suite", apt_suite + "/updates") + msconf.set("security", "components", " ".join(apt_components)) + msconf.set("security", "omitdebsrc", "true") + + msconf.add_section("isar") + msconf.set("isar", "source", isar_source) + msconf.set("isar", "suite", isar_suite) + msconf.set("isar", "components", "main") + msconf.set("isar", "omitdebsrc", "true") + if isar_packages : + msconf.set("isar", "packages", + " ".join(isar_packages)) + + ret = "" + with io.StringIO() as out: + msconf.write(out) + ret = out.getvalue() + return ret diff --git a/meta/recipes-devtools/buildchroot/buildchroot.bb b/meta/recipes-devtools/buildchroot/buildchroot.bb index 570c0ad..59b4e7e 100644 --- a/meta/recipes-devtools/buildchroot/buildchroot.bb +++ b/meta/recipes-devtools/buildchroot/buildchroot.bb @@ -9,8 +9,7 @@ LICENSE = "gpl-2.0" LIC_FILES_CHKSUM = "file://${LAYERDIR_isar}/licenses/COPYING.GPLv2;md5=751419260aa954499f7abaabaa882bbe" FILESPATH =. "${LAYERDIR_core}/recipes-devtools/buildchroot/files:" -SRC_URI = "file://multistrap.conf.in \ - file://configscript.sh \ +SRC_URI = "file://configscript.sh \ file://setup.sh \ file://build.sh" PV = "1.0" @@ -37,32 +36,25 @@ do_build[dirs] = "${BUILDCHROOT_DIR}/isar-apt \ ${BUILDCHROOT_DIR}/sys" do_build[depends] = "isar-apt:do_cache_config" +MULTISTRAP_CONFIG_SCRIPT = "configscript.sh" +MULTISTRAP_APTSOURCES = "isar ${DISTRO_MULTICONF_APTSOURCES}" +MULTISTRAP_BOOTSTRAP = "${DISTRO_MULTICONF_BOOTSTRAP}" +MULTISTRAP_PREINSTALL = "${BUILDCHROOT_PREINSTALL}" +MULTISTRAP_ISAR_SOURCE = "file:///isar-apt" + do_build() { E="${@ bb.utils.export_proxies(d)}" chmod +x "${WORKDIR}/setup.sh" chmod +x "${WORKDIR}/configscript.sh" - # Multistrap accepts only relative path in configuration files, so get it: - cd ${TOPDIR} - WORKDIR_REL=${@ os.path.relpath(d.getVar("WORKDIR", True))} - - # Adjust multistrap config - sed -e 's|##BUILDCHROOT_PREINSTALL##|${BUILDCHROOT_PREINSTALL}|g' \ - -e 's|##DISTRO_MULTICONF_BOOTSTRAP##|${DISTRO_MULTICONF_BOOTSTRAP}|g' \ - -e 's|##DISTRO_MULTICONF_APTSOURCES##|${DISTRO_MULTICONF_APTSOURCES}|g' \ - -e 's|##DISTRO_APT_SOURCE##|${DISTRO_APT_SOURCE}|g' \ - -e 's|##DISTRO_APT_SOURCE_SEC##|${DISTRO_APT_SOURCE_SEC}|g' \ - -e 's|##DISTRO_SUITE##|${DISTRO_SUITE}|g' \ - -e 's|##DISTRO_COMPONENTS##|${DISTRO_COMPONENTS}|g' \ - -e 's|##CONFIG_SCRIPT##|./'"$WORKDIR_REL"'/configscript.sh|g' \ - -e 's|##SETUP_SCRIPT##|./'"$WORKDIR_REL"'/setup.sh|g' \ - -e 's|##DIR_HOOKS##|./'"$WORKDIR_REL"'/hooks_multistrap|g' \ - "${WORKDIR}/multistrap.conf.in" > "${WORKDIR}/multistrap.conf" + echo "${@ isar.multistrap.generate_multistrap_config(d) }" > \ + "${WORKDIR}/multistrap.conf" do_setup_mounts # Create root filesystem + cd "${TOPDIR}" sudo -E multistrap -a ${DISTRO_ARCH} -d "${BUILDCHROOT_DIR}" -f "${WORKDIR}/multistrap.conf" # Install package builder script diff --git a/meta/recipes-devtools/buildchroot/files/multistrap.conf.in b/meta/recipes-devtools/buildchroot/files/multistrap.conf.in deleted file mode 100644 index 480a4b8..0000000 --- a/meta/recipes-devtools/buildchroot/files/multistrap.conf.in +++ /dev/null @@ -1,37 +0,0 @@ -# This software is a part of ISAR. -# Copyright (C) 2015-2017 ilbers GmbH - -[General] -noauth=true -unpack=true -ignorenativearch=true -bootstrap=##DISTRO_MULTICONF_BOOTSTRAP## -aptsources=isar-apt ##DISTRO_MULTICONF_APTSOURCES## -configscript=##CONFIG_SCRIPT## -setupscript=##SETUP_SCRIPT## -hookdir=##DIR_HOOKS## - -[base] -source=##DISTRO_APT_SOURCE## -suite=##DISTRO_SUITE## -components=##DISTRO_COMPONENTS## -packages=##BUILDCHROOT_PREINSTALL## -omitdebsrc=true - -[updates] -source=##DISTRO_APT_SOURCE## -suite=##DISTRO_SUITE##-updates -components=##DISTRO_COMPONENTS## -omitdebsrc=true - -[security] -source=##DISTRO_APT_SOURCE_SEC## -suite=##DISTRO_SUITE##/updates -components=##DISTRO_COMPONENTS## -omitdebsrc=true - -[isar-apt] -source=file:///isar-apt -suite=isar -components=main -omitdebsrc=true -- 2.15.1