From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6652594697292218368 X-Received: by 2002:a1c:a655:: with SMTP id p82mr326729wme.15.1549288366784; Mon, 04 Feb 2019 05:52:46 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:6000:1152:: with SMTP id d18ls614290wrx.8.gmail; Mon, 04 Feb 2019 05:52:46 -0800 (PST) X-Google-Smtp-Source: AHgI3Ia3iw+YwmKkhiI7cS3VN1axm+cBOb0Ryyb2zp+npMg9TDhqyNv+DCv2JcCts7v/ydGnQWXR X-Received: by 2002:adf:d14e:: with SMTP id b14mr1893073wri.16.1549288366345; Mon, 04 Feb 2019 05:52:46 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1549288366; cv=none; d=google.com; s=arc-20160816; b=1Lg8ofMrkP/wFfcxi4Ma65bb2x7okLy5btBZZLtVYnC9iuaHbGacalUEXCo9kgqpOE RO3vJTyNxdnOF6h+COjtKvD5V7Nyuxxuw7uKsCmn7W9DQzIK4uN9Mb4R+YmZnYQMPOWg Pp6kVxmJD0zzbwcpxtgbzbCJUcqMbRP/UTfaOtgWmkvz5rl8y9AVViGVuatQJzNJrVhj RsUk2Jg90zdGO81Wh6p+S3N0iYfyDp6xzl+Hg0lyplpvclsr/vYH9plvhPbnk0ExV3uh kEbIhyEmSdvRetjeFjFddFksCq9H4HJDzwJK2w3APbJ2CJF3P/XOC83Bd4x0c43R2ejh e/dQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=LiAIXFQOB0DRibZuSTiwRiDK1yntrZsif9SjfMflcOw=; b=n/1cxHh2mM89W04NHudLO2vUpr6M6/XaT9ltsE+AnV2Ruc26jA8BhBPT5vBTZX/JbO aKJU10Qa9vPPHAD3HcA0/DOI2miAop/Z5dP7sslzlJ4XtZ7otSPiXw8+lnfeht9G+pbt 83h7avqUwwp0jVAAzEtlhoidfZdU32aobOgQ9qt+0w5h48H+DfgfkjbvRWKfTmegfUsP PBGdus577ooO2ubHa9EwRYtQZc2zDZMVW31LvHTUvYnYZzourfkPA3rHJFmE9Zrjiaj7 cggaz6CVq3aVfC2rnN1KCRyQvuac+UTA0j5ie0YURuhs76KB0hdjqHR6JFk/yWFmzInz agHQ== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of claudius.heine.ext@siemens.com designates 194.138.37.39 as permitted sender) smtp.mailfrom=claudius.heine.ext@siemens.com Return-Path: Received: from lizzard.sbs.de (lizzard.sbs.de. [194.138.37.39]) by gmr-mx.google.com with ESMTPS id g2si337095wrp.1.2019.02.04.05.52.46 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 04 Feb 2019 05:52:46 -0800 (PST) Received-SPF: pass (google.com: domain of claudius.heine.ext@siemens.com designates 194.138.37.39 as permitted sender) client-ip=194.138.37.39; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of claudius.heine.ext@siemens.com designates 194.138.37.39 as permitted sender) smtp.mailfrom=claudius.heine.ext@siemens.com Received: from mail2.sbs.de (mail2.sbs.de [192.129.41.66]) by lizzard.sbs.de (8.15.2/8.15.2) with ESMTPS id x14DqjqD009326 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 4 Feb 2019 14:52:45 +0100 Received: from ring.ppmd.siemens.net (linux-ses-ext02.ppmd.siemens.net [139.25.69.181]) by mail2.sbs.de (8.15.2/8.15.2) with ESMTP id x14DqiMg028713; Mon, 4 Feb 2019 14:52:45 +0100 From: claudius.heine.ext@siemens.com To: isar-users@googlegroups.com Cc: Claudius Heine Subject: [PATCH 1/1] meta: added do_transform_template task as templating system and switch Date: Mon, 4 Feb 2019 14:52:41 +0100 Message-Id: <20190204135241.32000-2-claudius.heine.ext@siemens.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190204135241.32000-1-claudius.heine.ext@siemens.com> References: <20190204135241.32000-1-claudius.heine.ext@siemens.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-TUID: TTibdlHBKpg0 From: Claudius Heine The new introduced variables are TEMPLATE_FILES and TEMPLATE_VARS. TEMPLATE_FILES is a list of files that should be processed in the `do_transform_template` task and TEMPLATE_VARS is a list of bitbake variable names that should be substituted in the template file. Signed-off-by: Claudius Heine --- doc/technical_overview.md | 25 ++++++++ meta/classes/base.bbclass | 1 + meta/classes/dpkg-base.bbclass | 2 +- meta/classes/template.bbclass | 62 +++++++++++++++++++ .../debian/{changelog => changelog.tmpl} | 2 +- meta/recipes-bsp/u-boot/files/debian/control | 19 ------ .../u-boot/files/debian/control.tmpl | 19 ++++++ meta/recipes-bsp/u-boot/u-boot-custom.inc | 12 ++-- .../debian/{changelog => changelog.tmpl} | 2 +- .../linux-module/files/debian/control | 11 ---- .../linux-module/files/debian/control.tmpl | 11 ++++ meta/recipes-kernel/linux-module/module.inc | 7 +-- 12 files changed, 129 insertions(+), 44 deletions(-) create mode 100644 meta/classes/template.bbclass rename meta/recipes-bsp/u-boot/files/debian/{changelog => changelog.tmpl} (74%) delete mode 100644 meta/recipes-bsp/u-boot/files/debian/control create mode 100644 meta/recipes-bsp/u-boot/files/debian/control.tmpl rename meta/recipes-kernel/linux-module/files/debian/{changelog => changelog.tmpl} (74%) delete mode 100644 meta/recipes-kernel/linux-module/files/debian/control create mode 100644 meta/recipes-kernel/linux-module/files/debian/control.tmpl diff --git a/doc/technical_overview.md b/doc/technical_overview.md index 7efbeee..2608727 100644 --- a/doc/technical_overview.md +++ b/doc/technical_overview.md @@ -283,3 +283,28 @@ cross-compilation, this is done by the same tools as for native compilation. Depending on ISAR_CROSS_COMPILE value, additional architecture specifiers are passed to build tools automatically, so this is absolutely transparent from the user point of view. + +## 3.9 Additional features + +### 3.9.1 Template files + +A basic templating system implemented on top `envsubst(1)` is available. It +allows limited access to bitbake variables for file generation. + +This system is implemented in the `template.bbclass` and defines the +`do_transform_template` task, the `TEMPLATE_FILES` and `TEMPLATE_VARS` +variables. Tasks that use need to use files generated via templates need +to be executed after the `do_transform_template` task. + +The `TEMPLATE_FILES` variable contains a space seperated list of template +files and the `TEMPLATE_VARS` variable a space seperated list of bitbake +variable names. The `do_transform_template` task takes these and generates +files using `envsubst(1)`. The output files are placed in the same directory +as the template files. The name of the output files are eighter the name of the +template file with `.tmpl` removed from the file name end or, if the template +file did not end with `.tmpl`, the name of the template file with `.out` added +to the end. + +Only template files from the `WORKDIR` are accepted. Eighter specify relative +paths based on the recipes `WORKDIR` or absolute paths containing the `WORKDIR` +in the `TEMPLATE_FILES` variable. diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass index d4082de..5ab6fa4 100644 --- a/meta/classes/base.bbclass +++ b/meta/classes/base.bbclass @@ -25,6 +25,7 @@ die() { } inherit logging +inherit template # Derived from bitbake: bitbake/classes/base.bbclass addtask showdata diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass index f1b127c..fe96850 100644 --- a/meta/classes/dpkg-base.bbclass +++ b/meta/classes/dpkg-base.bbclass @@ -40,7 +40,7 @@ do_prepare_build() { true } -addtask prepare_build after do_patch before do_build +addtask prepare_build after do_patch do_transform_template before do_build do_prepare_build[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" # If Isar recipes depend on each other, they typically need the package # deployed to isar-apt diff --git a/meta/classes/template.bbclass b/meta/classes/template.bbclass new file mode 100644 index 0000000..716b7a5 --- /dev/null +++ b/meta/classes/template.bbclass @@ -0,0 +1,62 @@ +# This software is a part of ISAR. +# Copyright (C) Siemens AG, 2019 +# +# SPDX-License-Identifier: MIT + +TEMPLATE_FILES ??= "" +TEMPLATE_VARS ??= "PN PV DESCRIPTION HOMEPAGE MAINTAINER KERNEL_NAME MACHINE \ + DISTRO_ARCH" + +do_transform_template[vardeps] = "TEMPLATE_FILES ${TEMPLATE_VARS}" +do_transform_template[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" +python do_transform_template() { + import subprocess, contextlib + + workdir = os.path.normpath(d.getVar('WORKDIR', True)) + + template_vars = (d.getVar('TEMPLATE_VARS', True) or "").split() + if len(template_vars) == 0: + return + + template_files = (d.getVar('TEMPLATE_FILES', True) or "").split() + if len(template_files) == 0: + return + + cmd = "envsubst" + args = " ".join("\${{{}}}".format(i) for i in template_vars) + + # Copy current process environment and add template variables + # from bitbake data store: + env = os.environ.copy() + for varname in template_vars: + value = d.getVar(varname, True) + if value: + env.update({varname: value}) + + for template_file in template_files: + # Normpath and workdir checks should prevent accidential or + # uninformed changes to files outside of the tmp and workdirectoy + template_file = os.path.normpath(template_file) + + # Convert relative paths to absolut paths based on the workdir: + if not os.path.isabs(template_file): + template_file = os.path.normpath(os.path.join(workdir, template_file)) + + if not template_file.startswith(workdir): + bb.fatal("Template file ({}) is not within workdir ({})" + .format(template_file, workdir)) + + output_file = (os.path.splitext(template_file)[0] + if template_file.endswith(".tmpl") + else (template_file + ".out")) + bb.note("{} {} [in: {} out: {}]".format(cmd, args, + template_file, output_file)) + with contextlib.ExitStack() as stack: + input = stack.enter_context(open(template_file, 'rb')) + output = stack.enter_context(open(output_file, 'wb')) + process = subprocess.Popen([cmd, args], stdin=input, + stdout=output, env=env) + if process.wait() != 0: + bb.fatal("processing of template failed") +} +addtask do_transform_template after do_unpack before do_build diff --git a/meta/recipes-bsp/u-boot/files/debian/changelog b/meta/recipes-bsp/u-boot/files/debian/changelog.tmpl similarity index 74% rename from meta/recipes-bsp/u-boot/files/debian/changelog rename to meta/recipes-bsp/u-boot/files/debian/changelog.tmpl index c1c3516..6e59e06 100644 --- a/meta/recipes-bsp/u-boot/files/debian/changelog +++ b/meta/recipes-bsp/u-boot/files/debian/changelog.tmpl @@ -1,4 +1,4 @@ -@PN@ (@PV@) unstable; urgency=low +${PN} (${PV}) unstable; urgency=low * Generated package. diff --git a/meta/recipes-bsp/u-boot/files/debian/control b/meta/recipes-bsp/u-boot/files/debian/control deleted file mode 100644 index 6b4c839..0000000 --- a/meta/recipes-bsp/u-boot/files/debian/control +++ /dev/null @@ -1,19 +0,0 @@ -Source: @PN@ -Section: admin -Priority: optional -Standards-Version: 3.9.6 -Build-Depends: @BUILD_DEPENDS@ -Maintainer: ISAR project - -Package: u-boot-@MACHINE@ -Architecture: @DISTRO_ARCH@ -Description: @DESCRIPTION@, bootloader binaries - -Package: u-boot-@MACHINE@-dev -Architecture: @DISTRO_ARCH@ -Description: @DESCRIPTION@, bootloader libraries - -Package: u-boot-tools -Architecture: linux-any -Depends: ${shlibs:Depends}, ${misc:Depends} -Description: @DESCRIPTION@, companion tools diff --git a/meta/recipes-bsp/u-boot/files/debian/control.tmpl b/meta/recipes-bsp/u-boot/files/debian/control.tmpl new file mode 100644 index 0000000..5c1cc92 --- /dev/null +++ b/meta/recipes-bsp/u-boot/files/debian/control.tmpl @@ -0,0 +1,19 @@ +Source: ${PN} +Section: admin +Priority: optional +Standards-Version: 3.9.6 +Build-Depends: ${BUILD_DEPENDS} +Maintainer: ISAR project + +Package: u-boot-${MACHINE} +Architecture: ${DISTRO_ARCH} +Description: ${DESCRIPTION}, bootloader binaries + +Package: u-boot-${MACHINE}-dev +Architecture: ${DISTRO_ARCH} +Description: ${DESCRIPTION}, bootloader libraries + +Package: u-boot-tools +Architecture: linux-any +Depends: ${shlibs:Depends}, ${misc:Depends} +Description: ${DESCRIPTION}, companion tools diff --git a/meta/recipes-bsp/u-boot/u-boot-custom.inc b/meta/recipes-bsp/u-boot/u-boot-custom.inc index 4b38c88..c6aab43 100644 --- a/meta/recipes-bsp/u-boot/u-boot-custom.inc +++ b/meta/recipes-bsp/u-boot/u-boot-custom.inc @@ -15,16 +15,14 @@ inherit dpkg SRC_URI += "file://debian/" -U_BOOT_BUILD_DEPENDS ?= "bc, bison, flex, device-tree-compiler" +BUILD_DEPENDS ?= "bc, bison, flex, device-tree-compiler" + +TEMPLATE_FILES = "debian/changelog.tmpl \ + debian/control.tmpl" +TEMPLATE_VARS += "BUILD_DEPENDS" do_prepare_build() { cp -r ${WORKDIR}/debian ${S}/ - sed -i -e 's/@PN@/${PN}/g' -e 's/@PV@/${PV}/g' \ - -e 's/@BUILD_DEPENDS@/${U_BOOT_BUILD_DEPENDS}/g' \ - -e 's/@MACHINE@/${MACHINE}/g' \ - -e 's/@DISTRO_ARCH@/${DISTRO_ARCH}/g' \ - -e 's/@DESCRIPTION@/${DESCRIPTION}/g' \ - ${S}/debian/changelog ${S}/debian/control echo "${U_BOOT_BIN} /usr/lib/u-boot/${MACHINE}" > \ ${S}/debian/u-boot-${MACHINE}.install diff --git a/meta/recipes-kernel/linux-module/files/debian/changelog b/meta/recipes-kernel/linux-module/files/debian/changelog.tmpl similarity index 74% rename from meta/recipes-kernel/linux-module/files/debian/changelog rename to meta/recipes-kernel/linux-module/files/debian/changelog.tmpl index c1c3516..6e59e06 100644 --- a/meta/recipes-kernel/linux-module/files/debian/changelog +++ b/meta/recipes-kernel/linux-module/files/debian/changelog.tmpl @@ -1,4 +1,4 @@ -@PN@ (@PV@) unstable; urgency=low +${PN} (${PV}) unstable; urgency=low * Generated package. diff --git a/meta/recipes-kernel/linux-module/files/debian/control b/meta/recipes-kernel/linux-module/files/debian/control deleted file mode 100644 index 1ee634c..0000000 --- a/meta/recipes-kernel/linux-module/files/debian/control +++ /dev/null @@ -1,11 +0,0 @@ -Source: @PN@ -Section: kernel -Priority: optional -Standards-Version: 3.9.6 -Build-Depends: linux-headers-@KERNEL_NAME@ -Maintainer: ISAR project - -Package: @PN@ -Architecture: any -Depends: linux-image-@KERNEL_NAME@, kmod -Description: @DESCRIPTION@ diff --git a/meta/recipes-kernel/linux-module/files/debian/control.tmpl b/meta/recipes-kernel/linux-module/files/debian/control.tmpl new file mode 100644 index 0000000..3b3292d --- /dev/null +++ b/meta/recipes-kernel/linux-module/files/debian/control.tmpl @@ -0,0 +1,11 @@ +Source: ${PN} +Section: kernel +Priority: optional +Standards-Version: 3.9.6 +Build-Depends: linux-headers-${KERNEL_NAME} +Maintainer: ISAR project + +Package: ${PN} +Architecture: any +Depends: linux-image-${KERNEL_NAME}, kmod +Description: ${DESCRIPTION} diff --git a/meta/recipes-kernel/linux-module/module.inc b/meta/recipes-kernel/linux-module/module.inc index cb7b8ad..44edd58 100644 --- a/meta/recipes-kernel/linux-module/module.inc +++ b/meta/recipes-kernel/linux-module/module.inc @@ -19,12 +19,11 @@ AUTOLOAD ?= "" inherit dpkg +TEMPLATE_FILES = "debian/control.tmpl \ + debian/changelog.tmpl" + do_prepare_build() { cp -r ${WORKDIR}/debian ${S}/ - sed -i -e 's/@PN@/${PN}/g' -e 's/@PV@/${PV}/g' \ - -e 's/@KERNEL_NAME@/${KERNEL_NAME}/g' \ - -e 's/@DESCRIPTION@/${DESCRIPTION}/g' \ - ${S}/debian/changelog ${S}/debian/control for module in "${AUTOLOAD}"; do echo "echo $module >> /etc/modules" >> ${S}/debian/postinst -- 2.20.1