From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6652594697292218368 X-Received: by 2002:ac2:50d1:: with SMTP id h17mr286929lfm.15.1549365485594; Tue, 05 Feb 2019 03:18:05 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:ac2:50c9:: with SMTP id h9ls1962947lfm.6.gmail; Tue, 05 Feb 2019 03:18:05 -0800 (PST) X-Google-Smtp-Source: AHgI3IZqcJTkpINP8OsNt8qQqysOWaPFfsBkZugrUiteTXN4SpUrPDThlcoI6x9mjrKsxOYx766n X-Received: by 2002:ac2:5396:: with SMTP id g22mr287255lfh.9.1549365485030; Tue, 05 Feb 2019 03:18:05 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1549365485; cv=none; d=google.com; s=arc-20160816; b=ApVTE501ZOQcZauL90eneeHlC3ZtnENRKCK1tOaFpfAAWKcDgHVfNn4HjwxQtxwCsH T8xdY/jP0AJ1waE6SOVrWImF1IvPJg4xWRNlfZ826rigNOJuh/gfFcUd8Bsp69pMEvrB gA+s750loqTkFw94DeYhpFiQnsh3wUbGu4k8iBriePMJ/67DcIqOBMGgpxViAmyPb/sM 7AmhGlg0WOrceVvCUfsjdtwCGTPYNTKfjy73TqsFnmieiIrMfCOpC/sRGBhwyU5FWKTj E0++r7RbZ+coVGSENjw1tBsc8VrQ0ZajD3PQQe3UljaJmr6UD+6q7IChen1hqnOPaUmz 5VNA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=ZG9822axEvj54vrVAieT29glqkrXbzJ9HGspdWOvhwg=; b=sJllVlv+nKQ5TTkZN8PNJF8Jlwmb6hzqoFQBDtZsgdYwf52C/nwAsxczI78ehSCGVe e+Fkrtvht4KDWu6rM0phwKs/ejXIA8Ry6KWDp3Hm+KPHIYIo2zsN/UZAXs5MswlCqMTJ lOJnWOHTM9N77uoUi4N+1vA/GGonUIh5ZHd9hdOD8dJvBbxn/JQlIVQtZcp0z2QLQbY+ i3kHWl/cMI0itRoZ3upmX+b8mk1NNjAs1K2Kjmmo4o9CIhkMAuN97rxY1uMU7pXNrs5p a3BHV9CJ6bgWK9WKiU42rlG0bpbP65tRkB23qxgTb7v1REupb4zMv3YGaf1IJFE3cOzJ fDUg== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of claudius.heine.ext@siemens.com designates 194.138.37.40 as permitted sender) smtp.mailfrom=claudius.heine.ext@siemens.com Return-Path: Received: from gecko.sbs.de (gecko.sbs.de. [194.138.37.40]) by gmr-mx.google.com with ESMTPS id y18si933597lfe.3.2019.02.05.03.18.04 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 05 Feb 2019 03:18:04 -0800 (PST) Received-SPF: pass (google.com: domain of claudius.heine.ext@siemens.com designates 194.138.37.40 as permitted sender) client-ip=194.138.37.40; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of claudius.heine.ext@siemens.com designates 194.138.37.40 as permitted sender) smtp.mailfrom=claudius.heine.ext@siemens.com Received: from mail2.sbs.de (mail2.sbs.de [192.129.41.66]) by gecko.sbs.de (8.15.2/8.15.2) with ESMTPS id x15BI4rW017506 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 5 Feb 2019 12:18:04 +0100 Received: from ring.ppmd.siemens.net (linux-ses-ext02.ppmd.siemens.net [139.25.69.181]) by mail2.sbs.de (8.15.2/8.15.2) with ESMTP id x15BI3pN005612; Tue, 5 Feb 2019 12:18:03 +0100 From: claudius.heine.ext@siemens.com To: isar-users@googlegroups.com Cc: Claudius Heine Subject: [PATCH v2 1/1] meta: added do_transform_template task as templating system and switch Date: Tue, 5 Feb 2019 12:18:01 +0100 Message-Id: <20190205111801.14362-2-claudius.heine.ext@siemens.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190205111801.14362-1-claudius.heine.ext@siemens.com> References: <20190205111801.14362-1-claudius.heine.ext@siemens.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-TUID: 9641IQxlRsvJ From: Claudius Heine The new introduced variables are TEMPLATE_FILES and TEMPLATE_VARS. TEMPLATE_FILES is a list of files that should be processed in the `do_transform_template` task and TEMPLATE_VARS is a list of bitbake variable names that should be substituted in the template file. Signed-off-by: Claudius Heine --- doc/technical_overview.md | 25 ++++++++ meta/classes/base.bbclass | 1 + meta/classes/dpkg-base.bbclass | 2 +- meta/classes/template.bbclass | 62 +++++++++++++++++++ .../debian/{changelog => changelog.tmpl} | 2 +- meta/recipes-bsp/u-boot/files/debian/control | 19 ------ .../u-boot/files/debian/control.tmpl | 19 ++++++ meta/recipes-bsp/u-boot/u-boot-custom.inc | 12 ++-- .../debian/{changelog => changelog.tmpl} | 2 +- .../linux-module/files/debian/control | 11 ---- .../linux-module/files/debian/control.tmpl | 11 ++++ meta/recipes-kernel/linux-module/module.inc | 7 +-- 12 files changed, 129 insertions(+), 44 deletions(-) create mode 100644 meta/classes/template.bbclass rename meta/recipes-bsp/u-boot/files/debian/{changelog => changelog.tmpl} (74%) delete mode 100644 meta/recipes-bsp/u-boot/files/debian/control create mode 100644 meta/recipes-bsp/u-boot/files/debian/control.tmpl rename meta/recipes-kernel/linux-module/files/debian/{changelog => changelog.tmpl} (74%) delete mode 100644 meta/recipes-kernel/linux-module/files/debian/control create mode 100644 meta/recipes-kernel/linux-module/files/debian/control.tmpl diff --git a/doc/technical_overview.md b/doc/technical_overview.md index 7efbeee..0ddcabc 100644 --- a/doc/technical_overview.md +++ b/doc/technical_overview.md @@ -283,3 +283,28 @@ cross-compilation, this is done by the same tools as for native compilation. Depending on ISAR_CROSS_COMPILE value, additional architecture specifiers are passed to build tools automatically, so this is absolutely transparent from the user point of view. + +## 3.9 Additional features + +### 3.9.1 Template files + +A basic templating system implemented on top `envsubst(1)` is available. It +allows limited access to bitbake variables for file generation. + +This system is implemented in the `template.bbclass` and defines the +`do_transform_template` task, the `TEMPLATE_FILES` and `TEMPLATE_VARS` +variables. Tasks that use need to use files generated via templates need +to be executed after the `do_transform_template` task. + +The `TEMPLATE_FILES` variable contains a space separated list of template +files and the `TEMPLATE_VARS` variable a space separated list of bitbake +variable names. The `do_transform_template` task takes these and generates +files using `envsubst(1)`. The output files are placed in the same directory +as the template files. The name of the output files are either the name of the +template file with `.tmpl` removed from the file name end or, if the template +file did not end with `.tmpl`, the name of the template file with `.out` added +to the end. + +Only template files from the `WORKDIR` are accepted. Either specify relative +paths based on the recipes `WORKDIR` or absolute paths containing the `WORKDIR` +in the `TEMPLATE_FILES` variable. diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass index d4082de..5ab6fa4 100644 --- a/meta/classes/base.bbclass +++ b/meta/classes/base.bbclass @@ -25,6 +25,7 @@ die() { } inherit logging +inherit template # Derived from bitbake: bitbake/classes/base.bbclass addtask showdata diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass index f1b127c..fe96850 100644 --- a/meta/classes/dpkg-base.bbclass +++ b/meta/classes/dpkg-base.bbclass @@ -40,7 +40,7 @@ do_prepare_build() { true } -addtask prepare_build after do_patch before do_build +addtask prepare_build after do_patch do_transform_template before do_build do_prepare_build[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" # If Isar recipes depend on each other, they typically need the package # deployed to isar-apt diff --git a/meta/classes/template.bbclass b/meta/classes/template.bbclass new file mode 100644 index 0000000..716b7a5 --- /dev/null +++ b/meta/classes/template.bbclass @@ -0,0 +1,62 @@ +# This software is a part of ISAR. +# Copyright (C) Siemens AG, 2019 +# +# SPDX-License-Identifier: MIT + +TEMPLATE_FILES ??= "" +TEMPLATE_VARS ??= "PN PV DESCRIPTION HOMEPAGE MAINTAINER KERNEL_NAME MACHINE \ + DISTRO_ARCH" + +do_transform_template[vardeps] = "TEMPLATE_FILES ${TEMPLATE_VARS}" +do_transform_template[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" +python do_transform_template() { + import subprocess, contextlib + + workdir = os.path.normpath(d.getVar('WORKDIR', True)) + + template_vars = (d.getVar('TEMPLATE_VARS', True) or "").split() + if len(template_vars) == 0: + return + + template_files = (d.getVar('TEMPLATE_FILES', True) or "").split() + if len(template_files) == 0: + return + + cmd = "envsubst" + args = " ".join("\${{{}}}".format(i) for i in template_vars) + + # Copy current process environment and add template variables + # from bitbake data store: + env = os.environ.copy() + for varname in template_vars: + value = d.getVar(varname, True) + if value: + env.update({varname: value}) + + for template_file in template_files: + # Normpath and workdir checks should prevent accidential or + # uninformed changes to files outside of the tmp and workdirectoy + template_file = os.path.normpath(template_file) + + # Convert relative paths to absolut paths based on the workdir: + if not os.path.isabs(template_file): + template_file = os.path.normpath(os.path.join(workdir, template_file)) + + if not template_file.startswith(workdir): + bb.fatal("Template file ({}) is not within workdir ({})" + .format(template_file, workdir)) + + output_file = (os.path.splitext(template_file)[0] + if template_file.endswith(".tmpl") + else (template_file + ".out")) + bb.note("{} {} [in: {} out: {}]".format(cmd, args, + template_file, output_file)) + with contextlib.ExitStack() as stack: + input = stack.enter_context(open(template_file, 'rb')) + output = stack.enter_context(open(output_file, 'wb')) + process = subprocess.Popen([cmd, args], stdin=input, + stdout=output, env=env) + if process.wait() != 0: + bb.fatal("processing of template failed") +} +addtask do_transform_template after do_unpack before do_build diff --git a/meta/recipes-bsp/u-boot/files/debian/changelog b/meta/recipes-bsp/u-boot/files/debian/changelog.tmpl similarity index 74% rename from meta/recipes-bsp/u-boot/files/debian/changelog rename to meta/recipes-bsp/u-boot/files/debian/changelog.tmpl index c1c3516..6e59e06 100644 --- a/meta/recipes-bsp/u-boot/files/debian/changelog +++ b/meta/recipes-bsp/u-boot/files/debian/changelog.tmpl @@ -1,4 +1,4 @@ -@PN@ (@PV@) unstable; urgency=low +${PN} (${PV}) unstable; urgency=low * Generated package. diff --git a/meta/recipes-bsp/u-boot/files/debian/control b/meta/recipes-bsp/u-boot/files/debian/control deleted file mode 100644 index 6b4c839..0000000 --- a/meta/recipes-bsp/u-boot/files/debian/control +++ /dev/null @@ -1,19 +0,0 @@ -Source: @PN@ -Section: admin -Priority: optional -Standards-Version: 3.9.6 -Build-Depends: @BUILD_DEPENDS@ -Maintainer: ISAR project - -Package: u-boot-@MACHINE@ -Architecture: @DISTRO_ARCH@ -Description: @DESCRIPTION@, bootloader binaries - -Package: u-boot-@MACHINE@-dev -Architecture: @DISTRO_ARCH@ -Description: @DESCRIPTION@, bootloader libraries - -Package: u-boot-tools -Architecture: linux-any -Depends: ${shlibs:Depends}, ${misc:Depends} -Description: @DESCRIPTION@, companion tools diff --git a/meta/recipes-bsp/u-boot/files/debian/control.tmpl b/meta/recipes-bsp/u-boot/files/debian/control.tmpl new file mode 100644 index 0000000..5c1cc92 --- /dev/null +++ b/meta/recipes-bsp/u-boot/files/debian/control.tmpl @@ -0,0 +1,19 @@ +Source: ${PN} +Section: admin +Priority: optional +Standards-Version: 3.9.6 +Build-Depends: ${BUILD_DEPENDS} +Maintainer: ISAR project + +Package: u-boot-${MACHINE} +Architecture: ${DISTRO_ARCH} +Description: ${DESCRIPTION}, bootloader binaries + +Package: u-boot-${MACHINE}-dev +Architecture: ${DISTRO_ARCH} +Description: ${DESCRIPTION}, bootloader libraries + +Package: u-boot-tools +Architecture: linux-any +Depends: ${shlibs:Depends}, ${misc:Depends} +Description: ${DESCRIPTION}, companion tools diff --git a/meta/recipes-bsp/u-boot/u-boot-custom.inc b/meta/recipes-bsp/u-boot/u-boot-custom.inc index 4b38c88..c6aab43 100644 --- a/meta/recipes-bsp/u-boot/u-boot-custom.inc +++ b/meta/recipes-bsp/u-boot/u-boot-custom.inc @@ -15,16 +15,14 @@ inherit dpkg SRC_URI += "file://debian/" -U_BOOT_BUILD_DEPENDS ?= "bc, bison, flex, device-tree-compiler" +BUILD_DEPENDS ?= "bc, bison, flex, device-tree-compiler" + +TEMPLATE_FILES = "debian/changelog.tmpl \ + debian/control.tmpl" +TEMPLATE_VARS += "BUILD_DEPENDS" do_prepare_build() { cp -r ${WORKDIR}/debian ${S}/ - sed -i -e 's/@PN@/${PN}/g' -e 's/@PV@/${PV}/g' \ - -e 's/@BUILD_DEPENDS@/${U_BOOT_BUILD_DEPENDS}/g' \ - -e 's/@MACHINE@/${MACHINE}/g' \ - -e 's/@DISTRO_ARCH@/${DISTRO_ARCH}/g' \ - -e 's/@DESCRIPTION@/${DESCRIPTION}/g' \ - ${S}/debian/changelog ${S}/debian/control echo "${U_BOOT_BIN} /usr/lib/u-boot/${MACHINE}" > \ ${S}/debian/u-boot-${MACHINE}.install diff --git a/meta/recipes-kernel/linux-module/files/debian/changelog b/meta/recipes-kernel/linux-module/files/debian/changelog.tmpl similarity index 74% rename from meta/recipes-kernel/linux-module/files/debian/changelog rename to meta/recipes-kernel/linux-module/files/debian/changelog.tmpl index c1c3516..6e59e06 100644 --- a/meta/recipes-kernel/linux-module/files/debian/changelog +++ b/meta/recipes-kernel/linux-module/files/debian/changelog.tmpl @@ -1,4 +1,4 @@ -@PN@ (@PV@) unstable; urgency=low +${PN} (${PV}) unstable; urgency=low * Generated package. diff --git a/meta/recipes-kernel/linux-module/files/debian/control b/meta/recipes-kernel/linux-module/files/debian/control deleted file mode 100644 index 1ee634c..0000000 --- a/meta/recipes-kernel/linux-module/files/debian/control +++ /dev/null @@ -1,11 +0,0 @@ -Source: @PN@ -Section: kernel -Priority: optional -Standards-Version: 3.9.6 -Build-Depends: linux-headers-@KERNEL_NAME@ -Maintainer: ISAR project - -Package: @PN@ -Architecture: any -Depends: linux-image-@KERNEL_NAME@, kmod -Description: @DESCRIPTION@ diff --git a/meta/recipes-kernel/linux-module/files/debian/control.tmpl b/meta/recipes-kernel/linux-module/files/debian/control.tmpl new file mode 100644 index 0000000..3b3292d --- /dev/null +++ b/meta/recipes-kernel/linux-module/files/debian/control.tmpl @@ -0,0 +1,11 @@ +Source: ${PN} +Section: kernel +Priority: optional +Standards-Version: 3.9.6 +Build-Depends: linux-headers-${KERNEL_NAME} +Maintainer: ISAR project + +Package: ${PN} +Architecture: any +Depends: linux-image-${KERNEL_NAME}, kmod +Description: ${DESCRIPTION} diff --git a/meta/recipes-kernel/linux-module/module.inc b/meta/recipes-kernel/linux-module/module.inc index cb7b8ad..44edd58 100644 --- a/meta/recipes-kernel/linux-module/module.inc +++ b/meta/recipes-kernel/linux-module/module.inc @@ -19,12 +19,11 @@ AUTOLOAD ?= "" inherit dpkg +TEMPLATE_FILES = "debian/control.tmpl \ + debian/changelog.tmpl" + do_prepare_build() { cp -r ${WORKDIR}/debian ${S}/ - sed -i -e 's/@PN@/${PN}/g' -e 's/@PV@/${PV}/g' \ - -e 's/@KERNEL_NAME@/${KERNEL_NAME}/g' \ - -e 's/@DESCRIPTION@/${DESCRIPTION}/g' \ - ${S}/debian/changelog ${S}/debian/control for module in "${AUTOLOAD}"; do echo "echo $module >> /etc/modules" >> ${S}/debian/postinst -- 2.20.1