From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6652594697292218368 X-Received: by 2002:a1c:10c:: with SMTP id 12mr76915wmb.8.1550070836255; Wed, 13 Feb 2019 07:13:56 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:adf:ec82:: with SMTP id z2ls4064493wrn.1.gmail; Wed, 13 Feb 2019 07:13:55 -0800 (PST) X-Google-Smtp-Source: AHgI3IaYgjLM0xoTmeO+g5QlvgOGKVyCWG2wpTUJKvcd7emCwv/C0VJZEpFheFM6eMPhAAcVQYv8 X-Received: by 2002:adf:822f:: with SMTP id 44mr83850wrb.2.1550070835721; Wed, 13 Feb 2019 07:13:55 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550070835; cv=none; d=google.com; s=arc-20160816; b=PDOMqMldNvOev6h+ixnNZSiAjtIt39tuc7F+Nt7rjqAhcrL3nSFctL0e9LFDyE0qx4 NE+88eaZhCIfFaXtSbMOvjIuYvpx8n0d0/DLfN9+1D8t3BrppSDTEKqilUFg//jE0qv3 ZsvV+eSOqtoyINc6kG4Gpn7Rt0LIcFQxvilcNFqqvGXrwWBE0EfYSMHxvk/UFlCVpvLn 4i6cMSkPG59dW58q9RkUjxjVTjOsS1JGOabfcGUaP5FCjzp3Pxyi8GneTEv0NcLKjrdw 2MOxQG7Br7L2dC33IabAnOyFsyxd5Z27Ajj/0MEydmo0dZAhIFYgjBFv2VOIRHubzvMZ 6SUA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=f1Gnp+SWbRQG2gBq85BQKSaAGGhoUhlsBuld6RGepM0=; b=fIYkNZt4amduxL7kXz2EZUuThFHzwwbSgpDasCeLt6J6gOWwPbwkEwZ8F5Meo7oGWz ANMRvcCnnq1MLEuzBVl+FwCmbe3bqRrxfwPG/IoqtasT24VfFFLL1SsIpl4k6wklfzLJ UcA8LiHnu6TNSTJ32TjmmpdviJvzdVDFUhNdkOhD1Ay1xxskjSn8LSRziLmaNOL2TwqI 5iGmELt3jhHvuSr8jk2ujEU0FPRPoDYZaDx25zOHIGA1wWWCoBWiUoUDJKKyHa20U9e5 O6gzk94kru1iEu8meVNugfHzTXQ3hyltZ9VgUPvwBaMlr1qJKD23mAjFd3Fvba9cU3Er 7Trg== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of claudius.heine.ext@siemens.com designates 192.35.17.14 as permitted sender) smtp.mailfrom=claudius.heine.ext@siemens.com Return-Path: Received: from david.siemens.de (david.siemens.de. [192.35.17.14]) by gmr-mx.google.com with ESMTPS id c12si722492wrn.5.2019.02.13.07.13.55 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 13 Feb 2019 07:13:55 -0800 (PST) Received-SPF: pass (google.com: domain of claudius.heine.ext@siemens.com designates 192.35.17.14 as permitted sender) client-ip=192.35.17.14; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of claudius.heine.ext@siemens.com designates 192.35.17.14 as permitted sender) smtp.mailfrom=claudius.heine.ext@siemens.com Received: from mail1.sbs.de (mail1.sbs.de [192.129.41.35]) by david.siemens.de (8.15.2/8.15.2) with ESMTPS id x1DFDsjG013289 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 13 Feb 2019 16:13:55 +0100 Received: from ring.ppmd.siemens.net (linux-ses-ext02.ppmd.siemens.net [139.25.69.181]) by mail1.sbs.de (8.15.2/8.15.2) with ESMTP id x1DFDsiO005696; Wed, 13 Feb 2019 16:13:54 +0100 From: claudius.heine.ext@siemens.com To: isar-users@googlegroups.com Cc: Claudius Heine Subject: [PATCH v3 1/1] meta: added do_transform_template task as templating system and switch Date: Wed, 13 Feb 2019 16:13:46 +0100 Message-Id: <20190213151346.2498-2-claudius.heine.ext@siemens.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190213151346.2498-1-claudius.heine.ext@siemens.com> References: <20190213151346.2498-1-claudius.heine.ext@siemens.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-TUID: Tx/c8t9lm/Jw From: Claudius Heine The new introduced variables are TEMPLATE_FILES and TEMPLATE_VARS. TEMPLATE_FILES is a list of files that should be processed in the `do_transform_template` task and TEMPLATE_VARS is a list of bitbake variable names that should be substituted in the template file. Signed-off-by: Claudius Heine --- doc/technical_overview.md | 25 ++++++++ doc/user_manual.md | 1 + meta/classes/base.bbclass | 1 + meta/classes/dpkg-base.bbclass | 2 +- meta/classes/template.bbclass | 62 +++++++++++++++++++ .../debian/{changelog => changelog.tmpl} | 2 +- meta/recipes-bsp/u-boot/files/debian/control | 19 ------ .../u-boot/files/debian/control.tmpl | 19 ++++++ meta/recipes-bsp/u-boot/u-boot-custom.inc | 12 ++-- .../debian/{changelog => changelog.tmpl} | 2 +- .../linux-module/files/debian/control | 11 ---- .../linux-module/files/debian/control.tmpl | 11 ++++ meta/recipes-kernel/linux-module/module.inc | 7 +-- 13 files changed, 130 insertions(+), 44 deletions(-) create mode 100644 meta/classes/template.bbclass rename meta/recipes-bsp/u-boot/files/debian/{changelog => changelog.tmpl} (74%) delete mode 100644 meta/recipes-bsp/u-boot/files/debian/control create mode 100644 meta/recipes-bsp/u-boot/files/debian/control.tmpl rename meta/recipes-kernel/linux-module/files/debian/{changelog => changelog.tmpl} (74%) delete mode 100644 meta/recipes-kernel/linux-module/files/debian/control create mode 100644 meta/recipes-kernel/linux-module/files/debian/control.tmpl diff --git a/doc/technical_overview.md b/doc/technical_overview.md index 7efbeee..0ddcabc 100644 --- a/doc/technical_overview.md +++ b/doc/technical_overview.md @@ -283,3 +283,28 @@ cross-compilation, this is done by the same tools as for native compilation. Depending on ISAR_CROSS_COMPILE value, additional architecture specifiers are passed to build tools automatically, so this is absolutely transparent from the user point of view. + +## 3.9 Additional features + +### 3.9.1 Template files + +A basic templating system implemented on top `envsubst(1)` is available. It +allows limited access to bitbake variables for file generation. + +This system is implemented in the `template.bbclass` and defines the +`do_transform_template` task, the `TEMPLATE_FILES` and `TEMPLATE_VARS` +variables. Tasks that use need to use files generated via templates need +to be executed after the `do_transform_template` task. + +The `TEMPLATE_FILES` variable contains a space separated list of template +files and the `TEMPLATE_VARS` variable a space separated list of bitbake +variable names. The `do_transform_template` task takes these and generates +files using `envsubst(1)`. The output files are placed in the same directory +as the template files. The name of the output files are either the name of the +template file with `.tmpl` removed from the file name end or, if the template +file did not end with `.tmpl`, the name of the template file with `.out` added +to the end. + +Only template files from the `WORKDIR` are accepted. Either specify relative +paths based on the recipes `WORKDIR` or absolute paths containing the `WORKDIR` +in the `TEMPLATE_FILES` variable. diff --git a/doc/user_manual.md b/doc/user_manual.md index ff4939c..c1b154f 100644 --- a/doc/user_manual.md +++ b/doc/user_manual.md @@ -68,6 +68,7 @@ qemu-user-static binfmt-support sudo reprepro +gettext-base ``` Notes: diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass index 6ca93bf..4279a68 100644 --- a/meta/classes/base.bbclass +++ b/meta/classes/base.bbclass @@ -25,6 +25,7 @@ die() { } inherit logging +inherit template # Derived from bitbake: bitbake/classes/base.bbclass addtask showdata diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass index 175dc80..320869b 100644 --- a/meta/classes/dpkg-base.bbclass +++ b/meta/classes/dpkg-base.bbclass @@ -60,7 +60,7 @@ do_prepare_build() { true } -addtask prepare_build after do_patch before do_build +addtask prepare_build after do_patch do_transform_template before do_build do_prepare_build[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" # If Isar recipes depend on each other, they typically need the package # deployed to isar-apt diff --git a/meta/classes/template.bbclass b/meta/classes/template.bbclass new file mode 100644 index 0000000..00d7453 --- /dev/null +++ b/meta/classes/template.bbclass @@ -0,0 +1,62 @@ +# This software is a part of ISAR. +# Copyright (C) Siemens AG, 2019 +# +# SPDX-License-Identifier: MIT + +TEMPLATE_FILES ??= "" +TEMPLATE_VARS ??= "PN PV DESCRIPTION HOMEPAGE MAINTAINER KERNEL_NAME MACHINE \ + DISTRO_ARCH" + +do_transform_template[vardeps] = "TEMPLATE_FILES ${TEMPLATE_VARS}" +do_transform_template[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" +python do_transform_template() { + import subprocess, contextlib + + workdir = os.path.normpath(d.getVar('WORKDIR', True)) + + template_vars = (d.getVar('TEMPLATE_VARS', True) or "").split() + if len(template_vars) == 0: + return + + template_files = (d.getVar('TEMPLATE_FILES', True) or "").split() + if len(template_files) == 0: + return + + cmd = "envsubst" + args = " ".join("\${{{}}}".format(i) for i in template_vars) + + # Copy current process environment and add template variables + # from bitbake data store: + env = os.environ.copy() + for varname in template_vars: + value = d.getVar(varname, True) + if value: + env.update({varname: value}) + + for template_file in template_files: + # Normpath and workdir checks should prevent accidential or + # uninformed changes to files outside of the tmp and workdirectoy + template_file = os.path.normpath(template_file) + + # Convert relative paths to absolut paths based on the workdir: + if not os.path.isabs(template_file): + template_file = os.path.normpath(os.path.join(workdir, template_file)) + + if not template_file.startswith(workdir): + bb.fatal("Template file ({}) is not within workdir ({})" + .format(template_file, workdir)) + + output_file = (os.path.splitext(template_file)[0] + if template_file.endswith(".tmpl") + else (template_file + ".out")) + bb.note("{} {} [in: {} out: {}]".format(cmd, args, + template_file, output_file)) + with contextlib.ExitStack() as stack: + input = stack.enter_context(open(template_file, 'rb')) + output = stack.enter_context(open(output_file, 'wb')) + process = stack.enter_context(subprocess.Popen([cmd, args], stdin=input, + stdout=output, env=env)) + if process.wait() != 0: + bb.fatal("processing of template failed") +} +addtask do_transform_template after do_unpack diff --git a/meta/recipes-bsp/u-boot/files/debian/changelog b/meta/recipes-bsp/u-boot/files/debian/changelog.tmpl similarity index 74% rename from meta/recipes-bsp/u-boot/files/debian/changelog rename to meta/recipes-bsp/u-boot/files/debian/changelog.tmpl index c1c3516..6e59e06 100644 --- a/meta/recipes-bsp/u-boot/files/debian/changelog +++ b/meta/recipes-bsp/u-boot/files/debian/changelog.tmpl @@ -1,4 +1,4 @@ -@PN@ (@PV@) unstable; urgency=low +${PN} (${PV}) unstable; urgency=low * Generated package. diff --git a/meta/recipes-bsp/u-boot/files/debian/control b/meta/recipes-bsp/u-boot/files/debian/control deleted file mode 100644 index 6b4c839..0000000 --- a/meta/recipes-bsp/u-boot/files/debian/control +++ /dev/null @@ -1,19 +0,0 @@ -Source: @PN@ -Section: admin -Priority: optional -Standards-Version: 3.9.6 -Build-Depends: @BUILD_DEPENDS@ -Maintainer: ISAR project - -Package: u-boot-@MACHINE@ -Architecture: @DISTRO_ARCH@ -Description: @DESCRIPTION@, bootloader binaries - -Package: u-boot-@MACHINE@-dev -Architecture: @DISTRO_ARCH@ -Description: @DESCRIPTION@, bootloader libraries - -Package: u-boot-tools -Architecture: linux-any -Depends: ${shlibs:Depends}, ${misc:Depends} -Description: @DESCRIPTION@, companion tools diff --git a/meta/recipes-bsp/u-boot/files/debian/control.tmpl b/meta/recipes-bsp/u-boot/files/debian/control.tmpl new file mode 100644 index 0000000..5c1cc92 --- /dev/null +++ b/meta/recipes-bsp/u-boot/files/debian/control.tmpl @@ -0,0 +1,19 @@ +Source: ${PN} +Section: admin +Priority: optional +Standards-Version: 3.9.6 +Build-Depends: ${BUILD_DEPENDS} +Maintainer: ISAR project + +Package: u-boot-${MACHINE} +Architecture: ${DISTRO_ARCH} +Description: ${DESCRIPTION}, bootloader binaries + +Package: u-boot-${MACHINE}-dev +Architecture: ${DISTRO_ARCH} +Description: ${DESCRIPTION}, bootloader libraries + +Package: u-boot-tools +Architecture: linux-any +Depends: ${shlibs:Depends}, ${misc:Depends} +Description: ${DESCRIPTION}, companion tools diff --git a/meta/recipes-bsp/u-boot/u-boot-custom.inc b/meta/recipes-bsp/u-boot/u-boot-custom.inc index 4b38c88..c6aab43 100644 --- a/meta/recipes-bsp/u-boot/u-boot-custom.inc +++ b/meta/recipes-bsp/u-boot/u-boot-custom.inc @@ -15,16 +15,14 @@ inherit dpkg SRC_URI += "file://debian/" -U_BOOT_BUILD_DEPENDS ?= "bc, bison, flex, device-tree-compiler" +BUILD_DEPENDS ?= "bc, bison, flex, device-tree-compiler" + +TEMPLATE_FILES = "debian/changelog.tmpl \ + debian/control.tmpl" +TEMPLATE_VARS += "BUILD_DEPENDS" do_prepare_build() { cp -r ${WORKDIR}/debian ${S}/ - sed -i -e 's/@PN@/${PN}/g' -e 's/@PV@/${PV}/g' \ - -e 's/@BUILD_DEPENDS@/${U_BOOT_BUILD_DEPENDS}/g' \ - -e 's/@MACHINE@/${MACHINE}/g' \ - -e 's/@DISTRO_ARCH@/${DISTRO_ARCH}/g' \ - -e 's/@DESCRIPTION@/${DESCRIPTION}/g' \ - ${S}/debian/changelog ${S}/debian/control echo "${U_BOOT_BIN} /usr/lib/u-boot/${MACHINE}" > \ ${S}/debian/u-boot-${MACHINE}.install diff --git a/meta/recipes-kernel/linux-module/files/debian/changelog b/meta/recipes-kernel/linux-module/files/debian/changelog.tmpl similarity index 74% rename from meta/recipes-kernel/linux-module/files/debian/changelog rename to meta/recipes-kernel/linux-module/files/debian/changelog.tmpl index c1c3516..6e59e06 100644 --- a/meta/recipes-kernel/linux-module/files/debian/changelog +++ b/meta/recipes-kernel/linux-module/files/debian/changelog.tmpl @@ -1,4 +1,4 @@ -@PN@ (@PV@) unstable; urgency=low +${PN} (${PV}) unstable; urgency=low * Generated package. diff --git a/meta/recipes-kernel/linux-module/files/debian/control b/meta/recipes-kernel/linux-module/files/debian/control deleted file mode 100644 index 1ee634c..0000000 --- a/meta/recipes-kernel/linux-module/files/debian/control +++ /dev/null @@ -1,11 +0,0 @@ -Source: @PN@ -Section: kernel -Priority: optional -Standards-Version: 3.9.6 -Build-Depends: linux-headers-@KERNEL_NAME@ -Maintainer: ISAR project - -Package: @PN@ -Architecture: any -Depends: linux-image-@KERNEL_NAME@, kmod -Description: @DESCRIPTION@ diff --git a/meta/recipes-kernel/linux-module/files/debian/control.tmpl b/meta/recipes-kernel/linux-module/files/debian/control.tmpl new file mode 100644 index 0000000..3b3292d --- /dev/null +++ b/meta/recipes-kernel/linux-module/files/debian/control.tmpl @@ -0,0 +1,11 @@ +Source: ${PN} +Section: kernel +Priority: optional +Standards-Version: 3.9.6 +Build-Depends: linux-headers-${KERNEL_NAME} +Maintainer: ISAR project + +Package: ${PN} +Architecture: any +Depends: linux-image-${KERNEL_NAME}, kmod +Description: ${DESCRIPTION} diff --git a/meta/recipes-kernel/linux-module/module.inc b/meta/recipes-kernel/linux-module/module.inc index cb7b8ad..44edd58 100644 --- a/meta/recipes-kernel/linux-module/module.inc +++ b/meta/recipes-kernel/linux-module/module.inc @@ -19,12 +19,11 @@ AUTOLOAD ?= "" inherit dpkg +TEMPLATE_FILES = "debian/control.tmpl \ + debian/changelog.tmpl" + do_prepare_build() { cp -r ${WORKDIR}/debian ${S}/ - sed -i -e 's/@PN@/${PN}/g' -e 's/@PV@/${PV}/g' \ - -e 's/@KERNEL_NAME@/${KERNEL_NAME}/g' \ - -e 's/@DESCRIPTION@/${DESCRIPTION}/g' \ - ${S}/debian/changelog ${S}/debian/control for module in "${AUTOLOAD}"; do echo "echo $module >> /etc/modules" >> ${S}/debian/postinst -- 2.20.1