* [PATCH v2 0/1] Template system
@ 2019-02-05 11:18 claudius.heine.ext
2019-02-05 11:18 ` [PATCH v2 1/1] meta: added do_transform_template task as templating system and switch claudius.heine.ext
0 siblings, 1 reply; 4+ messages in thread
From: claudius.heine.ext @ 2019-02-05 11:18 UTC (permalink / raw)
To: isar-users; +Cc: Claudius Heine
From: Claudius Heine <ch@denx.de>
Hi,
here the v2 of the templating system patchset.
Claudius
Changes from v1:
- fixed spelling mistakes of documentation
Claudius Heine (1):
meta: added do_transform_template task as templating system and switch
doc/technical_overview.md | 25 ++++++++
meta/classes/base.bbclass | 1 +
meta/classes/dpkg-base.bbclass | 2 +-
meta/classes/template.bbclass | 62 +++++++++++++++++++
.../debian/{changelog => changelog.tmpl} | 2 +-
meta/recipes-bsp/u-boot/files/debian/control | 19 ------
.../u-boot/files/debian/control.tmpl | 19 ++++++
meta/recipes-bsp/u-boot/u-boot-custom.inc | 12 ++--
.../debian/{changelog => changelog.tmpl} | 2 +-
.../linux-module/files/debian/control | 11 ----
.../linux-module/files/debian/control.tmpl | 11 ++++
meta/recipes-kernel/linux-module/module.inc | 7 +--
12 files changed, 129 insertions(+), 44 deletions(-)
create mode 100644 meta/classes/template.bbclass
rename meta/recipes-bsp/u-boot/files/debian/{changelog => changelog.tmpl} (74%)
delete mode 100644 meta/recipes-bsp/u-boot/files/debian/control
create mode 100644 meta/recipes-bsp/u-boot/files/debian/control.tmpl
rename meta/recipes-kernel/linux-module/files/debian/{changelog => changelog.tmpl} (74%)
delete mode 100644 meta/recipes-kernel/linux-module/files/debian/control
create mode 100644 meta/recipes-kernel/linux-module/files/debian/control.tmpl
--
2.20.1
^ permalink raw reply [flat|nested] 4+ messages in thread
* [PATCH v2 1/1] meta: added do_transform_template task as templating system and switch
2019-02-05 11:18 [PATCH v2 0/1] Template system claudius.heine.ext
@ 2019-02-05 11:18 ` claudius.heine.ext
2019-02-06 13:37 ` Henning Schild
2019-02-12 10:27 ` Claudius Heine
0 siblings, 2 replies; 4+ messages in thread
From: claudius.heine.ext @ 2019-02-05 11:18 UTC (permalink / raw)
To: isar-users; +Cc: Claudius Heine
From: Claudius Heine <ch@denx.de>
The new introduced variables are TEMPLATE_FILES and TEMPLATE_VARS.
TEMPLATE_FILES is a list of files that should be processed in the
`do_transform_template` task and TEMPLATE_VARS is a list of bitbake
variable names that should be substituted in the template file.
Signed-off-by: Claudius Heine <ch@denx.de>
---
doc/technical_overview.md | 25 ++++++++
meta/classes/base.bbclass | 1 +
meta/classes/dpkg-base.bbclass | 2 +-
meta/classes/template.bbclass | 62 +++++++++++++++++++
.../debian/{changelog => changelog.tmpl} | 2 +-
meta/recipes-bsp/u-boot/files/debian/control | 19 ------
.../u-boot/files/debian/control.tmpl | 19 ++++++
meta/recipes-bsp/u-boot/u-boot-custom.inc | 12 ++--
.../debian/{changelog => changelog.tmpl} | 2 +-
.../linux-module/files/debian/control | 11 ----
.../linux-module/files/debian/control.tmpl | 11 ++++
meta/recipes-kernel/linux-module/module.inc | 7 +--
12 files changed, 129 insertions(+), 44 deletions(-)
create mode 100644 meta/classes/template.bbclass
rename meta/recipes-bsp/u-boot/files/debian/{changelog => changelog.tmpl} (74%)
delete mode 100644 meta/recipes-bsp/u-boot/files/debian/control
create mode 100644 meta/recipes-bsp/u-boot/files/debian/control.tmpl
rename meta/recipes-kernel/linux-module/files/debian/{changelog => changelog.tmpl} (74%)
delete mode 100644 meta/recipes-kernel/linux-module/files/debian/control
create mode 100644 meta/recipes-kernel/linux-module/files/debian/control.tmpl
diff --git a/doc/technical_overview.md b/doc/technical_overview.md
index 7efbeee..0ddcabc 100644
--- a/doc/technical_overview.md
+++ b/doc/technical_overview.md
@@ -283,3 +283,28 @@ cross-compilation, this is done by the same tools as for native compilation.
Depending on ISAR_CROSS_COMPILE value, additional architecture specifiers are
passed to build tools automatically, so this is absolutely transparent from
the user point of view.
+
+## 3.9 Additional features
+
+### 3.9.1 Template files
+
+A basic templating system implemented on top `envsubst(1)` is available. It
+allows limited access to bitbake variables for file generation.
+
+This system is implemented in the `template.bbclass` and defines the
+`do_transform_template` task, the `TEMPLATE_FILES` and `TEMPLATE_VARS`
+variables. Tasks that use need to use files generated via templates need
+to be executed after the `do_transform_template` task.
+
+The `TEMPLATE_FILES` variable contains a space separated list of template
+files and the `TEMPLATE_VARS` variable a space separated list of bitbake
+variable names. The `do_transform_template` task takes these and generates
+files using `envsubst(1)`. The output files are placed in the same directory
+as the template files. The name of the output files are either the name of the
+template file with `.tmpl` removed from the file name end or, if the template
+file did not end with `.tmpl`, the name of the template file with `.out` added
+to the end.
+
+Only template files from the `WORKDIR` are accepted. Either specify relative
+paths based on the recipes `WORKDIR` or absolute paths containing the `WORKDIR`
+in the `TEMPLATE_FILES` variable.
diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
index d4082de..5ab6fa4 100644
--- a/meta/classes/base.bbclass
+++ b/meta/classes/base.bbclass
@@ -25,6 +25,7 @@ die() {
}
inherit logging
+inherit template
# Derived from bitbake: bitbake/classes/base.bbclass
addtask showdata
diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass
index f1b127c..fe96850 100644
--- a/meta/classes/dpkg-base.bbclass
+++ b/meta/classes/dpkg-base.bbclass
@@ -40,7 +40,7 @@ do_prepare_build() {
true
}
-addtask prepare_build after do_patch before do_build
+addtask prepare_build after do_patch do_transform_template before do_build
do_prepare_build[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
# If Isar recipes depend on each other, they typically need the package
# deployed to isar-apt
diff --git a/meta/classes/template.bbclass b/meta/classes/template.bbclass
new file mode 100644
index 0000000..716b7a5
--- /dev/null
+++ b/meta/classes/template.bbclass
@@ -0,0 +1,62 @@
+# This software is a part of ISAR.
+# Copyright (C) Siemens AG, 2019
+#
+# SPDX-License-Identifier: MIT
+
+TEMPLATE_FILES ??= ""
+TEMPLATE_VARS ??= "PN PV DESCRIPTION HOMEPAGE MAINTAINER KERNEL_NAME MACHINE \
+ DISTRO_ARCH"
+
+do_transform_template[vardeps] = "TEMPLATE_FILES ${TEMPLATE_VARS}"
+do_transform_template[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
+python do_transform_template() {
+ import subprocess, contextlib
+
+ workdir = os.path.normpath(d.getVar('WORKDIR', True))
+
+ template_vars = (d.getVar('TEMPLATE_VARS', True) or "").split()
+ if len(template_vars) == 0:
+ return
+
+ template_files = (d.getVar('TEMPLATE_FILES', True) or "").split()
+ if len(template_files) == 0:
+ return
+
+ cmd = "envsubst"
+ args = " ".join("\${{{}}}".format(i) for i in template_vars)
+
+ # Copy current process environment and add template variables
+ # from bitbake data store:
+ env = os.environ.copy()
+ for varname in template_vars:
+ value = d.getVar(varname, True)
+ if value:
+ env.update({varname: value})
+
+ for template_file in template_files:
+ # Normpath and workdir checks should prevent accidential or
+ # uninformed changes to files outside of the tmp and workdirectoy
+ template_file = os.path.normpath(template_file)
+
+ # Convert relative paths to absolut paths based on the workdir:
+ if not os.path.isabs(template_file):
+ template_file = os.path.normpath(os.path.join(workdir, template_file))
+
+ if not template_file.startswith(workdir):
+ bb.fatal("Template file ({}) is not within workdir ({})"
+ .format(template_file, workdir))
+
+ output_file = (os.path.splitext(template_file)[0]
+ if template_file.endswith(".tmpl")
+ else (template_file + ".out"))
+ bb.note("{} {} [in: {} out: {}]".format(cmd, args,
+ template_file, output_file))
+ with contextlib.ExitStack() as stack:
+ input = stack.enter_context(open(template_file, 'rb'))
+ output = stack.enter_context(open(output_file, 'wb'))
+ process = subprocess.Popen([cmd, args], stdin=input,
+ stdout=output, env=env)
+ if process.wait() != 0:
+ bb.fatal("processing of template failed")
+}
+addtask do_transform_template after do_unpack before do_build
diff --git a/meta/recipes-bsp/u-boot/files/debian/changelog b/meta/recipes-bsp/u-boot/files/debian/changelog.tmpl
similarity index 74%
rename from meta/recipes-bsp/u-boot/files/debian/changelog
rename to meta/recipes-bsp/u-boot/files/debian/changelog.tmpl
index c1c3516..6e59e06 100644
--- a/meta/recipes-bsp/u-boot/files/debian/changelog
+++ b/meta/recipes-bsp/u-boot/files/debian/changelog.tmpl
@@ -1,4 +1,4 @@
-@PN@ (@PV@) unstable; urgency=low
+${PN} (${PV}) unstable; urgency=low
* Generated package.
diff --git a/meta/recipes-bsp/u-boot/files/debian/control b/meta/recipes-bsp/u-boot/files/debian/control
deleted file mode 100644
index 6b4c839..0000000
--- a/meta/recipes-bsp/u-boot/files/debian/control
+++ /dev/null
@@ -1,19 +0,0 @@
-Source: @PN@
-Section: admin
-Priority: optional
-Standards-Version: 3.9.6
-Build-Depends: @BUILD_DEPENDS@
-Maintainer: ISAR project <isar-users@googlegroups.com>
-
-Package: u-boot-@MACHINE@
-Architecture: @DISTRO_ARCH@
-Description: @DESCRIPTION@, bootloader binaries
-
-Package: u-boot-@MACHINE@-dev
-Architecture: @DISTRO_ARCH@
-Description: @DESCRIPTION@, bootloader libraries
-
-Package: u-boot-tools
-Architecture: linux-any
-Depends: ${shlibs:Depends}, ${misc:Depends}
-Description: @DESCRIPTION@, companion tools
diff --git a/meta/recipes-bsp/u-boot/files/debian/control.tmpl b/meta/recipes-bsp/u-boot/files/debian/control.tmpl
new file mode 100644
index 0000000..5c1cc92
--- /dev/null
+++ b/meta/recipes-bsp/u-boot/files/debian/control.tmpl
@@ -0,0 +1,19 @@
+Source: ${PN}
+Section: admin
+Priority: optional
+Standards-Version: 3.9.6
+Build-Depends: ${BUILD_DEPENDS}
+Maintainer: ISAR project <isar-users@googlegroups.com>
+
+Package: u-boot-${MACHINE}
+Architecture: ${DISTRO_ARCH}
+Description: ${DESCRIPTION}, bootloader binaries
+
+Package: u-boot-${MACHINE}-dev
+Architecture: ${DISTRO_ARCH}
+Description: ${DESCRIPTION}, bootloader libraries
+
+Package: u-boot-tools
+Architecture: linux-any
+Depends: ${shlibs:Depends}, ${misc:Depends}
+Description: ${DESCRIPTION}, companion tools
diff --git a/meta/recipes-bsp/u-boot/u-boot-custom.inc b/meta/recipes-bsp/u-boot/u-boot-custom.inc
index 4b38c88..c6aab43 100644
--- a/meta/recipes-bsp/u-boot/u-boot-custom.inc
+++ b/meta/recipes-bsp/u-boot/u-boot-custom.inc
@@ -15,16 +15,14 @@ inherit dpkg
SRC_URI += "file://debian/"
-U_BOOT_BUILD_DEPENDS ?= "bc, bison, flex, device-tree-compiler"
+BUILD_DEPENDS ?= "bc, bison, flex, device-tree-compiler"
+
+TEMPLATE_FILES = "debian/changelog.tmpl \
+ debian/control.tmpl"
+TEMPLATE_VARS += "BUILD_DEPENDS"
do_prepare_build() {
cp -r ${WORKDIR}/debian ${S}/
- sed -i -e 's/@PN@/${PN}/g' -e 's/@PV@/${PV}/g' \
- -e 's/@BUILD_DEPENDS@/${U_BOOT_BUILD_DEPENDS}/g' \
- -e 's/@MACHINE@/${MACHINE}/g' \
- -e 's/@DISTRO_ARCH@/${DISTRO_ARCH}/g' \
- -e 's/@DESCRIPTION@/${DESCRIPTION}/g' \
- ${S}/debian/changelog ${S}/debian/control
echo "${U_BOOT_BIN} /usr/lib/u-boot/${MACHINE}" > \
${S}/debian/u-boot-${MACHINE}.install
diff --git a/meta/recipes-kernel/linux-module/files/debian/changelog b/meta/recipes-kernel/linux-module/files/debian/changelog.tmpl
similarity index 74%
rename from meta/recipes-kernel/linux-module/files/debian/changelog
rename to meta/recipes-kernel/linux-module/files/debian/changelog.tmpl
index c1c3516..6e59e06 100644
--- a/meta/recipes-kernel/linux-module/files/debian/changelog
+++ b/meta/recipes-kernel/linux-module/files/debian/changelog.tmpl
@@ -1,4 +1,4 @@
-@PN@ (@PV@) unstable; urgency=low
+${PN} (${PV}) unstable; urgency=low
* Generated package.
diff --git a/meta/recipes-kernel/linux-module/files/debian/control b/meta/recipes-kernel/linux-module/files/debian/control
deleted file mode 100644
index 1ee634c..0000000
--- a/meta/recipes-kernel/linux-module/files/debian/control
+++ /dev/null
@@ -1,11 +0,0 @@
-Source: @PN@
-Section: kernel
-Priority: optional
-Standards-Version: 3.9.6
-Build-Depends: linux-headers-@KERNEL_NAME@
-Maintainer: ISAR project <isar-users@googlegroups.com>
-
-Package: @PN@
-Architecture: any
-Depends: linux-image-@KERNEL_NAME@, kmod
-Description: @DESCRIPTION@
diff --git a/meta/recipes-kernel/linux-module/files/debian/control.tmpl b/meta/recipes-kernel/linux-module/files/debian/control.tmpl
new file mode 100644
index 0000000..3b3292d
--- /dev/null
+++ b/meta/recipes-kernel/linux-module/files/debian/control.tmpl
@@ -0,0 +1,11 @@
+Source: ${PN}
+Section: kernel
+Priority: optional
+Standards-Version: 3.9.6
+Build-Depends: linux-headers-${KERNEL_NAME}
+Maintainer: ISAR project <isar-users@googlegroups.com>
+
+Package: ${PN}
+Architecture: any
+Depends: linux-image-${KERNEL_NAME}, kmod
+Description: ${DESCRIPTION}
diff --git a/meta/recipes-kernel/linux-module/module.inc b/meta/recipes-kernel/linux-module/module.inc
index cb7b8ad..44edd58 100644
--- a/meta/recipes-kernel/linux-module/module.inc
+++ b/meta/recipes-kernel/linux-module/module.inc
@@ -19,12 +19,11 @@ AUTOLOAD ?= ""
inherit dpkg
+TEMPLATE_FILES = "debian/control.tmpl \
+ debian/changelog.tmpl"
+
do_prepare_build() {
cp -r ${WORKDIR}/debian ${S}/
- sed -i -e 's/@PN@/${PN}/g' -e 's/@PV@/${PV}/g' \
- -e 's/@KERNEL_NAME@/${KERNEL_NAME}/g' \
- -e 's/@DESCRIPTION@/${DESCRIPTION}/g' \
- ${S}/debian/changelog ${S}/debian/control
for module in "${AUTOLOAD}"; do
echo "echo $module >> /etc/modules" >> ${S}/debian/postinst
--
2.20.1
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH v2 1/1] meta: added do_transform_template task as templating system and switch
2019-02-05 11:18 ` [PATCH v2 1/1] meta: added do_transform_template task as templating system and switch claudius.heine.ext
@ 2019-02-06 13:37 ` Henning Schild
2019-02-12 10:27 ` Claudius Heine
1 sibling, 0 replies; 4+ messages in thread
From: Henning Schild @ 2019-02-06 13:37 UTC (permalink / raw)
To: [ext] claudius.heine.ext@siemens.com; +Cc: isar-users, Claudius Heine
Looks good and useful to me. I did not try it but since the
example-module is a user of it we do have CI coverage.
Henning
Am Tue, 5 Feb 2019 12:18:01 +0100
schrieb "[ext] claudius.heine.ext@siemens.com"
<claudius.heine.ext@siemens.com>:
> From: Claudius Heine <ch@denx.de>
>
> The new introduced variables are TEMPLATE_FILES and TEMPLATE_VARS.
> TEMPLATE_FILES is a list of files that should be processed in the
> `do_transform_template` task and TEMPLATE_VARS is a list of bitbake
> variable names that should be substituted in the template file.
>
> Signed-off-by: Claudius Heine <ch@denx.de>
> ---
> doc/technical_overview.md | 25 ++++++++
> meta/classes/base.bbclass | 1 +
> meta/classes/dpkg-base.bbclass | 2 +-
> meta/classes/template.bbclass | 62
> +++++++++++++++++++ .../debian/{changelog => changelog.tmpl} |
> 2 +- meta/recipes-bsp/u-boot/files/debian/control | 19 ------
> .../u-boot/files/debian/control.tmpl | 19 ++++++
> meta/recipes-bsp/u-boot/u-boot-custom.inc | 12 ++--
> .../debian/{changelog => changelog.tmpl} | 2 +-
> .../linux-module/files/debian/control | 11 ----
> .../linux-module/files/debian/control.tmpl | 11 ++++
> meta/recipes-kernel/linux-module/module.inc | 7 +--
> 12 files changed, 129 insertions(+), 44 deletions(-)
> create mode 100644 meta/classes/template.bbclass
> rename meta/recipes-bsp/u-boot/files/debian/{changelog =>
> changelog.tmpl} (74%) delete mode 100644
> meta/recipes-bsp/u-boot/files/debian/control create mode 100644
> meta/recipes-bsp/u-boot/files/debian/control.tmpl rename
> meta/recipes-kernel/linux-module/files/debian/{changelog =>
> changelog.tmpl} (74%) delete mode 100644
> meta/recipes-kernel/linux-module/files/debian/control create mode
> 100644 meta/recipes-kernel/linux-module/files/debian/control.tmpl
>
> diff --git a/doc/technical_overview.md b/doc/technical_overview.md
> index 7efbeee..0ddcabc 100644
> --- a/doc/technical_overview.md
> +++ b/doc/technical_overview.md
> @@ -283,3 +283,28 @@ cross-compilation, this is done by the same
> tools as for native compilation. Depending on ISAR_CROSS_COMPILE
> value, additional architecture specifiers are passed to build tools
> automatically, so this is absolutely transparent from the user point
> of view. +
> +## 3.9 Additional features
> +
> +### 3.9.1 Template files
> +
> +A basic templating system implemented on top `envsubst(1)` is
> available. It +allows limited access to bitbake variables for file
> generation. +
> +This system is implemented in the `template.bbclass` and defines the
> +`do_transform_template` task, the `TEMPLATE_FILES` and
> `TEMPLATE_VARS` +variables. Tasks that use need to use files
> generated via templates need +to be executed after the
> `do_transform_template` task. +
> +The `TEMPLATE_FILES` variable contains a space separated list of
> template +files and the `TEMPLATE_VARS` variable a space separated
> list of bitbake +variable names. The `do_transform_template` task
> takes these and generates +files using `envsubst(1)`. The output
> files are placed in the same directory +as the template files. The
> name of the output files are either the name of the +template file
> with `.tmpl` removed from the file name end or, if the template +file
> did not end with `.tmpl`, the name of the template file with `.out`
> added +to the end. +
> +Only template files from the `WORKDIR` are accepted. Either specify
> relative +paths based on the recipes `WORKDIR` or absolute paths
> containing the `WORKDIR` +in the `TEMPLATE_FILES` variable.
> diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
> index d4082de..5ab6fa4 100644
> --- a/meta/classes/base.bbclass
> +++ b/meta/classes/base.bbclass
> @@ -25,6 +25,7 @@ die() {
> }
>
> inherit logging
> +inherit template
>
> # Derived from bitbake: bitbake/classes/base.bbclass
> addtask showdata
> diff --git a/meta/classes/dpkg-base.bbclass
> b/meta/classes/dpkg-base.bbclass index f1b127c..fe96850 100644
> --- a/meta/classes/dpkg-base.bbclass
> +++ b/meta/classes/dpkg-base.bbclass
> @@ -40,7 +40,7 @@ do_prepare_build() {
> true
> }
>
> -addtask prepare_build after do_patch before do_build
> +addtask prepare_build after do_patch do_transform_template before
> do_build do_prepare_build[stamp-extra-info] =
> "${DISTRO}-${DISTRO_ARCH}" # If Isar recipes depend on each other,
> they typically need the package # deployed to isar-apt
> diff --git a/meta/classes/template.bbclass
> b/meta/classes/template.bbclass new file mode 100644
> index 0000000..716b7a5
> --- /dev/null
> +++ b/meta/classes/template.bbclass
> @@ -0,0 +1,62 @@
> +# This software is a part of ISAR.
> +# Copyright (C) Siemens AG, 2019
> +#
> +# SPDX-License-Identifier: MIT
> +
> +TEMPLATE_FILES ??= ""
> +TEMPLATE_VARS ??= "PN PV DESCRIPTION HOMEPAGE MAINTAINER KERNEL_NAME
> MACHINE \
> + DISTRO_ARCH"
> +
> +do_transform_template[vardeps] = "TEMPLATE_FILES ${TEMPLATE_VARS}"
> +do_transform_template[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
> +python do_transform_template() {
> + import subprocess, contextlib
> +
> + workdir = os.path.normpath(d.getVar('WORKDIR', True))
> +
> + template_vars = (d.getVar('TEMPLATE_VARS', True) or "").split()
> + if len(template_vars) == 0:
> + return
> +
> + template_files = (d.getVar('TEMPLATE_FILES', True) or "").split()
> + if len(template_files) == 0:
> + return
> +
> + cmd = "envsubst"
> + args = " ".join("\${{{}}}".format(i) for i in template_vars)
> +
> + # Copy current process environment and add template variables
> + # from bitbake data store:
> + env = os.environ.copy()
> + for varname in template_vars:
> + value = d.getVar(varname, True)
> + if value:
> + env.update({varname: value})
> +
> + for template_file in template_files:
> + # Normpath and workdir checks should prevent accidential or
> + # uninformed changes to files outside of the tmp and
> workdirectoy
> + template_file = os.path.normpath(template_file)
> +
> + # Convert relative paths to absolut paths based on the
> workdir:
> + if not os.path.isabs(template_file):
> + template_file = os.path.normpath(os.path.join(workdir,
> template_file)) +
> + if not template_file.startswith(workdir):
> + bb.fatal("Template file ({}) is not within workdir ({})"
> + .format(template_file, workdir))
> +
> + output_file = (os.path.splitext(template_file)[0]
> + if template_file.endswith(".tmpl")
> + else (template_file + ".out"))
> + bb.note("{} {} [in: {} out: {}]".format(cmd, args,
> + template_file,
> output_file))
> + with contextlib.ExitStack() as stack:
> + input = stack.enter_context(open(template_file, 'rb'))
> + output = stack.enter_context(open(output_file, 'wb'))
> + process = subprocess.Popen([cmd, args], stdin=input,
> + stdout=output, env=env)
> + if process.wait() != 0:
> + bb.fatal("processing of template failed")
> +}
> +addtask do_transform_template after do_unpack before do_build
> diff --git a/meta/recipes-bsp/u-boot/files/debian/changelog
> b/meta/recipes-bsp/u-boot/files/debian/changelog.tmpl similarity
> index 74% rename from meta/recipes-bsp/u-boot/files/debian/changelog
> rename to meta/recipes-bsp/u-boot/files/debian/changelog.tmpl
> index c1c3516..6e59e06 100644
> --- a/meta/recipes-bsp/u-boot/files/debian/changelog
> +++ b/meta/recipes-bsp/u-boot/files/debian/changelog.tmpl
> @@ -1,4 +1,4 @@
> -@PN@ (@PV@) unstable; urgency=low
> +${PN} (${PV}) unstable; urgency=low
>
> * Generated package.
>
> diff --git a/meta/recipes-bsp/u-boot/files/debian/control
> b/meta/recipes-bsp/u-boot/files/debian/control deleted file mode
> 100644 index 6b4c839..0000000
> --- a/meta/recipes-bsp/u-boot/files/debian/control
> +++ /dev/null
> @@ -1,19 +0,0 @@
> -Source: @PN@
> -Section: admin
> -Priority: optional
> -Standards-Version: 3.9.6
> -Build-Depends: @BUILD_DEPENDS@
> -Maintainer: ISAR project <isar-users@googlegroups.com>
> -
> -Package: u-boot-@MACHINE@
> -Architecture: @DISTRO_ARCH@
> -Description: @DESCRIPTION@, bootloader binaries
> -
> -Package: u-boot-@MACHINE@-dev
> -Architecture: @DISTRO_ARCH@
> -Description: @DESCRIPTION@, bootloader libraries
> -
> -Package: u-boot-tools
> -Architecture: linux-any
> -Depends: ${shlibs:Depends}, ${misc:Depends}
> -Description: @DESCRIPTION@, companion tools
> diff --git a/meta/recipes-bsp/u-boot/files/debian/control.tmpl
> b/meta/recipes-bsp/u-boot/files/debian/control.tmpl new file mode
> 100644 index 0000000..5c1cc92
> --- /dev/null
> +++ b/meta/recipes-bsp/u-boot/files/debian/control.tmpl
> @@ -0,0 +1,19 @@
> +Source: ${PN}
> +Section: admin
> +Priority: optional
> +Standards-Version: 3.9.6
> +Build-Depends: ${BUILD_DEPENDS}
> +Maintainer: ISAR project <isar-users@googlegroups.com>
> +
> +Package: u-boot-${MACHINE}
> +Architecture: ${DISTRO_ARCH}
> +Description: ${DESCRIPTION}, bootloader binaries
> +
> +Package: u-boot-${MACHINE}-dev
> +Architecture: ${DISTRO_ARCH}
> +Description: ${DESCRIPTION}, bootloader libraries
> +
> +Package: u-boot-tools
> +Architecture: linux-any
> +Depends: ${shlibs:Depends}, ${misc:Depends}
> +Description: ${DESCRIPTION}, companion tools
> diff --git a/meta/recipes-bsp/u-boot/u-boot-custom.inc
> b/meta/recipes-bsp/u-boot/u-boot-custom.inc index 4b38c88..c6aab43
> 100644 --- a/meta/recipes-bsp/u-boot/u-boot-custom.inc
> +++ b/meta/recipes-bsp/u-boot/u-boot-custom.inc
> @@ -15,16 +15,14 @@ inherit dpkg
>
> SRC_URI += "file://debian/"
>
> -U_BOOT_BUILD_DEPENDS ?= "bc, bison, flex, device-tree-compiler"
> +BUILD_DEPENDS ?= "bc, bison, flex, device-tree-compiler"
> +
> +TEMPLATE_FILES = "debian/changelog.tmpl \
> + debian/control.tmpl"
> +TEMPLATE_VARS += "BUILD_DEPENDS"
>
> do_prepare_build() {
> cp -r ${WORKDIR}/debian ${S}/
> - sed -i -e 's/@PN@/${PN}/g' -e 's/@PV@/${PV}/g' \
> - -e 's/@BUILD_DEPENDS@/${U_BOOT_BUILD_DEPENDS}/g' \
> - -e 's/@MACHINE@/${MACHINE}/g' \
> - -e 's/@DISTRO_ARCH@/${DISTRO_ARCH}/g' \
> - -e 's/@DESCRIPTION@/${DESCRIPTION}/g' \
> - ${S}/debian/changelog ${S}/debian/control
>
> echo "${U_BOOT_BIN} /usr/lib/u-boot/${MACHINE}" > \
> ${S}/debian/u-boot-${MACHINE}.install
> diff --git a/meta/recipes-kernel/linux-module/files/debian/changelog
> b/meta/recipes-kernel/linux-module/files/debian/changelog.tmpl
> similarity index 74% rename from
> meta/recipes-kernel/linux-module/files/debian/changelog rename to
> meta/recipes-kernel/linux-module/files/debian/changelog.tmpl index
> c1c3516..6e59e06 100644 ---
> a/meta/recipes-kernel/linux-module/files/debian/changelog +++
> b/meta/recipes-kernel/linux-module/files/debian/changelog.tmpl @@
> -1,4 +1,4 @@ -@PN@ (@PV@) unstable; urgency=low
> +${PN} (${PV}) unstable; urgency=low
>
> * Generated package.
>
> diff --git a/meta/recipes-kernel/linux-module/files/debian/control
> b/meta/recipes-kernel/linux-module/files/debian/control deleted file
> mode 100644 index 1ee634c..0000000
> --- a/meta/recipes-kernel/linux-module/files/debian/control
> +++ /dev/null
> @@ -1,11 +0,0 @@
> -Source: @PN@
> -Section: kernel
> -Priority: optional
> -Standards-Version: 3.9.6
> -Build-Depends: linux-headers-@KERNEL_NAME@
> -Maintainer: ISAR project <isar-users@googlegroups.com>
> -
> -Package: @PN@
> -Architecture: any
> -Depends: linux-image-@KERNEL_NAME@, kmod
> -Description: @DESCRIPTION@
> diff --git
> a/meta/recipes-kernel/linux-module/files/debian/control.tmpl
> b/meta/recipes-kernel/linux-module/files/debian/control.tmpl new file
> mode 100644 index 0000000..3b3292d --- /dev/null
> +++ b/meta/recipes-kernel/linux-module/files/debian/control.tmpl
> @@ -0,0 +1,11 @@
> +Source: ${PN}
> +Section: kernel
> +Priority: optional
> +Standards-Version: 3.9.6
> +Build-Depends: linux-headers-${KERNEL_NAME}
> +Maintainer: ISAR project <isar-users@googlegroups.com>
> +
> +Package: ${PN}
> +Architecture: any
> +Depends: linux-image-${KERNEL_NAME}, kmod
> +Description: ${DESCRIPTION}
> diff --git a/meta/recipes-kernel/linux-module/module.inc
> b/meta/recipes-kernel/linux-module/module.inc index cb7b8ad..44edd58
> 100644 --- a/meta/recipes-kernel/linux-module/module.inc
> +++ b/meta/recipes-kernel/linux-module/module.inc
> @@ -19,12 +19,11 @@ AUTOLOAD ?= ""
>
> inherit dpkg
>
> +TEMPLATE_FILES = "debian/control.tmpl \
> + debian/changelog.tmpl"
> +
> do_prepare_build() {
> cp -r ${WORKDIR}/debian ${S}/
> - sed -i -e 's/@PN@/${PN}/g' -e 's/@PV@/${PV}/g' \
> - -e 's/@KERNEL_NAME@/${KERNEL_NAME}/g' \
> - -e 's/@DESCRIPTION@/${DESCRIPTION}/g' \
> - ${S}/debian/changelog ${S}/debian/control
>
> for module in "${AUTOLOAD}"; do
> echo "echo $module >> /etc/modules" >> ${S}/debian/postinst
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH v2 1/1] meta: added do_transform_template task as templating system and switch
2019-02-05 11:18 ` [PATCH v2 1/1] meta: added do_transform_template task as templating system and switch claudius.heine.ext
2019-02-06 13:37 ` Henning Schild
@ 2019-02-12 10:27 ` Claudius Heine
1 sibling, 0 replies; 4+ messages in thread
From: Claudius Heine @ 2019-02-12 10:27 UTC (permalink / raw)
To: isar-users; +Cc: Claudius Heine
ping
On 05/02/2019 12.18, [ext] claudius.heine.ext@siemens.com wrote:
> From: Claudius Heine <ch@denx.de>
>
> The new introduced variables are TEMPLATE_FILES and TEMPLATE_VARS.
> TEMPLATE_FILES is a list of files that should be processed in the
> `do_transform_template` task and TEMPLATE_VARS is a list of bitbake
> variable names that should be substituted in the template file.
>
> Signed-off-by: Claudius Heine <ch@denx.de>
> ---
> doc/technical_overview.md | 25 ++++++++
> meta/classes/base.bbclass | 1 +
> meta/classes/dpkg-base.bbclass | 2 +-
> meta/classes/template.bbclass | 62 +++++++++++++++++++
> .../debian/{changelog => changelog.tmpl} | 2 +-
> meta/recipes-bsp/u-boot/files/debian/control | 19 ------
> .../u-boot/files/debian/control.tmpl | 19 ++++++
> meta/recipes-bsp/u-boot/u-boot-custom.inc | 12 ++--
> .../debian/{changelog => changelog.tmpl} | 2 +-
> .../linux-module/files/debian/control | 11 ----
> .../linux-module/files/debian/control.tmpl | 11 ++++
> meta/recipes-kernel/linux-module/module.inc | 7 +--
> 12 files changed, 129 insertions(+), 44 deletions(-)
> create mode 100644 meta/classes/template.bbclass
> rename meta/recipes-bsp/u-boot/files/debian/{changelog => changelog.tmpl} (74%)
> delete mode 100644 meta/recipes-bsp/u-boot/files/debian/control
> create mode 100644 meta/recipes-bsp/u-boot/files/debian/control.tmpl
> rename meta/recipes-kernel/linux-module/files/debian/{changelog => changelog.tmpl} (74%)
> delete mode 100644 meta/recipes-kernel/linux-module/files/debian/control
> create mode 100644 meta/recipes-kernel/linux-module/files/debian/control.tmpl
>
> diff --git a/doc/technical_overview.md b/doc/technical_overview.md
> index 7efbeee..0ddcabc 100644
> --- a/doc/technical_overview.md
> +++ b/doc/technical_overview.md
> @@ -283,3 +283,28 @@ cross-compilation, this is done by the same tools as for native compilation.
> Depending on ISAR_CROSS_COMPILE value, additional architecture specifiers are
> passed to build tools automatically, so this is absolutely transparent from
> the user point of view.
> +
> +## 3.9 Additional features
> +
> +### 3.9.1 Template files
> +
> +A basic templating system implemented on top `envsubst(1)` is available. It
> +allows limited access to bitbake variables for file generation.
> +
> +This system is implemented in the `template.bbclass` and defines the
> +`do_transform_template` task, the `TEMPLATE_FILES` and `TEMPLATE_VARS`
> +variables. Tasks that use need to use files generated via templates need
> +to be executed after the `do_transform_template` task.
> +
> +The `TEMPLATE_FILES` variable contains a space separated list of template
> +files and the `TEMPLATE_VARS` variable a space separated list of bitbake
> +variable names. The `do_transform_template` task takes these and generates
> +files using `envsubst(1)`. The output files are placed in the same directory
> +as the template files. The name of the output files are either the name of the
> +template file with `.tmpl` removed from the file name end or, if the template
> +file did not end with `.tmpl`, the name of the template file with `.out` added
> +to the end.
> +
> +Only template files from the `WORKDIR` are accepted. Either specify relative
> +paths based on the recipes `WORKDIR` or absolute paths containing the `WORKDIR`
> +in the `TEMPLATE_FILES` variable.
> diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
> index d4082de..5ab6fa4 100644
> --- a/meta/classes/base.bbclass
> +++ b/meta/classes/base.bbclass
> @@ -25,6 +25,7 @@ die() {
> }
>
> inherit logging
> +inherit template
>
> # Derived from bitbake: bitbake/classes/base.bbclass
> addtask showdata
> diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass
> index f1b127c..fe96850 100644
> --- a/meta/classes/dpkg-base.bbclass
> +++ b/meta/classes/dpkg-base.bbclass
> @@ -40,7 +40,7 @@ do_prepare_build() {
> true
> }
>
> -addtask prepare_build after do_patch before do_build
> +addtask prepare_build after do_patch do_transform_template before do_build
> do_prepare_build[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
> # If Isar recipes depend on each other, they typically need the package
> # deployed to isar-apt
> diff --git a/meta/classes/template.bbclass b/meta/classes/template.bbclass
> new file mode 100644
> index 0000000..716b7a5
> --- /dev/null
> +++ b/meta/classes/template.bbclass
> @@ -0,0 +1,62 @@
> +# This software is a part of ISAR.
> +# Copyright (C) Siemens AG, 2019
> +#
> +# SPDX-License-Identifier: MIT
> +
> +TEMPLATE_FILES ??= ""
> +TEMPLATE_VARS ??= "PN PV DESCRIPTION HOMEPAGE MAINTAINER KERNEL_NAME MACHINE \
> + DISTRO_ARCH"
> +
> +do_transform_template[vardeps] = "TEMPLATE_FILES ${TEMPLATE_VARS}"
> +do_transform_template[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
> +python do_transform_template() {
> + import subprocess, contextlib
> +
> + workdir = os.path.normpath(d.getVar('WORKDIR', True))
> +
> + template_vars = (d.getVar('TEMPLATE_VARS', True) or "").split()
> + if len(template_vars) == 0:
> + return
> +
> + template_files = (d.getVar('TEMPLATE_FILES', True) or "").split()
> + if len(template_files) == 0:
> + return
> +
> + cmd = "envsubst"
> + args = " ".join("\${{{}}}".format(i) for i in template_vars)
> +
> + # Copy current process environment and add template variables
> + # from bitbake data store:
> + env = os.environ.copy()
> + for varname in template_vars:
> + value = d.getVar(varname, True)
> + if value:
> + env.update({varname: value})
> +
> + for template_file in template_files:
> + # Normpath and workdir checks should prevent accidential or
> + # uninformed changes to files outside of the tmp and workdirectoy
> + template_file = os.path.normpath(template_file)
> +
> + # Convert relative paths to absolut paths based on the workdir:
> + if not os.path.isabs(template_file):
> + template_file = os.path.normpath(os.path.join(workdir, template_file))
> +
> + if not template_file.startswith(workdir):
> + bb.fatal("Template file ({}) is not within workdir ({})"
> + .format(template_file, workdir))
> +
> + output_file = (os.path.splitext(template_file)[0]
> + if template_file.endswith(".tmpl")
> + else (template_file + ".out"))
> + bb.note("{} {} [in: {} out: {}]".format(cmd, args,
> + template_file, output_file))
> + with contextlib.ExitStack() as stack:
> + input = stack.enter_context(open(template_file, 'rb'))
> + output = stack.enter_context(open(output_file, 'wb'))
> + process = subprocess.Popen([cmd, args], stdin=input,
> + stdout=output, env=env)
> + if process.wait() != 0:
> + bb.fatal("processing of template failed")
> +}
> +addtask do_transform_template after do_unpack before do_build
> diff --git a/meta/recipes-bsp/u-boot/files/debian/changelog b/meta/recipes-bsp/u-boot/files/debian/changelog.tmpl
> similarity index 74%
> rename from meta/recipes-bsp/u-boot/files/debian/changelog
> rename to meta/recipes-bsp/u-boot/files/debian/changelog.tmpl
> index c1c3516..6e59e06 100644
> --- a/meta/recipes-bsp/u-boot/files/debian/changelog
> +++ b/meta/recipes-bsp/u-boot/files/debian/changelog.tmpl
> @@ -1,4 +1,4 @@
> -@PN@ (@PV@) unstable; urgency=low
> +${PN} (${PV}) unstable; urgency=low
>
> * Generated package.
>
> diff --git a/meta/recipes-bsp/u-boot/files/debian/control b/meta/recipes-bsp/u-boot/files/debian/control
> deleted file mode 100644
> index 6b4c839..0000000
> --- a/meta/recipes-bsp/u-boot/files/debian/control
> +++ /dev/null
> @@ -1,19 +0,0 @@
> -Source: @PN@
> -Section: admin
> -Priority: optional
> -Standards-Version: 3.9.6
> -Build-Depends: @BUILD_DEPENDS@
> -Maintainer: ISAR project <isar-users@googlegroups.com>
> -
> -Package: u-boot-@MACHINE@
> -Architecture: @DISTRO_ARCH@
> -Description: @DESCRIPTION@, bootloader binaries
> -
> -Package: u-boot-@MACHINE@-dev
> -Architecture: @DISTRO_ARCH@
> -Description: @DESCRIPTION@, bootloader libraries
> -
> -Package: u-boot-tools
> -Architecture: linux-any
> -Depends: ${shlibs:Depends}, ${misc:Depends}
> -Description: @DESCRIPTION@, companion tools
> diff --git a/meta/recipes-bsp/u-boot/files/debian/control.tmpl b/meta/recipes-bsp/u-boot/files/debian/control.tmpl
> new file mode 100644
> index 0000000..5c1cc92
> --- /dev/null
> +++ b/meta/recipes-bsp/u-boot/files/debian/control.tmpl
> @@ -0,0 +1,19 @@
> +Source: ${PN}
> +Section: admin
> +Priority: optional
> +Standards-Version: 3.9.6
> +Build-Depends: ${BUILD_DEPENDS}
> +Maintainer: ISAR project <isar-users@googlegroups.com>
> +
> +Package: u-boot-${MACHINE}
> +Architecture: ${DISTRO_ARCH}
> +Description: ${DESCRIPTION}, bootloader binaries
> +
> +Package: u-boot-${MACHINE}-dev
> +Architecture: ${DISTRO_ARCH}
> +Description: ${DESCRIPTION}, bootloader libraries
> +
> +Package: u-boot-tools
> +Architecture: linux-any
> +Depends: ${shlibs:Depends}, ${misc:Depends}
> +Description: ${DESCRIPTION}, companion tools
> diff --git a/meta/recipes-bsp/u-boot/u-boot-custom.inc b/meta/recipes-bsp/u-boot/u-boot-custom.inc
> index 4b38c88..c6aab43 100644
> --- a/meta/recipes-bsp/u-boot/u-boot-custom.inc
> +++ b/meta/recipes-bsp/u-boot/u-boot-custom.inc
> @@ -15,16 +15,14 @@ inherit dpkg
>
> SRC_URI += "file://debian/"
>
> -U_BOOT_BUILD_DEPENDS ?= "bc, bison, flex, device-tree-compiler"
> +BUILD_DEPENDS ?= "bc, bison, flex, device-tree-compiler"
> +
> +TEMPLATE_FILES = "debian/changelog.tmpl \
> + debian/control.tmpl"
> +TEMPLATE_VARS += "BUILD_DEPENDS"
>
> do_prepare_build() {
> cp -r ${WORKDIR}/debian ${S}/
> - sed -i -e 's/@PN@/${PN}/g' -e 's/@PV@/${PV}/g' \
> - -e 's/@BUILD_DEPENDS@/${U_BOOT_BUILD_DEPENDS}/g' \
> - -e 's/@MACHINE@/${MACHINE}/g' \
> - -e 's/@DISTRO_ARCH@/${DISTRO_ARCH}/g' \
> - -e 's/@DESCRIPTION@/${DESCRIPTION}/g' \
> - ${S}/debian/changelog ${S}/debian/control
>
> echo "${U_BOOT_BIN} /usr/lib/u-boot/${MACHINE}" > \
> ${S}/debian/u-boot-${MACHINE}.install
> diff --git a/meta/recipes-kernel/linux-module/files/debian/changelog b/meta/recipes-kernel/linux-module/files/debian/changelog.tmpl
> similarity index 74%
> rename from meta/recipes-kernel/linux-module/files/debian/changelog
> rename to meta/recipes-kernel/linux-module/files/debian/changelog.tmpl
> index c1c3516..6e59e06 100644
> --- a/meta/recipes-kernel/linux-module/files/debian/changelog
> +++ b/meta/recipes-kernel/linux-module/files/debian/changelog.tmpl
> @@ -1,4 +1,4 @@
> -@PN@ (@PV@) unstable; urgency=low
> +${PN} (${PV}) unstable; urgency=low
>
> * Generated package.
>
> diff --git a/meta/recipes-kernel/linux-module/files/debian/control b/meta/recipes-kernel/linux-module/files/debian/control
> deleted file mode 100644
> index 1ee634c..0000000
> --- a/meta/recipes-kernel/linux-module/files/debian/control
> +++ /dev/null
> @@ -1,11 +0,0 @@
> -Source: @PN@
> -Section: kernel
> -Priority: optional
> -Standards-Version: 3.9.6
> -Build-Depends: linux-headers-@KERNEL_NAME@
> -Maintainer: ISAR project <isar-users@googlegroups.com>
> -
> -Package: @PN@
> -Architecture: any
> -Depends: linux-image-@KERNEL_NAME@, kmod
> -Description: @DESCRIPTION@
> diff --git a/meta/recipes-kernel/linux-module/files/debian/control.tmpl b/meta/recipes-kernel/linux-module/files/debian/control.tmpl
> new file mode 100644
> index 0000000..3b3292d
> --- /dev/null
> +++ b/meta/recipes-kernel/linux-module/files/debian/control.tmpl
> @@ -0,0 +1,11 @@
> +Source: ${PN}
> +Section: kernel
> +Priority: optional
> +Standards-Version: 3.9.6
> +Build-Depends: linux-headers-${KERNEL_NAME}
> +Maintainer: ISAR project <isar-users@googlegroups.com>
> +
> +Package: ${PN}
> +Architecture: any
> +Depends: linux-image-${KERNEL_NAME}, kmod
> +Description: ${DESCRIPTION}
> diff --git a/meta/recipes-kernel/linux-module/module.inc b/meta/recipes-kernel/linux-module/module.inc
> index cb7b8ad..44edd58 100644
> --- a/meta/recipes-kernel/linux-module/module.inc
> +++ b/meta/recipes-kernel/linux-module/module.inc
> @@ -19,12 +19,11 @@ AUTOLOAD ?= ""
>
> inherit dpkg
>
> +TEMPLATE_FILES = "debian/control.tmpl \
> + debian/changelog.tmpl"
> +
> do_prepare_build() {
> cp -r ${WORKDIR}/debian ${S}/
> - sed -i -e 's/@PN@/${PN}/g' -e 's/@PV@/${PV}/g' \
> - -e 's/@KERNEL_NAME@/${KERNEL_NAME}/g' \
> - -e 's/@DESCRIPTION@/${DESCRIPTION}/g' \
> - ${S}/debian/changelog ${S}/debian/control
>
> for module in "${AUTOLOAD}"; do
> echo "echo $module >> /etc/modules" >> ${S}/debian/postinst
>
--
DENX Software Engineering GmbH, Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2019-02-12 10:27 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-02-05 11:18 [PATCH v2 0/1] Template system claudius.heine.ext
2019-02-05 11:18 ` [PATCH v2 1/1] meta: added do_transform_template task as templating system and switch claudius.heine.ext
2019-02-06 13:37 ` Henning Schild
2019-02-12 10:27 ` Claudius Heine
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox