From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6652594697292218368 X-Received: by 2002:a1c:2dd0:: with SMTP id t199mr272982wmt.21.1549460223787; Wed, 06 Feb 2019 05:37:03 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:adf:ce8b:: with SMTP id r11ls4816700wrn.5.gmail; Wed, 06 Feb 2019 05:37:03 -0800 (PST) X-Google-Smtp-Source: AHgI3IZUj7osfvBinIY006PSiREY6VIKDLYkKiH9lNk+J6rwsUmIwrPpf6CWjycQ61dKsCS8RGoS X-Received: by 2002:adf:f203:: with SMTP id p3mr574446wro.15.1549460223327; Wed, 06 Feb 2019 05:37:03 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1549460223; cv=none; d=google.com; s=arc-20160816; b=D7R1F9jz3/R0euYcJn6st8PdPS4NSHsRwi8Q0TuH4o5Mg61m9rYp7M3veMvxRJSDrh IKZe5vAjCKCZJG+j3EF8+wy+xpwgXxA3p21whYZhqqZfnFudgALE0CdVhG4yW5L7PUUi 0RCIp/xOPjGPUVTRB3WaXWSOhYJSR74Al+sdPiqg7WZifebbDYBS/5Ytjuzo5bJzNx7p W7vES/g4jmCAa1ASgULPNazJMZsoAhq545bht/zmoEPAXwEC1x5sIfnT+DsPNJFLEUTK Gz9rVdi5nr1bSNJwguxx9m2vPKzrNm1HPEow+QaVgcJ39VFP9N6zwPf+CwTe7ONNbYCl iZJw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date; bh=8U7Yvc/Xxzj5CNyx4KHTVbWbZw4Kp0/QJSfrf+UhoFE=; b=ZQERder/iNnGgbEyK9wXL2jFI2A0ZB0jllep/vnwZDEGPtcCdG8L6I/KQDpZk1orPk cCEHXWyfalTRKxU3c3QcjfENi2VADaf7tHfWSPORTkuX6UqP2rlx2GRWecA9mZFq5FqL vCD1OGz+Cwh7UckQqGCaR+qTYY5OtlCPJx7OV3nN8qIm/Sm1gpDHFPM90fzs/e168/Fl fbKswttrg59vT6I/6MjApNavhDs3RJiDnuET/0xX9yLqZSj7SJDfkOebvG0j3oh0EkS3 3w7LCAykQep2cfZ0DihIV0yo9MR2zMAwcd4VX0XIKG+3JUHGF/8E+DX5LjB5S8uKC7aD g6NA== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.2 as permitted sender) smtp.mailfrom=henning.schild@siemens.com Return-Path: Received: from thoth.sbs.de (thoth.sbs.de. [192.35.17.2]) by gmr-mx.google.com with ESMTPS id j9si113027wme.2.2019.02.06.05.37.03 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 06 Feb 2019 05:37:03 -0800 (PST) Received-SPF: pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.2 as permitted sender) client-ip=192.35.17.2; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.2 as permitted sender) smtp.mailfrom=henning.schild@siemens.com Received: from mail1.sbs.de (mail1.sbs.de [192.129.41.35]) by thoth.sbs.de (8.15.2/8.15.2) with ESMTPS id x16Db2gF015779 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 6 Feb 2019 14:37:02 +0100 Received: from md1za8fc.ad001.siemens.net ([139.25.69.48]) by mail1.sbs.de (8.15.2/8.15.2) with ESMTP id x16Db2Di014407; Wed, 6 Feb 2019 14:37:02 +0100 Date: Wed, 6 Feb 2019 14:37:01 +0100 From: Henning Schild To: "[ext] claudius.heine.ext@siemens.com" Cc: , Claudius Heine Subject: Re: [PATCH v2 1/1] meta: added do_transform_template task as templating system and switch Message-ID: <20190206143701.5433bd34@md1za8fc.ad001.siemens.net> In-Reply-To: <20190205111801.14362-2-claudius.heine.ext@siemens.com> References: <20190205111801.14362-1-claudius.heine.ext@siemens.com> <20190205111801.14362-2-claudius.heine.ext@siemens.com> X-Mailer: Claws Mail 3.15.0-dirty (GTK+ 2.24.32; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-TUID: OQE/KZv5GcQ0 Looks good and useful to me. I did not try it but since the example-module is a user of it we do have CI coverage. Henning Am Tue, 5 Feb 2019 12:18:01 +0100 schrieb "[ext] claudius.heine.ext@siemens.com" : > From: Claudius Heine > > The new introduced variables are TEMPLATE_FILES and TEMPLATE_VARS. > TEMPLATE_FILES is a list of files that should be processed in the > `do_transform_template` task and TEMPLATE_VARS is a list of bitbake > variable names that should be substituted in the template file. > > Signed-off-by: Claudius Heine > --- > doc/technical_overview.md | 25 ++++++++ > meta/classes/base.bbclass | 1 + > meta/classes/dpkg-base.bbclass | 2 +- > meta/classes/template.bbclass | 62 > +++++++++++++++++++ .../debian/{changelog => changelog.tmpl} | > 2 +- meta/recipes-bsp/u-boot/files/debian/control | 19 ------ > .../u-boot/files/debian/control.tmpl | 19 ++++++ > meta/recipes-bsp/u-boot/u-boot-custom.inc | 12 ++-- > .../debian/{changelog => changelog.tmpl} | 2 +- > .../linux-module/files/debian/control | 11 ---- > .../linux-module/files/debian/control.tmpl | 11 ++++ > meta/recipes-kernel/linux-module/module.inc | 7 +-- > 12 files changed, 129 insertions(+), 44 deletions(-) > create mode 100644 meta/classes/template.bbclass > rename meta/recipes-bsp/u-boot/files/debian/{changelog => > changelog.tmpl} (74%) delete mode 100644 > meta/recipes-bsp/u-boot/files/debian/control create mode 100644 > meta/recipes-bsp/u-boot/files/debian/control.tmpl rename > meta/recipes-kernel/linux-module/files/debian/{changelog => > changelog.tmpl} (74%) delete mode 100644 > meta/recipes-kernel/linux-module/files/debian/control create mode > 100644 meta/recipes-kernel/linux-module/files/debian/control.tmpl > > diff --git a/doc/technical_overview.md b/doc/technical_overview.md > index 7efbeee..0ddcabc 100644 > --- a/doc/technical_overview.md > +++ b/doc/technical_overview.md > @@ -283,3 +283,28 @@ cross-compilation, this is done by the same > tools as for native compilation. Depending on ISAR_CROSS_COMPILE > value, additional architecture specifiers are passed to build tools > automatically, so this is absolutely transparent from the user point > of view. + > +## 3.9 Additional features > + > +### 3.9.1 Template files > + > +A basic templating system implemented on top `envsubst(1)` is > available. It +allows limited access to bitbake variables for file > generation. + > +This system is implemented in the `template.bbclass` and defines the > +`do_transform_template` task, the `TEMPLATE_FILES` and > `TEMPLATE_VARS` +variables. Tasks that use need to use files > generated via templates need +to be executed after the > `do_transform_template` task. + > +The `TEMPLATE_FILES` variable contains a space separated list of > template +files and the `TEMPLATE_VARS` variable a space separated > list of bitbake +variable names. The `do_transform_template` task > takes these and generates +files using `envsubst(1)`. The output > files are placed in the same directory +as the template files. The > name of the output files are either the name of the +template file > with `.tmpl` removed from the file name end or, if the template +file > did not end with `.tmpl`, the name of the template file with `.out` > added +to the end. + > +Only template files from the `WORKDIR` are accepted. Either specify > relative +paths based on the recipes `WORKDIR` or absolute paths > containing the `WORKDIR` +in the `TEMPLATE_FILES` variable. > diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass > index d4082de..5ab6fa4 100644 > --- a/meta/classes/base.bbclass > +++ b/meta/classes/base.bbclass > @@ -25,6 +25,7 @@ die() { > } > > inherit logging > +inherit template > > # Derived from bitbake: bitbake/classes/base.bbclass > addtask showdata > diff --git a/meta/classes/dpkg-base.bbclass > b/meta/classes/dpkg-base.bbclass index f1b127c..fe96850 100644 > --- a/meta/classes/dpkg-base.bbclass > +++ b/meta/classes/dpkg-base.bbclass > @@ -40,7 +40,7 @@ do_prepare_build() { > true > } > > -addtask prepare_build after do_patch before do_build > +addtask prepare_build after do_patch do_transform_template before > do_build do_prepare_build[stamp-extra-info] = > "${DISTRO}-${DISTRO_ARCH}" # If Isar recipes depend on each other, > they typically need the package # deployed to isar-apt > diff --git a/meta/classes/template.bbclass > b/meta/classes/template.bbclass new file mode 100644 > index 0000000..716b7a5 > --- /dev/null > +++ b/meta/classes/template.bbclass > @@ -0,0 +1,62 @@ > +# This software is a part of ISAR. > +# Copyright (C) Siemens AG, 2019 > +# > +# SPDX-License-Identifier: MIT > + > +TEMPLATE_FILES ??= "" > +TEMPLATE_VARS ??= "PN PV DESCRIPTION HOMEPAGE MAINTAINER KERNEL_NAME > MACHINE \ > + DISTRO_ARCH" > + > +do_transform_template[vardeps] = "TEMPLATE_FILES ${TEMPLATE_VARS}" > +do_transform_template[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" > +python do_transform_template() { > + import subprocess, contextlib > + > + workdir = os.path.normpath(d.getVar('WORKDIR', True)) > + > + template_vars = (d.getVar('TEMPLATE_VARS', True) or "").split() > + if len(template_vars) == 0: > + return > + > + template_files = (d.getVar('TEMPLATE_FILES', True) or "").split() > + if len(template_files) == 0: > + return > + > + cmd = "envsubst" > + args = " ".join("\${{{}}}".format(i) for i in template_vars) > + > + # Copy current process environment and add template variables > + # from bitbake data store: > + env = os.environ.copy() > + for varname in template_vars: > + value = d.getVar(varname, True) > + if value: > + env.update({varname: value}) > + > + for template_file in template_files: > + # Normpath and workdir checks should prevent accidential or > + # uninformed changes to files outside of the tmp and > workdirectoy > + template_file = os.path.normpath(template_file) > + > + # Convert relative paths to absolut paths based on the > workdir: > + if not os.path.isabs(template_file): > + template_file = os.path.normpath(os.path.join(workdir, > template_file)) + > + if not template_file.startswith(workdir): > + bb.fatal("Template file ({}) is not within workdir ({})" > + .format(template_file, workdir)) > + > + output_file = (os.path.splitext(template_file)[0] > + if template_file.endswith(".tmpl") > + else (template_file + ".out")) > + bb.note("{} {} [in: {} out: {}]".format(cmd, args, > + template_file, > output_file)) > + with contextlib.ExitStack() as stack: > + input = stack.enter_context(open(template_file, 'rb')) > + output = stack.enter_context(open(output_file, 'wb')) > + process = subprocess.Popen([cmd, args], stdin=input, > + stdout=output, env=env) > + if process.wait() != 0: > + bb.fatal("processing of template failed") > +} > +addtask do_transform_template after do_unpack before do_build > diff --git a/meta/recipes-bsp/u-boot/files/debian/changelog > b/meta/recipes-bsp/u-boot/files/debian/changelog.tmpl similarity > index 74% rename from meta/recipes-bsp/u-boot/files/debian/changelog > rename to meta/recipes-bsp/u-boot/files/debian/changelog.tmpl > index c1c3516..6e59e06 100644 > --- a/meta/recipes-bsp/u-boot/files/debian/changelog > +++ b/meta/recipes-bsp/u-boot/files/debian/changelog.tmpl > @@ -1,4 +1,4 @@ > -@PN@ (@PV@) unstable; urgency=low > +${PN} (${PV}) unstable; urgency=low > > * Generated package. > > diff --git a/meta/recipes-bsp/u-boot/files/debian/control > b/meta/recipes-bsp/u-boot/files/debian/control deleted file mode > 100644 index 6b4c839..0000000 > --- a/meta/recipes-bsp/u-boot/files/debian/control > +++ /dev/null > @@ -1,19 +0,0 @@ > -Source: @PN@ > -Section: admin > -Priority: optional > -Standards-Version: 3.9.6 > -Build-Depends: @BUILD_DEPENDS@ > -Maintainer: ISAR project > - > -Package: u-boot-@MACHINE@ > -Architecture: @DISTRO_ARCH@ > -Description: @DESCRIPTION@, bootloader binaries > - > -Package: u-boot-@MACHINE@-dev > -Architecture: @DISTRO_ARCH@ > -Description: @DESCRIPTION@, bootloader libraries > - > -Package: u-boot-tools > -Architecture: linux-any > -Depends: ${shlibs:Depends}, ${misc:Depends} > -Description: @DESCRIPTION@, companion tools > diff --git a/meta/recipes-bsp/u-boot/files/debian/control.tmpl > b/meta/recipes-bsp/u-boot/files/debian/control.tmpl new file mode > 100644 index 0000000..5c1cc92 > --- /dev/null > +++ b/meta/recipes-bsp/u-boot/files/debian/control.tmpl > @@ -0,0 +1,19 @@ > +Source: ${PN} > +Section: admin > +Priority: optional > +Standards-Version: 3.9.6 > +Build-Depends: ${BUILD_DEPENDS} > +Maintainer: ISAR project > + > +Package: u-boot-${MACHINE} > +Architecture: ${DISTRO_ARCH} > +Description: ${DESCRIPTION}, bootloader binaries > + > +Package: u-boot-${MACHINE}-dev > +Architecture: ${DISTRO_ARCH} > +Description: ${DESCRIPTION}, bootloader libraries > + > +Package: u-boot-tools > +Architecture: linux-any > +Depends: ${shlibs:Depends}, ${misc:Depends} > +Description: ${DESCRIPTION}, companion tools > diff --git a/meta/recipes-bsp/u-boot/u-boot-custom.inc > b/meta/recipes-bsp/u-boot/u-boot-custom.inc index 4b38c88..c6aab43 > 100644 --- a/meta/recipes-bsp/u-boot/u-boot-custom.inc > +++ b/meta/recipes-bsp/u-boot/u-boot-custom.inc > @@ -15,16 +15,14 @@ inherit dpkg > > SRC_URI += "file://debian/" > > -U_BOOT_BUILD_DEPENDS ?= "bc, bison, flex, device-tree-compiler" > +BUILD_DEPENDS ?= "bc, bison, flex, device-tree-compiler" > + > +TEMPLATE_FILES = "debian/changelog.tmpl \ > + debian/control.tmpl" > +TEMPLATE_VARS += "BUILD_DEPENDS" > > do_prepare_build() { > cp -r ${WORKDIR}/debian ${S}/ > - sed -i -e 's/@PN@/${PN}/g' -e 's/@PV@/${PV}/g' \ > - -e 's/@BUILD_DEPENDS@/${U_BOOT_BUILD_DEPENDS}/g' \ > - -e 's/@MACHINE@/${MACHINE}/g' \ > - -e 's/@DISTRO_ARCH@/${DISTRO_ARCH}/g' \ > - -e 's/@DESCRIPTION@/${DESCRIPTION}/g' \ > - ${S}/debian/changelog ${S}/debian/control > > echo "${U_BOOT_BIN} /usr/lib/u-boot/${MACHINE}" > \ > ${S}/debian/u-boot-${MACHINE}.install > diff --git a/meta/recipes-kernel/linux-module/files/debian/changelog > b/meta/recipes-kernel/linux-module/files/debian/changelog.tmpl > similarity index 74% rename from > meta/recipes-kernel/linux-module/files/debian/changelog rename to > meta/recipes-kernel/linux-module/files/debian/changelog.tmpl index > c1c3516..6e59e06 100644 --- > a/meta/recipes-kernel/linux-module/files/debian/changelog +++ > b/meta/recipes-kernel/linux-module/files/debian/changelog.tmpl @@ > -1,4 +1,4 @@ -@PN@ (@PV@) unstable; urgency=low > +${PN} (${PV}) unstable; urgency=low > > * Generated package. > > diff --git a/meta/recipes-kernel/linux-module/files/debian/control > b/meta/recipes-kernel/linux-module/files/debian/control deleted file > mode 100644 index 1ee634c..0000000 > --- a/meta/recipes-kernel/linux-module/files/debian/control > +++ /dev/null > @@ -1,11 +0,0 @@ > -Source: @PN@ > -Section: kernel > -Priority: optional > -Standards-Version: 3.9.6 > -Build-Depends: linux-headers-@KERNEL_NAME@ > -Maintainer: ISAR project > - > -Package: @PN@ > -Architecture: any > -Depends: linux-image-@KERNEL_NAME@, kmod > -Description: @DESCRIPTION@ > diff --git > a/meta/recipes-kernel/linux-module/files/debian/control.tmpl > b/meta/recipes-kernel/linux-module/files/debian/control.tmpl new file > mode 100644 index 0000000..3b3292d --- /dev/null > +++ b/meta/recipes-kernel/linux-module/files/debian/control.tmpl > @@ -0,0 +1,11 @@ > +Source: ${PN} > +Section: kernel > +Priority: optional > +Standards-Version: 3.9.6 > +Build-Depends: linux-headers-${KERNEL_NAME} > +Maintainer: ISAR project > + > +Package: ${PN} > +Architecture: any > +Depends: linux-image-${KERNEL_NAME}, kmod > +Description: ${DESCRIPTION} > diff --git a/meta/recipes-kernel/linux-module/module.inc > b/meta/recipes-kernel/linux-module/module.inc index cb7b8ad..44edd58 > 100644 --- a/meta/recipes-kernel/linux-module/module.inc > +++ b/meta/recipes-kernel/linux-module/module.inc > @@ -19,12 +19,11 @@ AUTOLOAD ?= "" > > inherit dpkg > > +TEMPLATE_FILES = "debian/control.tmpl \ > + debian/changelog.tmpl" > + > do_prepare_build() { > cp -r ${WORKDIR}/debian ${S}/ > - sed -i -e 's/@PN@/${PN}/g' -e 's/@PV@/${PV}/g' \ > - -e 's/@KERNEL_NAME@/${KERNEL_NAME}/g' \ > - -e 's/@DESCRIPTION@/${DESCRIPTION}/g' \ > - ${S}/debian/changelog ${S}/debian/control > > for module in "${AUTOLOAD}"; do > echo "echo $module >> /etc/modules" >> ${S}/debian/postinst