From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6652594697292218368 X-Received: by 2002:a2e:165c:: with SMTP id 28-v6mr186654ljw.25.1549967253466; Tue, 12 Feb 2019 02:27:33 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a2e:8551:: with SMTP id u17-v6ls1911020ljj.3.gmail; Tue, 12 Feb 2019 02:27:32 -0800 (PST) X-Google-Smtp-Source: AHgI3IZo6tq/H/VdRw/PBK+5ulVJyp0UDLBIaEE+H8gTXsvDBKKhqcJoeYHOR2+K950Exj4QNrB4 X-Received: by 2002:a2e:91c4:: with SMTP id u4mr63601ljg.29.1549967252911; Tue, 12 Feb 2019 02:27:32 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1549967252; cv=none; d=google.com; s=arc-20160816; b=nqnx8Rur39boC64F5WJr+Irw2z1XHGHVK+fois/7ns3+kiWnhfgcUPW7EElGEAzqjg zDudBA7mEF8B0W8fzhbnfFT1u74AemIaPpNkxnqHKpXELtvU0/6upNAUiCnn89VAVwdF AirL6a0aiqKRTpkIu5lenPp1zbJ5ZLGNT1vMNm5NFJ+kineFGE4Ven8Rt5PufDNlFaaf xJkPgEomp36Z6QiO1GANeTLtCGsGrkzcvVSQXIvu1wMOWsdtCp6NP6F7HAAV2QxesXPy Wjq5UWK/m5Lc1F8IfvouhkuoYQzMJ0fCuPYgCiGHHJNImXmGT0ueolQ0Jm5qU0fR9dWq ED/Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:references:cc:to:from:subject; bh=mLVcpvl2x7V+Aff3zAesWAf70Bqe42xuz5E532hNfq4=; b=mTQHhGYyvOzPU4fkoRPOlerjvAQeiphPj0WcVn60Qn+WerNAidxeky4i9W5+qwF8kA /6p5yOZj9f6ZSdifiqCLvGIFFFt2oqUc9pKPNb5RUbX4QYHx5P8arJ7vic6W4RHlf+ex wnMBU5GpT8gpJEZ0C67ZWK2PErAyTQ20oUoEFxn1G3raMdQtYKopw9NrYVAuLi+hi4Hl iZaO8QMcf3PQr0Eu632hcueb6P93I0zqVKRgn0+FKtapCfiyqYU/pOtxm3tXR8a4YpXt bZhylzZGl3Bi/r5grlc1tClUEuqewgGt3G10VfJNiMI5moSr0usKwiufQCjsvfh8fZOa vF7g== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of claudius.heine.ext@siemens.com designates 192.35.17.14 as permitted sender) smtp.mailfrom=claudius.heine.ext@siemens.com Return-Path: Received: from david.siemens.de (david.siemens.de. [192.35.17.14]) by gmr-mx.google.com with ESMTPS id h17-v6si772109ljj.0.2019.02.12.02.27.32 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 12 Feb 2019 02:27:32 -0800 (PST) Received-SPF: pass (google.com: domain of claudius.heine.ext@siemens.com designates 192.35.17.14 as permitted sender) client-ip=192.35.17.14; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of claudius.heine.ext@siemens.com designates 192.35.17.14 as permitted sender) smtp.mailfrom=claudius.heine.ext@siemens.com Received: from mail1.sbs.de (mail1.sbs.de [192.129.41.35]) by david.siemens.de (8.15.2/8.15.2) with ESMTPS id x1CARVQx018025 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 12 Feb 2019 11:27:31 +0100 Received: from [139.25.69.181] (linux-ses-ext02.ppmd.siemens.net [139.25.69.181]) by mail1.sbs.de (8.15.2/8.15.2) with ESMTP id x1CARVc3008219; Tue, 12 Feb 2019 11:27:31 +0100 Subject: Re: [PATCH v2 1/1] meta: added do_transform_template task as templating system and switch From: Claudius Heine To: isar-users@googlegroups.com Cc: Claudius Heine References: <20190205111801.14362-1-claudius.heine.ext@siemens.com> <20190205111801.14362-2-claudius.heine.ext@siemens.com> Message-ID: Date: Tue, 12 Feb 2019 11:27:31 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.4.0 MIME-Version: 1.0 In-Reply-To: <20190205111801.14362-2-claudius.heine.ext@siemens.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-TUID: NDC1wE90zbBa ping On 05/02/2019 12.18, [ext] claudius.heine.ext@siemens.com wrote: > From: Claudius Heine > > The new introduced variables are TEMPLATE_FILES and TEMPLATE_VARS. > TEMPLATE_FILES is a list of files that should be processed in the > `do_transform_template` task and TEMPLATE_VARS is a list of bitbake > variable names that should be substituted in the template file. > > Signed-off-by: Claudius Heine > --- > doc/technical_overview.md | 25 ++++++++ > meta/classes/base.bbclass | 1 + > meta/classes/dpkg-base.bbclass | 2 +- > meta/classes/template.bbclass | 62 +++++++++++++++++++ > .../debian/{changelog => changelog.tmpl} | 2 +- > meta/recipes-bsp/u-boot/files/debian/control | 19 ------ > .../u-boot/files/debian/control.tmpl | 19 ++++++ > meta/recipes-bsp/u-boot/u-boot-custom.inc | 12 ++-- > .../debian/{changelog => changelog.tmpl} | 2 +- > .../linux-module/files/debian/control | 11 ---- > .../linux-module/files/debian/control.tmpl | 11 ++++ > meta/recipes-kernel/linux-module/module.inc | 7 +-- > 12 files changed, 129 insertions(+), 44 deletions(-) > create mode 100644 meta/classes/template.bbclass > rename meta/recipes-bsp/u-boot/files/debian/{changelog => changelog.tmpl} (74%) > delete mode 100644 meta/recipes-bsp/u-boot/files/debian/control > create mode 100644 meta/recipes-bsp/u-boot/files/debian/control.tmpl > rename meta/recipes-kernel/linux-module/files/debian/{changelog => changelog.tmpl} (74%) > delete mode 100644 meta/recipes-kernel/linux-module/files/debian/control > create mode 100644 meta/recipes-kernel/linux-module/files/debian/control.tmpl > > diff --git a/doc/technical_overview.md b/doc/technical_overview.md > index 7efbeee..0ddcabc 100644 > --- a/doc/technical_overview.md > +++ b/doc/technical_overview.md > @@ -283,3 +283,28 @@ cross-compilation, this is done by the same tools as for native compilation. > Depending on ISAR_CROSS_COMPILE value, additional architecture specifiers are > passed to build tools automatically, so this is absolutely transparent from > the user point of view. > + > +## 3.9 Additional features > + > +### 3.9.1 Template files > + > +A basic templating system implemented on top `envsubst(1)` is available. It > +allows limited access to bitbake variables for file generation. > + > +This system is implemented in the `template.bbclass` and defines the > +`do_transform_template` task, the `TEMPLATE_FILES` and `TEMPLATE_VARS` > +variables. Tasks that use need to use files generated via templates need > +to be executed after the `do_transform_template` task. > + > +The `TEMPLATE_FILES` variable contains a space separated list of template > +files and the `TEMPLATE_VARS` variable a space separated list of bitbake > +variable names. The `do_transform_template` task takes these and generates > +files using `envsubst(1)`. The output files are placed in the same directory > +as the template files. The name of the output files are either the name of the > +template file with `.tmpl` removed from the file name end or, if the template > +file did not end with `.tmpl`, the name of the template file with `.out` added > +to the end. > + > +Only template files from the `WORKDIR` are accepted. Either specify relative > +paths based on the recipes `WORKDIR` or absolute paths containing the `WORKDIR` > +in the `TEMPLATE_FILES` variable. > diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass > index d4082de..5ab6fa4 100644 > --- a/meta/classes/base.bbclass > +++ b/meta/classes/base.bbclass > @@ -25,6 +25,7 @@ die() { > } > > inherit logging > +inherit template > > # Derived from bitbake: bitbake/classes/base.bbclass > addtask showdata > diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass > index f1b127c..fe96850 100644 > --- a/meta/classes/dpkg-base.bbclass > +++ b/meta/classes/dpkg-base.bbclass > @@ -40,7 +40,7 @@ do_prepare_build() { > true > } > > -addtask prepare_build after do_patch before do_build > +addtask prepare_build after do_patch do_transform_template before do_build > do_prepare_build[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" > # If Isar recipes depend on each other, they typically need the package > # deployed to isar-apt > diff --git a/meta/classes/template.bbclass b/meta/classes/template.bbclass > new file mode 100644 > index 0000000..716b7a5 > --- /dev/null > +++ b/meta/classes/template.bbclass > @@ -0,0 +1,62 @@ > +# This software is a part of ISAR. > +# Copyright (C) Siemens AG, 2019 > +# > +# SPDX-License-Identifier: MIT > + > +TEMPLATE_FILES ??= "" > +TEMPLATE_VARS ??= "PN PV DESCRIPTION HOMEPAGE MAINTAINER KERNEL_NAME MACHINE \ > + DISTRO_ARCH" > + > +do_transform_template[vardeps] = "TEMPLATE_FILES ${TEMPLATE_VARS}" > +do_transform_template[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" > +python do_transform_template() { > + import subprocess, contextlib > + > + workdir = os.path.normpath(d.getVar('WORKDIR', True)) > + > + template_vars = (d.getVar('TEMPLATE_VARS', True) or "").split() > + if len(template_vars) == 0: > + return > + > + template_files = (d.getVar('TEMPLATE_FILES', True) or "").split() > + if len(template_files) == 0: > + return > + > + cmd = "envsubst" > + args = " ".join("\${{{}}}".format(i) for i in template_vars) > + > + # Copy current process environment and add template variables > + # from bitbake data store: > + env = os.environ.copy() > + for varname in template_vars: > + value = d.getVar(varname, True) > + if value: > + env.update({varname: value}) > + > + for template_file in template_files: > + # Normpath and workdir checks should prevent accidential or > + # uninformed changes to files outside of the tmp and workdirectoy > + template_file = os.path.normpath(template_file) > + > + # Convert relative paths to absolut paths based on the workdir: > + if not os.path.isabs(template_file): > + template_file = os.path.normpath(os.path.join(workdir, template_file)) > + > + if not template_file.startswith(workdir): > + bb.fatal("Template file ({}) is not within workdir ({})" > + .format(template_file, workdir)) > + > + output_file = (os.path.splitext(template_file)[0] > + if template_file.endswith(".tmpl") > + else (template_file + ".out")) > + bb.note("{} {} [in: {} out: {}]".format(cmd, args, > + template_file, output_file)) > + with contextlib.ExitStack() as stack: > + input = stack.enter_context(open(template_file, 'rb')) > + output = stack.enter_context(open(output_file, 'wb')) > + process = subprocess.Popen([cmd, args], stdin=input, > + stdout=output, env=env) > + if process.wait() != 0: > + bb.fatal("processing of template failed") > +} > +addtask do_transform_template after do_unpack before do_build > diff --git a/meta/recipes-bsp/u-boot/files/debian/changelog b/meta/recipes-bsp/u-boot/files/debian/changelog.tmpl > similarity index 74% > rename from meta/recipes-bsp/u-boot/files/debian/changelog > rename to meta/recipes-bsp/u-boot/files/debian/changelog.tmpl > index c1c3516..6e59e06 100644 > --- a/meta/recipes-bsp/u-boot/files/debian/changelog > +++ b/meta/recipes-bsp/u-boot/files/debian/changelog.tmpl > @@ -1,4 +1,4 @@ > -@PN@ (@PV@) unstable; urgency=low > +${PN} (${PV}) unstable; urgency=low > > * Generated package. > > diff --git a/meta/recipes-bsp/u-boot/files/debian/control b/meta/recipes-bsp/u-boot/files/debian/control > deleted file mode 100644 > index 6b4c839..0000000 > --- a/meta/recipes-bsp/u-boot/files/debian/control > +++ /dev/null > @@ -1,19 +0,0 @@ > -Source: @PN@ > -Section: admin > -Priority: optional > -Standards-Version: 3.9.6 > -Build-Depends: @BUILD_DEPENDS@ > -Maintainer: ISAR project > - > -Package: u-boot-@MACHINE@ > -Architecture: @DISTRO_ARCH@ > -Description: @DESCRIPTION@, bootloader binaries > - > -Package: u-boot-@MACHINE@-dev > -Architecture: @DISTRO_ARCH@ > -Description: @DESCRIPTION@, bootloader libraries > - > -Package: u-boot-tools > -Architecture: linux-any > -Depends: ${shlibs:Depends}, ${misc:Depends} > -Description: @DESCRIPTION@, companion tools > diff --git a/meta/recipes-bsp/u-boot/files/debian/control.tmpl b/meta/recipes-bsp/u-boot/files/debian/control.tmpl > new file mode 100644 > index 0000000..5c1cc92 > --- /dev/null > +++ b/meta/recipes-bsp/u-boot/files/debian/control.tmpl > @@ -0,0 +1,19 @@ > +Source: ${PN} > +Section: admin > +Priority: optional > +Standards-Version: 3.9.6 > +Build-Depends: ${BUILD_DEPENDS} > +Maintainer: ISAR project > + > +Package: u-boot-${MACHINE} > +Architecture: ${DISTRO_ARCH} > +Description: ${DESCRIPTION}, bootloader binaries > + > +Package: u-boot-${MACHINE}-dev > +Architecture: ${DISTRO_ARCH} > +Description: ${DESCRIPTION}, bootloader libraries > + > +Package: u-boot-tools > +Architecture: linux-any > +Depends: ${shlibs:Depends}, ${misc:Depends} > +Description: ${DESCRIPTION}, companion tools > diff --git a/meta/recipes-bsp/u-boot/u-boot-custom.inc b/meta/recipes-bsp/u-boot/u-boot-custom.inc > index 4b38c88..c6aab43 100644 > --- a/meta/recipes-bsp/u-boot/u-boot-custom.inc > +++ b/meta/recipes-bsp/u-boot/u-boot-custom.inc > @@ -15,16 +15,14 @@ inherit dpkg > > SRC_URI += "file://debian/" > > -U_BOOT_BUILD_DEPENDS ?= "bc, bison, flex, device-tree-compiler" > +BUILD_DEPENDS ?= "bc, bison, flex, device-tree-compiler" > + > +TEMPLATE_FILES = "debian/changelog.tmpl \ > + debian/control.tmpl" > +TEMPLATE_VARS += "BUILD_DEPENDS" > > do_prepare_build() { > cp -r ${WORKDIR}/debian ${S}/ > - sed -i -e 's/@PN@/${PN}/g' -e 's/@PV@/${PV}/g' \ > - -e 's/@BUILD_DEPENDS@/${U_BOOT_BUILD_DEPENDS}/g' \ > - -e 's/@MACHINE@/${MACHINE}/g' \ > - -e 's/@DISTRO_ARCH@/${DISTRO_ARCH}/g' \ > - -e 's/@DESCRIPTION@/${DESCRIPTION}/g' \ > - ${S}/debian/changelog ${S}/debian/control > > echo "${U_BOOT_BIN} /usr/lib/u-boot/${MACHINE}" > \ > ${S}/debian/u-boot-${MACHINE}.install > diff --git a/meta/recipes-kernel/linux-module/files/debian/changelog b/meta/recipes-kernel/linux-module/files/debian/changelog.tmpl > similarity index 74% > rename from meta/recipes-kernel/linux-module/files/debian/changelog > rename to meta/recipes-kernel/linux-module/files/debian/changelog.tmpl > index c1c3516..6e59e06 100644 > --- a/meta/recipes-kernel/linux-module/files/debian/changelog > +++ b/meta/recipes-kernel/linux-module/files/debian/changelog.tmpl > @@ -1,4 +1,4 @@ > -@PN@ (@PV@) unstable; urgency=low > +${PN} (${PV}) unstable; urgency=low > > * Generated package. > > diff --git a/meta/recipes-kernel/linux-module/files/debian/control b/meta/recipes-kernel/linux-module/files/debian/control > deleted file mode 100644 > index 1ee634c..0000000 > --- a/meta/recipes-kernel/linux-module/files/debian/control > +++ /dev/null > @@ -1,11 +0,0 @@ > -Source: @PN@ > -Section: kernel > -Priority: optional > -Standards-Version: 3.9.6 > -Build-Depends: linux-headers-@KERNEL_NAME@ > -Maintainer: ISAR project > - > -Package: @PN@ > -Architecture: any > -Depends: linux-image-@KERNEL_NAME@, kmod > -Description: @DESCRIPTION@ > diff --git a/meta/recipes-kernel/linux-module/files/debian/control.tmpl b/meta/recipes-kernel/linux-module/files/debian/control.tmpl > new file mode 100644 > index 0000000..3b3292d > --- /dev/null > +++ b/meta/recipes-kernel/linux-module/files/debian/control.tmpl > @@ -0,0 +1,11 @@ > +Source: ${PN} > +Section: kernel > +Priority: optional > +Standards-Version: 3.9.6 > +Build-Depends: linux-headers-${KERNEL_NAME} > +Maintainer: ISAR project > + > +Package: ${PN} > +Architecture: any > +Depends: linux-image-${KERNEL_NAME}, kmod > +Description: ${DESCRIPTION} > diff --git a/meta/recipes-kernel/linux-module/module.inc b/meta/recipes-kernel/linux-module/module.inc > index cb7b8ad..44edd58 100644 > --- a/meta/recipes-kernel/linux-module/module.inc > +++ b/meta/recipes-kernel/linux-module/module.inc > @@ -19,12 +19,11 @@ AUTOLOAD ?= "" > > inherit dpkg > > +TEMPLATE_FILES = "debian/control.tmpl \ > + debian/changelog.tmpl" > + > do_prepare_build() { > cp -r ${WORKDIR}/debian ${S}/ > - sed -i -e 's/@PN@/${PN}/g' -e 's/@PV@/${PV}/g' \ > - -e 's/@KERNEL_NAME@/${KERNEL_NAME}/g' \ > - -e 's/@DESCRIPTION@/${DESCRIPTION}/g' \ > - ${S}/debian/changelog ${S}/debian/control > > for module in "${AUTOLOAD}"; do > echo "echo $module >> /etc/modules" >> ${S}/debian/postinst > -- DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de