* [PATCH v3 0/1] Template system
@ 2019-02-13 15:13 claudius.heine.ext
2019-02-13 15:13 ` [PATCH v3 1/1] meta: added do_transform_template task as templating system and switch claudius.heine.ext
2019-02-14 7:54 ` [PATCH v3 0/1] Template system Maxim Yu. Osipov
0 siblings, 2 replies; 4+ messages in thread
From: claudius.heine.ext @ 2019-02-13 15:13 UTC (permalink / raw)
To: isar-users; +Cc: Claudius Heine
From: Claudius Heine <ch@denx.de>
Hi,
so after having access to the ilbers ci I could find some issues with
this patch.
I will try to describe the issue I think happened here:
Initially I added the 'do_transform_template' task with a 'before
do_build' that normally just triggers the template generation process if
the default task 'do_build' is run. The 'do_transform_template' task has
the default '${DISTRO}-${DISTRO_ARCH}' stamp-extra-info that we have to
add to pretty much all new tasks in isar.
That might be ok for normal recipes, but the buildchroot-* recipes add
stuff to the virtual default `do_build` and that now depend on the
`do_transform_template` task and its dependencies.
Currently that is a bit to complex for me to exactly figure out how
those errors get triggered, but since that works in the internal CI, but
not on ilbers, that messes up the bitbake task order.
So my solution is to remove the 'before do_build', now
'do_transform_template' is no longer executed per default. 'dpkg-base'
and the image classes add their own dependencies if they need that task.
Currently the build a ilbers ci still fails, because apparently
gettext-base is not installed there. I added gettext-base as a
dependency to the user_manual.md
regards,
Claudius
Changes from v2:
- put subprocess into context manager
- added 'gettext-base' isar dependency
- remove 'before do_build' from addtask
Changes from v1:
- fixed spelling mistakes of documentation
Claudius Heine (1):
meta: added do_transform_template task as templating system and switch
doc/technical_overview.md | 25 ++++++++
doc/user_manual.md | 1 +
meta/classes/base.bbclass | 1 +
meta/classes/dpkg-base.bbclass | 2 +-
meta/classes/template.bbclass | 62 +++++++++++++++++++
.../debian/{changelog => changelog.tmpl} | 2 +-
meta/recipes-bsp/u-boot/files/debian/control | 19 ------
.../u-boot/files/debian/control.tmpl | 19 ++++++
meta/recipes-bsp/u-boot/u-boot-custom.inc | 12 ++--
.../debian/{changelog => changelog.tmpl} | 2 +-
.../linux-module/files/debian/control | 11 ----
.../linux-module/files/debian/control.tmpl | 11 ++++
meta/recipes-kernel/linux-module/module.inc | 7 +--
13 files changed, 130 insertions(+), 44 deletions(-)
create mode 100644 meta/classes/template.bbclass
rename meta/recipes-bsp/u-boot/files/debian/{changelog => changelog.tmpl} (74%)
delete mode 100644 meta/recipes-bsp/u-boot/files/debian/control
create mode 100644 meta/recipes-bsp/u-boot/files/debian/control.tmpl
rename meta/recipes-kernel/linux-module/files/debian/{changelog => changelog.tmpl} (74%)
delete mode 100644 meta/recipes-kernel/linux-module/files/debian/control
create mode 100644 meta/recipes-kernel/linux-module/files/debian/control.tmpl
--
2.20.1
^ permalink raw reply [flat|nested] 4+ messages in thread
* [PATCH v3 1/1] meta: added do_transform_template task as templating system and switch
2019-02-13 15:13 [PATCH v3 0/1] Template system claudius.heine.ext
@ 2019-02-13 15:13 ` claudius.heine.ext
2019-02-14 7:54 ` [PATCH v3 0/1] Template system Maxim Yu. Osipov
1 sibling, 0 replies; 4+ messages in thread
From: claudius.heine.ext @ 2019-02-13 15:13 UTC (permalink / raw)
To: isar-users; +Cc: Claudius Heine
From: Claudius Heine <ch@denx.de>
The new introduced variables are TEMPLATE_FILES and TEMPLATE_VARS.
TEMPLATE_FILES is a list of files that should be processed in the
`do_transform_template` task and TEMPLATE_VARS is a list of bitbake
variable names that should be substituted in the template file.
Signed-off-by: Claudius Heine <ch@denx.de>
---
doc/technical_overview.md | 25 ++++++++
doc/user_manual.md | 1 +
meta/classes/base.bbclass | 1 +
meta/classes/dpkg-base.bbclass | 2 +-
meta/classes/template.bbclass | 62 +++++++++++++++++++
.../debian/{changelog => changelog.tmpl} | 2 +-
meta/recipes-bsp/u-boot/files/debian/control | 19 ------
.../u-boot/files/debian/control.tmpl | 19 ++++++
meta/recipes-bsp/u-boot/u-boot-custom.inc | 12 ++--
.../debian/{changelog => changelog.tmpl} | 2 +-
.../linux-module/files/debian/control | 11 ----
.../linux-module/files/debian/control.tmpl | 11 ++++
meta/recipes-kernel/linux-module/module.inc | 7 +--
13 files changed, 130 insertions(+), 44 deletions(-)
create mode 100644 meta/classes/template.bbclass
rename meta/recipes-bsp/u-boot/files/debian/{changelog => changelog.tmpl} (74%)
delete mode 100644 meta/recipes-bsp/u-boot/files/debian/control
create mode 100644 meta/recipes-bsp/u-boot/files/debian/control.tmpl
rename meta/recipes-kernel/linux-module/files/debian/{changelog => changelog.tmpl} (74%)
delete mode 100644 meta/recipes-kernel/linux-module/files/debian/control
create mode 100644 meta/recipes-kernel/linux-module/files/debian/control.tmpl
diff --git a/doc/technical_overview.md b/doc/technical_overview.md
index 7efbeee..0ddcabc 100644
--- a/doc/technical_overview.md
+++ b/doc/technical_overview.md
@@ -283,3 +283,28 @@ cross-compilation, this is done by the same tools as for native compilation.
Depending on ISAR_CROSS_COMPILE value, additional architecture specifiers are
passed to build tools automatically, so this is absolutely transparent from
the user point of view.
+
+## 3.9 Additional features
+
+### 3.9.1 Template files
+
+A basic templating system implemented on top `envsubst(1)` is available. It
+allows limited access to bitbake variables for file generation.
+
+This system is implemented in the `template.bbclass` and defines the
+`do_transform_template` task, the `TEMPLATE_FILES` and `TEMPLATE_VARS`
+variables. Tasks that use need to use files generated via templates need
+to be executed after the `do_transform_template` task.
+
+The `TEMPLATE_FILES` variable contains a space separated list of template
+files and the `TEMPLATE_VARS` variable a space separated list of bitbake
+variable names. The `do_transform_template` task takes these and generates
+files using `envsubst(1)`. The output files are placed in the same directory
+as the template files. The name of the output files are either the name of the
+template file with `.tmpl` removed from the file name end or, if the template
+file did not end with `.tmpl`, the name of the template file with `.out` added
+to the end.
+
+Only template files from the `WORKDIR` are accepted. Either specify relative
+paths based on the recipes `WORKDIR` or absolute paths containing the `WORKDIR`
+in the `TEMPLATE_FILES` variable.
diff --git a/doc/user_manual.md b/doc/user_manual.md
index ff4939c..c1b154f 100644
--- a/doc/user_manual.md
+++ b/doc/user_manual.md
@@ -68,6 +68,7 @@ qemu-user-static
binfmt-support
sudo
reprepro
+gettext-base
```
Notes:
diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
index 6ca93bf..4279a68 100644
--- a/meta/classes/base.bbclass
+++ b/meta/classes/base.bbclass
@@ -25,6 +25,7 @@ die() {
}
inherit logging
+inherit template
# Derived from bitbake: bitbake/classes/base.bbclass
addtask showdata
diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass
index 175dc80..320869b 100644
--- a/meta/classes/dpkg-base.bbclass
+++ b/meta/classes/dpkg-base.bbclass
@@ -60,7 +60,7 @@ do_prepare_build() {
true
}
-addtask prepare_build after do_patch before do_build
+addtask prepare_build after do_patch do_transform_template before do_build
do_prepare_build[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
# If Isar recipes depend on each other, they typically need the package
# deployed to isar-apt
diff --git a/meta/classes/template.bbclass b/meta/classes/template.bbclass
new file mode 100644
index 0000000..00d7453
--- /dev/null
+++ b/meta/classes/template.bbclass
@@ -0,0 +1,62 @@
+# This software is a part of ISAR.
+# Copyright (C) Siemens AG, 2019
+#
+# SPDX-License-Identifier: MIT
+
+TEMPLATE_FILES ??= ""
+TEMPLATE_VARS ??= "PN PV DESCRIPTION HOMEPAGE MAINTAINER KERNEL_NAME MACHINE \
+ DISTRO_ARCH"
+
+do_transform_template[vardeps] = "TEMPLATE_FILES ${TEMPLATE_VARS}"
+do_transform_template[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
+python do_transform_template() {
+ import subprocess, contextlib
+
+ workdir = os.path.normpath(d.getVar('WORKDIR', True))
+
+ template_vars = (d.getVar('TEMPLATE_VARS', True) or "").split()
+ if len(template_vars) == 0:
+ return
+
+ template_files = (d.getVar('TEMPLATE_FILES', True) or "").split()
+ if len(template_files) == 0:
+ return
+
+ cmd = "envsubst"
+ args = " ".join("\${{{}}}".format(i) for i in template_vars)
+
+ # Copy current process environment and add template variables
+ # from bitbake data store:
+ env = os.environ.copy()
+ for varname in template_vars:
+ value = d.getVar(varname, True)
+ if value:
+ env.update({varname: value})
+
+ for template_file in template_files:
+ # Normpath and workdir checks should prevent accidential or
+ # uninformed changes to files outside of the tmp and workdirectoy
+ template_file = os.path.normpath(template_file)
+
+ # Convert relative paths to absolut paths based on the workdir:
+ if not os.path.isabs(template_file):
+ template_file = os.path.normpath(os.path.join(workdir, template_file))
+
+ if not template_file.startswith(workdir):
+ bb.fatal("Template file ({}) is not within workdir ({})"
+ .format(template_file, workdir))
+
+ output_file = (os.path.splitext(template_file)[0]
+ if template_file.endswith(".tmpl")
+ else (template_file + ".out"))
+ bb.note("{} {} [in: {} out: {}]".format(cmd, args,
+ template_file, output_file))
+ with contextlib.ExitStack() as stack:
+ input = stack.enter_context(open(template_file, 'rb'))
+ output = stack.enter_context(open(output_file, 'wb'))
+ process = stack.enter_context(subprocess.Popen([cmd, args], stdin=input,
+ stdout=output, env=env))
+ if process.wait() != 0:
+ bb.fatal("processing of template failed")
+}
+addtask do_transform_template after do_unpack
diff --git a/meta/recipes-bsp/u-boot/files/debian/changelog b/meta/recipes-bsp/u-boot/files/debian/changelog.tmpl
similarity index 74%
rename from meta/recipes-bsp/u-boot/files/debian/changelog
rename to meta/recipes-bsp/u-boot/files/debian/changelog.tmpl
index c1c3516..6e59e06 100644
--- a/meta/recipes-bsp/u-boot/files/debian/changelog
+++ b/meta/recipes-bsp/u-boot/files/debian/changelog.tmpl
@@ -1,4 +1,4 @@
-@PN@ (@PV@) unstable; urgency=low
+${PN} (${PV}) unstable; urgency=low
* Generated package.
diff --git a/meta/recipes-bsp/u-boot/files/debian/control b/meta/recipes-bsp/u-boot/files/debian/control
deleted file mode 100644
index 6b4c839..0000000
--- a/meta/recipes-bsp/u-boot/files/debian/control
+++ /dev/null
@@ -1,19 +0,0 @@
-Source: @PN@
-Section: admin
-Priority: optional
-Standards-Version: 3.9.6
-Build-Depends: @BUILD_DEPENDS@
-Maintainer: ISAR project <isar-users@googlegroups.com>
-
-Package: u-boot-@MACHINE@
-Architecture: @DISTRO_ARCH@
-Description: @DESCRIPTION@, bootloader binaries
-
-Package: u-boot-@MACHINE@-dev
-Architecture: @DISTRO_ARCH@
-Description: @DESCRIPTION@, bootloader libraries
-
-Package: u-boot-tools
-Architecture: linux-any
-Depends: ${shlibs:Depends}, ${misc:Depends}
-Description: @DESCRIPTION@, companion tools
diff --git a/meta/recipes-bsp/u-boot/files/debian/control.tmpl b/meta/recipes-bsp/u-boot/files/debian/control.tmpl
new file mode 100644
index 0000000..5c1cc92
--- /dev/null
+++ b/meta/recipes-bsp/u-boot/files/debian/control.tmpl
@@ -0,0 +1,19 @@
+Source: ${PN}
+Section: admin
+Priority: optional
+Standards-Version: 3.9.6
+Build-Depends: ${BUILD_DEPENDS}
+Maintainer: ISAR project <isar-users@googlegroups.com>
+
+Package: u-boot-${MACHINE}
+Architecture: ${DISTRO_ARCH}
+Description: ${DESCRIPTION}, bootloader binaries
+
+Package: u-boot-${MACHINE}-dev
+Architecture: ${DISTRO_ARCH}
+Description: ${DESCRIPTION}, bootloader libraries
+
+Package: u-boot-tools
+Architecture: linux-any
+Depends: ${shlibs:Depends}, ${misc:Depends}
+Description: ${DESCRIPTION}, companion tools
diff --git a/meta/recipes-bsp/u-boot/u-boot-custom.inc b/meta/recipes-bsp/u-boot/u-boot-custom.inc
index 4b38c88..c6aab43 100644
--- a/meta/recipes-bsp/u-boot/u-boot-custom.inc
+++ b/meta/recipes-bsp/u-boot/u-boot-custom.inc
@@ -15,16 +15,14 @@ inherit dpkg
SRC_URI += "file://debian/"
-U_BOOT_BUILD_DEPENDS ?= "bc, bison, flex, device-tree-compiler"
+BUILD_DEPENDS ?= "bc, bison, flex, device-tree-compiler"
+
+TEMPLATE_FILES = "debian/changelog.tmpl \
+ debian/control.tmpl"
+TEMPLATE_VARS += "BUILD_DEPENDS"
do_prepare_build() {
cp -r ${WORKDIR}/debian ${S}/
- sed -i -e 's/@PN@/${PN}/g' -e 's/@PV@/${PV}/g' \
- -e 's/@BUILD_DEPENDS@/${U_BOOT_BUILD_DEPENDS}/g' \
- -e 's/@MACHINE@/${MACHINE}/g' \
- -e 's/@DISTRO_ARCH@/${DISTRO_ARCH}/g' \
- -e 's/@DESCRIPTION@/${DESCRIPTION}/g' \
- ${S}/debian/changelog ${S}/debian/control
echo "${U_BOOT_BIN} /usr/lib/u-boot/${MACHINE}" > \
${S}/debian/u-boot-${MACHINE}.install
diff --git a/meta/recipes-kernel/linux-module/files/debian/changelog b/meta/recipes-kernel/linux-module/files/debian/changelog.tmpl
similarity index 74%
rename from meta/recipes-kernel/linux-module/files/debian/changelog
rename to meta/recipes-kernel/linux-module/files/debian/changelog.tmpl
index c1c3516..6e59e06 100644
--- a/meta/recipes-kernel/linux-module/files/debian/changelog
+++ b/meta/recipes-kernel/linux-module/files/debian/changelog.tmpl
@@ -1,4 +1,4 @@
-@PN@ (@PV@) unstable; urgency=low
+${PN} (${PV}) unstable; urgency=low
* Generated package.
diff --git a/meta/recipes-kernel/linux-module/files/debian/control b/meta/recipes-kernel/linux-module/files/debian/control
deleted file mode 100644
index 1ee634c..0000000
--- a/meta/recipes-kernel/linux-module/files/debian/control
+++ /dev/null
@@ -1,11 +0,0 @@
-Source: @PN@
-Section: kernel
-Priority: optional
-Standards-Version: 3.9.6
-Build-Depends: linux-headers-@KERNEL_NAME@
-Maintainer: ISAR project <isar-users@googlegroups.com>
-
-Package: @PN@
-Architecture: any
-Depends: linux-image-@KERNEL_NAME@, kmod
-Description: @DESCRIPTION@
diff --git a/meta/recipes-kernel/linux-module/files/debian/control.tmpl b/meta/recipes-kernel/linux-module/files/debian/control.tmpl
new file mode 100644
index 0000000..3b3292d
--- /dev/null
+++ b/meta/recipes-kernel/linux-module/files/debian/control.tmpl
@@ -0,0 +1,11 @@
+Source: ${PN}
+Section: kernel
+Priority: optional
+Standards-Version: 3.9.6
+Build-Depends: linux-headers-${KERNEL_NAME}
+Maintainer: ISAR project <isar-users@googlegroups.com>
+
+Package: ${PN}
+Architecture: any
+Depends: linux-image-${KERNEL_NAME}, kmod
+Description: ${DESCRIPTION}
diff --git a/meta/recipes-kernel/linux-module/module.inc b/meta/recipes-kernel/linux-module/module.inc
index cb7b8ad..44edd58 100644
--- a/meta/recipes-kernel/linux-module/module.inc
+++ b/meta/recipes-kernel/linux-module/module.inc
@@ -19,12 +19,11 @@ AUTOLOAD ?= ""
inherit dpkg
+TEMPLATE_FILES = "debian/control.tmpl \
+ debian/changelog.tmpl"
+
do_prepare_build() {
cp -r ${WORKDIR}/debian ${S}/
- sed -i -e 's/@PN@/${PN}/g' -e 's/@PV@/${PV}/g' \
- -e 's/@KERNEL_NAME@/${KERNEL_NAME}/g' \
- -e 's/@DESCRIPTION@/${DESCRIPTION}/g' \
- ${S}/debian/changelog ${S}/debian/control
for module in "${AUTOLOAD}"; do
echo "echo $module >> /etc/modules" >> ${S}/debian/postinst
--
2.20.1
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH v3 0/1] Template system
2019-02-13 15:13 [PATCH v3 0/1] Template system claudius.heine.ext
2019-02-13 15:13 ` [PATCH v3 1/1] meta: added do_transform_template task as templating system and switch claudius.heine.ext
@ 2019-02-14 7:54 ` Maxim Yu. Osipov
2019-02-14 8:07 ` Claudius Heine
1 sibling, 1 reply; 4+ messages in thread
From: Maxim Yu. Osipov @ 2019-02-14 7:54 UTC (permalink / raw)
To: claudius.heine.ext, isar-users; +Cc: Claudius Heine
Hi Claudius,
Good news:
I've installed gettext-base in ilbers CI and with this version of patch
your series applied over current next was built OK:
http://isar-build.org:8080/job/isar_mosipov_next/142/consoleFull
Only smoke test for qemui386-buster failed - but this is not relevant to
your patches (this sporadically happens and needs to be investigated
separately).
Bad news:
So far build of de0-nano-soc-stretch target with enabled
CROSS_COMPILATION was not included into CI.
After upgrade of CI server I'll include de0-nano-soc-stretch target into
CI "fast" build as this is a supported target which in particular builds
linux kernel and u-boot.
The corresponding patch will be sent afterwards.
So I tried to build de0-nano-soc-stretch target (on my branch
https://github.com/ilbers/isar/commits/mosipov-next) with enabled
CROSS_COMPILATION - and it fails after applying your patches. See log
below. Build of de0-nano-soc-stretch with enabled CROSS_COMPILATION on
current 'next' passes OK.
========
bitbake multiconfig:de0-nano-soc-stretch:isar-image-base
<snip>
NOTE: Executing RunQueue Tasks
ERROR: mc:de0-nano-soc-stretch:u-boot-2018.09-r0 do_install_builddeps:
Function failed: do_install_builddeps (log file is located at
/home/myo/work/isar/src/trunk/isar/build/tmp/work/debian-stretch-armhf/u-boot-2018.09-r0/temp/log.do_install_builddeps.5395)
ERROR: Logfile of failure stored in:
/home/myo/work/isar/src/trunk/isar/build/tmp/work/debian-stretch-armhf/u-boot-2018.09-r0/temp/log.do_install_builddeps.5395
Log data follows:
| DEBUG: Executing shell function do_install_builddeps
| Get:1 file:/isar-apt isar InRelease
| Ign:1 file:/isar-apt isar InRelease
| Get:2 file:/isar-apt isar Release [3563 B]
| Get:2 file:/isar-apt isar Release [3563 B]
| Get:3 file:/isar-apt isar Release.gpg
| Ign:3 file:/isar-apt isar Release.gpg
| Get:4 file:/isar-apt isar/main armhf Packages [544 B]
| Reading package lists...
| mk-build-deps: warning: debian/changelog(l1): badly formatted
heading line
| LINE: ${PN} (${PV}) unstable; urgency=low
| mk-build-deps: warning: debian/changelog(l2): found blank line
where expected first heading
| mk-build-deps: warning: debian/changelog(l3): found change data
where expected first heading
| LINE: * Generated package.
| mk-build-deps: warning: unknown information field '' in input data in
parsed version of changelog
| hostname: Name or service not known
| dpkg-architecture: warning: specified GNU system type
arm-linux-gnueabihf does not match CC system type x86_64-linux-gnu, try
setting a correct CC environment variable
| dh_testdir
| dh_testroot
| dh_prep
| dh_testdir
| dh_testroot
| dh_install
| dh_install: Compatibility levels before 9 are deprecated (level 7 in use)
| dh_installdocs
| dh_installdocs: Compatibility levels before 9 are deprecated (level 7
in use)
| dh_installchangelogs
| dpkg-parsechangelog: warning: debian/changelog(l1): badly
formatted heading line
| LINE: ${PN}-cross-build-deps (1.0) unstable; urgency=low
| dpkg-parsechangelog: warning: debian/changelog(l2): found blank
line where expected first heading
| dpkg-parsechangelog: warning: debian/changelog(l3): found change
data where expected first heading
| LINE: * First version
| dpkg-parsechangelog: warning: unknown information field '' in input
data in parsed version of changelog
| dh_compress
| find: 'debian/-cross-build-deps': No such file or directory
| dh_fixperms
| dh_installdeb
| dh_installdeb: Compatibility levels before 9 are deprecated (level 7
in use)
| dh_gencontrol
| dpkg-gencontrol: error: illegal package name '${PN}-cross-build-deps':
character '$' not allowed
| dh_gencontrol: dpkg-gencontrol -p${PN}-cross-build-deps
-ldebian/changelog -Tdebian/${PN}-cross-build-deps.substvars
-Pdebian/${PN}-cross-build-deps returned exit code 255
| debian/rules:20: recipe for target 'binary-indep' failed
| make: *** [binary-indep] Error 2
| Error in the build process: exit status 2
| dpkg: error: cannot access archive
'${PN}-cross-build-deps_1.0_armhf.deb': No such file or directory
| mk-build-deps: dpkg --unpack failed
| WARNING: exit code 2 from a shell command.
| ERROR: Function failed: do_install_builddeps (log file is located at
/home/myo/work/isar/src/trunk/isar/build/tmp/work/debian-stretch-armhf/u-boot-2018.09-r0/temp/log.do_install_builddeps.5395)
======
On 2/13/19 4:13 PM, claudius.heine.ext@siemens.com wrote:
> From: Claudius Heine <ch@denx.de>
>
> Hi,
>
> so after having access to the ilbers ci I could find some issues with
> this patch.
>
> I will try to describe the issue I think happened here:
>
> Initially I added the 'do_transform_template' task with a 'before
> do_build' that normally just triggers the template generation process if
> the default task 'do_build' is run. The 'do_transform_template' task has
> the default '${DISTRO}-${DISTRO_ARCH}' stamp-extra-info that we have to
> add to pretty much all new tasks in isar.
>
> That might be ok for normal recipes, but the buildchroot-* recipes add
> stuff to the virtual default `do_build` and that now depend on the
> `do_transform_template` task and its dependencies.
>
> Currently that is a bit to complex for me to exactly figure out how
> those errors get triggered, but since that works in the internal CI, but
> not on ilbers, that messes up the bitbake task order.
>
> So my solution is to remove the 'before do_build', now
> 'do_transform_template' is no longer executed per default. 'dpkg-base'
> and the image classes add their own dependencies if they need that task.
>
> Currently the build a ilbers ci still fails, because apparently
> gettext-base is not installed there. I added gettext-base as a
> dependency to the user_manual.md
>
> regards,
> Claudius
>
> Changes from v2:
> - put subprocess into context manager
> - added 'gettext-base' isar dependency
> - remove 'before do_build' from addtask
>
> Changes from v1:
> - fixed spelling mistakes of documentation
>
> Claudius Heine (1):
> meta: added do_transform_template task as templating system and switch
>
> doc/technical_overview.md | 25 ++++++++
> doc/user_manual.md | 1 +
> meta/classes/base.bbclass | 1 +
> meta/classes/dpkg-base.bbclass | 2 +-
> meta/classes/template.bbclass | 62 +++++++++++++++++++
> .../debian/{changelog => changelog.tmpl} | 2 +-
> meta/recipes-bsp/u-boot/files/debian/control | 19 ------
> .../u-boot/files/debian/control.tmpl | 19 ++++++
> meta/recipes-bsp/u-boot/u-boot-custom.inc | 12 ++--
> .../debian/{changelog => changelog.tmpl} | 2 +-
> .../linux-module/files/debian/control | 11 ----
> .../linux-module/files/debian/control.tmpl | 11 ++++
> meta/recipes-kernel/linux-module/module.inc | 7 +--
> 13 files changed, 130 insertions(+), 44 deletions(-)
> create mode 100644 meta/classes/template.bbclass
> rename meta/recipes-bsp/u-boot/files/debian/{changelog => changelog.tmpl} (74%)
> delete mode 100644 meta/recipes-bsp/u-boot/files/debian/control
> create mode 100644 meta/recipes-bsp/u-boot/files/debian/control.tmpl
> rename meta/recipes-kernel/linux-module/files/debian/{changelog => changelog.tmpl} (74%)
> delete mode 100644 meta/recipes-kernel/linux-module/files/debian/control
> create mode 100644 meta/recipes-kernel/linux-module/files/debian/control.tmpl
>
--
Maxim Osipov
ilbers GmbH
Maria-Merian-Str. 8
85521 Ottobrunn
Germany
+49 (151) 6517 6917
mosipov@ilbers.de
http://ilbers.de/
Commercial register Munich, HRB 214197
General Manager: Baurzhan Ismagulov
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH v3 0/1] Template system
2019-02-14 7:54 ` [PATCH v3 0/1] Template system Maxim Yu. Osipov
@ 2019-02-14 8:07 ` Claudius Heine
0 siblings, 0 replies; 4+ messages in thread
From: Claudius Heine @ 2019-02-14 8:07 UTC (permalink / raw)
To: Maxim Yu. Osipov, isar-users; +Cc: Claudius Heine
Hi Maxim,
On 14/02/2019 08.54, Maxim Yu. Osipov wrote:
> Hi Claudius,
>
> Good news:
>
> I've installed gettext-base in ilbers CI and with this version of patch
> your series applied over current next was built OK:
> http://isar-build.org:8080/job/isar_mosipov_next/142/consoleFull
>
> Only smoke test for qemui386-buster failed - but this is not relevant to
> your patches (this sporadically happens and needs to be investigated
> separately).
>
> Bad news:
>
> So far build of de0-nano-soc-stretch target with enabled
> CROSS_COMPILATION was not included into CI.
> After upgrade of CI server I'll include de0-nano-soc-stretch target into
> CI "fast" build as this is a supported target which in particular builds
> linux kernel and u-boot.
> The corresponding patch will be sent afterwards.
>
> So I tried to build de0-nano-soc-stretch target (on my branch
> https://github.com/ilbers/isar/commits/mosipov-next) with enabled
> CROSS_COMPILATION - and it fails after applying your patches. See log
> below. Build of de0-nano-soc-stretch with enabled CROSS_COMPILATION on
> current 'next' passes OK.
Thanks for testing and finding that issue. But it is a shame that this
was not included in the ci build.
>
>
> ========
> bitbake multiconfig:de0-nano-soc-stretch:isar-image-base
>
> <snip>
>
> NOTE: Executing RunQueue Tasks
> ERROR: mc:de0-nano-soc-stretch:u-boot-2018.09-r0 do_install_builddeps:
> Function failed: do_install_builddeps (log file is located at
> /home/myo/work/isar/src/trunk/isar/build/tmp/work/debian-stretch-armhf/u-boot-2018.09-r0/temp/log.do_install_builddeps.5395)
>
> ERROR: Logfile of failure stored in:
> /home/myo/work/isar/src/trunk/isar/build/tmp/work/debian-stretch-armhf/u-boot-2018.09-r0/temp/log.do_install_builddeps.5395
>
> Log data follows:
> | DEBUG: Executing shell function do_install_builddeps
> | Get:1 file:/isar-apt isar InRelease
> | Ign:1 file:/isar-apt isar InRelease
> | Get:2 file:/isar-apt isar Release [3563 B]
> | Get:2 file:/isar-apt isar Release [3563 B]
> | Get:3 file:/isar-apt isar Release.gpg
> | Ign:3 file:/isar-apt isar Release.gpg
> | Get:4 file:/isar-apt isar/main armhf Packages [544 B]
> | Reading package lists...
> | mk-build-deps: warning: debian/changelog(l1): badly formatted
> heading line
> | LINE: ${PN} (${PV}) unstable; urgency=low
> | mk-build-deps: warning: debian/changelog(l2): found blank line
> where expected first heading
> | mk-build-deps: warning: debian/changelog(l3): found change data
> where expected first heading
> | LINE: * Generated package.
> | mk-build-deps: warning: unknown information field '' in input data in
> parsed version of changelog
> | hostname: Name or service not known
> | dpkg-architecture: warning: specified GNU system type
> arm-linux-gnueabihf does not match CC system type x86_64-linux-gnu, try
> setting a correct CC environment variable
> | dh_testdir
> | dh_testroot
> | dh_prep
> | dh_testdir
> | dh_testroot
> | dh_install
> | dh_install: Compatibility levels before 9 are deprecated (level 7 in use)
> | dh_installdocs
> | dh_installdocs: Compatibility levels before 9 are deprecated (level 7
> in use)
> | dh_installchangelogs
> | dpkg-parsechangelog: warning: debian/changelog(l1): badly
> formatted heading line
> | LINE: ${PN}-cross-build-deps (1.0) unstable; urgency=low
> | dpkg-parsechangelog: warning: debian/changelog(l2): found blank
> line where expected first heading
> | dpkg-parsechangelog: warning: debian/changelog(l3): found change
> data where expected first heading
> | LINE: * First version
> | dpkg-parsechangelog: warning: unknown information field '' in input
> data in parsed version of changelog
> | dh_compress
> | find: 'debian/-cross-build-deps': No such file or directory
> | dh_fixperms
> | dh_installdeb
> | dh_installdeb: Compatibility levels before 9 are deprecated (level 7
> in use)
> | dh_gencontrol
> | dpkg-gencontrol: error: illegal package name '${PN}-cross-build-deps':
> character '$' not allowed
> | dh_gencontrol: dpkg-gencontrol -p${PN}-cross-build-deps
> -ldebian/changelog -Tdebian/${PN}-cross-build-deps.substvars
> -Pdebian/${PN}-cross-build-deps returned exit code 255
> | debian/rules:20: recipe for target 'binary-indep' failed
> | make: *** [binary-indep] Error 2
> | Error in the build process: exit status 2
> | dpkg: error: cannot access archive
> '${PN}-cross-build-deps_1.0_armhf.deb': No such file or directory
OK, so for some reason PN is not replaced correctly. I will investigate
that and send v4.
regards,
Claudius
> | mk-build-deps: dpkg --unpack failed
> | WARNING: exit code 2 from a shell command.
> | ERROR: Function failed: do_install_builddeps (log file is located at
> /home/myo/work/isar/src/trunk/isar/build/tmp/work/debian-stretch-armhf/u-boot-2018.09-r0/temp/log.do_install_builddeps.5395)
>
> ======
>
>
>
> On 2/13/19 4:13 PM, claudius.heine.ext@siemens.com wrote:
>> From: Claudius Heine <ch@denx.de>
>>
>> Hi,
>>
>> so after having access to the ilbers ci I could find some issues with
>> this patch.
>>
>> I will try to describe the issue I think happened here:
>>
>> Initially I added the 'do_transform_template' task with a 'before
>> do_build' that normally just triggers the template generation process if
>> the default task 'do_build' is run. The 'do_transform_template' task has
>> the default '${DISTRO}-${DISTRO_ARCH}' stamp-extra-info that we have to
>> add to pretty much all new tasks in isar.
>>
>> That might be ok for normal recipes, but the buildchroot-* recipes add
>> stuff to the virtual default `do_build` and that now depend on the
>> `do_transform_template` task and its dependencies.
>>
>> Currently that is a bit to complex for me to exactly figure out how
>> those errors get triggered, but since that works in the internal CI, but
>> not on ilbers, that messes up the bitbake task order.
>>
>> So my solution is to remove the 'before do_build', now
>> 'do_transform_template' is no longer executed per default. 'dpkg-base'
>> and the image classes add their own dependencies if they need that task.
>>
>> Currently the build a ilbers ci still fails, because apparently
>> gettext-base is not installed there. I added gettext-base as a
>> dependency to the user_manual.md
>>
>> regards,
>> Claudius
>>
>> Changes from v2:
>> - put subprocess into context manager
>> - added 'gettext-base' isar dependency
>> - remove 'before do_build' from addtask
>>
>> Changes from v1:
>> - fixed spelling mistakes of documentation
>>
>> Claudius Heine (1):
>> meta: added do_transform_template task as templating system and switch
>>
>> doc/technical_overview.md | 25 ++++++++
>> doc/user_manual.md | 1 +
>> meta/classes/base.bbclass | 1 +
>> meta/classes/dpkg-base.bbclass | 2 +-
>> meta/classes/template.bbclass | 62 +++++++++++++++++++
>> .../debian/{changelog => changelog.tmpl} | 2 +-
>> meta/recipes-bsp/u-boot/files/debian/control | 19 ------
>> .../u-boot/files/debian/control.tmpl | 19 ++++++
>> meta/recipes-bsp/u-boot/u-boot-custom.inc | 12 ++--
>> .../debian/{changelog => changelog.tmpl} | 2 +-
>> .../linux-module/files/debian/control | 11 ----
>> .../linux-module/files/debian/control.tmpl | 11 ++++
>> meta/recipes-kernel/linux-module/module.inc | 7 +--
>> 13 files changed, 130 insertions(+), 44 deletions(-)
>> create mode 100644 meta/classes/template.bbclass
>> rename meta/recipes-bsp/u-boot/files/debian/{changelog =>
>> changelog.tmpl} (74%)
>> delete mode 100644 meta/recipes-bsp/u-boot/files/debian/control
>> create mode 100644 meta/recipes-bsp/u-boot/files/debian/control.tmpl
>> rename meta/recipes-kernel/linux-module/files/debian/{changelog =>
>> changelog.tmpl} (74%)
>> delete mode 100644
>> meta/recipes-kernel/linux-module/files/debian/control
>> create mode 100644
>> meta/recipes-kernel/linux-module/files/debian/control.tmpl
>>
>
>
--
DENX Software Engineering GmbH, Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2019-02-14 8:07 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-02-13 15:13 [PATCH v3 0/1] Template system claudius.heine.ext
2019-02-13 15:13 ` [PATCH v3 1/1] meta: added do_transform_template task as templating system and switch claudius.heine.ext
2019-02-14 7:54 ` [PATCH v3 0/1] Template system Maxim Yu. Osipov
2019-02-14 8:07 ` Claudius Heine
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox