* [PATCH 0/7] linux-custom recipe rework
@ 2019-11-05 13:55 Cedric Hombourger
2019-11-05 13:55 ` [PATCH 1/7] recipes-kernel/linux: make KERNEL_DEFCONFIG support in-tree defconfigs Cedric Hombourger
` (7 more replies)
0 siblings, 8 replies; 83+ messages in thread
From: Cedric Hombourger @ 2019-11-05 13:55 UTC (permalink / raw)
To: isar-users; +Cc: Cedric Hombourger
The Linux kernel is packaged a bit differently between Debian variants
and in particular for the perf package(s) in Debian vs Ubuntu. Since
there is no way the kernel build scripts would ever get distro specific
support for building more than the kernel and headers, we need our own
build/packaging scripts. The approach is documented in the custom_kernel.md
file included in this patch series.
This changeset was tested against ci_build.sh and some images were booted
on hardware available here. No regressions were detected (to date).
It is probably a good time to seek comments on the changeset and ideally
get it merged before starting some of the work-packages such as getting
perf packaged for Debian and Ubuntu distros.
Cedric Hombourger (7):
recipes-kernel/linux: make KERNEL_DEFCONFIG support in-tree defconfigs
linux-mainline: fix stripping of .0 from the kernel version
linux-mainline: update from 4.19.0 to 4.19.80
linux-custom: add support for kernel config fragments
linux-mainline: disable support for HFS to demonstrate use of config fragments
buildchroot-host: install qemu-static to support hybrid cross-compiles
linux-custom: rewrite to no longer depend on the kernel's builddeb
doc/custom_kernel.md | 181 ++++++++++++++++
.../recipes-kernel/linux/files/no-fs-hfs.cfg | 2 +
...ne_4.19.0.bb => linux-mainline_4.19.80.bb} | 8 +-
.../buildchroot/buildchroot-host.bb | 6 +
.../linux/files/build-kernel.sh | 129 ------------
meta/recipes-kernel/linux/files/debian/compat | 1 +
.../linux/files/debian/control.tmpl | 36 ++++
meta/recipes-kernel/linux/files/debian/files | 5 +
.../linux/files/debian/isar/build.tmpl | 36 ++++
.../linux/files/debian/isar/clean.tmpl | 20 ++
.../linux/files/debian/isar/common.tmpl | 60 ++++++
.../linux/files/debian/isar/install.tmpl | 197 ++++++++++++++++++
.../files/debian/linux-image.postinst.tmpl | 22 ++
.../files/debian/linux-image.postrm.tmpl | 18 ++
.../linux/files/debian/rules.tmpl | 39 ++++
meta/recipes-kernel/linux/linux-custom.inc | 196 ++++++++++++-----
16 files changed, 772 insertions(+), 184 deletions(-)
create mode 100644 doc/custom_kernel.md
create mode 100644 meta-isar/recipes-kernel/linux/files/no-fs-hfs.cfg
rename meta-isar/recipes-kernel/linux/{linux-mainline_4.19.0.bb => linux-mainline_4.19.80.bb} (67%)
delete mode 100644 meta/recipes-kernel/linux/files/build-kernel.sh
create mode 100644 meta/recipes-kernel/linux/files/debian/compat
create mode 100644 meta/recipes-kernel/linux/files/debian/control.tmpl
create mode 100644 meta/recipes-kernel/linux/files/debian/files
create mode 100644 meta/recipes-kernel/linux/files/debian/isar/build.tmpl
create mode 100644 meta/recipes-kernel/linux/files/debian/isar/clean.tmpl
create mode 100644 meta/recipes-kernel/linux/files/debian/isar/common.tmpl
create mode 100644 meta/recipes-kernel/linux/files/debian/isar/install.tmpl
create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl
create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl
create mode 100755 meta/recipes-kernel/linux/files/debian/rules.tmpl
--
2.20.1
^ permalink raw reply [flat|nested] 83+ messages in thread* [PATCH 1/7] recipes-kernel/linux: make KERNEL_DEFCONFIG support in-tree defconfigs 2019-11-05 13:55 [PATCH 0/7] linux-custom recipe rework Cedric Hombourger @ 2019-11-05 13:55 ` Cedric Hombourger 2019-11-05 13:55 ` [PATCH 2/7] linux-mainline: fix stripping of .0 from the kernel version Cedric Hombourger ` (6 subsequent siblings) 7 siblings, 0 replies; 83+ messages in thread From: Cedric Hombourger @ 2019-11-05 13:55 UTC (permalink / raw) To: isar-users; +Cc: Cedric Hombourger The recipe expected the specified KERNEL_DEFCONFIG to be provided as a SRC_URI while the kernel already provides many useful defconfigs. The recipe will now check if the specified configuration can be found in WORKDIR. If it is found, it will then be used via "make olddefconfig" as before. It will otherwise assume that it is an in-tree defconfig and attempt a "make foobar_defconfig" (where foobar_defconfig is the user-provided KERNEL_DEFCONFIG setting). To support this change, a KERNEL_CONFIG_TARGET variable is passed to the build-kernel.sh script via the environment. Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com> --- meta/recipes-kernel/linux/files/build-kernel.sh | 6 +----- meta/recipes-kernel/linux/linux-custom.inc | 9 ++++++++- 2 files changed, 9 insertions(+), 6 deletions(-) diff --git a/meta/recipes-kernel/linux/files/build-kernel.sh b/meta/recipes-kernel/linux/files/build-kernel.sh index b46cefa..386e58b 100644 --- a/meta/recipes-kernel/linux/files/build-kernel.sh +++ b/meta/recipes-kernel/linux/files/build-kernel.sh @@ -36,11 +36,7 @@ REPACK_DIR="$1/../repack" REPACK_LINUX_IMAGE_DIR="${REPACK_DIR}/linux-image" REPACK_LINUX_HEADERS_DIR="${REPACK_DIR}/linux-headers" -if [ -e .config ]; then - make olddefconfig -else - make defconfig -fi +make ${KERNEL_CONFIG_TARGET} || exit ${?} KV=$( make -s kernelrelease ) if [ "${KV}" != "${PV}" ]; then diff --git a/meta/recipes-kernel/linux/linux-custom.inc b/meta/recipes-kernel/linux/linux-custom.inc index b597e25..0c185d8 100644 --- a/meta/recipes-kernel/linux/linux-custom.inc +++ b/meta/recipes-kernel/linux/linux-custom.inc @@ -61,8 +61,14 @@ addtask devshell after do_install_builddeps dpkg_runbuild() { chmod +x ${WORKDIR}/build-kernel.sh + KERNEL_CONFIG_TARGET="${KERNEL_DEFCONFIG}" if [ -n "${KERNEL_DEFCONFIG}" ]; then - cp ${WORKDIR}/${KERNEL_DEFCONFIG} ${S}/.config + if [ -e "${WORKDIR}/${KERNEL_DEFCONFIG}" ]; then + cp ${WORKDIR}/${KERNEL_DEFCONFIG} ${S}/.config + KERNEL_CONFIG_TARGET="olddefconfig" + fi + else + KERNEL_CONFIG_TARGET="defconfig" fi E="${@ bb.utils.export_proxies(d)}" @@ -71,6 +77,7 @@ dpkg_runbuild() { export KERNEL_NAME=${KERNEL_NAME_PROVIDED} export KBUILD_DEPENDS="${KBUILD_DEPENDS}" + export KERNEL_CONFIG_TARGET export KERNEL_DEBIAN_DEPENDS="${KERNEL_DEBIAN_DEPENDS}" export KERNEL_FILE="${KERNEL_FILE}" export KERNEL_HEADERS_DEBIAN_DEPENDS="${KERNEL_HEADERS_DEBIAN_DEPENDS}" -- 2.20.1 ^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH 2/7] linux-mainline: fix stripping of .0 from the kernel version 2019-11-05 13:55 [PATCH 0/7] linux-custom recipe rework Cedric Hombourger 2019-11-05 13:55 ` [PATCH 1/7] recipes-kernel/linux: make KERNEL_DEFCONFIG support in-tree defconfigs Cedric Hombourger @ 2019-11-05 13:55 ` Cedric Hombourger 2019-11-05 13:55 ` [PATCH 3/7] linux-mainline: update from 4.19.0 to 4.19.80 Cedric Hombourger ` (5 subsequent siblings) 7 siblings, 0 replies; 83+ messages in thread From: Cedric Hombourger @ 2019-11-05 13:55 UTC (permalink / raw) To: isar-users; +Cc: Cedric Hombourger The strip() function was used to strip the trailing .0 from the kernel version since it is omitted in the upstream archive file names. That function is actually taking a character set as argument not a substring. This was causing ARCHIVE_VERSION to expand to "4.19.8" when the user provided version was "4.19.80". The inline python code was changed to check if the version ends with ".0" and only then remove the last two characters. Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com> --- meta-isar/recipes-kernel/linux/linux-mainline_4.19.0.bb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/meta-isar/recipes-kernel/linux/linux-mainline_4.19.0.bb b/meta-isar/recipes-kernel/linux/linux-mainline_4.19.0.bb index e50e149..127f6e2 100644 --- a/meta-isar/recipes-kernel/linux/linux-mainline_4.19.0.bb +++ b/meta-isar/recipes-kernel/linux/linux-mainline_4.19.0.bb @@ -7,7 +7,7 @@ require recipes-kernel/linux/linux-custom.inc -ARCHIVE_VERSION = "${@d.getVar('PV').strip('.0')}" +ARCHIVE_VERSION = "${@ d.getVar('PV')[:-2] if d.getVar('PV').endswith('.0') else d.getVar('PV') }" SRC_URI += " \ https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-${ARCHIVE_VERSION}.tar.xz \ -- 2.20.1 ^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH 3/7] linux-mainline: update from 4.19.0 to 4.19.80 2019-11-05 13:55 [PATCH 0/7] linux-custom recipe rework Cedric Hombourger 2019-11-05 13:55 ` [PATCH 1/7] recipes-kernel/linux: make KERNEL_DEFCONFIG support in-tree defconfigs Cedric Hombourger 2019-11-05 13:55 ` [PATCH 2/7] linux-mainline: fix stripping of .0 from the kernel version Cedric Hombourger @ 2019-11-05 13:55 ` Cedric Hombourger 2019-11-05 13:55 ` [PATCH 4/7] linux-custom: add support for kernel config fragments Cedric Hombourger ` (4 subsequent siblings) 7 siblings, 0 replies; 83+ messages in thread From: Cedric Hombourger @ 2019-11-05 13:55 UTC (permalink / raw) To: isar-users; +Cc: Cedric Hombourger In preparation for a new machine addition, update the linux-mainline kernel from 4.19.0 to 4.19.80. This is needed to get some of the mips fixes that were integrated in this longterm kernel version (and in particular: MIPS: have "plain" make calls build dtbs for selected platforms, i.e. 637dfa0fad6d91a9a709dc70549a6d20fa77f615 upstream) Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com> --- .../{linux-mainline_4.19.0.bb => linux-mainline_4.19.80.bb} | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) rename meta-isar/recipes-kernel/linux/{linux-mainline_4.19.0.bb => linux-mainline_4.19.80.bb} (87%) diff --git a/meta-isar/recipes-kernel/linux/linux-mainline_4.19.0.bb b/meta-isar/recipes-kernel/linux/linux-mainline_4.19.80.bb similarity index 87% rename from meta-isar/recipes-kernel/linux/linux-mainline_4.19.0.bb rename to meta-isar/recipes-kernel/linux/linux-mainline_4.19.80.bb index 127f6e2..b41d1a8 100644 --- a/meta-isar/recipes-kernel/linux/linux-mainline_4.19.0.bb +++ b/meta-isar/recipes-kernel/linux/linux-mainline_4.19.80.bb @@ -12,7 +12,7 @@ ARCHIVE_VERSION = "${@ d.getVar('PV')[:-2] if d.getVar('PV').endswith('.0') else SRC_URI += " \ https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-${ARCHIVE_VERSION}.tar.xz \ file://x86_64_defconfig" -SRC_URI[sha256sum] = "0c68f5655528aed4f99dae71a5b259edc93239fa899e2df79c055275c21749a1" +SRC_URI[sha256sum] = "80a9ba764e088aa7fddfef5a97c0236905e291468a37832243b6f3828d36e7ec" SRC_URI_append_de0-nano-soc = " \ file://0001-ARM-dts-socfpga-Rename-socfpga_cyclone5_de0_-sockit-.patch" -- 2.20.1 ^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH 4/7] linux-custom: add support for kernel config fragments 2019-11-05 13:55 [PATCH 0/7] linux-custom recipe rework Cedric Hombourger ` (2 preceding siblings ...) 2019-11-05 13:55 ` [PATCH 3/7] linux-mainline: update from 4.19.0 to 4.19.80 Cedric Hombourger @ 2019-11-05 13:55 ` Cedric Hombourger 2019-11-05 13:57 ` [PATCH 5/7] linux-mainline: disable support for HFS to demonstrate use of " Cedric Hombourger ` (3 subsequent siblings) 7 siblings, 0 replies; 83+ messages in thread From: Cedric Hombourger @ 2019-11-05 13:55 UTC (permalink / raw) To: isar-users; +Cc: Cedric Hombourger Check for .cfg files listed in SRC_URI and merge them with the main defconfig using the kernel merge_config script. The optional "apply" parameter may be used for the specified fragment to be ignore (set to "no"). Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com> --- .../linux/files/build-kernel.sh | 3 +++ meta/recipes-kernel/linux/linux-custom.inc | 23 +++++++++++++++++++ 2 files changed, 26 insertions(+) diff --git a/meta/recipes-kernel/linux/files/build-kernel.sh b/meta/recipes-kernel/linux/files/build-kernel.sh index 386e58b..f56e96c 100644 --- a/meta/recipes-kernel/linux/files/build-kernel.sh +++ b/meta/recipes-kernel/linux/files/build-kernel.sh @@ -37,6 +37,9 @@ REPACK_LINUX_IMAGE_DIR="${REPACK_DIR}/linux-image" REPACK_LINUX_HEADERS_DIR="${REPACK_DIR}/linux-headers" make ${KERNEL_CONFIG_TARGET} || exit ${?} +if [ -n "${KERNEL_FRAGMENTS}" ]; then + scripts/kconfig/merge_config.sh -m .config ${KERNEL_FRAGMENTS} +fi KV=$( make -s kernelrelease ) if [ "${KV}" != "${PV}" ]; then diff --git a/meta/recipes-kernel/linux/linux-custom.inc b/meta/recipes-kernel/linux/linux-custom.inc index 0c185d8..ca91f64 100644 --- a/meta/recipes-kernel/linux/linux-custom.inc +++ b/meta/recipes-kernel/linux/linux-custom.inc @@ -18,6 +18,19 @@ python() { 'linux-headers-' + kernel_name) } +def config_fragments(d): + fragments = [] + sources = d.getVar("SRC_URI").split() + for s in sources: + _, _, local, _, _, parm = bb.fetch.decodeurl(s) + apply = parm.get("apply") + if apply == "no": + continue + base, ext = os.path.splitext(os.path.basename(local)) + if ext and ext in (".cfg"): + fragments.append(local) + return fragments + inherit dpkg-base SRC_URI += "file://build-kernel.sh" @@ -71,6 +84,16 @@ dpkg_runbuild() { KERNEL_CONFIG_TARGET="defconfig" fi + # copy config fragments over to the kernel tree + src_frags="${@ " ".join(config_fragments(d)) }" + out_frags="${@ " ".join(map(lambda frag: 'debian/fragments/' + frag, config_fragments(d))) }" + rm -rf ${S}/debian/fragments + if [ -n "${src_frags}" ]; then + mkdir -p ${S}/debian/fragments + (cd ${WORKDIR} && cp ${src_frags} ${S}/debian/fragments/) + fi + export KERNEL_FRAGMENTS="${out_frags}" + E="${@ bb.utils.export_proxies(d)}" export PV=${PV} -- 2.20.1 ^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH 5/7] linux-mainline: disable support for HFS to demonstrate use of config fragments 2019-11-05 13:55 [PATCH 0/7] linux-custom recipe rework Cedric Hombourger ` (3 preceding siblings ...) 2019-11-05 13:55 ` [PATCH 4/7] linux-custom: add support for kernel config fragments Cedric Hombourger @ 2019-11-05 13:57 ` Cedric Hombourger 2019-11-05 13:57 ` [PATCH 6/7] buildchroot-host: install qemu-static to support hybrid cross-compiles Cedric Hombourger 2019-11-05 13:57 ` [PATCH 7/7] linux-custom: rewrite to no longer depend on the kernel's builddeb Cedric Hombourger 2019-11-05 15:32 ` [PATCH 0/7] linux-custom recipe rework Jan Kiszka ` (2 subsequent siblings) 7 siblings, 2 replies; 83+ messages in thread From: Cedric Hombourger @ 2019-11-05 13:57 UTC (permalink / raw) To: isar-users; +Cc: Cedric Hombourger Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com> --- meta-isar/recipes-kernel/linux/files/no-fs-hfs.cfg | 2 ++ meta-isar/recipes-kernel/linux/linux-mainline_4.19.80.bb | 4 +++- 2 files changed, 5 insertions(+), 1 deletion(-) create mode 100644 meta-isar/recipes-kernel/linux/files/no-fs-hfs.cfg diff --git a/meta-isar/recipes-kernel/linux/files/no-fs-hfs.cfg b/meta-isar/recipes-kernel/linux/files/no-fs-hfs.cfg new file mode 100644 index 0000000..5dbfadb --- /dev/null +++ b/meta-isar/recipes-kernel/linux/files/no-fs-hfs.cfg @@ -0,0 +1,2 @@ +# CONFIG_FS_HFS is not set +# CONFIG_HFSPLUS_FS is not set diff --git a/meta-isar/recipes-kernel/linux/linux-mainline_4.19.80.bb b/meta-isar/recipes-kernel/linux/linux-mainline_4.19.80.bb index b41d1a8..83038f0 100644 --- a/meta-isar/recipes-kernel/linux/linux-mainline_4.19.80.bb +++ b/meta-isar/recipes-kernel/linux/linux-mainline_4.19.80.bb @@ -11,7 +11,9 @@ ARCHIVE_VERSION = "${@ d.getVar('PV')[:-2] if d.getVar('PV').endswith('.0') else SRC_URI += " \ https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-${ARCHIVE_VERSION}.tar.xz \ - file://x86_64_defconfig" + file://x86_64_defconfig \ + file://no-fs-hfs.cfg" + SRC_URI[sha256sum] = "80a9ba764e088aa7fddfef5a97c0236905e291468a37832243b6f3828d36e7ec" SRC_URI_append_de0-nano-soc = " \ -- 2.20.1 ^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH 6/7] buildchroot-host: install qemu-static to support hybrid cross-compiles 2019-11-05 13:57 ` [PATCH 5/7] linux-mainline: disable support for HFS to demonstrate use of " Cedric Hombourger @ 2019-11-05 13:57 ` Cedric Hombourger 2019-11-05 14:52 ` Jan Kiszka 2019-11-05 13:57 ` [PATCH 7/7] linux-custom: rewrite to no longer depend on the kernel's builddeb Cedric Hombourger 1 sibling, 1 reply; 83+ messages in thread From: Cedric Hombourger @ 2019-11-05 13:57 UTC (permalink / raw) To: isar-users; +Cc: Cedric Hombourger The Linux kernel has great support for cross-compiling the kernel image and modules. There is however no support/mechanism for cross-compiling the build tools from its "scripts" directory. While HOSTCC may be set to use our cross-compiler, the kernel build infrasture would then try to run foreign-arch binaries such as fixdep. The easiest and least intrusive way to support this is to enable execution of such binaries via binfmt/qemu like we do in the buildchroot-target environment. Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com> --- meta/recipes-devtools/buildchroot/buildchroot-host.bb | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/meta/recipes-devtools/buildchroot/buildchroot-host.bb b/meta/recipes-devtools/buildchroot/buildchroot-host.bb index 408ad39..2e76acb 100644 --- a/meta/recipes-devtools/buildchroot/buildchroot-host.bb +++ b/meta/recipes-devtools/buildchroot/buildchroot-host.bb @@ -15,3 +15,9 @@ BUILDCHROOT_PREINSTALL ?= " \ ${BUILDCHROOT_PREINSTALL_COMMON} \ libc6:${DISTRO_ARCH} \ crossbuild-essential-${DISTRO_ARCH}" + +buildchroot_install_files_append() { + if [ -e '/usr/bin/qemu-${QEMU_ARCH}-static' ]; then + sudo cp '/usr/bin/qemu-${QEMU_ARCH}-static' '${BUILDCHROOT_DIR}/usr/bin/qemu-${QEMU_ARCH}-static' + fi +} -- 2.20.1 ^ permalink raw reply [flat|nested] 83+ messages in thread
* Re: [PATCH 6/7] buildchroot-host: install qemu-static to support hybrid cross-compiles 2019-11-05 13:57 ` [PATCH 6/7] buildchroot-host: install qemu-static to support hybrid cross-compiles Cedric Hombourger @ 2019-11-05 14:52 ` Jan Kiszka 2019-11-05 15:00 ` Cedric Hombourger 0 siblings, 1 reply; 83+ messages in thread From: Jan Kiszka @ 2019-11-05 14:52 UTC (permalink / raw) To: Cedric Hombourger, isar-users On 05.11.19 14:57, Cedric Hombourger wrote: > The Linux kernel has great support for cross-compiling the kernel image > and modules. There is however no support/mechanism for cross-compiling the > build tools from its "scripts" directory. While HOSTCC may be set to use > our cross-compiler, the kernel build infrasture would then try to run > foreign-arch binaries such as fixdep. The easiest and least intrusive way > to support this is to enable execution of such binaries via binfmt/qemu > like we do in the buildchroot-target environment. What's the use case? Or why didn't we hit this issue so far? Jan > > Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com> > --- > meta/recipes-devtools/buildchroot/buildchroot-host.bb | 6 ++++++ > 1 file changed, 6 insertions(+) > > diff --git a/meta/recipes-devtools/buildchroot/buildchroot-host.bb b/meta/recipes-devtools/buildchroot/buildchroot-host.bb > index 408ad39..2e76acb 100644 > --- a/meta/recipes-devtools/buildchroot/buildchroot-host.bb > +++ b/meta/recipes-devtools/buildchroot/buildchroot-host.bb > @@ -15,3 +15,9 @@ BUILDCHROOT_PREINSTALL ?= " \ > ${BUILDCHROOT_PREINSTALL_COMMON} \ > libc6:${DISTRO_ARCH} \ > crossbuild-essential-${DISTRO_ARCH}" > + > +buildchroot_install_files_append() { > + if [ -e '/usr/bin/qemu-${QEMU_ARCH}-static' ]; then > + sudo cp '/usr/bin/qemu-${QEMU_ARCH}-static' '${BUILDCHROOT_DIR}/usr/bin/qemu-${QEMU_ARCH}-static' > + fi > +} > -- Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 83+ messages in thread
* Re: [PATCH 6/7] buildchroot-host: install qemu-static to support hybrid cross-compiles 2019-11-05 14:52 ` Jan Kiszka @ 2019-11-05 15:00 ` Cedric Hombourger 2019-11-05 15:27 ` Jan Kiszka 0 siblings, 1 reply; 83+ messages in thread From: Cedric Hombourger @ 2019-11-05 15:00 UTC (permalink / raw) To: Jan Kiszka, isar-users On 11/5/2019 3:52 PM, Jan Kiszka wrote: > On 05.11.19 14:57, Cedric Hombourger wrote: >> The Linux kernel has great support for cross-compiling the kernel image >> and modules. There is however no support/mechanism for cross-compiling the >> build tools from its "scripts" directory. While HOSTCC may be set to use >> our cross-compiler, the kernel build infrasture would then try to run >> foreign-arch binaries such as fixdep. The easiest and least intrusive way >> to support this is to enable execution of such binaries via binfmt/qemu >> like we do in the buildchroot-target environment. > What's the use case? Or why didn't we hit this issue so far? when building with the kernel builddeb script, the linux-headers package will include scripts/basic/fixdep compiled for amd64 even though your target architecture is e.g. armhf this went unnoticed because the kernel does not use debhelper and therefore does not benefit from the extra checks/stuff that it does while migrating to custom build/packaging scripts using debhelper, that packaging mistake caused dpkg-buildpackage to fail so things worked before by luck and also because nobody is actually doing on target development/builds (on non x86 machines that is). it is however desirable to have our linux-headers package for arch x (e.g. arm64) include arm64 binaries and not amd64 hope that makes more sense now > Jan > >> Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com> >> --- >> meta/recipes-devtools/buildchroot/buildchroot-host.bb | 6 ++++++ >> 1 file changed, 6 insertions(+) >> >> diff --git a/meta/recipes-devtools/buildchroot/buildchroot-host.bb b/meta/recipes-devtools/buildchroot/buildchroot-host.bb >> index 408ad39..2e76acb 100644 >> --- a/meta/recipes-devtools/buildchroot/buildchroot-host.bb >> +++ b/meta/recipes-devtools/buildchroot/buildchroot-host.bb >> @@ -15,3 +15,9 @@ BUILDCHROOT_PREINSTALL ?= " \ >> ${BUILDCHROOT_PREINSTALL_COMMON} \ >> libc6:${DISTRO_ARCH} \ >> crossbuild-essential-${DISTRO_ARCH}" >> + >> +buildchroot_install_files_append() { >> + if [ -e '/usr/bin/qemu-${QEMU_ARCH}-static' ]; then >> + sudo cp '/usr/bin/qemu-${QEMU_ARCH}-static' '${BUILDCHROOT_DIR}/usr/bin/qemu-${QEMU_ARCH}-static' >> + fi >> +} >> ^ permalink raw reply [flat|nested] 83+ messages in thread
* Re: [PATCH 6/7] buildchroot-host: install qemu-static to support hybrid cross-compiles 2019-11-05 15:00 ` Cedric Hombourger @ 2019-11-05 15:27 ` Jan Kiszka 0 siblings, 0 replies; 83+ messages in thread From: Jan Kiszka @ 2019-11-05 15:27 UTC (permalink / raw) To: Cedric Hombourger, isar-users On 05.11.19 16:00, Cedric Hombourger wrote: > On 11/5/2019 3:52 PM, Jan Kiszka wrote: >> On 05.11.19 14:57, Cedric Hombourger wrote: >>> The Linux kernel has great support for cross-compiling the kernel image >>> and modules. There is however no support/mechanism for >>> cross-compiling the >>> build tools from its "scripts" directory. While HOSTCC may be set to use >>> our cross-compiler, the kernel build infrasture would then try to run >>> foreign-arch binaries such as fixdep. The easiest and least intrusive >>> way >>> to support this is to enable execution of such binaries via binfmt/qemu >>> like we do in the buildchroot-target environment. >> What's the use case? Or why didn't we hit this issue so far? > > when building with the kernel builddeb script, the linux-headers package > will include scripts/basic/fixdep compiled for amd64 even though your > target architecture is e.g. armhf > > this went unnoticed because the kernel does not use debhelper and > therefore does not benefit from the extra checks/stuff that it does > > while migrating to custom build/packaging scripts using debhelper, that > packaging mistake caused dpkg-buildpackage to fail > > so things worked before by luck and also because nobody is actually > doing on target development/builds (on non x86 machines that is). it is > however desirable to have our linux-headers package for arch x (e.g. > arm64) include arm64 binaries and not amd64 > > hope that makes more sense now See my comment on patch 7: This here is also only papering over the real issue. Jan -- Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH 7/7] linux-custom: rewrite to no longer depend on the kernel's builddeb 2019-11-05 13:57 ` [PATCH 5/7] linux-mainline: disable support for HFS to demonstrate use of " Cedric Hombourger 2019-11-05 13:57 ` [PATCH 6/7] buildchroot-host: install qemu-static to support hybrid cross-compiles Cedric Hombourger @ 2019-11-05 13:57 ` Cedric Hombourger 2019-11-05 15:26 ` Jan Kiszka 2019-11-07 11:44 ` Gylstorff Quirin 1 sibling, 2 replies; 83+ messages in thread From: Cedric Hombourger @ 2019-11-05 13:57 UTC (permalink / raw) To: isar-users; +Cc: Cedric Hombourger Regain control over the packaging of the Linux kernel by providing our own debian recipes (debian,{control,rules} files and friends) instead of using the kernel's builddeb script. This will allow generation of packages for "perf" (not included in this changeset) with per distro scheme (Debian and Ubuntu have differences) and maintain compatibility with upstream packages generated from different source packages but dependent on a certain deployment scheme (e.g. linux-base for Debian) Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com> --- doc/custom_kernel.md | 181 ++++++++++++++++ .../linux/files/build-kernel.sh | 128 ------------ meta/recipes-kernel/linux/files/debian/compat | 1 + .../linux/files/debian/control.tmpl | 36 ++++ meta/recipes-kernel/linux/files/debian/files | 5 + .../linux/files/debian/isar/build.tmpl | 36 ++++ .../linux/files/debian/isar/clean.tmpl | 20 ++ .../linux/files/debian/isar/common.tmpl | 60 ++++++ .../linux/files/debian/isar/install.tmpl | 197 ++++++++++++++++++ .../files/debian/linux-image.postinst.tmpl | 22 ++ .../files/debian/linux-image.postrm.tmpl | 18 ++ .../linux/files/debian/rules.tmpl | 39 ++++ meta/recipes-kernel/linux/linux-custom.inc | 172 ++++++++++----- 13 files changed, 732 insertions(+), 183 deletions(-) create mode 100644 doc/custom_kernel.md delete mode 100644 meta/recipes-kernel/linux/files/build-kernel.sh create mode 100644 meta/recipes-kernel/linux/files/debian/compat create mode 100644 meta/recipes-kernel/linux/files/debian/control.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/files create mode 100644 meta/recipes-kernel/linux/files/debian/isar/build.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/isar/clean.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/isar/common.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/isar/install.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl create mode 100755 meta/recipes-kernel/linux/files/debian/rules.tmpl diff --git a/doc/custom_kernel.md b/doc/custom_kernel.md new file mode 100644 index 0000000..5ab2599 --- /dev/null +++ b/doc/custom_kernel.md @@ -0,0 +1,181 @@ +# Custom kernel recipe for Isar + +## Contents + + - [Summary](#summary) + - [Proposal owners](#proposal-owners) + - [Current status](#current-status) + - [Detailed description](#detailed-description) + - [Benefit to Isar](#benefit-to-isar) + - [How can I help?](#how-can-i-help) + - [Scope](#scope) + - [How to test](#how-to-test) + - [Dependencies](#dependencies) + - [Documentation](#documentation) + - [Questions and answers](#questions-and-answers) + +## Summary + +Isar currently uses the Linux kernel's builddeb script to generate linux-image, +linux-headers and linux-libc-dev packages for your custom kernel sources. The +main benefit of this approach was that the low level details of how to package +the Linux kernel image, modules and headers were left in the kernel. There are +however some drawbacks: Isar had to repack generated packages to introduce some +of the changes it needs such as suffixing the kernel packages with the name of +the custom kernel (e.g. with "mainline" for the "linux-mainline" recipe), lack +of automatic dependencies (the builddeb script from the kernel does not use +debhelper), it was difficult to add packages especially distribution specific +packages (maintain version specific patch in Isar vs being able to upstream +distribution specific changes upstream), etc. This change is about being less +dependent on the kernel packaging scripts (which aren't used by major distros +anyway) and regain control on how we want or need the Linux kernel packaged. + +## Proposal owners + + * name: [Cedric Hombourger](https://github.com/chombourger) + +## Current status + +### Tests + +The following were recently checked: + + * Custom kernels in meta-isar build without (known) failures + * initrd image gets generated when the kernel is added to the rootfs + * `ISAR_CROSS_COMPILE` is supported + * `KERNEL_FILE` is honored + * `KERNEL_DEFCONFIG` may either be a file (specified via `SRC_URI`) or a + kernel-provided defconfig + * Config fragments get merged with the user-specified kernel configuration + * Path to installed DTBs may be configured (new) + * Configuration and build from a `devshell` + * Support for the `KERNEL_LIBC_DEV_DEPLOY` directive + +while the following needs work: + + * Check if custom-kernel builds are reproducible + +### Remaining work items + + * Check if there is a better way to cross compile kernel "scripts" such as + `fixdep` or `modpost` than manually compiling them (they are compiled for + the build machine (amd64) and not for the target (e.g. armhf) even though + we are cross-compiling - yet the linux-headers package should be shipping + "scripts" for the target machine) + +## Detailed description + +### Requirements + +#### Current + +Isar has had a `linux-custom` recipe for a long time and counts many users. It is therefore +paramount to maintain features that it has today with no or little changes to the APIs. The +following requirements were identified and will be maintained: + + 1. Sources to the custom Linux kernel may be specified via `SRC_URI` + + 2. Configure the kernel via an in-tree or an external `defconfig` + + 3. Support kernel configuration tweaks via configuration fragments (`.cfg` files) + + 4. Patches to the linux kernel may be specified via `SRC_URI` + + 5. Ensure that the Isar recipe `PV` matches the kernel release identifier (`KERNEL_RELEASE`) + + 6. Produce a `linux-image` package that ships the kernel image and modules + + 7. Allow the name of the kernel image to be changed via `KERNEL_FILE` (defaults to `vmlinuz`) + + 8. Produce a `linux-headers` package which includes kernel headers and kbuild scripts/tools + + 9. The `linux-headers` package shall support native and cross compiles of out-of-tree kernel modules + + 10. Produce a `linux-libc-dev` package to support user-land builds + + 11. Only ship the `linux-libc-dev` package if instructed to (`KERNEL_LIBC_DEV_DEPLOY` equals to `"1"`) + + 12. Support both native and cross compiles (`ISAR_CROSS_COMPILE`) + + 13. Support for the following kernel architectures: + + * arm + * arm64 + * mips + * x86 + + 14. Support `devshell` (kernel configuration shall be applied) + +#### New requirements (proposed) + +This proposal includes the following new requirements: + + 1. Support creation of additional packages + + 2. Allow per-distro packaging tweaks + +### Approach + +The new implementation will have the `builddeb` code ported from the Linux kernel included to the +Isar environment. It will be refactor to support pre and post hooks. Hooks may be used by external +layers to extend the kernel packaging instructions provided by "stock" Isar or by Isar itself to +implement distro-specific packages or tweaks. + +Hooks need the ability to add `build` and `install` steps which can be achieved by having the main +`build` and `install` steps call pre and post scripts. This will however solve only one side of the +problem: new packages should be declared in the `debian/control` file. The packaging process +(initiated by `debian/rules`) will use a baseline `control` file when started and a mechanism will +be provided for hooks to append `control` blocks as they are processed. + +### Future work + +In the event where this proposal is accepted, here are a few future projects that are being +considered: + + 1. Package `perf` as Debian does + + 2. Add `ubuntu` as an Isar supported target distro and validate `linux-custom` recipe + +## Benefit to Isar + + * Control the kernel build and packaging process + + * No longer need to repack packages + + * Provide hooks for custom kernel recipes to add build/install directives + +## How can I help? + + * Check if you are able to build your own linux-custom recipes with this change + * Check if packages generated by this recipe are compatible with your Debian-based distro + * Review and provide comments on this changeset + * Create templates for distro-specific packages (e.g. perf) + +## Scope + + * Proposal owners: proposal owners are to communicate the idea to isar-users and make sure + that all of their requirements are met (to the extent they are already supported with the + current solution) or may be met in the future. + + * Other developers: port your out-of-tree linux-custom recipes to this new solution and + report issues. + +## How to test? + +The linux-mainline recipe may be used for some basic testing. This recipe is being used by the +following machines: + + * de0-nano-soc + * qemumipsel + +## Dependencies + + * None + +## Documentation + + * Document user-visible variables added by this proposal + +## Questions and answers + + * None diff --git a/meta/recipes-kernel/linux/files/build-kernel.sh b/meta/recipes-kernel/linux/files/build-kernel.sh deleted file mode 100644 index f56e96c..0000000 --- a/meta/recipes-kernel/linux/files/build-kernel.sh +++ /dev/null @@ -1,128 +0,0 @@ -#!/bin/bash -# -# Custom kernel build -# -# This software is a part of ISAR. -# Copyright (c) Siemens AG, 2018 -# -# SPDX-License-Identifier: MIT - -source /isar/common.sh - -host_arch=$(dpkg --print-architecture) - -if [ "$host_arch" != "$target_arch" ]; then - case $target_arch in - armhf) - export ARCH=arm - export CROSS_COMPILE="arm-linux-gnueabihf-" - ;; - arm64) - export ARCH=arm64 - export CROSS_COMPILE="aarch64-linux-gnu-" - ;; - mipsel) - export ARCH=mips - export CROSS_COMPILE="mipsel-linux-gnu-" - ;; - *) - echo "error: unsupported architecture ($target_arch)" - exit 1 - ;; - esac -fi - -REPACK_DIR="$1/../repack" -REPACK_LINUX_IMAGE_DIR="${REPACK_DIR}/linux-image" -REPACK_LINUX_HEADERS_DIR="${REPACK_DIR}/linux-headers" - -make ${KERNEL_CONFIG_TARGET} || exit ${?} -if [ -n "${KERNEL_FRAGMENTS}" ]; then - scripts/kconfig/merge_config.sh -m .config ${KERNEL_FRAGMENTS} -fi - -KV=$( make -s kernelrelease ) -if [ "${KV}" != "${PV}" ]; then - echo "ERROR: Recipe PV is \"${PV}\" but should be \"${KV}\"" 1>&2 - echo "ERROR: Probably due to CONFIG_LOCALVERSION" 1>&2 - exit 1 -fi - -rm -f .version -KBUILD_DEBARCH=$target_arch make -j $(($(nproc) * 2)) deb-pkg - -rm -rf "${REPACK_DIR}" -mkdir -p "${REPACK_DIR}" -mkdir -p "${REPACK_LINUX_IMAGE_DIR}" -mkdir -p "${REPACK_LINUX_HEADERS_DIR}" - -cp -a debian "${REPACK_DIR}" - -# dpkg-gencontrol performs cross-incompatible checks on the -# Architecture field; trick it to accept the control file -sed -i "s/Architecture: .*/Architecture: any/" "${REPACK_DIR}/debian/control" - -cd .. - -dpkg-deb -R linux-image-${PV}_${PV}-1_*.deb "${REPACK_LINUX_IMAGE_DIR}" -dpkg-deb -R linux-headers-${PV}_${PV}-1_*.deb "${REPACK_LINUX_HEADERS_DIR}" - -dpkg-gencontrol -crepack/debian/control \ - -lrepack/debian/changelog \ - -frepack/debian/files \ - -plinux-image-${PV} \ - -P"${REPACK_LINUX_IMAGE_DIR}" \ - -DPackage="linux-image-${KERNEL_NAME}" \ - -DSection=kernel \ - -DPriority=required \ - -DDepends="${KERNEL_DEBIAN_DEPENDS}" \ - -DArchitecture=$target_arch - -# Add Debian-like link installation to postinst -mkdir -p ${REPACK_LINUX_IMAGE_DIR}/lib/modules/${PV} -touch "${REPACK_LINUX_IMAGE_DIR}/lib/modules/${PV}/.fresh-install" -sed -i "${REPACK_LINUX_IMAGE_DIR}/DEBIAN/postinst" \ - -e "/^set -e$/a\\ -\\ -if [ -f /lib/modules/${PV}/.fresh-install ]; then\\ - change=install\\ -else\\ - change=upgrade\\ -fi\\ -linux-update-symlinks \$change ${PV} /boot/${KERNEL_FILE}-${PV}\\ -rm -f /lib/modules/${PV}/.fresh-install" - -# Add Debian-like link removal to postrm -sed -i "${REPACK_LINUX_IMAGE_DIR}/DEBIAN/postrm" \ - -e "/^set -e$/a\\ -\\ -rm -f /lib/modules/${PV}/.fresh-install\\ -\\ -if [ \"\$1\" != upgrade ] && command -v linux-update-symlinks >/dev/null; then\\ - linux-update-symlinks remove ${PV} /boot/${KERNEL_FILE}-${PV}\\ -fi" - -# Make sure arm64 kernels are decompressed -if [ "$target_arch" = "arm64" ]; then - kernel_file="${REPACK_LINUX_IMAGE_DIR}/boot/${KERNEL_FILE}-${PV}" - mv "${kernel_file}" "${kernel_file}.gz" - gunzip "${kernel_file}.gz" -fi - -dpkg-gencontrol -crepack/debian/control \ - -lrepack/debian/changelog \ - -frepack/debian/files \ - -plinux-headers-${PV} \ - -P"${REPACK_LINUX_HEADERS_DIR}" \ - -Vkernel:debarch="${KERNEL_NAME}" \ - -DPackage="linux-headers-${KERNEL_NAME}" \ - -DSection=kernel \ - -DDepends="${KERNEL_HEADERS_DEBIAN_DEPENDS}" \ - -DArchitecture=$target_arch - -fakeroot dpkg-deb -b "${REPACK_LINUX_IMAGE_DIR}" \ - linux-image-${KERNEL_NAME}_${PV}-1_${KERNEL_NAME}.deb -rm -f linux-image-${PV}_${PV}-1_*.deb -fakeroot dpkg-deb -b "${REPACK_LINUX_HEADERS_DIR}" \ - linux-headers-${KERNEL_NAME}_${PV}-1_${KERNEL_NAME}.deb -rm -f linux-headers-${PV}_${PV}-1_*.deb diff --git a/meta/recipes-kernel/linux/files/debian/compat b/meta/recipes-kernel/linux/files/debian/compat new file mode 100644 index 0000000..ec63514 --- /dev/null +++ b/meta/recipes-kernel/linux/files/debian/compat @@ -0,0 +1 @@ +9 diff --git a/meta/recipes-kernel/linux/files/debian/control.tmpl b/meta/recipes-kernel/linux/files/debian/control.tmpl new file mode 100644 index 0000000..1295466 --- /dev/null +++ b/meta/recipes-kernel/linux/files/debian/control.tmpl @@ -0,0 +1,36 @@ +Source: linux-${KERNEL_NAME_PROVIDED} +Section: kernel +Priority: optional +Maintainer: ${MAINTAINER} +Build-Depends: bc, kmod, cpio, ${KBUILD_DEPENDS} +Homepage: http://www.kernel.org/ + +Package: linux-image-${KERNEL_NAME_PROVIDED} +Architecture: any +Depends: ${KERNEL_DEBIAN_DEPENDS} +Description: ${KERNEL_NAME_PROVIDED} Linux kernel, version ${PV} + This package contains the Linux kernel, modules and corresponding other + files, version: ${PV}. + +Package: linux-headers-${KERNEL_NAME_PROVIDED} +Architecture: any +Depends: ${KERNEL_HEADERS_DEBIAN_DEPENDS}, ${perl:Depends}, ${shlib:Depends} +Description: ${KERNEL_NAME_PROVIDED} Linux kernel headers for ${PV} + This package provides kernel header files for ${PV} on ${DISTRO_ARCH} + . + This is useful for people who need to build external modules + +Package: linux-libc-dev +Section: devel +Provides: linux-kernel-headers +Architecture: any +Description: Linux support headers for userspace development + This package provides userspaces headers from the Linux kernel. These headers + are used by the installed headers for GNU glibc and other system libraries. + +Package: linux-image-${KERNEL_NAME_PROVIDED}-dbg +Section: debug +Architecture: any +Description: Linux kernel debugging symbols for ${PV} + This package will come in handy if you need to debug the kernel. It provides + all the necessary debug symbols for the kernel and its modules. diff --git a/meta/recipes-kernel/linux/files/debian/files b/meta/recipes-kernel/linux/files/debian/files new file mode 100644 index 0000000..a8ace32 --- /dev/null +++ b/meta/recipes-kernel/linux/files/debian/files @@ -0,0 +1,5 @@ +linux-headers_4.19.80-1_mipsel.deb kernel optional +linux-image-dbg_4.19.80-1_mipsel.deb debug optional +linux-image_4.19.80-1_mipsel.deb kernel optional +linux-libc-dev_4.19.80-1_mipsel.deb devel optional +linux_4.19.80-1_mipsel.buildinfo kernel optional diff --git a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl new file mode 100644 index 0000000..ce79d56 --- /dev/null +++ b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl @@ -0,0 +1,36 @@ +#!/bin/sh +# Copyright (c) Mentor Graphics, a Siemens business, 2019 +# SPDX-License-Identifier: MIT + +# Load common stuff +. ${S}/debian/isar/common || exit ${?} + +do_build() { + + # Print a few things that are of particular interest + print_settings + + # Trace what we do here + set -x + + # Process existing kernel configuration to make sure it is complete + # (use defaults for options that were not specified) + ${MAKE} O=${KERNEL_BUILD_DIR} olddefconfig || exit ${?} + + # Build the Linux kernel + ${MAKE} O=${KERNEL_BUILD_DIR} -j $(nproc) || exit ${?} + + # Stop tracing + set +x +} + +print_settings() { + cat <<EOF +# Build settings: +# --------------- +# ARCH=${ARCH} +# CROSS_COMPILE=${CROSS_COMPILE} +EOF +} + +main build ${*} diff --git a/meta/recipes-kernel/linux/files/debian/isar/clean.tmpl b/meta/recipes-kernel/linux/files/debian/isar/clean.tmpl new file mode 100644 index 0000000..2aa3742 --- /dev/null +++ b/meta/recipes-kernel/linux/files/debian/isar/clean.tmpl @@ -0,0 +1,20 @@ +#!/bin/sh +# Copyright (c) Mentor Graphics, a Siemens business, 2019 +# SPDX-License-Identifier: MIT + +# Load common stuff +. ${S}/debian/isar/common || exit ${?} + +do_clean() { + + # Trace what we do here + set -x + + rm -rf ${deb_img_dir} ${deb_dbg_dir} + rm -rf ${deb_kern_hdrdir} ${deb_libc_hdr_dir} + + # Stop tracing + set +x +} + +main clean ${*} diff --git a/meta/recipes-kernel/linux/files/debian/isar/common.tmpl b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl new file mode 100644 index 0000000..b392b46 --- /dev/null +++ b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl @@ -0,0 +1,60 @@ +#!/bin/sh +# Copyright (c) Mentor Graphics, a Siemens business, 2019 +# SPDX-License-Identifier: MIT + +# Isar settings +ARCH=${KERNEL_ARCH} +KERNEL_PKG_IMAGE=linux-image-${KERNEL_NAME_PROVIDED} +KERNEL_PKG_KERN_HEADERS=linux-headers-${KERNEL_NAME_PROVIDED} +KERNEL_PKG_LIBC_HEADERS=linux-libc-dev + +# Constants +KCONF=.config + +# Target directories +deb_top_dir=${S}/debian +deb_img_dir=${deb_top_dir}/${KERNEL_PKG_IMAGE} +deb_dbg_dir=${deb_img_dir}-dbg +deb_dtb_dir=${deb_img_dir}/${KERNEL_DTB_DIR} +deb_kern_hdr_dir=${deb_top_dir}/${KERNEL_PKG_KERN_HEADERS} +deb_libc_hdr_dir=${deb_top_dir}/${KERNEL_PKG_LIBC_HEADERS} + +main() { + target=${1} + + if [ ! -f ${S}/debian/isar/${target} ]; then + echo "error: ${target} is not a supported build target!" >&2 + return 1 + fi + + # check for scripts to be sourced + for f in ${S}/debian/isar/defaults.d/${target}/*; do + [ -f ${f} ] || continue + . ${f} || return ${?} + done + + # variables to be exported + export ARCH + + # are we cross-compiling? + BUILD_ARCH=$(dpkg-architecture -qDEB_BUILD_ARCH) + if [ "${BUILD_ARCH}" = "${DISTRO_ARCH}" ]; then + # no, make sure CROSS_COMPILE isn't set + unset CROSS_COMPILE + fi + + # check for pre-target scripts + for f in ${S}/debian/isar/pre.d/${target}/*; do + [ -f ${f} ] || continue + sh ${f} || return ${?} + done + + # call the actual target script + do_${target} || return ${?} + + # check for post-target scripts + for f in ${S}/debian/isar/post.d/${target}/*; do + [ -f ${f} ] || continue + sh ${f} || return ${?} + done +} diff --git a/meta/recipes-kernel/linux/files/debian/isar/install.tmpl b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl new file mode 100644 index 0000000..2055669 --- /dev/null +++ b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl @@ -0,0 +1,197 @@ +#!/bin/sh +# Copyright (c) Mentor Graphics, a Siemens business, 2019 +# SPDX-License-Identifier: MIT + +# Load common stuff +. ${S}/debian/isar/common || exit ${?} + +do_install() { + + # check if our kernel was configured + if [ ! -f "${O}/.config" ]; then + echo "error: kernel not configured!" >&2 + return 1 + fi + + # load its configuration + . ${O}/.config + + kimage="$(${MAKE} O=${O} -s --no-print-directory image_name)" + case "${ARCH}" in + mips|powerpc) kimage_path="boot/vmlinux-${PV}" ;; + um) kimage_path="usr/bin/vmlinux-${PV}" ;; + *) kimage_path="boot/vmlinuz-${PV}" ;; + esac + + print_settings + + # Stop on error + set -e + + # Trace what we do here + set -x + + install_image + install_hooks + install_dtbs + install_kmods + install_headers + + # Stop tracing + set +x +} + +print_settings() { + cat <<EOF +Install settings: +----------------- +deb_dtb_dir=${deb_dtb_dir} +deb_hdr_dir=${deb_hdr_dir} +kimage=${kimage} +kimage_path=${kimage_path} + +EOF +} + +install_image() { + install -m 755 -d ${deb_img_dir}/$(dirname ${kimage_path}) + cp ${O}/${kimage} ${deb_img_dir}/${kimage_path} + install_image_debug +} + +install_image_debug() { + # Different tools want the image in different locations + # perf + mkdir -p ${deb_dbg_dir}/usr/lib/debug/lib/modules/${PV}/ + cp vmlinux ${deb_dbg_dir}/usr/lib/debug/lib/modules/${PV}/ + # systemtap + mkdir -p ${deb_dbg_dir}/usr/lib/debug/boot/ + ln -s ../lib/modules/$version/vmlinux ${deb_dbg_dir}/usr/lib/debug/boot/vmlinux-${PV} + # kdump-tools + ln -s lib/modules/${PV}/vmlinux ${deb_dbg_dir}/usr/lib/debug/vmlinux-${PV} +} + +install_hooks() { + install -m 755 -d ${deb_img_dir}/etc/kernel/install.d + install -m 755 -d ${deb_img_dir}/etc/kernel/postinst.d + install -m 755 -d ${deb_img_dir}/etc/kernel/postrm.d + install -m 755 -d ${deb_img_dir}/etc/kernel/prerm.d +} + +install_dtbs() { + [ -n "${CONFIG_OF}" ] || return 0 + ${MAKE} O=${O} INSTALL_DTBS_PATH=${deb_dtb_dir} dtbs_install +} + +install_kmods() { + [ -n "${CONFIG_MODULES}" ] || return 0 + ${MAKE} O=${O} INSTALL_MOD_PATH=${deb_img_dir} modules_install + touch ${deb_img_dir}/lib/modules/${PV}/.fresh-install + rm -fv ${deb_img_dir}/lib/modules/${PV}/build + rm -fv ${deb_img_dir}/lib/modules/${PV}/source + install_kmods_debug +} + +kmods_sign() { + [ -n "${CONFIG_MODULE_SIG_ALL}" ] || return 0 + ${MAKE} O=${O} INSTALL_MOD_PATH=${deb_img_dir} modules_sign +} + +install_kmods_debug() { + [ -n "${CONFIG_DEBUG_INFO}" ] || return 0 + + kmod_inst_dir=${deb_img_dir}/lib/modules + kmod_debug_dir=${deb_dbg_dir}/usr/lib/debug + + # copy kernels modules to usr/lib/debug + mkdir -p ${kmod_debug_dir} + tar -C ${kmod_inst_dir}/ -cO --exclude='modules.*' . | tar -C ${kmod_debug_dir}/ -xf - + # strip everything but debug sections for modules in usr/lib/debug + find ${kmod_debug_dir} -name *.ko -exec ${CROSS_COMPILE}objcopy --only-keep-debug {} \; + # and strip debug sections from modules in lib/modules + find ${kmod_inst_dir} -name *.ko -exec ${CROSS_COMPILE}objcopy --strip-debug {} \; + + # re-sign stripped kernel modules + kmods_sign +} + +headers_check() { + ${MAKE} O=${O} headers_check +} + +libc_headers() { + mkdir -p ${deb_libc_hdr_dir} + ${MAKE} O=${O} headers_install INSTALL_HDR_PATH=${deb_libc_hdr_dir}/usr + host_arch=$(dpkg-architecture -a${DISTRO_ARCH} -qDEB_HOST_MULTIARCH) + mkdir ${deb_libc_hdr_dir}/usr/include/${host_arch} + mv ${deb_libc_hdr_dir}/usr/include/asm ${deb_libc_hdr_dir}/usr/include/${host_arch}/ +} + +# the kernel does not seem to provide a way to cross-compile its tools, we however +# have qemu-user-static binaries in our buildchroot. Create a new build tree for the +# scripts and override HOSTCC to force a cross-compile +kernel_tools_build() { + odir="${S}/build-scripts" + mflags="HOSTCC=${CROSS_COMPILE}gcc O=${odir} NOSTDINC_FLAGS=" + rm -rf ${odir} && mkdir -p ${odir} && cp ${S}/${KERNEL_BUILD_DIR}/.config ${odir} + (unset LD_PRELOAD; cd ${odir} && make -C ${S} ${mflags} olddefconfig scripts) + (cd ${odir}; find scripts -type f -executable -exec file {} \;|grep ELF|cut -d: -f1) >${src_hdr_files} + tar -C ${odir} -cf - -T - <${src_hdr_files} | tar -C ${destdir} -xf - +} + +kernel_tools() { + # remove object files + find ${destdir}/scripts -type f -name '*.o' |xargs rm -f + + # we're all done if we aren't cross-compiling + [ -n "${CROSS_COMPILE}" ] || return 0 + + # remove ELF executables from the linux-headers package + find ${destdir}/scripts -type f -exec file {} \;|grep ELF|cut -d: -f1|xargs rm -fv + + # cross-compile kernel tools to be shipped with linux-headers + kernel_tools_build +} + +kernel_headers() { + destdir=${deb_kern_hdr_dir}/${KERNEL_HEADERS_DIR} + src_hdr_files=$(mktemp) + obj_hdr_files=$(mktemp) + + mkdir -p ${destdir} + mkdir -p ${deb_kern_hdr_dir}/lib/modules/${PV} + + (cd ${S}; find . -name 'Makefile*' -o -name 'Kconfig*' -o -name '*.pl') >>${src_hdr_files} + (cd ${S}; find arch/*/include include scripts -type f -o -type l) >>${src_hdr_files} + (cd ${S}; find arch/${ARCH} -name module.lds -o -name Kbuild.platforms -o -name Platform) >>${src_hdr_files} + (cd ${S}; find $(find arch/${ARCH} -name include -o -name scripts -type d) -type f) >>${src_hdr_files} + + (cd ${O}; find arch/${ARCH}/include Module.symvers include scripts -type f) >>${obj_hdr_files} + if [ -n "${CONFIG_STACK_VALIDATION}" ]; then + (cd ${O}; find tools/objtool -type f -executable) >>${obj_hdr_files} + fi + if [ -n "${CONFIG_GCC_PLUGINS}" ]; then + (cd ${O}; find scripts/gcc-plugins -name *.so -o -name gcc-common.h) >>${obj_hdr_files} + fi + + # deploy files that were matched above + tar -C ${S} -cf - -T - <${src_hdr_files} | tar -C ${destdir} -xf - + tar -C ${O} -cf - -T - <${obj_hdr_files} | tar -C ${destdir} -xf - + + # add the kernel config + cp ${O}/${KCONF} ${destdir}/.config + + # handle kernel development tools + kernel_tools + + # create symlinks + ln -sf /${KERNEL_HEADERS_DIR} ${deb_kern_hdr_dir}/lib/modules/${PV}/build +} + +install_headers() { + headers_check + libc_headers + kernel_headers +} + +main install ${*} diff --git a/meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl b/meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl new file mode 100644 index 0000000..fd89121 --- /dev/null +++ b/meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl @@ -0,0 +1,22 @@ +#!/bin/sh + +set -e +set -x + +if [ -f /lib/modules/${PV}/.fresh-install ]; then + change=install +else + change=upgrade +fi + +linux-update-symlinks $change ${PV} /boot/${KERNEL_FILE}-${PV} +rm -f /lib/modules/${PV}/.fresh-install + +# Pass maintainer script parameters to hook scripts +export DEB_MAINT_PARAMS="$*" + +# Tell initramfs builder whether it's wanted +export INITRD=Yes + +test -d /etc/kernel/postinst.d && run-parts --arg="${PV}" --arg="/boot/${KERNEL_FILE}-${PV}" /etc/kernel/postinst.d +exit 0 diff --git a/meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl b/meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl new file mode 100644 index 0000000..9d88218 --- /dev/null +++ b/meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl @@ -0,0 +1,18 @@ +#!/bin/sh + +set -e + +rm -f /lib/modules/${PV}/.fresh-install + +if [ "$1" != upgrade ] && command -v linux-update-symlinks >/dev/null; then + linux-update-symlinks remove ${PV} /boot/${KERNEL_FILE}-${PV} +fi + +# Pass maintainer script parameters to hook scripts +export DEB_MAINT_PARAMS="$*" + +# Tell initramfs builder whether it's wanted +export INITRD=Yes + +test -d /etc/kernel/postrm.d && run-parts --arg="${PV}" --arg="/boot/${KERNEL_FILE}-${PV}" /etc/kernel/postrm.d +exit 0 diff --git a/meta/recipes-kernel/linux/files/debian/rules.tmpl b/meta/recipes-kernel/linux/files/debian/rules.tmpl new file mode 100755 index 0000000..93c8a5b --- /dev/null +++ b/meta/recipes-kernel/linux/files/debian/rules.tmpl @@ -0,0 +1,39 @@ +#!/usr/bin/make -f + +export CROSS_COMPILE=$(DEB_HOST_GNU_TYPE)- + +O:=$(CURDIR)/${KERNEL_BUILD_DIR} +S:=$(CURDIR) +deb_top_dir:=$(S)/debian + +DH_VERBOSE=1 + +# Dynamic variables to be passed to Isar build scripts +isar_env=$(strip \ + export CROSS_COMPILE='$(CROSS_COMPILE)' && \ + export MAKE='$(MAKE)' && \ + export O='${O}' \ + export S='${S}' \ +) + +%: + dh $(@) + +.PHONY: override_dh_auto_clean +override_dh_auto_clean: + $(isar_env) && sh $(deb_top_dir)/isar/clean + +.PHONY: override_dh_auto_build +override_dh_auto_build: + $(isar_env) && sh $(deb_top_dir)/isar/build + +override_dh_auto_install: + $(isar_env) && sh $(deb_top_dir)/isar/install + +.PHONY: override_dh_auto_test +override_dh_auto_test: + true + +PHONY: override_dh_strip +override_dh_strip: + dh_strip -Xvmlinux --no-automatic-dbgsym diff --git a/meta/recipes-kernel/linux/linux-custom.inc b/meta/recipes-kernel/linux/linux-custom.inc index ca91f64..36086be 100644 --- a/meta/recipes-kernel/linux/linux-custom.inc +++ b/meta/recipes-kernel/linux/linux-custom.inc @@ -2,22 +2,104 @@ # # This software is a part of ISAR. # Copyright (c) Siemens AG, 2018 +# Copyright (c) Mentor Graphics, a Siemens business, 2019 # # SPDX-License-Identifier: MIT -FILESEXTRAPATHS_prepend := "${FILE_DIRNAME}/files:" +# Settings that would typically be done from the custom kernel recipe +# ------------------------------------------------------------------- DESCRIPTION ?= "Custom kernel" +MAINTAINER ?= "isar-users <isar-users@googlegroups.com>" + +KBUILD_DEPENDS ?= "build-essential:native, \ + libelf-dev:native, \ + libncurses-dev:native, \ + libssl-dev:native, \ + bc, \ + bison, \ + cpio, \ + flex, \ + git, \ + kmod, \ + libssl-dev," + +KERNEL_DEBIAN_DEPENDS ?= "initramfs-tools | linux-initramfs-tool, \ + kmod, \ + linux-base (>= 4.3~)," + +KERNEL_HEADERS_DEBIAN_DEPENDS ?= "libc6, \ + libssl1.1," + +KERNEL_LIBC_DEV_DEPLOY ?= "0" + +# Settings that may be changed on a per distro, machine or layer basis +# -------------------------------------------------------------------- + +KERNEL_DTB_DIR ?= "usr/lib/linux-${KERNEL_NAME_PROVIDED}-${PV}" +KERNEL_HEADERS_DIR ?= "usr/src/linux-headers-${PV}" -KERNEL_NAME_PROVIDED ?= "${@ d.getVar('PN', True).partition('linux-')[2]}" KERNEL_DEFCONFIG ?= "" +# Add our template meta-data to the sources +FILESEXTRAPATHS_prepend := "${FILE_DIRNAME}/files:" +SRC_URI += "file://debian" + +# Variables and files that make our templates +# ------------------------------------------- + +TEMPLATE_FILES += " \ + debian/control.tmpl \ + debian/isar/build.tmpl \ + debian/isar/clean.tmpl \ + debian/isar/common.tmpl \ + debian/isar/install.tmpl \ + debian/linux-image.postinst.tmpl \ + debian/linux-image.postrm.tmpl \ + debian/rules.tmpl \ +" + +TEMPLATE_VARS += " \ + KBUILD_DEPENDS \ + KERNEL_ARCH \ + KERNEL_DEBIAN_DEPENDS \ + KERNEL_BUILD_DIR \ + KERNEL_DTB_DIR \ + KERNEL_FILE \ + KERNEL_HEADERS_DEBIAN_DEPENDS \ + KERNEL_HEADERS_DIR \ + KERNEL_NAME_PROVIDED \ +" + +inherit dpkg +inherit template + +# Derive name of the kernel packages from the name of this recipe +KERNEL_NAME_PROVIDED ?= "${@ d.getVar('PN', True).partition('linux-')[2]}" + +# Make bitbake know we will be producing linux-image and linux-headers packages python() { kernel_name = d.getVar("KERNEL_NAME_PROVIDED", True) d.setVar('PROVIDES', 'linux-image-' + kernel_name + ' ' + \ 'linux-headers-' + kernel_name) } +def get_kernel_arch(d): + distro_arch = d.getVar("DISTRO_ARCH") + if distro_arch == "amd64": + kernel_arch = "x86" + elif distro_arch == "arm64": + kernel_arch = "arm64" + elif distro_arch == "armhf": + kernel_arch = "arm" + elif distro_arch == "mipsel": + kernel_arch = "mips" + else: + kernel_arch = "" + return kernel_arch + +KERNEL_ARCH ??= "${@get_kernel_arch(d)}" + def config_fragments(d): fragments = [] sources = d.getVar("SRC_URI").split() @@ -31,57 +113,35 @@ def config_fragments(d): fragments.append(local) return fragments -inherit dpkg-base - -SRC_URI += "file://build-kernel.sh" +do_prepare_build_prepend() { + # copy meta-data over to source tree + rm -rf ${S}/debian + cp -r ${WORKDIR}/debian ${S}/ -KBUILD_DEPENDS ?= " \ - build-essential:native \ - libssl-dev \ - libelf-dev \ - bc \ - git \ - kmod \ - bison \ - flex \ - cpio \ - libncurses-dev" -KERNEL_DEBIAN_DEPENDS ?= "initramfs-tools | linux-initramfs-tool, kmod, linux-base (>= 4.3~)" -KERNEL_HEADERS_DEBIAN_DEPENDS ?= "libc6, libssl1.1" + # remove templates from the source tree + find ${S}/debian -name *.tmpl | xargs rm -f -KERNEL_LIBC_DEV_DEPLOY ?= "0" + # rename install/remove hooks to match user-specified name for our linux-image package + mv ${S}/debian/linux-image.postinst ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.postinst + mv ${S}/debian/linux-image.postrm ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.postrm -do_install_builddeps() { - dpkg_do_mounts - E="${@ bb.utils.export_proxies(d)}" - sudo -E chroot ${BUILDCHROOT_DIR} \ - apt-get update \ - -o Dir::Etc::SourceList="sources.list.d/isar-apt.list" \ - -o Dir::Etc::SourceParts="-" \ - -o APT::Get::List-Cleanup="0" - sudo -E chroot ${BUILDCHROOT_DIR} \ - apt-get install \ - -y -o Debug::pkgProblemResolver=yes \ - --no-install-recommends ${KBUILD_DEPENDS} - dpkg_undo_mounts + # produce a changelog for our kernel build + deb_add_changelog } -addtask install_builddeps after do_prepare_build before do_dpkg_build -# apt and reprepro may not run in parallel, acquire the Isar lock -do_install_builddeps[lockfiles] += "${REPO_ISAR_DIR}/isar.lock" - -addtask devshell after do_install_builddeps +# build directory for our "full" kernel build +KERNEL_BUILD_DIR = "build-full" -dpkg_runbuild() { - chmod +x ${WORKDIR}/build-kernel.sh - KERNEL_CONFIG_TARGET="${KERNEL_DEFCONFIG}" +dpkg_configure_kernel() { + config_target="${KERNEL_DEFCONFIG}" + rm -rf ${S}/${KERNEL_BUILD_DIR} && mkdir -p ${S}/${KERNEL_BUILD_DIR} if [ -n "${KERNEL_DEFCONFIG}" ]; then if [ -e "${WORKDIR}/${KERNEL_DEFCONFIG}" ]; then - cp ${WORKDIR}/${KERNEL_DEFCONFIG} ${S}/.config - KERNEL_CONFIG_TARGET="olddefconfig" + cp ${WORKDIR}/${KERNEL_DEFCONFIG} ${S}/${KERNEL_BUILD_DIR}/.config + config_target="olddefconfig" fi else - KERNEL_CONFIG_TARGET="defconfig" + config_target="defconfig" fi # copy config fragments over to the kernel tree @@ -92,21 +152,23 @@ dpkg_runbuild() { mkdir -p ${S}/debian/fragments (cd ${WORKDIR} && cp ${src_frags} ${S}/debian/fragments/) fi - export KERNEL_FRAGMENTS="${out_frags}" - E="${@ bb.utils.export_proxies(d)}" - - export PV=${PV} - export KERNEL_NAME=${KERNEL_NAME_PROVIDED} - - export KBUILD_DEPENDS="${KBUILD_DEPENDS}" - export KERNEL_CONFIG_TARGET - export KERNEL_DEBIAN_DEPENDS="${KERNEL_DEBIAN_DEPENDS}" - export KERNEL_FILE="${KERNEL_FILE}" - export KERNEL_HEADERS_DEBIAN_DEPENDS="${KERNEL_HEADERS_DEBIAN_DEPENDS}" + sudo -E chroot --userspec=$(id -u):$(id -g) ${BUILDCHROOT_DIR} sh -c " \ + export ARCH=${KERNEL_ARCH} && \ + cd ${PP}/${PPS} && \ + make O=${KERNEL_BUILD_DIR} ${config_target} && \ + ./scripts/kconfig/merge_config.sh \ + -O ${KERNEL_BUILD_DIR}/ \ + ${KERNEL_BUILD_DIR}/.config \ + ${out_frags} \ + " +} - sudo -E chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} ${PP}/build-kernel.sh ${PP}/${PPS} ${DISTRO_ARCH} +dpkg_runbuild_prepend() { + dpkg_configure_kernel +} +dpkg_runbuild_append() { if [ "${KERNEL_LIBC_DEV_DEPLOY}" != "1" ]; then rm -f ${WORKDIR}/linux-libc-dev_${PV}*.deb fi -- 2.20.1 ^ permalink raw reply [flat|nested] 83+ messages in thread
* Re: [PATCH 7/7] linux-custom: rewrite to no longer depend on the kernel's builddeb 2019-11-05 13:57 ` [PATCH 7/7] linux-custom: rewrite to no longer depend on the kernel's builddeb Cedric Hombourger @ 2019-11-05 15:26 ` Jan Kiszka 2019-11-05 16:20 ` Cedric Hombourger ` (2 more replies) 2019-11-07 11:44 ` Gylstorff Quirin 1 sibling, 3 replies; 83+ messages in thread From: Jan Kiszka @ 2019-11-05 15:26 UTC (permalink / raw) To: Cedric Hombourger, isar-users On 05.11.19 14:57, Cedric Hombourger wrote: > Regain control over the packaging of the Linux kernel by providing our > own debian recipes (debian,{control,rules} files and friends) instead > of using the kernel's builddeb script. This will allow generation of > packages for "perf" (not included in this changeset) with per distro > scheme (Debian and Ubuntu have differences) and maintain compatibility > with upstream packages generated from different source packages but > dependent on a certain deployment scheme (e.g. linux-base for Debian) > > Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com> > --- > doc/custom_kernel.md | 181 ++++++++++++++++ > .../linux/files/build-kernel.sh | 128 ------------ > meta/recipes-kernel/linux/files/debian/compat | 1 + > .../linux/files/debian/control.tmpl | 36 ++++ > meta/recipes-kernel/linux/files/debian/files | 5 + > .../linux/files/debian/isar/build.tmpl | 36 ++++ > .../linux/files/debian/isar/clean.tmpl | 20 ++ > .../linux/files/debian/isar/common.tmpl | 60 ++++++ > .../linux/files/debian/isar/install.tmpl | 197 ++++++++++++++++++ > .../files/debian/linux-image.postinst.tmpl | 22 ++ > .../files/debian/linux-image.postrm.tmpl | 18 ++ > .../linux/files/debian/rules.tmpl | 39 ++++ > meta/recipes-kernel/linux/linux-custom.inc | 172 ++++++++++----- > 13 files changed, 732 insertions(+), 183 deletions(-) > create mode 100644 doc/custom_kernel.md > delete mode 100644 meta/recipes-kernel/linux/files/build-kernel.sh > create mode 100644 meta/recipes-kernel/linux/files/debian/compat > create mode 100644 meta/recipes-kernel/linux/files/debian/control.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/files > create mode 100644 meta/recipes-kernel/linux/files/debian/isar/build.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/isar/clean.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/isar/common.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/isar/install.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl > create mode 100755 meta/recipes-kernel/linux/files/debian/rules.tmpl > > diff --git a/doc/custom_kernel.md b/doc/custom_kernel.md > new file mode 100644 > index 0000000..5ab2599 > --- /dev/null > +++ b/doc/custom_kernel.md > @@ -0,0 +1,181 @@ > +# Custom kernel recipe for Isar > + > +## Contents > + > + - [Summary](#summary) > + - [Proposal owners](#proposal-owners) > + - [Current status](#current-status) > + - [Detailed description](#detailed-description) > + - [Benefit to Isar](#benefit-to-isar) > + - [How can I help?](#how-can-i-help) > + - [Scope](#scope) > + - [How to test](#how-to-test) > + - [Dependencies](#dependencies) > + - [Documentation](#documentation) > + - [Questions and answers](#questions-and-answers) > + > +## Summary > + > +Isar currently uses the Linux kernel's builddeb script to generate linux-image, > +linux-headers and linux-libc-dev packages for your custom kernel sources. The > +main benefit of this approach was that the low level details of how to package > +the Linux kernel image, modules and headers were left in the kernel. There are > +however some drawbacks: Isar had to repack generated packages to introduce some > +of the changes it needs such as suffixing the kernel packages with the name of > +the custom kernel (e.g. with "mainline" for the "linux-mainline" recipe), lack > +of automatic dependencies (the builddeb script from the kernel does not use > +debhelper), it was difficult to add packages especially distribution specific > +packages (maintain version specific patch in Isar vs being able to upstream > +distribution specific changes upstream), etc. This change is about being less > +dependent on the kernel packaging scripts (which aren't used by major distros > +anyway) and regain control on how we want or need the Linux kernel packaged. > + > +## Proposal owners > + > + * name: [Cedric Hombourger](https://github.com/chombourger) > + > +## Current status > + > +### Tests > + > +The following were recently checked: > + > + * Custom kernels in meta-isar build without (known) failures > + * initrd image gets generated when the kernel is added to the rootfs > + * `ISAR_CROSS_COMPILE` is supported > + * `KERNEL_FILE` is honored > + * `KERNEL_DEFCONFIG` may either be a file (specified via `SRC_URI`) or a > + kernel-provided defconfig > + * Config fragments get merged with the user-specified kernel configuration > + * Path to installed DTBs may be configured (new) > + * Configuration and build from a `devshell` > + * Support for the `KERNEL_LIBC_DEV_DEPLOY` directive > + > +while the following needs work: > + > + * Check if custom-kernel builds are reproducible > + > +### Remaining work items > + > + * Check if there is a better way to cross compile kernel "scripts" such as > + `fixdep` or `modpost` than manually compiling them (they are compiled for > + the build machine (amd64) and not for the target (e.g. armhf) even though > + we are cross-compiling - yet the linux-headers package should be shipping > + "scripts" for the target machine) > + > +## Detailed description > + > +### Requirements > + > +#### Current > + > +Isar has had a `linux-custom` recipe for a long time and counts many users. It is therefore > +paramount to maintain features that it has today with no or little changes to the APIs. The > +following requirements were identified and will be maintained: > + > + 1. Sources to the custom Linux kernel may be specified via `SRC_URI` > + > + 2. Configure the kernel via an in-tree or an external `defconfig` Reference to KERNEL_DEFCONFIG is missing. > + > + 3. Support kernel configuration tweaks via configuration fragments (`.cfg` files) > + > + 4. Patches to the linux kernel may be specified via `SRC_URI` > + > + 5. Ensure that the Isar recipe `PV` matches the kernel release identifier (`KERNEL_RELEASE`) > + > + 6. Produce a `linux-image` package that ships the kernel image and modules > + > + 7. Allow the name of the kernel image to be changed via `KERNEL_FILE` (defaults to `vmlinuz`) > + > + 8. Produce a `linux-headers` package which includes kernel headers and kbuild scripts/tools > + > + 9. The `linux-headers` package shall support native and cross compiles of out-of-tree kernel modules One package to serve them all? Is that the use case for patch 6? IOW, we are now building linux-headers for the target, not the host OS anymore? What's the slowdown compare to cross-linux-headers? The current policy is that, when cross-building, our -dev packagages (which is what linux-headers falls under) will work natively with the buildchroot. The future path is clearly generating fitting packages for both host and target. Or is there anything in the linux-headers that prevents providing both? > + > + 10. Produce a `linux-libc-dev` package to support user-land builds This should be opt-in only: No need to build it when we don't ship it. Just add the related fragement to debian/control when KERNEL_LIBC_DEV_DEPLOY is set. > + > + 11. Only ship the `linux-libc-dev` package if instructed to (`KERNEL_LIBC_DEV_DEPLOY` equals to `"1"`) > + > + 12. Support both native and cross compiles (`ISAR_CROSS_COMPILE`) > + > + 13. Support for the following kernel architectures: > + > + * arm > + * arm64 > + * mips > + * x86 > + > + 14. Support `devshell` (kernel configuration shall be applied) > + > +#### New requirements (proposed) > + > +This proposal includes the following new requirements: > + > + 1. Support creation of additional packages > + > + 2. Allow per-distro packaging tweaks > + > +### Approach > + > +The new implementation will have the `builddeb` code ported from the Linux kernel included to the > +Isar environment. It will be refactor to support pre and post hooks. Hooks may be used by external > +layers to extend the kernel packaging instructions provided by "stock" Isar or by Isar itself to > +implement distro-specific packages or tweaks. > + > +Hooks need the ability to add `build` and `install` steps which can be achieved by having the main > +`build` and `install` steps call pre and post scripts. This will however solve only one side of the > +problem: new packages should be declared in the `debian/control` file. The packaging process > +(initiated by `debian/rules`) will use a baseline `control` file when started and a mechanism will > +be provided for hooks to append `control` blocks as they are processed. Is this a section addressing users or reviewers? The latter audience is better addressed via the commit log. > + > +### Future work > + > +In the event where this proposal is accepted, here are a few future projects that are being > +considered: > + > + 1. Package `perf` as Debian does > + > + 2. Add `ubuntu` as an Isar supported target distro and validate `linux-custom` recipe > + > +## Benefit to Isar > + > + * Control the kernel build and packaging process > + > + * No longer need to repack packages > + > + * Provide hooks for custom kernel recipes to add build/install directives Careful with hooks! They tend to pile up mess. Where exactly do you want them? What for? > + > +## How can I help? > + > + * Check if you are able to build your own linux-custom recipes with this change > + * Check if packages generated by this recipe are compatible with your Debian-based distro > + * Review and provide comments on this changeset > + * Create templates for distro-specific packages (e.g. perf) > + > +## Scope > + > + * Proposal owners: proposal owners are to communicate the idea to isar-users and make sure > + that all of their requirements are met (to the extent they are already supported with the > + current solution) or may be met in the future. > + > + * Other developers: port your out-of-tree linux-custom recipes to this new solution and > + report issues. > + > +## How to test? > + > +The linux-mainline recipe may be used for some basic testing. This recipe is being used by the > +following machines: > + > + * de0-nano-soc > + * qemumipsel > + > +## Dependencies > + > + * None > + > +## Documentation > + > + * Document user-visible variables added by this proposal > + > +## Questions and answers > + > + * None > diff --git a/meta/recipes-kernel/linux/files/build-kernel.sh b/meta/recipes-kernel/linux/files/build-kernel.sh > deleted file mode 100644 > index f56e96c..0000000 > --- a/meta/recipes-kernel/linux/files/build-kernel.sh > +++ /dev/null > @@ -1,128 +0,0 @@ > -#!/bin/bash > -# > -# Custom kernel build > -# > -# This software is a part of ISAR. > -# Copyright (c) Siemens AG, 2018 > -# > -# SPDX-License-Identifier: MIT > - > -source /isar/common.sh > - > -host_arch=$(dpkg --print-architecture) > - > -if [ "$host_arch" != "$target_arch" ]; then > - case $target_arch in > - armhf) > - export ARCH=arm > - export CROSS_COMPILE="arm-linux-gnueabihf-" > - ;; > - arm64) > - export ARCH=arm64 > - export CROSS_COMPILE="aarch64-linux-gnu-" > - ;; > - mipsel) > - export ARCH=mips > - export CROSS_COMPILE="mipsel-linux-gnu-" > - ;; > - *) > - echo "error: unsupported architecture ($target_arch)" > - exit 1 > - ;; > - esac > -fi > - > -REPACK_DIR="$1/../repack" > -REPACK_LINUX_IMAGE_DIR="${REPACK_DIR}/linux-image" > -REPACK_LINUX_HEADERS_DIR="${REPACK_DIR}/linux-headers" > - > -make ${KERNEL_CONFIG_TARGET} || exit ${?} > -if [ -n "${KERNEL_FRAGMENTS}" ]; then > - scripts/kconfig/merge_config.sh -m .config ${KERNEL_FRAGMENTS} > -fi > - > -KV=$( make -s kernelrelease ) > -if [ "${KV}" != "${PV}" ]; then > - echo "ERROR: Recipe PV is \"${PV}\" but should be \"${KV}\"" 1>&2 > - echo "ERROR: Probably due to CONFIG_LOCALVERSION" 1>&2 > - exit 1 > -fi > - > -rm -f .version > -KBUILD_DEBARCH=$target_arch make -j $(($(nproc) * 2)) deb-pkg > - > -rm -rf "${REPACK_DIR}" > -mkdir -p "${REPACK_DIR}" > -mkdir -p "${REPACK_LINUX_IMAGE_DIR}" > -mkdir -p "${REPACK_LINUX_HEADERS_DIR}" > - > -cp -a debian "${REPACK_DIR}" > - > -# dpkg-gencontrol performs cross-incompatible checks on the > -# Architecture field; trick it to accept the control file > -sed -i "s/Architecture: .*/Architecture: any/" "${REPACK_DIR}/debian/control" > - > -cd .. > - > -dpkg-deb -R linux-image-${PV}_${PV}-1_*.deb "${REPACK_LINUX_IMAGE_DIR}" > -dpkg-deb -R linux-headers-${PV}_${PV}-1_*.deb "${REPACK_LINUX_HEADERS_DIR}" > - > -dpkg-gencontrol -crepack/debian/control \ > - -lrepack/debian/changelog \ > - -frepack/debian/files \ > - -plinux-image-${PV} \ > - -P"${REPACK_LINUX_IMAGE_DIR}" \ > - -DPackage="linux-image-${KERNEL_NAME}" \ > - -DSection=kernel \ > - -DPriority=required \ > - -DDepends="${KERNEL_DEBIAN_DEPENDS}" \ > - -DArchitecture=$target_arch > - > -# Add Debian-like link installation to postinst > -mkdir -p ${REPACK_LINUX_IMAGE_DIR}/lib/modules/${PV} > -touch "${REPACK_LINUX_IMAGE_DIR}/lib/modules/${PV}/.fresh-install" > -sed -i "${REPACK_LINUX_IMAGE_DIR}/DEBIAN/postinst" \ > - -e "/^set -e$/a\\ > -\\ > -if [ -f /lib/modules/${PV}/.fresh-install ]; then\\ > - change=install\\ > -else\\ > - change=upgrade\\ > -fi\\ > -linux-update-symlinks \$change ${PV} /boot/${KERNEL_FILE}-${PV}\\ > -rm -f /lib/modules/${PV}/.fresh-install" > - > -# Add Debian-like link removal to postrm > -sed -i "${REPACK_LINUX_IMAGE_DIR}/DEBIAN/postrm" \ > - -e "/^set -e$/a\\ > -\\ > -rm -f /lib/modules/${PV}/.fresh-install\\ > -\\ > -if [ \"\$1\" != upgrade ] && command -v linux-update-symlinks >/dev/null; then\\ > - linux-update-symlinks remove ${PV} /boot/${KERNEL_FILE}-${PV}\\ > -fi" > - > -# Make sure arm64 kernels are decompressed > -if [ "$target_arch" = "arm64" ]; then > - kernel_file="${REPACK_LINUX_IMAGE_DIR}/boot/${KERNEL_FILE}-${PV}" > - mv "${kernel_file}" "${kernel_file}.gz" > - gunzip "${kernel_file}.gz" > -fi > - > -dpkg-gencontrol -crepack/debian/control \ > - -lrepack/debian/changelog \ > - -frepack/debian/files \ > - -plinux-headers-${PV} \ > - -P"${REPACK_LINUX_HEADERS_DIR}" \ > - -Vkernel:debarch="${KERNEL_NAME}" \ > - -DPackage="linux-headers-${KERNEL_NAME}" \ > - -DSection=kernel \ > - -DDepends="${KERNEL_HEADERS_DEBIAN_DEPENDS}" \ > - -DArchitecture=$target_arch > - > -fakeroot dpkg-deb -b "${REPACK_LINUX_IMAGE_DIR}" \ > - linux-image-${KERNEL_NAME}_${PV}-1_${KERNEL_NAME}.deb > -rm -f linux-image-${PV}_${PV}-1_*.deb > -fakeroot dpkg-deb -b "${REPACK_LINUX_HEADERS_DIR}" \ > - linux-headers-${KERNEL_NAME}_${PV}-1_${KERNEL_NAME}.deb > -rm -f linux-headers-${PV}_${PV}-1_*.deb > diff --git a/meta/recipes-kernel/linux/files/debian/compat b/meta/recipes-kernel/linux/files/debian/compat > new file mode 100644 > index 0000000..ec63514 > --- /dev/null > +++ b/meta/recipes-kernel/linux/files/debian/compat > @@ -0,0 +1 @@ > +9 > diff --git a/meta/recipes-kernel/linux/files/debian/control.tmpl b/meta/recipes-kernel/linux/files/debian/control.tmpl > new file mode 100644 > index 0000000..1295466 > --- /dev/null > +++ b/meta/recipes-kernel/linux/files/debian/control.tmpl > @@ -0,0 +1,36 @@ > +Source: linux-${KERNEL_NAME_PROVIDED} > +Section: kernel > +Priority: optional > +Maintainer: ${MAINTAINER} > +Build-Depends: bc, kmod, cpio, ${KBUILD_DEPENDS} > +Homepage: http://www.kernel.org/ Standard-Version? > + > +Package: linux-image-${KERNEL_NAME_PROVIDED} > +Architecture: any > +Depends: ${KERNEL_DEBIAN_DEPENDS} > +Description: ${KERNEL_NAME_PROVIDED} Linux kernel, version ${PV} > + This package contains the Linux kernel, modules and corresponding other > + files, version: ${PV}. > + > +Package: linux-headers-${KERNEL_NAME_PROVIDED} > +Architecture: any > +Depends: ${KERNEL_HEADERS_DEBIAN_DEPENDS}, ${perl:Depends}, ${shlib:Depends} > +Description: ${KERNEL_NAME_PROVIDED} Linux kernel headers for ${PV} > + This package provides kernel header files for ${PV} on ${DISTRO_ARCH} > + . > + This is useful for people who need to build external modules > + > +Package: linux-libc-dev See above, should not be here by default. > +Section: devel > +Provides: linux-kernel-headers > +Architecture: any > +Description: Linux support headers for userspace development > + This package provides userspaces headers from the Linux kernel. These headers > + are used by the installed headers for GNU glibc and other system libraries. > + > +Package: linux-image-${KERNEL_NAME_PROVIDED}-dbg > +Section: debug > +Architecture: any > +Description: Linux kernel debugging symbols for ${PV} > + This package will come in handy if you need to debug the kernel. It provides > + all the necessary debug symbols for the kernel and its modules. > diff --git a/meta/recipes-kernel/linux/files/debian/files b/meta/recipes-kernel/linux/files/debian/files > new file mode 100644 > index 0000000..a8ace32 > --- /dev/null > +++ b/meta/recipes-kernel/linux/files/debian/files > @@ -0,0 +1,5 @@ > +linux-headers_4.19.80-1_mipsel.deb kernel optional > +linux-image-dbg_4.19.80-1_mipsel.deb debug optional > +linux-image_4.19.80-1_mipsel.deb kernel optional > +linux-libc-dev_4.19.80-1_mipsel.deb devel optional > +linux_4.19.80-1_mipsel.buildinfo kernel optional Looks like an accidentally added file. > diff --git a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl > new file mode 100644 > index 0000000..ce79d56 > --- /dev/null > +++ b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl > @@ -0,0 +1,36 @@ > +#!/bin/sh > +# Copyright (c) Mentor Graphics, a Siemens business, 2019 > +# SPDX-License-Identifier: MIT > + > +# Load common stuff > +. ${S}/debian/isar/common || exit ${?} > + > +do_build() { > + > + # Print a few things that are of particular interest > + print_settings > + > + # Trace what we do here > + set -x > + > + # Process existing kernel configuration to make sure it is complete > + # (use defaults for options that were not specified) > + ${MAKE} O=${KERNEL_BUILD_DIR} olddefconfig || exit ${?} > + > + # Build the Linux kernel > + ${MAKE} O=${KERNEL_BUILD_DIR} -j $(nproc) || exit ${?} > + > + # Stop tracing > + set +x > +} > + > +print_settings() { > + cat <<EOF > +# Build settings: > +# --------------- > +# ARCH=${ARCH} > +# CROSS_COMPILE=${CROSS_COMPILE} > +EOF > +} > + > +main build ${*} > diff --git a/meta/recipes-kernel/linux/files/debian/isar/clean.tmpl b/meta/recipes-kernel/linux/files/debian/isar/clean.tmpl > new file mode 100644 > index 0000000..2aa3742 > --- /dev/null > +++ b/meta/recipes-kernel/linux/files/debian/isar/clean.tmpl > @@ -0,0 +1,20 @@ > +#!/bin/sh > +# Copyright (c) Mentor Graphics, a Siemens business, 2019 > +# SPDX-License-Identifier: MIT > + > +# Load common stuff > +. ${S}/debian/isar/common || exit ${?} > + > +do_clean() { > + > + # Trace what we do here > + set -x > + > + rm -rf ${deb_img_dir} ${deb_dbg_dir} > + rm -rf ${deb_kern_hdrdir} ${deb_libc_hdr_dir} I suppose we rather (or also?) want "make clean" here for the build artifacts. Just make sure to save/restore the debian folder across that step. > + > + # Stop tracing > + set +x > +} > + > +main clean ${*} > diff --git a/meta/recipes-kernel/linux/files/debian/isar/common.tmpl b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl > new file mode 100644 > index 0000000..b392b46 > --- /dev/null > +++ b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl > @@ -0,0 +1,60 @@ > +#!/bin/sh > +# Copyright (c) Mentor Graphics, a Siemens business, 2019 > +# SPDX-License-Identifier: MIT > + > +# Isar settings > +ARCH=${KERNEL_ARCH} > +KERNEL_PKG_IMAGE=linux-image-${KERNEL_NAME_PROVIDED} > +KERNEL_PKG_KERN_HEADERS=linux-headers-${KERNEL_NAME_PROVIDED} > +KERNEL_PKG_LIBC_HEADERS=linux-libc-dev > + > +# Constants > +KCONF=.config > + > +# Target directories > +deb_top_dir=${S}/debian > +deb_img_dir=${deb_top_dir}/${KERNEL_PKG_IMAGE} > +deb_dbg_dir=${deb_img_dir}-dbg > +deb_dtb_dir=${deb_img_dir}/${KERNEL_DTB_DIR} > +deb_kern_hdr_dir=${deb_top_dir}/${KERNEL_PKG_KERN_HEADERS} > +deb_libc_hdr_dir=${deb_top_dir}/${KERNEL_PKG_LIBC_HEADERS} > + > +main() { > + target=${1} > + > + if [ ! -f ${S}/debian/isar/${target} ]; then > + echo "error: ${target} is not a supported build target!" >&2 > + return 1 > + fi > + > + # check for scripts to be sourced > + for f in ${S}/debian/isar/defaults.d/${target}/*; do > + [ -f ${f} ] || continue > + . ${f} || return ${?} > + done > + > + # variables to be exported > + export ARCH > + > + # are we cross-compiling? > + BUILD_ARCH=$(dpkg-architecture -qDEB_BUILD_ARCH) > + if [ "${BUILD_ARCH}" = "${DISTRO_ARCH}" ]; then > + # no, make sure CROSS_COMPILE isn't set > + unset CROSS_COMPILE > + fi > + > + # check for pre-target scripts > + for f in ${S}/debian/isar/pre.d/${target}/*; do > + [ -f ${f} ] || continue > + sh ${f} || return ${?} > + done > + > + # call the actual target script > + do_${target} || return ${?} > + > + # check for post-target scripts > + for f in ${S}/debian/isar/post.d/${target}/*; do > + [ -f ${f} ] || continue > + sh ${f} || return ${?} > + done > +} > diff --git a/meta/recipes-kernel/linux/files/debian/isar/install.tmpl b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl > new file mode 100644 > index 0000000..2055669 > --- /dev/null > +++ b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl > @@ -0,0 +1,197 @@ > +#!/bin/sh > +# Copyright (c) Mentor Graphics, a Siemens business, 2019 > +# SPDX-License-Identifier: MIT > + > +# Load common stuff > +. ${S}/debian/isar/common || exit ${?} > + > +do_install() { > + > + # check if our kernel was configured > + if [ ! -f "${O}/.config" ]; then > + echo "error: kernel not configured!" >&2 > + return 1 > + fi > + > + # load its configuration > + . ${O}/.config > + > + kimage="$(${MAKE} O=${O} -s --no-print-directory image_name)" > + case "${ARCH}" in > + mips|powerpc) kimage_path="boot/vmlinux-${PV}" ;; > + um) kimage_path="usr/bin/vmlinux-${PV}" ;; > + *) kimage_path="boot/vmlinuz-${PV}" ;; > + esac > + > + print_settings > + > + # Stop on error > + set -e > + > + # Trace what we do here > + set -x > + > + install_image > + install_hooks > + install_dtbs > + install_kmods > + install_headers > + > + # Stop tracing > + set +x > +} > + > +print_settings() { > + cat <<EOF > +Install settings: > +----------------- > +deb_dtb_dir=${deb_dtb_dir} > +deb_hdr_dir=${deb_hdr_dir} > +kimage=${kimage} > +kimage_path=${kimage_path} > + > +EOF > +} > + > +install_image() { > + install -m 755 -d ${deb_img_dir}/$(dirname ${kimage_path}) > + cp ${O}/${kimage} ${deb_img_dir}/${kimage_path} > + install_image_debug > +} > + > +install_image_debug() { > + # Different tools want the image in different locations > + # perf > + mkdir -p ${deb_dbg_dir}/usr/lib/debug/lib/modules/${PV}/ > + cp vmlinux ${deb_dbg_dir}/usr/lib/debug/lib/modules/${PV}/ > + # systemtap > + mkdir -p ${deb_dbg_dir}/usr/lib/debug/boot/ > + ln -s ../lib/modules/$version/vmlinux ${deb_dbg_dir}/usr/lib/debug/boot/vmlinux-${PV} > + # kdump-tools > + ln -s lib/modules/${PV}/vmlinux ${deb_dbg_dir}/usr/lib/debug/vmlinux-${PV} > +} > + > +install_hooks() { > + install -m 755 -d ${deb_img_dir}/etc/kernel/install.d > + install -m 755 -d ${deb_img_dir}/etc/kernel/postinst.d > + install -m 755 -d ${deb_img_dir}/etc/kernel/postrm.d > + install -m 755 -d ${deb_img_dir}/etc/kernel/prerm.d > +} > + > +install_dtbs() { > + [ -n "${CONFIG_OF}" ] || return 0 > + ${MAKE} O=${O} INSTALL_DTBS_PATH=${deb_dtb_dir} dtbs_install > +} > + > +install_kmods() { > + [ -n "${CONFIG_MODULES}" ] || return 0 > + ${MAKE} O=${O} INSTALL_MOD_PATH=${deb_img_dir} modules_install > + touch ${deb_img_dir}/lib/modules/${PV}/.fresh-install > + rm -fv ${deb_img_dir}/lib/modules/${PV}/build > + rm -fv ${deb_img_dir}/lib/modules/${PV}/source > + install_kmods_debug > +} > + > +kmods_sign() { > + [ -n "${CONFIG_MODULE_SIG_ALL}" ] || return 0 > + ${MAKE} O=${O} INSTALL_MOD_PATH=${deb_img_dir} modules_sign > +} > + > +install_kmods_debug() { > + [ -n "${CONFIG_DEBUG_INFO}" ] || return 0 > + > + kmod_inst_dir=${deb_img_dir}/lib/modules > + kmod_debug_dir=${deb_dbg_dir}/usr/lib/debug > + > + # copy kernels modules to usr/lib/debug > + mkdir -p ${kmod_debug_dir} > + tar -C ${kmod_inst_dir}/ -cO --exclude='modules.*' . | tar -C ${kmod_debug_dir}/ -xf - > + # strip everything but debug sections for modules in usr/lib/debug > + find ${kmod_debug_dir} -name *.ko -exec ${CROSS_COMPILE}objcopy --only-keep-debug {} \; > + # and strip debug sections from modules in lib/modules > + find ${kmod_inst_dir} -name *.ko -exec ${CROSS_COMPILE}objcopy --strip-debug {} \; > + > + # re-sign stripped kernel modules > + kmods_sign > +} > + > +headers_check() { > + ${MAKE} O=${O} headers_check > +} > + > +libc_headers() { > + mkdir -p ${deb_libc_hdr_dir} > + ${MAKE} O=${O} headers_install INSTALL_HDR_PATH=${deb_libc_hdr_dir}/usr > + host_arch=$(dpkg-architecture -a${DISTRO_ARCH} -qDEB_HOST_MULTIARCH) > + mkdir ${deb_libc_hdr_dir}/usr/include/${host_arch} > + mv ${deb_libc_hdr_dir}/usr/include/asm ${deb_libc_hdr_dir}/usr/include/${host_arch}/ > +} > + > +# the kernel does not seem to provide a way to cross-compile its tools, we however > +# have qemu-user-static binaries in our buildchroot. Create a new build tree for the > +# scripts and override HOSTCC to force a cross-compile > +kernel_tools_build() { > + odir="${S}/build-scripts" > + mflags="HOSTCC=${CROSS_COMPILE}gcc O=${odir} NOSTDINC_FLAGS=" > + rm -rf ${odir} && mkdir -p ${odir} && cp ${S}/${KERNEL_BUILD_DIR}/.config ${odir} > + (unset LD_PRELOAD; cd ${odir} && make -C ${S} ${mflags} olddefconfig scripts) > + (cd ${odir}; find scripts -type f -executable -exec file {} \;|grep ELF|cut -d: -f1) >${src_hdr_files} > + tar -C ${odir} -cf - -T - <${src_hdr_files} | tar -C ${destdir} -xf - > +} > + > +kernel_tools() { > + # remove object files > + find ${destdir}/scripts -type f -name '*.o' |xargs rm -f > + > + # we're all done if we aren't cross-compiling > + [ -n "${CROSS_COMPILE}" ] || return 0 > + > + # remove ELF executables from the linux-headers package > + find ${destdir}/scripts -type f -exec file {} \;|grep ELF|cut -d: -f1|xargs rm -fv > + > + # cross-compile kernel tools to be shipped with linux-headers > + kernel_tools_build > +} > + > +kernel_headers() { > + destdir=${deb_kern_hdr_dir}/${KERNEL_HEADERS_DIR} > + src_hdr_files=$(mktemp) > + obj_hdr_files=$(mktemp) > + > + mkdir -p ${destdir} > + mkdir -p ${deb_kern_hdr_dir}/lib/modules/${PV} > + > + (cd ${S}; find . -name 'Makefile*' -o -name 'Kconfig*' -o -name '*.pl') >>${src_hdr_files} > + (cd ${S}; find arch/*/include include scripts -type f -o -type l) >>${src_hdr_files} > + (cd ${S}; find arch/${ARCH} -name module.lds -o -name Kbuild.platforms -o -name Platform) >>${src_hdr_files} > + (cd ${S}; find $(find arch/${ARCH} -name include -o -name scripts -type d) -type f) >>${src_hdr_files} > + > + (cd ${O}; find arch/${ARCH}/include Module.symvers include scripts -type f) >>${obj_hdr_files} > + if [ -n "${CONFIG_STACK_VALIDATION}" ]; then > + (cd ${O}; find tools/objtool -type f -executable) >>${obj_hdr_files} > + fi > + if [ -n "${CONFIG_GCC_PLUGINS}" ]; then > + (cd ${O}; find scripts/gcc-plugins -name *.so -o -name gcc-common.h) >>${obj_hdr_files} > + fi > + > + # deploy files that were matched above > + tar -C ${S} -cf - -T - <${src_hdr_files} | tar -C ${destdir} -xf - > + tar -C ${O} -cf - -T - <${obj_hdr_files} | tar -C ${destdir} -xf - > + > + # add the kernel config > + cp ${O}/${KCONF} ${destdir}/.config > + > + # handle kernel development tools > + kernel_tools > + > + # create symlinks > + ln -sf /${KERNEL_HEADERS_DIR} ${deb_kern_hdr_dir}/lib/modules/${PV}/build > +} > + > +install_headers() { > + headers_check > + libc_headers > + kernel_headers > +} > + > +main install ${*} > diff --git a/meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl b/meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl > new file mode 100644 > index 0000000..fd89121 > --- /dev/null > +++ b/meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl > @@ -0,0 +1,22 @@ > +#!/bin/sh > + > +set -e > +set -x > + > +if [ -f /lib/modules/${PV}/.fresh-install ]; then > + change=install > +else > + change=upgrade > +fi > + > +linux-update-symlinks $change ${PV} /boot/${KERNEL_FILE}-${PV} > +rm -f /lib/modules/${PV}/.fresh-install > + > +# Pass maintainer script parameters to hook scripts > +export DEB_MAINT_PARAMS="$*" > + > +# Tell initramfs builder whether it's wanted > +export INITRD=Yes I've noticed while hacking my version that this should follow CONFIG_BLK_DEV_INITRD. > + > +test -d /etc/kernel/postinst.d && run-parts --arg="${PV}" --arg="/boot/${KERNEL_FILE}-${PV}" /etc/kernel/postinst.d > +exit 0 > diff --git a/meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl b/meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl > new file mode 100644 > index 0000000..9d88218 > --- /dev/null > +++ b/meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl > @@ -0,0 +1,18 @@ > +#!/bin/sh > + > +set -e > + > +rm -f /lib/modules/${PV}/.fresh-install > + > +if [ "$1" != upgrade ] && command -v linux-update-symlinks >/dev/null; then > + linux-update-symlinks remove ${PV} /boot/${KERNEL_FILE}-${PV} > +fi > + > +# Pass maintainer script parameters to hook scripts > +export DEB_MAINT_PARAMS="$*" > + > +# Tell initramfs builder whether it's wanted > +export INITRD=Yes > + > +test -d /etc/kernel/postrm.d && run-parts --arg="${PV}" --arg="/boot/${KERNEL_FILE}-${PV}" /etc/kernel/postrm.d > +exit 0 > diff --git a/meta/recipes-kernel/linux/files/debian/rules.tmpl b/meta/recipes-kernel/linux/files/debian/rules.tmpl There is quite some duplication between postinst and postrm already. Moreover, I think you are missing preinst and prerm. I think all four should probably be generated from one template. > new file mode 100755 > index 0000000..93c8a5b > --- /dev/null > +++ b/meta/recipes-kernel/linux/files/debian/rules.tmpl > @@ -0,0 +1,39 @@ > +#!/usr/bin/make -f > + > +export CROSS_COMPILE=$(DEB_HOST_GNU_TYPE)- > + > +O:=$(CURDIR)/${KERNEL_BUILD_DIR} > +S:=$(CURDIR) > +deb_top_dir:=$(S)/debian > + > +DH_VERBOSE=1 > + > +# Dynamic variables to be passed to Isar build scripts > +isar_env=$(strip \ > + export CROSS_COMPILE='$(CROSS_COMPILE)' && \ > + export MAKE='$(MAKE)' && \ > + export O='${O}' \ > + export S='${S}' \ > +) > + > +%: > + dh $(@) > + > +.PHONY: override_dh_auto_clean > +override_dh_auto_clean: > + $(isar_env) && sh $(deb_top_dir)/isar/clean > + > +.PHONY: override_dh_auto_build > +override_dh_auto_build: > + $(isar_env) && sh $(deb_top_dir)/isar/build > + > +override_dh_auto_install: > + $(isar_env) && sh $(deb_top_dir)/isar/install > + > +.PHONY: override_dh_auto_test > +override_dh_auto_test: > + true > + > +PHONY: override_dh_strip > +override_dh_strip: > + dh_strip -Xvmlinux --no-automatic-dbgsym > diff --git a/meta/recipes-kernel/linux/linux-custom.inc b/meta/recipes-kernel/linux/linux-custom.inc > index ca91f64..36086be 100644 > --- a/meta/recipes-kernel/linux/linux-custom.inc > +++ b/meta/recipes-kernel/linux/linux-custom.inc > @@ -2,22 +2,104 @@ > # > # This software is a part of ISAR. > # Copyright (c) Siemens AG, 2018 > +# Copyright (c) Mentor Graphics, a Siemens business, 2019 > # > # SPDX-License-Identifier: MIT > > -FILESEXTRAPATHS_prepend := "${FILE_DIRNAME}/files:" > +# Settings that would typically be done from the custom kernel recipe > +# ------------------------------------------------------------------- > > DESCRIPTION ?= "Custom kernel" > +MAINTAINER ?= "isar-users <isar-users@googlegroups.com>" > + > +KBUILD_DEPENDS ?= "build-essential:native, \ > + libelf-dev:native, \ > + libncurses-dev:native, \ > + libssl-dev:native, \ > + bc, \ > + bison, \ > + cpio, \ > + flex, \ > + git, \ > + kmod, \ > + libssl-dev," > + > +KERNEL_DEBIAN_DEPENDS ?= "initramfs-tools | linux-initramfs-tool, \ > + kmod, \ > + linux-base (>= 4.3~)," > + > +KERNEL_HEADERS_DEBIAN_DEPENDS ?= "libc6, \ > + libssl1.1," > + > +KERNEL_LIBC_DEV_DEPLOY ?= "0" > + > +# Settings that may be changed on a per distro, machine or layer basis > +# -------------------------------------------------------------------- > + > +KERNEL_DTB_DIR ?= "usr/lib/linux-${KERNEL_NAME_PROVIDED}-${PV}" > +KERNEL_HEADERS_DIR ?= "usr/src/linux-headers-${PV}" > > -KERNEL_NAME_PROVIDED ?= "${@ d.getVar('PN', True).partition('linux-')[2]}" > KERNEL_DEFCONFIG ?= "" > > +# Add our template meta-data to the sources > +FILESEXTRAPATHS_prepend := "${FILE_DIRNAME}/files:" > +SRC_URI += "file://debian" > + > +# Variables and files that make our templates > +# ------------------------------------------- > + > +TEMPLATE_FILES += " \ > + debian/control.tmpl \ > + debian/isar/build.tmpl \ > + debian/isar/clean.tmpl \ > + debian/isar/common.tmpl \ > + debian/isar/install.tmpl \ > + debian/linux-image.postinst.tmpl \ > + debian/linux-image.postrm.tmpl \ > + debian/rules.tmpl \ > +" > + > +TEMPLATE_VARS += " \ > + KBUILD_DEPENDS \ > + KERNEL_ARCH \ > + KERNEL_DEBIAN_DEPENDS \ > + KERNEL_BUILD_DIR \ > + KERNEL_DTB_DIR \ > + KERNEL_FILE \ > + KERNEL_HEADERS_DEBIAN_DEPENDS \ > + KERNEL_HEADERS_DIR \ > + KERNEL_NAME_PROVIDED \ > +" > + > +inherit dpkg > +inherit template > + > +# Derive name of the kernel packages from the name of this recipe > +KERNEL_NAME_PROVIDED ?= "${@ d.getVar('PN', True).partition('linux-')[2]}" > + > +# Make bitbake know we will be producing linux-image and linux-headers packages > python() { > kernel_name = d.getVar("KERNEL_NAME_PROVIDED", True) > d.setVar('PROVIDES', 'linux-image-' + kernel_name + ' ' + \ > 'linux-headers-' + kernel_name) > } > > +def get_kernel_arch(d): > + distro_arch = d.getVar("DISTRO_ARCH") > + if distro_arch == "amd64": > + kernel_arch = "x86" > + elif distro_arch == "arm64": > + kernel_arch = "arm64" > + elif distro_arch == "armhf": > + kernel_arch = "arm" > + elif distro_arch == "mipsel": > + kernel_arch = "mips" > + else: > + kernel_arch = "" > + return kernel_arch > + > +KERNEL_ARCH ??= "${@get_kernel_arch(d)}" > + > def config_fragments(d): > fragments = [] > sources = d.getVar("SRC_URI").split() > @@ -31,57 +113,35 @@ def config_fragments(d): > fragments.append(local) > return fragments > > -inherit dpkg-base > - > -SRC_URI += "file://build-kernel.sh" > +do_prepare_build_prepend() { > + # copy meta-data over to source tree > + rm -rf ${S}/debian > + cp -r ${WORKDIR}/debian ${S}/ > > -KBUILD_DEPENDS ?= " \ > - build-essential:native \ > - libssl-dev \ > - libelf-dev \ > - bc \ > - git \ > - kmod \ > - bison \ > - flex \ > - cpio \ > - libncurses-dev" > -KERNEL_DEBIAN_DEPENDS ?= "initramfs-tools | linux-initramfs-tool, kmod, linux-base (>= 4.3~)" > -KERNEL_HEADERS_DEBIAN_DEPENDS ?= "libc6, libssl1.1" > + # remove templates from the source tree > + find ${S}/debian -name *.tmpl | xargs rm -f > > -KERNEL_LIBC_DEV_DEPLOY ?= "0" > + # rename install/remove hooks to match user-specified name for our linux-image package > + mv ${S}/debian/linux-image.postinst ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.postinst > + mv ${S}/debian/linux-image.postrm ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.postrm > > -do_install_builddeps() { > - dpkg_do_mounts > - E="${@ bb.utils.export_proxies(d)}" > - sudo -E chroot ${BUILDCHROOT_DIR} \ > - apt-get update \ > - -o Dir::Etc::SourceList="sources.list.d/isar-apt.list" \ > - -o Dir::Etc::SourceParts="-" \ > - -o APT::Get::List-Cleanup="0" > - sudo -E chroot ${BUILDCHROOT_DIR} \ > - apt-get install \ > - -y -o Debug::pkgProblemResolver=yes \ > - --no-install-recommends ${KBUILD_DEPENDS} > - dpkg_undo_mounts > + # produce a changelog for our kernel build > + deb_add_changelog > } > > -addtask install_builddeps after do_prepare_build before do_dpkg_build > -# apt and reprepro may not run in parallel, acquire the Isar lock > -do_install_builddeps[lockfiles] += "${REPO_ISAR_DIR}/isar.lock" > - > -addtask devshell after do_install_builddeps > +# build directory for our "full" kernel build > +KERNEL_BUILD_DIR = "build-full" > > -dpkg_runbuild() { > - chmod +x ${WORKDIR}/build-kernel.sh > - KERNEL_CONFIG_TARGET="${KERNEL_DEFCONFIG}" > +dpkg_configure_kernel() { > + config_target="${KERNEL_DEFCONFIG}" > + rm -rf ${S}/${KERNEL_BUILD_DIR} && mkdir -p ${S}/${KERNEL_BUILD_DIR} > if [ -n "${KERNEL_DEFCONFIG}" ]; then > if [ -e "${WORKDIR}/${KERNEL_DEFCONFIG}" ]; then > - cp ${WORKDIR}/${KERNEL_DEFCONFIG} ${S}/.config > - KERNEL_CONFIG_TARGET="olddefconfig" > + cp ${WORKDIR}/${KERNEL_DEFCONFIG} ${S}/${KERNEL_BUILD_DIR}/.config > + config_target="olddefconfig" > fi > else > - KERNEL_CONFIG_TARGET="defconfig" > + config_target="defconfig" > fi > > # copy config fragments over to the kernel tree > @@ -92,21 +152,23 @@ dpkg_runbuild() { > mkdir -p ${S}/debian/fragments > (cd ${WORKDIR} && cp ${src_frags} ${S}/debian/fragments/) > fi > - export KERNEL_FRAGMENTS="${out_frags}" > > - E="${@ bb.utils.export_proxies(d)}" > - > - export PV=${PV} > - export KERNEL_NAME=${KERNEL_NAME_PROVIDED} > - > - export KBUILD_DEPENDS="${KBUILD_DEPENDS}" > - export KERNEL_CONFIG_TARGET > - export KERNEL_DEBIAN_DEPENDS="${KERNEL_DEBIAN_DEPENDS}" > - export KERNEL_FILE="${KERNEL_FILE}" > - export KERNEL_HEADERS_DEBIAN_DEPENDS="${KERNEL_HEADERS_DEBIAN_DEPENDS}" > + sudo -E chroot --userspec=$(id -u):$(id -g) ${BUILDCHROOT_DIR} sh -c " \ > + export ARCH=${KERNEL_ARCH} && \ > + cd ${PP}/${PPS} && \ > + make O=${KERNEL_BUILD_DIR} ${config_target} && \ > + ./scripts/kconfig/merge_config.sh \ > + -O ${KERNEL_BUILD_DIR}/ \ > + ${KERNEL_BUILD_DIR}/.config \ > + ${out_frags} \ > + " > +} > > - sudo -E chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} ${PP}/build-kernel.sh ${PP}/${PPS} ${DISTRO_ARCH} > +dpkg_runbuild_prepend() { > + dpkg_configure_kernel > +} > > +dpkg_runbuild_append() { > if [ "${KERNEL_LIBC_DEV_DEPLOY}" != "1" ]; then > rm -f ${WORKDIR}/linux-libc-dev_${PV}*.deb > fi > Valuable step forward! I still need to check some more details against my unfinished prototype, but it seems it generally a superset of it. Jan -- Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 83+ messages in thread
* Re: [PATCH 7/7] linux-custom: rewrite to no longer depend on the kernel's builddeb 2019-11-05 15:26 ` Jan Kiszka @ 2019-11-05 16:20 ` Cedric Hombourger 2019-11-05 16:22 ` Jan Kiszka 2019-11-05 20:16 ` chombourger 2 siblings, 0 replies; 83+ messages in thread From: Cedric Hombourger @ 2019-11-05 16:20 UTC (permalink / raw) To: Jan Kiszka, isar-users On 11/5/2019 4:26 PM, Jan Kiszka wrote: > On 05.11.19 14:57, Cedric Hombourger wrote: >> Regain control over the packaging of the Linux kernel by providing our >> own debian recipes (debian,{control,rules} files and friends) instead >> of using the kernel's builddeb script. This will allow generation of >> packages for "perf" (not included in this changeset) with per distro >> scheme (Debian and Ubuntu have differences) and maintain compatibility >> with upstream packages generated from different source packages but >> dependent on a certain deployment scheme (e.g. linux-base for Debian) >> >> Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com> >> --- >> doc/custom_kernel.md | 181 ++++++++++++++++ >> .../linux/files/build-kernel.sh | 128 ------------ >> meta/recipes-kernel/linux/files/debian/compat | 1 + >> .../linux/files/debian/control.tmpl | 36 ++++ >> meta/recipes-kernel/linux/files/debian/files | 5 + >> .../linux/files/debian/isar/build.tmpl | 36 ++++ >> .../linux/files/debian/isar/clean.tmpl | 20 ++ >> .../linux/files/debian/isar/common.tmpl | 60 ++++++ >> .../linux/files/debian/isar/install.tmpl | 197 ++++++++++++++++++ >> .../files/debian/linux-image.postinst.tmpl | 22 ++ >> .../files/debian/linux-image.postrm.tmpl | 18 ++ >> .../linux/files/debian/rules.tmpl | 39 ++++ >> meta/recipes-kernel/linux/linux-custom.inc | 172 ++++++++++----- >> 13 files changed, 732 insertions(+), 183 deletions(-) >> create mode 100644 doc/custom_kernel.md >> delete mode 100644 meta/recipes-kernel/linux/files/build-kernel.sh >> create mode 100644 meta/recipes-kernel/linux/files/debian/compat >> create mode 100644 meta/recipes-kernel/linux/files/debian/control.tmpl >> create mode 100644 meta/recipes-kernel/linux/files/debian/files >> create mode 100644 meta/recipes-kernel/linux/files/debian/isar/build.tmpl >> create mode 100644 meta/recipes-kernel/linux/files/debian/isar/clean.tmpl >> create mode 100644 meta/recipes-kernel/linux/files/debian/isar/common.tmpl >> create mode 100644 meta/recipes-kernel/linux/files/debian/isar/install.tmpl >> create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl >> create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl >> create mode 100755 meta/recipes-kernel/linux/files/debian/rules.tmpl >> >> diff --git a/doc/custom_kernel.md b/doc/custom_kernel.md >> new file mode 100644 >> index 0000000..5ab2599 >> --- /dev/null >> +++ b/doc/custom_kernel.md >> @@ -0,0 +1,181 @@ >> +# Custom kernel recipe for Isar >> + >> +## Contents >> + >> + - [Summary](#summary) >> + - [Proposal owners](#proposal-owners) >> + - [Current status](#current-status) >> + - [Detailed description](#detailed-description) >> + - [Benefit to Isar](#benefit-to-isar) >> + - [How can I help?](#how-can-i-help) >> + - [Scope](#scope) >> + - [How to test](#how-to-test) >> + - [Dependencies](#dependencies) >> + - [Documentation](#documentation) >> + - [Questions and answers](#questions-and-answers) >> + >> +## Summary >> + >> +Isar currently uses the Linux kernel's builddeb script to generate linux-image, >> +linux-headers and linux-libc-dev packages for your custom kernel sources. The >> +main benefit of this approach was that the low level details of how to package >> +the Linux kernel image, modules and headers were left in the kernel. There are >> +however some drawbacks: Isar had to repack generated packages to introduce some >> +of the changes it needs such as suffixing the kernel packages with the name of >> +the custom kernel (e.g. with "mainline" for the "linux-mainline" recipe), lack >> +of automatic dependencies (the builddeb script from the kernel does not use >> +debhelper), it was difficult to add packages especially distribution specific >> +packages (maintain version specific patch in Isar vs being able to upstream >> +distribution specific changes upstream), etc. This change is about being less >> +dependent on the kernel packaging scripts (which aren't used by major distros >> +anyway) and regain control on how we want or need the Linux kernel packaged. >> + >> +## Proposal owners >> + >> + * name: [Cedric Hombourger](https://github.com/chombourger) >> + >> +## Current status >> + >> +### Tests >> + >> +The following were recently checked: >> + >> + * Custom kernels in meta-isar build without (known) failures >> + * initrd image gets generated when the kernel is added to the rootfs >> + * `ISAR_CROSS_COMPILE` is supported >> + * `KERNEL_FILE` is honored >> + * `KERNEL_DEFCONFIG` may either be a file (specified via `SRC_URI`) or a >> + kernel-provided defconfig >> + * Config fragments get merged with the user-specified kernel configuration >> + * Path to installed DTBs may be configured (new) >> + * Configuration and build from a `devshell` >> + * Support for the `KERNEL_LIBC_DEV_DEPLOY` directive >> + >> +while the following needs work: >> + >> + * Check if custom-kernel builds are reproducible >> + >> +### Remaining work items >> + >> + * Check if there is a better way to cross compile kernel "scripts" such as >> + `fixdep` or `modpost` than manually compiling them (they are compiled for >> + the build machine (amd64) and not for the target (e.g. armhf) even though >> + we are cross-compiling - yet the linux-headers package should be shipping >> + "scripts" for the target machine) >> + >> +## Detailed description >> + >> +### Requirements >> + >> +#### Current >> + >> +Isar has had a `linux-custom` recipe for a long time and counts many users. It is therefore >> +paramount to maintain features that it has today with no or little changes to the APIs. The >> +following requirements were identified and will be maintained: >> + >> + 1. Sources to the custom Linux kernel may be specified via `SRC_URI` >> + >> + 2. Configure the kernel via an in-tree or an external `defconfig` > Reference to KERNEL_DEFCONFIG is missing. Added (locally) > >> + >> + 3. Support kernel configuration tweaks via configuration fragments (`.cfg` files) >> + >> + 4. Patches to the linux kernel may be specified via `SRC_URI` >> + >> + 5. Ensure that the Isar recipe `PV` matches the kernel release identifier (`KERNEL_RELEASE`) >> + >> + 6. Produce a `linux-image` package that ships the kernel image and modules >> + >> + 7. Allow the name of the kernel image to be changed via `KERNEL_FILE` (defaults to `vmlinuz`) >> + >> + 8. Produce a `linux-headers` package which includes kernel headers and kbuild scripts/tools >> + >> + 9. The `linux-headers` package shall support native and cross compiles of out-of-tree kernel modules > One package to serve them all? Is that the use case for patch 6? IOW, we > are now building linux-headers for the target, not the host OS anymore? > What's the slowdown compare to cross-linux-headers? well to the best of my knowledge Debian does not have a packaging scheme defined to support cross-compiles of out-of-tree kernel modules. That's something we can definitely invent but would recommend we make it a phase 2 enhancement. If I missed something and overlooked what Debian provides, please let me know! the overall build times with the new build scripts are way faster overall but I'll get some numbers for you > The current policy is that, when cross-building, our -dev packagages > (which is what linux-headers falls under) will work natively with the > buildchroot. The future path is clearly generating fitting packages for > both host and target. Or is there anything in the linux-headers that > prevents providing both? we probably should generate linux-headers for both amd64 and DISTRO_ARCH for the buildchroot to pick up the one we need for both scenario but would need to check if we can keep debhelper happy when it sees binaries for an architecture not matching the main architecture we are targeting > >> + >> + 10. Produce a `linux-libc-dev` package to support user-land builds > This should be opt-in only: No need to build it when we don't ship it. > Just add the related fragement to debian/control when > KERNEL_LIBC_DEV_DEPLOY is set. Ok. Will use BuildProfiles to achieve this > >> + >> + 11. Only ship the `linux-libc-dev` package if instructed to (`KERNEL_LIBC_DEV_DEPLOY` equals to `"1"`) >> + >> + 12. Support both native and cross compiles (`ISAR_CROSS_COMPILE`) >> + >> + 13. Support for the following kernel architectures: >> + >> + * arm >> + * arm64 >> + * mips >> + * x86 >> + >> + 14. Support `devshell` (kernel configuration shall be applied) >> + >> +#### New requirements (proposed) >> + >> +This proposal includes the following new requirements: >> + >> + 1. Support creation of additional packages >> + >> + 2. Allow per-distro packaging tweaks >> + >> +### Approach >> + >> +The new implementation will have the `builddeb` code ported from the Linux kernel included to the >> +Isar environment. It will be refactor to support pre and post hooks. Hooks may be used by external >> +layers to extend the kernel packaging instructions provided by "stock" Isar or by Isar itself to >> +implement distro-specific packages or tweaks. >> + >> +Hooks need the ability to add `build` and `install` steps which can be achieved by having the main >> +`build` and `install` steps call pre and post scripts. This will however solve only one side of the >> +problem: new packages should be declared in the `debian/control` file. The packaging process >> +(initiated by `debian/rules`) will use a baseline `control` file when started and a mechanism will >> +be provided for hooks to append `control` blocks as they are processed. > Is this a section addressing users or reviewers? The latter audience is > better addressed via the commit log. reviewers - it is anticipated that the rework will be a multi-phase project > >> + >> +### Future work >> + >> +In the event where this proposal is accepted, here are a few future projects that are being >> +considered: >> + >> + 1. Package `perf` as Debian does >> + >> + 2. Add `ubuntu` as an Isar supported target distro and validate `linux-custom` recipe >> + >> +## Benefit to Isar >> + >> + * Control the kernel build and packaging process >> + >> + * No longer need to repack packages >> + >> + * Provide hooks for custom kernel recipes to add build/install directives > Careful with hooks! They tend to pile up mess. Where exactly do you want > them? What for? They would be be internal to the recipe and used to handle distro differences for packages such as perf If we are happy to have if [ "${BASE_DISTRO}" = "ubuntu" ]; then do_x; fi in the code (such as install.tmpl) and handle tweaks in-place then I can easily remove calls to .d scripts (e.g. debian/post.d/install/) > >> + >> +## How can I help? >> + >> + * Check if you are able to build your own linux-custom recipes with this change >> + * Check if packages generated by this recipe are compatible with your Debian-based distro >> + * Review and provide comments on this changeset >> + * Create templates for distro-specific packages (e.g. perf) >> + >> +## Scope >> + >> + * Proposal owners: proposal owners are to communicate the idea to isar-users and make sure >> + that all of their requirements are met (to the extent they are already supported with the >> + current solution) or may be met in the future. >> + >> + * Other developers: port your out-of-tree linux-custom recipes to this new solution and >> + report issues. >> + >> +## How to test? >> + >> +The linux-mainline recipe may be used for some basic testing. This recipe is being used by the >> +following machines: >> + >> + * de0-nano-soc >> + * qemumipsel >> + >> +## Dependencies >> + >> + * None >> + >> +## Documentation >> + >> + * Document user-visible variables added by this proposal >> + >> +## Questions and answers >> + >> + * None >> diff --git a/meta/recipes-kernel/linux/files/build-kernel.sh b/meta/recipes-kernel/linux/files/build-kernel.sh >> deleted file mode 100644 >> index f56e96c..0000000 >> --- a/meta/recipes-kernel/linux/files/build-kernel.sh >> +++ /dev/null >> @@ -1,128 +0,0 @@ >> -#!/bin/bash >> -# >> -# Custom kernel build >> -# >> -# This software is a part of ISAR. >> -# Copyright (c) Siemens AG, 2018 >> -# >> -# SPDX-License-Identifier: MIT >> - >> -source /isar/common.sh >> - >> -host_arch=$(dpkg --print-architecture) >> - >> -if [ "$host_arch" != "$target_arch" ]; then >> - case $target_arch in >> - armhf) >> - export ARCH=arm >> - export CROSS_COMPILE="arm-linux-gnueabihf-" >> - ;; >> - arm64) >> - export ARCH=arm64 >> - export CROSS_COMPILE="aarch64-linux-gnu-" >> - ;; >> - mipsel) >> - export ARCH=mips >> - export CROSS_COMPILE="mipsel-linux-gnu-" >> - ;; >> - *) >> - echo "error: unsupported architecture ($target_arch)" >> - exit 1 >> - ;; >> - esac >> -fi >> - >> -REPACK_DIR="$1/../repack" >> -REPACK_LINUX_IMAGE_DIR="${REPACK_DIR}/linux-image" >> -REPACK_LINUX_HEADERS_DIR="${REPACK_DIR}/linux-headers" >> - >> -make ${KERNEL_CONFIG_TARGET} || exit ${?} >> -if [ -n "${KERNEL_FRAGMENTS}" ]; then >> - scripts/kconfig/merge_config.sh -m .config ${KERNEL_FRAGMENTS} >> -fi >> - >> -KV=$( make -s kernelrelease ) >> -if [ "${KV}" != "${PV}" ]; then >> - echo "ERROR: Recipe PV is \"${PV}\" but should be \"${KV}\"" 1>&2 >> - echo "ERROR: Probably due to CONFIG_LOCALVERSION" 1>&2 >> - exit 1 >> -fi >> - >> -rm -f .version >> -KBUILD_DEBARCH=$target_arch make -j $(($(nproc) * 2)) deb-pkg >> - >> -rm -rf "${REPACK_DIR}" >> -mkdir -p "${REPACK_DIR}" >> -mkdir -p "${REPACK_LINUX_IMAGE_DIR}" >> -mkdir -p "${REPACK_LINUX_HEADERS_DIR}" >> - >> -cp -a debian "${REPACK_DIR}" >> - >> -# dpkg-gencontrol performs cross-incompatible checks on the >> -# Architecture field; trick it to accept the control file >> -sed -i "s/Architecture: .*/Architecture: any/" "${REPACK_DIR}/debian/control" >> - >> -cd .. >> - >> -dpkg-deb -R linux-image-${PV}_${PV}-1_*.deb "${REPACK_LINUX_IMAGE_DIR}" >> -dpkg-deb -R linux-headers-${PV}_${PV}-1_*.deb "${REPACK_LINUX_HEADERS_DIR}" >> - >> -dpkg-gencontrol -crepack/debian/control \ >> - -lrepack/debian/changelog \ >> - -frepack/debian/files \ >> - -plinux-image-${PV} \ >> - -P"${REPACK_LINUX_IMAGE_DIR}" \ >> - -DPackage="linux-image-${KERNEL_NAME}" \ >> - -DSection=kernel \ >> - -DPriority=required \ >> - -DDepends="${KERNEL_DEBIAN_DEPENDS}" \ >> - -DArchitecture=$target_arch >> - >> -# Add Debian-like link installation to postinst >> -mkdir -p ${REPACK_LINUX_IMAGE_DIR}/lib/modules/${PV} >> -touch "${REPACK_LINUX_IMAGE_DIR}/lib/modules/${PV}/.fresh-install" >> -sed -i "${REPACK_LINUX_IMAGE_DIR}/DEBIAN/postinst" \ >> - -e "/^set -e$/a\\ >> -\\ >> -if [ -f /lib/modules/${PV}/.fresh-install ]; then\\ >> - change=install\\ >> -else\\ >> - change=upgrade\\ >> -fi\\ >> -linux-update-symlinks \$change ${PV} /boot/${KERNEL_FILE}-${PV}\\ >> -rm -f /lib/modules/${PV}/.fresh-install" >> - >> -# Add Debian-like link removal to postrm >> -sed -i "${REPACK_LINUX_IMAGE_DIR}/DEBIAN/postrm" \ >> - -e "/^set -e$/a\\ >> -\\ >> -rm -f /lib/modules/${PV}/.fresh-install\\ >> -\\ >> -if [ \"\$1\" != upgrade ] && command -v linux-update-symlinks >/dev/null; then\\ >> - linux-update-symlinks remove ${PV} /boot/${KERNEL_FILE}-${PV}\\ >> -fi" >> - >> -# Make sure arm64 kernels are decompressed >> -if [ "$target_arch" = "arm64" ]; then >> - kernel_file="${REPACK_LINUX_IMAGE_DIR}/boot/${KERNEL_FILE}-${PV}" >> - mv "${kernel_file}" "${kernel_file}.gz" >> - gunzip "${kernel_file}.gz" >> -fi >> - >> -dpkg-gencontrol -crepack/debian/control \ >> - -lrepack/debian/changelog \ >> - -frepack/debian/files \ >> - -plinux-headers-${PV} \ >> - -P"${REPACK_LINUX_HEADERS_DIR}" \ >> - -Vkernel:debarch="${KERNEL_NAME}" \ >> - -DPackage="linux-headers-${KERNEL_NAME}" \ >> - -DSection=kernel \ >> - -DDepends="${KERNEL_HEADERS_DEBIAN_DEPENDS}" \ >> - -DArchitecture=$target_arch >> - >> -fakeroot dpkg-deb -b "${REPACK_LINUX_IMAGE_DIR}" \ >> - linux-image-${KERNEL_NAME}_${PV}-1_${KERNEL_NAME}.deb >> -rm -f linux-image-${PV}_${PV}-1_*.deb >> -fakeroot dpkg-deb -b "${REPACK_LINUX_HEADERS_DIR}" \ >> - linux-headers-${KERNEL_NAME}_${PV}-1_${KERNEL_NAME}.deb >> -rm -f linux-headers-${PV}_${PV}-1_*.deb >> diff --git a/meta/recipes-kernel/linux/files/debian/compat b/meta/recipes-kernel/linux/files/debian/compat >> new file mode 100644 >> index 0000000..ec63514 >> --- /dev/null >> +++ b/meta/recipes-kernel/linux/files/debian/compat >> @@ -0,0 +1 @@ >> +9 >> diff --git a/meta/recipes-kernel/linux/files/debian/control.tmpl b/meta/recipes-kernel/linux/files/debian/control.tmpl >> new file mode 100644 >> index 0000000..1295466 >> --- /dev/null >> +++ b/meta/recipes-kernel/linux/files/debian/control.tmpl >> @@ -0,0 +1,36 @@ >> +Source: linux-${KERNEL_NAME_PROVIDED} >> +Section: kernel >> +Priority: optional >> +Maintainer: ${MAINTAINER} >> +Build-Depends: bc, kmod, cpio, ${KBUILD_DEPENDS} >> +Homepage: http://www.kernel.org/ > Standard-Version? > >> + >> +Package: linux-image-${KERNEL_NAME_PROVIDED} >> +Architecture: any >> +Depends: ${KERNEL_DEBIAN_DEPENDS} >> +Description: ${KERNEL_NAME_PROVIDED} Linux kernel, version ${PV} >> + This package contains the Linux kernel, modules and corresponding other >> + files, version: ${PV}. >> + >> +Package: linux-headers-${KERNEL_NAME_PROVIDED} >> +Architecture: any >> +Depends: ${KERNEL_HEADERS_DEBIAN_DEPENDS}, ${perl:Depends}, ${shlib:Depends} >> +Description: ${KERNEL_NAME_PROVIDED} Linux kernel headers for ${PV} >> + This package provides kernel header files for ${PV} on ${DISTRO_ARCH} >> + . >> + This is useful for people who need to build external modules >> + >> +Package: linux-libc-dev > See above, should not be here by default. > >> +Section: devel >> +Provides: linux-kernel-headers >> +Architecture: any >> +Description: Linux support headers for userspace development >> + This package provides userspaces headers from the Linux kernel. These headers >> + are used by the installed headers for GNU glibc and other system libraries. >> + >> +Package: linux-image-${KERNEL_NAME_PROVIDED}-dbg >> +Section: debug >> +Architecture: any >> +Description: Linux kernel debugging symbols for ${PV} >> + This package will come in handy if you need to debug the kernel. It provides >> + all the necessary debug symbols for the kernel and its modules. >> diff --git a/meta/recipes-kernel/linux/files/debian/files b/meta/recipes-kernel/linux/files/debian/files >> new file mode 100644 >> index 0000000..a8ace32 >> --- /dev/null >> +++ b/meta/recipes-kernel/linux/files/debian/files >> @@ -0,0 +1,5 @@ >> +linux-headers_4.19.80-1_mipsel.deb kernel optional >> +linux-image-dbg_4.19.80-1_mipsel.deb debug optional >> +linux-image_4.19.80-1_mipsel.deb kernel optional >> +linux-libc-dev_4.19.80-1_mipsel.deb devel optional >> +linux_4.19.80-1_mipsel.buildinfo kernel optional > Looks like an accidentally added file. > >> diff --git a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl >> new file mode 100644 >> index 0000000..ce79d56 >> --- /dev/null >> +++ b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl >> @@ -0,0 +1,36 @@ >> +#!/bin/sh >> +# Copyright (c) Mentor Graphics, a Siemens business, 2019 >> +# SPDX-License-Identifier: MIT >> + >> +# Load common stuff >> +. ${S}/debian/isar/common || exit ${?} >> + >> +do_build() { >> + >> + # Print a few things that are of particular interest >> + print_settings >> + >> + # Trace what we do here >> + set -x >> + >> + # Process existing kernel configuration to make sure it is complete >> + # (use defaults for options that were not specified) >> + ${MAKE} O=${KERNEL_BUILD_DIR} olddefconfig || exit ${?} >> + >> + # Build the Linux kernel >> + ${MAKE} O=${KERNEL_BUILD_DIR} -j $(nproc) || exit ${?} >> + >> + # Stop tracing >> + set +x >> +} >> + >> +print_settings() { >> + cat <<EOF >> +# Build settings: >> +# --------------- >> +# ARCH=${ARCH} >> +# CROSS_COMPILE=${CROSS_COMPILE} >> +EOF >> +} >> + >> +main build ${*} >> diff --git a/meta/recipes-kernel/linux/files/debian/isar/clean.tmpl b/meta/recipes-kernel/linux/files/debian/isar/clean.tmpl >> new file mode 100644 >> index 0000000..2aa3742 >> --- /dev/null >> +++ b/meta/recipes-kernel/linux/files/debian/isar/clean.tmpl >> @@ -0,0 +1,20 @@ >> +#!/bin/sh >> +# Copyright (c) Mentor Graphics, a Siemens business, 2019 >> +# SPDX-License-Identifier: MIT >> + >> +# Load common stuff >> +. ${S}/debian/isar/common || exit ${?} >> + >> +do_clean() { >> + >> + # Trace what we do here >> + set -x >> + >> + rm -rf ${deb_img_dir} ${deb_dbg_dir} >> + rm -rf ${deb_kern_hdrdir} ${deb_libc_hdr_dir} > I suppose we rather (or also?) want "make clean" here for the build > artifacts. Just make sure to save/restore the debian folder across that > step. I had it but do we really want it? if we do, you won't get incremental builds > >> + >> + # Stop tracing >> + set +x >> +} >> + >> +main clean ${*} >> diff --git a/meta/recipes-kernel/linux/files/debian/isar/common.tmpl b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl >> new file mode 100644 >> index 0000000..b392b46 >> --- /dev/null >> +++ b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl >> @@ -0,0 +1,60 @@ >> +#!/bin/sh >> +# Copyright (c) Mentor Graphics, a Siemens business, 2019 >> +# SPDX-License-Identifier: MIT >> + >> +# Isar settings >> +ARCH=${KERNEL_ARCH} >> +KERNEL_PKG_IMAGE=linux-image-${KERNEL_NAME_PROVIDED} >> +KERNEL_PKG_KERN_HEADERS=linux-headers-${KERNEL_NAME_PROVIDED} >> +KERNEL_PKG_LIBC_HEADERS=linux-libc-dev >> + >> +# Constants >> +KCONF=.config >> + >> +# Target directories >> +deb_top_dir=${S}/debian >> +deb_img_dir=${deb_top_dir}/${KERNEL_PKG_IMAGE} >> +deb_dbg_dir=${deb_img_dir}-dbg >> +deb_dtb_dir=${deb_img_dir}/${KERNEL_DTB_DIR} >> +deb_kern_hdr_dir=${deb_top_dir}/${KERNEL_PKG_KERN_HEADERS} >> +deb_libc_hdr_dir=${deb_top_dir}/${KERNEL_PKG_LIBC_HEADERS} >> + >> +main() { >> + target=${1} >> + >> + if [ ! -f ${S}/debian/isar/${target} ]; then >> + echo "error: ${target} is not a supported build target!" >&2 >> + return 1 >> + fi >> + >> + # check for scripts to be sourced >> + for f in ${S}/debian/isar/defaults.d/${target}/*; do >> + [ -f ${f} ] || continue >> + . ${f} || return ${?} >> + done >> + >> + # variables to be exported >> + export ARCH >> + >> + # are we cross-compiling? >> + BUILD_ARCH=$(dpkg-architecture -qDEB_BUILD_ARCH) >> + if [ "${BUILD_ARCH}" = "${DISTRO_ARCH}" ]; then >> + # no, make sure CROSS_COMPILE isn't set >> + unset CROSS_COMPILE >> + fi >> + >> + # check for pre-target scripts >> + for f in ${S}/debian/isar/pre.d/${target}/*; do >> + [ -f ${f} ] || continue >> + sh ${f} || return ${?} >> + done >> + >> + # call the actual target script >> + do_${target} || return ${?} >> + >> + # check for post-target scripts >> + for f in ${S}/debian/isar/post.d/${target}/*; do >> + [ -f ${f} ] || continue >> + sh ${f} || return ${?} >> + done >> +} >> diff --git a/meta/recipes-kernel/linux/files/debian/isar/install.tmpl b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl >> new file mode 100644 >> index 0000000..2055669 >> --- /dev/null >> +++ b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl >> @@ -0,0 +1,197 @@ >> +#!/bin/sh >> +# Copyright (c) Mentor Graphics, a Siemens business, 2019 >> +# SPDX-License-Identifier: MIT >> + >> +# Load common stuff >> +. ${S}/debian/isar/common || exit ${?} >> + >> +do_install() { >> + >> + # check if our kernel was configured >> + if [ ! -f "${O}/.config" ]; then >> + echo "error: kernel not configured!" >&2 >> + return 1 >> + fi >> + >> + # load its configuration >> + . ${O}/.config >> + >> + kimage="$(${MAKE} O=${O} -s --no-print-directory image_name)" >> + case "${ARCH}" in >> + mips|powerpc) kimage_path="boot/vmlinux-${PV}" ;; >> + um) kimage_path="usr/bin/vmlinux-${PV}" ;; >> + *) kimage_path="boot/vmlinuz-${PV}" ;; >> + esac >> + >> + print_settings >> + >> + # Stop on error >> + set -e >> + >> + # Trace what we do here >> + set -x >> + >> + install_image >> + install_hooks >> + install_dtbs >> + install_kmods >> + install_headers >> + >> + # Stop tracing >> + set +x >> +} >> + >> +print_settings() { >> + cat <<EOF >> +Install settings: >> +----------------- >> +deb_dtb_dir=${deb_dtb_dir} >> +deb_hdr_dir=${deb_hdr_dir} >> +kimage=${kimage} >> +kimage_path=${kimage_path} >> + >> +EOF >> +} >> + >> +install_image() { >> + install -m 755 -d ${deb_img_dir}/$(dirname ${kimage_path}) >> + cp ${O}/${kimage} ${deb_img_dir}/${kimage_path} >> + install_image_debug >> +} >> + >> +install_image_debug() { >> + # Different tools want the image in different locations >> + # perf >> + mkdir -p ${deb_dbg_dir}/usr/lib/debug/lib/modules/${PV}/ >> + cp vmlinux ${deb_dbg_dir}/usr/lib/debug/lib/modules/${PV}/ >> + # systemtap >> + mkdir -p ${deb_dbg_dir}/usr/lib/debug/boot/ >> + ln -s ../lib/modules/$version/vmlinux ${deb_dbg_dir}/usr/lib/debug/boot/vmlinux-${PV} >> + # kdump-tools >> + ln -s lib/modules/${PV}/vmlinux ${deb_dbg_dir}/usr/lib/debug/vmlinux-${PV} >> +} >> + >> +install_hooks() { >> + install -m 755 -d ${deb_img_dir}/etc/kernel/install.d >> + install -m 755 -d ${deb_img_dir}/etc/kernel/postinst.d >> + install -m 755 -d ${deb_img_dir}/etc/kernel/postrm.d >> + install -m 755 -d ${deb_img_dir}/etc/kernel/prerm.d >> +} >> + >> +install_dtbs() { >> + [ -n "${CONFIG_OF}" ] || return 0 >> + ${MAKE} O=${O} INSTALL_DTBS_PATH=${deb_dtb_dir} dtbs_install >> +} >> + >> +install_kmods() { >> + [ -n "${CONFIG_MODULES}" ] || return 0 >> + ${MAKE} O=${O} INSTALL_MOD_PATH=${deb_img_dir} modules_install >> + touch ${deb_img_dir}/lib/modules/${PV}/.fresh-install >> + rm -fv ${deb_img_dir}/lib/modules/${PV}/build >> + rm -fv ${deb_img_dir}/lib/modules/${PV}/source >> + install_kmods_debug >> +} >> + >> +kmods_sign() { >> + [ -n "${CONFIG_MODULE_SIG_ALL}" ] || return 0 >> + ${MAKE} O=${O} INSTALL_MOD_PATH=${deb_img_dir} modules_sign >> +} >> + >> +install_kmods_debug() { >> + [ -n "${CONFIG_DEBUG_INFO}" ] || return 0 >> + >> + kmod_inst_dir=${deb_img_dir}/lib/modules >> + kmod_debug_dir=${deb_dbg_dir}/usr/lib/debug >> + >> + # copy kernels modules to usr/lib/debug >> + mkdir -p ${kmod_debug_dir} >> + tar -C ${kmod_inst_dir}/ -cO --exclude='modules.*' . | tar -C ${kmod_debug_dir}/ -xf - >> + # strip everything but debug sections for modules in usr/lib/debug >> + find ${kmod_debug_dir} -name *.ko -exec ${CROSS_COMPILE}objcopy --only-keep-debug {} \; >> + # and strip debug sections from modules in lib/modules >> + find ${kmod_inst_dir} -name *.ko -exec ${CROSS_COMPILE}objcopy --strip-debug {} \; >> + >> + # re-sign stripped kernel modules >> + kmods_sign >> +} >> + >> +headers_check() { >> + ${MAKE} O=${O} headers_check >> +} >> + >> +libc_headers() { >> + mkdir -p ${deb_libc_hdr_dir} >> + ${MAKE} O=${O} headers_install INSTALL_HDR_PATH=${deb_libc_hdr_dir}/usr >> + host_arch=$(dpkg-architecture -a${DISTRO_ARCH} -qDEB_HOST_MULTIARCH) >> + mkdir ${deb_libc_hdr_dir}/usr/include/${host_arch} >> + mv ${deb_libc_hdr_dir}/usr/include/asm ${deb_libc_hdr_dir}/usr/include/${host_arch}/ >> +} >> + >> +# the kernel does not seem to provide a way to cross-compile its tools, we however >> +# have qemu-user-static binaries in our buildchroot. Create a new build tree for the >> +# scripts and override HOSTCC to force a cross-compile >> +kernel_tools_build() { >> + odir="${S}/build-scripts" >> + mflags="HOSTCC=${CROSS_COMPILE}gcc O=${odir} NOSTDINC_FLAGS=" >> + rm -rf ${odir} && mkdir -p ${odir} && cp ${S}/${KERNEL_BUILD_DIR}/.config ${odir} >> + (unset LD_PRELOAD; cd ${odir} && make -C ${S} ${mflags} olddefconfig scripts) >> + (cd ${odir}; find scripts -type f -executable -exec file {} \;|grep ELF|cut -d: -f1) >${src_hdr_files} >> + tar -C ${odir} -cf - -T - <${src_hdr_files} | tar -C ${destdir} -xf - >> +} >> + >> +kernel_tools() { >> + # remove object files >> + find ${destdir}/scripts -type f -name '*.o' |xargs rm -f >> + >> + # we're all done if we aren't cross-compiling >> + [ -n "${CROSS_COMPILE}" ] || return 0 >> + >> + # remove ELF executables from the linux-headers package >> + find ${destdir}/scripts -type f -exec file {} \;|grep ELF|cut -d: -f1|xargs rm -fv >> + >> + # cross-compile kernel tools to be shipped with linux-headers >> + kernel_tools_build >> +} >> + >> +kernel_headers() { >> + destdir=${deb_kern_hdr_dir}/${KERNEL_HEADERS_DIR} >> + src_hdr_files=$(mktemp) >> + obj_hdr_files=$(mktemp) >> + >> + mkdir -p ${destdir} >> + mkdir -p ${deb_kern_hdr_dir}/lib/modules/${PV} >> + >> + (cd ${S}; find . -name 'Makefile*' -o -name 'Kconfig*' -o -name '*.pl') >>${src_hdr_files} >> + (cd ${S}; find arch/*/include include scripts -type f -o -type l) >>${src_hdr_files} >> + (cd ${S}; find arch/${ARCH} -name module.lds -o -name Kbuild.platforms -o -name Platform) >>${src_hdr_files} >> + (cd ${S}; find $(find arch/${ARCH} -name include -o -name scripts -type d) -type f) >>${src_hdr_files} >> + >> + (cd ${O}; find arch/${ARCH}/include Module.symvers include scripts -type f) >>${obj_hdr_files} >> + if [ -n "${CONFIG_STACK_VALIDATION}" ]; then >> + (cd ${O}; find tools/objtool -type f -executable) >>${obj_hdr_files} >> + fi >> + if [ -n "${CONFIG_GCC_PLUGINS}" ]; then >> + (cd ${O}; find scripts/gcc-plugins -name *.so -o -name gcc-common.h) >>${obj_hdr_files} >> + fi >> + >> + # deploy files that were matched above >> + tar -C ${S} -cf - -T - <${src_hdr_files} | tar -C ${destdir} -xf - >> + tar -C ${O} -cf - -T - <${obj_hdr_files} | tar -C ${destdir} -xf - >> + >> + # add the kernel config >> + cp ${O}/${KCONF} ${destdir}/.config >> + >> + # handle kernel development tools >> + kernel_tools >> + >> + # create symlinks >> + ln -sf /${KERNEL_HEADERS_DIR} ${deb_kern_hdr_dir}/lib/modules/${PV}/build >> +} >> + >> +install_headers() { >> + headers_check >> + libc_headers >> + kernel_headers >> +} >> + >> +main install ${*} >> diff --git a/meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl b/meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl >> new file mode 100644 >> index 0000000..fd89121 >> --- /dev/null >> +++ b/meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl >> @@ -0,0 +1,22 @@ >> +#!/bin/sh >> + >> +set -e >> +set -x >> + >> +if [ -f /lib/modules/${PV}/.fresh-install ]; then >> + change=install >> +else >> + change=upgrade >> +fi >> + >> +linux-update-symlinks $change ${PV} /boot/${KERNEL_FILE}-${PV} >> +rm -f /lib/modules/${PV}/.fresh-install >> + >> +# Pass maintainer script parameters to hook scripts >> +export DEB_MAINT_PARAMS="$*" >> + >> +# Tell initramfs builder whether it's wanted >> +export INITRD=Yes > I've noticed while hacking my version that this should follow > CONFIG_BLK_DEV_INITRD. oh true. I meant to have it too and forgot. good catch! >> + >> +test -d /etc/kernel/postinst.d && run-parts --arg="${PV}" --arg="/boot/${KERNEL_FILE}-${PV}" /etc/kernel/postinst.d >> +exit 0 >> diff --git a/meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl b/meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl >> new file mode 100644 >> index 0000000..9d88218 >> --- /dev/null >> +++ b/meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl >> @@ -0,0 +1,18 @@ >> +#!/bin/sh >> + >> +set -e >> + >> +rm -f /lib/modules/${PV}/.fresh-install >> + >> +if [ "$1" != upgrade ] && command -v linux-update-symlinks >/dev/null; then >> + linux-update-symlinks remove ${PV} /boot/${KERNEL_FILE}-${PV} >> +fi >> + >> +# Pass maintainer script parameters to hook scripts >> +export DEB_MAINT_PARAMS="$*" >> + >> +# Tell initramfs builder whether it's wanted >> +export INITRD=Yes >> + >> +test -d /etc/kernel/postrm.d && run-parts --arg="${PV}" --arg="/boot/${KERNEL_FILE}-${PV}" /etc/kernel/postrm.d >> +exit 0 >> diff --git a/meta/recipes-kernel/linux/files/debian/rules.tmpl b/meta/recipes-kernel/linux/files/debian/rules.tmpl > There is quite some duplication between postinst and postrm already. > Moreover, I think you are missing preinst and prerm. I think all four > should probably be generated from one template. Ok > >> new file mode 100755 >> index 0000000..93c8a5b >> --- /dev/null >> +++ b/meta/recipes-kernel/linux/files/debian/rules.tmpl >> @@ -0,0 +1,39 @@ >> +#!/usr/bin/make -f >> + >> +export CROSS_COMPILE=$(DEB_HOST_GNU_TYPE)- >> + >> +O:=$(CURDIR)/${KERNEL_BUILD_DIR} >> +S:=$(CURDIR) >> +deb_top_dir:=$(S)/debian >> + >> +DH_VERBOSE=1 >> + >> +# Dynamic variables to be passed to Isar build scripts >> +isar_env=$(strip \ >> + export CROSS_COMPILE='$(CROSS_COMPILE)' && \ >> + export MAKE='$(MAKE)' && \ >> + export O='${O}' \ >> + export S='${S}' \ >> +) >> + >> +%: >> + dh $(@) >> + >> +.PHONY: override_dh_auto_clean >> +override_dh_auto_clean: >> + $(isar_env) && sh $(deb_top_dir)/isar/clean >> + >> +.PHONY: override_dh_auto_build >> +override_dh_auto_build: >> + $(isar_env) && sh $(deb_top_dir)/isar/build >> + >> +override_dh_auto_install: >> + $(isar_env) && sh $(deb_top_dir)/isar/install >> + >> +.PHONY: override_dh_auto_test >> +override_dh_auto_test: >> + true >> + >> +PHONY: override_dh_strip >> +override_dh_strip: >> + dh_strip -Xvmlinux --no-automatic-dbgsym >> diff --git a/meta/recipes-kernel/linux/linux-custom.inc b/meta/recipes-kernel/linux/linux-custom.inc >> index ca91f64..36086be 100644 >> --- a/meta/recipes-kernel/linux/linux-custom.inc >> +++ b/meta/recipes-kernel/linux/linux-custom.inc >> @@ -2,22 +2,104 @@ >> # >> # This software is a part of ISAR. >> # Copyright (c) Siemens AG, 2018 >> +# Copyright (c) Mentor Graphics, a Siemens business, 2019 >> # >> # SPDX-License-Identifier: MIT >> >> -FILESEXTRAPATHS_prepend := "${FILE_DIRNAME}/files:" >> +# Settings that would typically be done from the custom kernel recipe >> +# ------------------------------------------------------------------- >> >> DESCRIPTION ?= "Custom kernel" >> +MAINTAINER ?= "isar-users <isar-users@googlegroups.com>" >> + >> +KBUILD_DEPENDS ?= "build-essential:native, \ >> + libelf-dev:native, \ >> + libncurses-dev:native, \ >> + libssl-dev:native, \ >> + bc, \ >> + bison, \ >> + cpio, \ >> + flex, \ >> + git, \ >> + kmod, \ >> + libssl-dev," >> + >> +KERNEL_DEBIAN_DEPENDS ?= "initramfs-tools | linux-initramfs-tool, \ >> + kmod, \ >> + linux-base (>= 4.3~)," >> + >> +KERNEL_HEADERS_DEBIAN_DEPENDS ?= "libc6, \ >> + libssl1.1," >> + >> +KERNEL_LIBC_DEV_DEPLOY ?= "0" >> + >> +# Settings that may be changed on a per distro, machine or layer basis >> +# -------------------------------------------------------------------- >> + >> +KERNEL_DTB_DIR ?= "usr/lib/linux-${KERNEL_NAME_PROVIDED}-${PV}" >> +KERNEL_HEADERS_DIR ?= "usr/src/linux-headers-${PV}" >> >> -KERNEL_NAME_PROVIDED ?= "${@ d.getVar('PN', True).partition('linux-')[2]}" >> KERNEL_DEFCONFIG ?= "" >> >> +# Add our template meta-data to the sources >> +FILESEXTRAPATHS_prepend := "${FILE_DIRNAME}/files:" >> +SRC_URI += "file://debian" >> + >> +# Variables and files that make our templates >> +# ------------------------------------------- >> + >> +TEMPLATE_FILES += " \ >> + debian/control.tmpl \ >> + debian/isar/build.tmpl \ >> + debian/isar/clean.tmpl \ >> + debian/isar/common.tmpl \ >> + debian/isar/install.tmpl \ >> + debian/linux-image.postinst.tmpl \ >> + debian/linux-image.postrm.tmpl \ >> + debian/rules.tmpl \ >> +" >> + >> +TEMPLATE_VARS += " \ >> + KBUILD_DEPENDS \ >> + KERNEL_ARCH \ >> + KERNEL_DEBIAN_DEPENDS \ >> + KERNEL_BUILD_DIR \ >> + KERNEL_DTB_DIR \ >> + KERNEL_FILE \ >> + KERNEL_HEADERS_DEBIAN_DEPENDS \ >> + KERNEL_HEADERS_DIR \ >> + KERNEL_NAME_PROVIDED \ >> +" >> + >> +inherit dpkg >> +inherit template >> + >> +# Derive name of the kernel packages from the name of this recipe >> +KERNEL_NAME_PROVIDED ?= "${@ d.getVar('PN', True).partition('linux-')[2]}" >> + >> +# Make bitbake know we will be producing linux-image and linux-headers packages >> python() { >> kernel_name = d.getVar("KERNEL_NAME_PROVIDED", True) >> d.setVar('PROVIDES', 'linux-image-' + kernel_name + ' ' + \ >> 'linux-headers-' + kernel_name) >> } >> >> +def get_kernel_arch(d): >> + distro_arch = d.getVar("DISTRO_ARCH") >> + if distro_arch == "amd64": >> + kernel_arch = "x86" >> + elif distro_arch == "arm64": >> + kernel_arch = "arm64" >> + elif distro_arch == "armhf": >> + kernel_arch = "arm" >> + elif distro_arch == "mipsel": >> + kernel_arch = "mips" >> + else: >> + kernel_arch = "" >> + return kernel_arch >> + >> +KERNEL_ARCH ??= "${@get_kernel_arch(d)}" >> + >> def config_fragments(d): >> fragments = [] >> sources = d.getVar("SRC_URI").split() >> @@ -31,57 +113,35 @@ def config_fragments(d): >> fragments.append(local) >> return fragments >> >> -inherit dpkg-base >> - >> -SRC_URI += "file://build-kernel.sh" >> +do_prepare_build_prepend() { >> + # copy meta-data over to source tree >> + rm -rf ${S}/debian >> + cp -r ${WORKDIR}/debian ${S}/ >> >> -KBUILD_DEPENDS ?= " \ >> - build-essential:native \ >> - libssl-dev \ >> - libelf-dev \ >> - bc \ >> - git \ >> - kmod \ >> - bison \ >> - flex \ >> - cpio \ >> - libncurses-dev" >> -KERNEL_DEBIAN_DEPENDS ?= "initramfs-tools | linux-initramfs-tool, kmod, linux-base (>= 4.3~)" >> -KERNEL_HEADERS_DEBIAN_DEPENDS ?= "libc6, libssl1.1" >> + # remove templates from the source tree >> + find ${S}/debian -name *.tmpl | xargs rm -f >> >> -KERNEL_LIBC_DEV_DEPLOY ?= "0" >> + # rename install/remove hooks to match user-specified name for our linux-image package >> + mv ${S}/debian/linux-image.postinst ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.postinst >> + mv ${S}/debian/linux-image.postrm ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.postrm >> >> -do_install_builddeps() { >> - dpkg_do_mounts >> - E="${@ bb.utils.export_proxies(d)}" >> - sudo -E chroot ${BUILDCHROOT_DIR} \ >> - apt-get update \ >> - -o Dir::Etc::SourceList="sources.list.d/isar-apt.list" \ >> - -o Dir::Etc::SourceParts="-" \ >> - -o APT::Get::List-Cleanup="0" >> - sudo -E chroot ${BUILDCHROOT_DIR} \ >> - apt-get install \ >> - -y -o Debug::pkgProblemResolver=yes \ >> - --no-install-recommends ${KBUILD_DEPENDS} >> - dpkg_undo_mounts >> + # produce a changelog for our kernel build >> + deb_add_changelog >> } >> >> -addtask install_builddeps after do_prepare_build before do_dpkg_build >> -# apt and reprepro may not run in parallel, acquire the Isar lock >> -do_install_builddeps[lockfiles] += "${REPO_ISAR_DIR}/isar.lock" >> - >> -addtask devshell after do_install_builddeps >> +# build directory for our "full" kernel build >> +KERNEL_BUILD_DIR = "build-full" >> >> -dpkg_runbuild() { >> - chmod +x ${WORKDIR}/build-kernel.sh >> - KERNEL_CONFIG_TARGET="${KERNEL_DEFCONFIG}" >> +dpkg_configure_kernel() { >> + config_target="${KERNEL_DEFCONFIG}" >> + rm -rf ${S}/${KERNEL_BUILD_DIR} && mkdir -p ${S}/${KERNEL_BUILD_DIR} >> if [ -n "${KERNEL_DEFCONFIG}" ]; then >> if [ -e "${WORKDIR}/${KERNEL_DEFCONFIG}" ]; then >> - cp ${WORKDIR}/${KERNEL_DEFCONFIG} ${S}/.config >> - KERNEL_CONFIG_TARGET="olddefconfig" >> + cp ${WORKDIR}/${KERNEL_DEFCONFIG} ${S}/${KERNEL_BUILD_DIR}/.config >> + config_target="olddefconfig" >> fi >> else >> - KERNEL_CONFIG_TARGET="defconfig" >> + config_target="defconfig" >> fi >> >> # copy config fragments over to the kernel tree >> @@ -92,21 +152,23 @@ dpkg_runbuild() { >> mkdir -p ${S}/debian/fragments >> (cd ${WORKDIR} && cp ${src_frags} ${S}/debian/fragments/) >> fi >> - export KERNEL_FRAGMENTS="${out_frags}" >> >> - E="${@ bb.utils.export_proxies(d)}" >> - >> - export PV=${PV} >> - export KERNEL_NAME=${KERNEL_NAME_PROVIDED} >> - >> - export KBUILD_DEPENDS="${KBUILD_DEPENDS}" >> - export KERNEL_CONFIG_TARGET >> - export KERNEL_DEBIAN_DEPENDS="${KERNEL_DEBIAN_DEPENDS}" >> - export KERNEL_FILE="${KERNEL_FILE}" >> - export KERNEL_HEADERS_DEBIAN_DEPENDS="${KERNEL_HEADERS_DEBIAN_DEPENDS}" >> + sudo -E chroot --userspec=$(id -u):$(id -g) ${BUILDCHROOT_DIR} sh -c " \ >> + export ARCH=${KERNEL_ARCH} && \ >> + cd ${PP}/${PPS} && \ >> + make O=${KERNEL_BUILD_DIR} ${config_target} && \ >> + ./scripts/kconfig/merge_config.sh \ >> + -O ${KERNEL_BUILD_DIR}/ \ >> + ${KERNEL_BUILD_DIR}/.config \ >> + ${out_frags} \ >> + " >> +} >> >> - sudo -E chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} ${PP}/build-kernel.sh ${PP}/${PPS} ${DISTRO_ARCH} >> +dpkg_runbuild_prepend() { >> + dpkg_configure_kernel >> +} >> >> +dpkg_runbuild_append() { >> if [ "${KERNEL_LIBC_DEV_DEPLOY}" != "1" ]; then >> rm -f ${WORKDIR}/linux-libc-dev_${PV}*.deb >> fi >> > Valuable step forward! I still need to check some more details against > my unfinished prototype, but it seems it generally a superset of it. And I appreciate your quick feedback! > > Jan > ^ permalink raw reply [flat|nested] 83+ messages in thread
* Re: [PATCH 7/7] linux-custom: rewrite to no longer depend on the kernel's builddeb 2019-11-05 15:26 ` Jan Kiszka 2019-11-05 16:20 ` Cedric Hombourger @ 2019-11-05 16:22 ` Jan Kiszka 2019-11-05 16:25 ` Jan Kiszka 2019-11-05 20:16 ` chombourger 2 siblings, 1 reply; 83+ messages in thread From: Jan Kiszka @ 2019-11-05 16:22 UTC (permalink / raw) To: Cedric Hombourger, isar-users On 05.11.19 16:26, [ext] Jan Kiszka wrote: > On 05.11.19 14:57, Cedric Hombourger wrote: >> Regain control over the packaging of the Linux kernel by providing our >> own debian recipes (debian,{control,rules} files and friends) instead >> of using the kernel's builddeb script. This will allow generation of >> packages for "perf" (not included in this changeset) with per distro >> scheme (Debian and Ubuntu have differences) and maintain compatibility >> with upstream packages generated from different source packages but >> dependent on a certain deployment scheme (e.g. linux-base for Debian) >> >> Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com> >> --- >> doc/custom_kernel.md | 181 ++++++++++++++++ >> .../linux/files/build-kernel.sh | 128 ------------ >> meta/recipes-kernel/linux/files/debian/compat | 1 + >> .../linux/files/debian/control.tmpl | 36 ++++ >> meta/recipes-kernel/linux/files/debian/files | 5 + >> .../linux/files/debian/isar/build.tmpl | 36 ++++ >> .../linux/files/debian/isar/clean.tmpl | 20 ++ >> .../linux/files/debian/isar/common.tmpl | 60 ++++++ >> .../linux/files/debian/isar/install.tmpl | 197 ++++++++++++++++++ >> .../files/debian/linux-image.postinst.tmpl | 22 ++ >> .../files/debian/linux-image.postrm.tmpl | 18 ++ >> .../linux/files/debian/rules.tmpl | 39 ++++ >> meta/recipes-kernel/linux/linux-custom.inc | 172 ++++++++++----- >> 13 files changed, 732 insertions(+), 183 deletions(-) >> create mode 100644 doc/custom_kernel.md >> delete mode 100644 meta/recipes-kernel/linux/files/build-kernel.sh >> create mode 100644 meta/recipes-kernel/linux/files/debian/compat >> create mode 100644 meta/recipes-kernel/linux/files/debian/control.tmpl >> create mode 100644 meta/recipes-kernel/linux/files/debian/files >> create mode 100644 meta/recipes-kernel/linux/files/debian/isar/build.tmpl >> create mode 100644 meta/recipes-kernel/linux/files/debian/isar/clean.tmpl >> create mode 100644 meta/recipes-kernel/linux/files/debian/isar/common.tmpl >> create mode 100644 meta/recipes-kernel/linux/files/debian/isar/install.tmpl >> create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl >> create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl >> create mode 100755 meta/recipes-kernel/linux/files/debian/rules.tmpl >> >> diff --git a/doc/custom_kernel.md b/doc/custom_kernel.md >> new file mode 100644 >> index 0000000..5ab2599 >> --- /dev/null >> +++ b/doc/custom_kernel.md >> @@ -0,0 +1,181 @@ >> +# Custom kernel recipe for Isar >> + >> +## Contents >> + >> + - [Summary](#summary) >> + - [Proposal owners](#proposal-owners) >> + - [Current status](#current-status) >> + - [Detailed description](#detailed-description) >> + - [Benefit to Isar](#benefit-to-isar) >> + - [How can I help?](#how-can-i-help) >> + - [Scope](#scope) >> + - [How to test](#how-to-test) >> + - [Dependencies](#dependencies) >> + - [Documentation](#documentation) >> + - [Questions and answers](#questions-and-answers) >> + >> +## Summary >> + >> +Isar currently uses the Linux kernel's builddeb script to generate linux-image, >> +linux-headers and linux-libc-dev packages for your custom kernel sources. The >> +main benefit of this approach was that the low level details of how to package >> +the Linux kernel image, modules and headers were left in the kernel. There are >> +however some drawbacks: Isar had to repack generated packages to introduce some >> +of the changes it needs such as suffixing the kernel packages with the name of >> +the custom kernel (e.g. with "mainline" for the "linux-mainline" recipe), lack >> +of automatic dependencies (the builddeb script from the kernel does not use >> +debhelper), it was difficult to add packages especially distribution specific >> +packages (maintain version specific patch in Isar vs being able to upstream >> +distribution specific changes upstream), etc. This change is about being less >> +dependent on the kernel packaging scripts (which aren't used by major distros >> +anyway) and regain control on how we want or need the Linux kernel packaged. >> + >> +## Proposal owners >> + >> + * name: [Cedric Hombourger](https://github.com/chombourger) >> + >> +## Current status >> + >> +### Tests >> + >> +The following were recently checked: >> + >> + * Custom kernels in meta-isar build without (known) failures >> + * initrd image gets generated when the kernel is added to the rootfs >> + * `ISAR_CROSS_COMPILE` is supported >> + * `KERNEL_FILE` is honored >> + * `KERNEL_DEFCONFIG` may either be a file (specified via `SRC_URI`) or a >> + kernel-provided defconfig >> + * Config fragments get merged with the user-specified kernel configuration >> + * Path to installed DTBs may be configured (new) >> + * Configuration and build from a `devshell` >> + * Support for the `KERNEL_LIBC_DEV_DEPLOY` directive >> + >> +while the following needs work: >> + >> + * Check if custom-kernel builds are reproducible >> + >> +### Remaining work items >> + >> + * Check if there is a better way to cross compile kernel "scripts" such as >> + `fixdep` or `modpost` than manually compiling them (they are compiled for >> + the build machine (amd64) and not for the target (e.g. armhf) even though >> + we are cross-compiling - yet the linux-headers package should be shipping >> + "scripts" for the target machine) >> + >> +## Detailed description >> + >> +### Requirements >> + >> +#### Current >> + >> +Isar has had a `linux-custom` recipe for a long time and counts many users. It is therefore >> +paramount to maintain features that it has today with no or little changes to the APIs. The >> +following requirements were identified and will be maintained: >> + >> + 1. Sources to the custom Linux kernel may be specified via `SRC_URI` >> + >> + 2. Configure the kernel via an in-tree or an external `defconfig` > > Reference to KERNEL_DEFCONFIG is missing. > >> + >> + 3. Support kernel configuration tweaks via configuration fragments (`.cfg` files) >> + >> + 4. Patches to the linux kernel may be specified via `SRC_URI` >> + >> + 5. Ensure that the Isar recipe `PV` matches the kernel release identifier (`KERNEL_RELEASE`) >> + >> + 6. Produce a `linux-image` package that ships the kernel image and modules >> + >> + 7. Allow the name of the kernel image to be changed via `KERNEL_FILE` (defaults to `vmlinuz`) >> + >> + 8. Produce a `linux-headers` package which includes kernel headers and kbuild scripts/tools >> + >> + 9. The `linux-headers` package shall support native and cross compiles of out-of-tree kernel modules > > One package to serve them all? Is that the use case for patch 6? IOW, we > are now building linux-headers for the target, not the host OS anymore? > What's the slowdown compare to cross-linux-headers? > > The current policy is that, when cross-building, our -dev packagages > (which is what linux-headers falls under) will work natively with the > buildchroot. The future path is clearly generating fitting packages for > both host and target. Or is there anything in the linux-headers that > prevents providing both? > >> + >> + 10. Produce a `linux-libc-dev` package to support user-land builds > > This should be opt-in only: No need to build it when we don't ship it. > Just add the related fragement to debian/control when > KERNEL_LIBC_DEV_DEPLOY is set. > >> + >> + 11. Only ship the `linux-libc-dev` package if instructed to (`KERNEL_LIBC_DEV_DEPLOY` equals to `"1"`) >> + >> + 12. Support both native and cross compiles (`ISAR_CROSS_COMPILE`) >> + >> + 13. Support for the following kernel architectures: >> + >> + * arm >> + * arm64 >> + * mips >> + * x86 >> + >> + 14. Support `devshell` (kernel configuration shall be applied) >> + >> +#### New requirements (proposed) >> + >> +This proposal includes the following new requirements: >> + >> + 1. Support creation of additional packages >> + >> + 2. Allow per-distro packaging tweaks >> + >> +### Approach >> + >> +The new implementation will have the `builddeb` code ported from the Linux kernel included to the >> +Isar environment. It will be refactor to support pre and post hooks. Hooks may be used by external >> +layers to extend the kernel packaging instructions provided by "stock" Isar or by Isar itself to >> +implement distro-specific packages or tweaks. >> + >> +Hooks need the ability to add `build` and `install` steps which can be achieved by having the main >> +`build` and `install` steps call pre and post scripts. This will however solve only one side of the >> +problem: new packages should be declared in the `debian/control` file. The packaging process >> +(initiated by `debian/rules`) will use a baseline `control` file when started and a mechanism will >> +be provided for hooks to append `control` blocks as they are processed. > > Is this a section addressing users or reviewers? The latter audience is > better addressed via the commit log. > >> + >> +### Future work >> + >> +In the event where this proposal is accepted, here are a few future projects that are being >> +considered: >> + >> + 1. Package `perf` as Debian does >> + >> + 2. Add `ubuntu` as an Isar supported target distro and validate `linux-custom` recipe >> + >> +## Benefit to Isar >> + >> + * Control the kernel build and packaging process >> + >> + * No longer need to repack packages >> + >> + * Provide hooks for custom kernel recipes to add build/install directives > > Careful with hooks! They tend to pile up mess. Where exactly do you want > them? What for? > >> + >> +## How can I help? >> + >> + * Check if you are able to build your own linux-custom recipes with this change >> + * Check if packages generated by this recipe are compatible with your Debian-based distro >> + * Review and provide comments on this changeset >> + * Create templates for distro-specific packages (e.g. perf) >> + >> +## Scope >> + >> + * Proposal owners: proposal owners are to communicate the idea to isar-users and make sure >> + that all of their requirements are met (to the extent they are already supported with the >> + current solution) or may be met in the future. >> + >> + * Other developers: port your out-of-tree linux-custom recipes to this new solution and >> + report issues. >> + >> +## How to test? >> + >> +The linux-mainline recipe may be used for some basic testing. This recipe is being used by the >> +following machines: >> + >> + * de0-nano-soc >> + * qemumipsel >> + >> +## Dependencies >> + >> + * None >> + >> +## Documentation >> + >> + * Document user-visible variables added by this proposal >> + >> +## Questions and answers >> + >> + * None >> diff --git a/meta/recipes-kernel/linux/files/build-kernel.sh b/meta/recipes-kernel/linux/files/build-kernel.sh >> deleted file mode 100644 >> index f56e96c..0000000 >> --- a/meta/recipes-kernel/linux/files/build-kernel.sh >> +++ /dev/null >> @@ -1,128 +0,0 @@ >> -#!/bin/bash >> -# >> -# Custom kernel build >> -# >> -# This software is a part of ISAR. >> -# Copyright (c) Siemens AG, 2018 >> -# >> -# SPDX-License-Identifier: MIT >> - >> -source /isar/common.sh >> - >> -host_arch=$(dpkg --print-architecture) >> - >> -if [ "$host_arch" != "$target_arch" ]; then >> - case $target_arch in >> - armhf) >> - export ARCH=arm >> - export CROSS_COMPILE="arm-linux-gnueabihf-" >> - ;; >> - arm64) >> - export ARCH=arm64 >> - export CROSS_COMPILE="aarch64-linux-gnu-" >> - ;; >> - mipsel) >> - export ARCH=mips >> - export CROSS_COMPILE="mipsel-linux-gnu-" >> - ;; >> - *) >> - echo "error: unsupported architecture ($target_arch)" >> - exit 1 >> - ;; >> - esac >> -fi >> - >> -REPACK_DIR="$1/../repack" >> -REPACK_LINUX_IMAGE_DIR="${REPACK_DIR}/linux-image" >> -REPACK_LINUX_HEADERS_DIR="${REPACK_DIR}/linux-headers" >> - >> -make ${KERNEL_CONFIG_TARGET} || exit ${?} >> -if [ -n "${KERNEL_FRAGMENTS}" ]; then >> - scripts/kconfig/merge_config.sh -m .config ${KERNEL_FRAGMENTS} >> -fi >> - >> -KV=$( make -s kernelrelease ) >> -if [ "${KV}" != "${PV}" ]; then >> - echo "ERROR: Recipe PV is \"${PV}\" but should be \"${KV}\"" 1>&2 >> - echo "ERROR: Probably due to CONFIG_LOCALVERSION" 1>&2 >> - exit 1 >> -fi >> - >> -rm -f .version >> -KBUILD_DEBARCH=$target_arch make -j $(($(nproc) * 2)) deb-pkg >> - >> -rm -rf "${REPACK_DIR}" >> -mkdir -p "${REPACK_DIR}" >> -mkdir -p "${REPACK_LINUX_IMAGE_DIR}" >> -mkdir -p "${REPACK_LINUX_HEADERS_DIR}" >> - >> -cp -a debian "${REPACK_DIR}" >> - >> -# dpkg-gencontrol performs cross-incompatible checks on the >> -# Architecture field; trick it to accept the control file >> -sed -i "s/Architecture: .*/Architecture: any/" "${REPACK_DIR}/debian/control" >> - >> -cd .. >> - >> -dpkg-deb -R linux-image-${PV}_${PV}-1_*.deb "${REPACK_LINUX_IMAGE_DIR}" >> -dpkg-deb -R linux-headers-${PV}_${PV}-1_*.deb "${REPACK_LINUX_HEADERS_DIR}" >> - >> -dpkg-gencontrol -crepack/debian/control \ >> - -lrepack/debian/changelog \ >> - -frepack/debian/files \ >> - -plinux-image-${PV} \ >> - -P"${REPACK_LINUX_IMAGE_DIR}" \ >> - -DPackage="linux-image-${KERNEL_NAME}" \ >> - -DSection=kernel \ >> - -DPriority=required \ >> - -DDepends="${KERNEL_DEBIAN_DEPENDS}" \ >> - -DArchitecture=$target_arch >> - >> -# Add Debian-like link installation to postinst >> -mkdir -p ${REPACK_LINUX_IMAGE_DIR}/lib/modules/${PV} >> -touch "${REPACK_LINUX_IMAGE_DIR}/lib/modules/${PV}/.fresh-install" >> -sed -i "${REPACK_LINUX_IMAGE_DIR}/DEBIAN/postinst" \ >> - -e "/^set -e$/a\\ >> -\\ >> -if [ -f /lib/modules/${PV}/.fresh-install ]; then\\ >> - change=install\\ >> -else\\ >> - change=upgrade\\ >> -fi\\ >> -linux-update-symlinks \$change ${PV} /boot/${KERNEL_FILE}-${PV}\\ >> -rm -f /lib/modules/${PV}/.fresh-install" >> - >> -# Add Debian-like link removal to postrm >> -sed -i "${REPACK_LINUX_IMAGE_DIR}/DEBIAN/postrm" \ >> - -e "/^set -e$/a\\ >> -\\ >> -rm -f /lib/modules/${PV}/.fresh-install\\ >> -\\ >> -if [ \"\$1\" != upgrade ] && command -v linux-update-symlinks >/dev/null; then\\ >> - linux-update-symlinks remove ${PV} /boot/${KERNEL_FILE}-${PV}\\ >> -fi" >> - >> -# Make sure arm64 kernels are decompressed >> -if [ "$target_arch" = "arm64" ]; then >> - kernel_file="${REPACK_LINUX_IMAGE_DIR}/boot/${KERNEL_FILE}-${PV}" >> - mv "${kernel_file}" "${kernel_file}.gz" >> - gunzip "${kernel_file}.gz" >> -fi >> - >> -dpkg-gencontrol -crepack/debian/control \ >> - -lrepack/debian/changelog \ >> - -frepack/debian/files \ >> - -plinux-headers-${PV} \ >> - -P"${REPACK_LINUX_HEADERS_DIR}" \ >> - -Vkernel:debarch="${KERNEL_NAME}" \ >> - -DPackage="linux-headers-${KERNEL_NAME}" \ >> - -DSection=kernel \ >> - -DDepends="${KERNEL_HEADERS_DEBIAN_DEPENDS}" \ >> - -DArchitecture=$target_arch >> - >> -fakeroot dpkg-deb -b "${REPACK_LINUX_IMAGE_DIR}" \ >> - linux-image-${KERNEL_NAME}_${PV}-1_${KERNEL_NAME}.deb >> -rm -f linux-image-${PV}_${PV}-1_*.deb >> -fakeroot dpkg-deb -b "${REPACK_LINUX_HEADERS_DIR}" \ >> - linux-headers-${KERNEL_NAME}_${PV}-1_${KERNEL_NAME}.deb >> -rm -f linux-headers-${PV}_${PV}-1_*.deb >> diff --git a/meta/recipes-kernel/linux/files/debian/compat b/meta/recipes-kernel/linux/files/debian/compat >> new file mode 100644 >> index 0000000..ec63514 >> --- /dev/null >> +++ b/meta/recipes-kernel/linux/files/debian/compat >> @@ -0,0 +1 @@ >> +9 >> diff --git a/meta/recipes-kernel/linux/files/debian/control.tmpl b/meta/recipes-kernel/linux/files/debian/control.tmpl >> new file mode 100644 >> index 0000000..1295466 >> --- /dev/null >> +++ b/meta/recipes-kernel/linux/files/debian/control.tmpl >> @@ -0,0 +1,36 @@ >> +Source: linux-${KERNEL_NAME_PROVIDED} >> +Section: kernel >> +Priority: optional >> +Maintainer: ${MAINTAINER} >> +Build-Depends: bc, kmod, cpio, ${KBUILD_DEPENDS} >> +Homepage: http://www.kernel.org/ > > Standard-Version? > >> + >> +Package: linux-image-${KERNEL_NAME_PROVIDED} >> +Architecture: any >> +Depends: ${KERNEL_DEBIAN_DEPENDS} >> +Description: ${KERNEL_NAME_PROVIDED} Linux kernel, version ${PV} >> + This package contains the Linux kernel, modules and corresponding other >> + files, version: ${PV}. >> + >> +Package: linux-headers-${KERNEL_NAME_PROVIDED} >> +Architecture: any >> +Depends: ${KERNEL_HEADERS_DEBIAN_DEPENDS}, ${perl:Depends}, ${shlib:Depends} >> +Description: ${KERNEL_NAME_PROVIDED} Linux kernel headers for ${PV} >> + This package provides kernel header files for ${PV} on ${DISTRO_ARCH} >> + . >> + This is useful for people who need to build external modules >> + >> +Package: linux-libc-dev > > See above, should not be here by default. > >> +Section: devel >> +Provides: linux-kernel-headers >> +Architecture: any >> +Description: Linux support headers for userspace development >> + This package provides userspaces headers from the Linux kernel. These headers >> + are used by the installed headers for GNU glibc and other system libraries. >> + >> +Package: linux-image-${KERNEL_NAME_PROVIDED}-dbg >> +Section: debug >> +Architecture: any >> +Description: Linux kernel debugging symbols for ${PV} >> + This package will come in handy if you need to debug the kernel. It provides >> + all the necessary debug symbols for the kernel and its modules. >> diff --git a/meta/recipes-kernel/linux/files/debian/files b/meta/recipes-kernel/linux/files/debian/files >> new file mode 100644 >> index 0000000..a8ace32 >> --- /dev/null >> +++ b/meta/recipes-kernel/linux/files/debian/files >> @@ -0,0 +1,5 @@ >> +linux-headers_4.19.80-1_mipsel.deb kernel optional >> +linux-image-dbg_4.19.80-1_mipsel.deb debug optional >> +linux-image_4.19.80-1_mipsel.deb kernel optional >> +linux-libc-dev_4.19.80-1_mipsel.deb devel optional >> +linux_4.19.80-1_mipsel.buildinfo kernel optional > > Looks like an accidentally added file. > >> diff --git a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl >> new file mode 100644 >> index 0000000..ce79d56 >> --- /dev/null >> +++ b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl >> @@ -0,0 +1,36 @@ >> +#!/bin/sh >> +# Copyright (c) Mentor Graphics, a Siemens business, 2019 >> +# SPDX-License-Identifier: MIT >> + >> +# Load common stuff >> +. ${S}/debian/isar/common || exit ${?} >> + >> +do_build() { >> + >> + # Print a few things that are of particular interest >> + print_settings >> + >> + # Trace what we do here >> + set -x >> + >> + # Process existing kernel configuration to make sure it is complete >> + # (use defaults for options that were not specified) >> + ${MAKE} O=${KERNEL_BUILD_DIR} olddefconfig || exit ${?} >> + >> + # Build the Linux kernel >> + ${MAKE} O=${KERNEL_BUILD_DIR} -j $(nproc) || exit ${?} >> + >> + # Stop tracing >> + set +x >> +} >> + >> +print_settings() { >> + cat <<EOF >> +# Build settings: >> +# --------------- >> +# ARCH=${ARCH} >> +# CROSS_COMPILE=${CROSS_COMPILE} >> +EOF >> +} >> + >> +main build ${*} >> diff --git a/meta/recipes-kernel/linux/files/debian/isar/clean.tmpl b/meta/recipes-kernel/linux/files/debian/isar/clean.tmpl >> new file mode 100644 >> index 0000000..2aa3742 >> --- /dev/null >> +++ b/meta/recipes-kernel/linux/files/debian/isar/clean.tmpl >> @@ -0,0 +1,20 @@ >> +#!/bin/sh >> +# Copyright (c) Mentor Graphics, a Siemens business, 2019 >> +# SPDX-License-Identifier: MIT >> + >> +# Load common stuff >> +. ${S}/debian/isar/common || exit ${?} >> + >> +do_clean() { >> + >> + # Trace what we do here >> + set -x >> + >> + rm -rf ${deb_img_dir} ${deb_dbg_dir} >> + rm -rf ${deb_kern_hdrdir} ${deb_libc_hdr_dir} > > I suppose we rather (or also?) want "make clean" here for the build > artifacts. Just make sure to save/restore the debian folder across that > step. > >> + >> + # Stop tracing >> + set +x >> +} >> + >> +main clean ${*} >> diff --git a/meta/recipes-kernel/linux/files/debian/isar/common.tmpl b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl >> new file mode 100644 >> index 0000000..b392b46 >> --- /dev/null >> +++ b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl >> @@ -0,0 +1,60 @@ >> +#!/bin/sh >> +# Copyright (c) Mentor Graphics, a Siemens business, 2019 >> +# SPDX-License-Identifier: MIT >> + >> +# Isar settings >> +ARCH=${KERNEL_ARCH} >> +KERNEL_PKG_IMAGE=linux-image-${KERNEL_NAME_PROVIDED} >> +KERNEL_PKG_KERN_HEADERS=linux-headers-${KERNEL_NAME_PROVIDED} >> +KERNEL_PKG_LIBC_HEADERS=linux-libc-dev >> + >> +# Constants >> +KCONF=.config >> + >> +# Target directories >> +deb_top_dir=${S}/debian >> +deb_img_dir=${deb_top_dir}/${KERNEL_PKG_IMAGE} >> +deb_dbg_dir=${deb_img_dir}-dbg >> +deb_dtb_dir=${deb_img_dir}/${KERNEL_DTB_DIR} >> +deb_kern_hdr_dir=${deb_top_dir}/${KERNEL_PKG_KERN_HEADERS} >> +deb_libc_hdr_dir=${deb_top_dir}/${KERNEL_PKG_LIBC_HEADERS} >> + >> +main() { >> + target=${1} >> + >> + if [ ! -f ${S}/debian/isar/${target} ]; then >> + echo "error: ${target} is not a supported build target!" >&2 >> + return 1 >> + fi >> + >> + # check for scripts to be sourced >> + for f in ${S}/debian/isar/defaults.d/${target}/*; do >> + [ -f ${f} ] || continue >> + . ${f} || return ${?} >> + done >> + >> + # variables to be exported >> + export ARCH >> + >> + # are we cross-compiling? >> + BUILD_ARCH=$(dpkg-architecture -qDEB_BUILD_ARCH) >> + if [ "${BUILD_ARCH}" = "${DISTRO_ARCH}" ]; then >> + # no, make sure CROSS_COMPILE isn't set >> + unset CROSS_COMPILE >> + fi >> + >> + # check for pre-target scripts >> + for f in ${S}/debian/isar/pre.d/${target}/*; do >> + [ -f ${f} ] || continue >> + sh ${f} || return ${?} >> + done >> + >> + # call the actual target script >> + do_${target} || return ${?} >> + >> + # check for post-target scripts >> + for f in ${S}/debian/isar/post.d/${target}/*; do >> + [ -f ${f} ] || continue >> + sh ${f} || return ${?} >> + done >> +} >> diff --git a/meta/recipes-kernel/linux/files/debian/isar/install.tmpl b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl >> new file mode 100644 >> index 0000000..2055669 >> --- /dev/null >> +++ b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl >> @@ -0,0 +1,197 @@ >> +#!/bin/sh >> +# Copyright (c) Mentor Graphics, a Siemens business, 2019 >> +# SPDX-License-Identifier: MIT >> + >> +# Load common stuff >> +. ${S}/debian/isar/common || exit ${?} >> + >> +do_install() { >> + >> + # check if our kernel was configured >> + if [ ! -f "${O}/.config" ]; then >> + echo "error: kernel not configured!" >&2 >> + return 1 >> + fi >> + >> + # load its configuration >> + . ${O}/.config >> + >> + kimage="$(${MAKE} O=${O} -s --no-print-directory image_name)" >> + case "${ARCH}" in >> + mips|powerpc) kimage_path="boot/vmlinux-${PV}" ;; >> + um) kimage_path="usr/bin/vmlinux-${PV}" ;; >> + *) kimage_path="boot/vmlinuz-${PV}" ;; >> + esac >> + >> + print_settings >> + >> + # Stop on error >> + set -e >> + >> + # Trace what we do here >> + set -x >> + >> + install_image >> + install_hooks >> + install_dtbs >> + install_kmods >> + install_headers >> + >> + # Stop tracing >> + set +x >> +} >> + >> +print_settings() { >> + cat <<EOF >> +Install settings: >> +----------------- >> +deb_dtb_dir=${deb_dtb_dir} >> +deb_hdr_dir=${deb_hdr_dir} >> +kimage=${kimage} >> +kimage_path=${kimage_path} >> + >> +EOF >> +} >> + >> +install_image() { >> + install -m 755 -d ${deb_img_dir}/$(dirname ${kimage_path}) >> + cp ${O}/${kimage} ${deb_img_dir}/${kimage_path} >> + install_image_debug >> +} >> + >> +install_image_debug() { >> + # Different tools want the image in different locations >> + # perf >> + mkdir -p ${deb_dbg_dir}/usr/lib/debug/lib/modules/${PV}/ >> + cp vmlinux ${deb_dbg_dir}/usr/lib/debug/lib/modules/${PV}/ >> + # systemtap >> + mkdir -p ${deb_dbg_dir}/usr/lib/debug/boot/ >> + ln -s ../lib/modules/$version/vmlinux ${deb_dbg_dir}/usr/lib/debug/boot/vmlinux-${PV} >> + # kdump-tools >> + ln -s lib/modules/${PV}/vmlinux ${deb_dbg_dir}/usr/lib/debug/vmlinux-${PV} >> +} >> + >> +install_hooks() { >> + install -m 755 -d ${deb_img_dir}/etc/kernel/install.d >> + install -m 755 -d ${deb_img_dir}/etc/kernel/postinst.d >> + install -m 755 -d ${deb_img_dir}/etc/kernel/postrm.d >> + install -m 755 -d ${deb_img_dir}/etc/kernel/prerm.d >> +} >> + >> +install_dtbs() { >> + [ -n "${CONFIG_OF}" ] || return 0 >> + ${MAKE} O=${O} INSTALL_DTBS_PATH=${deb_dtb_dir} dtbs_install >> +} >> + >> +install_kmods() { >> + [ -n "${CONFIG_MODULES}" ] || return 0 >> + ${MAKE} O=${O} INSTALL_MOD_PATH=${deb_img_dir} modules_install >> + touch ${deb_img_dir}/lib/modules/${PV}/.fresh-install >> + rm -fv ${deb_img_dir}/lib/modules/${PV}/build >> + rm -fv ${deb_img_dir}/lib/modules/${PV}/source >> + install_kmods_debug >> +} >> + >> +kmods_sign() { >> + [ -n "${CONFIG_MODULE_SIG_ALL}" ] || return 0 >> + ${MAKE} O=${O} INSTALL_MOD_PATH=${deb_img_dir} modules_sign >> +} >> + >> +install_kmods_debug() { >> + [ -n "${CONFIG_DEBUG_INFO}" ] || return 0 >> + >> + kmod_inst_dir=${deb_img_dir}/lib/modules >> + kmod_debug_dir=${deb_dbg_dir}/usr/lib/debug >> + >> + # copy kernels modules to usr/lib/debug >> + mkdir -p ${kmod_debug_dir} >> + tar -C ${kmod_inst_dir}/ -cO --exclude='modules.*' . | tar -C ${kmod_debug_dir}/ -xf - >> + # strip everything but debug sections for modules in usr/lib/debug >> + find ${kmod_debug_dir} -name *.ko -exec ${CROSS_COMPILE}objcopy --only-keep-debug {} \; >> + # and strip debug sections from modules in lib/modules >> + find ${kmod_inst_dir} -name *.ko -exec ${CROSS_COMPILE}objcopy --strip-debug {} \; >> + >> + # re-sign stripped kernel modules >> + kmods_sign >> +} >> + >> +headers_check() { >> + ${MAKE} O=${O} headers_check >> +} >> + >> +libc_headers() { >> + mkdir -p ${deb_libc_hdr_dir} >> + ${MAKE} O=${O} headers_install INSTALL_HDR_PATH=${deb_libc_hdr_dir}/usr >> + host_arch=$(dpkg-architecture -a${DISTRO_ARCH} -qDEB_HOST_MULTIARCH) >> + mkdir ${deb_libc_hdr_dir}/usr/include/${host_arch} >> + mv ${deb_libc_hdr_dir}/usr/include/asm ${deb_libc_hdr_dir}/usr/include/${host_arch}/ >> +} >> + >> +# the kernel does not seem to provide a way to cross-compile its tools, we however >> +# have qemu-user-static binaries in our buildchroot. Create a new build tree for the >> +# scripts and override HOSTCC to force a cross-compile >> +kernel_tools_build() { >> + odir="${S}/build-scripts" >> + mflags="HOSTCC=${CROSS_COMPILE}gcc O=${odir} NOSTDINC_FLAGS=" >> + rm -rf ${odir} && mkdir -p ${odir} && cp ${S}/${KERNEL_BUILD_DIR}/.config ${odir} >> + (unset LD_PRELOAD; cd ${odir} && make -C ${S} ${mflags} olddefconfig scripts) >> + (cd ${odir}; find scripts -type f -executable -exec file {} \;|grep ELF|cut -d: -f1) >${src_hdr_files} >> + tar -C ${odir} -cf - -T - <${src_hdr_files} | tar -C ${destdir} -xf - >> +} >> + >> +kernel_tools() { >> + # remove object files >> + find ${destdir}/scripts -type f -name '*.o' |xargs rm -f >> + >> + # we're all done if we aren't cross-compiling >> + [ -n "${CROSS_COMPILE}" ] || return 0 >> + >> + # remove ELF executables from the linux-headers package >> + find ${destdir}/scripts -type f -exec file {} \;|grep ELF|cut -d: -f1|xargs rm -fv >> + >> + # cross-compile kernel tools to be shipped with linux-headers >> + kernel_tools_build >> +} >> + >> +kernel_headers() { >> + destdir=${deb_kern_hdr_dir}/${KERNEL_HEADERS_DIR} >> + src_hdr_files=$(mktemp) >> + obj_hdr_files=$(mktemp) >> + >> + mkdir -p ${destdir} >> + mkdir -p ${deb_kern_hdr_dir}/lib/modules/${PV} >> + >> + (cd ${S}; find . -name 'Makefile*' -o -name 'Kconfig*' -o -name '*.pl') >>${src_hdr_files} >> + (cd ${S}; find arch/*/include include scripts -type f -o -type l) >>${src_hdr_files} >> + (cd ${S}; find arch/${ARCH} -name module.lds -o -name Kbuild.platforms -o -name Platform) >>${src_hdr_files} >> + (cd ${S}; find $(find arch/${ARCH} -name include -o -name scripts -type d) -type f) >>${src_hdr_files} >> + >> + (cd ${O}; find arch/${ARCH}/include Module.symvers include scripts -type f) >>${obj_hdr_files} >> + if [ -n "${CONFIG_STACK_VALIDATION}" ]; then >> + (cd ${O}; find tools/objtool -type f -executable) >>${obj_hdr_files} >> + fi >> + if [ -n "${CONFIG_GCC_PLUGINS}" ]; then >> + (cd ${O}; find scripts/gcc-plugins -name *.so -o -name gcc-common.h) >>${obj_hdr_files} >> + fi >> + >> + # deploy files that were matched above >> + tar -C ${S} -cf - -T - <${src_hdr_files} | tar -C ${destdir} -xf - >> + tar -C ${O} -cf - -T - <${obj_hdr_files} | tar -C ${destdir} -xf - >> + >> + # add the kernel config >> + cp ${O}/${KCONF} ${destdir}/.config >> + >> + # handle kernel development tools >> + kernel_tools >> + >> + # create symlinks >> + ln -sf /${KERNEL_HEADERS_DIR} ${deb_kern_hdr_dir}/lib/modules/${PV}/build >> +} >> + >> +install_headers() { >> + headers_check >> + libc_headers >> + kernel_headers >> +} >> + >> +main install ${*} >> diff --git a/meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl b/meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl >> new file mode 100644 >> index 0000000..fd89121 >> --- /dev/null >> +++ b/meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl >> @@ -0,0 +1,22 @@ >> +#!/bin/sh >> + >> +set -e >> +set -x >> + >> +if [ -f /lib/modules/${PV}/.fresh-install ]; then >> + change=install >> +else >> + change=upgrade >> +fi >> + >> +linux-update-symlinks $change ${PV} /boot/${KERNEL_FILE}-${PV} >> +rm -f /lib/modules/${PV}/.fresh-install >> + >> +# Pass maintainer script parameters to hook scripts >> +export DEB_MAINT_PARAMS="$*" >> + >> +# Tell initramfs builder whether it's wanted >> +export INITRD=Yes > > I've noticed while hacking my version that this should follow > CONFIG_BLK_DEV_INITRD. > >> + >> +test -d /etc/kernel/postinst.d && run-parts --arg="${PV}" --arg="/boot/${KERNEL_FILE}-${PV}" /etc/kernel/postinst.d >> +exit 0 >> diff --git a/meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl b/meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl >> new file mode 100644 >> index 0000000..9d88218 >> --- /dev/null >> +++ b/meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl >> @@ -0,0 +1,18 @@ >> +#!/bin/sh >> + >> +set -e >> + >> +rm -f /lib/modules/${PV}/.fresh-install >> + >> +if [ "$1" != upgrade ] && command -v linux-update-symlinks >/dev/null; then >> + linux-update-symlinks remove ${PV} /boot/${KERNEL_FILE}-${PV} >> +fi >> + >> +# Pass maintainer script parameters to hook scripts >> +export DEB_MAINT_PARAMS="$*" >> + >> +# Tell initramfs builder whether it's wanted >> +export INITRD=Yes >> + >> +test -d /etc/kernel/postrm.d && run-parts --arg="${PV}" --arg="/boot/${KERNEL_FILE}-${PV}" /etc/kernel/postrm.d >> +exit 0 >> diff --git a/meta/recipes-kernel/linux/files/debian/rules.tmpl b/meta/recipes-kernel/linux/files/debian/rules.tmpl > > There is quite some duplication between postinst and postrm already. > Moreover, I think you are missing preinst and prerm. I think all four > should probably be generated from one template. > >> new file mode 100755 >> index 0000000..93c8a5b >> --- /dev/null >> +++ b/meta/recipes-kernel/linux/files/debian/rules.tmpl >> @@ -0,0 +1,39 @@ >> +#!/usr/bin/make -f >> + >> +export CROSS_COMPILE=$(DEB_HOST_GNU_TYPE)- >> + >> +O:=$(CURDIR)/${KERNEL_BUILD_DIR} >> +S:=$(CURDIR) >> +deb_top_dir:=$(S)/debian >> + >> +DH_VERBOSE=1 >> + >> +# Dynamic variables to be passed to Isar build scripts >> +isar_env=$(strip \ >> + export CROSS_COMPILE='$(CROSS_COMPILE)' && \ >> + export MAKE='$(MAKE)' && \ >> + export O='${O}' \ >> + export S='${S}' \ >> +) >> + >> +%: >> + dh $(@) >> + >> +.PHONY: override_dh_auto_clean >> +override_dh_auto_clean: >> + $(isar_env) && sh $(deb_top_dir)/isar/clean >> + >> +.PHONY: override_dh_auto_build >> +override_dh_auto_build: >> + $(isar_env) && sh $(deb_top_dir)/isar/build >> + >> +override_dh_auto_install: >> + $(isar_env) && sh $(deb_top_dir)/isar/install >> + >> +.PHONY: override_dh_auto_test >> +override_dh_auto_test: >> + true >> + >> +PHONY: override_dh_strip >> +override_dh_strip: >> + dh_strip -Xvmlinux --no-automatic-dbgsym >> diff --git a/meta/recipes-kernel/linux/linux-custom.inc b/meta/recipes-kernel/linux/linux-custom.inc >> index ca91f64..36086be 100644 >> --- a/meta/recipes-kernel/linux/linux-custom.inc >> +++ b/meta/recipes-kernel/linux/linux-custom.inc >> @@ -2,22 +2,104 @@ >> # >> # This software is a part of ISAR. >> # Copyright (c) Siemens AG, 2018 >> +# Copyright (c) Mentor Graphics, a Siemens business, 2019 >> # >> # SPDX-License-Identifier: MIT >> >> -FILESEXTRAPATHS_prepend := "${FILE_DIRNAME}/files:" >> +# Settings that would typically be done from the custom kernel recipe >> +# ------------------------------------------------------------------- >> >> DESCRIPTION ?= "Custom kernel" >> +MAINTAINER ?= "isar-users <isar-users@googlegroups.com>" >> + >> +KBUILD_DEPENDS ?= "build-essential:native, \ >> + libelf-dev:native, \ >> + libncurses-dev:native, \ >> + libssl-dev:native, \ >> + bc, \ >> + bison, \ >> + cpio, \ >> + flex, \ >> + git, \ >> + kmod, \ >> + libssl-dev," >> + >> +KERNEL_DEBIAN_DEPENDS ?= "initramfs-tools | linux-initramfs-tool, \ >> + kmod, \ >> + linux-base (>= 4.3~)," >> + >> +KERNEL_HEADERS_DEBIAN_DEPENDS ?= "libc6, \ >> + libssl1.1," >> + >> +KERNEL_LIBC_DEV_DEPLOY ?= "0" >> + >> +# Settings that may be changed on a per distro, machine or layer basis >> +# -------------------------------------------------------------------- >> + >> +KERNEL_DTB_DIR ?= "usr/lib/linux-${KERNEL_NAME_PROVIDED}-${PV}" >> +KERNEL_HEADERS_DIR ?= "usr/src/linux-headers-${PV}" >> >> -KERNEL_NAME_PROVIDED ?= "${@ d.getVar('PN', True).partition('linux-')[2]}" >> KERNEL_DEFCONFIG ?= "" >> >> +# Add our template meta-data to the sources >> +FILESEXTRAPATHS_prepend := "${FILE_DIRNAME}/files:" >> +SRC_URI += "file://debian" >> + >> +# Variables and files that make our templates >> +# ------------------------------------------- >> + >> +TEMPLATE_FILES += " \ >> + debian/control.tmpl \ >> + debian/isar/build.tmpl \ >> + debian/isar/clean.tmpl \ >> + debian/isar/common.tmpl \ >> + debian/isar/install.tmpl \ >> + debian/linux-image.postinst.tmpl \ >> + debian/linux-image.postrm.tmpl \ >> + debian/rules.tmpl \ >> +" >> + >> +TEMPLATE_VARS += " \ >> + KBUILD_DEPENDS \ >> + KERNEL_ARCH \ >> + KERNEL_DEBIAN_DEPENDS \ >> + KERNEL_BUILD_DIR \ >> + KERNEL_DTB_DIR \ >> + KERNEL_FILE \ >> + KERNEL_HEADERS_DEBIAN_DEPENDS \ >> + KERNEL_HEADERS_DIR \ >> + KERNEL_NAME_PROVIDED \ >> +" >> + >> +inherit dpkg >> +inherit template >> + >> +# Derive name of the kernel packages from the name of this recipe >> +KERNEL_NAME_PROVIDED ?= "${@ d.getVar('PN', True).partition('linux-')[2]}" >> + >> +# Make bitbake know we will be producing linux-image and linux-headers packages >> python() { >> kernel_name = d.getVar("KERNEL_NAME_PROVIDED", True) >> d.setVar('PROVIDES', 'linux-image-' + kernel_name + ' ' + \ >> 'linux-headers-' + kernel_name) >> } >> >> +def get_kernel_arch(d): >> + distro_arch = d.getVar("DISTRO_ARCH") >> + if distro_arch == "amd64": >> + kernel_arch = "x86" >> + elif distro_arch == "arm64": >> + kernel_arch = "arm64" >> + elif distro_arch == "armhf": >> + kernel_arch = "arm" >> + elif distro_arch == "mipsel": >> + kernel_arch = "mips" >> + else: >> + kernel_arch = "" >> + return kernel_arch >> + >> +KERNEL_ARCH ??= "${@get_kernel_arch(d)}" >> + >> def config_fragments(d): >> fragments = [] >> sources = d.getVar("SRC_URI").split() >> @@ -31,57 +113,35 @@ def config_fragments(d): >> fragments.append(local) >> return fragments >> >> -inherit dpkg-base >> - >> -SRC_URI += "file://build-kernel.sh" >> +do_prepare_build_prepend() { >> + # copy meta-data over to source tree >> + rm -rf ${S}/debian >> + cp -r ${WORKDIR}/debian ${S}/ >> >> -KBUILD_DEPENDS ?= " \ >> - build-essential:native \ >> - libssl-dev \ >> - libelf-dev \ >> - bc \ >> - git \ >> - kmod \ >> - bison \ >> - flex \ >> - cpio \ >> - libncurses-dev" >> -KERNEL_DEBIAN_DEPENDS ?= "initramfs-tools | linux-initramfs-tool, kmod, linux-base (>= 4.3~)" >> -KERNEL_HEADERS_DEBIAN_DEPENDS ?= "libc6, libssl1.1" >> + # remove templates from the source tree >> + find ${S}/debian -name *.tmpl | xargs rm -f >> >> -KERNEL_LIBC_DEV_DEPLOY ?= "0" >> + # rename install/remove hooks to match user-specified name for our linux-image package >> + mv ${S}/debian/linux-image.postinst ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.postinst >> + mv ${S}/debian/linux-image.postrm ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.postrm >> >> -do_install_builddeps() { >> - dpkg_do_mounts >> - E="${@ bb.utils.export_proxies(d)}" >> - sudo -E chroot ${BUILDCHROOT_DIR} \ >> - apt-get update \ >> - -o Dir::Etc::SourceList="sources.list.d/isar-apt.list" \ >> - -o Dir::Etc::SourceParts="-" \ >> - -o APT::Get::List-Cleanup="0" >> - sudo -E chroot ${BUILDCHROOT_DIR} \ >> - apt-get install \ >> - -y -o Debug::pkgProblemResolver=yes \ >> - --no-install-recommends ${KBUILD_DEPENDS} >> - dpkg_undo_mounts >> + # produce a changelog for our kernel build >> + deb_add_changelog >> } >> >> -addtask install_builddeps after do_prepare_build before do_dpkg_build >> -# apt and reprepro may not run in parallel, acquire the Isar lock >> -do_install_builddeps[lockfiles] += "${REPO_ISAR_DIR}/isar.lock" >> - >> -addtask devshell after do_install_builddeps >> +# build directory for our "full" kernel build >> +KERNEL_BUILD_DIR = "build-full" >> >> -dpkg_runbuild() { >> - chmod +x ${WORKDIR}/build-kernel.sh >> - KERNEL_CONFIG_TARGET="${KERNEL_DEFCONFIG}" >> +dpkg_configure_kernel() { >> + config_target="${KERNEL_DEFCONFIG}" >> + rm -rf ${S}/${KERNEL_BUILD_DIR} && mkdir -p ${S}/${KERNEL_BUILD_DIR} >> if [ -n "${KERNEL_DEFCONFIG}" ]; then >> if [ -e "${WORKDIR}/${KERNEL_DEFCONFIG}" ]; then >> - cp ${WORKDIR}/${KERNEL_DEFCONFIG} ${S}/.config >> - KERNEL_CONFIG_TARGET="olddefconfig" >> + cp ${WORKDIR}/${KERNEL_DEFCONFIG} ${S}/${KERNEL_BUILD_DIR}/.config >> + config_target="olddefconfig" >> fi >> else >> - KERNEL_CONFIG_TARGET="defconfig" >> + config_target="defconfig" >> fi >> >> # copy config fragments over to the kernel tree >> @@ -92,21 +152,23 @@ dpkg_runbuild() { >> mkdir -p ${S}/debian/fragments >> (cd ${WORKDIR} && cp ${src_frags} ${S}/debian/fragments/) >> fi >> - export KERNEL_FRAGMENTS="${out_frags}" >> >> - E="${@ bb.utils.export_proxies(d)}" >> - >> - export PV=${PV} >> - export KERNEL_NAME=${KERNEL_NAME_PROVIDED} >> - >> - export KBUILD_DEPENDS="${KBUILD_DEPENDS}" >> - export KERNEL_CONFIG_TARGET >> - export KERNEL_DEBIAN_DEPENDS="${KERNEL_DEBIAN_DEPENDS}" >> - export KERNEL_FILE="${KERNEL_FILE}" >> - export KERNEL_HEADERS_DEBIAN_DEPENDS="${KERNEL_HEADERS_DEBIAN_DEPENDS}" >> + sudo -E chroot --userspec=$(id -u):$(id -g) ${BUILDCHROOT_DIR} sh -c " \ >> + export ARCH=${KERNEL_ARCH} && \ >> + cd ${PP}/${PPS} && \ >> + make O=${KERNEL_BUILD_DIR} ${config_target} && \ >> + ./scripts/kconfig/merge_config.sh \ >> + -O ${KERNEL_BUILD_DIR}/ \ >> + ${KERNEL_BUILD_DIR}/.config \ >> + ${out_frags} \ >> + " >> +} >> >> - sudo -E chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} ${PP}/build-kernel.sh ${PP}/${PPS} ${DISTRO_ARCH} >> +dpkg_runbuild_prepend() { >> + dpkg_configure_kernel >> +} >> >> +dpkg_runbuild_append() { >> if [ "${KERNEL_LIBC_DEV_DEPLOY}" != "1" ]; then >> rm -f ${WORKDIR}/linux-libc-dev_${PV}*.deb >> fi >> > > Valuable step forward! I still need to check some more details against > my unfinished prototype, but it seems it generally a superset of it. Found a first difference: if $(call is_enabled,CONFIG_OF_EARLY_FLATTREE) && \ [ -d arch/${ARCH}/boot/dts ]; then \ $(MAKE) INSTALL_DTBS_PATH=debian/tmp/usr/lib/linux-image-${PV} dtbs_install; \ fi Jan -- Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 83+ messages in thread
* Re: [PATCH 7/7] linux-custom: rewrite to no longer depend on the kernel's builddeb 2019-11-05 16:22 ` Jan Kiszka @ 2019-11-05 16:25 ` Jan Kiszka 0 siblings, 0 replies; 83+ messages in thread From: Jan Kiszka @ 2019-11-05 16:25 UTC (permalink / raw) To: Cedric Hombourger, isar-users On 05.11.19 17:22, Jan Kiszka wrote: > On 05.11.19 16:26, [ext] Jan Kiszka wrote: >> On 05.11.19 14:57, Cedric Hombourger wrote: >>> Regain control over the packaging of the Linux kernel by providing our >>> own debian recipes (debian,{control,rules} files and friends) instead >>> of using the kernel's builddeb script. This will allow generation of >>> packages for "perf" (not included in this changeset) with per distro >>> scheme (Debian and Ubuntu have differences) and maintain compatibility >>> with upstream packages generated from different source packages but >>> dependent on a certain deployment scheme (e.g. linux-base for Debian) >>> >>> Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com> >>> --- >>> doc/custom_kernel.md | 181 ++++++++++++++++ >>> .../linux/files/build-kernel.sh | 128 ------------ >>> meta/recipes-kernel/linux/files/debian/compat | 1 + >>> .../linux/files/debian/control.tmpl | 36 ++++ >>> meta/recipes-kernel/linux/files/debian/files | 5 + >>> .../linux/files/debian/isar/build.tmpl | 36 ++++ >>> .../linux/files/debian/isar/clean.tmpl | 20 ++ >>> .../linux/files/debian/isar/common.tmpl | 60 ++++++ >>> .../linux/files/debian/isar/install.tmpl | 197 ++++++++++++++++++ >>> .../files/debian/linux-image.postinst.tmpl | 22 ++ >>> .../files/debian/linux-image.postrm.tmpl | 18 ++ >>> .../linux/files/debian/rules.tmpl | 39 ++++ >>> meta/recipes-kernel/linux/linux-custom.inc | 172 ++++++++++----- >>> 13 files changed, 732 insertions(+), 183 deletions(-) >>> create mode 100644 doc/custom_kernel.md >>> delete mode 100644 meta/recipes-kernel/linux/files/build-kernel.sh >>> create mode 100644 meta/recipes-kernel/linux/files/debian/compat >>> create mode 100644 meta/recipes-kernel/linux/files/debian/control.tmpl >>> create mode 100644 meta/recipes-kernel/linux/files/debian/files >>> create mode 100644 meta/recipes-kernel/linux/files/debian/isar/build.tmpl >>> create mode 100644 meta/recipes-kernel/linux/files/debian/isar/clean.tmpl >>> create mode 100644 meta/recipes-kernel/linux/files/debian/isar/common.tmpl >>> create mode 100644 meta/recipes-kernel/linux/files/debian/isar/install.tmpl >>> create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl >>> create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl >>> create mode 100755 meta/recipes-kernel/linux/files/debian/rules.tmpl >>> >>> diff --git a/doc/custom_kernel.md b/doc/custom_kernel.md >>> new file mode 100644 >>> index 0000000..5ab2599 >>> --- /dev/null >>> +++ b/doc/custom_kernel.md >>> @@ -0,0 +1,181 @@ >>> +# Custom kernel recipe for Isar >>> + >>> +## Contents >>> + >>> + - [Summary](#summary) >>> + - [Proposal owners](#proposal-owners) >>> + - [Current status](#current-status) >>> + - [Detailed description](#detailed-description) >>> + - [Benefit to Isar](#benefit-to-isar) >>> + - [How can I help?](#how-can-i-help) >>> + - [Scope](#scope) >>> + - [How to test](#how-to-test) >>> + - [Dependencies](#dependencies) >>> + - [Documentation](#documentation) >>> + - [Questions and answers](#questions-and-answers) >>> + >>> +## Summary >>> + >>> +Isar currently uses the Linux kernel's builddeb script to generate linux-image, >>> +linux-headers and linux-libc-dev packages for your custom kernel sources. The >>> +main benefit of this approach was that the low level details of how to package >>> +the Linux kernel image, modules and headers were left in the kernel. There are >>> +however some drawbacks: Isar had to repack generated packages to introduce some >>> +of the changes it needs such as suffixing the kernel packages with the name of >>> +the custom kernel (e.g. with "mainline" for the "linux-mainline" recipe), lack >>> +of automatic dependencies (the builddeb script from the kernel does not use >>> +debhelper), it was difficult to add packages especially distribution specific >>> +packages (maintain version specific patch in Isar vs being able to upstream >>> +distribution specific changes upstream), etc. This change is about being less >>> +dependent on the kernel packaging scripts (which aren't used by major distros >>> +anyway) and regain control on how we want or need the Linux kernel packaged. >>> + >>> +## Proposal owners >>> + >>> + * name: [Cedric Hombourger](https://github.com/chombourger) >>> + >>> +## Current status >>> + >>> +### Tests >>> + >>> +The following were recently checked: >>> + >>> + * Custom kernels in meta-isar build without (known) failures >>> + * initrd image gets generated when the kernel is added to the rootfs >>> + * `ISAR_CROSS_COMPILE` is supported >>> + * `KERNEL_FILE` is honored >>> + * `KERNEL_DEFCONFIG` may either be a file (specified via `SRC_URI`) or a >>> + kernel-provided defconfig >>> + * Config fragments get merged with the user-specified kernel configuration >>> + * Path to installed DTBs may be configured (new) >>> + * Configuration and build from a `devshell` >>> + * Support for the `KERNEL_LIBC_DEV_DEPLOY` directive >>> + >>> +while the following needs work: >>> + >>> + * Check if custom-kernel builds are reproducible >>> + >>> +### Remaining work items >>> + >>> + * Check if there is a better way to cross compile kernel "scripts" such as >>> + `fixdep` or `modpost` than manually compiling them (they are compiled for >>> + the build machine (amd64) and not for the target (e.g. armhf) even though >>> + we are cross-compiling - yet the linux-headers package should be shipping >>> + "scripts" for the target machine) >>> + >>> +## Detailed description >>> + >>> +### Requirements >>> + >>> +#### Current >>> + >>> +Isar has had a `linux-custom` recipe for a long time and counts many users. It is therefore >>> +paramount to maintain features that it has today with no or little changes to the APIs. The >>> +following requirements were identified and will be maintained: >>> + >>> + 1. Sources to the custom Linux kernel may be specified via `SRC_URI` >>> + >>> + 2. Configure the kernel via an in-tree or an external `defconfig` >> >> Reference to KERNEL_DEFCONFIG is missing. >> >>> + >>> + 3. Support kernel configuration tweaks via configuration fragments (`.cfg` files) >>> + >>> + 4. Patches to the linux kernel may be specified via `SRC_URI` >>> + >>> + 5. Ensure that the Isar recipe `PV` matches the kernel release identifier (`KERNEL_RELEASE`) >>> + >>> + 6. Produce a `linux-image` package that ships the kernel image and modules >>> + >>> + 7. Allow the name of the kernel image to be changed via `KERNEL_FILE` (defaults to `vmlinuz`) >>> + >>> + 8. Produce a `linux-headers` package which includes kernel headers and kbuild scripts/tools >>> + >>> + 9. The `linux-headers` package shall support native and cross compiles of out-of-tree kernel modules >> >> One package to serve them all? Is that the use case for patch 6? IOW, we >> are now building linux-headers for the target, not the host OS anymore? >> What's the slowdown compare to cross-linux-headers? >> >> The current policy is that, when cross-building, our -dev packagages >> (which is what linux-headers falls under) will work natively with the >> buildchroot. The future path is clearly generating fitting packages for >> both host and target. Or is there anything in the linux-headers that >> prevents providing both? >> >>> + >>> + 10. Produce a `linux-libc-dev` package to support user-land builds >> >> This should be opt-in only: No need to build it when we don't ship it. >> Just add the related fragement to debian/control when >> KERNEL_LIBC_DEV_DEPLOY is set. >> >>> + >>> + 11. Only ship the `linux-libc-dev` package if instructed to (`KERNEL_LIBC_DEV_DEPLOY` equals to `"1"`) >>> + >>> + 12. Support both native and cross compiles (`ISAR_CROSS_COMPILE`) >>> + >>> + 13. Support for the following kernel architectures: >>> + >>> + * arm >>> + * arm64 >>> + * mips >>> + * x86 >>> + >>> + 14. Support `devshell` (kernel configuration shall be applied) >>> + >>> +#### New requirements (proposed) >>> + >>> +This proposal includes the following new requirements: >>> + >>> + 1. Support creation of additional packages >>> + >>> + 2. Allow per-distro packaging tweaks >>> + >>> +### Approach >>> + >>> +The new implementation will have the `builddeb` code ported from the Linux kernel included to the >>> +Isar environment. It will be refactor to support pre and post hooks. Hooks may be used by external >>> +layers to extend the kernel packaging instructions provided by "stock" Isar or by Isar itself to >>> +implement distro-specific packages or tweaks. >>> + >>> +Hooks need the ability to add `build` and `install` steps which can be achieved by having the main >>> +`build` and `install` steps call pre and post scripts. This will however solve only one side of the >>> +problem: new packages should be declared in the `debian/control` file. The packaging process >>> +(initiated by `debian/rules`) will use a baseline `control` file when started and a mechanism will >>> +be provided for hooks to append `control` blocks as they are processed. >> >> Is this a section addressing users or reviewers? The latter audience is >> better addressed via the commit log. >> >>> + >>> +### Future work >>> + >>> +In the event where this proposal is accepted, here are a few future projects that are being >>> +considered: >>> + >>> + 1. Package `perf` as Debian does >>> + >>> + 2. Add `ubuntu` as an Isar supported target distro and validate `linux-custom` recipe >>> + >>> +## Benefit to Isar >>> + >>> + * Control the kernel build and packaging process >>> + >>> + * No longer need to repack packages >>> + >>> + * Provide hooks for custom kernel recipes to add build/install directives >> >> Careful with hooks! They tend to pile up mess. Where exactly do you want >> them? What for? >> >>> + >>> +## How can I help? >>> + >>> + * Check if you are able to build your own linux-custom recipes with this change >>> + * Check if packages generated by this recipe are compatible with your Debian-based distro >>> + * Review and provide comments on this changeset >>> + * Create templates for distro-specific packages (e.g. perf) >>> + >>> +## Scope >>> + >>> + * Proposal owners: proposal owners are to communicate the idea to isar-users and make sure >>> + that all of their requirements are met (to the extent they are already supported with the >>> + current solution) or may be met in the future. >>> + >>> + * Other developers: port your out-of-tree linux-custom recipes to this new solution and >>> + report issues. >>> + >>> +## How to test? >>> + >>> +The linux-mainline recipe may be used for some basic testing. This recipe is being used by the >>> +following machines: >>> + >>> + * de0-nano-soc >>> + * qemumipsel >>> + >>> +## Dependencies >>> + >>> + * None >>> + >>> +## Documentation >>> + >>> + * Document user-visible variables added by this proposal >>> + >>> +## Questions and answers >>> + >>> + * None >>> diff --git a/meta/recipes-kernel/linux/files/build-kernel.sh b/meta/recipes-kernel/linux/files/build-kernel.sh >>> deleted file mode 100644 >>> index f56e96c..0000000 >>> --- a/meta/recipes-kernel/linux/files/build-kernel.sh >>> +++ /dev/null >>> @@ -1,128 +0,0 @@ >>> -#!/bin/bash >>> -# >>> -# Custom kernel build >>> -# >>> -# This software is a part of ISAR. >>> -# Copyright (c) Siemens AG, 2018 >>> -# >>> -# SPDX-License-Identifier: MIT >>> - >>> -source /isar/common.sh >>> - >>> -host_arch=$(dpkg --print-architecture) >>> - >>> -if [ "$host_arch" != "$target_arch" ]; then >>> - case $target_arch in >>> - armhf) >>> - export ARCH=arm >>> - export CROSS_COMPILE="arm-linux-gnueabihf-" >>> - ;; >>> - arm64) >>> - export ARCH=arm64 >>> - export CROSS_COMPILE="aarch64-linux-gnu-" >>> - ;; >>> - mipsel) >>> - export ARCH=mips >>> - export CROSS_COMPILE="mipsel-linux-gnu-" >>> - ;; >>> - *) >>> - echo "error: unsupported architecture ($target_arch)" >>> - exit 1 >>> - ;; >>> - esac >>> -fi >>> - >>> -REPACK_DIR="$1/../repack" >>> -REPACK_LINUX_IMAGE_DIR="${REPACK_DIR}/linux-image" >>> -REPACK_LINUX_HEADERS_DIR="${REPACK_DIR}/linux-headers" >>> - >>> -make ${KERNEL_CONFIG_TARGET} || exit ${?} >>> -if [ -n "${KERNEL_FRAGMENTS}" ]; then >>> - scripts/kconfig/merge_config.sh -m .config ${KERNEL_FRAGMENTS} >>> -fi >>> - >>> -KV=$( make -s kernelrelease ) >>> -if [ "${KV}" != "${PV}" ]; then >>> - echo "ERROR: Recipe PV is \"${PV}\" but should be \"${KV}\"" 1>&2 >>> - echo "ERROR: Probably due to CONFIG_LOCALVERSION" 1>&2 >>> - exit 1 >>> -fi >>> - >>> -rm -f .version >>> -KBUILD_DEBARCH=$target_arch make -j $(($(nproc) * 2)) deb-pkg >>> - >>> -rm -rf "${REPACK_DIR}" >>> -mkdir -p "${REPACK_DIR}" >>> -mkdir -p "${REPACK_LINUX_IMAGE_DIR}" >>> -mkdir -p "${REPACK_LINUX_HEADERS_DIR}" >>> - >>> -cp -a debian "${REPACK_DIR}" >>> - >>> -# dpkg-gencontrol performs cross-incompatible checks on the >>> -# Architecture field; trick it to accept the control file >>> -sed -i "s/Architecture: .*/Architecture: any/" "${REPACK_DIR}/debian/control" >>> - >>> -cd .. >>> - >>> -dpkg-deb -R linux-image-${PV}_${PV}-1_*.deb "${REPACK_LINUX_IMAGE_DIR}" >>> -dpkg-deb -R linux-headers-${PV}_${PV}-1_*.deb "${REPACK_LINUX_HEADERS_DIR}" >>> - >>> -dpkg-gencontrol -crepack/debian/control \ >>> - -lrepack/debian/changelog \ >>> - -frepack/debian/files \ >>> - -plinux-image-${PV} \ >>> - -P"${REPACK_LINUX_IMAGE_DIR}" \ >>> - -DPackage="linux-image-${KERNEL_NAME}" \ >>> - -DSection=kernel \ >>> - -DPriority=required \ >>> - -DDepends="${KERNEL_DEBIAN_DEPENDS}" \ >>> - -DArchitecture=$target_arch >>> - >>> -# Add Debian-like link installation to postinst >>> -mkdir -p ${REPACK_LINUX_IMAGE_DIR}/lib/modules/${PV} >>> -touch "${REPACK_LINUX_IMAGE_DIR}/lib/modules/${PV}/.fresh-install" >>> -sed -i "${REPACK_LINUX_IMAGE_DIR}/DEBIAN/postinst" \ >>> - -e "/^set -e$/a\\ >>> -\\ >>> -if [ -f /lib/modules/${PV}/.fresh-install ]; then\\ >>> - change=install\\ >>> -else\\ >>> - change=upgrade\\ >>> -fi\\ >>> -linux-update-symlinks \$change ${PV} /boot/${KERNEL_FILE}-${PV}\\ >>> -rm -f /lib/modules/${PV}/.fresh-install" >>> - >>> -# Add Debian-like link removal to postrm >>> -sed -i "${REPACK_LINUX_IMAGE_DIR}/DEBIAN/postrm" \ >>> - -e "/^set -e$/a\\ >>> -\\ >>> -rm -f /lib/modules/${PV}/.fresh-install\\ >>> -\\ >>> -if [ \"\$1\" != upgrade ] && command -v linux-update-symlinks >/dev/null; then\\ >>> - linux-update-symlinks remove ${PV} /boot/${KERNEL_FILE}-${PV}\\ >>> -fi" >>> - >>> -# Make sure arm64 kernels are decompressed >>> -if [ "$target_arch" = "arm64" ]; then >>> - kernel_file="${REPACK_LINUX_IMAGE_DIR}/boot/${KERNEL_FILE}-${PV}" >>> - mv "${kernel_file}" "${kernel_file}.gz" >>> - gunzip "${kernel_file}.gz" >>> -fi >>> - >>> -dpkg-gencontrol -crepack/debian/control \ >>> - -lrepack/debian/changelog \ >>> - -frepack/debian/files \ >>> - -plinux-headers-${PV} \ >>> - -P"${REPACK_LINUX_HEADERS_DIR}" \ >>> - -Vkernel:debarch="${KERNEL_NAME}" \ >>> - -DPackage="linux-headers-${KERNEL_NAME}" \ >>> - -DSection=kernel \ >>> - -DDepends="${KERNEL_HEADERS_DEBIAN_DEPENDS}" \ >>> - -DArchitecture=$target_arch >>> - >>> -fakeroot dpkg-deb -b "${REPACK_LINUX_IMAGE_DIR}" \ >>> - linux-image-${KERNEL_NAME}_${PV}-1_${KERNEL_NAME}.deb >>> -rm -f linux-image-${PV}_${PV}-1_*.deb >>> -fakeroot dpkg-deb -b "${REPACK_LINUX_HEADERS_DIR}" \ >>> - linux-headers-${KERNEL_NAME}_${PV}-1_${KERNEL_NAME}.deb >>> -rm -f linux-headers-${PV}_${PV}-1_*.deb >>> diff --git a/meta/recipes-kernel/linux/files/debian/compat b/meta/recipes-kernel/linux/files/debian/compat >>> new file mode 100644 >>> index 0000000..ec63514 >>> --- /dev/null >>> +++ b/meta/recipes-kernel/linux/files/debian/compat >>> @@ -0,0 +1 @@ >>> +9 >>> diff --git a/meta/recipes-kernel/linux/files/debian/control.tmpl b/meta/recipes-kernel/linux/files/debian/control.tmpl >>> new file mode 100644 >>> index 0000000..1295466 >>> --- /dev/null >>> +++ b/meta/recipes-kernel/linux/files/debian/control.tmpl >>> @@ -0,0 +1,36 @@ >>> +Source: linux-${KERNEL_NAME_PROVIDED} >>> +Section: kernel >>> +Priority: optional >>> +Maintainer: ${MAINTAINER} >>> +Build-Depends: bc, kmod, cpio, ${KBUILD_DEPENDS} >>> +Homepage: http://www.kernel.org/ >> >> Standard-Version? >> >>> + >>> +Package: linux-image-${KERNEL_NAME_PROVIDED} >>> +Architecture: any >>> +Depends: ${KERNEL_DEBIAN_DEPENDS} >>> +Description: ${KERNEL_NAME_PROVIDED} Linux kernel, version ${PV} >>> + This package contains the Linux kernel, modules and corresponding other >>> + files, version: ${PV}. >>> + >>> +Package: linux-headers-${KERNEL_NAME_PROVIDED} >>> +Architecture: any >>> +Depends: ${KERNEL_HEADERS_DEBIAN_DEPENDS}, ${perl:Depends}, ${shlib:Depends} >>> +Description: ${KERNEL_NAME_PROVIDED} Linux kernel headers for ${PV} >>> + This package provides kernel header files for ${PV} on ${DISTRO_ARCH} >>> + . >>> + This is useful for people who need to build external modules >>> + >>> +Package: linux-libc-dev >> >> See above, should not be here by default. >> >>> +Section: devel >>> +Provides: linux-kernel-headers >>> +Architecture: any >>> +Description: Linux support headers for userspace development >>> + This package provides userspaces headers from the Linux kernel. These headers >>> + are used by the installed headers for GNU glibc and other system libraries. >>> + >>> +Package: linux-image-${KERNEL_NAME_PROVIDED}-dbg >>> +Section: debug >>> +Architecture: any >>> +Description: Linux kernel debugging symbols for ${PV} >>> + This package will come in handy if you need to debug the kernel. It provides >>> + all the necessary debug symbols for the kernel and its modules. >>> diff --git a/meta/recipes-kernel/linux/files/debian/files b/meta/recipes-kernel/linux/files/debian/files >>> new file mode 100644 >>> index 0000000..a8ace32 >>> --- /dev/null >>> +++ b/meta/recipes-kernel/linux/files/debian/files >>> @@ -0,0 +1,5 @@ >>> +linux-headers_4.19.80-1_mipsel.deb kernel optional >>> +linux-image-dbg_4.19.80-1_mipsel.deb debug optional >>> +linux-image_4.19.80-1_mipsel.deb kernel optional >>> +linux-libc-dev_4.19.80-1_mipsel.deb devel optional >>> +linux_4.19.80-1_mipsel.buildinfo kernel optional >> >> Looks like an accidentally added file. >> >>> diff --git a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl >>> new file mode 100644 >>> index 0000000..ce79d56 >>> --- /dev/null >>> +++ b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl >>> @@ -0,0 +1,36 @@ >>> +#!/bin/sh >>> +# Copyright (c) Mentor Graphics, a Siemens business, 2019 >>> +# SPDX-License-Identifier: MIT >>> + >>> +# Load common stuff >>> +. ${S}/debian/isar/common || exit ${?} >>> + >>> +do_build() { >>> + >>> + # Print a few things that are of particular interest >>> + print_settings >>> + >>> + # Trace what we do here >>> + set -x >>> + >>> + # Process existing kernel configuration to make sure it is complete >>> + # (use defaults for options that were not specified) >>> + ${MAKE} O=${KERNEL_BUILD_DIR} olddefconfig || exit ${?} >>> + >>> + # Build the Linux kernel >>> + ${MAKE} O=${KERNEL_BUILD_DIR} -j $(nproc) || exit ${?} >>> + >>> + # Stop tracing >>> + set +x >>> +} >>> + >>> +print_settings() { >>> + cat <<EOF >>> +# Build settings: >>> +# --------------- >>> +# ARCH=${ARCH} >>> +# CROSS_COMPILE=${CROSS_COMPILE} >>> +EOF >>> +} >>> + >>> +main build ${*} >>> diff --git a/meta/recipes-kernel/linux/files/debian/isar/clean.tmpl b/meta/recipes-kernel/linux/files/debian/isar/clean.tmpl >>> new file mode 100644 >>> index 0000000..2aa3742 >>> --- /dev/null >>> +++ b/meta/recipes-kernel/linux/files/debian/isar/clean.tmpl >>> @@ -0,0 +1,20 @@ >>> +#!/bin/sh >>> +# Copyright (c) Mentor Graphics, a Siemens business, 2019 >>> +# SPDX-License-Identifier: MIT >>> + >>> +# Load common stuff >>> +. ${S}/debian/isar/common || exit ${?} >>> + >>> +do_clean() { >>> + >>> + # Trace what we do here >>> + set -x >>> + >>> + rm -rf ${deb_img_dir} ${deb_dbg_dir} >>> + rm -rf ${deb_kern_hdrdir} ${deb_libc_hdr_dir} >> >> I suppose we rather (or also?) want "make clean" here for the build >> artifacts. Just make sure to save/restore the debian folder across that >> step. >> >>> + >>> + # Stop tracing >>> + set +x >>> +} >>> + >>> +main clean ${*} >>> diff --git a/meta/recipes-kernel/linux/files/debian/isar/common.tmpl b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl >>> new file mode 100644 >>> index 0000000..b392b46 >>> --- /dev/null >>> +++ b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl >>> @@ -0,0 +1,60 @@ >>> +#!/bin/sh >>> +# Copyright (c) Mentor Graphics, a Siemens business, 2019 >>> +# SPDX-License-Identifier: MIT >>> + >>> +# Isar settings >>> +ARCH=${KERNEL_ARCH} >>> +KERNEL_PKG_IMAGE=linux-image-${KERNEL_NAME_PROVIDED} >>> +KERNEL_PKG_KERN_HEADERS=linux-headers-${KERNEL_NAME_PROVIDED} >>> +KERNEL_PKG_LIBC_HEADERS=linux-libc-dev >>> + >>> +# Constants >>> +KCONF=.config >>> + >>> +# Target directories >>> +deb_top_dir=${S}/debian >>> +deb_img_dir=${deb_top_dir}/${KERNEL_PKG_IMAGE} >>> +deb_dbg_dir=${deb_img_dir}-dbg >>> +deb_dtb_dir=${deb_img_dir}/${KERNEL_DTB_DIR} >>> +deb_kern_hdr_dir=${deb_top_dir}/${KERNEL_PKG_KERN_HEADERS} >>> +deb_libc_hdr_dir=${deb_top_dir}/${KERNEL_PKG_LIBC_HEADERS} >>> + >>> +main() { >>> + target=${1} >>> + >>> + if [ ! -f ${S}/debian/isar/${target} ]; then >>> + echo "error: ${target} is not a supported build target!" >&2 >>> + return 1 >>> + fi >>> + >>> + # check for scripts to be sourced >>> + for f in ${S}/debian/isar/defaults.d/${target}/*; do >>> + [ -f ${f} ] || continue >>> + . ${f} || return ${?} >>> + done >>> + >>> + # variables to be exported >>> + export ARCH >>> + >>> + # are we cross-compiling? >>> + BUILD_ARCH=$(dpkg-architecture -qDEB_BUILD_ARCH) >>> + if [ "${BUILD_ARCH}" = "${DISTRO_ARCH}" ]; then >>> + # no, make sure CROSS_COMPILE isn't set >>> + unset CROSS_COMPILE >>> + fi >>> + >>> + # check for pre-target scripts >>> + for f in ${S}/debian/isar/pre.d/${target}/*; do >>> + [ -f ${f} ] || continue >>> + sh ${f} || return ${?} >>> + done >>> + >>> + # call the actual target script >>> + do_${target} || return ${?} >>> + >>> + # check for post-target scripts >>> + for f in ${S}/debian/isar/post.d/${target}/*; do >>> + [ -f ${f} ] || continue >>> + sh ${f} || return ${?} >>> + done >>> +} >>> diff --git a/meta/recipes-kernel/linux/files/debian/isar/install.tmpl b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl >>> new file mode 100644 >>> index 0000000..2055669 >>> --- /dev/null >>> +++ b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl >>> @@ -0,0 +1,197 @@ >>> +#!/bin/sh >>> +# Copyright (c) Mentor Graphics, a Siemens business, 2019 >>> +# SPDX-License-Identifier: MIT >>> + >>> +# Load common stuff >>> +. ${S}/debian/isar/common || exit ${?} >>> + >>> +do_install() { >>> + >>> + # check if our kernel was configured >>> + if [ ! -f "${O}/.config" ]; then >>> + echo "error: kernel not configured!" >&2 >>> + return 1 >>> + fi >>> + >>> + # load its configuration >>> + . ${O}/.config >>> + >>> + kimage="$(${MAKE} O=${O} -s --no-print-directory image_name)" >>> + case "${ARCH}" in >>> + mips|powerpc) kimage_path="boot/vmlinux-${PV}" ;; >>> + um) kimage_path="usr/bin/vmlinux-${PV}" ;; >>> + *) kimage_path="boot/vmlinuz-${PV}" ;; >>> + esac >>> + >>> + print_settings >>> + >>> + # Stop on error >>> + set -e >>> + >>> + # Trace what we do here >>> + set -x >>> + >>> + install_image >>> + install_hooks >>> + install_dtbs >>> + install_kmods >>> + install_headers >>> + >>> + # Stop tracing >>> + set +x >>> +} >>> + >>> +print_settings() { >>> + cat <<EOF >>> +Install settings: >>> +----------------- >>> +deb_dtb_dir=${deb_dtb_dir} >>> +deb_hdr_dir=${deb_hdr_dir} >>> +kimage=${kimage} >>> +kimage_path=${kimage_path} >>> + >>> +EOF >>> +} >>> + >>> +install_image() { >>> + install -m 755 -d ${deb_img_dir}/$(dirname ${kimage_path}) >>> + cp ${O}/${kimage} ${deb_img_dir}/${kimage_path} >>> + install_image_debug >>> +} >>> + >>> +install_image_debug() { >>> + # Different tools want the image in different locations >>> + # perf >>> + mkdir -p ${deb_dbg_dir}/usr/lib/debug/lib/modules/${PV}/ >>> + cp vmlinux ${deb_dbg_dir}/usr/lib/debug/lib/modules/${PV}/ >>> + # systemtap >>> + mkdir -p ${deb_dbg_dir}/usr/lib/debug/boot/ >>> + ln -s ../lib/modules/$version/vmlinux ${deb_dbg_dir}/usr/lib/debug/boot/vmlinux-${PV} >>> + # kdump-tools >>> + ln -s lib/modules/${PV}/vmlinux ${deb_dbg_dir}/usr/lib/debug/vmlinux-${PV} >>> +} >>> + >>> +install_hooks() { >>> + install -m 755 -d ${deb_img_dir}/etc/kernel/install.d >>> + install -m 755 -d ${deb_img_dir}/etc/kernel/postinst.d >>> + install -m 755 -d ${deb_img_dir}/etc/kernel/postrm.d >>> + install -m 755 -d ${deb_img_dir}/etc/kernel/prerm.d >>> +} >>> + >>> +install_dtbs() { >>> + [ -n "${CONFIG_OF}" ] || return 0 >>> + ${MAKE} O=${O} INSTALL_DTBS_PATH=${deb_dtb_dir} dtbs_install >>> +} >>> + >>> +install_kmods() { >>> + [ -n "${CONFIG_MODULES}" ] || return 0 >>> + ${MAKE} O=${O} INSTALL_MOD_PATH=${deb_img_dir} modules_install >>> + touch ${deb_img_dir}/lib/modules/${PV}/.fresh-install >>> + rm -fv ${deb_img_dir}/lib/modules/${PV}/build >>> + rm -fv ${deb_img_dir}/lib/modules/${PV}/source >>> + install_kmods_debug >>> +} >>> + >>> +kmods_sign() { >>> + [ -n "${CONFIG_MODULE_SIG_ALL}" ] || return 0 >>> + ${MAKE} O=${O} INSTALL_MOD_PATH=${deb_img_dir} modules_sign >>> +} >>> + >>> +install_kmods_debug() { >>> + [ -n "${CONFIG_DEBUG_INFO}" ] || return 0 >>> + >>> + kmod_inst_dir=${deb_img_dir}/lib/modules >>> + kmod_debug_dir=${deb_dbg_dir}/usr/lib/debug >>> + >>> + # copy kernels modules to usr/lib/debug >>> + mkdir -p ${kmod_debug_dir} >>> + tar -C ${kmod_inst_dir}/ -cO --exclude='modules.*' . | tar -C ${kmod_debug_dir}/ -xf - >>> + # strip everything but debug sections for modules in usr/lib/debug >>> + find ${kmod_debug_dir} -name *.ko -exec ${CROSS_COMPILE}objcopy --only-keep-debug {} \; >>> + # and strip debug sections from modules in lib/modules >>> + find ${kmod_inst_dir} -name *.ko -exec ${CROSS_COMPILE}objcopy --strip-debug {} \; >>> + >>> + # re-sign stripped kernel modules >>> + kmods_sign >>> +} >>> + >>> +headers_check() { >>> + ${MAKE} O=${O} headers_check >>> +} >>> + >>> +libc_headers() { >>> + mkdir -p ${deb_libc_hdr_dir} >>> + ${MAKE} O=${O} headers_install INSTALL_HDR_PATH=${deb_libc_hdr_dir}/usr >>> + host_arch=$(dpkg-architecture -a${DISTRO_ARCH} -qDEB_HOST_MULTIARCH) >>> + mkdir ${deb_libc_hdr_dir}/usr/include/${host_arch} >>> + mv ${deb_libc_hdr_dir}/usr/include/asm ${deb_libc_hdr_dir}/usr/include/${host_arch}/ >>> +} >>> + >>> +# the kernel does not seem to provide a way to cross-compile its tools, we however >>> +# have qemu-user-static binaries in our buildchroot. Create a new build tree for the >>> +# scripts and override HOSTCC to force a cross-compile >>> +kernel_tools_build() { >>> + odir="${S}/build-scripts" >>> + mflags="HOSTCC=${CROSS_COMPILE}gcc O=${odir} NOSTDINC_FLAGS=" >>> + rm -rf ${odir} && mkdir -p ${odir} && cp ${S}/${KERNEL_BUILD_DIR}/.config ${odir} >>> + (unset LD_PRELOAD; cd ${odir} && make -C ${S} ${mflags} olddefconfig scripts) >>> + (cd ${odir}; find scripts -type f -executable -exec file {} \;|grep ELF|cut -d: -f1) >${src_hdr_files} >>> + tar -C ${odir} -cf - -T - <${src_hdr_files} | tar -C ${destdir} -xf - >>> +} >>> + >>> +kernel_tools() { >>> + # remove object files >>> + find ${destdir}/scripts -type f -name '*.o' |xargs rm -f >>> + >>> + # we're all done if we aren't cross-compiling >>> + [ -n "${CROSS_COMPILE}" ] || return 0 >>> + >>> + # remove ELF executables from the linux-headers package >>> + find ${destdir}/scripts -type f -exec file {} \;|grep ELF|cut -d: -f1|xargs rm -fv >>> + >>> + # cross-compile kernel tools to be shipped with linux-headers >>> + kernel_tools_build >>> +} >>> + >>> +kernel_headers() { >>> + destdir=${deb_kern_hdr_dir}/${KERNEL_HEADERS_DIR} >>> + src_hdr_files=$(mktemp) >>> + obj_hdr_files=$(mktemp) >>> + >>> + mkdir -p ${destdir} >>> + mkdir -p ${deb_kern_hdr_dir}/lib/modules/${PV} >>> + >>> + (cd ${S}; find . -name 'Makefile*' -o -name 'Kconfig*' -o -name '*.pl') >>${src_hdr_files} >>> + (cd ${S}; find arch/*/include include scripts -type f -o -type l) >>${src_hdr_files} >>> + (cd ${S}; find arch/${ARCH} -name module.lds -o -name Kbuild.platforms -o -name Platform) >>${src_hdr_files} >>> + (cd ${S}; find $(find arch/${ARCH} -name include -o -name scripts -type d) -type f) >>${src_hdr_files} >>> + >>> + (cd ${O}; find arch/${ARCH}/include Module.symvers include scripts -type f) >>${obj_hdr_files} >>> + if [ -n "${CONFIG_STACK_VALIDATION}" ]; then >>> + (cd ${O}; find tools/objtool -type f -executable) >>${obj_hdr_files} >>> + fi >>> + if [ -n "${CONFIG_GCC_PLUGINS}" ]; then >>> + (cd ${O}; find scripts/gcc-plugins -name *.so -o -name gcc-common.h) >>${obj_hdr_files} >>> + fi >>> + >>> + # deploy files that were matched above >>> + tar -C ${S} -cf - -T - <${src_hdr_files} | tar -C ${destdir} -xf - >>> + tar -C ${O} -cf - -T - <${obj_hdr_files} | tar -C ${destdir} -xf - >>> + >>> + # add the kernel config >>> + cp ${O}/${KCONF} ${destdir}/.config >>> + >>> + # handle kernel development tools >>> + kernel_tools >>> + >>> + # create symlinks >>> + ln -sf /${KERNEL_HEADERS_DIR} ${deb_kern_hdr_dir}/lib/modules/${PV}/build >>> +} >>> + >>> +install_headers() { >>> + headers_check >>> + libc_headers >>> + kernel_headers >>> +} >>> + >>> +main install ${*} >>> diff --git a/meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl b/meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl >>> new file mode 100644 >>> index 0000000..fd89121 >>> --- /dev/null >>> +++ b/meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl >>> @@ -0,0 +1,22 @@ >>> +#!/bin/sh >>> + >>> +set -e >>> +set -x >>> + >>> +if [ -f /lib/modules/${PV}/.fresh-install ]; then >>> + change=install >>> +else >>> + change=upgrade >>> +fi >>> + >>> +linux-update-symlinks $change ${PV} /boot/${KERNEL_FILE}-${PV} >>> +rm -f /lib/modules/${PV}/.fresh-install >>> + >>> +# Pass maintainer script parameters to hook scripts >>> +export DEB_MAINT_PARAMS="$*" >>> + >>> +# Tell initramfs builder whether it's wanted >>> +export INITRD=Yes >> >> I've noticed while hacking my version that this should follow >> CONFIG_BLK_DEV_INITRD. >> >>> + >>> +test -d /etc/kernel/postinst.d && run-parts --arg="${PV}" --arg="/boot/${KERNEL_FILE}-${PV}" /etc/kernel/postinst.d >>> +exit 0 >>> diff --git a/meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl b/meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl >>> new file mode 100644 >>> index 0000000..9d88218 >>> --- /dev/null >>> +++ b/meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl >>> @@ -0,0 +1,18 @@ >>> +#!/bin/sh >>> + >>> +set -e >>> + >>> +rm -f /lib/modules/${PV}/.fresh-install >>> + >>> +if [ "$1" != upgrade ] && command -v linux-update-symlinks >/dev/null; then >>> + linux-update-symlinks remove ${PV} /boot/${KERNEL_FILE}-${PV} >>> +fi >>> + >>> +# Pass maintainer script parameters to hook scripts >>> +export DEB_MAINT_PARAMS="$*" >>> + >>> +# Tell initramfs builder whether it's wanted >>> +export INITRD=Yes >>> + >>> +test -d /etc/kernel/postrm.d && run-parts --arg="${PV}" --arg="/boot/${KERNEL_FILE}-${PV}" /etc/kernel/postrm.d >>> +exit 0 >>> diff --git a/meta/recipes-kernel/linux/files/debian/rules.tmpl b/meta/recipes-kernel/linux/files/debian/rules.tmpl >> >> There is quite some duplication between postinst and postrm already. >> Moreover, I think you are missing preinst and prerm. I think all four >> should probably be generated from one template. >> >>> new file mode 100755 >>> index 0000000..93c8a5b >>> --- /dev/null >>> +++ b/meta/recipes-kernel/linux/files/debian/rules.tmpl >>> @@ -0,0 +1,39 @@ >>> +#!/usr/bin/make -f >>> + >>> +export CROSS_COMPILE=$(DEB_HOST_GNU_TYPE)- >>> + >>> +O:=$(CURDIR)/${KERNEL_BUILD_DIR} >>> +S:=$(CURDIR) >>> +deb_top_dir:=$(S)/debian >>> + >>> +DH_VERBOSE=1 >>> + >>> +# Dynamic variables to be passed to Isar build scripts >>> +isar_env=$(strip \ >>> + export CROSS_COMPILE='$(CROSS_COMPILE)' && \ >>> + export MAKE='$(MAKE)' && \ >>> + export O='${O}' \ >>> + export S='${S}' \ >>> +) >>> + >>> +%: >>> + dh $(@) >>> + >>> +.PHONY: override_dh_auto_clean >>> +override_dh_auto_clean: >>> + $(isar_env) && sh $(deb_top_dir)/isar/clean >>> + >>> +.PHONY: override_dh_auto_build >>> +override_dh_auto_build: >>> + $(isar_env) && sh $(deb_top_dir)/isar/build >>> + >>> +override_dh_auto_install: >>> + $(isar_env) && sh $(deb_top_dir)/isar/install >>> + >>> +.PHONY: override_dh_auto_test >>> +override_dh_auto_test: >>> + true >>> + >>> +PHONY: override_dh_strip >>> +override_dh_strip: >>> + dh_strip -Xvmlinux --no-automatic-dbgsym >>> diff --git a/meta/recipes-kernel/linux/linux-custom.inc b/meta/recipes-kernel/linux/linux-custom.inc >>> index ca91f64..36086be 100644 >>> --- a/meta/recipes-kernel/linux/linux-custom.inc >>> +++ b/meta/recipes-kernel/linux/linux-custom.inc >>> @@ -2,22 +2,104 @@ >>> # >>> # This software is a part of ISAR. >>> # Copyright (c) Siemens AG, 2018 >>> +# Copyright (c) Mentor Graphics, a Siemens business, 2019 >>> # >>> # SPDX-License-Identifier: MIT >>> >>> -FILESEXTRAPATHS_prepend := "${FILE_DIRNAME}/files:" >>> +# Settings that would typically be done from the custom kernel recipe >>> +# ------------------------------------------------------------------- >>> >>> DESCRIPTION ?= "Custom kernel" >>> +MAINTAINER ?= "isar-users <isar-users@googlegroups.com>" >>> + >>> +KBUILD_DEPENDS ?= "build-essential:native, \ >>> + libelf-dev:native, \ >>> + libncurses-dev:native, \ >>> + libssl-dev:native, \ >>> + bc, \ >>> + bison, \ >>> + cpio, \ >>> + flex, \ >>> + git, \ >>> + kmod, \ >>> + libssl-dev," >>> + >>> +KERNEL_DEBIAN_DEPENDS ?= "initramfs-tools | linux-initramfs-tool, \ >>> + kmod, \ >>> + linux-base (>= 4.3~)," >>> + >>> +KERNEL_HEADERS_DEBIAN_DEPENDS ?= "libc6, \ >>> + libssl1.1," >>> + >>> +KERNEL_LIBC_DEV_DEPLOY ?= "0" >>> + >>> +# Settings that may be changed on a per distro, machine or layer basis >>> +# -------------------------------------------------------------------- >>> + >>> +KERNEL_DTB_DIR ?= "usr/lib/linux-${KERNEL_NAME_PROVIDED}-${PV}" >>> +KERNEL_HEADERS_DIR ?= "usr/src/linux-headers-${PV}" >>> >>> -KERNEL_NAME_PROVIDED ?= "${@ d.getVar('PN', True).partition('linux-')[2]}" >>> KERNEL_DEFCONFIG ?= "" >>> >>> +# Add our template meta-data to the sources >>> +FILESEXTRAPATHS_prepend := "${FILE_DIRNAME}/files:" >>> +SRC_URI += "file://debian" >>> + >>> +# Variables and files that make our templates >>> +# ------------------------------------------- >>> + >>> +TEMPLATE_FILES += " \ >>> + debian/control.tmpl \ >>> + debian/isar/build.tmpl \ >>> + debian/isar/clean.tmpl \ >>> + debian/isar/common.tmpl \ >>> + debian/isar/install.tmpl \ >>> + debian/linux-image.postinst.tmpl \ >>> + debian/linux-image.postrm.tmpl \ >>> + debian/rules.tmpl \ >>> +" >>> + >>> +TEMPLATE_VARS += " \ >>> + KBUILD_DEPENDS \ >>> + KERNEL_ARCH \ >>> + KERNEL_DEBIAN_DEPENDS \ >>> + KERNEL_BUILD_DIR \ >>> + KERNEL_DTB_DIR \ >>> + KERNEL_FILE \ >>> + KERNEL_HEADERS_DEBIAN_DEPENDS \ >>> + KERNEL_HEADERS_DIR \ >>> + KERNEL_NAME_PROVIDED \ >>> +" >>> + >>> +inherit dpkg >>> +inherit template >>> + >>> +# Derive name of the kernel packages from the name of this recipe >>> +KERNEL_NAME_PROVIDED ?= "${@ d.getVar('PN', True).partition('linux-')[2]}" >>> + >>> +# Make bitbake know we will be producing linux-image and linux-headers packages >>> python() { >>> kernel_name = d.getVar("KERNEL_NAME_PROVIDED", True) >>> d.setVar('PROVIDES', 'linux-image-' + kernel_name + ' ' + \ >>> 'linux-headers-' + kernel_name) >>> } >>> >>> +def get_kernel_arch(d): >>> + distro_arch = d.getVar("DISTRO_ARCH") >>> + if distro_arch == "amd64": >>> + kernel_arch = "x86" >>> + elif distro_arch == "arm64": >>> + kernel_arch = "arm64" >>> + elif distro_arch == "armhf": >>> + kernel_arch = "arm" >>> + elif distro_arch == "mipsel": >>> + kernel_arch = "mips" >>> + else: >>> + kernel_arch = "" >>> + return kernel_arch >>> + >>> +KERNEL_ARCH ??= "${@get_kernel_arch(d)}" >>> + >>> def config_fragments(d): >>> fragments = [] >>> sources = d.getVar("SRC_URI").split() >>> @@ -31,57 +113,35 @@ def config_fragments(d): >>> fragments.append(local) >>> return fragments >>> >>> -inherit dpkg-base >>> - >>> -SRC_URI += "file://build-kernel.sh" >>> +do_prepare_build_prepend() { >>> + # copy meta-data over to source tree >>> + rm -rf ${S}/debian >>> + cp -r ${WORKDIR}/debian ${S}/ >>> >>> -KBUILD_DEPENDS ?= " \ >>> - build-essential:native \ >>> - libssl-dev \ >>> - libelf-dev \ >>> - bc \ >>> - git \ >>> - kmod \ >>> - bison \ >>> - flex \ >>> - cpio \ >>> - libncurses-dev" >>> -KERNEL_DEBIAN_DEPENDS ?= "initramfs-tools | linux-initramfs-tool, kmod, linux-base (>= 4.3~)" >>> -KERNEL_HEADERS_DEBIAN_DEPENDS ?= "libc6, libssl1.1" >>> + # remove templates from the source tree >>> + find ${S}/debian -name *.tmpl | xargs rm -f >>> >>> -KERNEL_LIBC_DEV_DEPLOY ?= "0" >>> + # rename install/remove hooks to match user-specified name for our linux-image package >>> + mv ${S}/debian/linux-image.postinst ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.postinst >>> + mv ${S}/debian/linux-image.postrm ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.postrm >>> >>> -do_install_builddeps() { >>> - dpkg_do_mounts >>> - E="${@ bb.utils.export_proxies(d)}" >>> - sudo -E chroot ${BUILDCHROOT_DIR} \ >>> - apt-get update \ >>> - -o Dir::Etc::SourceList="sources.list.d/isar-apt.list" \ >>> - -o Dir::Etc::SourceParts="-" \ >>> - -o APT::Get::List-Cleanup="0" >>> - sudo -E chroot ${BUILDCHROOT_DIR} \ >>> - apt-get install \ >>> - -y -o Debug::pkgProblemResolver=yes \ >>> - --no-install-recommends ${KBUILD_DEPENDS} >>> - dpkg_undo_mounts >>> + # produce a changelog for our kernel build >>> + deb_add_changelog >>> } >>> >>> -addtask install_builddeps after do_prepare_build before do_dpkg_build >>> -# apt and reprepro may not run in parallel, acquire the Isar lock >>> -do_install_builddeps[lockfiles] += "${REPO_ISAR_DIR}/isar.lock" >>> - >>> -addtask devshell after do_install_builddeps >>> +# build directory for our "full" kernel build >>> +KERNEL_BUILD_DIR = "build-full" >>> >>> -dpkg_runbuild() { >>> - chmod +x ${WORKDIR}/build-kernel.sh >>> - KERNEL_CONFIG_TARGET="${KERNEL_DEFCONFIG}" >>> +dpkg_configure_kernel() { >>> + config_target="${KERNEL_DEFCONFIG}" >>> + rm -rf ${S}/${KERNEL_BUILD_DIR} && mkdir -p ${S}/${KERNEL_BUILD_DIR} >>> if [ -n "${KERNEL_DEFCONFIG}" ]; then >>> if [ -e "${WORKDIR}/${KERNEL_DEFCONFIG}" ]; then >>> - cp ${WORKDIR}/${KERNEL_DEFCONFIG} ${S}/.config >>> - KERNEL_CONFIG_TARGET="olddefconfig" >>> + cp ${WORKDIR}/${KERNEL_DEFCONFIG} ${S}/${KERNEL_BUILD_DIR}/.config >>> + config_target="olddefconfig" >>> fi >>> else >>> - KERNEL_CONFIG_TARGET="defconfig" >>> + config_target="defconfig" >>> fi >>> >>> # copy config fragments over to the kernel tree >>> @@ -92,21 +152,23 @@ dpkg_runbuild() { >>> mkdir -p ${S}/debian/fragments >>> (cd ${WORKDIR} && cp ${src_frags} ${S}/debian/fragments/) >>> fi >>> - export KERNEL_FRAGMENTS="${out_frags}" >>> >>> - E="${@ bb.utils.export_proxies(d)}" >>> - >>> - export PV=${PV} >>> - export KERNEL_NAME=${KERNEL_NAME_PROVIDED} >>> - >>> - export KBUILD_DEPENDS="${KBUILD_DEPENDS}" >>> - export KERNEL_CONFIG_TARGET >>> - export KERNEL_DEBIAN_DEPENDS="${KERNEL_DEBIAN_DEPENDS}" >>> - export KERNEL_FILE="${KERNEL_FILE}" >>> - export KERNEL_HEADERS_DEBIAN_DEPENDS="${KERNEL_HEADERS_DEBIAN_DEPENDS}" >>> + sudo -E chroot --userspec=$(id -u):$(id -g) ${BUILDCHROOT_DIR} sh -c " \ >>> + export ARCH=${KERNEL_ARCH} && \ >>> + cd ${PP}/${PPS} && \ >>> + make O=${KERNEL_BUILD_DIR} ${config_target} && \ >>> + ./scripts/kconfig/merge_config.sh \ >>> + -O ${KERNEL_BUILD_DIR}/ \ >>> + ${KERNEL_BUILD_DIR}/.config \ >>> + ${out_frags} \ >>> + " >>> +} >>> >>> - sudo -E chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} ${PP}/build-kernel.sh ${PP}/${PPS} ${DISTRO_ARCH} >>> +dpkg_runbuild_prepend() { >>> + dpkg_configure_kernel >>> +} >>> >>> +dpkg_runbuild_append() { >>> if [ "${KERNEL_LIBC_DEV_DEPLOY}" != "1" ]; then >>> rm -f ${WORKDIR}/linux-libc-dev_${PV}*.deb >>> fi >>> >> >> Valuable step forward! I still need to check some more details against >> my unfinished prototype, but it seems it generally a superset of it. > > Found a first difference: > > if $(call is_enabled,CONFIG_OF_EARLY_FLATTREE) && \ > [ -d arch/${ARCH}/boot/dts ]; then \ > $(MAKE) INSTALL_DTBS_PATH=debian/tmp/usr/lib/linux-image-${PV} dtbs_install; \ > fi Ah, you have the installation, just the condition is inaccurate. IIRC, CONFIG_OF_EARLY_FLATTREE comes directly from the upstream packaging rules. Jan -- Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 83+ messages in thread
* Re: [PATCH 7/7] linux-custom: rewrite to no longer depend on the kernel's builddeb 2019-11-05 15:26 ` Jan Kiszka 2019-11-05 16:20 ` Cedric Hombourger 2019-11-05 16:22 ` Jan Kiszka @ 2019-11-05 20:16 ` chombourger 2019-11-05 20:24 ` Jan Kiszka 2 siblings, 1 reply; 83+ messages in thread From: chombourger @ 2019-11-05 20:16 UTC (permalink / raw) To: isar-users [-- Attachment #1.1: Type: text/plain, Size: 44926 bytes --] On Tuesday, November 5, 2019 at 4:26:33 PM UTC+1, Jan Kiszka wrote: > > On 05.11.19 14:57, Cedric Hombourger wrote: > > Regain control over the packaging of the Linux kernel by providing our > > own debian recipes (debian,{control,rules} files and friends) instead > > of using the kernel's builddeb script. This will allow generation of > > packages for "perf" (not included in this changeset) with per distro > > scheme (Debian and Ubuntu have differences) and maintain compatibility > > with upstream packages generated from different source packages but > > dependent on a certain deployment scheme (e.g. linux-base for Debian) > > > > Signed-off-by: Cedric Hombourger <Cedric_H...@mentor.com <javascript:>> > > --- > > doc/custom_kernel.md | 181 ++++++++++++++++ > > .../linux/files/build-kernel.sh | 128 ------------ > > meta/recipes-kernel/linux/files/debian/compat | 1 + > > .../linux/files/debian/control.tmpl | 36 ++++ > > meta/recipes-kernel/linux/files/debian/files | 5 + > > .../linux/files/debian/isar/build.tmpl | 36 ++++ > > .../linux/files/debian/isar/clean.tmpl | 20 ++ > > .../linux/files/debian/isar/common.tmpl | 60 ++++++ > > .../linux/files/debian/isar/install.tmpl | 197 ++++++++++++++++++ > > .../files/debian/linux-image.postinst.tmpl | 22 ++ > > .../files/debian/linux-image.postrm.tmpl | 18 ++ > > .../linux/files/debian/rules.tmpl | 39 ++++ > > meta/recipes-kernel/linux/linux-custom.inc | 172 ++++++++++----- > > 13 files changed, 732 insertions(+), 183 deletions(-) > > create mode 100644 doc/custom_kernel.md > > delete mode 100644 meta/recipes-kernel/linux/files/build-kernel.sh > > create mode 100644 meta/recipes-kernel/linux/files/debian/compat > > create mode 100644 meta/recipes-kernel/linux/files/debian/control.tmpl > > create mode 100644 meta/recipes-kernel/linux/files/debian/files > > create mode 100644 > meta/recipes-kernel/linux/files/debian/isar/build.tmpl > > create mode 100644 > meta/recipes-kernel/linux/files/debian/isar/clean.tmpl > > create mode 100644 > meta/recipes-kernel/linux/files/debian/isar/common.tmpl > > create mode 100644 > meta/recipes-kernel/linux/files/debian/isar/install.tmpl > > create mode 100644 > meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl > > create mode 100644 > meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl > > create mode 100755 meta/recipes-kernel/linux/files/debian/rules.tmpl > > > > diff --git a/doc/custom_kernel.md b/doc/custom_kernel.md > > new file mode 100644 > > index 0000000..5ab2599 > > --- /dev/null > > +++ b/doc/custom_kernel.md > > @@ -0,0 +1,181 @@ > > +# Custom kernel recipe for Isar > > + > > +## Contents > > + > > + - [Summary](#summary) > > + - [Proposal owners](#proposal-owners) > > + - [Current status](#current-status) > > + - [Detailed description](#detailed-description) > > + - [Benefit to Isar](#benefit-to-isar) > > + - [How can I help?](#how-can-i-help) > > + - [Scope](#scope) > > + - [How to test](#how-to-test) > > + - [Dependencies](#dependencies) > > + - [Documentation](#documentation) > > + - [Questions and answers](#questions-and-answers) > > + > > +## Summary > > + > > +Isar currently uses the Linux kernel's builddeb script to generate > linux-image, > > +linux-headers and linux-libc-dev packages for your custom kernel > sources. The > > +main benefit of this approach was that the low level details of how to > package > > +the Linux kernel image, modules and headers were left in the kernel. > There are > > +however some drawbacks: Isar had to repack generated packages to > introduce some > > +of the changes it needs such as suffixing the kernel packages with the > name of > > +the custom kernel (e.g. with "mainline" for the "linux-mainline" > recipe), lack > > +of automatic dependencies (the builddeb script from the kernel does not > use > > +debhelper), it was difficult to add packages especially distribution > specific > > +packages (maintain version specific patch in Isar vs being able to > upstream > > +distribution specific changes upstream), etc. This change is about > being less > > +dependent on the kernel packaging scripts (which aren't used by major > distros > > +anyway) and regain control on how we want or need the Linux kernel > packaged. > > + > > +## Proposal owners > > + > > + * name: [Cedric Hombourger](https://github.com/chombourger) > > + > > +## Current status > > + > > +### Tests > > + > > +The following were recently checked: > > + > > + * Custom kernels in meta-isar build without (known) failures > > + * initrd image gets generated when the kernel is added to the rootfs > > + * `ISAR_CROSS_COMPILE` is supported > > + * `KERNEL_FILE` is honored > > + * `KERNEL_DEFCONFIG` may either be a file (specified via `SRC_URI`) or > a > > + kernel-provided defconfig > > + * Config fragments get merged with the user-specified kernel > configuration > > + * Path to installed DTBs may be configured (new) > > + * Configuration and build from a `devshell` > > + * Support for the `KERNEL_LIBC_DEV_DEPLOY` directive > > + > > +while the following needs work: > > + > > + * Check if custom-kernel builds are reproducible > > + > > +### Remaining work items > > + > > + * Check if there is a better way to cross compile kernel "scripts" > such as > > + `fixdep` or `modpost` than manually compiling them (they are > compiled for > > + the build machine (amd64) and not for the target (e.g. armhf) even > though > > + we are cross-compiling - yet the linux-headers package should be > shipping > > + "scripts" for the target machine) > > + > > +## Detailed description > > + > > +### Requirements > > + > > +#### Current > > + > > +Isar has had a `linux-custom` recipe for a long time and counts many > users. It is therefore > > +paramount to maintain features that it has today with no or little > changes to the APIs. The > > +following requirements were identified and will be maintained: > > + > > + 1. Sources to the custom Linux kernel may be specified via `SRC_URI` > > + > > + 2. Configure the kernel via an in-tree or an external `defconfig` > > Reference to KERNEL_DEFCONFIG is missing. > > > + > > + 3. Support kernel configuration tweaks via configuration fragments > (`.cfg` files) > > + > > + 4. Patches to the linux kernel may be specified via `SRC_URI` > > + > > + 5. Ensure that the Isar recipe `PV` matches the kernel release > identifier (`KERNEL_RELEASE`) > > + > > + 6. Produce a `linux-image` package that ships the kernel image and > modules > > + > > + 7. Allow the name of the kernel image to be changed via `KERNEL_FILE` > (defaults to `vmlinuz`) > > + > > + 8. Produce a `linux-headers` package which includes kernel headers and > kbuild scripts/tools > > + > > + 9. The `linux-headers` package shall support native and cross compiles > of out-of-tree kernel modules > > One package to serve them all? Is that the use case for patch 6? IOW, we > are now building linux-headers for the target, not the host OS anymore? > What's the slowdown compare to cross-linux-headers? > > The current policy is that, when cross-building, our -dev packagages > (which is what linux-headers falls under) will work natively with the > buildchroot. The future path is clearly generating fitting packages for > both host and target. Or is there anything in the linux-headers that > prevents providing both? > > > + > > + 10. Produce a `linux-libc-dev` package to support user-land builds > > This should be opt-in only: No need to build it when we don't ship it. > Just add the related fragement to debian/control when > KERNEL_LIBC_DEV_DEPLOY is set. > > > + > > + 11. Only ship the `linux-libc-dev` package if instructed to > (`KERNEL_LIBC_DEV_DEPLOY` equals to `"1"`) > > + > > + 12. Support both native and cross compiles (`ISAR_CROSS_COMPILE`) > > + > > + 13. Support for the following kernel architectures: > > + > > + * arm > > + * arm64 > > + * mips > > + * x86 > > + > > + 14. Support `devshell` (kernel configuration shall be applied) > > + > > +#### New requirements (proposed) > > + > > +This proposal includes the following new requirements: > > + > > + 1. Support creation of additional packages > > + > > + 2. Allow per-distro packaging tweaks > > + > > +### Approach > > + > > +The new implementation will have the `builddeb` code ported from the > Linux kernel included to the > > +Isar environment. It will be refactor to support pre and post hooks. > Hooks may be used by external > > +layers to extend the kernel packaging instructions provided by "stock" > Isar or by Isar itself to > > +implement distro-specific packages or tweaks. > > + > > +Hooks need the ability to add `build` and `install` steps which can be > achieved by having the main > > +`build` and `install` steps call pre and post scripts. This will > however solve only one side of the > > +problem: new packages should be declared in the `debian/control` file. > The packaging process > > +(initiated by `debian/rules`) will use a baseline `control` file when > started and a mechanism will > > +be provided for hooks to append `control` blocks as they are processed. > > Is this a section addressing users or reviewers? The latter audience is > better addressed via the commit log. > > > + > > +### Future work > > + > > +In the event where this proposal is accepted, here are a few future > projects that are being > > +considered: > > + > > + 1. Package `perf` as Debian does > > + > > + 2. Add `ubuntu` as an Isar supported target distro and validate > `linux-custom` recipe > > + > > +## Benefit to Isar > > + > > + * Control the kernel build and packaging process > > + > > + * No longer need to repack packages > > + > > + * Provide hooks for custom kernel recipes to add build/install > directives > > Careful with hooks! They tend to pile up mess. Where exactly do you want > them? What for? > > > + > > +## How can I help? > > + > > + * Check if you are able to build your own linux-custom recipes with > this change > > + * Check if packages generated by this recipe are compatible with your > Debian-based distro > > + * Review and provide comments on this changeset > > + * Create templates for distro-specific packages (e.g. perf) > > + > > +## Scope > > + > > + * Proposal owners: proposal owners are to communicate the idea to > isar-users and make sure > > + that all of their requirements are met (to the extent they are > already supported with the > > + current solution) or may be met in the future. > > + > > + * Other developers: port your out-of-tree linux-custom recipes to this > new solution and > > + report issues. > > + > > +## How to test? > > + > > +The linux-mainline recipe may be used for some basic testing. This > recipe is being used by the > > +following machines: > > + > > + * de0-nano-soc > > + * qemumipsel > > + > > +## Dependencies > > + > > + * None > > + > > +## Documentation > > + > > + * Document user-visible variables added by this proposal > > + > > +## Questions and answers > > + > > + * None > > diff --git a/meta/recipes-kernel/linux/files/build-kernel.sh > b/meta/recipes-kernel/linux/files/build-kernel.sh > > deleted file mode 100644 > > index f56e96c..0000000 > > --- a/meta/recipes-kernel/linux/files/build-kernel.sh > > +++ /dev/null > > @@ -1,128 +0,0 @@ > > -#!/bin/bash > > -# > > -# Custom kernel build > > -# > > -# This software is a part of ISAR. > > -# Copyright (c) Siemens AG, 2018 > > -# > > -# SPDX-License-Identifier: MIT > > - > > -source /isar/common.sh > > - > > -host_arch=$(dpkg --print-architecture) > > - > > -if [ "$host_arch" != "$target_arch" ]; then > > - case $target_arch in > > - armhf) > > - export ARCH=arm > > - export CROSS_COMPILE="arm-linux-gnueabihf-" > > - ;; > > - arm64) > > - export ARCH=arm64 > > - export CROSS_COMPILE="aarch64-linux-gnu-" > > - ;; > > - mipsel) > > - export ARCH=mips > > - export CROSS_COMPILE="mipsel-linux-gnu-" > > - ;; > > - *) > > - echo "error: unsupported architecture ($target_arch)" > > - exit 1 > > - ;; > > - esac > > -fi > > - > > -REPACK_DIR="$1/../repack" > > -REPACK_LINUX_IMAGE_DIR="${REPACK_DIR}/linux-image" > > -REPACK_LINUX_HEADERS_DIR="${REPACK_DIR}/linux-headers" > > - > > -make ${KERNEL_CONFIG_TARGET} || exit ${?} > > -if [ -n "${KERNEL_FRAGMENTS}" ]; then > > - scripts/kconfig/merge_config.sh -m .config ${KERNEL_FRAGMENTS} > > -fi > > - > > -KV=$( make -s kernelrelease ) > > -if [ "${KV}" != "${PV}" ]; then > > - echo "ERROR: Recipe PV is \"${PV}\" but should be \"${KV}\"" > 1>&2 > > - echo "ERROR: Probably due to CONFIG_LOCALVERSION" 1>&2 > > - exit 1 > > -fi > > - > > -rm -f .version > > -KBUILD_DEBARCH=$target_arch make -j $(($(nproc) * 2)) deb-pkg > > - > > -rm -rf "${REPACK_DIR}" > > -mkdir -p "${REPACK_DIR}" > > -mkdir -p "${REPACK_LINUX_IMAGE_DIR}" > > -mkdir -p "${REPACK_LINUX_HEADERS_DIR}" > > - > > -cp -a debian "${REPACK_DIR}" > > - > > -# dpkg-gencontrol performs cross-incompatible checks on the > > -# Architecture field; trick it to accept the control file > > -sed -i "s/Architecture: .*/Architecture: any/" > "${REPACK_DIR}/debian/control" > > - > > -cd .. > > - > > -dpkg-deb -R linux-image-${PV}_${PV}-1_*.deb "${REPACK_LINUX_IMAGE_DIR}" > > -dpkg-deb -R linux-headers-${PV}_${PV}-1_*.deb > "${REPACK_LINUX_HEADERS_DIR}" > > - > > -dpkg-gencontrol -crepack/debian/control \ > > - -lrepack/debian/changelog \ > > - -frepack/debian/files \ > > - -plinux-image-${PV} \ > > - -P"${REPACK_LINUX_IMAGE_DIR}" \ > > - -DPackage="linux-image-${KERNEL_NAME}" \ > > - -DSection=kernel \ > > - -DPriority=required \ > > - -DDepends="${KERNEL_DEBIAN_DEPENDS}" \ > > - -DArchitecture=$target_arch > > - > > -# Add Debian-like link installation to postinst > > -mkdir -p ${REPACK_LINUX_IMAGE_DIR}/lib/modules/${PV} > > -touch "${REPACK_LINUX_IMAGE_DIR}/lib/modules/${PV}/.fresh-install" > > -sed -i "${REPACK_LINUX_IMAGE_DIR}/DEBIAN/postinst" \ > > - -e "/^set -e$/a\\ > > -\\ > > -if [ -f /lib/modules/${PV}/.fresh-install ]; then\\ > > - change=install\\ > > -else\\ > > - change=upgrade\\ > > -fi\\ > > -linux-update-symlinks \$change ${PV} /boot/${KERNEL_FILE}-${PV}\\ > > -rm -f /lib/modules/${PV}/.fresh-install" > > - > > -# Add Debian-like link removal to postrm > > -sed -i "${REPACK_LINUX_IMAGE_DIR}/DEBIAN/postrm" \ > > - -e "/^set -e$/a\\ > > -\\ > > -rm -f /lib/modules/${PV}/.fresh-install\\ > > -\\ > > -if [ \"\$1\" != upgrade ] && command -v linux-update-symlinks > >/dev/null; then\\ > > - linux-update-symlinks remove ${PV} > /boot/${KERNEL_FILE}-${PV}\\ > > -fi" > > - > > -# Make sure arm64 kernels are decompressed > > -if [ "$target_arch" = "arm64" ]; then > > > - kernel_file="${REPACK_LINUX_IMAGE_DIR}/boot/${KERNEL_FILE}-${PV}" > > - mv "${kernel_file}" "${kernel_file}.gz" > > - gunzip "${kernel_file}.gz" > > -fi > > - > > -dpkg-gencontrol -crepack/debian/control \ > > - -lrepack/debian/changelog \ > > - -frepack/debian/files \ > > - -plinux-headers-${PV} \ > > - -P"${REPACK_LINUX_HEADERS_DIR}" \ > > - -Vkernel:debarch="${KERNEL_NAME}" \ > > - -DPackage="linux-headers-${KERNEL_NAME}" \ > > - -DSection=kernel \ > > - -DDepends="${KERNEL_HEADERS_DEBIAN_DEPENDS}" \ > > - -DArchitecture=$target_arch > > - > > -fakeroot dpkg-deb -b "${REPACK_LINUX_IMAGE_DIR}" \ > > - linux-image-${KERNEL_NAME}_${PV}-1_${KERNEL_NAME}.deb > > -rm -f linux-image-${PV}_${PV}-1_*.deb > > -fakeroot dpkg-deb -b "${REPACK_LINUX_HEADERS_DIR}" \ > > - linux-headers-${KERNEL_NAME}_${PV}-1_${KERNEL_NAME}.deb > > -rm -f linux-headers-${PV}_${PV}-1_*.deb > > diff --git a/meta/recipes-kernel/linux/files/debian/compat > b/meta/recipes-kernel/linux/files/debian/compat > > new file mode 100644 > > index 0000000..ec63514 > > --- /dev/null > > +++ b/meta/recipes-kernel/linux/files/debian/compat > > @@ -0,0 +1 @@ > > +9 > > diff --git a/meta/recipes-kernel/linux/files/debian/control.tmpl > b/meta/recipes-kernel/linux/files/debian/control.tmpl > > new file mode 100644 > > index 0000000..1295466 > > --- /dev/null > > +++ b/meta/recipes-kernel/linux/files/debian/control.tmpl > > @@ -0,0 +1,36 @@ > > +Source: linux-${KERNEL_NAME_PROVIDED} > > +Section: kernel > > +Priority: optional > > +Maintainer: ${MAINTAINER} > > +Build-Depends: bc, kmod, cpio, ${KBUILD_DEPENDS} > > +Homepage: http://www.kernel.org/ > > Standard-Version? > > > + > > +Package: linux-image-${KERNEL_NAME_PROVIDED} > > +Architecture: any > > +Depends: ${KERNEL_DEBIAN_DEPENDS} > > +Description: ${KERNEL_NAME_PROVIDED} Linux kernel, version ${PV} > > + This package contains the Linux kernel, modules and corresponding > other > > + files, version: ${PV}. > > + > > +Package: linux-headers-${KERNEL_NAME_PROVIDED} > > +Architecture: any > > +Depends: ${KERNEL_HEADERS_DEBIAN_DEPENDS}, ${perl:Depends}, > ${shlib:Depends} > > +Description: ${KERNEL_NAME_PROVIDED} Linux kernel headers for ${PV} > > + This package provides kernel header files for ${PV} on ${DISTRO_ARCH} > > + . > > + This is useful for people who need to build external modules > > + > > +Package: linux-libc-dev > > See above, should not be here by default. > > > +Section: devel > > +Provides: linux-kernel-headers > > +Architecture: any > > +Description: Linux support headers for userspace development > > + This package provides userspaces headers from the Linux kernel. These > headers > > + are used by the installed headers for GNU glibc and other system > libraries. > > + > > +Package: linux-image-${KERNEL_NAME_PROVIDED}-dbg > > +Section: debug > > +Architecture: any > > +Description: Linux kernel debugging symbols for ${PV} > > + This package will come in handy if you need to debug the kernel. It > provides > > + all the necessary debug symbols for the kernel and its modules. > > diff --git a/meta/recipes-kernel/linux/files/debian/files > b/meta/recipes-kernel/linux/files/debian/files > > new file mode 100644 > > index 0000000..a8ace32 > > --- /dev/null > > +++ b/meta/recipes-kernel/linux/files/debian/files > > @@ -0,0 +1,5 @@ > > +linux-headers_4.19.80-1_mipsel.deb kernel optional > > +linux-image-dbg_4.19.80-1_mipsel.deb debug optional > > +linux-image_4.19.80-1_mipsel.deb kernel optional > > +linux-libc-dev_4.19.80-1_mipsel.deb devel optional > > +linux_4.19.80-1_mipsel.buildinfo kernel optional > > Looks like an accidentally added file. > > > diff --git a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl > b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl > > new file mode 100644 > > index 0000000..ce79d56 > > --- /dev/null > > +++ b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl > > @@ -0,0 +1,36 @@ > > +#!/bin/sh > > +# Copyright (c) Mentor Graphics, a Siemens business, 2019 > > +# SPDX-License-Identifier: MIT > > + > > +# Load common stuff > > +. ${S}/debian/isar/common || exit ${?} > > + > > +do_build() { > > + > > + # Print a few things that are of particular interest > > + print_settings > > + > > + # Trace what we do here > > + set -x > > + > > + # Process existing kernel configuration to make sure it is complete > > + # (use defaults for options that were not specified) > > + ${MAKE} O=${KERNEL_BUILD_DIR} olddefconfig || exit ${?} > > + > > + # Build the Linux kernel > > + ${MAKE} O=${KERNEL_BUILD_DIR} -j $(nproc) || exit ${?} > > + > > + # Stop tracing > > + set +x > > +} > > + > > +print_settings() { > > + cat <<EOF > > +# Build settings: > > +# --------------- > > +# ARCH=${ARCH} > > +# CROSS_COMPILE=${CROSS_COMPILE} > > +EOF > > +} > > + > > +main build ${*} > > diff --git a/meta/recipes-kernel/linux/files/debian/isar/clean.tmpl > b/meta/recipes-kernel/linux/files/debian/isar/clean.tmpl > > new file mode 100644 > > index 0000000..2aa3742 > > --- /dev/null > > +++ b/meta/recipes-kernel/linux/files/debian/isar/clean.tmpl > > @@ -0,0 +1,20 @@ > > +#!/bin/sh > > +# Copyright (c) Mentor Graphics, a Siemens business, 2019 > > +# SPDX-License-Identifier: MIT > > + > > +# Load common stuff > > +. ${S}/debian/isar/common || exit ${?} > > + > > +do_clean() { > > + > > + # Trace what we do here > > + set -x > > + > > + rm -rf ${deb_img_dir} ${deb_dbg_dir} > > + rm -rf ${deb_kern_hdrdir} ${deb_libc_hdr_dir} > > I suppose we rather (or also?) want "make clean" here for the build > artifacts. Just make sure to save/restore the debian folder across that > step. > > > + > > + # Stop tracing > > + set +x > > +} > > + > > +main clean ${*} > > diff --git a/meta/recipes-kernel/linux/files/debian/isar/common.tmpl > b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl > > new file mode 100644 > > index 0000000..b392b46 > > --- /dev/null > > +++ b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl > > @@ -0,0 +1,60 @@ > > +#!/bin/sh > > +# Copyright (c) Mentor Graphics, a Siemens business, 2019 > > +# SPDX-License-Identifier: MIT > > + > > +# Isar settings > > +ARCH=${KERNEL_ARCH} > > +KERNEL_PKG_IMAGE=linux-image-${KERNEL_NAME_PROVIDED} > > +KERNEL_PKG_KERN_HEADERS=linux-headers-${KERNEL_NAME_PROVIDED} > > +KERNEL_PKG_LIBC_HEADERS=linux-libc-dev > > + > > +# Constants > > +KCONF=.config > > + > > +# Target directories > > +deb_top_dir=${S}/debian > > +deb_img_dir=${deb_top_dir}/${KERNEL_PKG_IMAGE} > > +deb_dbg_dir=${deb_img_dir}-dbg > > +deb_dtb_dir=${deb_img_dir}/${KERNEL_DTB_DIR} > > +deb_kern_hdr_dir=${deb_top_dir}/${KERNEL_PKG_KERN_HEADERS} > > +deb_libc_hdr_dir=${deb_top_dir}/${KERNEL_PKG_LIBC_HEADERS} > > + > > +main() { > > + target=${1} > > + > > + if [ ! -f ${S}/debian/isar/${target} ]; then > > + echo "error: ${target} is not a supported build target!" >&2 > > + return 1 > > + fi > > + > > + # check for scripts to be sourced > > + for f in ${S}/debian/isar/defaults.d/${target}/*; do > > + [ -f ${f} ] || continue > > + . ${f} || return ${?} > > + done > > + > > + # variables to be exported > > + export ARCH > > + > > + # are we cross-compiling? > > + BUILD_ARCH=$(dpkg-architecture -qDEB_BUILD_ARCH) > > + if [ "${BUILD_ARCH}" = "${DISTRO_ARCH}" ]; then > > + # no, make sure CROSS_COMPILE isn't set > > + unset CROSS_COMPILE > > + fi > > + > > + # check for pre-target scripts > > + for f in ${S}/debian/isar/pre.d/${target}/*; do > > + [ -f ${f} ] || continue > > + sh ${f} || return ${?} > > + done > > + > > + # call the actual target script > > + do_${target} || return ${?} > > + > > + # check for post-target scripts > > + for f in ${S}/debian/isar/post.d/${target}/*; do > > + [ -f ${f} ] || continue > > + sh ${f} || return ${?} > > + done > > +} > > diff --git a/meta/recipes-kernel/linux/files/debian/isar/install.tmpl > b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl > > new file mode 100644 > > index 0000000..2055669 > > --- /dev/null > > +++ b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl > > @@ -0,0 +1,197 @@ > > +#!/bin/sh > > +# Copyright (c) Mentor Graphics, a Siemens business, 2019 > > +# SPDX-License-Identifier: MIT > > + > > +# Load common stuff > > +. ${S}/debian/isar/common || exit ${?} > > + > > +do_install() { > > + > > + # check if our kernel was configured > > + if [ ! -f "${O}/.config" ]; then > > + echo "error: kernel not configured!" >&2 > > + return 1 > > + fi > > + > > + # load its configuration > > + . ${O}/.config > > + > > + kimage="$(${MAKE} O=${O} -s --no-print-directory image_name)" > > + case "${ARCH}" in > > + mips|powerpc) kimage_path="boot/vmlinux-${PV}" ;; > > + um) kimage_path="usr/bin/vmlinux-${PV}" ;; > > + *) kimage_path="boot/vmlinuz-${PV}" ;; > > + esac > > + > > + print_settings > > + > > + # Stop on error > > + set -e > > + > > + # Trace what we do here > > + set -x > > + > > + install_image > > + install_hooks > > + install_dtbs > > + install_kmods > > + install_headers > > + > > + # Stop tracing > > + set +x > > +} > > + > > +print_settings() { > > + cat <<EOF > > +Install settings: > > +----------------- > > +deb_dtb_dir=${deb_dtb_dir} > > +deb_hdr_dir=${deb_hdr_dir} > > +kimage=${kimage} > > +kimage_path=${kimage_path} > > + > > +EOF > > +} > > + > > +install_image() { > > + install -m 755 -d ${deb_img_dir}/$(dirname ${kimage_path}) > > + cp ${O}/${kimage} ${deb_img_dir}/${kimage_path} > > + install_image_debug > > +} > > + > > +install_image_debug() { > > + # Different tools want the image in different locations > > + # perf > > + mkdir -p ${deb_dbg_dir}/usr/lib/debug/lib/modules/${PV}/ > > + cp vmlinux ${deb_dbg_dir}/usr/lib/debug/lib/modules/${PV}/ > > + # systemtap > > + mkdir -p ${deb_dbg_dir}/usr/lib/debug/boot/ > > + ln -s ../lib/modules/$version/vmlinux > ${deb_dbg_dir}/usr/lib/debug/boot/vmlinux-${PV} > > + # kdump-tools > > + ln -s lib/modules/${PV}/vmlinux > ${deb_dbg_dir}/usr/lib/debug/vmlinux-${PV} > > +} > > + > > +install_hooks() { > > + install -m 755 -d ${deb_img_dir}/etc/kernel/install.d > > + install -m 755 -d ${deb_img_dir}/etc/kernel/postinst.d > > + install -m 755 -d ${deb_img_dir}/etc/kernel/postrm.d > > + install -m 755 -d ${deb_img_dir}/etc/kernel/prerm.d > > +} > > + > > +install_dtbs() { > > + [ -n "${CONFIG_OF}" ] || return 0 > > + ${MAKE} O=${O} INSTALL_DTBS_PATH=${deb_dtb_dir} dtbs_install > > +} > > + > > +install_kmods() { > > + [ -n "${CONFIG_MODULES}" ] || return 0 > > + ${MAKE} O=${O} INSTALL_MOD_PATH=${deb_img_dir} modules_install > > + touch ${deb_img_dir}/lib/modules/${PV}/.fresh-install > > + rm -fv ${deb_img_dir}/lib/modules/${PV}/build > > + rm -fv ${deb_img_dir}/lib/modules/${PV}/source > > + install_kmods_debug > > +} > > + > > +kmods_sign() { > > + [ -n "${CONFIG_MODULE_SIG_ALL}" ] || return 0 > > + ${MAKE} O=${O} INSTALL_MOD_PATH=${deb_img_dir} modules_sign > > +} > > + > > +install_kmods_debug() { > > + [ -n "${CONFIG_DEBUG_INFO}" ] || return 0 > > + > > + kmod_inst_dir=${deb_img_dir}/lib/modules > > + kmod_debug_dir=${deb_dbg_dir}/usr/lib/debug > > + > > + # copy kernels modules to usr/lib/debug > > + mkdir -p ${kmod_debug_dir} > > + tar -C ${kmod_inst_dir}/ -cO --exclude='modules.*' . | tar -C > ${kmod_debug_dir}/ -xf - > > + # strip everything but debug sections for modules in usr/lib/debug > > + find ${kmod_debug_dir} -name *.ko -exec ${CROSS_COMPILE}objcopy > --only-keep-debug {} \; > > + # and strip debug sections from modules in lib/modules > > + find ${kmod_inst_dir} -name *.ko -exec ${CROSS_COMPILE}objcopy > --strip-debug {} \; > > + > > + # re-sign stripped kernel modules > > + kmods_sign > > +} > > + > > +headers_check() { > > + ${MAKE} O=${O} headers_check > > +} > > + > > +libc_headers() { > > + mkdir -p ${deb_libc_hdr_dir} > > + ${MAKE} O=${O} headers_install > INSTALL_HDR_PATH=${deb_libc_hdr_dir}/usr > > + host_arch=$(dpkg-architecture -a${DISTRO_ARCH} > -qDEB_HOST_MULTIARCH) > > + mkdir ${deb_libc_hdr_dir}/usr/include/${host_arch} > > + mv ${deb_libc_hdr_dir}/usr/include/asm > ${deb_libc_hdr_dir}/usr/include/${host_arch}/ > > +} > > + > > +# the kernel does not seem to provide a way to cross-compile its tools, > we however > > +# have qemu-user-static binaries in our buildchroot. Create a new build > tree for the > > +# scripts and override HOSTCC to force a cross-compile > > +kernel_tools_build() { > > + odir="${S}/build-scripts" > > + mflags="HOSTCC=${CROSS_COMPILE}gcc O=${odir} NOSTDINC_FLAGS=" > > + rm -rf ${odir} && mkdir -p ${odir} && cp > ${S}/${KERNEL_BUILD_DIR}/.config ${odir} > > + (unset LD_PRELOAD; cd ${odir} && make -C ${S} ${mflags} > olddefconfig scripts) > > + (cd ${odir}; find scripts -type f -executable -exec file {} \;|grep > ELF|cut -d: -f1) >${src_hdr_files} > > + tar -C ${odir} -cf - -T - <${src_hdr_files} | tar -C ${destdir} -xf > - > > +} > > + > > +kernel_tools() { > > + # remove object files > > + find ${destdir}/scripts -type f -name '*.o' |xargs rm -f > > + > > + # we're all done if we aren't cross-compiling > > + [ -n "${CROSS_COMPILE}" ] || return 0 > > + > > + # remove ELF executables from the linux-headers package > > + find ${destdir}/scripts -type f -exec file {} \;|grep ELF|cut -d: > -f1|xargs rm -fv > > + > > + # cross-compile kernel tools to be shipped with linux-headers > > + kernel_tools_build > > +} > > + > > +kernel_headers() { > > + destdir=${deb_kern_hdr_dir}/${KERNEL_HEADERS_DIR} > > + src_hdr_files=$(mktemp) > > + obj_hdr_files=$(mktemp) > > + > > + mkdir -p ${destdir} > > + mkdir -p ${deb_kern_hdr_dir}/lib/modules/${PV} > > + > > + (cd ${S}; find . -name 'Makefile*' -o -name 'Kconfig*' -o -name > '*.pl') >>${src_hdr_files} > > + (cd ${S}; find arch/*/include include scripts -type f -o -type l) > >>${src_hdr_files} > > + (cd ${S}; find arch/${ARCH} -name module.lds -o -name > Kbuild.platforms -o -name Platform) >>${src_hdr_files} > > + (cd ${S}; find $(find arch/${ARCH} -name include -o -name scripts > -type d) -type f) >>${src_hdr_files} > > + > > + (cd ${O}; find arch/${ARCH}/include Module.symvers include scripts > -type f) >>${obj_hdr_files} > > + if [ -n "${CONFIG_STACK_VALIDATION}" ]; then > > + (cd ${O}; find tools/objtool -type f -executable) > >>${obj_hdr_files} > > + fi > > + if [ -n "${CONFIG_GCC_PLUGINS}" ]; then > > + (cd ${O}; find scripts/gcc-plugins -name *.so -o -name > gcc-common.h) >>${obj_hdr_files} > > + fi > > + > > + # deploy files that were matched above > > + tar -C ${S} -cf - -T - <${src_hdr_files} | tar -C ${destdir} -xf - > > + tar -C ${O} -cf - -T - <${obj_hdr_files} | tar -C ${destdir} -xf - > > + > > + # add the kernel config > > + cp ${O}/${KCONF} ${destdir}/.config > > + > > + # handle kernel development tools > > + kernel_tools > > + > > + # create symlinks > > + ln -sf /${KERNEL_HEADERS_DIR} > ${deb_kern_hdr_dir}/lib/modules/${PV}/build > > +} > > + > > +install_headers() { > > + headers_check > > + libc_headers > > + kernel_headers > > +} > > + > > +main install ${*} > > diff --git > a/meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl > b/meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl > > new file mode 100644 > > index 0000000..fd89121 > > --- /dev/null > > +++ b/meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl > > @@ -0,0 +1,22 @@ > > +#!/bin/sh > > + > > +set -e > > +set -x > > + > > +if [ -f /lib/modules/${PV}/.fresh-install ]; then > > + change=install > > +else > > + change=upgrade > > +fi > > + > > +linux-update-symlinks $change ${PV} /boot/${KERNEL_FILE}-${PV} > > +rm -f /lib/modules/${PV}/.fresh-install > > + > > +# Pass maintainer script parameters to hook scripts > > +export DEB_MAINT_PARAMS="$*" > > + > > +# Tell initramfs builder whether it's wanted > > +export INITRD=Yes > > I've noticed while hacking my version that this should follow > CONFIG_BLK_DEV_INITRD. > > > + > > +test -d /etc/kernel/postinst.d && run-parts --arg="${PV}" > --arg="/boot/${KERNEL_FILE}-${PV}" /etc/kernel/postinst.d > > +exit 0 > > diff --git > a/meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl > b/meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl > > new file mode 100644 > > index 0000000..9d88218 > > --- /dev/null > > +++ b/meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl > > @@ -0,0 +1,18 @@ > > +#!/bin/sh > > + > > +set -e > > + > > +rm -f /lib/modules/${PV}/.fresh-install > > + > > +if [ "$1" != upgrade ] && command -v linux-update-symlinks >/dev/null; > then > > + linux-update-symlinks remove ${PV} /boot/${KERNEL_FILE}-${PV} > > +fi > > + > > +# Pass maintainer script parameters to hook scripts > > +export DEB_MAINT_PARAMS="$*" > > + > > +# Tell initramfs builder whether it's wanted > > +export INITRD=Yes > > + > > +test -d /etc/kernel/postrm.d && run-parts --arg="${PV}" > --arg="/boot/${KERNEL_FILE}-${PV}" /etc/kernel/postrm.d > > +exit 0 > > diff --git a/meta/recipes-kernel/linux/files/debian/rules.tmpl > b/meta/recipes-kernel/linux/files/debian/rules.tmpl > > There is quite some duplication between postinst and postrm already. > Moreover, I think you are missing preinst and prerm. I think all four > should probably be generated from one template. > Ok I have checked upstream scripts and it seems that they all differ see https://salsa.debian.org/kernel-team/linux/tree/master/debian/templates I would recommend we just copy over their scripts but use our templating system instead of theirs (use ${var} instead of @var@) > > > new file mode 100755 > > index 0000000..93c8a5b > > --- /dev/null > > +++ b/meta/recipes-kernel/linux/files/debian/rules.tmpl > > @@ -0,0 +1,39 @@ > > +#!/usr/bin/make -f > > + > > +export CROSS_COMPILE=$(DEB_HOST_GNU_TYPE)- > > + > > +O:=$(CURDIR)/${KERNEL_BUILD_DIR} > > +S:=$(CURDIR) > > +deb_top_dir:=$(S)/debian > > + > > +DH_VERBOSE=1 > > + > > +# Dynamic variables to be passed to Isar build scripts > > +isar_env=$(strip \ > > + export CROSS_COMPILE='$(CROSS_COMPILE)' && \ > > + export MAKE='$(MAKE)' && \ > > + export O='${O}' \ > > + export S='${S}' \ > > +) > > + > > +%: > > + dh $(@) > > + > > +.PHONY: override_dh_auto_clean > > +override_dh_auto_clean: > > + $(isar_env) && sh $(deb_top_dir)/isar/clean > > + > > +.PHONY: override_dh_auto_build > > +override_dh_auto_build: > > + $(isar_env) && sh $(deb_top_dir)/isar/build > > + > > +override_dh_auto_install: > > + $(isar_env) && sh $(deb_top_dir)/isar/install > > + > > +.PHONY: override_dh_auto_test > > +override_dh_auto_test: > > + true > > + > > +PHONY: override_dh_strip > > +override_dh_strip: > > + dh_strip -Xvmlinux --no-automatic-dbgsym > > diff --git a/meta/recipes-kernel/linux/linux-custom.inc > b/meta/recipes-kernel/linux/linux-custom.inc > > index ca91f64..36086be 100644 > > --- a/meta/recipes-kernel/linux/linux-custom.inc > > +++ b/meta/recipes-kernel/linux/linux-custom.inc > > @@ -2,22 +2,104 @@ > > # > > # This software is a part of ISAR. > > # Copyright (c) Siemens AG, 2018 > > +# Copyright (c) Mentor Graphics, a Siemens business, 2019 > > # > > # SPDX-License-Identifier: MIT > > > > -FILESEXTRAPATHS_prepend := "${FILE_DIRNAME}/files:" > > +# Settings that would typically be done from the custom kernel recipe > > +# ------------------------------------------------------------------- > > > > DESCRIPTION ?= "Custom kernel" > > +MAINTAINER ?= "isar-users <isar-...@googlegroups.com <javascript:>>" > > + > > +KBUILD_DEPENDS ?= "build-essential:native, \ > > + libelf-dev:native, \ > > + libncurses-dev:native, \ > > + libssl-dev:native, \ > > + bc, \ > > + bison, \ > > + cpio, \ > > + flex, \ > > + git, \ > > + kmod, \ > > + libssl-dev," > > + > > +KERNEL_DEBIAN_DEPENDS ?= "initramfs-tools | linux-initramfs-tool, \ > > + kmod, \ > > + linux-base (>= 4.3~)," > > + > > +KERNEL_HEADERS_DEBIAN_DEPENDS ?= "libc6, \ > > + libssl1.1," > > + > > +KERNEL_LIBC_DEV_DEPLOY ?= "0" > > + > > +# Settings that may be changed on a per distro, machine or layer basis > > +# -------------------------------------------------------------------- > > + > > +KERNEL_DTB_DIR ?= "usr/lib/linux-${KERNEL_NAME_PROVIDED}-${PV}" > > +KERNEL_HEADERS_DIR ?= "usr/src/linux-headers-${PV}" > > > > -KERNEL_NAME_PROVIDED ?= "${@ d.getVar('PN', > True).partition('linux-')[2]}" > > KERNEL_DEFCONFIG ?= "" > > > > +# Add our template meta-data to the sources > > +FILESEXTRAPATHS_prepend := "${FILE_DIRNAME}/files:" > > +SRC_URI += "file://debian" > > + > > +# Variables and files that make our templates > > +# ------------------------------------------- > > + > > +TEMPLATE_FILES += " \ > > + debian/control.tmpl \ > > + debian/isar/build.tmpl \ > > + debian/isar/clean.tmpl \ > > + debian/isar/common.tmpl \ > > + debian/isar/install.tmpl \ > > + debian/linux-image.postinst.tmpl \ > > + debian/linux-image.postrm.tmpl \ > > + debian/rules.tmpl \ > > +" > > + > > +TEMPLATE_VARS += " \ > > + KBUILD_DEPENDS \ > > + KERNEL_ARCH \ > > + KERNEL_DEBIAN_DEPENDS \ > > + KERNEL_BUILD_DIR \ > > + KERNEL_DTB_DIR \ > > + KERNEL_FILE \ > > + KERNEL_HEADERS_DEBIAN_DEPENDS \ > > + KERNEL_HEADERS_DIR \ > > + KERNEL_NAME_PROVIDED \ > > +" > > + > > +inherit dpkg > > +inherit template > > + > > +# Derive name of the kernel packages from the name of this recipe > > +KERNEL_NAME_PROVIDED ?= "${@ d.getVar('PN', > True).partition('linux-')[2]}" > > + > > +# Make bitbake know we will be producing linux-image and linux-headers > packages > > python() { > > kernel_name = d.getVar("KERNEL_NAME_PROVIDED", True) > > d.setVar('PROVIDES', 'linux-image-' + kernel_name + ' ' + \ > > 'linux-headers-' + kernel_name) > > } > > > > +def get_kernel_arch(d): > > + distro_arch = d.getVar("DISTRO_ARCH") > > + if distro_arch == "amd64": > > + kernel_arch = "x86" > > + elif distro_arch == "arm64": > > + kernel_arch = "arm64" > > + elif distro_arch == "armhf": > > + kernel_arch = "arm" > > + elif distro_arch == "mipsel": > > + kernel_arch = "mips" > > + else: > > + kernel_arch = "" > > + return kernel_arch > > + > > +KERNEL_ARCH ??= "${@get_kernel_arch(d)}" > > + > > def config_fragments(d): > > fragments = [] > > sources = d.getVar("SRC_URI").split() > > @@ -31,57 +113,35 @@ def config_fragments(d): > > fragments.append(local) > > return fragments > > > > -inherit dpkg-base > > - > > -SRC_URI += "file://build-kernel.sh" > > +do_prepare_build_prepend() { > > + # copy meta-data over to source tree > > + rm -rf ${S}/debian > > + cp -r ${WORKDIR}/debian ${S}/ > > > > -KBUILD_DEPENDS ?= " \ > > - build-essential:native \ > > - libssl-dev \ > > - libelf-dev \ > > - bc \ > > - git \ > > - kmod \ > > - bison \ > > - flex \ > > - cpio \ > > - libncurses-dev" > > -KERNEL_DEBIAN_DEPENDS ?= "initramfs-tools | linux-initramfs-tool, kmod, > linux-base (>= 4.3~)" > > -KERNEL_HEADERS_DEBIAN_DEPENDS ?= "libc6, libssl1.1" > > + # remove templates from the source tree > > + find ${S}/debian -name *.tmpl | xargs rm -f > > > > -KERNEL_LIBC_DEV_DEPLOY ?= "0" > > + # rename install/remove hooks to match user-specified name for > our linux-image package > > + mv ${S}/debian/linux-image.postinst > ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.postinst > > + mv ${S}/debian/linux-image.postrm > ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.postrm > > > > -do_install_builddeps() { > > - dpkg_do_mounts > > - E="${@ bb.utils.export_proxies(d)}" > > - sudo -E chroot ${BUILDCHROOT_DIR} \ > > - apt-get update \ > > - -o > Dir::Etc::SourceList="sources.list.d/isar-apt.list" \ > > - -o Dir::Etc::SourceParts="-" \ > > - -o APT::Get::List-Cleanup="0" > > - sudo -E chroot ${BUILDCHROOT_DIR} \ > > - apt-get install \ > > - -y -o Debug::pkgProblemResolver=yes \ > > - --no-install-recommends ${KBUILD_DEPENDS} > > - dpkg_undo_mounts > > + # produce a changelog for our kernel build > > + deb_add_changelog > > } > > > > -addtask install_builddeps after do_prepare_build before do_dpkg_build > > -# apt and reprepro may not run in parallel, acquire the Isar lock > > -do_install_builddeps[lockfiles] += "${REPO_ISAR_DIR}/isar.lock" > > - > > -addtask devshell after do_install_builddeps > > +# build directory for our "full" kernel build > > +KERNEL_BUILD_DIR = "build-full" > > > > -dpkg_runbuild() { > > - chmod +x ${WORKDIR}/build-kernel.sh > > - KERNEL_CONFIG_TARGET="${KERNEL_DEFCONFIG}" > > +dpkg_configure_kernel() { > > + config_target="${KERNEL_DEFCONFIG}" > > + rm -rf ${S}/${KERNEL_BUILD_DIR} && mkdir -p > ${S}/${KERNEL_BUILD_DIR} > > if [ -n "${KERNEL_DEFCONFIG}" ]; then > > if [ -e "${WORKDIR}/${KERNEL_DEFCONFIG}" ]; then > > - cp ${WORKDIR}/${KERNEL_DEFCONFIG} ${S}/.config > > - KERNEL_CONFIG_TARGET="olddefconfig" > > + cp ${WORKDIR}/${KERNEL_DEFCONFIG} > ${S}/${KERNEL_BUILD_DIR}/.config > > + config_target="olddefconfig" > > fi > > else > > - KERNEL_CONFIG_TARGET="defconfig" > > + config_target="defconfig" > > fi > > > > # copy config fragments over to the kernel tree > > @@ -92,21 +152,23 @@ dpkg_runbuild() { > > mkdir -p ${S}/debian/fragments > > (cd ${WORKDIR} && cp ${src_frags} > ${S}/debian/fragments/) > > fi > > - export KERNEL_FRAGMENTS="${out_frags}" > > > > - E="${@ bb.utils.export_proxies(d)}" > > - > > - export PV=${PV} > > - export KERNEL_NAME=${KERNEL_NAME_PROVIDED} > > - > > - export KBUILD_DEPENDS="${KBUILD_DEPENDS}" > > - export KERNEL_CONFIG_TARGET > > - export KERNEL_DEBIAN_DEPENDS="${KERNEL_DEBIAN_DEPENDS}" > > - export KERNEL_FILE="${KERNEL_FILE}" > > - export > KERNEL_HEADERS_DEBIAN_DEPENDS="${KERNEL_HEADERS_DEBIAN_DEPENDS}" > > + sudo -E chroot --userspec=$(id -u):$(id -g) ${BUILDCHROOT_DIR} > sh -c " \ > > + export ARCH=${KERNEL_ARCH} && > \ > > + cd ${PP}/${PPS} && > \ > > + make O=${KERNEL_BUILD_DIR} ${config_target} && > \ > > + ./scripts/kconfig/merge_config.sh > \ > > + -O ${KERNEL_BUILD_DIR}/ > \ > > + ${KERNEL_BUILD_DIR}/.config > \ > > + ${out_frags} > \ > > + " > > +} > > > > - sudo -E chroot --userspec=$( id -u ):$( id -g ) > ${BUILDCHROOT_DIR} ${PP}/build-kernel.sh ${PP}/${PPS} ${DISTRO_ARCH} > > +dpkg_runbuild_prepend() { > > + dpkg_configure_kernel > > +} > > > > +dpkg_runbuild_append() { > > if [ "${KERNEL_LIBC_DEV_DEPLOY}" != "1" ]; then > > rm -f ${WORKDIR}/linux-libc-dev_${PV}*.deb > > fi > > > > Valuable step forward! I still need to check some more details against > my unfinished prototype, but it seems it generally a superset of it. > > Jan > > -- > Siemens AG, Corporate Technology, CT RDA IOT SES-DE > Corporate Competence Center Embedded Linux > [-- Attachment #1.2: Type: text/html, Size: 56839 bytes --] ^ permalink raw reply [flat|nested] 83+ messages in thread
* Re: [PATCH 7/7] linux-custom: rewrite to no longer depend on the kernel's builddeb 2019-11-05 20:16 ` chombourger @ 2019-11-05 20:24 ` Jan Kiszka 0 siblings, 0 replies; 83+ messages in thread From: Jan Kiszka @ 2019-11-05 20:24 UTC (permalink / raw) To: chombourger, isar-users On 05.11.19 21:16, chombourger@gmail.com wrote: > On Tuesday, November 5, 2019 at 4:26:33 PM UTC+1, Jan Kiszka wrote: > > On 05.11.19 14:57, Cedric Hombourger wrote: > > diff --git > a/meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl > b/meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl > > new file mode 100644 > > index 0000000..9d88218 > > --- /dev/null > > +++ b/meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl > > @@ -0,0 +1,18 @@ > > +#!/bin/sh > > + > > +set -e > > + > > +rm -f /lib/modules/${PV}/.fresh-install > > + > > +if [ "$1" != upgrade ] && command -v linux-update-symlinks > >/dev/null; then > > + linux-update-symlinks remove ${PV} > /boot/${KERNEL_FILE}-${PV} > > +fi > > + > > +# Pass maintainer script parameters to hook scripts > > +export DEB_MAINT_PARAMS="$*" > > + > > +# Tell initramfs builder whether it's wanted > > +export INITRD=Yes > > + > > +test -d /etc/kernel/postrm.d && run-parts --arg="${PV}" > --arg="/boot/${KERNEL_FILE}-${PV}" /etc/kernel/postrm.d > > +exit 0 > > diff --git a/meta/recipes-kernel/linux/files/debian/rules.tmpl > b/meta/recipes-kernel/linux/files/debian/rules.tmpl > > There is quite some duplication between postinst and postrm already. > Moreover, I think you are missing preinst and prerm. I think all four > should probably be generated from one template. > > > Ok I have checked upstream scripts and it seems that they all differ > see https://salsa.debian.org/kernel-team/linux/tree/master/debian/templates > > I would recommend we just copy over their scripts but use our templating > system instead of theirs > (use ${var} instead of @var@) > Yes, seems reasonable. I checked but didn't spot anything that seems specific to their kernel. So far we lived from the scripted that the kernel generates, and those have much higher commonality. But they likely miss some useful bits. Jan -- Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 83+ messages in thread
* Re: [PATCH 7/7] linux-custom: rewrite to no longer depend on the kernel's builddeb 2019-11-05 13:57 ` [PATCH 7/7] linux-custom: rewrite to no longer depend on the kernel's builddeb Cedric Hombourger 2019-11-05 15:26 ` Jan Kiszka @ 2019-11-07 11:44 ` Gylstorff Quirin 2019-11-07 11:55 ` Cedric Hombourger 2019-11-07 13:20 ` Cedric Hombourger 1 sibling, 2 replies; 83+ messages in thread From: Gylstorff Quirin @ 2019-11-07 11:44 UTC (permalink / raw) To: isar-users, Cedric_Hombourger On 11/5/19 2:57 PM, Cedric Hombourger wrote: > -KV=$( make -s kernelrelease ) > -if [ "${KV}" != "${PV}" ]; then > - echo "ERROR: Recipe PV is \"${PV}\" but should be \"${KV}\"" 1>&2 > - echo "ERROR: Probably due to CONFIG_LOCALVERSION" 1>&2 > - exit 1 > -fi > - Can we now use CONFIG_LOCALVERSION in a kernel configuration? Quirin ^ permalink raw reply [flat|nested] 83+ messages in thread
* Re: [PATCH 7/7] linux-custom: rewrite to no longer depend on the kernel's builddeb 2019-11-07 11:44 ` Gylstorff Quirin @ 2019-11-07 11:55 ` Cedric Hombourger 2019-11-07 13:20 ` Cedric Hombourger 1 sibling, 0 replies; 83+ messages in thread From: Cedric Hombourger @ 2019-11-07 11:55 UTC (permalink / raw) To: Gylstorff Quirin, isar-users On 11/7/19 12:44 PM, Gylstorff Quirin wrote: > > > On 11/5/19 2:57 PM, Cedric Hombourger wrote: >> -KV=$( make -s kernelrelease ) >> -if [ "${KV}" != "${PV}" ]; then >> - echo "ERROR: Recipe PV is \"${PV}\" but should be \"${KV}\"" 1>&2 >> - echo "ERROR: Probably due to CONFIG_LOCALVERSION" 1>&2 >> - exit 1 >> -fi >> - > > Can we now use CONFIG_LOCALVERSION in a kernel configuration? good catch. let me check. > > Quirin ^ permalink raw reply [flat|nested] 83+ messages in thread
* Re: [PATCH 7/7] linux-custom: rewrite to no longer depend on the kernel's builddeb 2019-11-07 11:44 ` Gylstorff Quirin 2019-11-07 11:55 ` Cedric Hombourger @ 2019-11-07 13:20 ` Cedric Hombourger 2019-11-07 15:43 ` Gylstorff Quirin 1 sibling, 1 reply; 83+ messages in thread From: Cedric Hombourger @ 2019-11-07 13:20 UTC (permalink / raw) To: Gylstorff Quirin, isar-users On 11/7/19 12:44 PM, Gylstorff Quirin wrote: > > > On 11/5/19 2:57 PM, Cedric Hombourger wrote: >> -KV=$( make -s kernelrelease ) >> -if [ "${KV}" != "${PV}" ]; then >> - echo "ERROR: Recipe PV is \"${PV}\" but should be \"${KV}\"" 1>&2 >> - echo "ERROR: Probably due to CONFIG_LOCALVERSION" 1>&2 >> - exit 1 >> -fi >> - > > Can we now use CONFIG_LOCALVERSION in a kernel configuration? looks like I need to handle that. do we want to maintain the current behavior and make sure that PV and $(make -s kernelrelease) match or do we want to accept something like (pseudo-code): KR=$(make -s kernelrelease) if [ "${PV}" != "${KR}" -a "${PV}${CONFIG_LOCALVERSION}" != "${KR}" ]; then echo "ERROR: Recipe PV (${PV}) does not seem to match the kernelrelease (${KR})!" 1>&2 echo "ERROR: Make sure the kernel version in your PV setting and/or CONFIG_LOCALVERSION are aligned" 1>&2 exit 1 fi > > Quirin ^ permalink raw reply [flat|nested] 83+ messages in thread
* Re: [PATCH 7/7] linux-custom: rewrite to no longer depend on the kernel's builddeb 2019-11-07 13:20 ` Cedric Hombourger @ 2019-11-07 15:43 ` Gylstorff Quirin 0 siblings, 0 replies; 83+ messages in thread From: Gylstorff Quirin @ 2019-11-07 15:43 UTC (permalink / raw) To: Cedric Hombourger, isar-users On 11/7/19 2:20 PM, Cedric Hombourger wrote: > > On 11/7/19 12:44 PM, Gylstorff Quirin wrote: >> >> >> On 11/5/19 2:57 PM, Cedric Hombourger wrote: >>> -KV=$( make -s kernelrelease ) >>> -if [ "${KV}" != "${PV}" ]; then >>> - echo "ERROR: Recipe PV is \"${PV}\" but should be \"${KV}\"" 1>&2 >>> - echo "ERROR: Probably due to CONFIG_LOCALVERSION" 1>&2 >>> - exit 1 >>> -fi >>> - >> >> Can we now use CONFIG_LOCALVERSION in a kernel configuration? > > looks like I need to handle that. > > do we want to maintain the current behavior and make sure that PV and > $(make -s kernelrelease) match or do we want to accept something like > (pseudo-code): > > KR=$(make -s kernelrelease) > if [ "${PV}" != "${KR}" -a "${PV}${CONFIG_LOCALVERSION}" != "${KR}" ]; then > echo "ERROR: Recipe PV (${PV}) does not seem to match the > kernelrelease (${KR})!" 1>&2 > echo "ERROR: Make sure the kernel version in your PV setting and/or > CONFIG_LOCALVERSION are aligned" 1>&2 > exit 1 > fi > From my understanding should it be allowed that the package version and the version differ. What happens if we let the user set KR + CONFIG_LOCALVERSION as Version like Debian? Example current 4.19 Kernel image: Package: linux-image-4.19.0-6-amd64 Version: 4.19.67-2+deb10u1 Built-Using: linux (= 4.19.67-2+deb10u1) Priority: optional Section: kernel The original code was added to avoid confusion. build-kernel.sh: Handle kernel configs with CONFIG_LOCALVERSION Setting this option in your kernel config will result in filenames that build-kernel.sh does not expect. To not introduce confusion between recipe versions and debian package versions simply require the recipe to have a PV matching "make kernelrelease". Signed-off-by: Henning Schild <henning.schild@siemens.com> Quirin ^ permalink raw reply [flat|nested] 83+ messages in thread
* Re: [PATCH 0/7] linux-custom recipe rework 2019-11-05 13:55 [PATCH 0/7] linux-custom recipe rework Cedric Hombourger ` (4 preceding siblings ...) 2019-11-05 13:57 ` [PATCH 5/7] linux-mainline: disable support for HFS to demonstrate use of " Cedric Hombourger @ 2019-11-05 15:32 ` Jan Kiszka 2019-11-05 15:55 ` Henning Schild 2019-11-07 9:23 ` [PATCH L-C v2 " Cedric Hombourger 7 siblings, 0 replies; 83+ messages in thread From: Jan Kiszka @ 2019-11-05 15:32 UTC (permalink / raw) To: Cedric Hombourger, isar-users On 05.11.19 14:55, Cedric Hombourger wrote: > The Linux kernel is packaged a bit differently between Debian variants > and in particular for the perf package(s) in Debian vs Ubuntu. Since > there is no way the kernel build scripts would ever get distro specific > support for building more than the kernel and headers, we need our own > build/packaging scripts. The approach is documented in the custom_kernel.md > file included in this patch series. > > This changeset was tested against ci_build.sh and some images were booted > on hardware available here. No regressions were detected (to date). > > It is probably a good time to seek comments on the changeset and ideally > get it merged before starting some of the work-packages such as getting > perf packaged for Debian and Ubuntu distros. > > Cedric Hombourger (7): > recipes-kernel/linux: make KERNEL_DEFCONFIG support in-tree defconfigs > linux-mainline: fix stripping of .0 from the kernel version > linux-mainline: update from 4.19.0 to 4.19.80 > linux-custom: add support for kernel config fragments > linux-mainline: disable support for HFS to demonstrate use of config fragments > buildchroot-host: install qemu-static to support hybrid cross-compiles > linux-custom: rewrite to no longer depend on the kernel's builddeb > > doc/custom_kernel.md | 181 ++++++++++++++++ > .../recipes-kernel/linux/files/no-fs-hfs.cfg | 2 + > ...ne_4.19.0.bb => linux-mainline_4.19.80.bb} | 8 +- > .../buildchroot/buildchroot-host.bb | 6 + > .../linux/files/build-kernel.sh | 129 ------------ > meta/recipes-kernel/linux/files/debian/compat | 1 + > .../linux/files/debian/control.tmpl | 36 ++++ > meta/recipes-kernel/linux/files/debian/files | 5 + > .../linux/files/debian/isar/build.tmpl | 36 ++++ > .../linux/files/debian/isar/clean.tmpl | 20 ++ > .../linux/files/debian/isar/common.tmpl | 60 ++++++ > .../linux/files/debian/isar/install.tmpl | 197 ++++++++++++++++++ > .../files/debian/linux-image.postinst.tmpl | 22 ++ > .../files/debian/linux-image.postrm.tmpl | 18 ++ > .../linux/files/debian/rules.tmpl | 39 ++++ > meta/recipes-kernel/linux/linux-custom.inc | 196 ++++++++++++----- > 16 files changed, 772 insertions(+), 184 deletions(-) > create mode 100644 doc/custom_kernel.md > create mode 100644 meta-isar/recipes-kernel/linux/files/no-fs-hfs.cfg > rename meta-isar/recipes-kernel/linux/{linux-mainline_4.19.0.bb => linux-mainline_4.19.80.bb} (67%) > delete mode 100644 meta/recipes-kernel/linux/files/build-kernel.sh > create mode 100644 meta/recipes-kernel/linux/files/debian/compat > create mode 100644 meta/recipes-kernel/linux/files/debian/control.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/files > create mode 100644 meta/recipes-kernel/linux/files/debian/isar/build.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/isar/clean.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/isar/common.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/isar/install.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl > create mode 100755 meta/recipes-kernel/linux/files/debian/rules.tmpl > Should have been v3? ;) Jan -- Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 83+ messages in thread
* Re: [PATCH 0/7] linux-custom recipe rework 2019-11-05 13:55 [PATCH 0/7] linux-custom recipe rework Cedric Hombourger ` (5 preceding siblings ...) 2019-11-05 15:32 ` [PATCH 0/7] linux-custom recipe rework Jan Kiszka @ 2019-11-05 15:55 ` Henning Schild 2019-11-05 16:00 ` Cedric Hombourger 2019-11-07 9:23 ` [PATCH L-C v2 " Cedric Hombourger 7 siblings, 1 reply; 83+ messages in thread From: Henning Schild @ 2019-11-05 15:55 UTC (permalink / raw) To: Cedric Hombourger; +Cc: isar-users Is that not v3? and what has changed since v2? Henning On Tue, 5 Nov 2019 14:55:38 +0100 Cedric Hombourger <Cedric_Hombourger@mentor.com> wrote: > The Linux kernel is packaged a bit differently between Debian variants > and in particular for the perf package(s) in Debian vs Ubuntu. Since > there is no way the kernel build scripts would ever get distro > specific support for building more than the kernel and headers, we > need our own build/packaging scripts. The approach is documented in > the custom_kernel.md file included in this patch series. > > This changeset was tested against ci_build.sh and some images were > booted on hardware available here. No regressions were detected (to > date). > > It is probably a good time to seek comments on the changeset and > ideally get it merged before starting some of the work-packages such > as getting perf packaged for Debian and Ubuntu distros. > > Cedric Hombourger (7): > recipes-kernel/linux: make KERNEL_DEFCONFIG support in-tree > defconfigs linux-mainline: fix stripping of .0 from the kernel version > linux-mainline: update from 4.19.0 to 4.19.80 > linux-custom: add support for kernel config fragments > linux-mainline: disable support for HFS to demonstrate use of > config fragments buildchroot-host: install qemu-static to support > hybrid cross-compiles linux-custom: rewrite to no longer depend on > the kernel's builddeb > > doc/custom_kernel.md | 181 ++++++++++++++++ > .../recipes-kernel/linux/files/no-fs-hfs.cfg | 2 + > ...ne_4.19.0.bb => linux-mainline_4.19.80.bb} | 8 +- > .../buildchroot/buildchroot-host.bb | 6 + > .../linux/files/build-kernel.sh | 129 ------------ > meta/recipes-kernel/linux/files/debian/compat | 1 + > .../linux/files/debian/control.tmpl | 36 ++++ > meta/recipes-kernel/linux/files/debian/files | 5 + > .../linux/files/debian/isar/build.tmpl | 36 ++++ > .../linux/files/debian/isar/clean.tmpl | 20 ++ > .../linux/files/debian/isar/common.tmpl | 60 ++++++ > .../linux/files/debian/isar/install.tmpl | 197 > ++++++++++++++++++ .../files/debian/linux-image.postinst.tmpl | > 22 ++ .../files/debian/linux-image.postrm.tmpl | 18 ++ > .../linux/files/debian/rules.tmpl | 39 ++++ > meta/recipes-kernel/linux/linux-custom.inc | 196 ++++++++++++----- > 16 files changed, 772 insertions(+), 184 deletions(-) > create mode 100644 doc/custom_kernel.md > create mode 100644 meta-isar/recipes-kernel/linux/files/no-fs-hfs.cfg > rename meta-isar/recipes-kernel/linux/{linux-mainline_4.19.0.bb => > linux-mainline_4.19.80.bb} (67%) delete mode 100644 > meta/recipes-kernel/linux/files/build-kernel.sh create mode 100644 > meta/recipes-kernel/linux/files/debian/compat create mode 100644 > meta/recipes-kernel/linux/files/debian/control.tmpl create mode > 100644 meta/recipes-kernel/linux/files/debian/files create mode > 100644 meta/recipes-kernel/linux/files/debian/isar/build.tmpl create > mode 100644 meta/recipes-kernel/linux/files/debian/isar/clean.tmpl > create mode 100644 > meta/recipes-kernel/linux/files/debian/isar/common.tmpl create mode > 100644 meta/recipes-kernel/linux/files/debian/isar/install.tmpl > create mode 100644 > meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl > create mode 100644 > meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl create > mode 100755 meta/recipes-kernel/linux/files/debian/rules.tmpl > ^ permalink raw reply [flat|nested] 83+ messages in thread
* Re: [PATCH 0/7] linux-custom recipe rework 2019-11-05 15:55 ` Henning Schild @ 2019-11-05 16:00 ` Cedric Hombourger 2019-11-05 16:19 ` Henning Schild 0 siblings, 1 reply; 83+ messages in thread From: Cedric Hombourger @ 2019-11-05 16:00 UTC (permalink / raw) To: Henning Schild; +Cc: isar-users On 11/5/2019 4:55 PM, Henning Schild wrote: > Is that not v3? and what has changed since v2? this is the first time the reworked linux-custom recipe has been submitted (definitely a v1) I have included other patches that got me there and were previously submitted as mean to give you guys a single set to review but I can go either way and keep the little tweaks I worked on before separate and have this PR only contain the 2 patches that were added. I leave it up to you.. whatever is easier for you guys. > > Henning > > On Tue, 5 Nov 2019 14:55:38 +0100 > Cedric Hombourger <Cedric_Hombourger@mentor.com> wrote: > >> The Linux kernel is packaged a bit differently between Debian variants >> and in particular for the perf package(s) in Debian vs Ubuntu. Since >> there is no way the kernel build scripts would ever get distro >> specific support for building more than the kernel and headers, we >> need our own build/packaging scripts. The approach is documented in >> the custom_kernel.md file included in this patch series. >> >> This changeset was tested against ci_build.sh and some images were >> booted on hardware available here. No regressions were detected (to >> date). >> >> It is probably a good time to seek comments on the changeset and >> ideally get it merged before starting some of the work-packages such >> as getting perf packaged for Debian and Ubuntu distros. >> >> Cedric Hombourger (7): >> recipes-kernel/linux: make KERNEL_DEFCONFIG support in-tree >> defconfigs linux-mainline: fix stripping of .0 from the kernel version >> linux-mainline: update from 4.19.0 to 4.19.80 >> linux-custom: add support for kernel config fragments >> linux-mainline: disable support for HFS to demonstrate use of >> config fragments buildchroot-host: install qemu-static to support >> hybrid cross-compiles linux-custom: rewrite to no longer depend on >> the kernel's builddeb >> >> doc/custom_kernel.md | 181 ++++++++++++++++ >> .../recipes-kernel/linux/files/no-fs-hfs.cfg | 2 + >> ...ne_4.19.0.bb => linux-mainline_4.19.80.bb} | 8 +- >> .../buildchroot/buildchroot-host.bb | 6 + >> .../linux/files/build-kernel.sh | 129 ------------ >> meta/recipes-kernel/linux/files/debian/compat | 1 + >> .../linux/files/debian/control.tmpl | 36 ++++ >> meta/recipes-kernel/linux/files/debian/files | 5 + >> .../linux/files/debian/isar/build.tmpl | 36 ++++ >> .../linux/files/debian/isar/clean.tmpl | 20 ++ >> .../linux/files/debian/isar/common.tmpl | 60 ++++++ >> .../linux/files/debian/isar/install.tmpl | 197 >> ++++++++++++++++++ .../files/debian/linux-image.postinst.tmpl | >> 22 ++ .../files/debian/linux-image.postrm.tmpl | 18 ++ >> .../linux/files/debian/rules.tmpl | 39 ++++ >> meta/recipes-kernel/linux/linux-custom.inc | 196 ++++++++++++----- >> 16 files changed, 772 insertions(+), 184 deletions(-) >> create mode 100644 doc/custom_kernel.md >> create mode 100644 meta-isar/recipes-kernel/linux/files/no-fs-hfs.cfg >> rename meta-isar/recipes-kernel/linux/{linux-mainline_4.19.0.bb => >> linux-mainline_4.19.80.bb} (67%) delete mode 100644 >> meta/recipes-kernel/linux/files/build-kernel.sh create mode 100644 >> meta/recipes-kernel/linux/files/debian/compat create mode 100644 >> meta/recipes-kernel/linux/files/debian/control.tmpl create mode >> 100644 meta/recipes-kernel/linux/files/debian/files create mode >> 100644 meta/recipes-kernel/linux/files/debian/isar/build.tmpl create >> mode 100644 meta/recipes-kernel/linux/files/debian/isar/clean.tmpl >> create mode 100644 >> meta/recipes-kernel/linux/files/debian/isar/common.tmpl create mode >> 100644 meta/recipes-kernel/linux/files/debian/isar/install.tmpl >> create mode 100644 >> meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl >> create mode 100644 >> meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl create >> mode 100755 meta/recipes-kernel/linux/files/debian/rules.tmpl >> ^ permalink raw reply [flat|nested] 83+ messages in thread
* Re: [PATCH 0/7] linux-custom recipe rework 2019-11-05 16:00 ` Cedric Hombourger @ 2019-11-05 16:19 ` Henning Schild 0 siblings, 0 replies; 83+ messages in thread From: Henning Schild @ 2019-11-05 16:19 UTC (permalink / raw) To: Cedric Hombourger; +Cc: isar-users Am Tue, 5 Nov 2019 17:00:32 +0100 schrieb Cedric Hombourger <cedric_hombourger@mentor.com>: > On 11/5/2019 4:55 PM, Henning Schild wrote: > > Is that not v3? and what has changed since v2? > > this is the first time the reworked linux-custom recipe has been > submitted (definitely a v1) > > I have included other patches that got me there and were previously > submitted as mean to give you guys a single set to review but I can > go either way and keep the little tweaks I worked on before separate > and have this PR only contain the 2 patches that were added. I leave > it up to you.. whatever is easier for you guys. Ok, i just looked at the first few and was sure that i saw those before. Henning > > > > Henning > > > > On Tue, 5 Nov 2019 14:55:38 +0100 > > Cedric Hombourger <Cedric_Hombourger@mentor.com> wrote: > > > >> The Linux kernel is packaged a bit differently between Debian > >> variants and in particular for the perf package(s) in Debian vs > >> Ubuntu. Since there is no way the kernel build scripts would ever > >> get distro specific support for building more than the kernel and > >> headers, we need our own build/packaging scripts. The approach is > >> documented in the custom_kernel.md file included in this patch > >> series. > >> > >> This changeset was tested against ci_build.sh and some images were > >> booted on hardware available here. No regressions were detected (to > >> date). > >> > >> It is probably a good time to seek comments on the changeset and > >> ideally get it merged before starting some of the work-packages > >> such as getting perf packaged for Debian and Ubuntu distros. > >> > >> Cedric Hombourger (7): > >> recipes-kernel/linux: make KERNEL_DEFCONFIG support in-tree > >> defconfigs linux-mainline: fix stripping of .0 from the kernel > >> version linux-mainline: update from 4.19.0 to 4.19.80 > >> linux-custom: add support for kernel config fragments > >> linux-mainline: disable support for HFS to demonstrate use of > >> config fragments buildchroot-host: install qemu-static to support > >> hybrid cross-compiles linux-custom: rewrite to no longer depend on > >> the kernel's builddeb > >> > >> doc/custom_kernel.md | 181 > >> ++++++++++++++++ .../recipes-kernel/linux/files/no-fs-hfs.cfg | > >> 2 + ...ne_4.19.0.bb => linux-mainline_4.19.80.bb} | 8 +- > >> .../buildchroot/buildchroot-host.bb | 6 + > >> .../linux/files/build-kernel.sh | 129 ------------ > >> meta/recipes-kernel/linux/files/debian/compat | 1 + > >> .../linux/files/debian/control.tmpl | 36 ++++ > >> meta/recipes-kernel/linux/files/debian/files | 5 + > >> .../linux/files/debian/isar/build.tmpl | 36 ++++ > >> .../linux/files/debian/isar/clean.tmpl | 20 ++ > >> .../linux/files/debian/isar/common.tmpl | 60 ++++++ > >> .../linux/files/debian/isar/install.tmpl | 197 > >> ++++++++++++++++++ .../files/debian/linux-image.postinst.tmpl | > >> 22 ++ .../files/debian/linux-image.postrm.tmpl | 18 ++ > >> .../linux/files/debian/rules.tmpl | 39 ++++ > >> meta/recipes-kernel/linux/linux-custom.inc | 196 > >> ++++++++++++----- 16 files changed, 772 insertions(+), 184 > >> deletions(-) create mode 100644 doc/custom_kernel.md > >> create mode 100644 > >> meta-isar/recipes-kernel/linux/files/no-fs-hfs.cfg rename > >> meta-isar/recipes-kernel/linux/{linux-mainline_4.19.0.bb => > >> linux-mainline_4.19.80.bb} (67%) delete mode 100644 > >> meta/recipes-kernel/linux/files/build-kernel.sh create mode 100644 > >> meta/recipes-kernel/linux/files/debian/compat create mode 100644 > >> meta/recipes-kernel/linux/files/debian/control.tmpl create mode > >> 100644 meta/recipes-kernel/linux/files/debian/files create mode > >> 100644 meta/recipes-kernel/linux/files/debian/isar/build.tmpl > >> create mode 100644 > >> meta/recipes-kernel/linux/files/debian/isar/clean.tmpl create mode > >> 100644 meta/recipes-kernel/linux/files/debian/isar/common.tmpl > >> create mode 100644 > >> meta/recipes-kernel/linux/files/debian/isar/install.tmpl create > >> mode 100644 > >> meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl > >> create mode 100644 > >> meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl > >> create mode 100755 > >> meta/recipes-kernel/linux/files/debian/rules.tmpl ^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH L-C v2 0/7] linux-custom recipe rework 2019-11-05 13:55 [PATCH 0/7] linux-custom recipe rework Cedric Hombourger ` (6 preceding siblings ...) 2019-11-05 15:55 ` Henning Schild @ 2019-11-07 9:23 ` Cedric Hombourger 2019-11-07 9:23 ` [PATCH L-C v2 1/7] recipes-kernel/linux: make KERNEL_DEFCONFIG support in-tree defconfigs Cedric Hombourger ` (8 more replies) 7 siblings, 9 replies; 83+ messages in thread From: Cedric Hombourger @ 2019-11-07 9:23 UTC (permalink / raw) To: isar-users; +Cc: Cedric Hombourger The Linux kernel is packaged a bit differently between Debian variants and in particular for the perf package(s) in Debian vs Ubuntu. Since there is no way the kernel build scripts would ever get distro specific support for building more than the kernel and headers, we need our own build/packaging scripts. The approach is documented in the custom_kernel.md file included in this patch series. Version 2 of this changeset was tested against ci_build.sh and some images were booted on hardware available here. No regressions were detected (to date). This was also tested on the creator-ci40-marduk machine (subject of another patch series). Changes compared to v1: Add missing reference to KERNEL_DEFCONFIG in requirements list Amend postinst/postrm scripts according to CONFIG_BLK_DEV_INITRD Remove (unused) pre/post host hooks from debian/isar scripts Add preinst/prerm scripts shipped by the kernel Align {pre,post}{inst,rm} scripts with upstream Add ${BASE_DISTRO} to Build-Profiles Add nolibcdev to Build-Profiles if KERNEL_LIBC_DEV_DEPLOY != 1 Only build the libc-dev packages if it is needed Cedric Hombourger (7): recipes-kernel/linux: make KERNEL_DEFCONFIG support in-tree defconfigs linux-mainline: fix stripping of .0 from the kernel version linux-mainline: update from 4.19.0 to 4.19.80 linux-custom: add support for kernel config fragments linux-mainline: disable support for HFS to demonstrate use of config fragments buildchroot-host: install qemu-static to support hybrid cross-compiles linux-custom: rewrite to no longer depend on the kernel's builddeb doc/custom_kernel.md | 181 ++++++++++++++++ .../recipes-kernel/linux/files/no-fs-hfs.cfg | 2 + ...ne_4.19.0.bb => linux-mainline_4.19.80.bb} | 8 +- .../buildchroot/buildchroot-host.bb | 6 + .../linux/files/build-kernel.sh | 129 ------------ meta/recipes-kernel/linux/files/debian/compat | 1 + .../linux/files/debian/control.tmpl | 36 ++++ meta/recipes-kernel/linux/files/debian/files | 5 + .../linux/files/debian/isar/build.tmpl | 36 ++++ .../linux/files/debian/isar/clean.tmpl | 20 ++ .../linux/files/debian/isar/common.tmpl | 60 ++++++ .../linux/files/debian/isar/install.tmpl | 197 ++++++++++++++++++ .../files/debian/linux-image.postinst.tmpl | 22 ++ .../files/debian/linux-image.postrm.tmpl | 18 ++ .../linux/files/debian/rules.tmpl | 39 ++++ meta/recipes-kernel/linux/linux-custom.inc | 196 ++++++++++++----- 16 files changed, 772 insertions(+), 184 deletions(-) create mode 100644 doc/custom_kernel.md create mode 100644 meta-isar/recipes-kernel/linux/files/no-fs-hfs.cfg rename meta-isar/recipes-kernel/linux/{linux-mainline_4.19.0.bb => linux-mainline_4.19.80.bb} (67%) delete mode 100644 meta/recipes-kernel/linux/files/build-kernel.sh create mode 100644 meta/recipes-kernel/linux/files/debian/compat create mode 100644 meta/recipes-kernel/linux/files/debian/control.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/files create mode 100644 meta/recipes-kernel/linux/files/debian/isar/build.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/isar/clean.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/isar/common.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/isar/install.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl create mode 100755 meta/recipes-kernel/linux/files/debian/rules.tmpl -- 2.20.1 ^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH L-C v2 1/7] recipes-kernel/linux: make KERNEL_DEFCONFIG support in-tree defconfigs 2019-11-07 9:23 ` [PATCH L-C v2 " Cedric Hombourger @ 2019-11-07 9:23 ` Cedric Hombourger 2019-11-07 9:23 ` [PATCH L-C v2 2/7] linux-mainline: fix stripping of .0 from the kernel version Cedric Hombourger ` (7 subsequent siblings) 8 siblings, 0 replies; 83+ messages in thread From: Cedric Hombourger @ 2019-11-07 9:23 UTC (permalink / raw) To: isar-users; +Cc: Cedric Hombourger The recipe expected the specified KERNEL_DEFCONFIG to be provided as a SRC_URI while the kernel already provides many useful defconfigs. The recipe will now check if the specified configuration can be found in WORKDIR. If it is found, it will then be used via "make olddefconfig" as before. It will otherwise assume that it is an in-tree defconfig and attempt a "make foobar_defconfig" (where foobar_defconfig is the user-provided KERNEL_DEFCONFIG setting). To support this change, a KERNEL_CONFIG_TARGET variable is passed to the build-kernel.sh script via the environment. Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com> --- meta/recipes-kernel/linux/files/build-kernel.sh | 6 +----- meta/recipes-kernel/linux/linux-custom.inc | 9 ++++++++- 2 files changed, 9 insertions(+), 6 deletions(-) diff --git a/meta/recipes-kernel/linux/files/build-kernel.sh b/meta/recipes-kernel/linux/files/build-kernel.sh index b46cefa..386e58b 100644 --- a/meta/recipes-kernel/linux/files/build-kernel.sh +++ b/meta/recipes-kernel/linux/files/build-kernel.sh @@ -36,11 +36,7 @@ REPACK_DIR="$1/../repack" REPACK_LINUX_IMAGE_DIR="${REPACK_DIR}/linux-image" REPACK_LINUX_HEADERS_DIR="${REPACK_DIR}/linux-headers" -if [ -e .config ]; then - make olddefconfig -else - make defconfig -fi +make ${KERNEL_CONFIG_TARGET} || exit ${?} KV=$( make -s kernelrelease ) if [ "${KV}" != "${PV}" ]; then diff --git a/meta/recipes-kernel/linux/linux-custom.inc b/meta/recipes-kernel/linux/linux-custom.inc index b597e25..0c185d8 100644 --- a/meta/recipes-kernel/linux/linux-custom.inc +++ b/meta/recipes-kernel/linux/linux-custom.inc @@ -61,8 +61,14 @@ addtask devshell after do_install_builddeps dpkg_runbuild() { chmod +x ${WORKDIR}/build-kernel.sh + KERNEL_CONFIG_TARGET="${KERNEL_DEFCONFIG}" if [ -n "${KERNEL_DEFCONFIG}" ]; then - cp ${WORKDIR}/${KERNEL_DEFCONFIG} ${S}/.config + if [ -e "${WORKDIR}/${KERNEL_DEFCONFIG}" ]; then + cp ${WORKDIR}/${KERNEL_DEFCONFIG} ${S}/.config + KERNEL_CONFIG_TARGET="olddefconfig" + fi + else + KERNEL_CONFIG_TARGET="defconfig" fi E="${@ bb.utils.export_proxies(d)}" @@ -71,6 +77,7 @@ dpkg_runbuild() { export KERNEL_NAME=${KERNEL_NAME_PROVIDED} export KBUILD_DEPENDS="${KBUILD_DEPENDS}" + export KERNEL_CONFIG_TARGET export KERNEL_DEBIAN_DEPENDS="${KERNEL_DEBIAN_DEPENDS}" export KERNEL_FILE="${KERNEL_FILE}" export KERNEL_HEADERS_DEBIAN_DEPENDS="${KERNEL_HEADERS_DEBIAN_DEPENDS}" -- 2.20.1 ^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH L-C v2 2/7] linux-mainline: fix stripping of .0 from the kernel version 2019-11-07 9:23 ` [PATCH L-C v2 " Cedric Hombourger 2019-11-07 9:23 ` [PATCH L-C v2 1/7] recipes-kernel/linux: make KERNEL_DEFCONFIG support in-tree defconfigs Cedric Hombourger @ 2019-11-07 9:23 ` Cedric Hombourger 2019-11-07 9:23 ` [PATCH L-C v2 3/7] linux-mainline: update from 4.19.0 to 4.19.80 Cedric Hombourger ` (6 subsequent siblings) 8 siblings, 0 replies; 83+ messages in thread From: Cedric Hombourger @ 2019-11-07 9:23 UTC (permalink / raw) To: isar-users; +Cc: Cedric Hombourger The strip() function was used to strip the trailing .0 from the kernel version since it is omitted in the upstream archive file names. That function is actually taking a character set as argument not a substring. This was causing ARCHIVE_VERSION to expand to "4.19.8" when the user provided version was "4.19.80". The inline python code was changed to check if the version ends with ".0" and only then remove the last two characters. Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com> --- meta-isar/recipes-kernel/linux/linux-mainline_4.19.0.bb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/meta-isar/recipes-kernel/linux/linux-mainline_4.19.0.bb b/meta-isar/recipes-kernel/linux/linux-mainline_4.19.0.bb index e50e149..127f6e2 100644 --- a/meta-isar/recipes-kernel/linux/linux-mainline_4.19.0.bb +++ b/meta-isar/recipes-kernel/linux/linux-mainline_4.19.0.bb @@ -7,7 +7,7 @@ require recipes-kernel/linux/linux-custom.inc -ARCHIVE_VERSION = "${@d.getVar('PV').strip('.0')}" +ARCHIVE_VERSION = "${@ d.getVar('PV')[:-2] if d.getVar('PV').endswith('.0') else d.getVar('PV') }" SRC_URI += " \ https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-${ARCHIVE_VERSION}.tar.xz \ -- 2.20.1 ^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH L-C v2 3/7] linux-mainline: update from 4.19.0 to 4.19.80 2019-11-07 9:23 ` [PATCH L-C v2 " Cedric Hombourger 2019-11-07 9:23 ` [PATCH L-C v2 1/7] recipes-kernel/linux: make KERNEL_DEFCONFIG support in-tree defconfigs Cedric Hombourger 2019-11-07 9:23 ` [PATCH L-C v2 2/7] linux-mainline: fix stripping of .0 from the kernel version Cedric Hombourger @ 2019-11-07 9:23 ` Cedric Hombourger 2019-11-07 9:23 ` [PATCH L-C v2 4/7] linux-custom: add support for kernel config fragments Cedric Hombourger ` (5 subsequent siblings) 8 siblings, 0 replies; 83+ messages in thread From: Cedric Hombourger @ 2019-11-07 9:23 UTC (permalink / raw) To: isar-users; +Cc: Cedric Hombourger In preparation for a new machine addition, update the linux-mainline kernel from 4.19.0 to 4.19.80. This is needed to get some of the mips fixes that were integrated in this longterm kernel version (and in particular: MIPS: have "plain" make calls build dtbs for selected platforms, i.e. 637dfa0fad6d91a9a709dc70549a6d20fa77f615 upstream) Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com> --- .../{linux-mainline_4.19.0.bb => linux-mainline_4.19.80.bb} | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) rename meta-isar/recipes-kernel/linux/{linux-mainline_4.19.0.bb => linux-mainline_4.19.80.bb} (87%) diff --git a/meta-isar/recipes-kernel/linux/linux-mainline_4.19.0.bb b/meta-isar/recipes-kernel/linux/linux-mainline_4.19.80.bb similarity index 87% rename from meta-isar/recipes-kernel/linux/linux-mainline_4.19.0.bb rename to meta-isar/recipes-kernel/linux/linux-mainline_4.19.80.bb index 127f6e2..b41d1a8 100644 --- a/meta-isar/recipes-kernel/linux/linux-mainline_4.19.0.bb +++ b/meta-isar/recipes-kernel/linux/linux-mainline_4.19.80.bb @@ -12,7 +12,7 @@ ARCHIVE_VERSION = "${@ d.getVar('PV')[:-2] if d.getVar('PV').endswith('.0') else SRC_URI += " \ https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-${ARCHIVE_VERSION}.tar.xz \ file://x86_64_defconfig" -SRC_URI[sha256sum] = "0c68f5655528aed4f99dae71a5b259edc93239fa899e2df79c055275c21749a1" +SRC_URI[sha256sum] = "80a9ba764e088aa7fddfef5a97c0236905e291468a37832243b6f3828d36e7ec" SRC_URI_append_de0-nano-soc = " \ file://0001-ARM-dts-socfpga-Rename-socfpga_cyclone5_de0_-sockit-.patch" -- 2.20.1 ^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH L-C v2 4/7] linux-custom: add support for kernel config fragments 2019-11-07 9:23 ` [PATCH L-C v2 " Cedric Hombourger ` (2 preceding siblings ...) 2019-11-07 9:23 ` [PATCH L-C v2 3/7] linux-mainline: update from 4.19.0 to 4.19.80 Cedric Hombourger @ 2019-11-07 9:23 ` Cedric Hombourger 2019-11-07 9:23 ` [PATCH L-C v2 5/7] linux-mainline: disable support for HFS to demonstrate use of " Cedric Hombourger ` (4 subsequent siblings) 8 siblings, 0 replies; 83+ messages in thread From: Cedric Hombourger @ 2019-11-07 9:23 UTC (permalink / raw) To: isar-users; +Cc: Cedric Hombourger Check for .cfg files listed in SRC_URI and merge them with the main defconfig using the kernel merge_config script. The optional "apply" parameter may be used for the specified fragment to be ignore (set to "no"). Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com> --- .../linux/files/build-kernel.sh | 3 +++ meta/recipes-kernel/linux/linux-custom.inc | 23 +++++++++++++++++++ 2 files changed, 26 insertions(+) diff --git a/meta/recipes-kernel/linux/files/build-kernel.sh b/meta/recipes-kernel/linux/files/build-kernel.sh index 386e58b..f56e96c 100644 --- a/meta/recipes-kernel/linux/files/build-kernel.sh +++ b/meta/recipes-kernel/linux/files/build-kernel.sh @@ -37,6 +37,9 @@ REPACK_LINUX_IMAGE_DIR="${REPACK_DIR}/linux-image" REPACK_LINUX_HEADERS_DIR="${REPACK_DIR}/linux-headers" make ${KERNEL_CONFIG_TARGET} || exit ${?} +if [ -n "${KERNEL_FRAGMENTS}" ]; then + scripts/kconfig/merge_config.sh -m .config ${KERNEL_FRAGMENTS} +fi KV=$( make -s kernelrelease ) if [ "${KV}" != "${PV}" ]; then diff --git a/meta/recipes-kernel/linux/linux-custom.inc b/meta/recipes-kernel/linux/linux-custom.inc index 0c185d8..ca91f64 100644 --- a/meta/recipes-kernel/linux/linux-custom.inc +++ b/meta/recipes-kernel/linux/linux-custom.inc @@ -18,6 +18,19 @@ python() { 'linux-headers-' + kernel_name) } +def config_fragments(d): + fragments = [] + sources = d.getVar("SRC_URI").split() + for s in sources: + _, _, local, _, _, parm = bb.fetch.decodeurl(s) + apply = parm.get("apply") + if apply == "no": + continue + base, ext = os.path.splitext(os.path.basename(local)) + if ext and ext in (".cfg"): + fragments.append(local) + return fragments + inherit dpkg-base SRC_URI += "file://build-kernel.sh" @@ -71,6 +84,16 @@ dpkg_runbuild() { KERNEL_CONFIG_TARGET="defconfig" fi + # copy config fragments over to the kernel tree + src_frags="${@ " ".join(config_fragments(d)) }" + out_frags="${@ " ".join(map(lambda frag: 'debian/fragments/' + frag, config_fragments(d))) }" + rm -rf ${S}/debian/fragments + if [ -n "${src_frags}" ]; then + mkdir -p ${S}/debian/fragments + (cd ${WORKDIR} && cp ${src_frags} ${S}/debian/fragments/) + fi + export KERNEL_FRAGMENTS="${out_frags}" + E="${@ bb.utils.export_proxies(d)}" export PV=${PV} -- 2.20.1 ^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH L-C v2 5/7] linux-mainline: disable support for HFS to demonstrate use of config fragments 2019-11-07 9:23 ` [PATCH L-C v2 " Cedric Hombourger ` (3 preceding siblings ...) 2019-11-07 9:23 ` [PATCH L-C v2 4/7] linux-custom: add support for kernel config fragments Cedric Hombourger @ 2019-11-07 9:23 ` Cedric Hombourger 2019-11-07 9:23 ` [PATCH L-C v2 6/7] buildchroot-host: install qemu-static to support hybrid cross-compiles Cedric Hombourger ` (3 subsequent siblings) 8 siblings, 0 replies; 83+ messages in thread From: Cedric Hombourger @ 2019-11-07 9:23 UTC (permalink / raw) To: isar-users; +Cc: Cedric Hombourger Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com> --- meta-isar/recipes-kernel/linux/files/no-fs-hfs.cfg | 2 ++ meta-isar/recipes-kernel/linux/linux-mainline_4.19.80.bb | 4 +++- 2 files changed, 5 insertions(+), 1 deletion(-) create mode 100644 meta-isar/recipes-kernel/linux/files/no-fs-hfs.cfg diff --git a/meta-isar/recipes-kernel/linux/files/no-fs-hfs.cfg b/meta-isar/recipes-kernel/linux/files/no-fs-hfs.cfg new file mode 100644 index 0000000..5dbfadb --- /dev/null +++ b/meta-isar/recipes-kernel/linux/files/no-fs-hfs.cfg @@ -0,0 +1,2 @@ +# CONFIG_FS_HFS is not set +# CONFIG_HFSPLUS_FS is not set diff --git a/meta-isar/recipes-kernel/linux/linux-mainline_4.19.80.bb b/meta-isar/recipes-kernel/linux/linux-mainline_4.19.80.bb index b41d1a8..83038f0 100644 --- a/meta-isar/recipes-kernel/linux/linux-mainline_4.19.80.bb +++ b/meta-isar/recipes-kernel/linux/linux-mainline_4.19.80.bb @@ -11,7 +11,9 @@ ARCHIVE_VERSION = "${@ d.getVar('PV')[:-2] if d.getVar('PV').endswith('.0') else SRC_URI += " \ https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-${ARCHIVE_VERSION}.tar.xz \ - file://x86_64_defconfig" + file://x86_64_defconfig \ + file://no-fs-hfs.cfg" + SRC_URI[sha256sum] = "80a9ba764e088aa7fddfef5a97c0236905e291468a37832243b6f3828d36e7ec" SRC_URI_append_de0-nano-soc = " \ -- 2.20.1 ^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH L-C v2 6/7] buildchroot-host: install qemu-static to support hybrid cross-compiles 2019-11-07 9:23 ` [PATCH L-C v2 " Cedric Hombourger ` (4 preceding siblings ...) 2019-11-07 9:23 ` [PATCH L-C v2 5/7] linux-mainline: disable support for HFS to demonstrate use of " Cedric Hombourger @ 2019-11-07 9:23 ` Cedric Hombourger 2019-11-07 9:23 ` [PATCH L-C v2 7/7] linux-custom: rewrite to no longer depend on the kernel's builddeb Cedric Hombourger ` (2 subsequent siblings) 8 siblings, 0 replies; 83+ messages in thread From: Cedric Hombourger @ 2019-11-07 9:23 UTC (permalink / raw) To: isar-users; +Cc: Cedric Hombourger The Linux kernel has great support for cross-compiling the kernel image and modules. There is however no support/mechanism for cross-compiling the build tools from its "scripts" directory. While HOSTCC may be set to use our cross-compiler, the kernel build infrasture would then try to run foreign-arch binaries such as fixdep. The easiest and least intrusive way to support this is to enable execution of such binaries via binfmt/qemu like we do in the buildchroot-target environment. Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com> --- meta/recipes-devtools/buildchroot/buildchroot-host.bb | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/meta/recipes-devtools/buildchroot/buildchroot-host.bb b/meta/recipes-devtools/buildchroot/buildchroot-host.bb index 408ad39..2e76acb 100644 --- a/meta/recipes-devtools/buildchroot/buildchroot-host.bb +++ b/meta/recipes-devtools/buildchroot/buildchroot-host.bb @@ -15,3 +15,9 @@ BUILDCHROOT_PREINSTALL ?= " \ ${BUILDCHROOT_PREINSTALL_COMMON} \ libc6:${DISTRO_ARCH} \ crossbuild-essential-${DISTRO_ARCH}" + +buildchroot_install_files_append() { + if [ -e '/usr/bin/qemu-${QEMU_ARCH}-static' ]; then + sudo cp '/usr/bin/qemu-${QEMU_ARCH}-static' '${BUILDCHROOT_DIR}/usr/bin/qemu-${QEMU_ARCH}-static' + fi +} -- 2.20.1 ^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH L-C v2 7/7] linux-custom: rewrite to no longer depend on the kernel's builddeb 2019-11-07 9:23 ` [PATCH L-C v2 " Cedric Hombourger ` (5 preceding siblings ...) 2019-11-07 9:23 ` [PATCH L-C v2 6/7] buildchroot-host: install qemu-static to support hybrid cross-compiles Cedric Hombourger @ 2019-11-07 9:23 ` Cedric Hombourger 2019-11-07 11:50 ` Jan Kiszka 2019-11-09 8:51 ` Jan Kiszka 2019-11-07 11:39 ` [PATCH L-C v2 0/7] " Jan Kiszka 2019-11-08 7:07 ` [PATCH L-C v3 " Cedric Hombourger 8 siblings, 2 replies; 83+ messages in thread From: Cedric Hombourger @ 2019-11-07 9:23 UTC (permalink / raw) To: isar-users; +Cc: Cedric Hombourger Regain control over the packaging of the Linux kernel by providing our own debian recipes (debian,{control,rules} files and friends) instead of using the kernel's builddeb script. This will allow generation of packages for "perf" (not included in this changeset) with per distro scheme (Debian and Ubuntu have differences) and maintain compatibility with upstream packages generated from different source packages but dependent on a certain deployment scheme (e.g. linux-base for Debian) Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com> --- doc/custom_kernel.md | 181 ++++++++++++++++ .../linux/files/build-kernel.sh | 128 ----------- meta/recipes-kernel/linux/files/debian/compat | 1 + .../linux/files/debian/control.tmpl | 37 ++++ .../linux/files/debian/isar/build.tmpl | 36 ++++ .../linux/files/debian/isar/clean.tmpl | 20 ++ .../linux/files/debian/isar/common.tmpl | 48 +++++ .../linux/files/debian/isar/install.tmpl | 204 ++++++++++++++++++ .../files/debian/linux-image.postinst.tmpl | 29 +++ .../files/debian/linux-image.postrm.tmpl | 35 +++ .../files/debian/linux-image.preinst.tmpl | 25 +++ .../linux/files/debian/linux-image.prerm.tmpl | 21 ++ .../linux/files/debian/rules.tmpl | 41 ++++ meta/recipes-kernel/linux/linux-custom.inc | 179 ++++++++++----- 14 files changed, 800 insertions(+), 185 deletions(-) create mode 100644 doc/custom_kernel.md delete mode 100644 meta/recipes-kernel/linux/files/build-kernel.sh create mode 100644 meta/recipes-kernel/linux/files/debian/compat create mode 100644 meta/recipes-kernel/linux/files/debian/control.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/isar/build.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/isar/clean.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/isar/common.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/isar/install.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.preinst.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.prerm.tmpl create mode 100755 meta/recipes-kernel/linux/files/debian/rules.tmpl diff --git a/doc/custom_kernel.md b/doc/custom_kernel.md new file mode 100644 index 0000000..934325a --- /dev/null +++ b/doc/custom_kernel.md @@ -0,0 +1,181 @@ +# Custom kernel recipe for Isar + +## Contents + + - [Summary](#summary) + - [Proposal owners](#proposal-owners) + - [Current status](#current-status) + - [Detailed description](#detailed-description) + - [Benefit to Isar](#benefit-to-isar) + - [How can I help?](#how-can-i-help) + - [Scope](#scope) + - [How to test](#how-to-test) + - [Dependencies](#dependencies) + - [Documentation](#documentation) + - [Questions and answers](#questions-and-answers) + +## Summary + +Isar currently uses the Linux kernel's builddeb script to generate linux-image, +linux-headers and linux-libc-dev packages for your custom kernel sources. The +main benefit of this approach was that the low level details of how to package +the Linux kernel image, modules and headers were left in the kernel. There are +however some drawbacks: Isar had to repack generated packages to introduce some +of the changes it needs such as suffixing the kernel packages with the name of +the custom kernel (e.g. with "mainline" for the "linux-mainline" recipe), lack +of automatic dependencies (the builddeb script from the kernel does not use +debhelper), it was difficult to add packages especially distribution specific +packages (maintain version specific patch in Isar vs being able to upstream +distribution specific changes upstream), etc. This change is about being less +dependent on the kernel packaging scripts (which aren't used by major distros +anyway) and regain control on how we want or need the Linux kernel packaged. + +## Proposal owners + + * name: [Cedric Hombourger](https://github.com/chombourger) + +## Current status + +### Tests + +The following were recently checked: + + * Custom kernels in meta-isar build without (known) failures + * initrd image gets generated when the kernel is added to the rootfs + * `ISAR_CROSS_COMPILE` is supported + * `KERNEL_FILE` is honored + * `KERNEL_DEFCONFIG` may either be a file (specified via `SRC_URI`) or a + kernel-provided defconfig + * Config fragments get merged with the user-specified kernel configuration + * Path to installed DTBs may be configured (new) + * Configuration and build from a `devshell` + * Support for the `KERNEL_LIBC_DEV_DEPLOY` directive + +while the following needs work: + + * Check if custom-kernel builds are reproducible + +### Remaining work items + + * Check if there is a better way to cross compile kernel "scripts" such as + `fixdep` or `modpost` than manually compiling them (they are compiled for + the build machine (amd64) and not for the target (e.g. armhf) even though + we are cross-compiling - yet the linux-headers package should be shipping + "scripts" for the target machine) + +## Detailed description + +### Requirements + +#### Current + +Isar has had a `linux-custom` recipe for a long time and counts many users. It is therefore +paramount to maintain features that it has today with no or little changes to the APIs. The +following requirements were identified and will be maintained: + + 1. Sources to the custom Linux kernel may be specified via `SRC_URI` + + 2. Configure the kernel via an in-tree or an external `defconfig` via `KERNEL_DEFCONFIG` + + 3. Support kernel configuration tweaks via configuration fragments (`.cfg` files) + + 4. Patches to the linux kernel may be specified via `SRC_URI` + + 5. Ensure that the Isar recipe `PV` matches the kernel release identifier (`KERNEL_RELEASE`) + + 6. Produce a `linux-image` package that ships the kernel image and modules + + 7. Allow the name of the kernel image to be changed via `KERNEL_FILE` (defaults to `vmlinuz`) + + 8. Produce a `linux-headers` package which includes kernel headers and kbuild scripts/tools + + 9. The `linux-headers` package shall support native and cross compiles of out-of-tree kernel modules + + 10. Produce a `linux-libc-dev` package to support user-land builds + + 11. Only ship the `linux-libc-dev` package if instructed to (`KERNEL_LIBC_DEV_DEPLOY` equals to `"1"`) + + 12. Support both native and cross compiles (`ISAR_CROSS_COMPILE`) + + 13. Support for the following kernel architectures: + + * arm + * arm64 + * mips + * x86 + + 14. Support `devshell` (kernel configuration shall be applied) + +#### New requirements (proposed) + +This proposal includes the following new requirements: + + 1. Support creation of additional packages + + 2. Allow per-distro packaging tweaks + +### Approach + +The new implementation will have the `builddeb` code ported from the Linux kernel included to the +Isar environment. It will be refactor to support pre and post hooks. Hooks may be used by external +layers to extend the kernel packaging instructions provided by "stock" Isar or by Isar itself to +implement distro-specific packages or tweaks. + +Hooks need the ability to add `build` and `install` steps which can be achieved by having the main +`build` and `install` steps call pre and post scripts. This will however solve only one side of the +problem: new packages should be declared in the `debian/control` file. The packaging process +(initiated by `debian/rules`) will use a baseline `control` file when started and a mechanism will +be provided for hooks to append `control` blocks as they are processed. + +### Future work + +In the event where this proposal is accepted, here are a few future projects that are being +considered: + + 1. Package `perf` as Debian does + + 2. Add `ubuntu` as an Isar supported target distro and validate `linux-custom` recipe + +## Benefit to Isar + + * Control the kernel build and packaging process + + * No longer need to repack packages + + * Provide hooks for custom kernel recipes to add build/install directives + +## How can I help? + + * Check if you are able to build your own linux-custom recipes with this change + * Check if packages generated by this recipe are compatible with your Debian-based distro + * Review and provide comments on this changeset + * Create templates for distro-specific packages (e.g. perf) + +## Scope + + * Proposal owners: proposal owners are to communicate the idea to isar-users and make sure + that all of their requirements are met (to the extent they are already supported with the + current solution) or may be met in the future. + + * Other developers: port your out-of-tree linux-custom recipes to this new solution and + report issues. + +## How to test? + +The linux-mainline recipe may be used for some basic testing. This recipe is being used by the +following machines: + + * de0-nano-soc + * qemumipsel + +## Dependencies + + * None + +## Documentation + + * Document user-visible variables added by this proposal + +## Questions and answers + + * None diff --git a/meta/recipes-kernel/linux/files/build-kernel.sh b/meta/recipes-kernel/linux/files/build-kernel.sh deleted file mode 100644 index f56e96c..0000000 --- a/meta/recipes-kernel/linux/files/build-kernel.sh +++ /dev/null @@ -1,128 +0,0 @@ -#!/bin/bash -# -# Custom kernel build -# -# This software is a part of ISAR. -# Copyright (c) Siemens AG, 2018 -# -# SPDX-License-Identifier: MIT - -source /isar/common.sh - -host_arch=$(dpkg --print-architecture) - -if [ "$host_arch" != "$target_arch" ]; then - case $target_arch in - armhf) - export ARCH=arm - export CROSS_COMPILE="arm-linux-gnueabihf-" - ;; - arm64) - export ARCH=arm64 - export CROSS_COMPILE="aarch64-linux-gnu-" - ;; - mipsel) - export ARCH=mips - export CROSS_COMPILE="mipsel-linux-gnu-" - ;; - *) - echo "error: unsupported architecture ($target_arch)" - exit 1 - ;; - esac -fi - -REPACK_DIR="$1/../repack" -REPACK_LINUX_IMAGE_DIR="${REPACK_DIR}/linux-image" -REPACK_LINUX_HEADERS_DIR="${REPACK_DIR}/linux-headers" - -make ${KERNEL_CONFIG_TARGET} || exit ${?} -if [ -n "${KERNEL_FRAGMENTS}" ]; then - scripts/kconfig/merge_config.sh -m .config ${KERNEL_FRAGMENTS} -fi - -KV=$( make -s kernelrelease ) -if [ "${KV}" != "${PV}" ]; then - echo "ERROR: Recipe PV is \"${PV}\" but should be \"${KV}\"" 1>&2 - echo "ERROR: Probably due to CONFIG_LOCALVERSION" 1>&2 - exit 1 -fi - -rm -f .version -KBUILD_DEBARCH=$target_arch make -j $(($(nproc) * 2)) deb-pkg - -rm -rf "${REPACK_DIR}" -mkdir -p "${REPACK_DIR}" -mkdir -p "${REPACK_LINUX_IMAGE_DIR}" -mkdir -p "${REPACK_LINUX_HEADERS_DIR}" - -cp -a debian "${REPACK_DIR}" - -# dpkg-gencontrol performs cross-incompatible checks on the -# Architecture field; trick it to accept the control file -sed -i "s/Architecture: .*/Architecture: any/" "${REPACK_DIR}/debian/control" - -cd .. - -dpkg-deb -R linux-image-${PV}_${PV}-1_*.deb "${REPACK_LINUX_IMAGE_DIR}" -dpkg-deb -R linux-headers-${PV}_${PV}-1_*.deb "${REPACK_LINUX_HEADERS_DIR}" - -dpkg-gencontrol -crepack/debian/control \ - -lrepack/debian/changelog \ - -frepack/debian/files \ - -plinux-image-${PV} \ - -P"${REPACK_LINUX_IMAGE_DIR}" \ - -DPackage="linux-image-${KERNEL_NAME}" \ - -DSection=kernel \ - -DPriority=required \ - -DDepends="${KERNEL_DEBIAN_DEPENDS}" \ - -DArchitecture=$target_arch - -# Add Debian-like link installation to postinst -mkdir -p ${REPACK_LINUX_IMAGE_DIR}/lib/modules/${PV} -touch "${REPACK_LINUX_IMAGE_DIR}/lib/modules/${PV}/.fresh-install" -sed -i "${REPACK_LINUX_IMAGE_DIR}/DEBIAN/postinst" \ - -e "/^set -e$/a\\ -\\ -if [ -f /lib/modules/${PV}/.fresh-install ]; then\\ - change=install\\ -else\\ - change=upgrade\\ -fi\\ -linux-update-symlinks \$change ${PV} /boot/${KERNEL_FILE}-${PV}\\ -rm -f /lib/modules/${PV}/.fresh-install" - -# Add Debian-like link removal to postrm -sed -i "${REPACK_LINUX_IMAGE_DIR}/DEBIAN/postrm" \ - -e "/^set -e$/a\\ -\\ -rm -f /lib/modules/${PV}/.fresh-install\\ -\\ -if [ \"\$1\" != upgrade ] && command -v linux-update-symlinks >/dev/null; then\\ - linux-update-symlinks remove ${PV} /boot/${KERNEL_FILE}-${PV}\\ -fi" - -# Make sure arm64 kernels are decompressed -if [ "$target_arch" = "arm64" ]; then - kernel_file="${REPACK_LINUX_IMAGE_DIR}/boot/${KERNEL_FILE}-${PV}" - mv "${kernel_file}" "${kernel_file}.gz" - gunzip "${kernel_file}.gz" -fi - -dpkg-gencontrol -crepack/debian/control \ - -lrepack/debian/changelog \ - -frepack/debian/files \ - -plinux-headers-${PV} \ - -P"${REPACK_LINUX_HEADERS_DIR}" \ - -Vkernel:debarch="${KERNEL_NAME}" \ - -DPackage="linux-headers-${KERNEL_NAME}" \ - -DSection=kernel \ - -DDepends="${KERNEL_HEADERS_DEBIAN_DEPENDS}" \ - -DArchitecture=$target_arch - -fakeroot dpkg-deb -b "${REPACK_LINUX_IMAGE_DIR}" \ - linux-image-${KERNEL_NAME}_${PV}-1_${KERNEL_NAME}.deb -rm -f linux-image-${PV}_${PV}-1_*.deb -fakeroot dpkg-deb -b "${REPACK_LINUX_HEADERS_DIR}" \ - linux-headers-${KERNEL_NAME}_${PV}-1_${KERNEL_NAME}.deb -rm -f linux-headers-${PV}_${PV}-1_*.deb diff --git a/meta/recipes-kernel/linux/files/debian/compat b/meta/recipes-kernel/linux/files/debian/compat new file mode 100644 index 0000000..ec63514 --- /dev/null +++ b/meta/recipes-kernel/linux/files/debian/compat @@ -0,0 +1 @@ +9 diff --git a/meta/recipes-kernel/linux/files/debian/control.tmpl b/meta/recipes-kernel/linux/files/debian/control.tmpl new file mode 100644 index 0000000..d55096b --- /dev/null +++ b/meta/recipes-kernel/linux/files/debian/control.tmpl @@ -0,0 +1,37 @@ +Source: linux-${KERNEL_NAME_PROVIDED} +Section: kernel +Priority: optional +Maintainer: ${MAINTAINER} +Build-Depends: bc, kmod, cpio, ${KBUILD_DEPENDS} +Homepage: http://www.kernel.org/ + +Package: linux-image-${KERNEL_NAME_PROVIDED} +Architecture: any +Depends: ${KERNEL_DEBIAN_DEPENDS} +Description: ${KERNEL_NAME_PROVIDED} Linux kernel, version ${PV} + This package contains the Linux kernel, modules and corresponding other + files, version: ${PV}. + +Package: linux-headers-${KERNEL_NAME_PROVIDED} +Architecture: any +Depends: ${KERNEL_HEADERS_DEBIAN_DEPENDS}, ${perl:Depends}, ${shlib:Depends} +Description: ${KERNEL_NAME_PROVIDED} Linux kernel headers for ${PV} + This package provides kernel header files for ${PV} on ${DISTRO_ARCH} + . + This is useful for people who need to build external modules + +Package: linux-libc-dev +Build-Profiles: <!nolibcdev> +Section: devel +Provides: linux-kernel-headers +Architecture: any +Description: Linux support headers for userspace development + This package provides userspaces headers from the Linux kernel. These headers + are used by the installed headers for GNU glibc and other system libraries. + +Package: linux-image-${KERNEL_NAME_PROVIDED}-dbg +Section: debug +Architecture: any +Description: Linux kernel debugging symbols for ${PV} + This package will come in handy if you need to debug the kernel. It provides + all the necessary debug symbols for the kernel and its modules. diff --git a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl new file mode 100644 index 0000000..e4e75e3 --- /dev/null +++ b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl @@ -0,0 +1,36 @@ +#!/bin/bash +# Copyright (c) Mentor Graphics, a Siemens business, 2019 +# SPDX-License-Identifier: MIT + +# Load common stuff +. ${S}/debian/isar/common || exit ${?} + +do_build() { + + # Print a few things that are of particular interest + print_settings + + # Trace what we do here + set -x + + # Process existing kernel configuration to make sure it is complete + # (use defaults for options that were not specified) + ${MAKE} O=${KERNEL_BUILD_DIR} olddefconfig || exit ${?} + + # Build the Linux kernel + ${MAKE} O=${KERNEL_BUILD_DIR} -j $(nproc) || exit ${?} + + # Stop tracing + set +x +} + +print_settings() { + cat <<EOF +# Build settings: +# --------------- +# ARCH=${ARCH} +# CROSS_COMPILE=${CROSS_COMPILE} +EOF +} + +main build ${*} diff --git a/meta/recipes-kernel/linux/files/debian/isar/clean.tmpl b/meta/recipes-kernel/linux/files/debian/isar/clean.tmpl new file mode 100644 index 0000000..60046d2 --- /dev/null +++ b/meta/recipes-kernel/linux/files/debian/isar/clean.tmpl @@ -0,0 +1,20 @@ +#!/bin/bash +# Copyright (c) Mentor Graphics, a Siemens business, 2019 +# SPDX-License-Identifier: MIT + +# Load common stuff +. ${S}/debian/isar/common || exit ${?} + +do_clean() { + + # Trace what we do here + set -x + + rm -rf ${deb_img_dir} ${deb_dbg_dir} + rm -rf ${deb_kern_hdrdir} ${deb_libc_hdr_dir} + + # Stop tracing + set +x +} + +main clean ${*} diff --git a/meta/recipes-kernel/linux/files/debian/isar/common.tmpl b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl new file mode 100644 index 0000000..7a6b5dd --- /dev/null +++ b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl @@ -0,0 +1,48 @@ +#!/bin/bash +# Copyright (c) Mentor Graphics, a Siemens business, 2019 +# SPDX-License-Identifier: MIT + +# Isar settings +ARCH=${KERNEL_ARCH} +KERNEL_PKG_IMAGE=linux-image-${KERNEL_NAME_PROVIDED} +KERNEL_PKG_KERN_HEADERS=linux-headers-${KERNEL_NAME_PROVIDED} +KERNEL_PKG_LIBC_HEADERS=linux-libc-dev + +# Constants +KCONF=.config + +# Target directories +deb_top_dir=${S}/debian +deb_img_dir=${deb_top_dir}/${KERNEL_PKG_IMAGE} +deb_dbg_dir=${deb_img_dir}-dbg +deb_dtb_dir=${deb_img_dir}/${KERNEL_DTB_DIR} +deb_kern_hdr_dir=${deb_top_dir}/${KERNEL_PKG_KERN_HEADERS} +deb_libc_hdr_dir=${deb_top_dir}/${KERNEL_PKG_LIBC_HEADERS} + +main() { + target=${1} + + if [ ! -f ${S}/debian/isar/${target} ]; then + echo "error: ${target} is not a supported build target!" >&2 + return 1 + fi + + # create do_<x> variables for each package to be generated + for p in ${DEB_PACKAGES}; do + p="${p//-/_}" + eval do_${p}="1" + done + + # variables to be exported + export ARCH + + # are we cross-compiling? + BUILD_ARCH=$(dpkg-architecture -qDEB_BUILD_ARCH) + if [ "${BUILD_ARCH}" = "${DISTRO_ARCH}" ]; then + # no, make sure CROSS_COMPILE isn't set + unset CROSS_COMPILE + fi + + # call the actual target script + do_${target} || return ${?} +} diff --git a/meta/recipes-kernel/linux/files/debian/isar/install.tmpl b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl new file mode 100644 index 0000000..d586392 --- /dev/null +++ b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl @@ -0,0 +1,204 @@ +#!/bin/bash +# Copyright (c) Mentor Graphics, a Siemens business, 2019 +# SPDX-License-Identifier: MIT + +# Load common stuff +. ${S}/debian/isar/common || exit ${?} + +# Stop on error +set -e + +do_install() { + + # check if our kernel was configured + if [ ! -f "${O}/.config" ]; then + echo "error: kernel not configured!" >&2 + return 1 + fi + + # load its configuration + . ${O}/.config + + kimage="$(${MAKE} O=${O} -s --no-print-directory image_name)" + case "${ARCH}" in + mips|powerpc) kimage_path="boot/vmlinux-${PV}" ;; + um) kimage_path="usr/bin/vmlinux-${PV}" ;; + *) kimage_path="boot/vmlinuz-${PV}" ;; + esac + + print_settings + + # Trace what we do here + set -x + + install_image + install_hooks + install_dtbs + install_kmods + install_headers + + # Stop tracing + set +x +} + +print_settings() { + cat <<EOF +Install settings: +----------------- +deb_dtb_dir=${deb_dtb_dir} +deb_hdr_dir=${deb_hdr_dir} +kimage=${kimage} +kimage_path=${kimage_path} + +EOF +} + +install_image() { + install -m 755 -d ${deb_img_dir}/$(dirname ${kimage_path}) + cp ${O}/${kimage} ${deb_img_dir}/${kimage_path} + install_image_debug +} + +install_image_debug() { + # Different tools want the image in different locations + # perf + mkdir -p ${deb_dbg_dir}/usr/lib/debug/lib/modules/${PV}/ + cp ${O}/vmlinux ${deb_dbg_dir}/usr/lib/debug/lib/modules/${PV}/ + # systemtap + mkdir -p ${deb_dbg_dir}/usr/lib/debug/boot/ + ln -s ../lib/modules/$version/vmlinux ${deb_dbg_dir}/usr/lib/debug/boot/vmlinux-${PV} + # kdump-tools + ln -s lib/modules/${PV}/vmlinux ${deb_dbg_dir}/usr/lib/debug/vmlinux-${PV} +} + +install_hooks() { + install -m 755 -d ${deb_img_dir}/etc/kernel/install.d + install -m 755 -d ${deb_img_dir}/etc/kernel/postinst.d + install -m 755 -d ${deb_img_dir}/etc/kernel/postrm.d + install -m 755 -d ${deb_img_dir}/etc/kernel/prerm.d + + initrd="No" + [ -z "${CONFIG_BLK_DEV_INITRD}" ] || initrd="Yes" + + for script in postinst postrm preinst prerm; do + sed -i -e "s,INITRD=[A-Za-z0-9]*,INITRD=${initrd},g" ${S}/debian/linux-image*.${script} + done +} + +install_dtbs() { + [ -n "${CONFIG_OF}" ] || return 0 + ${MAKE} O=${O} INSTALL_DTBS_PATH=${deb_dtb_dir} dtbs_install +} + +install_kmods() { + [ -n "${CONFIG_MODULES}" ] || return 0 + ${MAKE} O=${O} INSTALL_MOD_PATH=${deb_img_dir} modules_install + touch ${deb_img_dir}/lib/modules/${PV}/.fresh-install + rm -fv ${deb_img_dir}/lib/modules/${PV}/build + rm -fv ${deb_img_dir}/lib/modules/${PV}/source + install_kmods_debug +} + +kmods_sign() { + [ -n "${CONFIG_MODULE_SIG_ALL}" ] || return 0 + ${MAKE} O=${O} INSTALL_MOD_PATH=${deb_img_dir} modules_sign +} + +install_kmods_debug() { + [ -n "${CONFIG_DEBUG_INFO}" ] || return 0 + + kmod_inst_dir=${deb_img_dir}/lib/modules + kmod_debug_dir=${deb_dbg_dir}/usr/lib/debug + + # copy kernels modules to usr/lib/debug + mkdir -p ${kmod_debug_dir} + tar -C ${kmod_inst_dir}/ -cO --exclude='modules.*' . | tar -C ${kmod_debug_dir}/ -xf - + # strip everything but debug sections for modules in usr/lib/debug + find ${kmod_debug_dir} -name *.ko -exec ${CROSS_COMPILE}objcopy --only-keep-debug {} \; + # and strip debug sections from modules in lib/modules + find ${kmod_inst_dir} -name *.ko -exec ${CROSS_COMPILE}objcopy --strip-debug {} \; + + # re-sign stripped kernel modules + kmods_sign +} + +headers_check() { + ${MAKE} O=${O} headers_check +} + +libc_headers() { + mkdir -p ${deb_libc_hdr_dir} + ${MAKE} O=${O} headers_install INSTALL_HDR_PATH=${deb_libc_hdr_dir}/usr + host_arch=$(dpkg-architecture -a${DISTRO_ARCH} -qDEB_HOST_MULTIARCH) + mkdir ${deb_libc_hdr_dir}/usr/include/${host_arch} + mv ${deb_libc_hdr_dir}/usr/include/asm ${deb_libc_hdr_dir}/usr/include/${host_arch}/ +} + +# the kernel does not seem to provide a way to cross-compile its tools, we however +# have qemu-user-static binaries in our buildchroot. Create a new build tree for the +# scripts and override HOSTCC to force a cross-compile +kernel_tools_build() { + odir="${S}/build-scripts" + mflags="HOSTCC=${CROSS_COMPILE}gcc O=${odir} NOSTDINC_FLAGS=" + rm -rf ${odir} && mkdir -p ${odir} && cp ${S}/${KERNEL_BUILD_DIR}/.config ${odir} + (unset LD_PRELOAD; cd ${odir} && make -C ${S} ${mflags} olddefconfig scripts) + (cd ${odir}; find scripts -type f -executable -exec file {} \;|grep ELF|cut -d: -f1) >${src_hdr_files} + tar -C ${odir} -cf - -T - <${src_hdr_files} | tar -C ${destdir} -xf - +} + +kernel_tools() { + # remove object files + find ${destdir}/scripts -type f -name '*.o' |xargs rm -f + + # we're all done if we aren't cross-compiling + [ -n "${CROSS_COMPILE}" ] || return 0 + + # remove ELF executables from the linux-headers package + find ${destdir}/scripts -type f -exec file {} \;|grep ELF|cut -d: -f1|xargs rm -fv + + # cross-compile kernel tools to be shipped with linux-headers + kernel_tools_build +} + +kernel_headers() { + destdir=${deb_kern_hdr_dir}/${KERNEL_HEADERS_DIR} + src_hdr_files=$(mktemp) + obj_hdr_files=$(mktemp) + + mkdir -p ${destdir} + mkdir -p ${deb_kern_hdr_dir}/lib/modules/${PV} + + (cd ${S}; find . -name 'Makefile*' -o -name 'Kconfig*' -o -name '*.pl') >>${src_hdr_files} + (cd ${S}; find arch/*/include include scripts -type f -o -type l) >>${src_hdr_files} + (cd ${S}; find arch/${ARCH} -name module.lds -o -name Kbuild.platforms -o -name Platform) >>${src_hdr_files} + (cd ${S}; find $(find arch/${ARCH} -name include -o -name scripts -type d) -type f) >>${src_hdr_files} + + (cd ${O}; find arch/${ARCH}/include Module.symvers include scripts -type f) >>${obj_hdr_files} + if [ -n "${CONFIG_STACK_VALIDATION}" ]; then + (cd ${O}; find tools/objtool -type f -executable) >>${obj_hdr_files} + fi + if [ -n "${CONFIG_GCC_PLUGINS}" ]; then + (cd ${O}; find scripts/gcc-plugins -name *.so -o -name gcc-common.h) >>${obj_hdr_files} + fi + + # deploy files that were matched above + tar -C ${S} -cf - -T - <${src_hdr_files} | tar -C ${destdir} -xf - + tar -C ${O} -cf - -T - <${obj_hdr_files} | tar -C ${destdir} -xf - + + # add the kernel config + cp ${O}/${KCONF} ${destdir}/.config + + # handle kernel development tools + kernel_tools + + # create symlinks + ln -sf /${KERNEL_HEADERS_DIR} ${deb_kern_hdr_dir}/lib/modules/${PV}/build +} + +install_headers() { + headers_check + [ -z "${do_linux_libc_headers}" ] || libc_headers + kernel_headers +} + +main install ${*} diff --git a/meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl b/meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl new file mode 100644 index 0000000..58fb4a1 --- /dev/null +++ b/meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl @@ -0,0 +1,29 @@ +#!/bin/sh +# based on https://salsa.debian.org/kernel-team/linux/blob/master/debian/templates/image.postinst.in + +# Tell initramfs builder whether it's wanted +export INITRD=Yes + +version=${PV} +image_path=/boot/${KERNEL_FILE}-${PV} + +if [ "$1" != configure ]; then + exit 0 +fi + +depmod $version + +if [ -f /lib/modules/$version/.fresh-install ]; then + change=install +else + change=upgrade +fi +linux-update-symlinks $change $version $image_path +rm -f /lib/modules/$version/.fresh-install + +if [ -d /etc/kernel/postinst.d ]; then + DEB_MAINT_PARAMS="$*" run-parts --report --exit-on-error --arg=$version \ + --arg=$image_path /etc/kernel/postinst.d +fi + +exit 0 diff --git a/meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl b/meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl new file mode 100644 index 0000000..56325cf --- /dev/null +++ b/meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl @@ -0,0 +1,35 @@ +#!/bin/sh -e +# based on https://salsa.debian.org/kernel-team/linux/blob/master/debian/templates/image.postrm.in + +# Tell initramfs builder whether it's wanted +export INITRD=Yes + +version=${PV} +image_path=/boot/${KERNEL_FILE}/${PV} + +rm -f /lib/modules/$version/.fresh-install + +if [ "$1" != upgrade ] && command -v linux-update-symlinks >/dev/null; then + linux-update-symlinks remove $version $image_path +fi + +if [ -d /etc/kernel/postrm.d ]; then + DEB_MAINT_PARAMS="$*" run-parts --report --exit-on-error --arg=$version \ + --arg=$image_path /etc/kernel/postrm.d +fi + +if [ "$1" = purge ]; then + for extra_file in modules.dep modules.isapnpmap modules.pcimap \ + modules.usbmap modules.parportmap \ + modules.generic_string modules.ieee1394map \ + modules.ieee1394map modules.pnpbiosmap \ + modules.alias modules.ccwmap modules.inputmap \ + modules.symbols modules.ofmap \ + modules.seriomap modules.\*.bin \ + modules.softdep modules.devname; do + eval rm -f /lib/modules/$version/$extra_file + done + rmdir /lib/modules/$version || true +fi + +exit 0 diff --git a/meta/recipes-kernel/linux/files/debian/linux-image.preinst.tmpl b/meta/recipes-kernel/linux/files/debian/linux-image.preinst.tmpl new file mode 100644 index 0000000..4440893 --- /dev/null +++ b/meta/recipes-kernel/linux/files/debian/linux-image.preinst.tmpl @@ -0,0 +1,25 @@ +#!/bin/sh -e +# based on https://salsa.debian.org/kernel-team/linux/blob/master/debian/templates/image.preinst.in + +# Tell initramfs builder whether it's wanted +export INITRD=Yes + +version=${PV} +image_path=/boot/${KERNEL_FILE}-${PV} + +if [ "$1" = abort-upgrade ]; then + exit 0 +fi + +if [ "$1" = install ]; then + # Create a flag file for postinst + mkdir -p /lib/modules/$version + touch /lib/modules/$version/.fresh-install +fi + +if [ -d /etc/kernel/preinst.d ]; then + DEB_MAINT_PARAMS="$*" run-parts --report --exit-on-error --arg=$version \ + --arg=$image_path /etc/kernel/preinst.d +fi + +exit 0 diff --git a/meta/recipes-kernel/linux/files/debian/linux-image.prerm.tmpl b/meta/recipes-kernel/linux/files/debian/linux-image.prerm.tmpl new file mode 100644 index 0000000..d5d5ccb --- /dev/null +++ b/meta/recipes-kernel/linux/files/debian/linux-image.prerm.tmpl @@ -0,0 +1,21 @@ +#!/bin/sh -e +# based on https://salsa.debian.org/kernel-team/linux/blob/master/debian/templates/image.prerm.in + +# Tell initramfs builder whether it's wanted +export INITRD=Yes + +version=${PV} +image_path=/boot/${KERNEL_FILE}-${PV} + +if [ "$1" != remove ]; then + exit 0 +fi + +linux-check-removal $version + +if [ -d /etc/kernel/prerm.d ]; then + DEB_MAINT_PARAMS="$*" run-parts --report --exit-on-error --arg=$version \ + --arg=$image_path /etc/kernel/prerm.d +fi + +exit 0 diff --git a/meta/recipes-kernel/linux/files/debian/rules.tmpl b/meta/recipes-kernel/linux/files/debian/rules.tmpl new file mode 100755 index 0000000..1728dda --- /dev/null +++ b/meta/recipes-kernel/linux/files/debian/rules.tmpl @@ -0,0 +1,41 @@ +#!/usr/bin/make -f + +export CROSS_COMPILE:=$(DEB_HOST_GNU_TYPE)- +export DEB_PACKAGES:=$(shell dh_listpackages) + +O:=$(CURDIR)/${KERNEL_BUILD_DIR} +S:=$(CURDIR) +deb_top_dir:=$(S)/debian + +DH_VERBOSE=1 + +# Dynamic variables to be passed to Isar build scripts +isar_env=$(strip \ + export CROSS_COMPILE='$(CROSS_COMPILE)' && \ + export DEB_PACKAGES='$(DEB_PACKAGES)' && \ + export MAKE='$(MAKE)' && \ + export O='${O}' \ + export S='${S}' \ +) + +%: + dh $(@) + +.PHONY: override_dh_auto_clean +override_dh_auto_clean: + $(isar_env) && bash $(deb_top_dir)/isar/clean + +.PHONY: override_dh_auto_build +override_dh_auto_build: + $(isar_env) && bash $(deb_top_dir)/isar/build + +override_dh_auto_install: + $(isar_env) && bash $(deb_top_dir)/isar/install + +.PHONY: override_dh_auto_test +override_dh_auto_test: + true + +PHONY: override_dh_strip +override_dh_strip: + dh_strip -Xvmlinux --no-automatic-dbgsym diff --git a/meta/recipes-kernel/linux/linux-custom.inc b/meta/recipes-kernel/linux/linux-custom.inc index ca91f64..00e90d0 100644 --- a/meta/recipes-kernel/linux/linux-custom.inc +++ b/meta/recipes-kernel/linux/linux-custom.inc @@ -2,22 +2,106 @@ # # This software is a part of ISAR. # Copyright (c) Siemens AG, 2018 +# Copyright (c) Mentor Graphics, a Siemens business, 2019 # # SPDX-License-Identifier: MIT -FILESEXTRAPATHS_prepend := "${FILE_DIRNAME}/files:" +# Settings that would typically be done from the custom kernel recipe +# ------------------------------------------------------------------- DESCRIPTION ?= "Custom kernel" +MAINTAINER ?= "isar-users <isar-users@googlegroups.com>" + +KBUILD_DEPENDS ?= "build-essential:native, \ + libelf-dev:native, \ + libncurses-dev:native, \ + libssl-dev:native, \ + bc, \ + bison, \ + cpio, \ + flex, \ + git, \ + kmod, \ + libssl-dev," + +KERNEL_DEBIAN_DEPENDS ?= "initramfs-tools | linux-initramfs-tool, \ + kmod, \ + linux-base (>= 4.3~)," + +KERNEL_HEADERS_DEBIAN_DEPENDS ?= "libc6, \ + libssl1.1," + +KERNEL_LIBC_DEV_DEPLOY ?= "0" + +# Settings that may be changed on a per distro, machine or layer basis +# -------------------------------------------------------------------- + +KERNEL_DTB_DIR ?= "usr/lib/linux-${KERNEL_NAME_PROVIDED}-${PV}" +KERNEL_HEADERS_DIR ?= "usr/src/linux-headers-${PV}" -KERNEL_NAME_PROVIDED ?= "${@ d.getVar('PN', True).partition('linux-')[2]}" KERNEL_DEFCONFIG ?= "" +# Add our template meta-data to the sources +FILESEXTRAPATHS_prepend := "${FILE_DIRNAME}/files:" +SRC_URI += "file://debian" + +# Variables and files that make our templates +# ------------------------------------------- + +TEMPLATE_FILES += " \ + debian/control.tmpl \ + debian/isar/build.tmpl \ + debian/isar/clean.tmpl \ + debian/isar/common.tmpl \ + debian/isar/install.tmpl \ + debian/linux-image.postinst.tmpl \ + debian/linux-image.postrm.tmpl \ + debian/linux-image.preinst.tmpl \ + debian/linux-image.prerm.tmpl \ + debian/rules.tmpl \ +" + +TEMPLATE_VARS += " \ + KBUILD_DEPENDS \ + KERNEL_ARCH \ + KERNEL_DEBIAN_DEPENDS \ + KERNEL_BUILD_DIR \ + KERNEL_DTB_DIR \ + KERNEL_FILE \ + KERNEL_HEADERS_DEBIAN_DEPENDS \ + KERNEL_HEADERS_DIR \ + KERNEL_NAME_PROVIDED \ +" + +inherit dpkg +inherit template + +# Derive name of the kernel packages from the name of this recipe +KERNEL_NAME_PROVIDED ?= "${@ d.getVar('PN', True).partition('linux-')[2]}" + +# Make bitbake know we will be producing linux-image and linux-headers packages python() { kernel_name = d.getVar("KERNEL_NAME_PROVIDED", True) d.setVar('PROVIDES', 'linux-image-' + kernel_name + ' ' + \ 'linux-headers-' + kernel_name) } +def get_kernel_arch(d): + distro_arch = d.getVar("DISTRO_ARCH") + if distro_arch == "amd64": + kernel_arch = "x86" + elif distro_arch == "arm64": + kernel_arch = "arm64" + elif distro_arch == "armhf": + kernel_arch = "arm" + elif distro_arch == "mipsel": + kernel_arch = "mips" + else: + kernel_arch = "" + return kernel_arch + +KERNEL_ARCH ??= "${@get_kernel_arch(d)}" + def config_fragments(d): fragments = [] sources = d.getVar("SRC_URI").split() @@ -31,57 +115,37 @@ def config_fragments(d): fragments.append(local) return fragments -inherit dpkg-base - -SRC_URI += "file://build-kernel.sh" +do_prepare_build_prepend() { + # copy meta-data over to source tree + rm -rf ${S}/debian + cp -r ${WORKDIR}/debian ${S}/ -KBUILD_DEPENDS ?= " \ - build-essential:native \ - libssl-dev \ - libelf-dev \ - bc \ - git \ - kmod \ - bison \ - flex \ - cpio \ - libncurses-dev" -KERNEL_DEBIAN_DEPENDS ?= "initramfs-tools | linux-initramfs-tool, kmod, linux-base (>= 4.3~)" -KERNEL_HEADERS_DEBIAN_DEPENDS ?= "libc6, libssl1.1" + # remove templates from the source tree + find ${S}/debian -name *.tmpl | xargs rm -f -KERNEL_LIBC_DEV_DEPLOY ?= "0" + # rename install/remove hooks to match user-specified name for our linux-image package + mv ${S}/debian/linux-image.postinst ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.postinst + mv ${S}/debian/linux-image.postrm ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.postrm + mv ${S}/debian/linux-image.preinst ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.preinst + mv ${S}/debian/linux-image.prerm ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.prerm -do_install_builddeps() { - dpkg_do_mounts - E="${@ bb.utils.export_proxies(d)}" - sudo -E chroot ${BUILDCHROOT_DIR} \ - apt-get update \ - -o Dir::Etc::SourceList="sources.list.d/isar-apt.list" \ - -o Dir::Etc::SourceParts="-" \ - -o APT::Get::List-Cleanup="0" - sudo -E chroot ${BUILDCHROOT_DIR} \ - apt-get install \ - -y -o Debug::pkgProblemResolver=yes \ - --no-install-recommends ${KBUILD_DEPENDS} - dpkg_undo_mounts + # produce a changelog for our kernel build + deb_add_changelog } -addtask install_builddeps after do_prepare_build before do_dpkg_build -# apt and reprepro may not run in parallel, acquire the Isar lock -do_install_builddeps[lockfiles] += "${REPO_ISAR_DIR}/isar.lock" - -addtask devshell after do_install_builddeps +# build directory for our "full" kernel build +KERNEL_BUILD_DIR = "build-full" -dpkg_runbuild() { - chmod +x ${WORKDIR}/build-kernel.sh - KERNEL_CONFIG_TARGET="${KERNEL_DEFCONFIG}" +dpkg_configure_kernel() { + config_target="${KERNEL_DEFCONFIG}" + rm -rf ${S}/${KERNEL_BUILD_DIR} && mkdir -p ${S}/${KERNEL_BUILD_DIR} if [ -n "${KERNEL_DEFCONFIG}" ]; then if [ -e "${WORKDIR}/${KERNEL_DEFCONFIG}" ]; then - cp ${WORKDIR}/${KERNEL_DEFCONFIG} ${S}/.config - KERNEL_CONFIG_TARGET="olddefconfig" + cp ${WORKDIR}/${KERNEL_DEFCONFIG} ${S}/${KERNEL_BUILD_DIR}/.config + config_target="olddefconfig" fi else - KERNEL_CONFIG_TARGET="defconfig" + config_target="defconfig" fi # copy config fragments over to the kernel tree @@ -92,22 +156,23 @@ dpkg_runbuild() { mkdir -p ${S}/debian/fragments (cd ${WORKDIR} && cp ${src_frags} ${S}/debian/fragments/) fi - export KERNEL_FRAGMENTS="${out_frags}" - E="${@ bb.utils.export_proxies(d)}" - - export PV=${PV} - export KERNEL_NAME=${KERNEL_NAME_PROVIDED} - - export KBUILD_DEPENDS="${KBUILD_DEPENDS}" - export KERNEL_CONFIG_TARGET - export KERNEL_DEBIAN_DEPENDS="${KERNEL_DEBIAN_DEPENDS}" - export KERNEL_FILE="${KERNEL_FILE}" - export KERNEL_HEADERS_DEBIAN_DEPENDS="${KERNEL_HEADERS_DEBIAN_DEPENDS}" - - sudo -E chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} ${PP}/build-kernel.sh ${PP}/${PPS} ${DISTRO_ARCH} + sudo -E chroot --userspec=$(id -u):$(id -g) ${BUILDCHROOT_DIR} sh -c " \ + export ARCH=${KERNEL_ARCH} && \ + cd ${PP}/${PPS} && \ + make O=${KERNEL_BUILD_DIR} ${config_target} && \ + ./scripts/kconfig/merge_config.sh \ + -O ${KERNEL_BUILD_DIR}/ \ + ${KERNEL_BUILD_DIR}/.config \ + ${out_frags} \ + " +} +dpkg_runbuild_prepend() { + profiles="${BASE_DISTRO}" if [ "${KERNEL_LIBC_DEV_DEPLOY}" != "1" ]; then - rm -f ${WORKDIR}/linux-libc-dev_${PV}*.deb + profiles="${profiles} nolibcdev" fi + export DEB_BUILD_PROFILES="${profiles}" + dpkg_configure_kernel } -- 2.20.1 ^ permalink raw reply [flat|nested] 83+ messages in thread
* Re: [PATCH L-C v2 7/7] linux-custom: rewrite to no longer depend on the kernel's builddeb 2019-11-07 9:23 ` [PATCH L-C v2 7/7] linux-custom: rewrite to no longer depend on the kernel's builddeb Cedric Hombourger @ 2019-11-07 11:50 ` Jan Kiszka 2019-11-07 11:52 ` Jan Kiszka 2019-11-07 11:53 ` Cedric Hombourger 2019-11-09 8:51 ` Jan Kiszka 1 sibling, 2 replies; 83+ messages in thread From: Jan Kiszka @ 2019-11-07 11:50 UTC (permalink / raw) To: Cedric Hombourger, isar-users On 07.11.19 10:23, Cedric Hombourger wrote: > Regain control over the packaging of the Linux kernel by providing our > own debian recipes (debian,{control,rules} files and friends) instead > of using the kernel's builddeb script. This will allow generation of > packages for "perf" (not included in this changeset) with per distro > scheme (Debian and Ubuntu have differences) and maintain compatibility > with upstream packages generated from different source packages but > dependent on a certain deployment scheme (e.g. linux-base for Debian) > > Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com> > --- > doc/custom_kernel.md | 181 ++++++++++++++++ This document still makes no sense to me. I would expect a user document, not an "approach proposal". If you patches are merged, it's no longer a proposal. /me no goes testing.... Jan -- Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 83+ messages in thread
* Re: [PATCH L-C v2 7/7] linux-custom: rewrite to no longer depend on the kernel's builddeb 2019-11-07 11:50 ` Jan Kiszka @ 2019-11-07 11:52 ` Jan Kiszka 2019-11-07 11:53 ` Cedric Hombourger 1 sibling, 0 replies; 83+ messages in thread From: Jan Kiszka @ 2019-11-07 11:52 UTC (permalink / raw) To: Cedric Hombourger, isar-users On 07.11.19 12:50, [ext] Jan Kiszka wrote: > On 07.11.19 10:23, Cedric Hombourger wrote: >> Regain control over the packaging of the Linux kernel by providing our >> own debian recipes (debian,{control,rules} files and friends) instead >> of using the kernel's builddeb script. This will allow generation of >> packages for "perf" (not included in this changeset) with per distro >> scheme (Debian and Ubuntu have differences) and maintain compatibility >> with upstream packages generated from different source packages but >> dependent on a certain deployment scheme (e.g. linux-base for Debian) >> >> Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com> >> --- >> doc/custom_kernel.md | 181 ++++++++++++++++ > > This document still makes no sense to me. I would expect a user > document, not an "approach proposal". If you patches are merged, it's no > longer a proposal. > > /me no goes testing.... "now" I mean. Integration also available via https://github.com/siemens/isar/commits/jan/next. Jan -- Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 83+ messages in thread
* Re: [PATCH L-C v2 7/7] linux-custom: rewrite to no longer depend on the kernel's builddeb 2019-11-07 11:50 ` Jan Kiszka 2019-11-07 11:52 ` Jan Kiszka @ 2019-11-07 11:53 ` Cedric Hombourger 2019-11-07 12:14 ` Jan Kiszka 1 sibling, 1 reply; 83+ messages in thread From: Cedric Hombourger @ 2019-11-07 11:53 UTC (permalink / raw) To: Jan Kiszka, isar-users On 11/7/19 12:50 PM, Jan Kiszka wrote: > On 07.11.19 10:23, Cedric Hombourger wrote: >> Regain control over the packaging of the Linux kernel by providing our >> own debian recipes (debian,{control,rules} files and friends) instead >> of using the kernel's builddeb script. This will allow generation of >> packages for "perf" (not included in this changeset) with per distro >> scheme (Debian and Ubuntu have differences) and maintain compatibility >> with upstream packages generated from different source packages but >> dependent on a certain deployment scheme (e.g. linux-base for Debian) >> >> Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com> >> --- >> doc/custom_kernel.md | 181 ++++++++++++++++ > This document still makes no sense to me. I would expect a user > document, not an "approach proposal". If you patches are merged, it's no > longer a proposal. well I think more work is needed to make the l-c recipe match features we can find in OE/YP for instance: (1) make it easy to add .dts files in SRC_URI and have them copied to ${S}/arch/${ARCH}/boot/dts and compiled (2) handling of distro specifics (e.g. perf for debian vs ubuntu) I would recommend that we turn it into a user document once we are reasonably happy with the state of l-c > /me no goes testing.... > > Jan > ^ permalink raw reply [flat|nested] 83+ messages in thread
* Re: [PATCH L-C v2 7/7] linux-custom: rewrite to no longer depend on the kernel's builddeb 2019-11-07 11:53 ` Cedric Hombourger @ 2019-11-07 12:14 ` Jan Kiszka 0 siblings, 0 replies; 83+ messages in thread From: Jan Kiszka @ 2019-11-07 12:14 UTC (permalink / raw) To: Cedric Hombourger, isar-users On 07.11.19 12:53, Cedric Hombourger wrote: > > On 11/7/19 12:50 PM, Jan Kiszka wrote: >> On 07.11.19 10:23, Cedric Hombourger wrote: >>> Regain control over the packaging of the Linux kernel by providing our >>> own debian recipes (debian,{control,rules} files and friends) instead >>> of using the kernel's builddeb script. This will allow generation of >>> packages for "perf" (not included in this changeset) with per distro >>> scheme (Debian and Ubuntu have differences) and maintain compatibility >>> with upstream packages generated from different source packages but >>> dependent on a certain deployment scheme (e.g. linux-base for Debian) >>> >>> Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com> >>> --- >>> doc/custom_kernel.md | 181 ++++++++++++++++ >> This document still makes no sense to me. I would expect a user >> document, not an "approach proposal". If you patches are merged, it's no >> longer a proposal. > > well I think more work is needed to make the l-c recipe match features > we can find in OE/YP > > for instance: (1) make it easy to add .dts files in SRC_URI and have > them copied to ${S}/arch/${ARCH}/boot/dts and compiled (2) handling of > distro specifics (e.g. perf for debian vs ubuntu) Nice-to-have, not mandatory to move forward. And dts files are trivially included by just copying them into the source tree (if you dislike the proper approach of patching). > > I would recommend that we turn it into a user document once we are > reasonably happy with the state of l-c Still makes no sense. The document reads as if what things still need to be implemented, but only the future section is future once the patches are in. Please fix. Jan -- Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 83+ messages in thread
* Re: [PATCH L-C v2 7/7] linux-custom: rewrite to no longer depend on the kernel's builddeb 2019-11-07 9:23 ` [PATCH L-C v2 7/7] linux-custom: rewrite to no longer depend on the kernel's builddeb Cedric Hombourger 2019-11-07 11:50 ` Jan Kiszka @ 2019-11-09 8:51 ` Jan Kiszka 2019-11-09 13:43 ` Cedric Hombourger 1 sibling, 1 reply; 83+ messages in thread From: Jan Kiszka @ 2019-11-09 8:51 UTC (permalink / raw) To: Cedric Hombourger, isar-users On 07.11.19 10:23, Cedric Hombourger wrote: > Regain control over the packaging of the Linux kernel by providing our > own debian recipes (debian,{control,rules} files and friends) instead > of using the kernel's builddeb script. This will allow generation of > packages for "perf" (not included in this changeset) with per distro > scheme (Debian and Ubuntu have differences) and maintain compatibility > with upstream packages generated from different source packages but > dependent on a certain deployment scheme (e.g. linux-base for Debian) > > Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com> > --- > doc/custom_kernel.md | 181 ++++++++++++++++ > .../linux/files/build-kernel.sh | 128 ----------- > meta/recipes-kernel/linux/files/debian/compat | 1 + > .../linux/files/debian/control.tmpl | 37 ++++ > .../linux/files/debian/isar/build.tmpl | 36 ++++ > .../linux/files/debian/isar/clean.tmpl | 20 ++ > .../linux/files/debian/isar/common.tmpl | 48 +++++ > .../linux/files/debian/isar/install.tmpl | 204 ++++++++++++++++++ > .../files/debian/linux-image.postinst.tmpl | 29 +++ > .../files/debian/linux-image.postrm.tmpl | 35 +++ > .../files/debian/linux-image.preinst.tmpl | 25 +++ > .../linux/files/debian/linux-image.prerm.tmpl | 21 ++ > .../linux/files/debian/rules.tmpl | 41 ++++ > meta/recipes-kernel/linux/linux-custom.inc | 179 ++++++++++----- > 14 files changed, 800 insertions(+), 185 deletions(-) > create mode 100644 doc/custom_kernel.md > delete mode 100644 meta/recipes-kernel/linux/files/build-kernel.sh > create mode 100644 meta/recipes-kernel/linux/files/debian/compat > create mode 100644 meta/recipes-kernel/linux/files/debian/control.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/isar/build.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/isar/clean.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/isar/common.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/isar/install.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.preinst.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.prerm.tmpl > create mode 100755 meta/recipes-kernel/linux/files/debian/rules.tmpl > > diff --git a/doc/custom_kernel.md b/doc/custom_kernel.md > new file mode 100644 > index 0000000..934325a > --- /dev/null > +++ b/doc/custom_kernel.md > @@ -0,0 +1,181 @@ > +# Custom kernel recipe for Isar > + > +## Contents > + > + - [Summary](#summary) > + - [Proposal owners](#proposal-owners) > + - [Current status](#current-status) > + - [Detailed description](#detailed-description) > + - [Benefit to Isar](#benefit-to-isar) > + - [How can I help?](#how-can-i-help) > + - [Scope](#scope) > + - [How to test](#how-to-test) > + - [Dependencies](#dependencies) > + - [Documentation](#documentation) > + - [Questions and answers](#questions-and-answers) > + > +## Summary > + > +Isar currently uses the Linux kernel's builddeb script to generate linux-image, > +linux-headers and linux-libc-dev packages for your custom kernel sources. The > +main benefit of this approach was that the low level details of how to package > +the Linux kernel image, modules and headers were left in the kernel. There are > +however some drawbacks: Isar had to repack generated packages to introduce some > +of the changes it needs such as suffixing the kernel packages with the name of > +the custom kernel (e.g. with "mainline" for the "linux-mainline" recipe), lack > +of automatic dependencies (the builddeb script from the kernel does not use > +debhelper), it was difficult to add packages especially distribution specific > +packages (maintain version specific patch in Isar vs being able to upstream > +distribution specific changes upstream), etc. This change is about being less > +dependent on the kernel packaging scripts (which aren't used by major distros > +anyway) and regain control on how we want or need the Linux kernel packaged. > + > +## Proposal owners > + > + * name: [Cedric Hombourger](https://github.com/chombourger) > + > +## Current status > + > +### Tests > + > +The following were recently checked: > + > + * Custom kernels in meta-isar build without (known) failures > + * initrd image gets generated when the kernel is added to the rootfs > + * `ISAR_CROSS_COMPILE` is supported > + * `KERNEL_FILE` is honored > + * `KERNEL_DEFCONFIG` may either be a file (specified via `SRC_URI`) or a > + kernel-provided defconfig > + * Config fragments get merged with the user-specified kernel configuration > + * Path to installed DTBs may be configured (new) > + * Configuration and build from a `devshell` > + * Support for the `KERNEL_LIBC_DEV_DEPLOY` directive > + > +while the following needs work: > + > + * Check if custom-kernel builds are reproducible > + > +### Remaining work items > + > + * Check if there is a better way to cross compile kernel "scripts" such as > + `fixdep` or `modpost` than manually compiling them (they are compiled for > + the build machine (amd64) and not for the target (e.g. armhf) even though > + we are cross-compiling - yet the linux-headers package should be shipping > + "scripts" for the target machine) > + > +## Detailed description > + > +### Requirements > + > +#### Current > + > +Isar has had a `linux-custom` recipe for a long time and counts many users. It is therefore > +paramount to maintain features that it has today with no or little changes to the APIs. The > +following requirements were identified and will be maintained: > + > + 1. Sources to the custom Linux kernel may be specified via `SRC_URI` > + > + 2. Configure the kernel via an in-tree or an external `defconfig` via `KERNEL_DEFCONFIG` > + > + 3. Support kernel configuration tweaks via configuration fragments (`.cfg` files) > + > + 4. Patches to the linux kernel may be specified via `SRC_URI` > + > + 5. Ensure that the Isar recipe `PV` matches the kernel release identifier (`KERNEL_RELEASE`) > + > + 6. Produce a `linux-image` package that ships the kernel image and modules > + > + 7. Allow the name of the kernel image to be changed via `KERNEL_FILE` (defaults to `vmlinuz`) > + > + 8. Produce a `linux-headers` package which includes kernel headers and kbuild scripts/tools > + > + 9. The `linux-headers` package shall support native and cross compiles of out-of-tree kernel modules > + > + 10. Produce a `linux-libc-dev` package to support user-land builds > + > + 11. Only ship the `linux-libc-dev` package if instructed to (`KERNEL_LIBC_DEV_DEPLOY` equals to `"1"`) > + > + 12. Support both native and cross compiles (`ISAR_CROSS_COMPILE`) > + > + 13. Support for the following kernel architectures: > + > + * arm > + * arm64 > + * mips > + * x86 > + > + 14. Support `devshell` (kernel configuration shall be applied) > + > +#### New requirements (proposed) > + > +This proposal includes the following new requirements: > + > + 1. Support creation of additional packages > + > + 2. Allow per-distro packaging tweaks > + > +### Approach > + > +The new implementation will have the `builddeb` code ported from the Linux kernel included to the > +Isar environment. It will be refactor to support pre and post hooks. Hooks may be used by external > +layers to extend the kernel packaging instructions provided by "stock" Isar or by Isar itself to > +implement distro-specific packages or tweaks. > + > +Hooks need the ability to add `build` and `install` steps which can be achieved by having the main > +`build` and `install` steps call pre and post scripts. This will however solve only one side of the > +problem: new packages should be declared in the `debian/control` file. The packaging process > +(initiated by `debian/rules`) will use a baseline `control` file when started and a mechanism will > +be provided for hooks to append `control` blocks as they are processed. > + > +### Future work > + > +In the event where this proposal is accepted, here are a few future projects that are being > +considered: > + > + 1. Package `perf` as Debian does > + > + 2. Add `ubuntu` as an Isar supported target distro and validate `linux-custom` recipe > + > +## Benefit to Isar > + > + * Control the kernel build and packaging process > + > + * No longer need to repack packages > + > + * Provide hooks for custom kernel recipes to add build/install directives > + > +## How can I help? > + > + * Check if you are able to build your own linux-custom recipes with this change > + * Check if packages generated by this recipe are compatible with your Debian-based distro > + * Review and provide comments on this changeset > + * Create templates for distro-specific packages (e.g. perf) > + > +## Scope > + > + * Proposal owners: proposal owners are to communicate the idea to isar-users and make sure > + that all of their requirements are met (to the extent they are already supported with the > + current solution) or may be met in the future. > + > + * Other developers: port your out-of-tree linux-custom recipes to this new solution and > + report issues. > + > +## How to test? > + > +The linux-mainline recipe may be used for some basic testing. This recipe is being used by the > +following machines: > + > + * de0-nano-soc > + * qemumipsel > + > +## Dependencies > + > + * None > + > +## Documentation > + > + * Document user-visible variables added by this proposal > + > +## Questions and answers > + > + * None > diff --git a/meta/recipes-kernel/linux/files/build-kernel.sh b/meta/recipes-kernel/linux/files/build-kernel.sh > deleted file mode 100644 > index f56e96c..0000000 > --- a/meta/recipes-kernel/linux/files/build-kernel.sh > +++ /dev/null > @@ -1,128 +0,0 @@ > -#!/bin/bash > -# > -# Custom kernel build > -# > -# This software is a part of ISAR. > -# Copyright (c) Siemens AG, 2018 > -# > -# SPDX-License-Identifier: MIT > - > -source /isar/common.sh > - > -host_arch=$(dpkg --print-architecture) > - > -if [ "$host_arch" != "$target_arch" ]; then > - case $target_arch in > - armhf) > - export ARCH=arm > - export CROSS_COMPILE="arm-linux-gnueabihf-" > - ;; > - arm64) > - export ARCH=arm64 > - export CROSS_COMPILE="aarch64-linux-gnu-" > - ;; > - mipsel) > - export ARCH=mips > - export CROSS_COMPILE="mipsel-linux-gnu-" > - ;; > - *) > - echo "error: unsupported architecture ($target_arch)" > - exit 1 > - ;; > - esac > -fi > - > -REPACK_DIR="$1/../repack" > -REPACK_LINUX_IMAGE_DIR="${REPACK_DIR}/linux-image" > -REPACK_LINUX_HEADERS_DIR="${REPACK_DIR}/linux-headers" > - > -make ${KERNEL_CONFIG_TARGET} || exit ${?} > -if [ -n "${KERNEL_FRAGMENTS}" ]; then > - scripts/kconfig/merge_config.sh -m .config ${KERNEL_FRAGMENTS} > -fi > - > -KV=$( make -s kernelrelease ) > -if [ "${KV}" != "${PV}" ]; then > - echo "ERROR: Recipe PV is \"${PV}\" but should be \"${KV}\"" 1>&2 > - echo "ERROR: Probably due to CONFIG_LOCALVERSION" 1>&2 > - exit 1 > -fi > - > -rm -f .version > -KBUILD_DEBARCH=$target_arch make -j $(($(nproc) * 2)) deb-pkg > - > -rm -rf "${REPACK_DIR}" > -mkdir -p "${REPACK_DIR}" > -mkdir -p "${REPACK_LINUX_IMAGE_DIR}" > -mkdir -p "${REPACK_LINUX_HEADERS_DIR}" > - > -cp -a debian "${REPACK_DIR}" > - > -# dpkg-gencontrol performs cross-incompatible checks on the > -# Architecture field; trick it to accept the control file > -sed -i "s/Architecture: .*/Architecture: any/" "${REPACK_DIR}/debian/control" > - > -cd .. > - > -dpkg-deb -R linux-image-${PV}_${PV}-1_*.deb "${REPACK_LINUX_IMAGE_DIR}" > -dpkg-deb -R linux-headers-${PV}_${PV}-1_*.deb "${REPACK_LINUX_HEADERS_DIR}" > - > -dpkg-gencontrol -crepack/debian/control \ > - -lrepack/debian/changelog \ > - -frepack/debian/files \ > - -plinux-image-${PV} \ > - -P"${REPACK_LINUX_IMAGE_DIR}" \ > - -DPackage="linux-image-${KERNEL_NAME}" \ > - -DSection=kernel \ > - -DPriority=required \ > - -DDepends="${KERNEL_DEBIAN_DEPENDS}" \ > - -DArchitecture=$target_arch > - > -# Add Debian-like link installation to postinst > -mkdir -p ${REPACK_LINUX_IMAGE_DIR}/lib/modules/${PV} > -touch "${REPACK_LINUX_IMAGE_DIR}/lib/modules/${PV}/.fresh-install" > -sed -i "${REPACK_LINUX_IMAGE_DIR}/DEBIAN/postinst" \ > - -e "/^set -e$/a\\ > -\\ > -if [ -f /lib/modules/${PV}/.fresh-install ]; then\\ > - change=install\\ > -else\\ > - change=upgrade\\ > -fi\\ > -linux-update-symlinks \$change ${PV} /boot/${KERNEL_FILE}-${PV}\\ > -rm -f /lib/modules/${PV}/.fresh-install" > - > -# Add Debian-like link removal to postrm > -sed -i "${REPACK_LINUX_IMAGE_DIR}/DEBIAN/postrm" \ > - -e "/^set -e$/a\\ > -\\ > -rm -f /lib/modules/${PV}/.fresh-install\\ > -\\ > -if [ \"\$1\" != upgrade ] && command -v linux-update-symlinks >/dev/null; then\\ > - linux-update-symlinks remove ${PV} /boot/${KERNEL_FILE}-${PV}\\ > -fi" > - > -# Make sure arm64 kernels are decompressed > -if [ "$target_arch" = "arm64" ]; then > - kernel_file="${REPACK_LINUX_IMAGE_DIR}/boot/${KERNEL_FILE}-${PV}" > - mv "${kernel_file}" "${kernel_file}.gz" > - gunzip "${kernel_file}.gz" > -fi Next regression: This got lost. Jan -- Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 83+ messages in thread
* Re: [PATCH L-C v2 7/7] linux-custom: rewrite to no longer depend on the kernel's builddeb 2019-11-09 8:51 ` Jan Kiszka @ 2019-11-09 13:43 ` Cedric Hombourger 2019-11-22 18:58 ` Jan Kiszka 0 siblings, 1 reply; 83+ messages in thread From: Cedric Hombourger @ 2019-11-09 13:43 UTC (permalink / raw) To: Jan Kiszka; +Cc: Cedric Hombourger, isar-users [-- Attachment #1: Type: text/plain, Size: 15156 bytes --] On Sat, 9 Nov 2019 at 09:51, Jan Kiszka <jan.kiszka@siemens.com> wrote: > On 07.11.19 10:23, Cedric Hombourger wrote: > > Regain control over the packaging of the Linux kernel by providing our > > own debian recipes (debian,{control,rules} files and friends) instead > > of using the kernel's builddeb script. This will allow generation of > > packages for "perf" (not included in this changeset) with per distro > > scheme (Debian and Ubuntu have differences) and maintain compatibility > > with upstream packages generated from different source packages but > > dependent on a certain deployment scheme (e.g. linux-base for Debian) > > > > Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com> > > --- > > doc/custom_kernel.md | 181 ++++++++++++++++ > > .../linux/files/build-kernel.sh | 128 ----------- > > meta/recipes-kernel/linux/files/debian/compat | 1 + > > .../linux/files/debian/control.tmpl | 37 ++++ > > .../linux/files/debian/isar/build.tmpl | 36 ++++ > > .../linux/files/debian/isar/clean.tmpl | 20 ++ > > .../linux/files/debian/isar/common.tmpl | 48 +++++ > > .../linux/files/debian/isar/install.tmpl | 204 ++++++++++++++++++ > > .../files/debian/linux-image.postinst.tmpl | 29 +++ > > .../files/debian/linux-image.postrm.tmpl | 35 +++ > > .../files/debian/linux-image.preinst.tmpl | 25 +++ > > .../linux/files/debian/linux-image.prerm.tmpl | 21 ++ > > .../linux/files/debian/rules.tmpl | 41 ++++ > > meta/recipes-kernel/linux/linux-custom.inc | 179 ++++++++++----- > > 14 files changed, 800 insertions(+), 185 deletions(-) > > create mode 100644 doc/custom_kernel.md > > delete mode 100644 meta/recipes-kernel/linux/files/build-kernel.sh > > create mode 100644 meta/recipes-kernel/linux/files/debian/compat > > create mode 100644 meta/recipes-kernel/linux/files/debian/control.tmpl > > create mode 100644 > meta/recipes-kernel/linux/files/debian/isar/build.tmpl > > create mode 100644 > meta/recipes-kernel/linux/files/debian/isar/clean.tmpl > > create mode 100644 > meta/recipes-kernel/linux/files/debian/isar/common.tmpl > > create mode 100644 > meta/recipes-kernel/linux/files/debian/isar/install.tmpl > > create mode 100644 > meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl > > create mode 100644 > meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl > > create mode 100644 > meta/recipes-kernel/linux/files/debian/linux-image.preinst.tmpl > > create mode 100644 > meta/recipes-kernel/linux/files/debian/linux-image.prerm.tmpl > > create mode 100755 meta/recipes-kernel/linux/files/debian/rules.tmpl > > > > diff --git a/doc/custom_kernel.md b/doc/custom_kernel.md > > new file mode 100644 > > index 0000000..934325a > > --- /dev/null > > +++ b/doc/custom_kernel.md > > @@ -0,0 +1,181 @@ > > +# Custom kernel recipe for Isar > > + > > +## Contents > > + > > + - [Summary](#summary) > > + - [Proposal owners](#proposal-owners) > > + - [Current status](#current-status) > > + - [Detailed description](#detailed-description) > > + - [Benefit to Isar](#benefit-to-isar) > > + - [How can I help?](#how-can-i-help) > > + - [Scope](#scope) > > + - [How to test](#how-to-test) > > + - [Dependencies](#dependencies) > > + - [Documentation](#documentation) > > + - [Questions and answers](#questions-and-answers) > > + > > +## Summary > > + > > +Isar currently uses the Linux kernel's builddeb script to generate > linux-image, > > +linux-headers and linux-libc-dev packages for your custom kernel > sources. The > > +main benefit of this approach was that the low level details of how to > package > > +the Linux kernel image, modules and headers were left in the kernel. > There are > > +however some drawbacks: Isar had to repack generated packages to > introduce some > > +of the changes it needs such as suffixing the kernel packages with the > name of > > +the custom kernel (e.g. with "mainline" for the "linux-mainline" > recipe), lack > > +of automatic dependencies (the builddeb script from the kernel does not > use > > +debhelper), it was difficult to add packages especially distribution > specific > > +packages (maintain version specific patch in Isar vs being able to > upstream > > +distribution specific changes upstream), etc. This change is about > being less > > +dependent on the kernel packaging scripts (which aren't used by major > distros > > +anyway) and regain control on how we want or need the Linux kernel > packaged. > > + > > +## Proposal owners > > + > > + * name: [Cedric Hombourger](https://github.com/chombourger) > > + > > +## Current status > > + > > +### Tests > > + > > +The following were recently checked: > > + > > + * Custom kernels in meta-isar build without (known) failures > > + * initrd image gets generated when the kernel is added to the rootfs > > + * `ISAR_CROSS_COMPILE` is supported > > + * `KERNEL_FILE` is honored > > + * `KERNEL_DEFCONFIG` may either be a file (specified via `SRC_URI`) or > a > > + kernel-provided defconfig > > + * Config fragments get merged with the user-specified kernel > configuration > > + * Path to installed DTBs may be configured (new) > > + * Configuration and build from a `devshell` > > + * Support for the `KERNEL_LIBC_DEV_DEPLOY` directive > > + > > +while the following needs work: > > + > > + * Check if custom-kernel builds are reproducible > > + > > +### Remaining work items > > + > > + * Check if there is a better way to cross compile kernel "scripts" > such as > > + `fixdep` or `modpost` than manually compiling them (they are > compiled for > > + the build machine (amd64) and not for the target (e.g. armhf) even > though > > + we are cross-compiling - yet the linux-headers package should be > shipping > > + "scripts" for the target machine) > > + > > +## Detailed description > > + > > +### Requirements > > + > > +#### Current > > + > > +Isar has had a `linux-custom` recipe for a long time and counts many > users. It is therefore > > +paramount to maintain features that it has today with no or little > changes to the APIs. The > > +following requirements were identified and will be maintained: > > + > > + 1. Sources to the custom Linux kernel may be specified via `SRC_URI` > > + > > + 2. Configure the kernel via an in-tree or an external `defconfig` via > `KERNEL_DEFCONFIG` > > + > > + 3. Support kernel configuration tweaks via configuration fragments > (`.cfg` files) > > + > > + 4. Patches to the linux kernel may be specified via `SRC_URI` > > + > > + 5. Ensure that the Isar recipe `PV` matches the kernel release > identifier (`KERNEL_RELEASE`) > > + > > + 6. Produce a `linux-image` package that ships the kernel image and > modules > > + > > + 7. Allow the name of the kernel image to be changed via `KERNEL_FILE` > (defaults to `vmlinuz`) > > + > > + 8. Produce a `linux-headers` package which includes kernel headers and > kbuild scripts/tools > > + > > + 9. The `linux-headers` package shall support native and cross compiles > of out-of-tree kernel modules > > + > > + 10. Produce a `linux-libc-dev` package to support user-land builds > > + > > + 11. Only ship the `linux-libc-dev` package if instructed to > (`KERNEL_LIBC_DEV_DEPLOY` equals to `"1"`) > > + > > + 12. Support both native and cross compiles (`ISAR_CROSS_COMPILE`) > > + > > + 13. Support for the following kernel architectures: > > + > > + * arm > > + * arm64 > > + * mips > > + * x86 > > + > > + 14. Support `devshell` (kernel configuration shall be applied) > > + > > +#### New requirements (proposed) > > + > > +This proposal includes the following new requirements: > > + > > + 1. Support creation of additional packages > > + > > + 2. Allow per-distro packaging tweaks > > + > > +### Approach > > + > > +The new implementation will have the `builddeb` code ported from the > Linux kernel included to the > > +Isar environment. It will be refactor to support pre and post hooks. > Hooks may be used by external > > +layers to extend the kernel packaging instructions provided by "stock" > Isar or by Isar itself to > > +implement distro-specific packages or tweaks. > > + > > +Hooks need the ability to add `build` and `install` steps which can be > achieved by having the main > > +`build` and `install` steps call pre and post scripts. This will > however solve only one side of the > > +problem: new packages should be declared in the `debian/control` file. > The packaging process > > +(initiated by `debian/rules`) will use a baseline `control` file when > started and a mechanism will > > +be provided for hooks to append `control` blocks as they are processed. > > + > > +### Future work > > + > > +In the event where this proposal is accepted, here are a few future > projects that are being > > +considered: > > + > > + 1. Package `perf` as Debian does > > + > > + 2. Add `ubuntu` as an Isar supported target distro and validate > `linux-custom` recipe > > + > > +## Benefit to Isar > > + > > + * Control the kernel build and packaging process > > + > > + * No longer need to repack packages > > + > > + * Provide hooks for custom kernel recipes to add build/install > directives > > + > > +## How can I help? > > + > > + * Check if you are able to build your own linux-custom recipes with > this change > > + * Check if packages generated by this recipe are compatible with your > Debian-based distro > > + * Review and provide comments on this changeset > > + * Create templates for distro-specific packages (e.g. perf) > > + > > +## Scope > > + > > + * Proposal owners: proposal owners are to communicate the idea to > isar-users and make sure > > + that all of their requirements are met (to the extent they are > already supported with the > > + current solution) or may be met in the future. > > + > > + * Other developers: port your out-of-tree linux-custom recipes to this > new solution and > > + report issues. > > + > > +## How to test? > > + > > +The linux-mainline recipe may be used for some basic testing. This > recipe is being used by the > > +following machines: > > + > > + * de0-nano-soc > > + * qemumipsel > > + > > +## Dependencies > > + > > + * None > > + > > +## Documentation > > + > > + * Document user-visible variables added by this proposal > > + > > +## Questions and answers > > + > > + * None > > diff --git a/meta/recipes-kernel/linux/files/build-kernel.sh > b/meta/recipes-kernel/linux/files/build-kernel.sh > > deleted file mode 100644 > > index f56e96c..0000000 > > --- a/meta/recipes-kernel/linux/files/build-kernel.sh > > +++ /dev/null > > @@ -1,128 +0,0 @@ > > -#!/bin/bash > > -# > > -# Custom kernel build > > -# > > -# This software is a part of ISAR. > > -# Copyright (c) Siemens AG, 2018 > > -# > > -# SPDX-License-Identifier: MIT > > - > > -source /isar/common.sh > > - > > -host_arch=$(dpkg --print-architecture) > > - > > -if [ "$host_arch" != "$target_arch" ]; then > > - case $target_arch in > > - armhf) > > - export ARCH=arm > > - export CROSS_COMPILE="arm-linux-gnueabihf-" > > - ;; > > - arm64) > > - export ARCH=arm64 > > - export CROSS_COMPILE="aarch64-linux-gnu-" > > - ;; > > - mipsel) > > - export ARCH=mips > > - export CROSS_COMPILE="mipsel-linux-gnu-" > > - ;; > > - *) > > - echo "error: unsupported architecture ($target_arch)" > > - exit 1 > > - ;; > > - esac > > -fi > > - > > -REPACK_DIR="$1/../repack" > > -REPACK_LINUX_IMAGE_DIR="${REPACK_DIR}/linux-image" > > -REPACK_LINUX_HEADERS_DIR="${REPACK_DIR}/linux-headers" > > - > > -make ${KERNEL_CONFIG_TARGET} || exit ${?} > > -if [ -n "${KERNEL_FRAGMENTS}" ]; then > > - scripts/kconfig/merge_config.sh -m .config ${KERNEL_FRAGMENTS} > > -fi > > - > > -KV=$( make -s kernelrelease ) > > -if [ "${KV}" != "${PV}" ]; then > > - echo "ERROR: Recipe PV is \"${PV}\" but should be \"${KV}\"" 1>&2 > > - echo "ERROR: Probably due to CONFIG_LOCALVERSION" 1>&2 > > - exit 1 > > -fi > > - > > -rm -f .version > > -KBUILD_DEBARCH=$target_arch make -j $(($(nproc) * 2)) deb-pkg > > - > > -rm -rf "${REPACK_DIR}" > > -mkdir -p "${REPACK_DIR}" > > -mkdir -p "${REPACK_LINUX_IMAGE_DIR}" > > -mkdir -p "${REPACK_LINUX_HEADERS_DIR}" > > - > > -cp -a debian "${REPACK_DIR}" > > - > > -# dpkg-gencontrol performs cross-incompatible checks on the > > -# Architecture field; trick it to accept the control file > > -sed -i "s/Architecture: .*/Architecture: any/" > "${REPACK_DIR}/debian/control" > > - > > -cd .. > > - > > -dpkg-deb -R linux-image-${PV}_${PV}-1_*.deb "${REPACK_LINUX_IMAGE_DIR}" > > -dpkg-deb -R linux-headers-${PV}_${PV}-1_*.deb > "${REPACK_LINUX_HEADERS_DIR}" > > - > > -dpkg-gencontrol -crepack/debian/control \ > > - -lrepack/debian/changelog \ > > - -frepack/debian/files \ > > - -plinux-image-${PV} \ > > - -P"${REPACK_LINUX_IMAGE_DIR}" \ > > - -DPackage="linux-image-${KERNEL_NAME}" \ > > - -DSection=kernel \ > > - -DPriority=required \ > > - -DDepends="${KERNEL_DEBIAN_DEPENDS}" \ > > - -DArchitecture=$target_arch > > - > > -# Add Debian-like link installation to postinst > > -mkdir -p ${REPACK_LINUX_IMAGE_DIR}/lib/modules/${PV} > > -touch "${REPACK_LINUX_IMAGE_DIR}/lib/modules/${PV}/.fresh-install" > > -sed -i "${REPACK_LINUX_IMAGE_DIR}/DEBIAN/postinst" \ > > - -e "/^set -e$/a\\ > > -\\ > > -if [ -f /lib/modules/${PV}/.fresh-install ]; then\\ > > - change=install\\ > > -else\\ > > - change=upgrade\\ > > -fi\\ > > -linux-update-symlinks \$change ${PV} /boot/${KERNEL_FILE}-${PV}\\ > > -rm -f /lib/modules/${PV}/.fresh-install" > > - > > -# Add Debian-like link removal to postrm > > -sed -i "${REPACK_LINUX_IMAGE_DIR}/DEBIAN/postrm" \ > > - -e "/^set -e$/a\\ > > -\\ > > -rm -f /lib/modules/${PV}/.fresh-install\\ > > -\\ > > -if [ \"\$1\" != upgrade ] && command -v linux-update-symlinks > >/dev/null; then\\ > > - linux-update-symlinks remove ${PV} /boot/${KERNEL_FILE}-${PV}\\ > > -fi" > > - > > -# Make sure arm64 kernels are decompressed > > -if [ "$target_arch" = "arm64" ]; then > > - kernel_file="${REPACK_LINUX_IMAGE_DIR}/boot/${KERNEL_FILE}-${PV}" > > - mv "${kernel_file}" "${kernel_file}.gz" > > - gunzip "${kernel_file}.gz" > > -fi > > Next regression: This got lost. > Fixed locally ( https://github.com/chombourger/isar-fork/blob/cedric/next/meta/recipes-kernel/linux/files/debian/isar/install.tmpl#L62 ) Started a test build to check Thanks for catching that subtle one > > Jan > > -- > Siemens AG, Corporate Technology, CT RDA IOT SES-DE > Corporate Competence Center Embedded Linux > > -- > You received this message because you are subscribed to a topic in the > Google Groups "isar-users" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/isar-users/XDeqow7oo9I/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > isar-users+unsubscribe@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/isar-users/d177ccc9-c18c-15c7-a356-92dd2c9fd8e6%40siemens.com > . > [-- Attachment #2: Type: text/html, Size: 19051 bytes --] ^ permalink raw reply [flat|nested] 83+ messages in thread
* Re: [PATCH L-C v2 7/7] linux-custom: rewrite to no longer depend on the kernel's builddeb 2019-11-09 13:43 ` Cedric Hombourger @ 2019-11-22 18:58 ` Jan Kiszka 2019-11-25 16:04 ` [PATCH L-C v4 0/5] linux-custom recipe rework Cedric Hombourger 0 siblings, 1 reply; 83+ messages in thread From: Jan Kiszka @ 2019-11-22 18:58 UTC (permalink / raw) To: Cedric Hombourger; +Cc: Cedric Hombourger, isar-users On 09.11.19 14:43, Cedric Hombourger wrote: > > > On Sat, 9 Nov 2019 at 09:51, Jan Kiszka <jan.kiszka@siemens.com > <mailto:jan.kiszka@siemens.com>> wrote: > > On 07.11.19 10:23, Cedric Hombourger wrote: > > Regain control over the packaging of the Linux kernel by > providing our > > own debian recipes (debian,{control,rules} files and friends) instead > > of using the kernel's builddeb script. This will allow generation of > > packages for "perf" (not included in this changeset) with per distro > > scheme (Debian and Ubuntu have differences) and maintain > compatibility > > with upstream packages generated from different source packages but > > dependent on a certain deployment scheme (e.g. linux-base for Debian) > > > > Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com > <mailto:Cedric_Hombourger@mentor.com>> > > --- > > doc/custom_kernel.md | 181 > ++++++++++++++++ > > .../linux/files/build-kernel.sh | 128 ----------- > > meta/recipes-kernel/linux/files/debian/compat | 1 + > > .../linux/files/debian/control.tmpl | 37 ++++ > > .../linux/files/debian/isar/build.tmpl | 36 ++++ > > .../linux/files/debian/isar/clean.tmpl | 20 ++ > > .../linux/files/debian/isar/common.tmpl | 48 +++++ > > .../linux/files/debian/isar/install.tmpl | 204 > ++++++++++++++++++ > > .../files/debian/linux-image.postinst.tmpl | 29 +++ > > .../files/debian/linux-image.postrm.tmpl | 35 +++ > > .../files/debian/linux-image.preinst.tmpl | 25 +++ > > .../linux/files/debian/linux-image.prerm.tmpl | 21 ++ > > .../linux/files/debian/rules.tmpl | 41 ++++ > > meta/recipes-kernel/linux/linux-custom.inc | 179 ++++++++++----- > > 14 files changed, 800 insertions(+), 185 deletions(-) > > create mode 100644 doc/custom_kernel.md > > delete mode 100644 meta/recipes-kernel/linux/files/build-kernel.sh > > create mode 100644 meta/recipes-kernel/linux/files/debian/compat > > create mode 100644 > meta/recipes-kernel/linux/files/debian/control.tmpl > > create mode 100644 > meta/recipes-kernel/linux/files/debian/isar/build.tmpl > > create mode 100644 > meta/recipes-kernel/linux/files/debian/isar/clean.tmpl > > create mode 100644 > meta/recipes-kernel/linux/files/debian/isar/common.tmpl > > create mode 100644 > meta/recipes-kernel/linux/files/debian/isar/install.tmpl > > create mode 100644 > meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl > > create mode 100644 > meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl > > create mode 100644 > meta/recipes-kernel/linux/files/debian/linux-image.preinst.tmpl > > create mode 100644 > meta/recipes-kernel/linux/files/debian/linux-image.prerm.tmpl > > create mode 100755 > meta/recipes-kernel/linux/files/debian/rules.tmpl > > > > diff --git a/doc/custom_kernel.md b/doc/custom_kernel.md > > new file mode 100644 > > index 0000000..934325a > > --- /dev/null > > +++ b/doc/custom_kernel.md > > @@ -0,0 +1,181 @@ > > +# Custom kernel recipe for Isar > > + > > +## Contents > > + > > + - [Summary](#summary) > > + - [Proposal owners](#proposal-owners) > > + - [Current status](#current-status) > > + - [Detailed description](#detailed-description) > > + - [Benefit to Isar](#benefit-to-isar) > > + - [How can I help?](#how-can-i-help) > > + - [Scope](#scope) > > + - [How to test](#how-to-test) > > + - [Dependencies](#dependencies) > > + - [Documentation](#documentation) > > + - [Questions and answers](#questions-and-answers) > > + > > +## Summary > > + > > +Isar currently uses the Linux kernel's builddeb script to > generate linux-image, > > +linux-headers and linux-libc-dev packages for your custom kernel > sources. The > > +main benefit of this approach was that the low level details of > how to package > > +the Linux kernel image, modules and headers were left in the > kernel. There are > > +however some drawbacks: Isar had to repack generated packages to > introduce some > > +of the changes it needs such as suffixing the kernel packages > with the name of > > +the custom kernel (e.g. with "mainline" for the "linux-mainline" > recipe), lack > > +of automatic dependencies (the builddeb script from the kernel > does not use > > +debhelper), it was difficult to add packages especially > distribution specific > > +packages (maintain version specific patch in Isar vs being able > to upstream > > +distribution specific changes upstream), etc. This change is > about being less > > +dependent on the kernel packaging scripts (which aren't used by > major distros > > +anyway) and regain control on how we want or need the Linux > kernel packaged. > > + > > +## Proposal owners > > + > > + * name: [Cedric Hombourger](https://github.com/chombourger) > > + > > +## Current status > > + > > +### Tests > > + > > +The following were recently checked: > > + > > + * Custom kernels in meta-isar build without (known) failures > > + * initrd image gets generated when the kernel is added to the > rootfs > > + * `ISAR_CROSS_COMPILE` is supported > > + * `KERNEL_FILE` is honored > > + * `KERNEL_DEFCONFIG` may either be a file (specified via > `SRC_URI`) or a > > + kernel-provided defconfig > > + * Config fragments get merged with the user-specified kernel > configuration > > + * Path to installed DTBs may be configured (new) > > + * Configuration and build from a `devshell` > > + * Support for the `KERNEL_LIBC_DEV_DEPLOY` directive > > + > > +while the following needs work: > > + > > + * Check if custom-kernel builds are reproducible > > + > > +### Remaining work items > > + > > + * Check if there is a better way to cross compile kernel > "scripts" such as > > + `fixdep` or `modpost` than manually compiling them (they are > compiled for > > + the build machine (amd64) and not for the target (e.g. armhf) > even though > > + we are cross-compiling - yet the linux-headers package should > be shipping > > + "scripts" for the target machine) > > + > > +## Detailed description > > + > > +### Requirements > > + > > +#### Current > > + > > +Isar has had a `linux-custom` recipe for a long time and counts > many users. It is therefore > > +paramount to maintain features that it has today with no or > little changes to the APIs. The > > +following requirements were identified and will be maintained: > > + > > + 1. Sources to the custom Linux kernel may be specified via > `SRC_URI` > > + > > + 2. Configure the kernel via an in-tree or an external > `defconfig` via `KERNEL_DEFCONFIG` > > + > > + 3. Support kernel configuration tweaks via configuration > fragments (`.cfg` files) > > + > > + 4. Patches to the linux kernel may be specified via `SRC_URI` > > + > > + 5. Ensure that the Isar recipe `PV` matches the kernel release > identifier (`KERNEL_RELEASE`) > > + > > + 6. Produce a `linux-image` package that ships the kernel image > and modules > > + > > + 7. Allow the name of the kernel image to be changed via > `KERNEL_FILE` (defaults to `vmlinuz`) > > + > > + 8. Produce a `linux-headers` package which includes kernel > headers and kbuild scripts/tools > > + > > + 9. The `linux-headers` package shall support native and cross > compiles of out-of-tree kernel modules > > + > > + 10. Produce a `linux-libc-dev` package to support user-land builds > > + > > + 11. Only ship the `linux-libc-dev` package if instructed to > (`KERNEL_LIBC_DEV_DEPLOY` equals to `"1"`) > > + > > + 12. Support both native and cross compiles (`ISAR_CROSS_COMPILE`) > > + > > + 13. Support for the following kernel architectures: > > + > > + * arm > > + * arm64 > > + * mips > > + * x86 > > + > > + 14. Support `devshell` (kernel configuration shall be applied) > > + > > +#### New requirements (proposed) > > + > > +This proposal includes the following new requirements: > > + > > + 1. Support creation of additional packages > > + > > + 2. Allow per-distro packaging tweaks > > + > > +### Approach > > + > > +The new implementation will have the `builddeb` code ported from > the Linux kernel included to the > > +Isar environment. It will be refactor to support pre and post > hooks. Hooks may be used by external > > +layers to extend the kernel packaging instructions provided by > "stock" Isar or by Isar itself to > > +implement distro-specific packages or tweaks. > > + > > +Hooks need the ability to add `build` and `install` steps which > can be achieved by having the main > > +`build` and `install` steps call pre and post scripts. This will > however solve only one side of the > > +problem: new packages should be declared in the `debian/control` > file. The packaging process > > +(initiated by `debian/rules`) will use a baseline `control` file > when started and a mechanism will > > +be provided for hooks to append `control` blocks as they are > processed. > > + > > +### Future work > > + > > +In the event where this proposal is accepted, here are a few > future projects that are being > > +considered: > > + > > + 1. Package `perf` as Debian does > > + > > + 2. Add `ubuntu` as an Isar supported target distro and validate > `linux-custom` recipe > > + > > +## Benefit to Isar > > + > > + * Control the kernel build and packaging process > > + > > + * No longer need to repack packages > > + > > + * Provide hooks for custom kernel recipes to add build/install > directives > > + > > +## How can I help? > > + > > + * Check if you are able to build your own linux-custom recipes > with this change > > + * Check if packages generated by this recipe are compatible > with your Debian-based distro > > + * Review and provide comments on this changeset > > + * Create templates for distro-specific packages (e.g. perf) > > + > > +## Scope > > + > > + * Proposal owners: proposal owners are to communicate the idea > to isar-users and make sure > > + that all of their requirements are met (to the extent they > are already supported with the > > + current solution) or may be met in the future. > > + > > + * Other developers: port your out-of-tree linux-custom recipes > to this new solution and > > + report issues. > > + > > +## How to test? > > + > > +The linux-mainline recipe may be used for some basic testing. > This recipe is being used by the > > +following machines: > > + > > + * de0-nano-soc > > + * qemumipsel > > + > > +## Dependencies > > + > > + * None > > + > > +## Documentation > > + > > + * Document user-visible variables added by this proposal > > + > > +## Questions and answers > > + > > + * None > > diff --git a/meta/recipes-kernel/linux/files/build-kernel.sh > b/meta/recipes-kernel/linux/files/build-kernel.sh > > deleted file mode 100644 > > index f56e96c..0000000 > > --- a/meta/recipes-kernel/linux/files/build-kernel.sh > > +++ /dev/null > > @@ -1,128 +0,0 @@ > > -#!/bin/bash > > -# > > -# Custom kernel build > > -# > > -# This software is a part of ISAR. > > -# Copyright (c) Siemens AG, 2018 > > -# > > -# SPDX-License-Identifier: MIT > > - > > -source /isar/common.sh > > - > > -host_arch=$(dpkg --print-architecture) > > - > > -if [ "$host_arch" != "$target_arch" ]; then > > - case $target_arch in > > - armhf) > > - export ARCH=arm > > - export CROSS_COMPILE="arm-linux-gnueabihf-" > > - ;; > > - arm64) > > - export ARCH=arm64 > > - export CROSS_COMPILE="aarch64-linux-gnu-" > > - ;; > > - mipsel) > > - export ARCH=mips > > - export CROSS_COMPILE="mipsel-linux-gnu-" > > - ;; > > - *) > > - echo "error: unsupported architecture ($target_arch)" > > - exit 1 > > - ;; > > - esac > > -fi > > - > > -REPACK_DIR="$1/../repack" > > -REPACK_LINUX_IMAGE_DIR="${REPACK_DIR}/linux-image" > > -REPACK_LINUX_HEADERS_DIR="${REPACK_DIR}/linux-headers" > > - > > -make ${KERNEL_CONFIG_TARGET} || exit ${?} > > -if [ -n "${KERNEL_FRAGMENTS}" ]; then > > - scripts/kconfig/merge_config.sh -m .config ${KERNEL_FRAGMENTS} > > -fi > > - > > -KV=$( make -s kernelrelease ) > > -if [ "${KV}" != "${PV}" ]; then > > - echo "ERROR: Recipe PV is \"${PV}\" but should be > \"${KV}\"" 1>&2 > > - echo "ERROR: Probably due to CONFIG_LOCALVERSION" 1>&2 > > - exit 1 > > -fi > > - > > -rm -f .version > > -KBUILD_DEBARCH=$target_arch make -j $(($(nproc) * 2)) deb-pkg > > - > > -rm -rf "${REPACK_DIR}" > > -mkdir -p "${REPACK_DIR}" > > -mkdir -p "${REPACK_LINUX_IMAGE_DIR}" > > -mkdir -p "${REPACK_LINUX_HEADERS_DIR}" > > - > > -cp -a debian "${REPACK_DIR}" > > - > > -# dpkg-gencontrol performs cross-incompatible checks on the > > -# Architecture field; trick it to accept the control file > > -sed -i "s/Architecture: .*/Architecture: any/" > "${REPACK_DIR}/debian/control" > > - > > -cd .. > > - > > -dpkg-deb -R linux-image-${PV}_${PV}-1_*.deb > "${REPACK_LINUX_IMAGE_DIR}" > > -dpkg-deb -R linux-headers-${PV}_${PV}-1_*.deb > "${REPACK_LINUX_HEADERS_DIR}" > > - > > -dpkg-gencontrol -crepack/debian/control \ > > - -lrepack/debian/changelog \ > > - -frepack/debian/files \ > > - -plinux-image-${PV} \ > > - -P"${REPACK_LINUX_IMAGE_DIR}" \ > > - -DPackage="linux-image-${KERNEL_NAME}" \ > > - -DSection=kernel \ > > - -DPriority=required \ > > - -DDepends="${KERNEL_DEBIAN_DEPENDS}" \ > > - -DArchitecture=$target_arch > > - > > -# Add Debian-like link installation to postinst > > -mkdir -p ${REPACK_LINUX_IMAGE_DIR}/lib/modules/${PV} > > -touch "${REPACK_LINUX_IMAGE_DIR}/lib/modules/${PV}/.fresh-install" > > -sed -i "${REPACK_LINUX_IMAGE_DIR}/DEBIAN/postinst" \ > > - -e "/^set -e$/a\\ > > -\\ > > -if [ -f /lib/modules/${PV}/.fresh-install ]; then\\ > > - change=install\\ > > -else\\ > > - change=upgrade\\ > > -fi\\ > > -linux-update-symlinks \$change ${PV} /boot/${KERNEL_FILE}-${PV}\\ > > -rm -f /lib/modules/${PV}/.fresh-install" > > - > > -# Add Debian-like link removal to postrm > > -sed -i "${REPACK_LINUX_IMAGE_DIR}/DEBIAN/postrm" \ > > - -e "/^set -e$/a\\ > > -\\ > > -rm -f /lib/modules/${PV}/.fresh-install\\ > > -\\ > > -if [ \"\$1\" != upgrade ] && command -v linux-update-symlinks > >/dev/null; then\\ > > - linux-update-symlinks remove ${PV} > /boot/${KERNEL_FILE}-${PV}\\ > > -fi" > > - > > -# Make sure arm64 kernels are decompressed > > -if [ "$target_arch" = "arm64" ]; then > > - > kernel_file="${REPACK_LINUX_IMAGE_DIR}/boot/${KERNEL_FILE}-${PV}" > > - mv "${kernel_file}" "${kernel_file}.gz" > > - gunzip "${kernel_file}.gz" > > -fi > > Next regression: This got lost. > > > Fixed locally > (https://github.com/chombourger/isar-fork/blob/cedric/next/meta/recipes-kernel/linux/files/debian/isar/install.tmpl#L62) > Started a test build to check > > Thanks for catching that subtle one > Do you plan to send a v4 soon? I'm playing with the "final" fix for patch.bbclass (i.e. an OE-core import), and that clashes with the current kernel build. Before patching that again... Jan -- Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH L-C v4 0/5] linux-custom recipe rework 2019-11-22 18:58 ` Jan Kiszka @ 2019-11-25 16:04 ` Cedric Hombourger 2019-11-25 16:04 ` [PATCH L-C v4 1/5] recipes-kernel/linux: make KERNEL_DEFCONFIG support in-tree defconfigs Cedric Hombourger ` (7 more replies) 0 siblings, 8 replies; 83+ messages in thread From: Cedric Hombourger @ 2019-11-25 16:04 UTC (permalink / raw) To: isar-users; +Cc: Cedric Hombourger The Linux kernel is packaged a bit differently between Debian variants and in particular for the perf package(s) in Debian vs Ubuntu. Since there is no way the kernel build scripts would ever get distro specific support for building more than the kernel and headers, we need our own build/packaging scripts. Version 4 of this changeset was tested against ci_build.sh and some images were booted on hardware available (including the de0-nano-soc). No regressions were detected (to date). This version is still relying on qemu-user-static being in the buildchroot-host I am unfortunetely getting little time to work on removing this dependency (for the avoidance of doubt, I understand the views of the maintainers). The purpose of this submission is to share progress. Changes compared to v3: Ship device trees where they can be found (checked on de0-nano-soc) Initialize LOCAL_VERSION from the recipe PV & PR Changes compared to v2: Rework custom_kernel.md Compare PV+/-LOCAL_VERSION and kernelrelease string Changes compared to v1: Add missing reference to KERNEL_DEFCONFIG in requirements list Amend postinst/postrm scripts according to CONFIG_BLK_DEV_INITRD Remove (unused) pre/post host hooks from debian/isar scripts Add preinst/prerm scripts shipped by the kernel Align {pre,post}{inst,rm} scripts with upstream Add ${BASE_DISTRO} to Build-Profiles Add nolibcdev to Build-Profiles if KERNEL_LIBC_DEV_DEPLOY != 1 Only build the libc-dev packages if it is needed Cedric Hombourger (5): recipes-kernel/linux: make KERNEL_DEFCONFIG support in-tree defconfigs linux-mainline: fix stripping of .0 from the kernel version linux-mainline: update from 4.19.0 to 4.19.80 buildchroot-host: install qemu-static to support hybrid cross-compiles linux-custom: rewrite to no longer depend on the kernel's builddeb doc/custom_kernel.md | 80 +++++++ ...ne_4.19.0.bb => linux-mainline_4.19.80.bb} | 4 +- .../buildchroot/buildchroot-host.bb | 6 + .../linux/files/build-kernel.sh | 129 ----------- meta/recipes-kernel/linux/files/debian/compat | 1 + .../linux/files/debian/control.tmpl | 37 +++ .../linux/files/debian/isar/build.tmpl | 45 ++++ .../linux/files/debian/isar/clean.tmpl | 21 ++ .../linux/files/debian/isar/common.tmpl | 50 ++++ .../linux/files/debian/isar/install.tmpl | 213 ++++++++++++++++++ .../linux/files/debian/isar/version.cfg.tmpl | 2 + .../files/debian/linux-image.postinst.tmpl | 29 +++ .../files/debian/linux-image.postrm.tmpl | 35 +++ .../files/debian/linux-image.preinst.tmpl | 25 ++ .../linux/files/debian/linux-image.prerm.tmpl | 21 ++ .../linux/files/debian/rules.tmpl | 39 ++++ meta/recipes-kernel/linux/linux-custom.inc | 209 ++++++++++++----- 17 files changed, 761 insertions(+), 185 deletions(-) create mode 100644 doc/custom_kernel.md rename meta-isar/recipes-kernel/linux/{linux-mainline_4.19.0.bb => linux-mainline_4.19.80.bb} (73%) delete mode 100644 meta/recipes-kernel/linux/files/build-kernel.sh create mode 100644 meta/recipes-kernel/linux/files/debian/compat create mode 100644 meta/recipes-kernel/linux/files/debian/control.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/isar/build.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/isar/clean.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/isar/common.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/isar/install.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/isar/version.cfg.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.preinst.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.prerm.tmpl create mode 100755 meta/recipes-kernel/linux/files/debian/rules.tmpl -- 2.20.1 ^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH L-C v4 1/5] recipes-kernel/linux: make KERNEL_DEFCONFIG support in-tree defconfigs 2019-11-25 16:04 ` [PATCH L-C v4 0/5] linux-custom recipe rework Cedric Hombourger @ 2019-11-25 16:04 ` Cedric Hombourger 2019-11-25 16:04 ` [PATCH L-C v4 2/5] linux-mainline: fix stripping of .0 from the kernel version Cedric Hombourger ` (6 subsequent siblings) 7 siblings, 0 replies; 83+ messages in thread From: Cedric Hombourger @ 2019-11-25 16:04 UTC (permalink / raw) To: isar-users; +Cc: Cedric Hombourger The recipe expected the specified KERNEL_DEFCONFIG to be provided as a SRC_URI while the kernel already provides many useful defconfigs. The recipe will now check if the specified configuration can be found in WORKDIR. If it is found, it will then be used via "make olddefconfig" as before. It will otherwise assume that it is an in-tree defconfig and attempt a "make foobar_defconfig" (where foobar_defconfig is the user-provided KERNEL_DEFCONFIG setting). To support this change, a KERNEL_CONFIG_TARGET variable is passed to the build-kernel.sh script via the environment. Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com> --- meta/recipes-kernel/linux/files/build-kernel.sh | 6 +----- meta/recipes-kernel/linux/linux-custom.inc | 9 ++++++++- 2 files changed, 9 insertions(+), 6 deletions(-) diff --git a/meta/recipes-kernel/linux/files/build-kernel.sh b/meta/recipes-kernel/linux/files/build-kernel.sh index b46cefa..386e58b 100644 --- a/meta/recipes-kernel/linux/files/build-kernel.sh +++ b/meta/recipes-kernel/linux/files/build-kernel.sh @@ -36,11 +36,7 @@ REPACK_DIR="$1/../repack" REPACK_LINUX_IMAGE_DIR="${REPACK_DIR}/linux-image" REPACK_LINUX_HEADERS_DIR="${REPACK_DIR}/linux-headers" -if [ -e .config ]; then - make olddefconfig -else - make defconfig -fi +make ${KERNEL_CONFIG_TARGET} || exit ${?} KV=$( make -s kernelrelease ) if [ "${KV}" != "${PV}" ]; then diff --git a/meta/recipes-kernel/linux/linux-custom.inc b/meta/recipes-kernel/linux/linux-custom.inc index 17f8464..712f52b 100644 --- a/meta/recipes-kernel/linux/linux-custom.inc +++ b/meta/recipes-kernel/linux/linux-custom.inc @@ -61,8 +61,14 @@ addtask devshell after do_install_builddeps dpkg_runbuild() { chmod +x ${WORKDIR}/build-kernel.sh + KERNEL_CONFIG_TARGET="${KERNEL_DEFCONFIG}" if [ -n "${KERNEL_DEFCONFIG}" ]; then - cp ${WORKDIR}/${KERNEL_DEFCONFIG} ${S}/.config + if [ -e "${WORKDIR}/${KERNEL_DEFCONFIG}" ]; then + cp ${WORKDIR}/${KERNEL_DEFCONFIG} ${S}/.config + KERNEL_CONFIG_TARGET="olddefconfig" + fi + else + KERNEL_CONFIG_TARGET="defconfig" fi E="${@ isar_export_proxies(d)}" @@ -71,6 +77,7 @@ dpkg_runbuild() { export KERNEL_NAME=${KERNEL_NAME_PROVIDED} export KBUILD_DEPENDS="${KBUILD_DEPENDS}" + export KERNEL_CONFIG_TARGET export KERNEL_DEBIAN_DEPENDS="${KERNEL_DEBIAN_DEPENDS}" export KERNEL_FILE="${KERNEL_FILE}" export KERNEL_HEADERS_DEBIAN_DEPENDS="${KERNEL_HEADERS_DEBIAN_DEPENDS}" -- 2.20.1 ^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH L-C v4 2/5] linux-mainline: fix stripping of .0 from the kernel version 2019-11-25 16:04 ` [PATCH L-C v4 0/5] linux-custom recipe rework Cedric Hombourger 2019-11-25 16:04 ` [PATCH L-C v4 1/5] recipes-kernel/linux: make KERNEL_DEFCONFIG support in-tree defconfigs Cedric Hombourger @ 2019-11-25 16:04 ` Cedric Hombourger 2019-11-25 16:04 ` [PATCH L-C v4 3/5] linux-mainline: update from 4.19.0 to 4.19.80 Cedric Hombourger ` (5 subsequent siblings) 7 siblings, 0 replies; 83+ messages in thread From: Cedric Hombourger @ 2019-11-25 16:04 UTC (permalink / raw) To: isar-users; +Cc: Cedric Hombourger The strip() function was used to strip the trailing .0 from the kernel version since it is omitted in the upstream archive file names. That function is actually taking a character set as argument not a substring. This was causing ARCHIVE_VERSION to expand to "4.19.8" when the user provided version was "4.19.80". The inline python code was changed to check if the version ends with ".0" and only then remove the last two characters. Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com> --- meta-isar/recipes-kernel/linux/linux-mainline_4.19.0.bb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/meta-isar/recipes-kernel/linux/linux-mainline_4.19.0.bb b/meta-isar/recipes-kernel/linux/linux-mainline_4.19.0.bb index e50e149..127f6e2 100644 --- a/meta-isar/recipes-kernel/linux/linux-mainline_4.19.0.bb +++ b/meta-isar/recipes-kernel/linux/linux-mainline_4.19.0.bb @@ -7,7 +7,7 @@ require recipes-kernel/linux/linux-custom.inc -ARCHIVE_VERSION = "${@d.getVar('PV').strip('.0')}" +ARCHIVE_VERSION = "${@ d.getVar('PV')[:-2] if d.getVar('PV').endswith('.0') else d.getVar('PV') }" SRC_URI += " \ https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-${ARCHIVE_VERSION}.tar.xz \ -- 2.20.1 ^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH L-C v4 3/5] linux-mainline: update from 4.19.0 to 4.19.80 2019-11-25 16:04 ` [PATCH L-C v4 0/5] linux-custom recipe rework Cedric Hombourger 2019-11-25 16:04 ` [PATCH L-C v4 1/5] recipes-kernel/linux: make KERNEL_DEFCONFIG support in-tree defconfigs Cedric Hombourger 2019-11-25 16:04 ` [PATCH L-C v4 2/5] linux-mainline: fix stripping of .0 from the kernel version Cedric Hombourger @ 2019-11-25 16:04 ` Cedric Hombourger 2019-11-25 16:04 ` [PATCH L-C v4 4/5] buildchroot-host: install qemu-static to support hybrid cross-compiles Cedric Hombourger ` (4 subsequent siblings) 7 siblings, 0 replies; 83+ messages in thread From: Cedric Hombourger @ 2019-11-25 16:04 UTC (permalink / raw) To: isar-users; +Cc: Cedric Hombourger In preparation for a new machine addition, update the linux-mainline kernel from 4.19.0 to 4.19.80. This is needed to get some of the mips fixes that were integrated in this longterm kernel version (and in particular: MIPS: have "plain" make calls build dtbs for selected platforms, i.e. 637dfa0fad6d91a9a709dc70549a6d20fa77f615 upstream) Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com> --- .../{linux-mainline_4.19.0.bb => linux-mainline_4.19.80.bb} | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) rename meta-isar/recipes-kernel/linux/{linux-mainline_4.19.0.bb => linux-mainline_4.19.80.bb} (87%) diff --git a/meta-isar/recipes-kernel/linux/linux-mainline_4.19.0.bb b/meta-isar/recipes-kernel/linux/linux-mainline_4.19.80.bb similarity index 87% rename from meta-isar/recipes-kernel/linux/linux-mainline_4.19.0.bb rename to meta-isar/recipes-kernel/linux/linux-mainline_4.19.80.bb index 127f6e2..b41d1a8 100644 --- a/meta-isar/recipes-kernel/linux/linux-mainline_4.19.0.bb +++ b/meta-isar/recipes-kernel/linux/linux-mainline_4.19.80.bb @@ -12,7 +12,7 @@ ARCHIVE_VERSION = "${@ d.getVar('PV')[:-2] if d.getVar('PV').endswith('.0') else SRC_URI += " \ https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-${ARCHIVE_VERSION}.tar.xz \ file://x86_64_defconfig" -SRC_URI[sha256sum] = "0c68f5655528aed4f99dae71a5b259edc93239fa899e2df79c055275c21749a1" +SRC_URI[sha256sum] = "80a9ba764e088aa7fddfef5a97c0236905e291468a37832243b6f3828d36e7ec" SRC_URI_append_de0-nano-soc = " \ file://0001-ARM-dts-socfpga-Rename-socfpga_cyclone5_de0_-sockit-.patch" -- 2.20.1 ^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH L-C v4 4/5] buildchroot-host: install qemu-static to support hybrid cross-compiles 2019-11-25 16:04 ` [PATCH L-C v4 0/5] linux-custom recipe rework Cedric Hombourger ` (2 preceding siblings ...) 2019-11-25 16:04 ` [PATCH L-C v4 3/5] linux-mainline: update from 4.19.0 to 4.19.80 Cedric Hombourger @ 2019-11-25 16:04 ` Cedric Hombourger 2019-11-25 16:06 ` [PATCH L-C v4 5/5] linux-custom: rewrite to no longer depend on the kernel's builddeb Cedric Hombourger ` (3 subsequent siblings) 7 siblings, 0 replies; 83+ messages in thread From: Cedric Hombourger @ 2019-11-25 16:04 UTC (permalink / raw) To: isar-users; +Cc: Cedric Hombourger The Linux kernel has great support for cross-compiling the kernel image and modules. There is however no support/mechanism for cross-compiling the build tools from its "scripts" directory. While HOSTCC may be set to use our cross-compiler, the kernel build infrasture would then try to run foreign-arch binaries such as fixdep. The easiest and least intrusive way to support this is to enable execution of such binaries via binfmt/qemu like we do in the buildchroot-target environment. Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com> --- meta/recipes-devtools/buildchroot/buildchroot-host.bb | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/meta/recipes-devtools/buildchroot/buildchroot-host.bb b/meta/recipes-devtools/buildchroot/buildchroot-host.bb index 408ad39..2e76acb 100644 --- a/meta/recipes-devtools/buildchroot/buildchroot-host.bb +++ b/meta/recipes-devtools/buildchroot/buildchroot-host.bb @@ -15,3 +15,9 @@ BUILDCHROOT_PREINSTALL ?= " \ ${BUILDCHROOT_PREINSTALL_COMMON} \ libc6:${DISTRO_ARCH} \ crossbuild-essential-${DISTRO_ARCH}" + +buildchroot_install_files_append() { + if [ -e '/usr/bin/qemu-${QEMU_ARCH}-static' ]; then + sudo cp '/usr/bin/qemu-${QEMU_ARCH}-static' '${BUILDCHROOT_DIR}/usr/bin/qemu-${QEMU_ARCH}-static' + fi +} -- 2.20.1 ^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH L-C v4 5/5] linux-custom: rewrite to no longer depend on the kernel's builddeb 2019-11-25 16:04 ` [PATCH L-C v4 0/5] linux-custom recipe rework Cedric Hombourger ` (3 preceding siblings ...) 2019-11-25 16:04 ` [PATCH L-C v4 4/5] buildchroot-host: install qemu-static to support hybrid cross-compiles Cedric Hombourger @ 2019-11-25 16:06 ` Cedric Hombourger 2019-11-25 19:55 ` Jan Kiszka 2019-11-26 17:10 ` Gylstorff Quirin 2019-11-25 16:16 ` [PATCH L-C v4 0/5] linux-custom recipe rework Jan Kiszka ` (2 subsequent siblings) 7 siblings, 2 replies; 83+ messages in thread From: Cedric Hombourger @ 2019-11-25 16:06 UTC (permalink / raw) To: isar-users; +Cc: Cedric Hombourger Regain control over the packaging of the Linux kernel by providing our own debian recipes (debian,{control,rules} files and friends) instead of using the kernel's builddeb script. This will allow generation of packages for "perf" (not included in this changeset) with per distro scheme (Debian and Ubuntu have differences) and maintain compatibility with upstream packages generated from different source packages but dependent on a certain deployment scheme (e.g. linux-base for Debian) Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com> --- doc/custom_kernel.md | 80 +++++++ .../linux/files/build-kernel.sh | 125 ---------- meta/recipes-kernel/linux/files/debian/compat | 1 + .../linux/files/debian/control.tmpl | 37 +++ .../linux/files/debian/isar/build.tmpl | 45 ++++ .../linux/files/debian/isar/clean.tmpl | 21 ++ .../linux/files/debian/isar/common.tmpl | 50 ++++ .../linux/files/debian/isar/install.tmpl | 213 ++++++++++++++++++ .../linux/files/debian/isar/version.cfg.tmpl | 2 + .../files/debian/linux-image.postinst.tmpl | 29 +++ .../files/debian/linux-image.postrm.tmpl | 35 +++ .../files/debian/linux-image.preinst.tmpl | 25 ++ .../linux/files/debian/linux-image.prerm.tmpl | 21 ++ .../linux/files/debian/rules.tmpl | 39 ++++ meta/recipes-kernel/linux/linux-custom.inc | 210 ++++++++++++----- 15 files changed, 750 insertions(+), 183 deletions(-) create mode 100644 doc/custom_kernel.md delete mode 100644 meta/recipes-kernel/linux/files/build-kernel.sh create mode 100644 meta/recipes-kernel/linux/files/debian/compat create mode 100644 meta/recipes-kernel/linux/files/debian/control.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/isar/build.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/isar/clean.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/isar/common.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/isar/install.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/isar/version.cfg.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.preinst.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.prerm.tmpl create mode 100755 meta/recipes-kernel/linux/files/debian/rules.tmpl diff --git a/doc/custom_kernel.md b/doc/custom_kernel.md new file mode 100644 index 0000000..31319d4 --- /dev/null +++ b/doc/custom_kernel.md @@ -0,0 +1,80 @@ +# Custom kernel recipe for Isar + +## Contents + + - [Summary](#summary) + - [Features](#features) + - [Future](#future) + - [Examples](#examples) + +## Summary + +Isar provides a recipe to build custom kernels for Debian-based distributions. +It uses templates to generate the debian meta-data (such as debian/control) and +Debian's [BuildProfiles](https://wiki.debian.org/BuildProfileSpec) to handle +some of the distro specific variations. It should be noted that Isar has moved +away from using the kernel's builddeb script since it would not generate all +the packages we need (and in particular perf). + +## Features + +The linux-custom recipe provides support for: + + 1. Sources to the custom Linux kernel may be specified via `SRC_URI` + + 2. Configure the kernel via an in-tree or an external `defconfig` via + `KERNEL_DEFCONFIG` + + 3. Integrate kernel configuration tweaks via configuration fragments (`.cfg` + files) + + 4. Patches to the linux kernel may be specified via `SRC_URI` + + 5. Ensure that the Isar recipe `PV` matches the kernel release identifier + (`KERNEL_RELEASE`) + + 6. Produce a `linux-image` package that ships the kernel image and modules + + 7. Allow the name of the kernel image to be changed via `KERNEL_FILE` (defaults + to `vmlinuz`) + + 8. Produce a `linux-headers` package which includes kernel headers and kbuild + scripts/tools + + 9. The `linux-headers` package shall support native and cross compiles of + out-of-tree kernel modules + + 10. Produce a `linux-libc-dev` package to support user-land builds + + 11. Only build/ship the `linux-libc-dev` package if instructed to + (`KERNEL_LIBC_DEV_DEPLOY` equals to `"1"`) + + 12. Support both native and cross compiles (`ISAR_CROSS_COMPILE`) + + 13. Support for the following kernel architectures: + + * arm + * arm64 + * mips + * x86 + + 14. Support `devshell` (kernel configuration shall be applied) + +## Future + +In the future, the recipe may be extended to: + + 1. Package perf + + 2. Support inclusion/build of dts files listed in `SRC_URI` + + 3. Be compatible with Ubuntu + +## Examples + +The linux-custom recipe is currently used by the linux-mainline package and is +used mainline recipe may be used for some basic testing. This recipe is being +used by the following machines: + + * de0-nano-soc + * qemumipsel diff --git a/meta/recipes-kernel/linux/files/build-kernel.sh b/meta/recipes-kernel/linux/files/build-kernel.sh deleted file mode 100644 index 386e58b..0000000 --- a/meta/recipes-kernel/linux/files/build-kernel.sh +++ /dev/null @@ -1,125 +0,0 @@ -#!/bin/bash -# -# Custom kernel build -# -# This software is a part of ISAR. -# Copyright (c) Siemens AG, 2018 -# -# SPDX-License-Identifier: MIT - -source /isar/common.sh - -host_arch=$(dpkg --print-architecture) - -if [ "$host_arch" != "$target_arch" ]; then - case $target_arch in - armhf) - export ARCH=arm - export CROSS_COMPILE="arm-linux-gnueabihf-" - ;; - arm64) - export ARCH=arm64 - export CROSS_COMPILE="aarch64-linux-gnu-" - ;; - mipsel) - export ARCH=mips - export CROSS_COMPILE="mipsel-linux-gnu-" - ;; - *) - echo "error: unsupported architecture ($target_arch)" - exit 1 - ;; - esac -fi - -REPACK_DIR="$1/../repack" -REPACK_LINUX_IMAGE_DIR="${REPACK_DIR}/linux-image" -REPACK_LINUX_HEADERS_DIR="${REPACK_DIR}/linux-headers" - -make ${KERNEL_CONFIG_TARGET} || exit ${?} - -KV=$( make -s kernelrelease ) -if [ "${KV}" != "${PV}" ]; then - echo "ERROR: Recipe PV is \"${PV}\" but should be \"${KV}\"" 1>&2 - echo "ERROR: Probably due to CONFIG_LOCALVERSION" 1>&2 - exit 1 -fi - -rm -f .version -KBUILD_DEBARCH=$target_arch make -j $(($(nproc) * 2)) deb-pkg - -rm -rf "${REPACK_DIR}" -mkdir -p "${REPACK_DIR}" -mkdir -p "${REPACK_LINUX_IMAGE_DIR}" -mkdir -p "${REPACK_LINUX_HEADERS_DIR}" - -cp -a debian "${REPACK_DIR}" - -# dpkg-gencontrol performs cross-incompatible checks on the -# Architecture field; trick it to accept the control file -sed -i "s/Architecture: .*/Architecture: any/" "${REPACK_DIR}/debian/control" - -cd .. - -dpkg-deb -R linux-image-${PV}_${PV}-1_*.deb "${REPACK_LINUX_IMAGE_DIR}" -dpkg-deb -R linux-headers-${PV}_${PV}-1_*.deb "${REPACK_LINUX_HEADERS_DIR}" - -dpkg-gencontrol -crepack/debian/control \ - -lrepack/debian/changelog \ - -frepack/debian/files \ - -plinux-image-${PV} \ - -P"${REPACK_LINUX_IMAGE_DIR}" \ - -DPackage="linux-image-${KERNEL_NAME}" \ - -DSection=kernel \ - -DPriority=required \ - -DDepends="${KERNEL_DEBIAN_DEPENDS}" \ - -DArchitecture=$target_arch - -# Add Debian-like link installation to postinst -mkdir -p ${REPACK_LINUX_IMAGE_DIR}/lib/modules/${PV} -touch "${REPACK_LINUX_IMAGE_DIR}/lib/modules/${PV}/.fresh-install" -sed -i "${REPACK_LINUX_IMAGE_DIR}/DEBIAN/postinst" \ - -e "/^set -e$/a\\ -\\ -if [ -f /lib/modules/${PV}/.fresh-install ]; then\\ - change=install\\ -else\\ - change=upgrade\\ -fi\\ -linux-update-symlinks \$change ${PV} /boot/${KERNEL_FILE}-${PV}\\ -rm -f /lib/modules/${PV}/.fresh-install" - -# Add Debian-like link removal to postrm -sed -i "${REPACK_LINUX_IMAGE_DIR}/DEBIAN/postrm" \ - -e "/^set -e$/a\\ -\\ -rm -f /lib/modules/${PV}/.fresh-install\\ -\\ -if [ \"\$1\" != upgrade ] && command -v linux-update-symlinks >/dev/null; then\\ - linux-update-symlinks remove ${PV} /boot/${KERNEL_FILE}-${PV}\\ -fi" - -# Make sure arm64 kernels are decompressed -if [ "$target_arch" = "arm64" ]; then - kernel_file="${REPACK_LINUX_IMAGE_DIR}/boot/${KERNEL_FILE}-${PV}" - mv "${kernel_file}" "${kernel_file}.gz" - gunzip "${kernel_file}.gz" -fi - -dpkg-gencontrol -crepack/debian/control \ - -lrepack/debian/changelog \ - -frepack/debian/files \ - -plinux-headers-${PV} \ - -P"${REPACK_LINUX_HEADERS_DIR}" \ - -Vkernel:debarch="${KERNEL_NAME}" \ - -DPackage="linux-headers-${KERNEL_NAME}" \ - -DSection=kernel \ - -DDepends="${KERNEL_HEADERS_DEBIAN_DEPENDS}" \ - -DArchitecture=$target_arch - -fakeroot dpkg-deb -b "${REPACK_LINUX_IMAGE_DIR}" \ - linux-image-${KERNEL_NAME}_${PV}-1_${KERNEL_NAME}.deb -rm -f linux-image-${PV}_${PV}-1_*.deb -fakeroot dpkg-deb -b "${REPACK_LINUX_HEADERS_DIR}" \ - linux-headers-${KERNEL_NAME}_${PV}-1_${KERNEL_NAME}.deb -rm -f linux-headers-${PV}_${PV}-1_*.deb diff --git a/meta/recipes-kernel/linux/files/debian/compat b/meta/recipes-kernel/linux/files/debian/compat new file mode 100644 index 0000000..ec63514 --- /dev/null +++ b/meta/recipes-kernel/linux/files/debian/compat @@ -0,0 +1 @@ +9 diff --git a/meta/recipes-kernel/linux/files/debian/control.tmpl b/meta/recipes-kernel/linux/files/debian/control.tmpl new file mode 100644 index 0000000..7236732 --- /dev/null +++ b/meta/recipes-kernel/linux/files/debian/control.tmpl @@ -0,0 +1,37 @@ +Source: linux-${KERNEL_NAME_PROVIDED} +Section: kernel +Priority: optional +Maintainer: ${MAINTAINER} +Build-Depends: bc, kmod, cpio, ${KBUILD_DEPENDS} +Homepage: http://www.kernel.org/ + +Package: linux-image-${KERNEL_NAME_PROVIDED} +Architecture: any +Depends: ${KERNEL_DEBIAN_DEPENDS} +Description: ${KERNEL_NAME_PROVIDED} Linux kernel, version ${PV}-${KERNEL_LOCALVERSION} + This package contains the Linux kernel, modules and corresponding other + files, version: ${PV}-${KERNEL_LOCALVERSION}. + +Package: linux-headers-${KERNEL_NAME_PROVIDED} +Architecture: any +Depends: ${KERNEL_HEADERS_DEBIAN_DEPENDS}, ${perl:Depends}, ${shlib:Depends} +Description: ${KERNEL_NAME_PROVIDED} Linux kernel headers for ${PV}-${KERNEL_LOCALVERSION} + This package provides kernel header files for ${PV}-${KERNEL_LOCALVERSION} on ${DISTRO_ARCH} + . + This is useful for people who need to build external modules + +Package: linux-libc-dev +Build-Profiles: <!nolibcdev> +Section: devel +Provides: linux-kernel-headers +Architecture: any +Description: Linux support headers for userspace development + This package provides userspaces headers from the Linux kernel. These headers + are used by the installed headers for GNU glibc and other system libraries. + +Package: linux-image-${KERNEL_NAME_PROVIDED}-dbg +Section: debug +Architecture: any +Description: Linux kernel debugging symbols for ${PV}-${KERNEL_LOCALVERSION} + This package will come in handy if you need to debug the kernel. It provides + all the necessary debug symbols for the kernel and its modules. diff --git a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl new file mode 100644 index 0000000..1775b19 --- /dev/null +++ b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl @@ -0,0 +1,45 @@ +#!/bin/bash +# Copyright (c) Mentor Graphics, a Siemens business, 2019 +# SPDX-License-Identifier: MIT + +# Load common stuff +. ${S}/debian/isar/common || exit ${?} + +do_build() { + + # Print a few things that are of particular interest + print_settings + + # Process existing kernel configuration to make sure it is complete + # (use defaults for options that were not specified) + ${MAKE} O=${KERNEL_BUILD_DIR} olddefconfig prepare || exit ${?} + + # Check if the recipe's PV makes sense + KR=$(${MAKE} O=${KERNEL_BUILD_DIR} -s --no-print-directory kernelrelease) + eval $(grep ^CONFIG_LOCALVERSION= ${KERNEL_BUILD_DIR}/${KCONF} || true) + if [ "${PV}-${KERNEL_LOCALVERSION}" != "${KR}" ]; then + echo "ERROR: Recipe version (${PV}-${KERNEL_LOCALVERSION}) does not seem to match the kernelrelease (${KR})!" 1>&2 + echo "ERROR: Make sure the kernel version in your NAME/PV/PR settings and/or CONFIG_LOCALVERSION are aligned" 1>&2 + exit 1 + fi + + # Trace what we do here + set -x + + # Build the Linux kernel + ${MAKE} O=${KERNEL_BUILD_DIR} -j $(nproc) || exit ${?} + + # Stop tracing + set +x +} + +print_settings() { + cat <<EOF +# Build settings: +# --------------- +# ARCH=${ARCH} +# CROSS_COMPILE=${CROSS_COMPILE} +EOF +} + +main build ${*} diff --git a/meta/recipes-kernel/linux/files/debian/isar/clean.tmpl b/meta/recipes-kernel/linux/files/debian/isar/clean.tmpl new file mode 100644 index 0000000..f903987 --- /dev/null +++ b/meta/recipes-kernel/linux/files/debian/isar/clean.tmpl @@ -0,0 +1,21 @@ +#!/bin/bash +# Copyright (c) Mentor Graphics, a Siemens business, 2019 +# SPDX-License-Identifier: MIT + +# Load common stuff +. ${S}/debian/isar/common || exit ${?} + +do_clean() { + + # Trace what we do here + set -x + + rm -rf ${deb_img_dir} ${deb_dbg_dir} + rm -rf ${deb_kern_hdrdir} + rm -rf ${deb_libc_hdr_dir} + + # Stop tracing + set +x +} + +main clean ${*} diff --git a/meta/recipes-kernel/linux/files/debian/isar/common.tmpl b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl new file mode 100644 index 0000000..44f67b3 --- /dev/null +++ b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl @@ -0,0 +1,50 @@ +#!/bin/bash +# Copyright (c) Mentor Graphics, a Siemens business, 2019 +# SPDX-License-Identifier: MIT + +# Isar settings +ARCH=${KERNEL_ARCH} +KERNEL_PKG_IMAGE=linux-image-${KERNEL_NAME_PROVIDED} +KERNEL_PKG_KERN_HEADERS=linux-headers-${KERNEL_NAME_PROVIDED} +KERNEL_PKG_LIBC_HEADERS=linux-libc-dev + +# Constants +KCONF=.config + +# Target directories +deb_top_dir=${S}/debian +deb_img_dir=${deb_top_dir}/${KERNEL_PKG_IMAGE} +deb_dbg_dir=${deb_img_dir}-dbg +deb_dtb_dir=${deb_img_dir}/${KERNEL_DTB_DIR} +deb_kern_hdr_dir=${deb_top_dir}/${KERNEL_PKG_KERN_HEADERS} +deb_libc_hdr_dir=${deb_top_dir}/${KERNEL_PKG_LIBC_HEADERS} + +# Array of packages to be generated +declare -A kern_pkgs + +main() { + local target=${1} + + if [ ! -f ${S}/debian/isar/${target} ]; then + echo "error: ${target} is not a supported build target!" >&2 + return 1 + fi + + # create do_<x> variables for each package to be generated + for p in $(unset DEB_HOST_ARCH; dh_listpackages); do + kern_pkgs["${p}"]="1" + done + + # variables to be exported + export ARCH + + # are we cross-compiling? + BUILD_ARCH=$(dpkg-architecture -qDEB_BUILD_ARCH) + if [ "${BUILD_ARCH}" = "${DISTRO_ARCH}" ]; then + # no, make sure CROSS_COMPILE isn't set + unset CROSS_COMPILE + fi + + # call the actual target script + do_${target} || return ${?} +} diff --git a/meta/recipes-kernel/linux/files/debian/isar/install.tmpl b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl new file mode 100644 index 0000000..806df5d --- /dev/null +++ b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl @@ -0,0 +1,213 @@ +#!/bin/bash +# Copyright (c) Mentor Graphics, a Siemens business, 2019 +# SPDX-License-Identifier: MIT + +# Load common stuff +. ${S}/debian/isar/common || exit ${?} + +# Stop on error +set -e + +do_install() { + + # check if our kernel was configured + if [ ! -f "${O}/.config" ]; then + echo "error: kernel not configured!" >&2 + return 1 + fi + + # load its configuration + . ${O}/.config + + kimage="$(${MAKE} O=${O} -s --no-print-directory image_name)" + krel="$(${MAKE} O=${O} -s --no-print-directory kernelrelease)" + case "${ARCH}" in + mips|powerpc) kimage_path="boot/vmlinux-${krel}" ;; + um) kimage_path="usr/bin/vmlinux-${krel}" ;; + *) kimage_path="boot/vmlinuz-${krel}" ;; + esac + + print_settings + + # Trace what we do here + set -x + + install_image + install_hooks + install_dtbs + install_kmods + install_headers + + # Stop tracing + set +x +} + +print_settings() { + cat <<EOF +Install settings: +----------------- +deb_dtb_dir=${deb_dtb_dir} +deb_hdr_dir=${deb_hdr_dir} +kimage=${kimage} +kimage_path=${kimage_path} + +EOF +} + +install_image() { + install -m 755 -d ${deb_img_dir}/$(dirname ${kimage_path}) + cp ${O}/${kimage} ${deb_img_dir}/${kimage_path} + + # Make sure arm64 kernels are decompressed + if [ "${ARCH}" = "arm64" ]; then + mv ${deb_img_dir}/${kimage_path} ${deb_img_dir}/${kimage_path}.gz + gunzip -f ${deb_img_dir}/${kimage_path}.gz + fi + + install_image_debug +} + +install_image_debug() { + # Different tools want the image in different locations + # perf + mkdir -p ${deb_dbg_dir}/usr/lib/debug/lib/modules/${krel}/ + cp ${O}/vmlinux ${deb_dbg_dir}/usr/lib/debug/lib/modules/${krel}/ + # systemtap + mkdir -p ${deb_dbg_dir}/usr/lib/debug/boot/ + ln -s ../lib/modules/$version/vmlinux ${deb_dbg_dir}/usr/lib/debug/boot/vmlinux-${krel} + # kdump-tools + ln -s lib/modules/${krel}/vmlinux ${deb_dbg_dir}/usr/lib/debug/vmlinux-${krel} +} + +install_hooks() { + install -m 755 -d ${deb_img_dir}/etc/kernel/install.d + install -m 755 -d ${deb_img_dir}/etc/kernel/postinst.d + install -m 755 -d ${deb_img_dir}/etc/kernel/postrm.d + install -m 755 -d ${deb_img_dir}/etc/kernel/prerm.d + + initrd="No" + [ -z "${CONFIG_BLK_DEV_INITRD}" ] || initrd="Yes" + + for script in postinst postrm preinst prerm; do + sed -i -e "s,INITRD=[A-Za-z0-9]*,INITRD=${initrd},g" ${S}/debian/linux-image*.${script} + sed -i -e "s,version=.*,version=${krel},g" ${S}/debian/linux-image*.${script} + done +} + +install_dtbs() { + [ -n "${CONFIG_OF}" ] || return 0 + ${MAKE} O=${O} INSTALL_DTBS_PATH=${deb_dtb_dir} dtbs_install +} + +install_kmods() { + [ -n "${CONFIG_MODULES}" ] || return 0 + ${MAKE} O=${O} INSTALL_MOD_PATH=${deb_img_dir} modules_install + touch ${deb_img_dir}/lib/modules/${krel}/.fresh-install + rm -fv ${deb_img_dir}/lib/modules/${krel}/build + rm -fv ${deb_img_dir}/lib/modules/${krel}/source + install_kmods_debug +} + +kmods_sign() { + [ -n "${CONFIG_MODULE_SIG_ALL}" ] || return 0 + ${MAKE} O=${O} INSTALL_MOD_PATH=${deb_img_dir} modules_sign +} + +install_kmods_debug() { + [ -n "${CONFIG_DEBUG_INFO}" ] || return 0 + + kmod_inst_dir=${deb_img_dir}/lib/modules + kmod_debug_dir=${deb_dbg_dir}/usr/lib/debug + + # copy kernels modules to usr/lib/debug + mkdir -p ${kmod_debug_dir} + tar -C ${kmod_inst_dir}/ -cO --exclude='modules.*' . | tar -C ${kmod_debug_dir}/ -xf - + # strip everything but debug sections for modules in usr/lib/debug + find ${kmod_debug_dir} -name *.ko -exec ${CROSS_COMPILE}objcopy --only-keep-debug {} \; + # and strip debug sections from modules in lib/modules + find ${kmod_inst_dir} -name *.ko -exec ${CROSS_COMPILE}objcopy --strip-debug {} \; + + # re-sign stripped kernel modules + kmods_sign +} + +headers_check() { + ${MAKE} O=${O} headers_check +} + +libc_headers() { + mkdir -p ${deb_libc_hdr_dir} + ${MAKE} O=${O} headers_install INSTALL_HDR_PATH=${deb_libc_hdr_dir}/usr + host_arch=$(dpkg-architecture -a${DISTRO_ARCH} -qDEB_HOST_MULTIARCH) + mkdir ${deb_libc_hdr_dir}/usr/include/${host_arch} + mv ${deb_libc_hdr_dir}/usr/include/asm ${deb_libc_hdr_dir}/usr/include/${host_arch}/ +} + +# the kernel does not seem to provide a way to cross-compile its tools, we however +# have qemu-user-static binaries in our buildchroot. Create a new build tree for the +# scripts and override HOSTCC to force a cross-compile +kernel_tools_build() { + odir="${S}/build-scripts" + mflags="HOSTCC=${CROSS_COMPILE}gcc O=${odir} NOSTDINC_FLAGS=" + rm -rf ${odir} && mkdir -p ${odir} && cp ${S}/${KERNEL_BUILD_DIR}/.config ${odir} + (unset LD_PRELOAD; cd ${odir} && make -C ${S} ${mflags} olddefconfig scripts) + (cd ${odir}; find scripts -type f -executable -exec file {} \;|grep ELF|cut -d: -f1) >${src_hdr_files} + tar -C ${odir} -cf - -T - <${src_hdr_files} | tar -C ${destdir} -xf - +} + +kernel_tools() { + # remove object files + find ${destdir}/scripts -type f -name '*.o' |xargs rm -f + + # we're all done if we aren't cross-compiling + [ -n "${CROSS_COMPILE}" ] || return 0 + + # remove ELF executables from the linux-headers package + find ${destdir}/scripts -type f -exec file {} \;|grep ELF|cut -d: -f1|xargs rm -fv + + # cross-compile kernel tools to be shipped with linux-headers + kernel_tools_build +} + +kernel_headers() { + destdir=${deb_kern_hdr_dir}/${KERNEL_HEADERS_DIR} + src_hdr_files=$(mktemp) + obj_hdr_files=$(mktemp) + + mkdir -p ${destdir} + mkdir -p ${deb_kern_hdr_dir}/lib/modules/${krel} + + (cd ${S}; find . -name 'Makefile*' -o -name 'Kconfig*' -o -name '*.pl') >>${src_hdr_files} + (cd ${S}; find arch/*/include include scripts -type f -o -type l) >>${src_hdr_files} + (cd ${S}; find arch/${ARCH} -name module.lds -o -name Kbuild.platforms -o -name Platform) >>${src_hdr_files} + (cd ${S}; find $(find arch/${ARCH} -name include -o -name scripts -type d) -type f) >>${src_hdr_files} + + (cd ${O}; find arch/${ARCH}/include Module.symvers include scripts -type f) >>${obj_hdr_files} + if [ -n "${CONFIG_STACK_VALIDATION}" ]; then + (cd ${O}; find tools/objtool -type f -executable) >>${obj_hdr_files} + fi + if [ -n "${CONFIG_GCC_PLUGINS}" ]; then + (cd ${O}; find scripts/gcc-plugins -name *.so -o -name gcc-common.h) >>${obj_hdr_files} + fi + + # deploy files that were matched above + tar -C ${S} -cf - -T - <${src_hdr_files} | tar -C ${destdir} -xf - + tar -C ${O} -cf - -T - <${obj_hdr_files} | tar -C ${destdir} -xf - + + # add the kernel config + cp ${O}/${KCONF} ${destdir}/.config + + # handle kernel development tools + kernel_tools + + # create symlinks + ln -sf /${KERNEL_HEADERS_DIR} ${deb_kern_hdr_dir}/lib/modules/${krel}/build +} + +install_headers() { + headers_check + [ -z ${kern_pkgs["linux-libc-headers"]} ] || libc_headers + kernel_headers +} + +main install ${*} diff --git a/meta/recipes-kernel/linux/files/debian/isar/version.cfg.tmpl b/meta/recipes-kernel/linux/files/debian/isar/version.cfg.tmpl new file mode 100644 index 0000000..09fa364 --- /dev/null +++ b/meta/recipes-kernel/linux/files/debian/isar/version.cfg.tmpl @@ -0,0 +1,2 @@ +CONFIG_LOCALVERSION="-${KERNEL_LOCALVERSION}" +# CONFIG_LOCALVERSION_AUTO is not set diff --git a/meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl b/meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl new file mode 100644 index 0000000..b373801 --- /dev/null +++ b/meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl @@ -0,0 +1,29 @@ +#!/bin/sh +# based on https://salsa.debian.org/kernel-team/linux/blob/master/debian/templates/image.postinst.in + +# Tell initramfs builder whether it's wanted +export INITRD=Yes + +version=${PV}-${KERNEL_LOCALVERSION} +image_path=/boot/${KERNEL_FILE}-${version} + +if [ "$1" != configure ]; then + exit 0 +fi + +depmod $version + +if [ -f /lib/modules/$version/.fresh-install ]; then + change=install +else + change=upgrade +fi +linux-update-symlinks $change $version $image_path +rm -f /lib/modules/$version/.fresh-install + +if [ -d /etc/kernel/postinst.d ]; then + DEB_MAINT_PARAMS="$*" run-parts --report --exit-on-error --arg=$version \ + --arg=$image_path /etc/kernel/postinst.d +fi + +exit 0 diff --git a/meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl b/meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl new file mode 100644 index 0000000..1b5dbbe --- /dev/null +++ b/meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl @@ -0,0 +1,35 @@ +#!/bin/sh -e +# based on https://salsa.debian.org/kernel-team/linux/blob/master/debian/templates/image.postrm.in + +# Tell initramfs builder whether it's wanted +export INITRD=Yes + +version=${PV}-${KERNEL_LOCALVERSION} +image_path=/boot/${KERNEL_FILE}-${version} + +rm -f /lib/modules/$version/.fresh-install + +if [ "$1" != upgrade ] && command -v linux-update-symlinks >/dev/null; then + linux-update-symlinks remove $version $image_path +fi + +if [ -d /etc/kernel/postrm.d ]; then + DEB_MAINT_PARAMS="$*" run-parts --report --exit-on-error --arg=$version \ + --arg=$image_path /etc/kernel/postrm.d +fi + +if [ "$1" = purge ]; then + for extra_file in modules.dep modules.isapnpmap modules.pcimap \ + modules.usbmap modules.parportmap \ + modules.generic_string modules.ieee1394map \ + modules.ieee1394map modules.pnpbiosmap \ + modules.alias modules.ccwmap modules.inputmap \ + modules.symbols modules.ofmap \ + modules.seriomap modules.\*.bin \ + modules.softdep modules.devname; do + eval rm -f /lib/modules/$version/$extra_file + done + rmdir /lib/modules/$version || true +fi + +exit 0 diff --git a/meta/recipes-kernel/linux/files/debian/linux-image.preinst.tmpl b/meta/recipes-kernel/linux/files/debian/linux-image.preinst.tmpl new file mode 100644 index 0000000..4f24019 --- /dev/null +++ b/meta/recipes-kernel/linux/files/debian/linux-image.preinst.tmpl @@ -0,0 +1,25 @@ +#!/bin/sh -e +# based on https://salsa.debian.org/kernel-team/linux/blob/master/debian/templates/image.preinst.in + +# Tell initramfs builder whether it's wanted +export INITRD=Yes + +version=${PV}-${KERNEL_LOCALVERSION} +image_path=/boot/${KERNEL_FILE}-${version} + +if [ "$1" = abort-upgrade ]; then + exit 0 +fi + +if [ "$1" = install ]; then + # Create a flag file for postinst + mkdir -p /lib/modules/$version + touch /lib/modules/$version/.fresh-install +fi + +if [ -d /etc/kernel/preinst.d ]; then + DEB_MAINT_PARAMS="$*" run-parts --report --exit-on-error --arg=$version \ + --arg=$image_path /etc/kernel/preinst.d +fi + +exit 0 diff --git a/meta/recipes-kernel/linux/files/debian/linux-image.prerm.tmpl b/meta/recipes-kernel/linux/files/debian/linux-image.prerm.tmpl new file mode 100644 index 0000000..cd697c0 --- /dev/null +++ b/meta/recipes-kernel/linux/files/debian/linux-image.prerm.tmpl @@ -0,0 +1,21 @@ +#!/bin/sh -e +# based on https://salsa.debian.org/kernel-team/linux/blob/master/debian/templates/image.prerm.in + +# Tell initramfs builder whether it's wanted +export INITRD=Yes + +version=${PV}-${KERNEL_LOCALVERSION} +image_path=/boot/${KERNEL_FILE}-${version} + +if [ "$1" != remove ]; then + exit 0 +fi + +linux-check-removal $version + +if [ -d /etc/kernel/prerm.d ]; then + DEB_MAINT_PARAMS="$*" run-parts --report --exit-on-error --arg=$version \ + --arg=$image_path /etc/kernel/prerm.d +fi + +exit 0 diff --git a/meta/recipes-kernel/linux/files/debian/rules.tmpl b/meta/recipes-kernel/linux/files/debian/rules.tmpl new file mode 100755 index 0000000..c024768 --- /dev/null +++ b/meta/recipes-kernel/linux/files/debian/rules.tmpl @@ -0,0 +1,39 @@ +#!/usr/bin/make -f + +CROSS_COMPILE:=$(DEB_HOST_GNU_TYPE)- + +O:=$(CURDIR)/${KERNEL_BUILD_DIR} +S:=$(CURDIR) +deb_top_dir:=$(S)/debian + +# Dynamic variables to be passed to Isar build scripts +isar_env=$(strip \ + export CROSS_COMPILE='$(CROSS_COMPILE)' && \ + export DEB_BUILD_PROFILES='$(DEB_BUILD_PROFILES)' && \ + export DEB_HOST_GNU_TYPE='$(DEB_HOST_GNU_TYPE)' && \ + export MAKE='$(MAKE)' && \ + export O='${O}' \ + export S='${S}' \ +) + +%: + dh $(@) + +.PHONY: override_dh_auto_clean +override_dh_auto_clean: + $(isar_env) && bash $(deb_top_dir)/isar/clean + +.PHONY: override_dh_auto_build +override_dh_auto_build: + $(isar_env) && bash $(deb_top_dir)/isar/build + +override_dh_auto_install: + $(isar_env) && bash $(deb_top_dir)/isar/install + +.PHONY: override_dh_auto_test +override_dh_auto_test: + true + +PHONY: override_dh_strip +override_dh_strip: + dh_strip -Xvmlinux --no-automatic-dbgsym diff --git a/meta/recipes-kernel/linux/linux-custom.inc b/meta/recipes-kernel/linux/linux-custom.inc index 712f52b..bf1646a 100644 --- a/meta/recipes-kernel/linux/linux-custom.inc +++ b/meta/recipes-kernel/linux/linux-custom.inc @@ -2,89 +2,183 @@ # # This software is a part of ISAR. # Copyright (c) Siemens AG, 2018 +# Copyright (c) Mentor Graphics, a Siemens business, 2019 # # SPDX-License-Identifier: MIT -FILESEXTRAPATHS_prepend := "${FILE_DIRNAME}/files:" +# Settings that would typically be done from the custom kernel recipe +# ------------------------------------------------------------------- +CHANGELOG_V = "${PV}+${PR}" DESCRIPTION ?= "Custom kernel" +MAINTAINER ?= "isar-users <isar-users@googlegroups.com>" + +KBUILD_DEPENDS ?= "build-essential:native, \ + libelf-dev:native, \ + libncurses-dev:native, \ + libssl-dev:native, \ + bc, \ + bison, \ + cpio, \ + flex, \ + git, \ + kmod, \ + libssl-dev," + +KERNEL_DEBIAN_DEPENDS ?= "initramfs-tools | linux-initramfs-tool, \ + kmod, \ + linux-base (>= 4.3~)," + +KERNEL_HEADERS_DEBIAN_DEPENDS ?= "libc6, \ + libssl1.1," + +KERNEL_LIBC_DEV_DEPLOY ?= "0" + +# Settings that may be changed on a per distro, machine or layer basis +# -------------------------------------------------------------------- + +KERNEL_LOCALVERSION ?= "-${KERNEL_NAME_PROVIDED}-${PR}" +KERNEL_DTB_DIR ?= "usr/lib/linux-image-${PV}-${KERNEL_LOCALVERSION}" +KERNEL_HEADERS_DIR ?= "usr/src/linux-headers-${PV}-${KERNEL_LOCALVERSION}" -KERNEL_NAME_PROVIDED ?= "${@ d.getVar('PN', True).partition('linux-')[2]}" KERNEL_DEFCONFIG ?= "" +# Add our template meta-data to the sources +FILESEXTRAPATHS_prepend := "${FILE_DIRNAME}/files:" +SRC_URI += "file://debian" + +# Variables and files that make our templates +# ------------------------------------------- + +TEMPLATE_FILES += " \ + debian/control.tmpl \ + debian/isar/build.tmpl \ + debian/isar/clean.tmpl \ + debian/isar/common.tmpl \ + debian/isar/install.tmpl \ + debian/isar/version.cfg.tmpl \ + debian/linux-image.postinst.tmpl \ + debian/linux-image.postrm.tmpl \ + debian/linux-image.preinst.tmpl \ + debian/linux-image.prerm.tmpl \ + debian/rules.tmpl \ +" + +TEMPLATE_VARS += " \ + HOST_ARCH \ + KBUILD_DEPENDS \ + KERNEL_ARCH \ + KERNEL_DEBIAN_DEPENDS \ + KERNEL_BUILD_DIR \ + KERNEL_DTB_DIR \ + KERNEL_FILE \ + KERNEL_HEADERS_DEBIAN_DEPENDS \ + KERNEL_HEADERS_DIR \ + KERNEL_LOCALVERSION \ + KERNEL_NAME_PROVIDED \ +" + +inherit dpkg +inherit template + +# Derive name of the kernel packages from the name of this recipe +KERNEL_NAME_PROVIDED ?= "${@ d.getVar('PN', True).partition('linux-')[2]}" + +# Make bitbake know we will be producing linux-image and linux-headers packages python() { kernel_name = d.getVar("KERNEL_NAME_PROVIDED", True) d.setVar('PROVIDES', 'linux-image-' + kernel_name + ' ' + \ 'linux-headers-' + kernel_name) } -inherit dpkg-base - -SRC_URI += "file://build-kernel.sh" - -KBUILD_DEPENDS ?= " \ - build-essential:native \ - libssl-dev \ - libelf-dev \ - bc \ - git \ - kmod \ - bison \ - flex \ - cpio \ - libncurses-dev" -KERNEL_DEBIAN_DEPENDS ?= "initramfs-tools | linux-initramfs-tool, kmod, linux-base (>= 4.3~)" -KERNEL_HEADERS_DEBIAN_DEPENDS ?= "libc6, libssl1.1" - -KERNEL_LIBC_DEV_DEPLOY ?= "0" - -do_install_builddeps() { - dpkg_do_mounts - E="${@ isar_export_proxies(d)}" - sudo -E chroot ${BUILDCHROOT_DIR} \ - apt-get update \ - -o Dir::Etc::SourceList="sources.list.d/isar-apt.list" \ - -o Dir::Etc::SourceParts="-" \ - -o APT::Get::List-Cleanup="0" - sudo -E chroot ${BUILDCHROOT_DIR} \ - apt-get install \ - -y -o Debug::pkgProblemResolver=yes \ - --no-install-recommends ${KBUILD_DEPENDS} - dpkg_undo_mounts +def get_kernel_arch(d): + distro_arch = d.getVar("DISTRO_ARCH") + if distro_arch == "amd64": + kernel_arch = "x86" + elif distro_arch == "arm64": + kernel_arch = "arm64" + elif distro_arch == "armhf": + kernel_arch = "arm" + elif distro_arch == "mipsel": + kernel_arch = "mips" + else: + kernel_arch = "" + return kernel_arch + +KERNEL_ARCH ??= "${@get_kernel_arch(d)}" + +def config_fragments(d): + fragments = [] + sources = d.getVar("SRC_URI").split() + for s in sources: + _, _, local, _, _, parm = bb.fetch.decodeurl(s) + apply = parm.get("apply") + if apply == "no": + continue + base, ext = os.path.splitext(os.path.basename(local)) + if ext and ext in (".cfg"): + fragments.append(local) + return fragments + +do_prepare_build_prepend() { + # copy meta-data over to source tree + rm -rf ${S}/debian + cp -r ${WORKDIR}/debian ${S}/ + + # remove templates from the source tree + find ${S}/debian -name *.tmpl | xargs rm -f + + # rename install/remove hooks to match user-specified name for our linux-image package + mv ${S}/debian/linux-image.postinst ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.postinst + mv ${S}/debian/linux-image.postrm ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.postrm + mv ${S}/debian/linux-image.preinst ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.preinst + mv ${S}/debian/linux-image.prerm ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.prerm + + # produce a changelog for our kernel build + deb_add_changelog } -addtask install_builddeps after do_prepare_build before do_dpkg_build -# apt and reprepro may not run in parallel, acquire the Isar lock -do_install_builddeps[lockfiles] += "${REPO_ISAR_DIR}/isar.lock" - -addtask devshell after do_install_builddeps +# build directory for our "full" kernel build +KERNEL_BUILD_DIR = "build-full" -dpkg_runbuild() { - chmod +x ${WORKDIR}/build-kernel.sh - KERNEL_CONFIG_TARGET="${KERNEL_DEFCONFIG}" +dpkg_configure_kernel() { + config_target="${KERNEL_DEFCONFIG}" + rm -rf ${S}/${KERNEL_BUILD_DIR} && mkdir -p ${S}/${KERNEL_BUILD_DIR} if [ -n "${KERNEL_DEFCONFIG}" ]; then if [ -e "${WORKDIR}/${KERNEL_DEFCONFIG}" ]; then - cp ${WORKDIR}/${KERNEL_DEFCONFIG} ${S}/.config - KERNEL_CONFIG_TARGET="olddefconfig" + cp ${WORKDIR}/${KERNEL_DEFCONFIG} ${S}/${KERNEL_BUILD_DIR}/.config + config_target="olddefconfig" fi else - KERNEL_CONFIG_TARGET="defconfig" + config_target="defconfig" fi - E="${@ isar_export_proxies(d)}" - - export PV=${PV} - export KERNEL_NAME=${KERNEL_NAME_PROVIDED} - - export KBUILD_DEPENDS="${KBUILD_DEPENDS}" - export KERNEL_CONFIG_TARGET - export KERNEL_DEBIAN_DEPENDS="${KERNEL_DEBIAN_DEPENDS}" - export KERNEL_FILE="${KERNEL_FILE}" - export KERNEL_HEADERS_DEBIAN_DEPENDS="${KERNEL_HEADERS_DEBIAN_DEPENDS}" + # copy config fragments over to the kernel tree + src_frags="${@ " ".join(config_fragments(d)) }" + out_frags="${@ " ".join(map(lambda frag: 'debian/fragments/' + frag, config_fragments(d))) }" + rm -rf ${S}/debian/fragments + if [ -n "${src_frags}" ]; then + mkdir -p ${S}/debian/fragments + (cd ${WORKDIR} && cp ${src_frags} ${S}/debian/fragments/) + fi - sudo -E chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} ${PP}/build-kernel.sh ${PP}/${PPS} ${DISTRO_ARCH} + sudo -E chroot --userspec=$(id -u):$(id -g) ${BUILDCHROOT_DIR} sh -c " \ + export ARCH=${KERNEL_ARCH} && \ + cd ${PP}/${PPS} && \ + make O=${KERNEL_BUILD_DIR} ${config_target} && \ + ./scripts/kconfig/merge_config.sh \ + -O ${KERNEL_BUILD_DIR}/ \ + ${KERNEL_BUILD_DIR}/.config \ + ${out_frags} \ + debian/isar/version.cfg \ + " +} +dpkg_runbuild_prepend() { + profiles="${BASE_DISTRO}" if [ "${KERNEL_LIBC_DEV_DEPLOY}" != "1" ]; then - rm -f ${WORKDIR}/linux-libc-dev_${PV}*.deb + profiles="${profiles} nolibcdev" fi + export DEB_BUILD_PROFILES="${profiles}" + dpkg_configure_kernel } -- 2.20.1 ^ permalink raw reply [flat|nested] 83+ messages in thread
* Re: [PATCH L-C v4 5/5] linux-custom: rewrite to no longer depend on the kernel's builddeb 2019-11-25 16:06 ` [PATCH L-C v4 5/5] linux-custom: rewrite to no longer depend on the kernel's builddeb Cedric Hombourger @ 2019-11-25 19:55 ` Jan Kiszka 2019-11-26 7:42 ` Cedric Hombourger 2019-11-26 17:10 ` Gylstorff Quirin 1 sibling, 1 reply; 83+ messages in thread From: Jan Kiszka @ 2019-11-25 19:55 UTC (permalink / raw) To: Cedric Hombourger, isar-users On 25.11.19 17:06, Cedric Hombourger wrote: > Regain control over the packaging of the Linux kernel by providing our > own debian recipes (debian,{control,rules} files and friends) instead > of using the kernel's builddeb script. This will allow generation of > packages for "perf" (not included in this changeset) with per distro > scheme (Debian and Ubuntu have differences) and maintain compatibility > with upstream packages generated from different source packages but > dependent on a certain deployment scheme (e.g. linux-base for Debian) > > Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com> > --- > doc/custom_kernel.md | 80 +++++++ > .../linux/files/build-kernel.sh | 125 ---------- > meta/recipes-kernel/linux/files/debian/compat | 1 + > .../linux/files/debian/control.tmpl | 37 +++ > .../linux/files/debian/isar/build.tmpl | 45 ++++ > .../linux/files/debian/isar/clean.tmpl | 21 ++ > .../linux/files/debian/isar/common.tmpl | 50 ++++ > .../linux/files/debian/isar/install.tmpl | 213 ++++++++++++++++++ > .../linux/files/debian/isar/version.cfg.tmpl | 2 + > .../files/debian/linux-image.postinst.tmpl | 29 +++ > .../files/debian/linux-image.postrm.tmpl | 35 +++ > .../files/debian/linux-image.preinst.tmpl | 25 ++ > .../linux/files/debian/linux-image.prerm.tmpl | 21 ++ > .../linux/files/debian/rules.tmpl | 39 ++++ > meta/recipes-kernel/linux/linux-custom.inc | 210 ++++++++++++----- > 15 files changed, 750 insertions(+), 183 deletions(-) > create mode 100644 doc/custom_kernel.md > delete mode 100644 meta/recipes-kernel/linux/files/build-kernel.sh > create mode 100644 meta/recipes-kernel/linux/files/debian/compat > create mode 100644 meta/recipes-kernel/linux/files/debian/control.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/isar/build.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/isar/clean.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/isar/common.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/isar/install.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/isar/version.cfg.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.preinst.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.prerm.tmpl > create mode 100755 meta/recipes-kernel/linux/files/debian/rules.tmpl > > diff --git a/doc/custom_kernel.md b/doc/custom_kernel.md > new file mode 100644 > index 0000000..31319d4 > --- /dev/null > +++ b/doc/custom_kernel.md > @@ -0,0 +1,80 @@ > +# Custom kernel recipe for Isar > + > +## Contents > + > + - [Summary](#summary) > + - [Features](#features) > + - [Future](#future) > + - [Examples](#examples) > + > +## Summary > + > +Isar provides a recipe to build custom kernels for Debian-based distributions. > +It uses templates to generate the debian meta-data (such as debian/control) and > +Debian's [BuildProfiles](https://wiki.debian.org/BuildProfileSpec) to handle > +some of the distro specific variations. It should be noted that Isar has moved > +away from using the kernel's builddeb script since it would not generate all > +the packages we need (and in particular perf). > + > +## Features > + > +The linux-custom recipe provides support for: > + > + 1. Sources to the custom Linux kernel may be specified via `SRC_URI` > + > + 2. Configure the kernel via an in-tree or an external `defconfig` via > + `KERNEL_DEFCONFIG` > + > + 3. Integrate kernel configuration tweaks via configuration fragments (`.cfg` > + files) > + > + 4. Patches to the linux kernel may be specified via `SRC_URI` > + > + 5. Ensure that the Isar recipe `PV` matches the kernel release identifier > + (`KERNEL_RELEASE`) > + > + 6. Produce a `linux-image` package that ships the kernel image and modules > + > + 7. Allow the name of the kernel image to be changed via `KERNEL_FILE` (defaults > + to `vmlinuz`) > + > + 8. Produce a `linux-headers` package which includes kernel headers and kbuild > + scripts/tools > + > + 9. The `linux-headers` package shall support native and cross compiles of > + out-of-tree kernel modules > + > + 10. Produce a `linux-libc-dev` package to support user-land builds > + > + 11. Only build/ship the `linux-libc-dev` package if instructed to > + (`KERNEL_LIBC_DEV_DEPLOY` equals to `"1"`) > + > + 12. Support both native and cross compiles (`ISAR_CROSS_COMPILE`) > + > + 13. Support for the following kernel architectures: > + > + * arm > + * arm64 > + * mips > + * x86 > + > + 14. Support `devshell` (kernel configuration shall be applied) > + > +## Future > + > +In the future, the recipe may be extended to: > + > + 1. Package perf > + > + 2. Support inclusion/build of dts files listed in `SRC_URI` > + > + 3. Be compatible with Ubuntu > + > +## Examples > + > +The linux-custom recipe is currently used by the linux-mainline package and is > +used mainline recipe may be used for some basic testing. This recipe is being > +used by the following machines: > + > + * de0-nano-soc > + * qemumipsel > diff --git a/meta/recipes-kernel/linux/files/build-kernel.sh b/meta/recipes-kernel/linux/files/build-kernel.sh > deleted file mode 100644 > index 386e58b..0000000 > --- a/meta/recipes-kernel/linux/files/build-kernel.sh > +++ /dev/null > @@ -1,125 +0,0 @@ > -#!/bin/bash > -# > -# Custom kernel build > -# > -# This software is a part of ISAR. > -# Copyright (c) Siemens AG, 2018 > -# > -# SPDX-License-Identifier: MIT > - > -source /isar/common.sh > - > -host_arch=$(dpkg --print-architecture) > - > -if [ "$host_arch" != "$target_arch" ]; then > - case $target_arch in > - armhf) > - export ARCH=arm > - export CROSS_COMPILE="arm-linux-gnueabihf-" > - ;; > - arm64) > - export ARCH=arm64 > - export CROSS_COMPILE="aarch64-linux-gnu-" > - ;; > - mipsel) > - export ARCH=mips > - export CROSS_COMPILE="mipsel-linux-gnu-" > - ;; > - *) > - echo "error: unsupported architecture ($target_arch)" > - exit 1 > - ;; > - esac > -fi > - > -REPACK_DIR="$1/../repack" > -REPACK_LINUX_IMAGE_DIR="${REPACK_DIR}/linux-image" > -REPACK_LINUX_HEADERS_DIR="${REPACK_DIR}/linux-headers" > - > -make ${KERNEL_CONFIG_TARGET} || exit ${?} > - > -KV=$( make -s kernelrelease ) > -if [ "${KV}" != "${PV}" ]; then > - echo "ERROR: Recipe PV is \"${PV}\" but should be \"${KV}\"" 1>&2 > - echo "ERROR: Probably due to CONFIG_LOCALVERSION" 1>&2 > - exit 1 > -fi > - > -rm -f .version > -KBUILD_DEBARCH=$target_arch make -j $(($(nproc) * 2)) deb-pkg > - > -rm -rf "${REPACK_DIR}" > -mkdir -p "${REPACK_DIR}" > -mkdir -p "${REPACK_LINUX_IMAGE_DIR}" > -mkdir -p "${REPACK_LINUX_HEADERS_DIR}" > - > -cp -a debian "${REPACK_DIR}" > - > -# dpkg-gencontrol performs cross-incompatible checks on the > -# Architecture field; trick it to accept the control file > -sed -i "s/Architecture: .*/Architecture: any/" "${REPACK_DIR}/debian/control" > - > -cd .. > - > -dpkg-deb -R linux-image-${PV}_${PV}-1_*.deb "${REPACK_LINUX_IMAGE_DIR}" > -dpkg-deb -R linux-headers-${PV}_${PV}-1_*.deb "${REPACK_LINUX_HEADERS_DIR}" > - > -dpkg-gencontrol -crepack/debian/control \ > - -lrepack/debian/changelog \ > - -frepack/debian/files \ > - -plinux-image-${PV} \ > - -P"${REPACK_LINUX_IMAGE_DIR}" \ > - -DPackage="linux-image-${KERNEL_NAME}" \ > - -DSection=kernel \ > - -DPriority=required \ > - -DDepends="${KERNEL_DEBIAN_DEPENDS}" \ > - -DArchitecture=$target_arch > - > -# Add Debian-like link installation to postinst > -mkdir -p ${REPACK_LINUX_IMAGE_DIR}/lib/modules/${PV} > -touch "${REPACK_LINUX_IMAGE_DIR}/lib/modules/${PV}/.fresh-install" > -sed -i "${REPACK_LINUX_IMAGE_DIR}/DEBIAN/postinst" \ > - -e "/^set -e$/a\\ > -\\ > -if [ -f /lib/modules/${PV}/.fresh-install ]; then\\ > - change=install\\ > -else\\ > - change=upgrade\\ > -fi\\ > -linux-update-symlinks \$change ${PV} /boot/${KERNEL_FILE}-${PV}\\ > -rm -f /lib/modules/${PV}/.fresh-install" > - > -# Add Debian-like link removal to postrm > -sed -i "${REPACK_LINUX_IMAGE_DIR}/DEBIAN/postrm" \ > - -e "/^set -e$/a\\ > -\\ > -rm -f /lib/modules/${PV}/.fresh-install\\ > -\\ > -if [ \"\$1\" != upgrade ] && command -v linux-update-symlinks >/dev/null; then\\ > - linux-update-symlinks remove ${PV} /boot/${KERNEL_FILE}-${PV}\\ > -fi" > - > -# Make sure arm64 kernels are decompressed > -if [ "$target_arch" = "arm64" ]; then > - kernel_file="${REPACK_LINUX_IMAGE_DIR}/boot/${KERNEL_FILE}-${PV}" > - mv "${kernel_file}" "${kernel_file}.gz" > - gunzip "${kernel_file}.gz" > -fi > - > -dpkg-gencontrol -crepack/debian/control \ > - -lrepack/debian/changelog \ > - -frepack/debian/files \ > - -plinux-headers-${PV} \ > - -P"${REPACK_LINUX_HEADERS_DIR}" \ > - -Vkernel:debarch="${KERNEL_NAME}" \ > - -DPackage="linux-headers-${KERNEL_NAME}" \ > - -DSection=kernel \ > - -DDepends="${KERNEL_HEADERS_DEBIAN_DEPENDS}" \ > - -DArchitecture=$target_arch > - > -fakeroot dpkg-deb -b "${REPACK_LINUX_IMAGE_DIR}" \ > - linux-image-${KERNEL_NAME}_${PV}-1_${KERNEL_NAME}.deb > -rm -f linux-image-${PV}_${PV}-1_*.deb > -fakeroot dpkg-deb -b "${REPACK_LINUX_HEADERS_DIR}" \ > - linux-headers-${KERNEL_NAME}_${PV}-1_${KERNEL_NAME}.deb > -rm -f linux-headers-${PV}_${PV}-1_*.deb > diff --git a/meta/recipes-kernel/linux/files/debian/compat b/meta/recipes-kernel/linux/files/debian/compat > new file mode 100644 > index 0000000..ec63514 > --- /dev/null > +++ b/meta/recipes-kernel/linux/files/debian/compat > @@ -0,0 +1 @@ > +9 > diff --git a/meta/recipes-kernel/linux/files/debian/control.tmpl b/meta/recipes-kernel/linux/files/debian/control.tmpl > new file mode 100644 > index 0000000..7236732 > --- /dev/null > +++ b/meta/recipes-kernel/linux/files/debian/control.tmpl > @@ -0,0 +1,37 @@ > +Source: linux-${KERNEL_NAME_PROVIDED} > +Section: kernel > +Priority: optional > +Maintainer: ${MAINTAINER} > +Build-Depends: bc, kmod, cpio, ${KBUILD_DEPENDS} > +Homepage: http://www.kernel.org/ > + > +Package: linux-image-${KERNEL_NAME_PROVIDED} > +Architecture: any > +Depends: ${KERNEL_DEBIAN_DEPENDS} > +Description: ${KERNEL_NAME_PROVIDED} Linux kernel, version ${PV}-${KERNEL_LOCALVERSION} > + This package contains the Linux kernel, modules and corresponding other > + files, version: ${PV}-${KERNEL_LOCALVERSION}. > + > +Package: linux-headers-${KERNEL_NAME_PROVIDED} > +Architecture: any > +Depends: ${KERNEL_HEADERS_DEBIAN_DEPENDS}, ${perl:Depends}, ${shlib:Depends} > +Description: ${KERNEL_NAME_PROVIDED} Linux kernel headers for ${PV}-${KERNEL_LOCALVERSION} > + This package provides kernel header files for ${PV}-${KERNEL_LOCALVERSION} on ${DISTRO_ARCH} > + . > + This is useful for people who need to build external modules > + > +Package: linux-libc-dev > +Build-Profiles: <!nolibcdev> > +Section: devel > +Provides: linux-kernel-headers > +Architecture: any > +Description: Linux support headers for userspace development > + This package provides userspaces headers from the Linux kernel. These headers > + are used by the installed headers for GNU glibc and other system libraries. > + > +Package: linux-image-${KERNEL_NAME_PROVIDED}-dbg > +Section: debug > +Architecture: any > +Description: Linux kernel debugging symbols for ${PV}-${KERNEL_LOCALVERSION} > + This package will come in handy if you need to debug the kernel. It provides > + all the necessary debug symbols for the kernel and its modules. > diff --git a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl > new file mode 100644 > index 0000000..1775b19 > --- /dev/null > +++ b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl > @@ -0,0 +1,45 @@ > +#!/bin/bash > +# Copyright (c) Mentor Graphics, a Siemens business, 2019 > +# SPDX-License-Identifier: MIT > + > +# Load common stuff > +. ${S}/debian/isar/common || exit ${?} > + > +do_build() { > + > + # Print a few things that are of particular interest > + print_settings > + > + # Process existing kernel configuration to make sure it is complete > + # (use defaults for options that were not specified) > + ${MAKE} O=${KERNEL_BUILD_DIR} olddefconfig prepare || exit ${?} > + > + # Check if the recipe's PV makes sense > + KR=$(${MAKE} O=${KERNEL_BUILD_DIR} -s --no-print-directory kernelrelease) > + eval $(grep ^CONFIG_LOCALVERSION= ${KERNEL_BUILD_DIR}/${KCONF} || true) > + if [ "${PV}-${KERNEL_LOCALVERSION}" != "${KR}" ]; then > + echo "ERROR: Recipe version (${PV}-${KERNEL_LOCALVERSION}) does not seem to match the kernelrelease (${KR})!" 1>&2 > + echo "ERROR: Make sure the kernel version in your NAME/PV/PR settings and/or CONFIG_LOCALVERSION are aligned" 1>&2 > + exit 1 > + fi > + > + # Trace what we do here > + set -x > + > + # Build the Linux kernel > + ${MAKE} O=${KERNEL_BUILD_DIR} -j $(nproc) || exit ${?} > + > + # Stop tracing > + set +x > +} > + > +print_settings() { > + cat <<EOF > +# Build settings: > +# --------------- > +# ARCH=${ARCH} > +# CROSS_COMPILE=${CROSS_COMPILE} > +EOF > +} > + > +main build ${*} > diff --git a/meta/recipes-kernel/linux/files/debian/isar/clean.tmpl b/meta/recipes-kernel/linux/files/debian/isar/clean.tmpl > new file mode 100644 > index 0000000..f903987 > --- /dev/null > +++ b/meta/recipes-kernel/linux/files/debian/isar/clean.tmpl > @@ -0,0 +1,21 @@ > +#!/bin/bash > +# Copyright (c) Mentor Graphics, a Siemens business, 2019 > +# SPDX-License-Identifier: MIT > + > +# Load common stuff > +. ${S}/debian/isar/common || exit ${?} > + > +do_clean() { > + > + # Trace what we do here > + set -x > + > + rm -rf ${deb_img_dir} ${deb_dbg_dir} > + rm -rf ${deb_kern_hdrdir} > + rm -rf ${deb_libc_hdr_dir} > + > + # Stop tracing > + set +x > +} > + > +main clean ${*} > diff --git a/meta/recipes-kernel/linux/files/debian/isar/common.tmpl b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl > new file mode 100644 > index 0000000..44f67b3 > --- /dev/null > +++ b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl > @@ -0,0 +1,50 @@ > +#!/bin/bash > +# Copyright (c) Mentor Graphics, a Siemens business, 2019 > +# SPDX-License-Identifier: MIT > + > +# Isar settings > +ARCH=${KERNEL_ARCH} > +KERNEL_PKG_IMAGE=linux-image-${KERNEL_NAME_PROVIDED} > +KERNEL_PKG_KERN_HEADERS=linux-headers-${KERNEL_NAME_PROVIDED} > +KERNEL_PKG_LIBC_HEADERS=linux-libc-dev > + > +# Constants > +KCONF=.config > + > +# Target directories > +deb_top_dir=${S}/debian > +deb_img_dir=${deb_top_dir}/${KERNEL_PKG_IMAGE} > +deb_dbg_dir=${deb_img_dir}-dbg > +deb_dtb_dir=${deb_img_dir}/${KERNEL_DTB_DIR} > +deb_kern_hdr_dir=${deb_top_dir}/${KERNEL_PKG_KERN_HEADERS} > +deb_libc_hdr_dir=${deb_top_dir}/${KERNEL_PKG_LIBC_HEADERS} > + > +# Array of packages to be generated > +declare -A kern_pkgs > + > +main() { > + local target=${1} > + > + if [ ! -f ${S}/debian/isar/${target} ]; then > + echo "error: ${target} is not a supported build target!" >&2 > + return 1 > + fi > + > + # create do_<x> variables for each package to be generated > + for p in $(unset DEB_HOST_ARCH; dh_listpackages); do > + kern_pkgs["${p}"]="1" > + done > + > + # variables to be exported > + export ARCH > + > + # are we cross-compiling? > + BUILD_ARCH=$(dpkg-architecture -qDEB_BUILD_ARCH) > + if [ "${BUILD_ARCH}" = "${DISTRO_ARCH}" ]; then > + # no, make sure CROSS_COMPILE isn't set > + unset CROSS_COMPILE > + fi > + > + # call the actual target script > + do_${target} || return ${?} > +} > diff --git a/meta/recipes-kernel/linux/files/debian/isar/install.tmpl b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl > new file mode 100644 > index 0000000..806df5d > --- /dev/null > +++ b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl > @@ -0,0 +1,213 @@ > +#!/bin/bash > +# Copyright (c) Mentor Graphics, a Siemens business, 2019 > +# SPDX-License-Identifier: MIT > + > +# Load common stuff > +. ${S}/debian/isar/common || exit ${?} > + > +# Stop on error > +set -e > + > +do_install() { > + > + # check if our kernel was configured > + if [ ! -f "${O}/.config" ]; then > + echo "error: kernel not configured!" >&2 > + return 1 > + fi > + > + # load its configuration > + . ${O}/.config > + > + kimage="$(${MAKE} O=${O} -s --no-print-directory image_name)" > + krel="$(${MAKE} O=${O} -s --no-print-directory kernelrelease)" > + case "${ARCH}" in > + mips|powerpc) kimage_path="boot/vmlinux-${krel}" ;; > + um) kimage_path="usr/bin/vmlinux-${krel}" ;; > + *) kimage_path="boot/vmlinuz-${krel}" ;; > + esac > + > + print_settings > + > + # Trace what we do here > + set -x > + > + install_image > + install_hooks > + install_dtbs > + install_kmods > + install_headers > + > + # Stop tracing > + set +x > +} > + > +print_settings() { > + cat <<EOF > +Install settings: > +----------------- > +deb_dtb_dir=${deb_dtb_dir} > +deb_hdr_dir=${deb_hdr_dir} > +kimage=${kimage} > +kimage_path=${kimage_path} > + > +EOF > +} > + > +install_image() { > + install -m 755 -d ${deb_img_dir}/$(dirname ${kimage_path}) > + cp ${O}/${kimage} ${deb_img_dir}/${kimage_path} > + > + # Make sure arm64 kernels are decompressed > + if [ "${ARCH}" = "arm64" ]; then > + mv ${deb_img_dir}/${kimage_path} ${deb_img_dir}/${kimage_path}.gz > + gunzip -f ${deb_img_dir}/${kimage_path}.gz > + fi > + > + install_image_debug > +} > + > +install_image_debug() { > + # Different tools want the image in different locations > + # perf > + mkdir -p ${deb_dbg_dir}/usr/lib/debug/lib/modules/${krel}/ > + cp ${O}/vmlinux ${deb_dbg_dir}/usr/lib/debug/lib/modules/${krel}/ > + # systemtap > + mkdir -p ${deb_dbg_dir}/usr/lib/debug/boot/ > + ln -s ../lib/modules/$version/vmlinux ${deb_dbg_dir}/usr/lib/debug/boot/vmlinux-${krel} > + # kdump-tools > + ln -s lib/modules/${krel}/vmlinux ${deb_dbg_dir}/usr/lib/debug/vmlinux-${krel} > +} > + > +install_hooks() { > + install -m 755 -d ${deb_img_dir}/etc/kernel/install.d > + install -m 755 -d ${deb_img_dir}/etc/kernel/postinst.d > + install -m 755 -d ${deb_img_dir}/etc/kernel/postrm.d > + install -m 755 -d ${deb_img_dir}/etc/kernel/prerm.d > + > + initrd="No" > + [ -z "${CONFIG_BLK_DEV_INITRD}" ] || initrd="Yes" > + > + for script in postinst postrm preinst prerm; do > + sed -i -e "s,INITRD=[A-Za-z0-9]*,INITRD=${initrd},g" ${S}/debian/linux-image*.${script} > + sed -i -e "s,version=.*,version=${krel},g" ${S}/debian/linux-image*.${script} > + done > +} > + > +install_dtbs() { > + [ -n "${CONFIG_OF}" ] || return 0 > + ${MAKE} O=${O} INSTALL_DTBS_PATH=${deb_dtb_dir} dtbs_install > +} > + > +install_kmods() { > + [ -n "${CONFIG_MODULES}" ] || return 0 > + ${MAKE} O=${O} INSTALL_MOD_PATH=${deb_img_dir} modules_install > + touch ${deb_img_dir}/lib/modules/${krel}/.fresh-install > + rm -fv ${deb_img_dir}/lib/modules/${krel}/build > + rm -fv ${deb_img_dir}/lib/modules/${krel}/source > + install_kmods_debug > +} > + > +kmods_sign() { > + [ -n "${CONFIG_MODULE_SIG_ALL}" ] || return 0 > + ${MAKE} O=${O} INSTALL_MOD_PATH=${deb_img_dir} modules_sign > +} > + > +install_kmods_debug() { > + [ -n "${CONFIG_DEBUG_INFO}" ] || return 0 > + > + kmod_inst_dir=${deb_img_dir}/lib/modules > + kmod_debug_dir=${deb_dbg_dir}/usr/lib/debug > + > + # copy kernels modules to usr/lib/debug > + mkdir -p ${kmod_debug_dir} > + tar -C ${kmod_inst_dir}/ -cO --exclude='modules.*' . | tar -C ${kmod_debug_dir}/ -xf - > + # strip everything but debug sections for modules in usr/lib/debug > + find ${kmod_debug_dir} -name *.ko -exec ${CROSS_COMPILE}objcopy --only-keep-debug {} \; > + # and strip debug sections from modules in lib/modules > + find ${kmod_inst_dir} -name *.ko -exec ${CROSS_COMPILE}objcopy --strip-debug {} \; > + > + # re-sign stripped kernel modules > + kmods_sign > +} > + > +headers_check() { > + ${MAKE} O=${O} headers_check > +} > + > +libc_headers() { > + mkdir -p ${deb_libc_hdr_dir} > + ${MAKE} O=${O} headers_install INSTALL_HDR_PATH=${deb_libc_hdr_dir}/usr > + host_arch=$(dpkg-architecture -a${DISTRO_ARCH} -qDEB_HOST_MULTIARCH) > + mkdir ${deb_libc_hdr_dir}/usr/include/${host_arch} > + mv ${deb_libc_hdr_dir}/usr/include/asm ${deb_libc_hdr_dir}/usr/include/${host_arch}/ > +} > + > +# the kernel does not seem to provide a way to cross-compile its tools, we however > +# have qemu-user-static binaries in our buildchroot. Create a new build tree for the > +# scripts and override HOSTCC to force a cross-compile > +kernel_tools_build() { > + odir="${S}/build-scripts" > + mflags="HOSTCC=${CROSS_COMPILE}gcc O=${odir} NOSTDINC_FLAGS=" > + rm -rf ${odir} && mkdir -p ${odir} && cp ${S}/${KERNEL_BUILD_DIR}/.config ${odir} > + (unset LD_PRELOAD; cd ${odir} && make -C ${S} ${mflags} olddefconfig scripts) > + (cd ${odir}; find scripts -type f -executable -exec file {} \;|grep ELF|cut -d: -f1) >${src_hdr_files} > + tar -C ${odir} -cf - -T - <${src_hdr_files} | tar -C ${destdir} -xf - > +} > + > +kernel_tools() { > + # remove object files > + find ${destdir}/scripts -type f -name '*.o' |xargs rm -f > + > + # we're all done if we aren't cross-compiling > + [ -n "${CROSS_COMPILE}" ] || return 0 > + > + # remove ELF executables from the linux-headers package > + find ${destdir}/scripts -type f -exec file {} \;|grep ELF|cut -d: -f1|xargs rm -fv > + > + # cross-compile kernel tools to be shipped with linux-headers > + kernel_tools_build > +} > + > +kernel_headers() { > + destdir=${deb_kern_hdr_dir}/${KERNEL_HEADERS_DIR} > + src_hdr_files=$(mktemp) > + obj_hdr_files=$(mktemp) > + > + mkdir -p ${destdir} > + mkdir -p ${deb_kern_hdr_dir}/lib/modules/${krel} > + > + (cd ${S}; find . -name 'Makefile*' -o -name 'Kconfig*' -o -name '*.pl') >>${src_hdr_files} > + (cd ${S}; find arch/*/include include scripts -type f -o -type l) >>${src_hdr_files} > + (cd ${S}; find arch/${ARCH} -name module.lds -o -name Kbuild.platforms -o -name Platform) >>${src_hdr_files} > + (cd ${S}; find $(find arch/${ARCH} -name include -o -name scripts -type d) -type f) >>${src_hdr_files} > + > + (cd ${O}; find arch/${ARCH}/include Module.symvers include scripts -type f) >>${obj_hdr_files} > + if [ -n "${CONFIG_STACK_VALIDATION}" ]; then > + (cd ${O}; find tools/objtool -type f -executable) >>${obj_hdr_files} > + fi > + if [ -n "${CONFIG_GCC_PLUGINS}" ]; then > + (cd ${O}; find scripts/gcc-plugins -name *.so -o -name gcc-common.h) >>${obj_hdr_files} > + fi > + > + # deploy files that were matched above > + tar -C ${S} -cf - -T - <${src_hdr_files} | tar -C ${destdir} -xf - > + tar -C ${O} -cf - -T - <${obj_hdr_files} | tar -C ${destdir} -xf - > + > + # add the kernel config > + cp ${O}/${KCONF} ${destdir}/.config > + > + # handle kernel development tools > + kernel_tools > + > + # create symlinks > + ln -sf /${KERNEL_HEADERS_DIR} ${deb_kern_hdr_dir}/lib/modules/${krel}/build > +} > + > +install_headers() { > + headers_check > + [ -z ${kern_pkgs["linux-libc-headers"]} ] || libc_headers > + kernel_headers > +} > + > +main install ${*} > diff --git a/meta/recipes-kernel/linux/files/debian/isar/version.cfg.tmpl b/meta/recipes-kernel/linux/files/debian/isar/version.cfg.tmpl > new file mode 100644 > index 0000000..09fa364 > --- /dev/null > +++ b/meta/recipes-kernel/linux/files/debian/isar/version.cfg.tmpl > @@ -0,0 +1,2 @@ > +CONFIG_LOCALVERSION="-${KERNEL_LOCALVERSION}" > +# CONFIG_LOCALVERSION_AUTO is not set > diff --git a/meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl b/meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl > new file mode 100644 > index 0000000..b373801 > --- /dev/null > +++ b/meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl > @@ -0,0 +1,29 @@ > +#!/bin/sh > +# based on https://salsa.debian.org/kernel-team/linux/blob/master/debian/templates/image.postinst.in > + > +# Tell initramfs builder whether it's wanted > +export INITRD=Yes > + > +version=${PV}-${KERNEL_LOCALVERSION} > +image_path=/boot/${KERNEL_FILE}-${version} > + > +if [ "$1" != configure ]; then > + exit 0 > +fi > + > +depmod $version > + > +if [ -f /lib/modules/$version/.fresh-install ]; then > + change=install > +else > + change=upgrade > +fi > +linux-update-symlinks $change $version $image_path > +rm -f /lib/modules/$version/.fresh-install > + > +if [ -d /etc/kernel/postinst.d ]; then > + DEB_MAINT_PARAMS="$*" run-parts --report --exit-on-error --arg=$version \ > + --arg=$image_path /etc/kernel/postinst.d > +fi > + > +exit 0 > diff --git a/meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl b/meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl > new file mode 100644 > index 0000000..1b5dbbe > --- /dev/null > +++ b/meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl > @@ -0,0 +1,35 @@ > +#!/bin/sh -e > +# based on https://salsa.debian.org/kernel-team/linux/blob/master/debian/templates/image.postrm.in > + > +# Tell initramfs builder whether it's wanted > +export INITRD=Yes > + > +version=${PV}-${KERNEL_LOCALVERSION} > +image_path=/boot/${KERNEL_FILE}-${version} > + > +rm -f /lib/modules/$version/.fresh-install > + > +if [ "$1" != upgrade ] && command -v linux-update-symlinks >/dev/null; then > + linux-update-symlinks remove $version $image_path > +fi > + > +if [ -d /etc/kernel/postrm.d ]; then > + DEB_MAINT_PARAMS="$*" run-parts --report --exit-on-error --arg=$version \ > + --arg=$image_path /etc/kernel/postrm.d > +fi > + > +if [ "$1" = purge ]; then > + for extra_file in modules.dep modules.isapnpmap modules.pcimap \ > + modules.usbmap modules.parportmap \ > + modules.generic_string modules.ieee1394map \ > + modules.ieee1394map modules.pnpbiosmap \ > + modules.alias modules.ccwmap modules.inputmap \ > + modules.symbols modules.ofmap \ > + modules.seriomap modules.\*.bin \ > + modules.softdep modules.devname; do > + eval rm -f /lib/modules/$version/$extra_file > + done > + rmdir /lib/modules/$version || true > +fi > + > +exit 0 > diff --git a/meta/recipes-kernel/linux/files/debian/linux-image.preinst.tmpl b/meta/recipes-kernel/linux/files/debian/linux-image.preinst.tmpl > new file mode 100644 > index 0000000..4f24019 > --- /dev/null > +++ b/meta/recipes-kernel/linux/files/debian/linux-image.preinst.tmpl > @@ -0,0 +1,25 @@ > +#!/bin/sh -e > +# based on https://salsa.debian.org/kernel-team/linux/blob/master/debian/templates/image.preinst.in > + > +# Tell initramfs builder whether it's wanted > +export INITRD=Yes > + > +version=${PV}-${KERNEL_LOCALVERSION} > +image_path=/boot/${KERNEL_FILE}-${version} > + > +if [ "$1" = abort-upgrade ]; then > + exit 0 > +fi > + > +if [ "$1" = install ]; then > + # Create a flag file for postinst > + mkdir -p /lib/modules/$version > + touch /lib/modules/$version/.fresh-install > +fi > + > +if [ -d /etc/kernel/preinst.d ]; then > + DEB_MAINT_PARAMS="$*" run-parts --report --exit-on-error --arg=$version \ > + --arg=$image_path /etc/kernel/preinst.d > +fi > + > +exit 0 > diff --git a/meta/recipes-kernel/linux/files/debian/linux-image.prerm.tmpl b/meta/recipes-kernel/linux/files/debian/linux-image.prerm.tmpl > new file mode 100644 > index 0000000..cd697c0 > --- /dev/null > +++ b/meta/recipes-kernel/linux/files/debian/linux-image.prerm.tmpl > @@ -0,0 +1,21 @@ > +#!/bin/sh -e > +# based on https://salsa.debian.org/kernel-team/linux/blob/master/debian/templates/image.prerm.in > + > +# Tell initramfs builder whether it's wanted > +export INITRD=Yes > + > +version=${PV}-${KERNEL_LOCALVERSION} > +image_path=/boot/${KERNEL_FILE}-${version} > + > +if [ "$1" != remove ]; then > + exit 0 > +fi > + > +linux-check-removal $version > + > +if [ -d /etc/kernel/prerm.d ]; then > + DEB_MAINT_PARAMS="$*" run-parts --report --exit-on-error --arg=$version \ > + --arg=$image_path /etc/kernel/prerm.d > +fi > + > +exit 0 > diff --git a/meta/recipes-kernel/linux/files/debian/rules.tmpl b/meta/recipes-kernel/linux/files/debian/rules.tmpl > new file mode 100755 > index 0000000..c024768 > --- /dev/null > +++ b/meta/recipes-kernel/linux/files/debian/rules.tmpl > @@ -0,0 +1,39 @@ > +#!/usr/bin/make -f > + > +CROSS_COMPILE:=$(DEB_HOST_GNU_TYPE)- > + > +O:=$(CURDIR)/${KERNEL_BUILD_DIR} > +S:=$(CURDIR) > +deb_top_dir:=$(S)/debian > + > +# Dynamic variables to be passed to Isar build scripts > +isar_env=$(strip \ > + export CROSS_COMPILE='$(CROSS_COMPILE)' && \ > + export DEB_BUILD_PROFILES='$(DEB_BUILD_PROFILES)' && \ > + export DEB_HOST_GNU_TYPE='$(DEB_HOST_GNU_TYPE)' && \ > + export MAKE='$(MAKE)' && \ > + export O='${O}' \ > + export S='${S}' \ > +) > + > +%: > + dh $(@) > + > +.PHONY: override_dh_auto_clean > +override_dh_auto_clean: > + $(isar_env) && bash $(deb_top_dir)/isar/clean > + > +.PHONY: override_dh_auto_build > +override_dh_auto_build: > + $(isar_env) && bash $(deb_top_dir)/isar/build > + > +override_dh_auto_install: > + $(isar_env) && bash $(deb_top_dir)/isar/install > + > +.PHONY: override_dh_auto_test > +override_dh_auto_test: > + true > + > +PHONY: override_dh_strip > +override_dh_strip: > + dh_strip -Xvmlinux --no-automatic-dbgsym > diff --git a/meta/recipes-kernel/linux/linux-custom.inc b/meta/recipes-kernel/linux/linux-custom.inc > index 712f52b..bf1646a 100644 > --- a/meta/recipes-kernel/linux/linux-custom.inc > +++ b/meta/recipes-kernel/linux/linux-custom.inc > @@ -2,89 +2,183 @@ > # > # This software is a part of ISAR. > # Copyright (c) Siemens AG, 2018 > +# Copyright (c) Mentor Graphics, a Siemens business, 2019 > # > # SPDX-License-Identifier: MIT > > -FILESEXTRAPATHS_prepend := "${FILE_DIRNAME}/files:" > +# Settings that would typically be done from the custom kernel recipe > +# ------------------------------------------------------------------- > > +CHANGELOG_V = "${PV}+${PR}" > DESCRIPTION ?= "Custom kernel" > +MAINTAINER ?= "isar-users <isar-users@googlegroups.com>" > + > +KBUILD_DEPENDS ?= "build-essential:native, \ > + libelf-dev:native, \ > + libncurses-dev:native, \ > + libssl-dev:native, \ > + bc, \ > + bison, \ > + cpio, \ > + flex, \ > + git, \ > + kmod, \ > + libssl-dev," > + > +KERNEL_DEBIAN_DEPENDS ?= "initramfs-tools | linux-initramfs-tool, \ > + kmod, \ > + linux-base (>= 4.3~)," > + > +KERNEL_HEADERS_DEBIAN_DEPENDS ?= "libc6, \ > + libssl1.1," > + > +KERNEL_LIBC_DEV_DEPLOY ?= "0" > + > +# Settings that may be changed on a per distro, machine or layer basis > +# -------------------------------------------------------------------- > + > +KERNEL_LOCALVERSION ?= "-${KERNEL_NAME_PROVIDED}-${PR}" > +KERNEL_DTB_DIR ?= "usr/lib/linux-image-${PV}-${KERNEL_LOCALVERSION}" > +KERNEL_HEADERS_DIR ?= "usr/src/linux-headers-${PV}-${KERNEL_LOCALVERSION}" > > -KERNEL_NAME_PROVIDED ?= "${@ d.getVar('PN', True).partition('linux-')[2]}" > KERNEL_DEFCONFIG ?= "" > > +# Add our template meta-data to the sources > +FILESEXTRAPATHS_prepend := "${FILE_DIRNAME}/files:" > +SRC_URI += "file://debian" > + > +# Variables and files that make our templates > +# ------------------------------------------- > + > +TEMPLATE_FILES += " \ > + debian/control.tmpl \ > + debian/isar/build.tmpl \ > + debian/isar/clean.tmpl \ > + debian/isar/common.tmpl \ > + debian/isar/install.tmpl \ > + debian/isar/version.cfg.tmpl \ > + debian/linux-image.postinst.tmpl \ > + debian/linux-image.postrm.tmpl \ > + debian/linux-image.preinst.tmpl \ > + debian/linux-image.prerm.tmpl \ > + debian/rules.tmpl \ > +" > + > +TEMPLATE_VARS += " \ > + HOST_ARCH \ > + KBUILD_DEPENDS \ > + KERNEL_ARCH \ > + KERNEL_DEBIAN_DEPENDS \ > + KERNEL_BUILD_DIR \ > + KERNEL_DTB_DIR \ > + KERNEL_FILE \ > + KERNEL_HEADERS_DEBIAN_DEPENDS \ > + KERNEL_HEADERS_DIR \ > + KERNEL_LOCALVERSION \ > + KERNEL_NAME_PROVIDED \ > +" > + > +inherit dpkg > +inherit template > + > +# Derive name of the kernel packages from the name of this recipe > +KERNEL_NAME_PROVIDED ?= "${@ d.getVar('PN', True).partition('linux-')[2]}" > + > +# Make bitbake know we will be producing linux-image and linux-headers packages > python() { > kernel_name = d.getVar("KERNEL_NAME_PROVIDED", True) > d.setVar('PROVIDES', 'linux-image-' + kernel_name + ' ' + \ > 'linux-headers-' + kernel_name) > } > > -inherit dpkg-base > - > -SRC_URI += "file://build-kernel.sh" > - > -KBUILD_DEPENDS ?= " \ > - build-essential:native \ > - libssl-dev \ > - libelf-dev \ > - bc \ > - git \ > - kmod \ > - bison \ > - flex \ > - cpio \ > - libncurses-dev" > -KERNEL_DEBIAN_DEPENDS ?= "initramfs-tools | linux-initramfs-tool, kmod, linux-base (>= 4.3~)" > -KERNEL_HEADERS_DEBIAN_DEPENDS ?= "libc6, libssl1.1" > - > -KERNEL_LIBC_DEV_DEPLOY ?= "0" > - > -do_install_builddeps() { > - dpkg_do_mounts > - E="${@ isar_export_proxies(d)}" > - sudo -E chroot ${BUILDCHROOT_DIR} \ > - apt-get update \ > - -o Dir::Etc::SourceList="sources.list.d/isar-apt.list" \ > - -o Dir::Etc::SourceParts="-" \ > - -o APT::Get::List-Cleanup="0" > - sudo -E chroot ${BUILDCHROOT_DIR} \ > - apt-get install \ > - -y -o Debug::pkgProblemResolver=yes \ > - --no-install-recommends ${KBUILD_DEPENDS} > - dpkg_undo_mounts > +def get_kernel_arch(d): > + distro_arch = d.getVar("DISTRO_ARCH") > + if distro_arch == "amd64": > + kernel_arch = "x86" > + elif distro_arch == "arm64": > + kernel_arch = "arm64" > + elif distro_arch == "armhf": > + kernel_arch = "arm" > + elif distro_arch == "mipsel": > + kernel_arch = "mips" > + else: > + kernel_arch = "" > + return kernel_arch > + > +KERNEL_ARCH ??= "${@get_kernel_arch(d)}" > + > +def config_fragments(d): > + fragments = [] > + sources = d.getVar("SRC_URI").split() > + for s in sources: > + _, _, local, _, _, parm = bb.fetch.decodeurl(s) > + apply = parm.get("apply") > + if apply == "no": > + continue > + base, ext = os.path.splitext(os.path.basename(local)) > + if ext and ext in (".cfg"): > + fragments.append(local) > + return fragments > + > +do_prepare_build_prepend() { > + # copy meta-data over to source tree > + rm -rf ${S}/debian > + cp -r ${WORKDIR}/debian ${S}/ > + > + # remove templates from the source tree > + find ${S}/debian -name *.tmpl | xargs rm -f > + > + # rename install/remove hooks to match user-specified name for our linux-image package > + mv ${S}/debian/linux-image.postinst ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.postinst > + mv ${S}/debian/linux-image.postrm ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.postrm > + mv ${S}/debian/linux-image.preinst ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.preinst > + mv ${S}/debian/linux-image.prerm ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.prerm > + > + # produce a changelog for our kernel build > + deb_add_changelog > } > > -addtask install_builddeps after do_prepare_build before do_dpkg_build > -# apt and reprepro may not run in parallel, acquire the Isar lock > -do_install_builddeps[lockfiles] += "${REPO_ISAR_DIR}/isar.lock" > - > -addtask devshell after do_install_builddeps > +# build directory for our "full" kernel build > +KERNEL_BUILD_DIR = "build-full" > > -dpkg_runbuild() { > - chmod +x ${WORKDIR}/build-kernel.sh > - KERNEL_CONFIG_TARGET="${KERNEL_DEFCONFIG}" > +dpkg_configure_kernel() { > + config_target="${KERNEL_DEFCONFIG}" > + rm -rf ${S}/${KERNEL_BUILD_DIR} && mkdir -p ${S}/${KERNEL_BUILD_DIR} > if [ -n "${KERNEL_DEFCONFIG}" ]; then > if [ -e "${WORKDIR}/${KERNEL_DEFCONFIG}" ]; then > - cp ${WORKDIR}/${KERNEL_DEFCONFIG} ${S}/.config > - KERNEL_CONFIG_TARGET="olddefconfig" > + cp ${WORKDIR}/${KERNEL_DEFCONFIG} ${S}/${KERNEL_BUILD_DIR}/.config > + config_target="olddefconfig" > fi > else > - KERNEL_CONFIG_TARGET="defconfig" > + config_target="defconfig" > fi > > - E="${@ isar_export_proxies(d)}" > - > - export PV=${PV} > - export KERNEL_NAME=${KERNEL_NAME_PROVIDED} > - > - export KBUILD_DEPENDS="${KBUILD_DEPENDS}" > - export KERNEL_CONFIG_TARGET > - export KERNEL_DEBIAN_DEPENDS="${KERNEL_DEBIAN_DEPENDS}" > - export KERNEL_FILE="${KERNEL_FILE}" > - export KERNEL_HEADERS_DEBIAN_DEPENDS="${KERNEL_HEADERS_DEBIAN_DEPENDS}" > + # copy config fragments over to the kernel tree > + src_frags="${@ " ".join(config_fragments(d)) }" > + out_frags="${@ " ".join(map(lambda frag: 'debian/fragments/' + frag, config_fragments(d))) }" > + rm -rf ${S}/debian/fragments > + if [ -n "${src_frags}" ]; then > + mkdir -p ${S}/debian/fragments > + (cd ${WORKDIR} && cp ${src_frags} ${S}/debian/fragments/) > + fi > > - sudo -E chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} ${PP}/build-kernel.sh ${PP}/${PPS} ${DISTRO_ARCH} > + sudo -E chroot --userspec=$(id -u):$(id -g) ${BUILDCHROOT_DIR} sh -c " \ > + export ARCH=${KERNEL_ARCH} && \ > + cd ${PP}/${PPS} && \ > + make O=${KERNEL_BUILD_DIR} ${config_target} && \ > + ./scripts/kconfig/merge_config.sh \ > + -O ${KERNEL_BUILD_DIR}/ \ > + ${KERNEL_BUILD_DIR}/.config \ > + ${out_frags} \ > + debian/isar/version.cfg \ > + " > +} > > +dpkg_runbuild_prepend() { > + profiles="${BASE_DISTRO}" > if [ "${KERNEL_LIBC_DEV_DEPLOY}" != "1" ]; then > - rm -f ${WORKDIR}/linux-libc-dev_${PV}*.deb > + profiles="${profiles} nolibcdev" > fi > + export DEB_BUILD_PROFILES="${profiles}" > + dpkg_configure_kernel > } > I would recommend this functional change to the patch to get away without patch 4: diff --git a/meta/recipes-kernel/linux/files/debian/isar/install.tmpl b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl index 806df5d..67b7ce3 100644 --- a/meta/recipes-kernel/linux/files/debian/isar/install.tmpl +++ b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl @@ -143,30 +143,9 @@ libc_headers() { mv ${deb_libc_hdr_dir}/usr/include/asm ${deb_libc_hdr_dir}/usr/include/${host_arch}/ } -# the kernel does not seem to provide a way to cross-compile its tools, we however -# have qemu-user-static binaries in our buildchroot. Create a new build tree for the -# scripts and override HOSTCC to force a cross-compile -kernel_tools_build() { - odir="${S}/build-scripts" - mflags="HOSTCC=${CROSS_COMPILE}gcc O=${odir} NOSTDINC_FLAGS=" - rm -rf ${odir} && mkdir -p ${odir} && cp ${S}/${KERNEL_BUILD_DIR}/.config ${odir} - (unset LD_PRELOAD; cd ${odir} && make -C ${S} ${mflags} olddefconfig scripts) - (cd ${odir}; find scripts -type f -executable -exec file {} \;|grep ELF|cut -d: -f1) >${src_hdr_files} - tar -C ${odir} -cf - -T - <${src_hdr_files} | tar -C ${destdir} -xf - -} - kernel_tools() { # remove object files find ${destdir}/scripts -type f -name '*.o' |xargs rm -f - - # we're all done if we aren't cross-compiling - [ -n "${CROSS_COMPILE}" ] || return 0 - - # remove ELF executables from the linux-headers package - find ${destdir}/scripts -type f -exec file {} \;|grep ELF|cut -d: -f1|xargs rm -fv - - # cross-compile kernel tools to be shipped with linux-headers - kernel_tools_build } kernel_headers() { diff --git a/meta/recipes-kernel/linux/files/debian/rules.tmpl b/meta/recipes-kernel/linux/files/debian/rules.tmpl index c024768..bea21f9 100755 --- a/meta/recipes-kernel/linux/files/debian/rules.tmpl +++ b/meta/recipes-kernel/linux/files/debian/rules.tmpl @@ -36,4 +36,4 @@ override_dh_auto_test: PHONY: override_dh_strip override_dh_strip: - dh_strip -Xvmlinux --no-automatic-dbgsym + unset DEB_HOST_GNU_TYPE && dh_strip -Xvmlinux --no-automatic-dbgsym Then we can introduce the correct building of linux-headers once we can generate both host and target packages with Isar. And we do not need to infect the host buildchroot. Jan -- Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 83+ messages in thread
* Re: [PATCH L-C v4 5/5] linux-custom: rewrite to no longer depend on the kernel's builddeb 2019-11-25 19:55 ` Jan Kiszka @ 2019-11-26 7:42 ` Cedric Hombourger 2019-11-26 9:13 ` Jan Kiszka 0 siblings, 1 reply; 83+ messages in thread From: Cedric Hombourger @ 2019-11-26 7:42 UTC (permalink / raw) To: Jan Kiszka, isar-users On 11/25/2019 8:55 PM, Jan Kiszka wrote: > On 25.11.19 17:06, Cedric Hombourger wrote: >> Regain control over the packaging of the Linux kernel by providing our >> own debian recipes (debian,{control,rules} files and friends) instead >> of using the kernel's builddeb script. This will allow generation of >> packages for "perf" (not included in this changeset) with per distro >> scheme (Debian and Ubuntu have differences) and maintain compatibility >> with upstream packages generated from different source packages but >> dependent on a certain deployment scheme (e.g. linux-base for Debian) >> >> Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com> >> --- >> doc/custom_kernel.md | 80 +++++++ >> .../linux/files/build-kernel.sh | 125 ---------- >> meta/recipes-kernel/linux/files/debian/compat | 1 + >> .../linux/files/debian/control.tmpl | 37 +++ >> .../linux/files/debian/isar/build.tmpl | 45 ++++ >> .../linux/files/debian/isar/clean.tmpl | 21 ++ >> .../linux/files/debian/isar/common.tmpl | 50 ++++ >> .../linux/files/debian/isar/install.tmpl | 213 ++++++++++++++++++ >> .../linux/files/debian/isar/version.cfg.tmpl | 2 + >> .../files/debian/linux-image.postinst.tmpl | 29 +++ >> .../files/debian/linux-image.postrm.tmpl | 35 +++ >> .../files/debian/linux-image.preinst.tmpl | 25 ++ >> .../linux/files/debian/linux-image.prerm.tmpl | 21 ++ >> .../linux/files/debian/rules.tmpl | 39 ++++ >> meta/recipes-kernel/linux/linux-custom.inc | 210 ++++++++++++----- >> 15 files changed, 750 insertions(+), 183 deletions(-) >> create mode 100644 doc/custom_kernel.md >> delete mode 100644 meta/recipes-kernel/linux/files/build-kernel.sh >> create mode 100644 meta/recipes-kernel/linux/files/debian/compat >> create mode 100644 meta/recipes-kernel/linux/files/debian/control.tmpl >> create mode 100644 meta/recipes-kernel/linux/files/debian/isar/build.tmpl >> create mode 100644 meta/recipes-kernel/linux/files/debian/isar/clean.tmpl >> create mode 100644 meta/recipes-kernel/linux/files/debian/isar/common.tmpl >> create mode 100644 meta/recipes-kernel/linux/files/debian/isar/install.tmpl >> create mode 100644 meta/recipes-kernel/linux/files/debian/isar/version.cfg.tmpl >> create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl >> create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl >> create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.preinst.tmpl >> create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.prerm.tmpl >> create mode 100755 meta/recipes-kernel/linux/files/debian/rules.tmpl >> >> diff --git a/doc/custom_kernel.md b/doc/custom_kernel.md >> new file mode 100644 >> index 0000000..31319d4 >> --- /dev/null >> +++ b/doc/custom_kernel.md >> @@ -0,0 +1,80 @@ >> +# Custom kernel recipe for Isar >> + >> +## Contents >> + >> + - [Summary](#summary) >> + - [Features](#features) >> + - [Future](#future) >> + - [Examples](#examples) >> + >> +## Summary >> + >> +Isar provides a recipe to build custom kernels for Debian-based distributions. >> +It uses templates to generate the debian meta-data (such as debian/control) and >> +Debian's [BuildProfiles](https://wiki.debian.org/BuildProfileSpec) to handle >> +some of the distro specific variations. It should be noted that Isar has moved >> +away from using the kernel's builddeb script since it would not generate all >> +the packages we need (and in particular perf). >> + >> +## Features >> + >> +The linux-custom recipe provides support for: >> + >> + 1. Sources to the custom Linux kernel may be specified via `SRC_URI` >> + >> + 2. Configure the kernel via an in-tree or an external `defconfig` via >> + `KERNEL_DEFCONFIG` >> + >> + 3. Integrate kernel configuration tweaks via configuration fragments (`.cfg` >> + files) >> + >> + 4. Patches to the linux kernel may be specified via `SRC_URI` >> + >> + 5. Ensure that the Isar recipe `PV` matches the kernel release identifier >> + (`KERNEL_RELEASE`) >> + >> + 6. Produce a `linux-image` package that ships the kernel image and modules >> + >> + 7. Allow the name of the kernel image to be changed via `KERNEL_FILE` (defaults >> + to `vmlinuz`) >> + >> + 8. Produce a `linux-headers` package which includes kernel headers and kbuild >> + scripts/tools >> + >> + 9. The `linux-headers` package shall support native and cross compiles of >> + out-of-tree kernel modules >> + >> + 10. Produce a `linux-libc-dev` package to support user-land builds >> + >> + 11. Only build/ship the `linux-libc-dev` package if instructed to >> + (`KERNEL_LIBC_DEV_DEPLOY` equals to `"1"`) >> + >> + 12. Support both native and cross compiles (`ISAR_CROSS_COMPILE`) >> + >> + 13. Support for the following kernel architectures: >> + >> + * arm >> + * arm64 >> + * mips >> + * x86 >> + >> + 14. Support `devshell` (kernel configuration shall be applied) >> + >> +## Future >> + >> +In the future, the recipe may be extended to: >> + >> + 1. Package perf >> + >> + 2. Support inclusion/build of dts files listed in `SRC_URI` >> + >> + 3. Be compatible with Ubuntu >> + >> +## Examples >> + >> +The linux-custom recipe is currently used by the linux-mainline package and is >> +used mainline recipe may be used for some basic testing. This recipe is being >> +used by the following machines: >> + >> + * de0-nano-soc >> + * qemumipsel >> diff --git a/meta/recipes-kernel/linux/files/build-kernel.sh b/meta/recipes-kernel/linux/files/build-kernel.sh >> deleted file mode 100644 >> index 386e58b..0000000 >> --- a/meta/recipes-kernel/linux/files/build-kernel.sh >> +++ /dev/null >> @@ -1,125 +0,0 @@ >> -#!/bin/bash >> -# >> -# Custom kernel build >> -# >> -# This software is a part of ISAR. >> -# Copyright (c) Siemens AG, 2018 >> -# >> -# SPDX-License-Identifier: MIT >> - >> -source /isar/common.sh >> - >> -host_arch=$(dpkg --print-architecture) >> - >> -if [ "$host_arch" != "$target_arch" ]; then >> - case $target_arch in >> - armhf) >> - export ARCH=arm >> - export CROSS_COMPILE="arm-linux-gnueabihf-" >> - ;; >> - arm64) >> - export ARCH=arm64 >> - export CROSS_COMPILE="aarch64-linux-gnu-" >> - ;; >> - mipsel) >> - export ARCH=mips >> - export CROSS_COMPILE="mipsel-linux-gnu-" >> - ;; >> - *) >> - echo "error: unsupported architecture ($target_arch)" >> - exit 1 >> - ;; >> - esac >> -fi >> - >> -REPACK_DIR="$1/../repack" >> -REPACK_LINUX_IMAGE_DIR="${REPACK_DIR}/linux-image" >> -REPACK_LINUX_HEADERS_DIR="${REPACK_DIR}/linux-headers" >> - >> -make ${KERNEL_CONFIG_TARGET} || exit ${?} >> - >> -KV=$( make -s kernelrelease ) >> -if [ "${KV}" != "${PV}" ]; then >> - echo "ERROR: Recipe PV is \"${PV}\" but should be \"${KV}\"" 1>&2 >> - echo "ERROR: Probably due to CONFIG_LOCALVERSION" 1>&2 >> - exit 1 >> -fi >> - >> -rm -f .version >> -KBUILD_DEBARCH=$target_arch make -j $(($(nproc) * 2)) deb-pkg >> - >> -rm -rf "${REPACK_DIR}" >> -mkdir -p "${REPACK_DIR}" >> -mkdir -p "${REPACK_LINUX_IMAGE_DIR}" >> -mkdir -p "${REPACK_LINUX_HEADERS_DIR}" >> - >> -cp -a debian "${REPACK_DIR}" >> - >> -# dpkg-gencontrol performs cross-incompatible checks on the >> -# Architecture field; trick it to accept the control file >> -sed -i "s/Architecture: .*/Architecture: any/" "${REPACK_DIR}/debian/control" >> - >> -cd .. >> - >> -dpkg-deb -R linux-image-${PV}_${PV}-1_*.deb "${REPACK_LINUX_IMAGE_DIR}" >> -dpkg-deb -R linux-headers-${PV}_${PV}-1_*.deb "${REPACK_LINUX_HEADERS_DIR}" >> - >> -dpkg-gencontrol -crepack/debian/control \ >> - -lrepack/debian/changelog \ >> - -frepack/debian/files \ >> - -plinux-image-${PV} \ >> - -P"${REPACK_LINUX_IMAGE_DIR}" \ >> - -DPackage="linux-image-${KERNEL_NAME}" \ >> - -DSection=kernel \ >> - -DPriority=required \ >> - -DDepends="${KERNEL_DEBIAN_DEPENDS}" \ >> - -DArchitecture=$target_arch >> - >> -# Add Debian-like link installation to postinst >> -mkdir -p ${REPACK_LINUX_IMAGE_DIR}/lib/modules/${PV} >> -touch "${REPACK_LINUX_IMAGE_DIR}/lib/modules/${PV}/.fresh-install" >> -sed -i "${REPACK_LINUX_IMAGE_DIR}/DEBIAN/postinst" \ >> - -e "/^set -e$/a\\ >> -\\ >> -if [ -f /lib/modules/${PV}/.fresh-install ]; then\\ >> - change=install\\ >> -else\\ >> - change=upgrade\\ >> -fi\\ >> -linux-update-symlinks \$change ${PV} /boot/${KERNEL_FILE}-${PV}\\ >> -rm -f /lib/modules/${PV}/.fresh-install" >> - >> -# Add Debian-like link removal to postrm >> -sed -i "${REPACK_LINUX_IMAGE_DIR}/DEBIAN/postrm" \ >> - -e "/^set -e$/a\\ >> -\\ >> -rm -f /lib/modules/${PV}/.fresh-install\\ >> -\\ >> -if [ \"\$1\" != upgrade ] && command -v linux-update-symlinks >/dev/null; then\\ >> - linux-update-symlinks remove ${PV} /boot/${KERNEL_FILE}-${PV}\\ >> -fi" >> - >> -# Make sure arm64 kernels are decompressed >> -if [ "$target_arch" = "arm64" ]; then >> - kernel_file="${REPACK_LINUX_IMAGE_DIR}/boot/${KERNEL_FILE}-${PV}" >> - mv "${kernel_file}" "${kernel_file}.gz" >> - gunzip "${kernel_file}.gz" >> -fi >> - >> -dpkg-gencontrol -crepack/debian/control \ >> - -lrepack/debian/changelog \ >> - -frepack/debian/files \ >> - -plinux-headers-${PV} \ >> - -P"${REPACK_LINUX_HEADERS_DIR}" \ >> - -Vkernel:debarch="${KERNEL_NAME}" \ >> - -DPackage="linux-headers-${KERNEL_NAME}" \ >> - -DSection=kernel \ >> - -DDepends="${KERNEL_HEADERS_DEBIAN_DEPENDS}" \ >> - -DArchitecture=$target_arch >> - >> -fakeroot dpkg-deb -b "${REPACK_LINUX_IMAGE_DIR}" \ >> - linux-image-${KERNEL_NAME}_${PV}-1_${KERNEL_NAME}.deb >> -rm -f linux-image-${PV}_${PV}-1_*.deb >> -fakeroot dpkg-deb -b "${REPACK_LINUX_HEADERS_DIR}" \ >> - linux-headers-${KERNEL_NAME}_${PV}-1_${KERNEL_NAME}.deb >> -rm -f linux-headers-${PV}_${PV}-1_*.deb >> diff --git a/meta/recipes-kernel/linux/files/debian/compat b/meta/recipes-kernel/linux/files/debian/compat >> new file mode 100644 >> index 0000000..ec63514 >> --- /dev/null >> +++ b/meta/recipes-kernel/linux/files/debian/compat >> @@ -0,0 +1 @@ >> +9 >> diff --git a/meta/recipes-kernel/linux/files/debian/control.tmpl b/meta/recipes-kernel/linux/files/debian/control.tmpl >> new file mode 100644 >> index 0000000..7236732 >> --- /dev/null >> +++ b/meta/recipes-kernel/linux/files/debian/control.tmpl >> @@ -0,0 +1,37 @@ >> +Source: linux-${KERNEL_NAME_PROVIDED} >> +Section: kernel >> +Priority: optional >> +Maintainer: ${MAINTAINER} >> +Build-Depends: bc, kmod, cpio, ${KBUILD_DEPENDS} >> +Homepage: http://www.kernel.org/ >> + >> +Package: linux-image-${KERNEL_NAME_PROVIDED} >> +Architecture: any >> +Depends: ${KERNEL_DEBIAN_DEPENDS} >> +Description: ${KERNEL_NAME_PROVIDED} Linux kernel, version ${PV}-${KERNEL_LOCALVERSION} >> + This package contains the Linux kernel, modules and corresponding other >> + files, version: ${PV}-${KERNEL_LOCALVERSION}. >> + >> +Package: linux-headers-${KERNEL_NAME_PROVIDED} >> +Architecture: any >> +Depends: ${KERNEL_HEADERS_DEBIAN_DEPENDS}, ${perl:Depends}, ${shlib:Depends} >> +Description: ${KERNEL_NAME_PROVIDED} Linux kernel headers for ${PV}-${KERNEL_LOCALVERSION} >> + This package provides kernel header files for ${PV}-${KERNEL_LOCALVERSION} on ${DISTRO_ARCH} >> + . >> + This is useful for people who need to build external modules >> + >> +Package: linux-libc-dev >> +Build-Profiles: <!nolibcdev> >> +Section: devel >> +Provides: linux-kernel-headers >> +Architecture: any >> +Description: Linux support headers for userspace development >> + This package provides userspaces headers from the Linux kernel. These headers >> + are used by the installed headers for GNU glibc and other system libraries. >> + >> +Package: linux-image-${KERNEL_NAME_PROVIDED}-dbg >> +Section: debug >> +Architecture: any >> +Description: Linux kernel debugging symbols for ${PV}-${KERNEL_LOCALVERSION} >> + This package will come in handy if you need to debug the kernel. It provides >> + all the necessary debug symbols for the kernel and its modules. >> diff --git a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl >> new file mode 100644 >> index 0000000..1775b19 >> --- /dev/null >> +++ b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl >> @@ -0,0 +1,45 @@ >> +#!/bin/bash >> +# Copyright (c) Mentor Graphics, a Siemens business, 2019 >> +# SPDX-License-Identifier: MIT >> + >> +# Load common stuff >> +. ${S}/debian/isar/common || exit ${?} >> + >> +do_build() { >> + >> + # Print a few things that are of particular interest >> + print_settings >> + >> + # Process existing kernel configuration to make sure it is complete >> + # (use defaults for options that were not specified) >> + ${MAKE} O=${KERNEL_BUILD_DIR} olddefconfig prepare || exit ${?} >> + >> + # Check if the recipe's PV makes sense >> + KR=$(${MAKE} O=${KERNEL_BUILD_DIR} -s --no-print-directory kernelrelease) >> + eval $(grep ^CONFIG_LOCALVERSION= ${KERNEL_BUILD_DIR}/${KCONF} || true) >> + if [ "${PV}-${KERNEL_LOCALVERSION}" != "${KR}" ]; then >> + echo "ERROR: Recipe version (${PV}-${KERNEL_LOCALVERSION}) does not seem to match the kernelrelease (${KR})!" 1>&2 >> + echo "ERROR: Make sure the kernel version in your NAME/PV/PR settings and/or CONFIG_LOCALVERSION are aligned" 1>&2 >> + exit 1 >> + fi >> + >> + # Trace what we do here >> + set -x >> + >> + # Build the Linux kernel >> + ${MAKE} O=${KERNEL_BUILD_DIR} -j $(nproc) || exit ${?} >> + >> + # Stop tracing >> + set +x >> +} >> + >> +print_settings() { >> + cat <<EOF >> +# Build settings: >> +# --------------- >> +# ARCH=${ARCH} >> +# CROSS_COMPILE=${CROSS_COMPILE} >> +EOF >> +} >> + >> +main build ${*} >> diff --git a/meta/recipes-kernel/linux/files/debian/isar/clean.tmpl b/meta/recipes-kernel/linux/files/debian/isar/clean.tmpl >> new file mode 100644 >> index 0000000..f903987 >> --- /dev/null >> +++ b/meta/recipes-kernel/linux/files/debian/isar/clean.tmpl >> @@ -0,0 +1,21 @@ >> +#!/bin/bash >> +# Copyright (c) Mentor Graphics, a Siemens business, 2019 >> +# SPDX-License-Identifier: MIT >> + >> +# Load common stuff >> +. ${S}/debian/isar/common || exit ${?} >> + >> +do_clean() { >> + >> + # Trace what we do here >> + set -x >> + >> + rm -rf ${deb_img_dir} ${deb_dbg_dir} >> + rm -rf ${deb_kern_hdrdir} >> + rm -rf ${deb_libc_hdr_dir} >> + >> + # Stop tracing >> + set +x >> +} >> + >> +main clean ${*} >> diff --git a/meta/recipes-kernel/linux/files/debian/isar/common.tmpl b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl >> new file mode 100644 >> index 0000000..44f67b3 >> --- /dev/null >> +++ b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl >> @@ -0,0 +1,50 @@ >> +#!/bin/bash >> +# Copyright (c) Mentor Graphics, a Siemens business, 2019 >> +# SPDX-License-Identifier: MIT >> + >> +# Isar settings >> +ARCH=${KERNEL_ARCH} >> +KERNEL_PKG_IMAGE=linux-image-${KERNEL_NAME_PROVIDED} >> +KERNEL_PKG_KERN_HEADERS=linux-headers-${KERNEL_NAME_PROVIDED} >> +KERNEL_PKG_LIBC_HEADERS=linux-libc-dev >> + >> +# Constants >> +KCONF=.config >> + >> +# Target directories >> +deb_top_dir=${S}/debian >> +deb_img_dir=${deb_top_dir}/${KERNEL_PKG_IMAGE} >> +deb_dbg_dir=${deb_img_dir}-dbg >> +deb_dtb_dir=${deb_img_dir}/${KERNEL_DTB_DIR} >> +deb_kern_hdr_dir=${deb_top_dir}/${KERNEL_PKG_KERN_HEADERS} >> +deb_libc_hdr_dir=${deb_top_dir}/${KERNEL_PKG_LIBC_HEADERS} >> + >> +# Array of packages to be generated >> +declare -A kern_pkgs >> + >> +main() { >> + local target=${1} >> + >> + if [ ! -f ${S}/debian/isar/${target} ]; then >> + echo "error: ${target} is not a supported build target!" >&2 >> + return 1 >> + fi >> + >> + # create do_<x> variables for each package to be generated >> + for p in $(unset DEB_HOST_ARCH; dh_listpackages); do >> + kern_pkgs["${p}"]="1" >> + done >> + >> + # variables to be exported >> + export ARCH >> + >> + # are we cross-compiling? >> + BUILD_ARCH=$(dpkg-architecture -qDEB_BUILD_ARCH) >> + if [ "${BUILD_ARCH}" = "${DISTRO_ARCH}" ]; then >> + # no, make sure CROSS_COMPILE isn't set >> + unset CROSS_COMPILE >> + fi >> + >> + # call the actual target script >> + do_${target} || return ${?} >> +} >> diff --git a/meta/recipes-kernel/linux/files/debian/isar/install.tmpl b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl >> new file mode 100644 >> index 0000000..806df5d >> --- /dev/null >> +++ b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl >> @@ -0,0 +1,213 @@ >> +#!/bin/bash >> +# Copyright (c) Mentor Graphics, a Siemens business, 2019 >> +# SPDX-License-Identifier: MIT >> + >> +# Load common stuff >> +. ${S}/debian/isar/common || exit ${?} >> + >> +# Stop on error >> +set -e >> + >> +do_install() { >> + >> + # check if our kernel was configured >> + if [ ! -f "${O}/.config" ]; then >> + echo "error: kernel not configured!" >&2 >> + return 1 >> + fi >> + >> + # load its configuration >> + . ${O}/.config >> + >> + kimage="$(${MAKE} O=${O} -s --no-print-directory image_name)" >> + krel="$(${MAKE} O=${O} -s --no-print-directory kernelrelease)" >> + case "${ARCH}" in >> + mips|powerpc) kimage_path="boot/vmlinux-${krel}" ;; >> + um) kimage_path="usr/bin/vmlinux-${krel}" ;; >> + *) kimage_path="boot/vmlinuz-${krel}" ;; >> + esac >> + >> + print_settings >> + >> + # Trace what we do here >> + set -x >> + >> + install_image >> + install_hooks >> + install_dtbs >> + install_kmods >> + install_headers >> + >> + # Stop tracing >> + set +x >> +} >> + >> +print_settings() { >> + cat <<EOF >> +Install settings: >> +----------------- >> +deb_dtb_dir=${deb_dtb_dir} >> +deb_hdr_dir=${deb_hdr_dir} >> +kimage=${kimage} >> +kimage_path=${kimage_path} >> + >> +EOF >> +} >> + >> +install_image() { >> + install -m 755 -d ${deb_img_dir}/$(dirname ${kimage_path}) >> + cp ${O}/${kimage} ${deb_img_dir}/${kimage_path} >> + >> + # Make sure arm64 kernels are decompressed >> + if [ "${ARCH}" = "arm64" ]; then >> + mv ${deb_img_dir}/${kimage_path} ${deb_img_dir}/${kimage_path}.gz >> + gunzip -f ${deb_img_dir}/${kimage_path}.gz >> + fi >> + >> + install_image_debug >> +} >> + >> +install_image_debug() { >> + # Different tools want the image in different locations >> + # perf >> + mkdir -p ${deb_dbg_dir}/usr/lib/debug/lib/modules/${krel}/ >> + cp ${O}/vmlinux ${deb_dbg_dir}/usr/lib/debug/lib/modules/${krel}/ >> + # systemtap >> + mkdir -p ${deb_dbg_dir}/usr/lib/debug/boot/ >> + ln -s ../lib/modules/$version/vmlinux ${deb_dbg_dir}/usr/lib/debug/boot/vmlinux-${krel} >> + # kdump-tools >> + ln -s lib/modules/${krel}/vmlinux ${deb_dbg_dir}/usr/lib/debug/vmlinux-${krel} >> +} >> + >> +install_hooks() { >> + install -m 755 -d ${deb_img_dir}/etc/kernel/install.d >> + install -m 755 -d ${deb_img_dir}/etc/kernel/postinst.d >> + install -m 755 -d ${deb_img_dir}/etc/kernel/postrm.d >> + install -m 755 -d ${deb_img_dir}/etc/kernel/prerm.d >> + >> + initrd="No" >> + [ -z "${CONFIG_BLK_DEV_INITRD}" ] || initrd="Yes" >> + >> + for script in postinst postrm preinst prerm; do >> + sed -i -e "s,INITRD=[A-Za-z0-9]*,INITRD=${initrd},g" ${S}/debian/linux-image*.${script} >> + sed -i -e "s,version=.*,version=${krel},g" ${S}/debian/linux-image*.${script} >> + done >> +} >> + >> +install_dtbs() { >> + [ -n "${CONFIG_OF}" ] || return 0 >> + ${MAKE} O=${O} INSTALL_DTBS_PATH=${deb_dtb_dir} dtbs_install >> +} >> + >> +install_kmods() { >> + [ -n "${CONFIG_MODULES}" ] || return 0 >> + ${MAKE} O=${O} INSTALL_MOD_PATH=${deb_img_dir} modules_install >> + touch ${deb_img_dir}/lib/modules/${krel}/.fresh-install >> + rm -fv ${deb_img_dir}/lib/modules/${krel}/build >> + rm -fv ${deb_img_dir}/lib/modules/${krel}/source >> + install_kmods_debug >> +} >> + >> +kmods_sign() { >> + [ -n "${CONFIG_MODULE_SIG_ALL}" ] || return 0 >> + ${MAKE} O=${O} INSTALL_MOD_PATH=${deb_img_dir} modules_sign >> +} >> + >> +install_kmods_debug() { >> + [ -n "${CONFIG_DEBUG_INFO}" ] || return 0 >> + >> + kmod_inst_dir=${deb_img_dir}/lib/modules >> + kmod_debug_dir=${deb_dbg_dir}/usr/lib/debug >> + >> + # copy kernels modules to usr/lib/debug >> + mkdir -p ${kmod_debug_dir} >> + tar -C ${kmod_inst_dir}/ -cO --exclude='modules.*' . | tar -C ${kmod_debug_dir}/ -xf - >> + # strip everything but debug sections for modules in usr/lib/debug >> + find ${kmod_debug_dir} -name *.ko -exec ${CROSS_COMPILE}objcopy --only-keep-debug {} \; >> + # and strip debug sections from modules in lib/modules >> + find ${kmod_inst_dir} -name *.ko -exec ${CROSS_COMPILE}objcopy --strip-debug {} \; >> + >> + # re-sign stripped kernel modules >> + kmods_sign >> +} >> + >> +headers_check() { >> + ${MAKE} O=${O} headers_check >> +} >> + >> +libc_headers() { >> + mkdir -p ${deb_libc_hdr_dir} >> + ${MAKE} O=${O} headers_install INSTALL_HDR_PATH=${deb_libc_hdr_dir}/usr >> + host_arch=$(dpkg-architecture -a${DISTRO_ARCH} -qDEB_HOST_MULTIARCH) >> + mkdir ${deb_libc_hdr_dir}/usr/include/${host_arch} >> + mv ${deb_libc_hdr_dir}/usr/include/asm ${deb_libc_hdr_dir}/usr/include/${host_arch}/ >> +} >> + >> +# the kernel does not seem to provide a way to cross-compile its tools, we however >> +# have qemu-user-static binaries in our buildchroot. Create a new build tree for the >> +# scripts and override HOSTCC to force a cross-compile >> +kernel_tools_build() { >> + odir="${S}/build-scripts" >> + mflags="HOSTCC=${CROSS_COMPILE}gcc O=${odir} NOSTDINC_FLAGS=" >> + rm -rf ${odir} && mkdir -p ${odir} && cp ${S}/${KERNEL_BUILD_DIR}/.config ${odir} >> + (unset LD_PRELOAD; cd ${odir} && make -C ${S} ${mflags} olddefconfig scripts) >> + (cd ${odir}; find scripts -type f -executable -exec file {} \;|grep ELF|cut -d: -f1) >${src_hdr_files} >> + tar -C ${odir} -cf - -T - <${src_hdr_files} | tar -C ${destdir} -xf - >> +} >> + >> +kernel_tools() { >> + # remove object files >> + find ${destdir}/scripts -type f -name '*.o' |xargs rm -f >> + >> + # we're all done if we aren't cross-compiling >> + [ -n "${CROSS_COMPILE}" ] || return 0 >> + >> + # remove ELF executables from the linux-headers package >> + find ${destdir}/scripts -type f -exec file {} \;|grep ELF|cut -d: -f1|xargs rm -fv >> + >> + # cross-compile kernel tools to be shipped with linux-headers >> + kernel_tools_build >> +} >> + >> +kernel_headers() { >> + destdir=${deb_kern_hdr_dir}/${KERNEL_HEADERS_DIR} >> + src_hdr_files=$(mktemp) >> + obj_hdr_files=$(mktemp) >> + >> + mkdir -p ${destdir} >> + mkdir -p ${deb_kern_hdr_dir}/lib/modules/${krel} >> + >> + (cd ${S}; find . -name 'Makefile*' -o -name 'Kconfig*' -o -name '*.pl') >>${src_hdr_files} >> + (cd ${S}; find arch/*/include include scripts -type f -o -type l) >>${src_hdr_files} >> + (cd ${S}; find arch/${ARCH} -name module.lds -o -name Kbuild.platforms -o -name Platform) >>${src_hdr_files} >> + (cd ${S}; find $(find arch/${ARCH} -name include -o -name scripts -type d) -type f) >>${src_hdr_files} >> + >> + (cd ${O}; find arch/${ARCH}/include Module.symvers include scripts -type f) >>${obj_hdr_files} >> + if [ -n "${CONFIG_STACK_VALIDATION}" ]; then >> + (cd ${O}; find tools/objtool -type f -executable) >>${obj_hdr_files} >> + fi >> + if [ -n "${CONFIG_GCC_PLUGINS}" ]; then >> + (cd ${O}; find scripts/gcc-plugins -name *.so -o -name gcc-common.h) >>${obj_hdr_files} >> + fi >> + >> + # deploy files that were matched above >> + tar -C ${S} -cf - -T - <${src_hdr_files} | tar -C ${destdir} -xf - >> + tar -C ${O} -cf - -T - <${obj_hdr_files} | tar -C ${destdir} -xf - >> + >> + # add the kernel config >> + cp ${O}/${KCONF} ${destdir}/.config >> + >> + # handle kernel development tools >> + kernel_tools >> + >> + # create symlinks >> + ln -sf /${KERNEL_HEADERS_DIR} ${deb_kern_hdr_dir}/lib/modules/${krel}/build >> +} >> + >> +install_headers() { >> + headers_check >> + [ -z ${kern_pkgs["linux-libc-headers"]} ] || libc_headers >> + kernel_headers >> +} >> + >> +main install ${*} >> diff --git a/meta/recipes-kernel/linux/files/debian/isar/version.cfg.tmpl b/meta/recipes-kernel/linux/files/debian/isar/version.cfg.tmpl >> new file mode 100644 >> index 0000000..09fa364 >> --- /dev/null >> +++ b/meta/recipes-kernel/linux/files/debian/isar/version.cfg.tmpl >> @@ -0,0 +1,2 @@ >> +CONFIG_LOCALVERSION="-${KERNEL_LOCALVERSION}" >> +# CONFIG_LOCALVERSION_AUTO is not set >> diff --git a/meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl b/meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl >> new file mode 100644 >> index 0000000..b373801 >> --- /dev/null >> +++ b/meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl >> @@ -0,0 +1,29 @@ >> +#!/bin/sh >> +# based on https://salsa.debian.org/kernel-team/linux/blob/master/debian/templates/image.postinst.in >> + >> +# Tell initramfs builder whether it's wanted >> +export INITRD=Yes >> + >> +version=${PV}-${KERNEL_LOCALVERSION} >> +image_path=/boot/${KERNEL_FILE}-${version} >> + >> +if [ "$1" != configure ]; then >> + exit 0 >> +fi >> + >> +depmod $version >> + >> +if [ -f /lib/modules/$version/.fresh-install ]; then >> + change=install >> +else >> + change=upgrade >> +fi >> +linux-update-symlinks $change $version $image_path >> +rm -f /lib/modules/$version/.fresh-install >> + >> +if [ -d /etc/kernel/postinst.d ]; then >> + DEB_MAINT_PARAMS="$*" run-parts --report --exit-on-error --arg=$version \ >> + --arg=$image_path /etc/kernel/postinst.d >> +fi >> + >> +exit 0 >> diff --git a/meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl b/meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl >> new file mode 100644 >> index 0000000..1b5dbbe >> --- /dev/null >> +++ b/meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl >> @@ -0,0 +1,35 @@ >> +#!/bin/sh -e >> +# based on https://salsa.debian.org/kernel-team/linux/blob/master/debian/templates/image.postrm.in >> + >> +# Tell initramfs builder whether it's wanted >> +export INITRD=Yes >> + >> +version=${PV}-${KERNEL_LOCALVERSION} >> +image_path=/boot/${KERNEL_FILE}-${version} >> + >> +rm -f /lib/modules/$version/.fresh-install >> + >> +if [ "$1" != upgrade ] && command -v linux-update-symlinks >/dev/null; then >> + linux-update-symlinks remove $version $image_path >> +fi >> + >> +if [ -d /etc/kernel/postrm.d ]; then >> + DEB_MAINT_PARAMS="$*" run-parts --report --exit-on-error --arg=$version \ >> + --arg=$image_path /etc/kernel/postrm.d >> +fi >> + >> +if [ "$1" = purge ]; then >> + for extra_file in modules.dep modules.isapnpmap modules.pcimap \ >> + modules.usbmap modules.parportmap \ >> + modules.generic_string modules.ieee1394map \ >> + modules.ieee1394map modules.pnpbiosmap \ >> + modules.alias modules.ccwmap modules.inputmap \ >> + modules.symbols modules.ofmap \ >> + modules.seriomap modules.\*.bin \ >> + modules.softdep modules.devname; do >> + eval rm -f /lib/modules/$version/$extra_file >> + done >> + rmdir /lib/modules/$version || true >> +fi >> + >> +exit 0 >> diff --git a/meta/recipes-kernel/linux/files/debian/linux-image.preinst.tmpl b/meta/recipes-kernel/linux/files/debian/linux-image.preinst.tmpl >> new file mode 100644 >> index 0000000..4f24019 >> --- /dev/null >> +++ b/meta/recipes-kernel/linux/files/debian/linux-image.preinst.tmpl >> @@ -0,0 +1,25 @@ >> +#!/bin/sh -e >> +# based on https://salsa.debian.org/kernel-team/linux/blob/master/debian/templates/image.preinst.in >> + >> +# Tell initramfs builder whether it's wanted >> +export INITRD=Yes >> + >> +version=${PV}-${KERNEL_LOCALVERSION} >> +image_path=/boot/${KERNEL_FILE}-${version} >> + >> +if [ "$1" = abort-upgrade ]; then >> + exit 0 >> +fi >> + >> +if [ "$1" = install ]; then >> + # Create a flag file for postinst >> + mkdir -p /lib/modules/$version >> + touch /lib/modules/$version/.fresh-install >> +fi >> + >> +if [ -d /etc/kernel/preinst.d ]; then >> + DEB_MAINT_PARAMS="$*" run-parts --report --exit-on-error --arg=$version \ >> + --arg=$image_path /etc/kernel/preinst.d >> +fi >> + >> +exit 0 >> diff --git a/meta/recipes-kernel/linux/files/debian/linux-image.prerm.tmpl b/meta/recipes-kernel/linux/files/debian/linux-image.prerm.tmpl >> new file mode 100644 >> index 0000000..cd697c0 >> --- /dev/null >> +++ b/meta/recipes-kernel/linux/files/debian/linux-image.prerm.tmpl >> @@ -0,0 +1,21 @@ >> +#!/bin/sh -e >> +# based on https://salsa.debian.org/kernel-team/linux/blob/master/debian/templates/image.prerm.in >> + >> +# Tell initramfs builder whether it's wanted >> +export INITRD=Yes >> + >> +version=${PV}-${KERNEL_LOCALVERSION} >> +image_path=/boot/${KERNEL_FILE}-${version} >> + >> +if [ "$1" != remove ]; then >> + exit 0 >> +fi >> + >> +linux-check-removal $version >> + >> +if [ -d /etc/kernel/prerm.d ]; then >> + DEB_MAINT_PARAMS="$*" run-parts --report --exit-on-error --arg=$version \ >> + --arg=$image_path /etc/kernel/prerm.d >> +fi >> + >> +exit 0 >> diff --git a/meta/recipes-kernel/linux/files/debian/rules.tmpl b/meta/recipes-kernel/linux/files/debian/rules.tmpl >> new file mode 100755 >> index 0000000..c024768 >> --- /dev/null >> +++ b/meta/recipes-kernel/linux/files/debian/rules.tmpl >> @@ -0,0 +1,39 @@ >> +#!/usr/bin/make -f >> + >> +CROSS_COMPILE:=$(DEB_HOST_GNU_TYPE)- >> + >> +O:=$(CURDIR)/${KERNEL_BUILD_DIR} >> +S:=$(CURDIR) >> +deb_top_dir:=$(S)/debian >> + >> +# Dynamic variables to be passed to Isar build scripts >> +isar_env=$(strip \ >> + export CROSS_COMPILE='$(CROSS_COMPILE)' && \ >> + export DEB_BUILD_PROFILES='$(DEB_BUILD_PROFILES)' && \ >> + export DEB_HOST_GNU_TYPE='$(DEB_HOST_GNU_TYPE)' && \ >> + export MAKE='$(MAKE)' && \ >> + export O='${O}' \ >> + export S='${S}' \ >> +) >> + >> +%: >> + dh $(@) >> + >> +.PHONY: override_dh_auto_clean >> +override_dh_auto_clean: >> + $(isar_env) && bash $(deb_top_dir)/isar/clean >> + >> +.PHONY: override_dh_auto_build >> +override_dh_auto_build: >> + $(isar_env) && bash $(deb_top_dir)/isar/build >> + >> +override_dh_auto_install: >> + $(isar_env) && bash $(deb_top_dir)/isar/install >> + >> +.PHONY: override_dh_auto_test >> +override_dh_auto_test: >> + true >> + >> +PHONY: override_dh_strip >> +override_dh_strip: >> + dh_strip -Xvmlinux --no-automatic-dbgsym >> diff --git a/meta/recipes-kernel/linux/linux-custom.inc b/meta/recipes-kernel/linux/linux-custom.inc >> index 712f52b..bf1646a 100644 >> --- a/meta/recipes-kernel/linux/linux-custom.inc >> +++ b/meta/recipes-kernel/linux/linux-custom.inc >> @@ -2,89 +2,183 @@ >> # >> # This software is a part of ISAR. >> # Copyright (c) Siemens AG, 2018 >> +# Copyright (c) Mentor Graphics, a Siemens business, 2019 >> # >> # SPDX-License-Identifier: MIT >> >> -FILESEXTRAPATHS_prepend := "${FILE_DIRNAME}/files:" >> +# Settings that would typically be done from the custom kernel recipe >> +# ------------------------------------------------------------------- >> >> +CHANGELOG_V = "${PV}+${PR}" >> DESCRIPTION ?= "Custom kernel" >> +MAINTAINER ?= "isar-users <isar-users@googlegroups.com>" >> + >> +KBUILD_DEPENDS ?= "build-essential:native, \ >> + libelf-dev:native, \ >> + libncurses-dev:native, \ >> + libssl-dev:native, \ >> + bc, \ >> + bison, \ >> + cpio, \ >> + flex, \ >> + git, \ >> + kmod, \ >> + libssl-dev," >> + >> +KERNEL_DEBIAN_DEPENDS ?= "initramfs-tools | linux-initramfs-tool, \ >> + kmod, \ >> + linux-base (>= 4.3~)," >> + >> +KERNEL_HEADERS_DEBIAN_DEPENDS ?= "libc6, \ >> + libssl1.1," >> + >> +KERNEL_LIBC_DEV_DEPLOY ?= "0" >> + >> +# Settings that may be changed on a per distro, machine or layer basis >> +# -------------------------------------------------------------------- >> + >> +KERNEL_LOCALVERSION ?= "-${KERNEL_NAME_PROVIDED}-${PR}" >> +KERNEL_DTB_DIR ?= "usr/lib/linux-image-${PV}-${KERNEL_LOCALVERSION}" >> +KERNEL_HEADERS_DIR ?= "usr/src/linux-headers-${PV}-${KERNEL_LOCALVERSION}" >> >> -KERNEL_NAME_PROVIDED ?= "${@ d.getVar('PN', True).partition('linux-')[2]}" >> KERNEL_DEFCONFIG ?= "" >> >> +# Add our template meta-data to the sources >> +FILESEXTRAPATHS_prepend := "${FILE_DIRNAME}/files:" >> +SRC_URI += "file://debian" >> + >> +# Variables and files that make our templates >> +# ------------------------------------------- >> + >> +TEMPLATE_FILES += " \ >> + debian/control.tmpl \ >> + debian/isar/build.tmpl \ >> + debian/isar/clean.tmpl \ >> + debian/isar/common.tmpl \ >> + debian/isar/install.tmpl \ >> + debian/isar/version.cfg.tmpl \ >> + debian/linux-image.postinst.tmpl \ >> + debian/linux-image.postrm.tmpl \ >> + debian/linux-image.preinst.tmpl \ >> + debian/linux-image.prerm.tmpl \ >> + debian/rules.tmpl \ >> +" >> + >> +TEMPLATE_VARS += " \ >> + HOST_ARCH \ >> + KBUILD_DEPENDS \ >> + KERNEL_ARCH \ >> + KERNEL_DEBIAN_DEPENDS \ >> + KERNEL_BUILD_DIR \ >> + KERNEL_DTB_DIR \ >> + KERNEL_FILE \ >> + KERNEL_HEADERS_DEBIAN_DEPENDS \ >> + KERNEL_HEADERS_DIR \ >> + KERNEL_LOCALVERSION \ >> + KERNEL_NAME_PROVIDED \ >> +" >> + >> +inherit dpkg >> +inherit template >> + >> +# Derive name of the kernel packages from the name of this recipe >> +KERNEL_NAME_PROVIDED ?= "${@ d.getVar('PN', True).partition('linux-')[2]}" >> + >> +# Make bitbake know we will be producing linux-image and linux-headers packages >> python() { >> kernel_name = d.getVar("KERNEL_NAME_PROVIDED", True) >> d.setVar('PROVIDES', 'linux-image-' + kernel_name + ' ' + \ >> 'linux-headers-' + kernel_name) >> } >> >> -inherit dpkg-base >> - >> -SRC_URI += "file://build-kernel.sh" >> - >> -KBUILD_DEPENDS ?= " \ >> - build-essential:native \ >> - libssl-dev \ >> - libelf-dev \ >> - bc \ >> - git \ >> - kmod \ >> - bison \ >> - flex \ >> - cpio \ >> - libncurses-dev" >> -KERNEL_DEBIAN_DEPENDS ?= "initramfs-tools | linux-initramfs-tool, kmod, linux-base (>= 4.3~)" >> -KERNEL_HEADERS_DEBIAN_DEPENDS ?= "libc6, libssl1.1" >> - >> -KERNEL_LIBC_DEV_DEPLOY ?= "0" >> - >> -do_install_builddeps() { >> - dpkg_do_mounts >> - E="${@ isar_export_proxies(d)}" >> - sudo -E chroot ${BUILDCHROOT_DIR} \ >> - apt-get update \ >> - -o Dir::Etc::SourceList="sources.list.d/isar-apt.list" \ >> - -o Dir::Etc::SourceParts="-" \ >> - -o APT::Get::List-Cleanup="0" >> - sudo -E chroot ${BUILDCHROOT_DIR} \ >> - apt-get install \ >> - -y -o Debug::pkgProblemResolver=yes \ >> - --no-install-recommends ${KBUILD_DEPENDS} >> - dpkg_undo_mounts >> +def get_kernel_arch(d): >> + distro_arch = d.getVar("DISTRO_ARCH") >> + if distro_arch == "amd64": >> + kernel_arch = "x86" >> + elif distro_arch == "arm64": >> + kernel_arch = "arm64" >> + elif distro_arch == "armhf": >> + kernel_arch = "arm" >> + elif distro_arch == "mipsel": >> + kernel_arch = "mips" >> + else: >> + kernel_arch = "" >> + return kernel_arch >> + >> +KERNEL_ARCH ??= "${@get_kernel_arch(d)}" >> + >> +def config_fragments(d): >> + fragments = [] >> + sources = d.getVar("SRC_URI").split() >> + for s in sources: >> + _, _, local, _, _, parm = bb.fetch.decodeurl(s) >> + apply = parm.get("apply") >> + if apply == "no": >> + continue >> + base, ext = os.path.splitext(os.path.basename(local)) >> + if ext and ext in (".cfg"): >> + fragments.append(local) >> + return fragments >> + >> +do_prepare_build_prepend() { >> + # copy meta-data over to source tree >> + rm -rf ${S}/debian >> + cp -r ${WORKDIR}/debian ${S}/ >> + >> + # remove templates from the source tree >> + find ${S}/debian -name *.tmpl | xargs rm -f >> + >> + # rename install/remove hooks to match user-specified name for our linux-image package >> + mv ${S}/debian/linux-image.postinst ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.postinst >> + mv ${S}/debian/linux-image.postrm ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.postrm >> + mv ${S}/debian/linux-image.preinst ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.preinst >> + mv ${S}/debian/linux-image.prerm ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.prerm >> + >> + # produce a changelog for our kernel build >> + deb_add_changelog >> } >> >> -addtask install_builddeps after do_prepare_build before do_dpkg_build >> -# apt and reprepro may not run in parallel, acquire the Isar lock >> -do_install_builddeps[lockfiles] += "${REPO_ISAR_DIR}/isar.lock" >> - >> -addtask devshell after do_install_builddeps >> +# build directory for our "full" kernel build >> +KERNEL_BUILD_DIR = "build-full" >> >> -dpkg_runbuild() { >> - chmod +x ${WORKDIR}/build-kernel.sh >> - KERNEL_CONFIG_TARGET="${KERNEL_DEFCONFIG}" >> +dpkg_configure_kernel() { >> + config_target="${KERNEL_DEFCONFIG}" >> + rm -rf ${S}/${KERNEL_BUILD_DIR} && mkdir -p ${S}/${KERNEL_BUILD_DIR} >> if [ -n "${KERNEL_DEFCONFIG}" ]; then >> if [ -e "${WORKDIR}/${KERNEL_DEFCONFIG}" ]; then >> - cp ${WORKDIR}/${KERNEL_DEFCONFIG} ${S}/.config >> - KERNEL_CONFIG_TARGET="olddefconfig" >> + cp ${WORKDIR}/${KERNEL_DEFCONFIG} ${S}/${KERNEL_BUILD_DIR}/.config >> + config_target="olddefconfig" >> fi >> else >> - KERNEL_CONFIG_TARGET="defconfig" >> + config_target="defconfig" >> fi >> >> - E="${@ isar_export_proxies(d)}" >> - >> - export PV=${PV} >> - export KERNEL_NAME=${KERNEL_NAME_PROVIDED} >> - >> - export KBUILD_DEPENDS="${KBUILD_DEPENDS}" >> - export KERNEL_CONFIG_TARGET >> - export KERNEL_DEBIAN_DEPENDS="${KERNEL_DEBIAN_DEPENDS}" >> - export KERNEL_FILE="${KERNEL_FILE}" >> - export KERNEL_HEADERS_DEBIAN_DEPENDS="${KERNEL_HEADERS_DEBIAN_DEPENDS}" >> + # copy config fragments over to the kernel tree >> + src_frags="${@ " ".join(config_fragments(d)) }" >> + out_frags="${@ " ".join(map(lambda frag: 'debian/fragments/' + frag, config_fragments(d))) }" >> + rm -rf ${S}/debian/fragments >> + if [ -n "${src_frags}" ]; then >> + mkdir -p ${S}/debian/fragments >> + (cd ${WORKDIR} && cp ${src_frags} ${S}/debian/fragments/) >> + fi >> >> - sudo -E chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} ${PP}/build-kernel.sh ${PP}/${PPS} ${DISTRO_ARCH} >> + sudo -E chroot --userspec=$(id -u):$(id -g) ${BUILDCHROOT_DIR} sh -c " \ >> + export ARCH=${KERNEL_ARCH} && \ >> + cd ${PP}/${PPS} && \ >> + make O=${KERNEL_BUILD_DIR} ${config_target} && \ >> + ./scripts/kconfig/merge_config.sh \ >> + -O ${KERNEL_BUILD_DIR}/ \ >> + ${KERNEL_BUILD_DIR}/.config \ >> + ${out_frags} \ >> + debian/isar/version.cfg \ >> + " >> +} >> >> +dpkg_runbuild_prepend() { >> + profiles="${BASE_DISTRO}" >> if [ "${KERNEL_LIBC_DEV_DEPLOY}" != "1" ]; then >> - rm -f ${WORKDIR}/linux-libc-dev_${PV}*.deb >> + profiles="${profiles} nolibcdev" >> fi >> + export DEB_BUILD_PROFILES="${profiles}" >> + dpkg_configure_kernel >> } >> > I would recommend this functional change to the patch to get away > without patch 4: > > diff --git a/meta/recipes-kernel/linux/files/debian/isar/install.tmpl b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl > index 806df5d..67b7ce3 100644 > --- a/meta/recipes-kernel/linux/files/debian/isar/install.tmpl > +++ b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl > @@ -143,30 +143,9 @@ libc_headers() { > mv ${deb_libc_hdr_dir}/usr/include/asm ${deb_libc_hdr_dir}/usr/include/${host_arch}/ > } > > -# the kernel does not seem to provide a way to cross-compile its tools, we however > -# have qemu-user-static binaries in our buildchroot. Create a new build tree for the > -# scripts and override HOSTCC to force a cross-compile > -kernel_tools_build() { > - odir="${S}/build-scripts" > - mflags="HOSTCC=${CROSS_COMPILE}gcc O=${odir} NOSTDINC_FLAGS=" > - rm -rf ${odir} && mkdir -p ${odir} && cp ${S}/${KERNEL_BUILD_DIR}/.config ${odir} > - (unset LD_PRELOAD; cd ${odir} && make -C ${S} ${mflags} olddefconfig scripts) > - (cd ${odir}; find scripts -type f -executable -exec file {} \;|grep ELF|cut -d: -f1) >${src_hdr_files} > - tar -C ${odir} -cf - -T - <${src_hdr_files} | tar -C ${destdir} -xf - > -} > - > kernel_tools() { > # remove object files > find ${destdir}/scripts -type f -name '*.o' |xargs rm -f > - > - # we're all done if we aren't cross-compiling > - [ -n "${CROSS_COMPILE}" ] || return 0 > - > - # remove ELF executables from the linux-headers package > - find ${destdir}/scripts -type f -exec file {} \;|grep ELF|cut -d: -f1|xargs rm -fv > - > - # cross-compile kernel tools to be shipped with linux-headers > - kernel_tools_build > } > > kernel_headers() { > diff --git a/meta/recipes-kernel/linux/files/debian/rules.tmpl b/meta/recipes-kernel/linux/files/debian/rules.tmpl > index c024768..bea21f9 100755 > --- a/meta/recipes-kernel/linux/files/debian/rules.tmpl > +++ b/meta/recipes-kernel/linux/files/debian/rules.tmpl > @@ -36,4 +36,4 @@ override_dh_auto_test: > > PHONY: override_dh_strip > override_dh_strip: > - dh_strip -Xvmlinux --no-automatic-dbgsym > + unset DEB_HOST_GNU_TYPE && dh_strip -Xvmlinux --no-automatic-dbgsym > > > Then we can introduce the correct building of linux-headers once we can > generate both host and target packages with Isar. And we do not need to > infect the host buildchroot. Hi Jan, Thanks for checking our options and devising this patch. Much appreciated. How should we proceed? Would you like to post v5 or should I? Should I add your Signed-off-by to the last commit (the one that heavily changes linux-custom.inc) to give you review/contribution credits? > > Jan > ^ permalink raw reply [flat|nested] 83+ messages in thread
* Re: [PATCH L-C v4 5/5] linux-custom: rewrite to no longer depend on the kernel's builddeb 2019-11-26 7:42 ` Cedric Hombourger @ 2019-11-26 9:13 ` Jan Kiszka 0 siblings, 0 replies; 83+ messages in thread From: Jan Kiszka @ 2019-11-26 9:13 UTC (permalink / raw) To: Cedric Hombourger, isar-users On 26.11.19 08:42, Cedric Hombourger wrote: > > On 11/25/2019 8:55 PM, Jan Kiszka wrote: >> On 25.11.19 17:06, Cedric Hombourger wrote: >>> Regain control over the packaging of the Linux kernel by providing our >>> own debian recipes (debian,{control,rules} files and friends) instead >>> of using the kernel's builddeb script. This will allow generation of >>> packages for "perf" (not included in this changeset) with per distro >>> scheme (Debian and Ubuntu have differences) and maintain compatibility >>> with upstream packages generated from different source packages but >>> dependent on a certain deployment scheme (e.g. linux-base for Debian) >>> >>> Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com> >>> --- >>> doc/custom_kernel.md | 80 +++++++ >>> .../linux/files/build-kernel.sh | 125 ---------- >>> meta/recipes-kernel/linux/files/debian/compat | 1 + >>> .../linux/files/debian/control.tmpl | 37 +++ >>> .../linux/files/debian/isar/build.tmpl | 45 ++++ >>> .../linux/files/debian/isar/clean.tmpl | 21 ++ >>> .../linux/files/debian/isar/common.tmpl | 50 ++++ >>> .../linux/files/debian/isar/install.tmpl | 213 >>> ++++++++++++++++++ >>> .../linux/files/debian/isar/version.cfg.tmpl | 2 + >>> .../files/debian/linux-image.postinst.tmpl | 29 +++ >>> .../files/debian/linux-image.postrm.tmpl | 35 +++ >>> .../files/debian/linux-image.preinst.tmpl | 25 ++ >>> .../linux/files/debian/linux-image.prerm.tmpl | 21 ++ >>> .../linux/files/debian/rules.tmpl | 39 ++++ >>> meta/recipes-kernel/linux/linux-custom.inc | 210 ++++++++++++----- >>> 15 files changed, 750 insertions(+), 183 deletions(-) >>> create mode 100644 doc/custom_kernel.md >>> delete mode 100644 meta/recipes-kernel/linux/files/build-kernel.sh >>> create mode 100644 meta/recipes-kernel/linux/files/debian/compat >>> create mode 100644 >>> meta/recipes-kernel/linux/files/debian/control.tmpl >>> create mode 100644 >>> meta/recipes-kernel/linux/files/debian/isar/build.tmpl >>> create mode 100644 >>> meta/recipes-kernel/linux/files/debian/isar/clean.tmpl >>> create mode 100644 >>> meta/recipes-kernel/linux/files/debian/isar/common.tmpl >>> create mode 100644 >>> meta/recipes-kernel/linux/files/debian/isar/install.tmpl >>> create mode 100644 >>> meta/recipes-kernel/linux/files/debian/isar/version.cfg.tmpl >>> create mode 100644 >>> meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl >>> create mode 100644 >>> meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl >>> create mode 100644 >>> meta/recipes-kernel/linux/files/debian/linux-image.preinst.tmpl >>> create mode 100644 >>> meta/recipes-kernel/linux/files/debian/linux-image.prerm.tmpl >>> create mode 100755 meta/recipes-kernel/linux/files/debian/rules.tmpl >>> >>> diff --git a/doc/custom_kernel.md b/doc/custom_kernel.md >>> new file mode 100644 >>> index 0000000..31319d4 >>> --- /dev/null >>> +++ b/doc/custom_kernel.md >>> @@ -0,0 +1,80 @@ >>> +# Custom kernel recipe for Isar >>> + >>> +## Contents >>> + >>> + - [Summary](#summary) >>> + - [Features](#features) >>> + - [Future](#future) >>> + - [Examples](#examples) >>> + >>> +## Summary >>> + >>> +Isar provides a recipe to build custom kernels for Debian-based >>> distributions. >>> +It uses templates to generate the debian meta-data (such as >>> debian/control) and >>> +Debian's [BuildProfiles](https://wiki.debian.org/BuildProfileSpec) >>> to handle >>> +some of the distro specific variations. It should be noted that Isar >>> has moved >>> +away from using the kernel's builddeb script since it would not >>> generate all >>> +the packages we need (and in particular perf). >>> + >>> +## Features >>> + >>> +The linux-custom recipe provides support for: >>> + >>> + 1. Sources to the custom Linux kernel may be specified via `SRC_URI` >>> + >>> + 2. Configure the kernel via an in-tree or an external `defconfig` via >>> + `KERNEL_DEFCONFIG` >>> + >>> + 3. Integrate kernel configuration tweaks via configuration >>> fragments (`.cfg` >>> + files) >>> + >>> + 4. Patches to the linux kernel may be specified via `SRC_URI` >>> + >>> + 5. Ensure that the Isar recipe `PV` matches the kernel release >>> identifier >>> + (`KERNEL_RELEASE`) >>> + >>> + 6. Produce a `linux-image` package that ships the kernel image and >>> modules >>> + >>> + 7. Allow the name of the kernel image to be changed via >>> `KERNEL_FILE` (defaults >>> + to `vmlinuz`) >>> + >>> + 8. Produce a `linux-headers` package which includes kernel headers >>> and kbuild >>> + scripts/tools >>> + >>> + 9. The `linux-headers` package shall support native and cross >>> compiles of >>> + out-of-tree kernel modules >>> + >>> + 10. Produce a `linux-libc-dev` package to support user-land builds >>> + >>> + 11. Only build/ship the `linux-libc-dev` package if instructed to >>> + (`KERNEL_LIBC_DEV_DEPLOY` equals to `"1"`) >>> + >>> + 12. Support both native and cross compiles (`ISAR_CROSS_COMPILE`) >>> + >>> + 13. Support for the following kernel architectures: >>> + >>> + * arm >>> + * arm64 >>> + * mips >>> + * x86 >>> + >>> + 14. Support `devshell` (kernel configuration shall be applied) >>> + >>> +## Future >>> + >>> +In the future, the recipe may be extended to: >>> + >>> + 1. Package perf >>> + >>> + 2. Support inclusion/build of dts files listed in `SRC_URI` >>> + >>> + 3. Be compatible with Ubuntu >>> + >>> +## Examples >>> + >>> +The linux-custom recipe is currently used by the linux-mainline >>> package and is >>> +used mainline recipe may be used for some basic testing. This recipe >>> is being >>> +used by the following machines: >>> + >>> + * de0-nano-soc >>> + * qemumipsel >>> diff --git a/meta/recipes-kernel/linux/files/build-kernel.sh >>> b/meta/recipes-kernel/linux/files/build-kernel.sh >>> deleted file mode 100644 >>> index 386e58b..0000000 >>> --- a/meta/recipes-kernel/linux/files/build-kernel.sh >>> +++ /dev/null >>> @@ -1,125 +0,0 @@ >>> -#!/bin/bash >>> -# >>> -# Custom kernel build >>> -# >>> -# This software is a part of ISAR. >>> -# Copyright (c) Siemens AG, 2018 >>> -# >>> -# SPDX-License-Identifier: MIT >>> - >>> -source /isar/common.sh >>> - >>> -host_arch=$(dpkg --print-architecture) >>> - >>> -if [ "$host_arch" != "$target_arch" ]; then >>> - case $target_arch in >>> - armhf) >>> - export ARCH=arm >>> - export CROSS_COMPILE="arm-linux-gnueabihf-" >>> - ;; >>> - arm64) >>> - export ARCH=arm64 >>> - export CROSS_COMPILE="aarch64-linux-gnu-" >>> - ;; >>> - mipsel) >>> - export ARCH=mips >>> - export CROSS_COMPILE="mipsel-linux-gnu-" >>> - ;; >>> - *) >>> - echo "error: unsupported architecture ($target_arch)" >>> - exit 1 >>> - ;; >>> - esac >>> -fi >>> - >>> -REPACK_DIR="$1/../repack" >>> -REPACK_LINUX_IMAGE_DIR="${REPACK_DIR}/linux-image" >>> -REPACK_LINUX_HEADERS_DIR="${REPACK_DIR}/linux-headers" >>> - >>> -make ${KERNEL_CONFIG_TARGET} || exit ${?} >>> - >>> -KV=$( make -s kernelrelease ) >>> -if [ "${KV}" != "${PV}" ]; then >>> - echo "ERROR: Recipe PV is \"${PV}\" but should be \"${KV}\"" 1>&2 >>> - echo "ERROR: Probably due to CONFIG_LOCALVERSION" 1>&2 >>> - exit 1 >>> -fi >>> - >>> -rm -f .version >>> -KBUILD_DEBARCH=$target_arch make -j $(($(nproc) * 2)) deb-pkg >>> - >>> -rm -rf "${REPACK_DIR}" >>> -mkdir -p "${REPACK_DIR}" >>> -mkdir -p "${REPACK_LINUX_IMAGE_DIR}" >>> -mkdir -p "${REPACK_LINUX_HEADERS_DIR}" >>> - >>> -cp -a debian "${REPACK_DIR}" >>> - >>> -# dpkg-gencontrol performs cross-incompatible checks on the >>> -# Architecture field; trick it to accept the control file >>> -sed -i "s/Architecture: .*/Architecture: any/" >>> "${REPACK_DIR}/debian/control" >>> - >>> -cd .. >>> - >>> -dpkg-deb -R linux-image-${PV}_${PV}-1_*.deb "${REPACK_LINUX_IMAGE_DIR}" >>> -dpkg-deb -R linux-headers-${PV}_${PV}-1_*.deb >>> "${REPACK_LINUX_HEADERS_DIR}" >>> - >>> -dpkg-gencontrol -crepack/debian/control \ >>> - -lrepack/debian/changelog \ >>> - -frepack/debian/files \ >>> - -plinux-image-${PV} \ >>> - -P"${REPACK_LINUX_IMAGE_DIR}" \ >>> - -DPackage="linux-image-${KERNEL_NAME}" \ >>> - -DSection=kernel \ >>> - -DPriority=required \ >>> - -DDepends="${KERNEL_DEBIAN_DEPENDS}" \ >>> - -DArchitecture=$target_arch >>> - >>> -# Add Debian-like link installation to postinst >>> -mkdir -p ${REPACK_LINUX_IMAGE_DIR}/lib/modules/${PV} >>> -touch "${REPACK_LINUX_IMAGE_DIR}/lib/modules/${PV}/.fresh-install" >>> -sed -i "${REPACK_LINUX_IMAGE_DIR}/DEBIAN/postinst" \ >>> - -e "/^set -e$/a\\ >>> -\\ >>> -if [ -f /lib/modules/${PV}/.fresh-install ]; then\\ >>> - change=install\\ >>> -else\\ >>> - change=upgrade\\ >>> -fi\\ >>> -linux-update-symlinks \$change ${PV} /boot/${KERNEL_FILE}-${PV}\\ >>> -rm -f /lib/modules/${PV}/.fresh-install" >>> - >>> -# Add Debian-like link removal to postrm >>> -sed -i "${REPACK_LINUX_IMAGE_DIR}/DEBIAN/postrm" \ >>> - -e "/^set -e$/a\\ >>> -\\ >>> -rm -f /lib/modules/${PV}/.fresh-install\\ >>> -\\ >>> -if [ \"\$1\" != upgrade ] && command -v linux-update-symlinks >>> >/dev/null; then\\ >>> - linux-update-symlinks remove ${PV} /boot/${KERNEL_FILE}-${PV}\\ >>> -fi" >>> - >>> -# Make sure arm64 kernels are decompressed >>> -if [ "$target_arch" = "arm64" ]; then >>> - kernel_file="${REPACK_LINUX_IMAGE_DIR}/boot/${KERNEL_FILE}-${PV}" >>> - mv "${kernel_file}" "${kernel_file}.gz" >>> - gunzip "${kernel_file}.gz" >>> -fi >>> - >>> -dpkg-gencontrol -crepack/debian/control \ >>> - -lrepack/debian/changelog \ >>> - -frepack/debian/files \ >>> - -plinux-headers-${PV} \ >>> - -P"${REPACK_LINUX_HEADERS_DIR}" \ >>> - -Vkernel:debarch="${KERNEL_NAME}" \ >>> - -DPackage="linux-headers-${KERNEL_NAME}" \ >>> - -DSection=kernel \ >>> - -DDepends="${KERNEL_HEADERS_DEBIAN_DEPENDS}" \ >>> - -DArchitecture=$target_arch >>> - >>> -fakeroot dpkg-deb -b "${REPACK_LINUX_IMAGE_DIR}" \ >>> - linux-image-${KERNEL_NAME}_${PV}-1_${KERNEL_NAME}.deb >>> -rm -f linux-image-${PV}_${PV}-1_*.deb >>> -fakeroot dpkg-deb -b "${REPACK_LINUX_HEADERS_DIR}" \ >>> - linux-headers-${KERNEL_NAME}_${PV}-1_${KERNEL_NAME}.deb >>> -rm -f linux-headers-${PV}_${PV}-1_*.deb >>> diff --git a/meta/recipes-kernel/linux/files/debian/compat >>> b/meta/recipes-kernel/linux/files/debian/compat >>> new file mode 100644 >>> index 0000000..ec63514 >>> --- /dev/null >>> +++ b/meta/recipes-kernel/linux/files/debian/compat >>> @@ -0,0 +1 @@ >>> +9 >>> diff --git a/meta/recipes-kernel/linux/files/debian/control.tmpl >>> b/meta/recipes-kernel/linux/files/debian/control.tmpl >>> new file mode 100644 >>> index 0000000..7236732 >>> --- /dev/null >>> +++ b/meta/recipes-kernel/linux/files/debian/control.tmpl >>> @@ -0,0 +1,37 @@ >>> +Source: linux-${KERNEL_NAME_PROVIDED} >>> +Section: kernel >>> +Priority: optional >>> +Maintainer: ${MAINTAINER} >>> +Build-Depends: bc, kmod, cpio, ${KBUILD_DEPENDS} >>> +Homepage: http://www.kernel.org/ >>> + >>> +Package: linux-image-${KERNEL_NAME_PROVIDED} >>> +Architecture: any >>> +Depends: ${KERNEL_DEBIAN_DEPENDS} >>> +Description: ${KERNEL_NAME_PROVIDED} Linux kernel, version >>> ${PV}-${KERNEL_LOCALVERSION} >>> + This package contains the Linux kernel, modules and corresponding >>> other >>> + files, version: ${PV}-${KERNEL_LOCALVERSION}. >>> + >>> +Package: linux-headers-${KERNEL_NAME_PROVIDED} >>> +Architecture: any >>> +Depends: ${KERNEL_HEADERS_DEBIAN_DEPENDS}, ${perl:Depends}, >>> ${shlib:Depends} >>> +Description: ${KERNEL_NAME_PROVIDED} Linux kernel headers for >>> ${PV}-${KERNEL_LOCALVERSION} >>> + This package provides kernel header files for >>> ${PV}-${KERNEL_LOCALVERSION} on ${DISTRO_ARCH} >>> + . >>> + This is useful for people who need to build external modules >>> + >>> +Package: linux-libc-dev >>> +Build-Profiles: <!nolibcdev> >>> +Section: devel >>> +Provides: linux-kernel-headers >>> +Architecture: any >>> +Description: Linux support headers for userspace development >>> + This package provides userspaces headers from the Linux kernel. >>> These headers >>> + are used by the installed headers for GNU glibc and other system >>> libraries. >>> + >>> +Package: linux-image-${KERNEL_NAME_PROVIDED}-dbg >>> +Section: debug >>> +Architecture: any >>> +Description: Linux kernel debugging symbols for >>> ${PV}-${KERNEL_LOCALVERSION} >>> + This package will come in handy if you need to debug the kernel. It >>> provides >>> + all the necessary debug symbols for the kernel and its modules. >>> diff --git a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl >>> b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl >>> new file mode 100644 >>> index 0000000..1775b19 >>> --- /dev/null >>> +++ b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl >>> @@ -0,0 +1,45 @@ >>> +#!/bin/bash >>> +# Copyright (c) Mentor Graphics, a Siemens business, 2019 >>> +# SPDX-License-Identifier: MIT >>> + >>> +# Load common stuff >>> +. ${S}/debian/isar/common || exit ${?} >>> + >>> +do_build() { >>> + >>> + # Print a few things that are of particular interest >>> + print_settings >>> + >>> + # Process existing kernel configuration to make sure it is complete >>> + # (use defaults for options that were not specified) >>> + ${MAKE} O=${KERNEL_BUILD_DIR} olddefconfig prepare || exit ${?} >>> + >>> + # Check if the recipe's PV makes sense >>> + KR=$(${MAKE} O=${KERNEL_BUILD_DIR} -s --no-print-directory >>> kernelrelease) >>> + eval $(grep ^CONFIG_LOCALVERSION= ${KERNEL_BUILD_DIR}/${KCONF} >>> || true) >>> + if [ "${PV}-${KERNEL_LOCALVERSION}" != "${KR}" ]; then >>> + echo "ERROR: Recipe version (${PV}-${KERNEL_LOCALVERSION}) >>> does not seem to match the kernelrelease (${KR})!" 1>&2 >>> + echo "ERROR: Make sure the kernel version in your NAME/PV/PR >>> settings and/or CONFIG_LOCALVERSION are aligned" 1>&2 >>> + exit 1 >>> + fi >>> + >>> + # Trace what we do here >>> + set -x >>> + >>> + # Build the Linux kernel >>> + ${MAKE} O=${KERNEL_BUILD_DIR} -j $(nproc) || exit ${?} >>> + >>> + # Stop tracing >>> + set +x >>> +} >>> + >>> +print_settings() { >>> + cat <<EOF >>> +# Build settings: >>> +# --------------- >>> +# ARCH=${ARCH} >>> +# CROSS_COMPILE=${CROSS_COMPILE} >>> +EOF >>> +} >>> + >>> +main build ${*} >>> diff --git a/meta/recipes-kernel/linux/files/debian/isar/clean.tmpl >>> b/meta/recipes-kernel/linux/files/debian/isar/clean.tmpl >>> new file mode 100644 >>> index 0000000..f903987 >>> --- /dev/null >>> +++ b/meta/recipes-kernel/linux/files/debian/isar/clean.tmpl >>> @@ -0,0 +1,21 @@ >>> +#!/bin/bash >>> +# Copyright (c) Mentor Graphics, a Siemens business, 2019 >>> +# SPDX-License-Identifier: MIT >>> + >>> +# Load common stuff >>> +. ${S}/debian/isar/common || exit ${?} >>> + >>> +do_clean() { >>> + >>> + # Trace what we do here >>> + set -x >>> + >>> + rm -rf ${deb_img_dir} ${deb_dbg_dir} >>> + rm -rf ${deb_kern_hdrdir} >>> + rm -rf ${deb_libc_hdr_dir} >>> + >>> + # Stop tracing >>> + set +x >>> +} >>> + >>> +main clean ${*} >>> diff --git a/meta/recipes-kernel/linux/files/debian/isar/common.tmpl >>> b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl >>> new file mode 100644 >>> index 0000000..44f67b3 >>> --- /dev/null >>> +++ b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl >>> @@ -0,0 +1,50 @@ >>> +#!/bin/bash >>> +# Copyright (c) Mentor Graphics, a Siemens business, 2019 >>> +# SPDX-License-Identifier: MIT >>> + >>> +# Isar settings >>> +ARCH=${KERNEL_ARCH} >>> +KERNEL_PKG_IMAGE=linux-image-${KERNEL_NAME_PROVIDED} >>> +KERNEL_PKG_KERN_HEADERS=linux-headers-${KERNEL_NAME_PROVIDED} >>> +KERNEL_PKG_LIBC_HEADERS=linux-libc-dev >>> + >>> +# Constants >>> +KCONF=.config >>> + >>> +# Target directories >>> +deb_top_dir=${S}/debian >>> +deb_img_dir=${deb_top_dir}/${KERNEL_PKG_IMAGE} >>> +deb_dbg_dir=${deb_img_dir}-dbg >>> +deb_dtb_dir=${deb_img_dir}/${KERNEL_DTB_DIR} >>> +deb_kern_hdr_dir=${deb_top_dir}/${KERNEL_PKG_KERN_HEADERS} >>> +deb_libc_hdr_dir=${deb_top_dir}/${KERNEL_PKG_LIBC_HEADERS} >>> + >>> +# Array of packages to be generated >>> +declare -A kern_pkgs >>> + >>> +main() { >>> + local target=${1} >>> + >>> + if [ ! -f ${S}/debian/isar/${target} ]; then >>> + echo "error: ${target} is not a supported build target!" >&2 >>> + return 1 >>> + fi >>> + >>> + # create do_<x> variables for each package to be generated >>> + for p in $(unset DEB_HOST_ARCH; dh_listpackages); do >>> + kern_pkgs["${p}"]="1" >>> + done >>> + >>> + # variables to be exported >>> + export ARCH >>> + >>> + # are we cross-compiling? >>> + BUILD_ARCH=$(dpkg-architecture -qDEB_BUILD_ARCH) >>> + if [ "${BUILD_ARCH}" = "${DISTRO_ARCH}" ]; then >>> + # no, make sure CROSS_COMPILE isn't set >>> + unset CROSS_COMPILE >>> + fi >>> + >>> + # call the actual target script >>> + do_${target} || return ${?} >>> +} >>> diff --git a/meta/recipes-kernel/linux/files/debian/isar/install.tmpl >>> b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl >>> new file mode 100644 >>> index 0000000..806df5d >>> --- /dev/null >>> +++ b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl >>> @@ -0,0 +1,213 @@ >>> +#!/bin/bash >>> +# Copyright (c) Mentor Graphics, a Siemens business, 2019 >>> +# SPDX-License-Identifier: MIT >>> + >>> +# Load common stuff >>> +. ${S}/debian/isar/common || exit ${?} >>> + >>> +# Stop on error >>> +set -e >>> + >>> +do_install() { >>> + >>> + # check if our kernel was configured >>> + if [ ! -f "${O}/.config" ]; then >>> + echo "error: kernel not configured!" >&2 >>> + return 1 >>> + fi >>> + >>> + # load its configuration >>> + . ${O}/.config >>> + >>> + kimage="$(${MAKE} O=${O} -s --no-print-directory image_name)" >>> + krel="$(${MAKE} O=${O} -s --no-print-directory kernelrelease)" >>> + case "${ARCH}" in >>> + mips|powerpc) kimage_path="boot/vmlinux-${krel}" ;; >>> + um) kimage_path="usr/bin/vmlinux-${krel}" ;; >>> + *) kimage_path="boot/vmlinuz-${krel}" ;; >>> + esac >>> + >>> + print_settings >>> + >>> + # Trace what we do here >>> + set -x >>> + >>> + install_image >>> + install_hooks >>> + install_dtbs >>> + install_kmods >>> + install_headers >>> + >>> + # Stop tracing >>> + set +x >>> +} >>> + >>> +print_settings() { >>> + cat <<EOF >>> +Install settings: >>> +----------------- >>> +deb_dtb_dir=${deb_dtb_dir} >>> +deb_hdr_dir=${deb_hdr_dir} >>> +kimage=${kimage} >>> +kimage_path=${kimage_path} >>> + >>> +EOF >>> +} >>> + >>> +install_image() { >>> + install -m 755 -d ${deb_img_dir}/$(dirname ${kimage_path}) >>> + cp ${O}/${kimage} ${deb_img_dir}/${kimage_path} >>> + >>> + # Make sure arm64 kernels are decompressed >>> + if [ "${ARCH}" = "arm64" ]; then >>> + mv ${deb_img_dir}/${kimage_path} >>> ${deb_img_dir}/${kimage_path}.gz >>> + gunzip -f ${deb_img_dir}/${kimage_path}.gz >>> + fi >>> + >>> + install_image_debug >>> +} >>> + >>> +install_image_debug() { >>> + # Different tools want the image in different locations >>> + # perf >>> + mkdir -p ${deb_dbg_dir}/usr/lib/debug/lib/modules/${krel}/ >>> + cp ${O}/vmlinux ${deb_dbg_dir}/usr/lib/debug/lib/modules/${krel}/ >>> + # systemtap >>> + mkdir -p ${deb_dbg_dir}/usr/lib/debug/boot/ >>> + ln -s ../lib/modules/$version/vmlinux >>> ${deb_dbg_dir}/usr/lib/debug/boot/vmlinux-${krel} >>> + # kdump-tools >>> + ln -s lib/modules/${krel}/vmlinux >>> ${deb_dbg_dir}/usr/lib/debug/vmlinux-${krel} >>> +} >>> + >>> +install_hooks() { >>> + install -m 755 -d ${deb_img_dir}/etc/kernel/install.d >>> + install -m 755 -d ${deb_img_dir}/etc/kernel/postinst.d >>> + install -m 755 -d ${deb_img_dir}/etc/kernel/postrm.d >>> + install -m 755 -d ${deb_img_dir}/etc/kernel/prerm.d >>> + >>> + initrd="No" >>> + [ -z "${CONFIG_BLK_DEV_INITRD}" ] || initrd="Yes" >>> + >>> + for script in postinst postrm preinst prerm; do >>> + sed -i -e "s,INITRD=[A-Za-z0-9]*,INITRD=${initrd},g" >>> ${S}/debian/linux-image*.${script} >>> + sed -i -e "s,version=.*,version=${krel},g" >>> ${S}/debian/linux-image*.${script} >>> + done >>> +} >>> + >>> +install_dtbs() { >>> + [ -n "${CONFIG_OF}" ] || return 0 >>> + ${MAKE} O=${O} INSTALL_DTBS_PATH=${deb_dtb_dir} dtbs_install >>> +} >>> + >>> +install_kmods() { >>> + [ -n "${CONFIG_MODULES}" ] || return 0 >>> + ${MAKE} O=${O} INSTALL_MOD_PATH=${deb_img_dir} modules_install >>> + touch ${deb_img_dir}/lib/modules/${krel}/.fresh-install >>> + rm -fv ${deb_img_dir}/lib/modules/${krel}/build >>> + rm -fv ${deb_img_dir}/lib/modules/${krel}/source >>> + install_kmods_debug >>> +} >>> + >>> +kmods_sign() { >>> + [ -n "${CONFIG_MODULE_SIG_ALL}" ] || return 0 >>> + ${MAKE} O=${O} INSTALL_MOD_PATH=${deb_img_dir} modules_sign >>> +} >>> + >>> +install_kmods_debug() { >>> + [ -n "${CONFIG_DEBUG_INFO}" ] || return 0 >>> + >>> + kmod_inst_dir=${deb_img_dir}/lib/modules >>> + kmod_debug_dir=${deb_dbg_dir}/usr/lib/debug >>> + >>> + # copy kernels modules to usr/lib/debug >>> + mkdir -p ${kmod_debug_dir} >>> + tar -C ${kmod_inst_dir}/ -cO --exclude='modules.*' . | tar -C >>> ${kmod_debug_dir}/ -xf - >>> + # strip everything but debug sections for modules in usr/lib/debug >>> + find ${kmod_debug_dir} -name *.ko -exec ${CROSS_COMPILE}objcopy >>> --only-keep-debug {} \; >>> + # and strip debug sections from modules in lib/modules >>> + find ${kmod_inst_dir} -name *.ko -exec ${CROSS_COMPILE}objcopy >>> --strip-debug {} \; >>> + >>> + # re-sign stripped kernel modules >>> + kmods_sign >>> +} >>> + >>> +headers_check() { >>> + ${MAKE} O=${O} headers_check >>> +} >>> + >>> +libc_headers() { >>> + mkdir -p ${deb_libc_hdr_dir} >>> + ${MAKE} O=${O} headers_install >>> INSTALL_HDR_PATH=${deb_libc_hdr_dir}/usr >>> + host_arch=$(dpkg-architecture -a${DISTRO_ARCH} >>> -qDEB_HOST_MULTIARCH) >>> + mkdir ${deb_libc_hdr_dir}/usr/include/${host_arch} >>> + mv ${deb_libc_hdr_dir}/usr/include/asm >>> ${deb_libc_hdr_dir}/usr/include/${host_arch}/ >>> +} >>> + >>> +# the kernel does not seem to provide a way to cross-compile its >>> tools, we however >>> +# have qemu-user-static binaries in our buildchroot. Create a new >>> build tree for the >>> +# scripts and override HOSTCC to force a cross-compile >>> +kernel_tools_build() { >>> + odir="${S}/build-scripts" >>> + mflags="HOSTCC=${CROSS_COMPILE}gcc O=${odir} NOSTDINC_FLAGS=" >>> + rm -rf ${odir} && mkdir -p ${odir} && cp >>> ${S}/${KERNEL_BUILD_DIR}/.config ${odir} >>> + (unset LD_PRELOAD; cd ${odir} && make -C ${S} ${mflags} >>> olddefconfig scripts) >>> + (cd ${odir}; find scripts -type f -executable -exec file {} >>> \;|grep ELF|cut -d: -f1) >${src_hdr_files} >>> + tar -C ${odir} -cf - -T - <${src_hdr_files} | tar -C ${destdir} >>> -xf - >>> +} >>> + >>> +kernel_tools() { >>> + # remove object files >>> + find ${destdir}/scripts -type f -name '*.o' |xargs rm -f >>> + >>> + # we're all done if we aren't cross-compiling >>> + [ -n "${CROSS_COMPILE}" ] || return 0 >>> + >>> + # remove ELF executables from the linux-headers package >>> + find ${destdir}/scripts -type f -exec file {} \;|grep ELF|cut >>> -d: -f1|xargs rm -fv >>> + >>> + # cross-compile kernel tools to be shipped with linux-headers >>> + kernel_tools_build >>> +} >>> + >>> +kernel_headers() { >>> + destdir=${deb_kern_hdr_dir}/${KERNEL_HEADERS_DIR} >>> + src_hdr_files=$(mktemp) >>> + obj_hdr_files=$(mktemp) >>> + >>> + mkdir -p ${destdir} >>> + mkdir -p ${deb_kern_hdr_dir}/lib/modules/${krel} >>> + >>> + (cd ${S}; find . -name 'Makefile*' -o -name 'Kconfig*' -o -name >>> '*.pl') >>${src_hdr_files} >>> + (cd ${S}; find arch/*/include include scripts -type f -o -type >>> l) >>${src_hdr_files} >>> + (cd ${S}; find arch/${ARCH} -name module.lds -o -name >>> Kbuild.platforms -o -name Platform) >>${src_hdr_files} >>> + (cd ${S}; find $(find arch/${ARCH} -name include -o -name >>> scripts -type d) -type f) >>${src_hdr_files} >>> + >>> + (cd ${O}; find arch/${ARCH}/include Module.symvers include >>> scripts -type f) >>${obj_hdr_files} >>> + if [ -n "${CONFIG_STACK_VALIDATION}" ]; then >>> + (cd ${O}; find tools/objtool -type f -executable) >>> >>${obj_hdr_files} >>> + fi >>> + if [ -n "${CONFIG_GCC_PLUGINS}" ]; then >>> + (cd ${O}; find scripts/gcc-plugins -name *.so -o -name >>> gcc-common.h) >>${obj_hdr_files} >>> + fi >>> + >>> + # deploy files that were matched above >>> + tar -C ${S} -cf - -T - <${src_hdr_files} | tar -C ${destdir} -xf - >>> + tar -C ${O} -cf - -T - <${obj_hdr_files} | tar -C ${destdir} -xf - >>> + >>> + # add the kernel config >>> + cp ${O}/${KCONF} ${destdir}/.config >>> + >>> + # handle kernel development tools >>> + kernel_tools >>> + >>> + # create symlinks >>> + ln -sf /${KERNEL_HEADERS_DIR} >>> ${deb_kern_hdr_dir}/lib/modules/${krel}/build >>> +} >>> + >>> +install_headers() { >>> + headers_check >>> + [ -z ${kern_pkgs["linux-libc-headers"]} ] || libc_headers >>> + kernel_headers >>> +} >>> + >>> +main install ${*} >>> diff --git >>> a/meta/recipes-kernel/linux/files/debian/isar/version.cfg.tmpl >>> b/meta/recipes-kernel/linux/files/debian/isar/version.cfg.tmpl >>> new file mode 100644 >>> index 0000000..09fa364 >>> --- /dev/null >>> +++ b/meta/recipes-kernel/linux/files/debian/isar/version.cfg.tmpl >>> @@ -0,0 +1,2 @@ >>> +CONFIG_LOCALVERSION="-${KERNEL_LOCALVERSION}" >>> +# CONFIG_LOCALVERSION_AUTO is not set >>> diff --git >>> a/meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl >>> b/meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl >>> new file mode 100644 >>> index 0000000..b373801 >>> --- /dev/null >>> +++ b/meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl >>> @@ -0,0 +1,29 @@ >>> +#!/bin/sh >>> +# based on >>> https://salsa.debian.org/kernel-team/linux/blob/master/debian/templates/image.postinst.in >>> >>> + >>> +# Tell initramfs builder whether it's wanted >>> +export INITRD=Yes >>> + >>> +version=${PV}-${KERNEL_LOCALVERSION} >>> +image_path=/boot/${KERNEL_FILE}-${version} >>> + >>> +if [ "$1" != configure ]; then >>> + exit 0 >>> +fi >>> + >>> +depmod $version >>> + >>> +if [ -f /lib/modules/$version/.fresh-install ]; then >>> + change=install >>> +else >>> + change=upgrade >>> +fi >>> +linux-update-symlinks $change $version $image_path >>> +rm -f /lib/modules/$version/.fresh-install >>> + >>> +if [ -d /etc/kernel/postinst.d ]; then >>> + DEB_MAINT_PARAMS="$*" run-parts --report --exit-on-error >>> --arg=$version \ >>> + --arg=$image_path /etc/kernel/postinst.d >>> +fi >>> + >>> +exit 0 >>> diff --git >>> a/meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl >>> b/meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl >>> new file mode 100644 >>> index 0000000..1b5dbbe >>> --- /dev/null >>> +++ b/meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl >>> @@ -0,0 +1,35 @@ >>> +#!/bin/sh -e >>> +# based on >>> https://salsa.debian.org/kernel-team/linux/blob/master/debian/templates/image.postrm.in >>> >>> + >>> +# Tell initramfs builder whether it's wanted >>> +export INITRD=Yes >>> + >>> +version=${PV}-${KERNEL_LOCALVERSION} >>> +image_path=/boot/${KERNEL_FILE}-${version} >>> + >>> +rm -f /lib/modules/$version/.fresh-install >>> + >>> +if [ "$1" != upgrade ] && command -v linux-update-symlinks >>> >/dev/null; then >>> + linux-update-symlinks remove $version $image_path >>> +fi >>> + >>> +if [ -d /etc/kernel/postrm.d ]; then >>> + DEB_MAINT_PARAMS="$*" run-parts --report --exit-on-error >>> --arg=$version \ >>> + --arg=$image_path /etc/kernel/postrm.d >>> +fi >>> + >>> +if [ "$1" = purge ]; then >>> + for extra_file in modules.dep modules.isapnpmap modules.pcimap \ >>> + modules.usbmap modules.parportmap \ >>> + modules.generic_string modules.ieee1394map \ >>> + modules.ieee1394map modules.pnpbiosmap \ >>> + modules.alias modules.ccwmap modules.inputmap \ >>> + modules.symbols modules.ofmap \ >>> + modules.seriomap modules.\*.bin \ >>> + modules.softdep modules.devname; do >>> + eval rm -f /lib/modules/$version/$extra_file >>> + done >>> + rmdir /lib/modules/$version || true >>> +fi >>> + >>> +exit 0 >>> diff --git >>> a/meta/recipes-kernel/linux/files/debian/linux-image.preinst.tmpl >>> b/meta/recipes-kernel/linux/files/debian/linux-image.preinst.tmpl >>> new file mode 100644 >>> index 0000000..4f24019 >>> --- /dev/null >>> +++ b/meta/recipes-kernel/linux/files/debian/linux-image.preinst.tmpl >>> @@ -0,0 +1,25 @@ >>> +#!/bin/sh -e >>> +# based on >>> https://salsa.debian.org/kernel-team/linux/blob/master/debian/templates/image.preinst.in >>> >>> + >>> +# Tell initramfs builder whether it's wanted >>> +export INITRD=Yes >>> + >>> +version=${PV}-${KERNEL_LOCALVERSION} >>> +image_path=/boot/${KERNEL_FILE}-${version} >>> + >>> +if [ "$1" = abort-upgrade ]; then >>> + exit 0 >>> +fi >>> + >>> +if [ "$1" = install ]; then >>> + # Create a flag file for postinst >>> + mkdir -p /lib/modules/$version >>> + touch /lib/modules/$version/.fresh-install >>> +fi >>> + >>> +if [ -d /etc/kernel/preinst.d ]; then >>> + DEB_MAINT_PARAMS="$*" run-parts --report --exit-on-error >>> --arg=$version \ >>> + --arg=$image_path /etc/kernel/preinst.d >>> +fi >>> + >>> +exit 0 >>> diff --git >>> a/meta/recipes-kernel/linux/files/debian/linux-image.prerm.tmpl >>> b/meta/recipes-kernel/linux/files/debian/linux-image.prerm.tmpl >>> new file mode 100644 >>> index 0000000..cd697c0 >>> --- /dev/null >>> +++ b/meta/recipes-kernel/linux/files/debian/linux-image.prerm.tmpl >>> @@ -0,0 +1,21 @@ >>> +#!/bin/sh -e >>> +# based on >>> https://salsa.debian.org/kernel-team/linux/blob/master/debian/templates/image.prerm.in >>> >>> + >>> +# Tell initramfs builder whether it's wanted >>> +export INITRD=Yes >>> + >>> +version=${PV}-${KERNEL_LOCALVERSION} >>> +image_path=/boot/${KERNEL_FILE}-${version} >>> + >>> +if [ "$1" != remove ]; then >>> + exit 0 >>> +fi >>> + >>> +linux-check-removal $version >>> + >>> +if [ -d /etc/kernel/prerm.d ]; then >>> + DEB_MAINT_PARAMS="$*" run-parts --report --exit-on-error >>> --arg=$version \ >>> + --arg=$image_path /etc/kernel/prerm.d >>> +fi >>> + >>> +exit 0 >>> diff --git a/meta/recipes-kernel/linux/files/debian/rules.tmpl >>> b/meta/recipes-kernel/linux/files/debian/rules.tmpl >>> new file mode 100755 >>> index 0000000..c024768 >>> --- /dev/null >>> +++ b/meta/recipes-kernel/linux/files/debian/rules.tmpl >>> @@ -0,0 +1,39 @@ >>> +#!/usr/bin/make -f >>> + >>> +CROSS_COMPILE:=$(DEB_HOST_GNU_TYPE)- >>> + >>> +O:=$(CURDIR)/${KERNEL_BUILD_DIR} >>> +S:=$(CURDIR) >>> +deb_top_dir:=$(S)/debian >>> + >>> +# Dynamic variables to be passed to Isar build scripts >>> +isar_env=$(strip \ >>> + export CROSS_COMPILE='$(CROSS_COMPILE)' && \ >>> + export DEB_BUILD_PROFILES='$(DEB_BUILD_PROFILES)' && \ >>> + export DEB_HOST_GNU_TYPE='$(DEB_HOST_GNU_TYPE)' && \ >>> + export MAKE='$(MAKE)' && \ >>> + export O='${O}' \ >>> + export S='${S}' \ >>> +) >>> + >>> +%: >>> + dh $(@) >>> + >>> +.PHONY: override_dh_auto_clean >>> +override_dh_auto_clean: >>> + $(isar_env) && bash $(deb_top_dir)/isar/clean >>> + >>> +.PHONY: override_dh_auto_build >>> +override_dh_auto_build: >>> + $(isar_env) && bash $(deb_top_dir)/isar/build >>> + >>> +override_dh_auto_install: >>> + $(isar_env) && bash $(deb_top_dir)/isar/install >>> + >>> +.PHONY: override_dh_auto_test >>> +override_dh_auto_test: >>> + true >>> + >>> +PHONY: override_dh_strip >>> +override_dh_strip: >>> + dh_strip -Xvmlinux --no-automatic-dbgsym >>> diff --git a/meta/recipes-kernel/linux/linux-custom.inc >>> b/meta/recipes-kernel/linux/linux-custom.inc >>> index 712f52b..bf1646a 100644 >>> --- a/meta/recipes-kernel/linux/linux-custom.inc >>> +++ b/meta/recipes-kernel/linux/linux-custom.inc >>> @@ -2,89 +2,183 @@ >>> # >>> # This software is a part of ISAR. >>> # Copyright (c) Siemens AG, 2018 >>> +# Copyright (c) Mentor Graphics, a Siemens business, 2019 >>> # >>> # SPDX-License-Identifier: MIT >>> -FILESEXTRAPATHS_prepend := "${FILE_DIRNAME}/files:" >>> +# Settings that would typically be done from the custom kernel recipe >>> +# ------------------------------------------------------------------- >>> +CHANGELOG_V = "${PV}+${PR}" >>> DESCRIPTION ?= "Custom kernel" >>> +MAINTAINER ?= "isar-users <isar-users@googlegroups.com>" >>> + >>> +KBUILD_DEPENDS ?= "build-essential:native, \ >>> + libelf-dev:native, \ >>> + libncurses-dev:native, \ >>> + libssl-dev:native, \ >>> + bc, \ >>> + bison, \ >>> + cpio, \ >>> + flex, \ >>> + git, \ >>> + kmod, \ >>> + libssl-dev," >>> + >>> +KERNEL_DEBIAN_DEPENDS ?= "initramfs-tools | linux-initramfs-tool, \ >>> + kmod, \ >>> + linux-base (>= 4.3~)," >>> + >>> +KERNEL_HEADERS_DEBIAN_DEPENDS ?= "libc6, \ >>> + libssl1.1," >>> + >>> +KERNEL_LIBC_DEV_DEPLOY ?= "0" >>> + >>> +# Settings that may be changed on a per distro, machine or layer basis >>> +# -------------------------------------------------------------------- >>> + >>> +KERNEL_LOCALVERSION ?= "-${KERNEL_NAME_PROVIDED}-${PR}" >>> +KERNEL_DTB_DIR ?= "usr/lib/linux-image-${PV}-${KERNEL_LOCALVERSION}" >>> +KERNEL_HEADERS_DIR ?= >>> "usr/src/linux-headers-${PV}-${KERNEL_LOCALVERSION}" >>> -KERNEL_NAME_PROVIDED ?= "${@ d.getVar('PN', >>> True).partition('linux-')[2]}" >>> KERNEL_DEFCONFIG ?= "" >>> +# Add our template meta-data to the sources >>> +FILESEXTRAPATHS_prepend := "${FILE_DIRNAME}/files:" >>> +SRC_URI += "file://debian" >>> + >>> +# Variables and files that make our templates >>> +# ------------------------------------------- >>> + >>> +TEMPLATE_FILES += " \ >>> + debian/control.tmpl \ >>> + debian/isar/build.tmpl \ >>> + debian/isar/clean.tmpl \ >>> + debian/isar/common.tmpl \ >>> + debian/isar/install.tmpl \ >>> + debian/isar/version.cfg.tmpl \ >>> + debian/linux-image.postinst.tmpl \ >>> + debian/linux-image.postrm.tmpl \ >>> + debian/linux-image.preinst.tmpl \ >>> + debian/linux-image.prerm.tmpl \ >>> + debian/rules.tmpl \ >>> +" >>> + >>> +TEMPLATE_VARS += " \ >>> + HOST_ARCH \ >>> + KBUILD_DEPENDS \ >>> + KERNEL_ARCH \ >>> + KERNEL_DEBIAN_DEPENDS \ >>> + KERNEL_BUILD_DIR \ >>> + KERNEL_DTB_DIR \ >>> + KERNEL_FILE \ >>> + KERNEL_HEADERS_DEBIAN_DEPENDS \ >>> + KERNEL_HEADERS_DIR \ >>> + KERNEL_LOCALVERSION \ >>> + KERNEL_NAME_PROVIDED \ >>> +" >>> + >>> +inherit dpkg >>> +inherit template >>> + >>> +# Derive name of the kernel packages from the name of this recipe >>> +KERNEL_NAME_PROVIDED ?= "${@ d.getVar('PN', >>> True).partition('linux-')[2]}" >>> + >>> +# Make bitbake know we will be producing linux-image and >>> linux-headers packages >>> python() { >>> kernel_name = d.getVar("KERNEL_NAME_PROVIDED", True) >>> d.setVar('PROVIDES', 'linux-image-' + kernel_name + ' ' + \ >>> 'linux-headers-' + kernel_name) >>> } >>> -inherit dpkg-base >>> - >>> -SRC_URI += "file://build-kernel.sh" >>> - >>> -KBUILD_DEPENDS ?= " \ >>> - build-essential:native \ >>> - libssl-dev \ >>> - libelf-dev \ >>> - bc \ >>> - git \ >>> - kmod \ >>> - bison \ >>> - flex \ >>> - cpio \ >>> - libncurses-dev" >>> -KERNEL_DEBIAN_DEPENDS ?= "initramfs-tools | linux-initramfs-tool, >>> kmod, linux-base (>= 4.3~)" >>> -KERNEL_HEADERS_DEBIAN_DEPENDS ?= "libc6, libssl1.1" >>> - >>> -KERNEL_LIBC_DEV_DEPLOY ?= "0" >>> - >>> -do_install_builddeps() { >>> - dpkg_do_mounts >>> - E="${@ isar_export_proxies(d)}" >>> - sudo -E chroot ${BUILDCHROOT_DIR} \ >>> - apt-get update \ >>> - -o Dir::Etc::SourceList="sources.list.d/isar-apt.list" \ >>> - -o Dir::Etc::SourceParts="-" \ >>> - -o APT::Get::List-Cleanup="0" >>> - sudo -E chroot ${BUILDCHROOT_DIR} \ >>> - apt-get install \ >>> - -y -o Debug::pkgProblemResolver=yes \ >>> - --no-install-recommends ${KBUILD_DEPENDS} >>> - dpkg_undo_mounts >>> +def get_kernel_arch(d): >>> + distro_arch = d.getVar("DISTRO_ARCH") >>> + if distro_arch == "amd64": >>> + kernel_arch = "x86" >>> + elif distro_arch == "arm64": >>> + kernel_arch = "arm64" >>> + elif distro_arch == "armhf": >>> + kernel_arch = "arm" >>> + elif distro_arch == "mipsel": >>> + kernel_arch = "mips" >>> + else: >>> + kernel_arch = "" >>> + return kernel_arch >>> + >>> +KERNEL_ARCH ??= "${@get_kernel_arch(d)}" >>> + >>> +def config_fragments(d): >>> + fragments = [] >>> + sources = d.getVar("SRC_URI").split() >>> + for s in sources: >>> + _, _, local, _, _, parm = bb.fetch.decodeurl(s) >>> + apply = parm.get("apply") >>> + if apply == "no": >>> + continue >>> + base, ext = os.path.splitext(os.path.basename(local)) >>> + if ext and ext in (".cfg"): >>> + fragments.append(local) >>> + return fragments >>> + >>> +do_prepare_build_prepend() { >>> + # copy meta-data over to source tree >>> + rm -rf ${S}/debian >>> + cp -r ${WORKDIR}/debian ${S}/ >>> + >>> + # remove templates from the source tree >>> + find ${S}/debian -name *.tmpl | xargs rm -f >>> + >>> + # rename install/remove hooks to match user-specified name for >>> our linux-image package >>> + mv ${S}/debian/linux-image.postinst >>> ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.postinst >>> + mv ${S}/debian/linux-image.postrm >>> ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.postrm >>> + mv ${S}/debian/linux-image.preinst >>> ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.preinst >>> + mv ${S}/debian/linux-image.prerm >>> ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.prerm >>> + >>> + # produce a changelog for our kernel build >>> + deb_add_changelog >>> } >>> -addtask install_builddeps after do_prepare_build before do_dpkg_build >>> -# apt and reprepro may not run in parallel, acquire the Isar lock >>> -do_install_builddeps[lockfiles] += "${REPO_ISAR_DIR}/isar.lock" >>> - >>> -addtask devshell after do_install_builddeps >>> +# build directory for our "full" kernel build >>> +KERNEL_BUILD_DIR = "build-full" >>> -dpkg_runbuild() { >>> - chmod +x ${WORKDIR}/build-kernel.sh >>> - KERNEL_CONFIG_TARGET="${KERNEL_DEFCONFIG}" >>> +dpkg_configure_kernel() { >>> + config_target="${KERNEL_DEFCONFIG}" >>> + rm -rf ${S}/${KERNEL_BUILD_DIR} && mkdir -p >>> ${S}/${KERNEL_BUILD_DIR} >>> if [ -n "${KERNEL_DEFCONFIG}" ]; then >>> if [ -e "${WORKDIR}/${KERNEL_DEFCONFIG}" ]; then >>> - cp ${WORKDIR}/${KERNEL_DEFCONFIG} ${S}/.config >>> - KERNEL_CONFIG_TARGET="olddefconfig" >>> + cp ${WORKDIR}/${KERNEL_DEFCONFIG} >>> ${S}/${KERNEL_BUILD_DIR}/.config >>> + config_target="olddefconfig" >>> fi >>> else >>> - KERNEL_CONFIG_TARGET="defconfig" >>> + config_target="defconfig" >>> fi >>> - E="${@ isar_export_proxies(d)}" >>> - >>> - export PV=${PV} >>> - export KERNEL_NAME=${KERNEL_NAME_PROVIDED} >>> - >>> - export KBUILD_DEPENDS="${KBUILD_DEPENDS}" >>> - export KERNEL_CONFIG_TARGET >>> - export KERNEL_DEBIAN_DEPENDS="${KERNEL_DEBIAN_DEPENDS}" >>> - export KERNEL_FILE="${KERNEL_FILE}" >>> - export >>> KERNEL_HEADERS_DEBIAN_DEPENDS="${KERNEL_HEADERS_DEBIAN_DEPENDS}" >>> + # copy config fragments over to the kernel tree >>> + src_frags="${@ " ".join(config_fragments(d)) }" >>> + out_frags="${@ " ".join(map(lambda frag: 'debian/fragments/' + >>> frag, config_fragments(d))) }" >>> + rm -rf ${S}/debian/fragments >>> + if [ -n "${src_frags}" ]; then >>> + mkdir -p ${S}/debian/fragments >>> + (cd ${WORKDIR} && cp ${src_frags} ${S}/debian/fragments/) >>> + fi >>> - sudo -E chroot --userspec=$( id -u ):$( id -g ) >>> ${BUILDCHROOT_DIR} ${PP}/build-kernel.sh ${PP}/${PPS} ${DISTRO_ARCH} >>> + sudo -E chroot --userspec=$(id -u):$(id -g) ${BUILDCHROOT_DIR} >>> sh -c " \ >>> + export ARCH=${KERNEL_ARCH} >>> && \ >>> + cd ${PP}/${PPS} >>> && \ >>> + make O=${KERNEL_BUILD_DIR} ${config_target} >>> && \ >>> + >>> ./scripts/kconfig/merge_config.sh \ >>> + -O ${KERNEL_BUILD_DIR}/ \ >>> + ${KERNEL_BUILD_DIR}/.config \ >>> + ${out_frags} \ >>> + >>> debian/isar/version.cfg \ >>> + " >>> +} >>> +dpkg_runbuild_prepend() { >>> + profiles="${BASE_DISTRO}" >>> if [ "${KERNEL_LIBC_DEV_DEPLOY}" != "1" ]; then >>> - rm -f ${WORKDIR}/linux-libc-dev_${PV}*.deb >>> + profiles="${profiles} nolibcdev" >>> fi >>> + export DEB_BUILD_PROFILES="${profiles}" >>> + dpkg_configure_kernel >>> } >>> >> I would recommend this functional change to the patch to get away >> without patch 4: >> >> diff --git a/meta/recipes-kernel/linux/files/debian/isar/install.tmpl >> b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl >> index 806df5d..67b7ce3 100644 >> --- a/meta/recipes-kernel/linux/files/debian/isar/install.tmpl >> +++ b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl >> @@ -143,30 +143,9 @@ libc_headers() { >> mv ${deb_libc_hdr_dir}/usr/include/asm >> ${deb_libc_hdr_dir}/usr/include/${host_arch}/ >> } >> -# the kernel does not seem to provide a way to cross-compile its >> tools, we however >> -# have qemu-user-static binaries in our buildchroot. Create a new >> build tree for the >> -# scripts and override HOSTCC to force a cross-compile >> -kernel_tools_build() { >> - odir="${S}/build-scripts" >> - mflags="HOSTCC=${CROSS_COMPILE}gcc O=${odir} NOSTDINC_FLAGS=" >> - rm -rf ${odir} && mkdir -p ${odir} && cp >> ${S}/${KERNEL_BUILD_DIR}/.config ${odir} >> - (unset LD_PRELOAD; cd ${odir} && make -C ${S} ${mflags} >> olddefconfig scripts) >> - (cd ${odir}; find scripts -type f -executable -exec file {} >> \;|grep ELF|cut -d: -f1) >${src_hdr_files} >> - tar -C ${odir} -cf - -T - <${src_hdr_files} | tar -C ${destdir} >> -xf - >> -} >> - >> kernel_tools() { >> # remove object files >> find ${destdir}/scripts -type f -name '*.o' |xargs rm -f >> - >> - # we're all done if we aren't cross-compiling >> - [ -n "${CROSS_COMPILE}" ] || return 0 >> - >> - # remove ELF executables from the linux-headers package >> - find ${destdir}/scripts -type f -exec file {} \;|grep ELF|cut -d: >> -f1|xargs rm -fv >> - >> - # cross-compile kernel tools to be shipped with linux-headers >> - kernel_tools_build >> } >> kernel_headers() { >> diff --git a/meta/recipes-kernel/linux/files/debian/rules.tmpl >> b/meta/recipes-kernel/linux/files/debian/rules.tmpl >> index c024768..bea21f9 100755 >> --- a/meta/recipes-kernel/linux/files/debian/rules.tmpl >> +++ b/meta/recipes-kernel/linux/files/debian/rules.tmpl >> @@ -36,4 +36,4 @@ override_dh_auto_test: >> PHONY: override_dh_strip >> override_dh_strip: >> - dh_strip -Xvmlinux --no-automatic-dbgsym >> + unset DEB_HOST_GNU_TYPE && dh_strip -Xvmlinux --no-automatic-dbgsym >> >> >> Then we can introduce the correct building of linux-headers once we can >> generate both host and target packages with Isar. And we do not need to >> infect the host buildchroot. > > Hi Jan, > > Thanks for checking our options and devising this patch. Much > appreciated. How should we proceed? Would you like to post v5 or should > I? Should I add your Signed-off-by to the last commit (the one that > heavily changes linux-custom.inc) to give you review/contribution credits? If you have the time, just fold my change in and send a v5. The only "substantial" addition of mine is the unset DEB_HOST_GNU_TYPE. Maybe we should also leave a comment in that line why this is currently needed. BTW, build tests with this against jailhouse-images worked fine. I'll push this also to our LAVA lab later on (via xenomai-images) to have a boot test on all popular archs. Jan -- Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 83+ messages in thread
* Re: [PATCH L-C v4 5/5] linux-custom: rewrite to no longer depend on the kernel's builddeb 2019-11-25 16:06 ` [PATCH L-C v4 5/5] linux-custom: rewrite to no longer depend on the kernel's builddeb Cedric Hombourger 2019-11-25 19:55 ` Jan Kiszka @ 2019-11-26 17:10 ` Gylstorff Quirin 1 sibling, 0 replies; 83+ messages in thread From: Gylstorff Quirin @ 2019-11-26 17:10 UTC (permalink / raw) To: isar-users On 11/25/19 5:06 PM, Cedric Hombourger wrote: > Regain control over the packaging of the Linux kernel by providing our > own debian recipes (debian,{control,rules} files and friends) instead > of using the kernel's builddeb script. This will allow generation of > packages for "perf" (not included in this changeset) with per distro > scheme (Debian and Ubuntu have differences) and maintain compatibility > with upstream packages generated from different source packages but > dependent on a certain deployment scheme (e.g. linux-base for Debian) > > Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com> > --- > doc/custom_kernel.md | 80 +++++++ > .../linux/files/build-kernel.sh | 125 ---------- > meta/recipes-kernel/linux/files/debian/compat | 1 + > .../linux/files/debian/control.tmpl | 37 +++ > .../linux/files/debian/isar/build.tmpl | 45 ++++ > .../linux/files/debian/isar/clean.tmpl | 21 ++ > .../linux/files/debian/isar/common.tmpl | 50 ++++ > .../linux/files/debian/isar/install.tmpl | 213 ++++++++++++++++++ > .../linux/files/debian/isar/version.cfg.tmpl | 2 + > .../files/debian/linux-image.postinst.tmpl | 29 +++ > .../files/debian/linux-image.postrm.tmpl | 35 +++ > .../files/debian/linux-image.preinst.tmpl | 25 ++ > .../linux/files/debian/linux-image.prerm.tmpl | 21 ++ > .../linux/files/debian/rules.tmpl | 39 ++++ > meta/recipes-kernel/linux/linux-custom.inc | 210 ++++++++++++----- > 15 files changed, 750 insertions(+), 183 deletions(-) > create mode 100644 doc/custom_kernel.md > delete mode 100644 meta/recipes-kernel/linux/files/build-kernel.sh > create mode 100644 meta/recipes-kernel/linux/files/debian/compat > create mode 100644 meta/recipes-kernel/linux/files/debian/control.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/isar/build.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/isar/clean.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/isar/common.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/isar/install.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/isar/version.cfg.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.preinst.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.prerm.tmpl > create mode 100755 meta/recipes-kernel/linux/files/debian/rules.tmpl > > diff --git a/doc/custom_kernel.md b/doc/custom_kernel.md > new file mode 100644 > index 0000000..31319d4 > --- /dev/null > +++ b/doc/custom_kernel.md > @@ -0,0 +1,80 @@ > +# Custom kernel recipe for Isar > + > +## Contents > + > + - [Summary](#summary) > + - [Features](#features) > + - [Future](#future) > + - [Examples](#examples) > + > +## Summary > + > +Isar provides a recipe to build custom kernels for Debian-based distributions. > +It uses templates to generate the debian meta-data (such as debian/control) and > +Debian's [BuildProfiles](https://wiki.debian.org/BuildProfileSpec) to handle > +some of the distro specific variations. It should be noted that Isar has moved > +away from using the kernel's builddeb script since it would not generate all > +the packages we need (and in particular perf). > + > +## Features > + > +The linux-custom recipe provides support for: > + > + 1. Sources to the custom Linux kernel may be specified via `SRC_URI` > + > + 2. Configure the kernel via an in-tree or an external `defconfig` via > + `KERNEL_DEFCONFIG` > + > + 3. Integrate kernel configuration tweaks via configuration fragments (`.cfg` > + files) > + > + 4. Patches to the linux kernel may be specified via `SRC_URI` > + > + 5. Ensure that the Isar recipe `PV` matches the kernel release identifier > + (`KERNEL_RELEASE`) > + > + 6. Produce a `linux-image` package that ships the kernel image and modules > + > + 7. Allow the name of the kernel image to be changed via `KERNEL_FILE` (defaults > + to `vmlinuz`) > + > + 8. Produce a `linux-headers` package which includes kernel headers and kbuild > + scripts/tools > + > + 9. The `linux-headers` package shall support native and cross compiles of > + out-of-tree kernel modules > + > + 10. Produce a `linux-libc-dev` package to support user-land builds > + > + 11. Only build/ship the `linux-libc-dev` package if instructed to > + (`KERNEL_LIBC_DEV_DEPLOY` equals to `"1"`) > + > + 12. Support both native and cross compiles (`ISAR_CROSS_COMPILE`) > + > + 13. Support for the following kernel architectures: > + > + * arm > + * arm64 > + * mips > + * x86 > + > + 14. Support `devshell` (kernel configuration shall be applied) > + > +## Future > + > +In the future, the recipe may be extended to: > + > + 1. Package perf > + > + 2. Support inclusion/build of dts files listed in `SRC_URI` > + > + 3. Be compatible with Ubuntu > + > +## Examples > + > +The linux-custom recipe is currently used by the linux-mainline package and is > +used mainline recipe may be used for some basic testing. This recipe is being > +used by the following machines: > + > + * de0-nano-soc > + * qemumipsel > diff --git a/meta/recipes-kernel/linux/files/build-kernel.sh b/meta/recipes-kernel/linux/files/build-kernel.sh > deleted file mode 100644 > index 386e58b..0000000 > --- a/meta/recipes-kernel/linux/files/build-kernel.sh > +++ /dev/null > @@ -1,125 +0,0 @@ > -#!/bin/bash > -# > -# Custom kernel build > -# > -# This software is a part of ISAR. > -# Copyright (c) Siemens AG, 2018 > -# > -# SPDX-License-Identifier: MIT > - > -source /isar/common.sh > - > -host_arch=$(dpkg --print-architecture) > - > -if [ "$host_arch" != "$target_arch" ]; then > - case $target_arch in > - armhf) > - export ARCH=arm > - export CROSS_COMPILE="arm-linux-gnueabihf-" > - ;; > - arm64) > - export ARCH=arm64 > - export CROSS_COMPILE="aarch64-linux-gnu-" > - ;; > - mipsel) > - export ARCH=mips > - export CROSS_COMPILE="mipsel-linux-gnu-" > - ;; > - *) > - echo "error: unsupported architecture ($target_arch)" > - exit 1 > - ;; > - esac > -fi > - > -REPACK_DIR="$1/../repack" > -REPACK_LINUX_IMAGE_DIR="${REPACK_DIR}/linux-image" > -REPACK_LINUX_HEADERS_DIR="${REPACK_DIR}/linux-headers" > - > -make ${KERNEL_CONFIG_TARGET} || exit ${?} > - > -KV=$( make -s kernelrelease ) > -if [ "${KV}" != "${PV}" ]; then > - echo "ERROR: Recipe PV is \"${PV}\" but should be \"${KV}\"" 1>&2 > - echo "ERROR: Probably due to CONFIG_LOCALVERSION" 1>&2 > - exit 1 > -fi > - > -rm -f .version > -KBUILD_DEBARCH=$target_arch make -j $(($(nproc) * 2)) deb-pkg > - > -rm -rf "${REPACK_DIR}" > -mkdir -p "${REPACK_DIR}" > -mkdir -p "${REPACK_LINUX_IMAGE_DIR}" > -mkdir -p "${REPACK_LINUX_HEADERS_DIR}" > - > -cp -a debian "${REPACK_DIR}" > - > -# dpkg-gencontrol performs cross-incompatible checks on the > -# Architecture field; trick it to accept the control file > -sed -i "s/Architecture: .*/Architecture: any/" "${REPACK_DIR}/debian/control" > - > -cd .. > - > -dpkg-deb -R linux-image-${PV}_${PV}-1_*.deb "${REPACK_LINUX_IMAGE_DIR}" > -dpkg-deb -R linux-headers-${PV}_${PV}-1_*.deb "${REPACK_LINUX_HEADERS_DIR}" > - > -dpkg-gencontrol -crepack/debian/control \ > - -lrepack/debian/changelog \ > - -frepack/debian/files \ > - -plinux-image-${PV} \ > - -P"${REPACK_LINUX_IMAGE_DIR}" \ > - -DPackage="linux-image-${KERNEL_NAME}" \ > - -DSection=kernel \ > - -DPriority=required \ > - -DDepends="${KERNEL_DEBIAN_DEPENDS}" \ > - -DArchitecture=$target_arch > - > -# Add Debian-like link installation to postinst > -mkdir -p ${REPACK_LINUX_IMAGE_DIR}/lib/modules/${PV} > -touch "${REPACK_LINUX_IMAGE_DIR}/lib/modules/${PV}/.fresh-install" > -sed -i "${REPACK_LINUX_IMAGE_DIR}/DEBIAN/postinst" \ > - -e "/^set -e$/a\\ > -\\ > -if [ -f /lib/modules/${PV}/.fresh-install ]; then\\ > - change=install\\ > -else\\ > - change=upgrade\\ > -fi\\ > -linux-update-symlinks \$change ${PV} /boot/${KERNEL_FILE}-${PV}\\ > -rm -f /lib/modules/${PV}/.fresh-install" > - > -# Add Debian-like link removal to postrm > -sed -i "${REPACK_LINUX_IMAGE_DIR}/DEBIAN/postrm" \ > - -e "/^set -e$/a\\ > -\\ > -rm -f /lib/modules/${PV}/.fresh-install\\ > -\\ > -if [ \"\$1\" != upgrade ] && command -v linux-update-symlinks >/dev/null; then\\ > - linux-update-symlinks remove ${PV} /boot/${KERNEL_FILE}-${PV}\\ > -fi" > - > -# Make sure arm64 kernels are decompressed > -if [ "$target_arch" = "arm64" ]; then > - kernel_file="${REPACK_LINUX_IMAGE_DIR}/boot/${KERNEL_FILE}-${PV}" > - mv "${kernel_file}" "${kernel_file}.gz" > - gunzip "${kernel_file}.gz" > -fi > - > -dpkg-gencontrol -crepack/debian/control \ > - -lrepack/debian/changelog \ > - -frepack/debian/files \ > - -plinux-headers-${PV} \ > - -P"${REPACK_LINUX_HEADERS_DIR}" \ > - -Vkernel:debarch="${KERNEL_NAME}" \ > - -DPackage="linux-headers-${KERNEL_NAME}" \ > - -DSection=kernel \ > - -DDepends="${KERNEL_HEADERS_DEBIAN_DEPENDS}" \ > - -DArchitecture=$target_arch > - > -fakeroot dpkg-deb -b "${REPACK_LINUX_IMAGE_DIR}" \ > - linux-image-${KERNEL_NAME}_${PV}-1_${KERNEL_NAME}.deb > -rm -f linux-image-${PV}_${PV}-1_*.deb > -fakeroot dpkg-deb -b "${REPACK_LINUX_HEADERS_DIR}" \ > - linux-headers-${KERNEL_NAME}_${PV}-1_${KERNEL_NAME}.deb > -rm -f linux-headers-${PV}_${PV}-1_*.deb > diff --git a/meta/recipes-kernel/linux/files/debian/compat b/meta/recipes-kernel/linux/files/debian/compat > new file mode 100644 > index 0000000..ec63514 > --- /dev/null > +++ b/meta/recipes-kernel/linux/files/debian/compat > @@ -0,0 +1 @@ > +9 > diff --git a/meta/recipes-kernel/linux/files/debian/control.tmpl b/meta/recipes-kernel/linux/files/debian/control.tmpl > new file mode 100644 > index 0000000..7236732 > --- /dev/null > +++ b/meta/recipes-kernel/linux/files/debian/control.tmpl > @@ -0,0 +1,37 @@ > +Source: linux-${KERNEL_NAME_PROVIDED} > +Section: kernel > +Priority: optional > +Maintainer: ${MAINTAINER} > +Build-Depends: bc, kmod, cpio, ${KBUILD_DEPENDS} > +Homepage: http://www.kernel.org/ > + > +Package: linux-image-${KERNEL_NAME_PROVIDED} > +Architecture: any > +Depends: ${KERNEL_DEBIAN_DEPENDS} > +Description: ${KERNEL_NAME_PROVIDED} Linux kernel, version ${PV}-${KERNEL_LOCALVERSION} > + This package contains the Linux kernel, modules and corresponding other > + files, version: ${PV}-${KERNEL_LOCALVERSION}. > + > +Package: linux-headers-${KERNEL_NAME_PROVIDED} > +Architecture: any > +Depends: ${KERNEL_HEADERS_DEBIAN_DEPENDS}, ${perl:Depends}, ${shlib:Depends} > +Description: ${KERNEL_NAME_PROVIDED} Linux kernel headers for ${PV}-${KERNEL_LOCALVERSION} > + This package provides kernel header files for ${PV}-${KERNEL_LOCALVERSION} on ${DISTRO_ARCH} > + . > + This is useful for people who need to build external modules > + > +Package: linux-libc-dev > +Build-Profiles: <!nolibcdev> > +Section: devel > +Provides: linux-kernel-headers > +Architecture: any > +Description: Linux support headers for userspace development > + This package provides userspaces headers from the Linux kernel. These headers > + are used by the installed headers for GNU glibc and other system libraries. > + > +Package: linux-image-${KERNEL_NAME_PROVIDED}-dbg > +Section: debug > +Architecture: any > +Description: Linux kernel debugging symbols for ${PV}-${KERNEL_LOCALVERSION} > + This package will come in handy if you need to debug the kernel. It provides > + all the necessary debug symbols for the kernel and its modules. > diff --git a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl > new file mode 100644 > index 0000000..1775b19 > --- /dev/null > +++ b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl > @@ -0,0 +1,45 @@ > +#!/bin/bash > +# Copyright (c) Mentor Graphics, a Siemens business, 2019 > +# SPDX-License-Identifier: MIT > + > +# Load common stuff > +. ${S}/debian/isar/common || exit ${?} > + > +do_build() { > + > + # Print a few things that are of particular interest > + print_settings > + > + # Process existing kernel configuration to make sure it is complete > + # (use defaults for options that were not specified) > + ${MAKE} O=${KERNEL_BUILD_DIR} olddefconfig prepare || exit ${?} > + > + # Check if the recipe's PV makes sense > + KR=$(${MAKE} O=${KERNEL_BUILD_DIR} -s --no-print-directory kernelrelease) > + eval $(grep ^CONFIG_LOCALVERSION= ${KERNEL_BUILD_DIR}/${KCONF} || true) > + if [ "${PV}-${KERNEL_LOCALVERSION}" != "${KR}" ]; then > + echo "ERROR: Recipe version (${PV}-${KERNEL_LOCALVERSION}) does not seem to match the kernelrelease (${KR})!" 1>&2 > + echo "ERROR: Make sure the kernel version in your NAME/PV/PR settings and/or CONFIG_LOCALVERSION are aligned" 1>&2 > + exit 1 > + fi we have some CI use case where we build the latest git release could we add something like this - if [ "${PV}-${KERNEL_LOCALVERSION}" != "${KR}" ]; then - echo "ERROR: Recipe version (${PV}-${KERNEL_LOCALVERSION}) does not seem to match the kernelrelease (${KR})!" 1>&2 - echo "ERROR: Make sure the kernel version in your NAME/PV/PR settings and/or CONFIG_LOCALVERSION are aligned" 1>&2 - exit 1 + if [ "${PV}" =~ "latest" ]; then + if [ "${PV}-${KERNEL_LOCALVERSION}" != "${KR}" ]; then + echo "ERROR: Recipe version (${PV}-${KERNEL_LOCALVERSION}) does not seem to match the kernelrelease (${KR})!" 1>&2 + echo "ERROR: Make sure the kernel version in your NAME/PV/PR settings and/or CONFIG_LOCALVERSION are aligned" 1>&2 + exit 1 + fi > + > + # Trace what we do here > + set -x > + > + # Build the Linux kernel > + ${MAKE} O=${KERNEL_BUILD_DIR} -j $(nproc) || exit ${?} > + > + # Stop tracing > + set +x > +} > + > +print_settings() { > + cat <<EOF > +# Build settings: > +# --------------- > +# ARCH=${ARCH} > +# CROSS_COMPILE=${CROSS_COMPILE} > +EOF > +} > + > +main build ${*} > diff --git a/meta/recipes-kernel/linux/files/debian/isar/clean.tmpl b/meta/recipes-kernel/linux/files/debian/isar/clean.tmpl > new file mode 100644 > index 0000000..f903987 > --- /dev/null > +++ b/meta/recipes-kernel/linux/files/debian/isar/clean.tmpl > @@ -0,0 +1,21 @@ > +#!/bin/bash > +# Copyright (c) Mentor Graphics, a Siemens business, 2019 > +# SPDX-License-Identifier: MIT > + > +# Load common stuff > +. ${S}/debian/isar/common || exit ${?} > + > +do_clean() { > + > + # Trace what we do here > + set -x > + > + rm -rf ${deb_img_dir} ${deb_dbg_dir} > + rm -rf ${deb_kern_hdrdir} > + rm -rf ${deb_libc_hdr_dir} > + > + # Stop tracing > + set +x > +} > + > +main clean ${*} > diff --git a/meta/recipes-kernel/linux/files/debian/isar/common.tmpl b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl > new file mode 100644 > index 0000000..44f67b3 > --- /dev/null > +++ b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl > @@ -0,0 +1,50 @@ > +#!/bin/bash > +# Copyright (c) Mentor Graphics, a Siemens business, 2019 > +# SPDX-License-Identifier: MIT > + > +# Isar settings > +ARCH=${KERNEL_ARCH} > +KERNEL_PKG_IMAGE=linux-image-${KERNEL_NAME_PROVIDED} > +KERNEL_PKG_KERN_HEADERS=linux-headers-${KERNEL_NAME_PROVIDED} > +KERNEL_PKG_LIBC_HEADERS=linux-libc-dev > + > +# Constants > +KCONF=.config > + > +# Target directories > +deb_top_dir=${S}/debian > +deb_img_dir=${deb_top_dir}/${KERNEL_PKG_IMAGE} > +deb_dbg_dir=${deb_img_dir}-dbg > +deb_dtb_dir=${deb_img_dir}/${KERNEL_DTB_DIR} > +deb_kern_hdr_dir=${deb_top_dir}/${KERNEL_PKG_KERN_HEADERS} > +deb_libc_hdr_dir=${deb_top_dir}/${KERNEL_PKG_LIBC_HEADERS} > + > +# Array of packages to be generated > +declare -A kern_pkgs > + > +main() { > + local target=${1} > + > + if [ ! -f ${S}/debian/isar/${target} ]; then > + echo "error: ${target} is not a supported build target!" >&2 > + return 1 > + fi > + > + # create do_<x> variables for each package to be generated > + for p in $(unset DEB_HOST_ARCH; dh_listpackages); do > + kern_pkgs["${p}"]="1" > + done > + > + # variables to be exported > + export ARCH > + > + # are we cross-compiling? > + BUILD_ARCH=$(dpkg-architecture -qDEB_BUILD_ARCH) > + if [ "${BUILD_ARCH}" = "${DISTRO_ARCH}" ]; then > + # no, make sure CROSS_COMPILE isn't set > + unset CROSS_COMPILE > + fi > + > + # call the actual target script > + do_${target} || return ${?} > +} > diff --git a/meta/recipes-kernel/linux/files/debian/isar/install.tmpl b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl > new file mode 100644 > index 0000000..806df5d > --- /dev/null > +++ b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl > @@ -0,0 +1,213 @@ > +#!/bin/bash > +# Copyright (c) Mentor Graphics, a Siemens business, 2019 > +# SPDX-License-Identifier: MIT > + > +# Load common stuff > +. ${S}/debian/isar/common || exit ${?} > + > +# Stop on error > +set -e > + > +do_install() { > + > + # check if our kernel was configured > + if [ ! -f "${O}/.config" ]; then > + echo "error: kernel not configured!" >&2 > + return 1 > + fi > + > + # load its configuration > + . ${O}/.config > + > + kimage="$(${MAKE} O=${O} -s --no-print-directory image_name)" > + krel="$(${MAKE} O=${O} -s --no-print-directory kernelrelease)" > + case "${ARCH}" in > + mips|powerpc) kimage_path="boot/vmlinux-${krel}" ;; > + um) kimage_path="usr/bin/vmlinux-${krel}" ;; > + *) kimage_path="boot/vmlinuz-${krel}" ;; > + esac > + > + print_settings > + > + # Trace what we do here > + set -x > + > + install_image > + install_hooks > + install_dtbs > + install_kmods > + install_headers > + > + # Stop tracing > + set +x > +} > + > +print_settings() { > + cat <<EOF > +Install settings: > +----------------- > +deb_dtb_dir=${deb_dtb_dir} > +deb_hdr_dir=${deb_hdr_dir} > +kimage=${kimage} > +kimage_path=${kimage_path} > + > +EOF > +} > + > +install_image() { > + install -m 755 -d ${deb_img_dir}/$(dirname ${kimage_path}) > + cp ${O}/${kimage} ${deb_img_dir}/${kimage_path} > + > + # Make sure arm64 kernels are decompressed > + if [ "${ARCH}" = "arm64" ]; then > + mv ${deb_img_dir}/${kimage_path} ${deb_img_dir}/${kimage_path}.gz > + gunzip -f ${deb_img_dir}/${kimage_path}.gz > + fi > + > + install_image_debug > +} > + > +install_image_debug() { > + # Different tools want the image in different locations > + # perf > + mkdir -p ${deb_dbg_dir}/usr/lib/debug/lib/modules/${krel}/ > + cp ${O}/vmlinux ${deb_dbg_dir}/usr/lib/debug/lib/modules/${krel}/ > + # systemtap > + mkdir -p ${deb_dbg_dir}/usr/lib/debug/boot/ > + ln -s ../lib/modules/$version/vmlinux ${deb_dbg_dir}/usr/lib/debug/boot/vmlinux-${krel} > + # kdump-tools > + ln -s lib/modules/${krel}/vmlinux ${deb_dbg_dir}/usr/lib/debug/vmlinux-${krel} > +} > + > +install_hooks() { > + install -m 755 -d ${deb_img_dir}/etc/kernel/install.d > + install -m 755 -d ${deb_img_dir}/etc/kernel/postinst.d > + install -m 755 -d ${deb_img_dir}/etc/kernel/postrm.d > + install -m 755 -d ${deb_img_dir}/etc/kernel/prerm.d > + > + initrd="No" > + [ -z "${CONFIG_BLK_DEV_INITRD}" ] || initrd="Yes" > + > + for script in postinst postrm preinst prerm; do > + sed -i -e "s,INITRD=[A-Za-z0-9]*,INITRD=${initrd},g" ${S}/debian/linux-image*.${script} > + sed -i -e "s,version=.*,version=${krel},g" ${S}/debian/linux-image*.${script} > + done > +} > + > +install_dtbs() { > + [ -n "${CONFIG_OF}" ] || return 0 > + ${MAKE} O=${O} INSTALL_DTBS_PATH=${deb_dtb_dir} dtbs_install > +} > + > +install_kmods() { > + [ -n "${CONFIG_MODULES}" ] || return 0 > + ${MAKE} O=${O} INSTALL_MOD_PATH=${deb_img_dir} modules_install > + touch ${deb_img_dir}/lib/modules/${krel}/.fresh-install > + rm -fv ${deb_img_dir}/lib/modules/${krel}/build > + rm -fv ${deb_img_dir}/lib/modules/${krel}/source > + install_kmods_debug > +} > + > +kmods_sign() { > + [ -n "${CONFIG_MODULE_SIG_ALL}" ] || return 0 > + ${MAKE} O=${O} INSTALL_MOD_PATH=${deb_img_dir} modules_sign > +} > + > +install_kmods_debug() { > + [ -n "${CONFIG_DEBUG_INFO}" ] || return 0 > + > + kmod_inst_dir=${deb_img_dir}/lib/modules > + kmod_debug_dir=${deb_dbg_dir}/usr/lib/debug > + > + # copy kernels modules to usr/lib/debug > + mkdir -p ${kmod_debug_dir} > + tar -C ${kmod_inst_dir}/ -cO --exclude='modules.*' . | tar -C ${kmod_debug_dir}/ -xf - > + # strip everything but debug sections for modules in usr/lib/debug > + find ${kmod_debug_dir} -name *.ko -exec ${CROSS_COMPILE}objcopy --only-keep-debug {} \; > + # and strip debug sections from modules in lib/modules > + find ${kmod_inst_dir} -name *.ko -exec ${CROSS_COMPILE}objcopy --strip-debug {} \; > + > + # re-sign stripped kernel modules > + kmods_sign > +} > + > +headers_check() { > + ${MAKE} O=${O} headers_check > +} > + > +libc_headers() { > + mkdir -p ${deb_libc_hdr_dir} > + ${MAKE} O=${O} headers_install INSTALL_HDR_PATH=${deb_libc_hdr_dir}/usr > + host_arch=$(dpkg-architecture -a${DISTRO_ARCH} -qDEB_HOST_MULTIARCH) > + mkdir ${deb_libc_hdr_dir}/usr/include/${host_arch} > + mv ${deb_libc_hdr_dir}/usr/include/asm ${deb_libc_hdr_dir}/usr/include/${host_arch}/ > +} > + > +# the kernel does not seem to provide a way to cross-compile its tools, we however > +# have qemu-user-static binaries in our buildchroot. Create a new build tree for the > +# scripts and override HOSTCC to force a cross-compile > +kernel_tools_build() { > + odir="${S}/build-scripts" > + mflags="HOSTCC=${CROSS_COMPILE}gcc O=${odir} NOSTDINC_FLAGS=" > + rm -rf ${odir} && mkdir -p ${odir} && cp ${S}/${KERNEL_BUILD_DIR}/.config ${odir} > + (unset LD_PRELOAD; cd ${odir} && make -C ${S} ${mflags} olddefconfig scripts) > + (cd ${odir}; find scripts -type f -executable -exec file {} \;|grep ELF|cut -d: -f1) >${src_hdr_files} > + tar -C ${odir} -cf - -T - <${src_hdr_files} | tar -C ${destdir} -xf - > +} > + > +kernel_tools() { > + # remove object files > + find ${destdir}/scripts -type f -name '*.o' |xargs rm -f > + > + # we're all done if we aren't cross-compiling > + [ -n "${CROSS_COMPILE}" ] || return 0 > + > + # remove ELF executables from the linux-headers package > + find ${destdir}/scripts -type f -exec file {} \;|grep ELF|cut -d: -f1|xargs rm -fv > + > + # cross-compile kernel tools to be shipped with linux-headers > + kernel_tools_build > +} > + > +kernel_headers() { > + destdir=${deb_kern_hdr_dir}/${KERNEL_HEADERS_DIR} > + src_hdr_files=$(mktemp) > + obj_hdr_files=$(mktemp) > + > + mkdir -p ${destdir} > + mkdir -p ${deb_kern_hdr_dir}/lib/modules/${krel} > + > + (cd ${S}; find . -name 'Makefile*' -o -name 'Kconfig*' -o -name '*.pl') >>${src_hdr_files} > + (cd ${S}; find arch/*/include include scripts -type f -o -type l) >>${src_hdr_files} > + (cd ${S}; find arch/${ARCH} -name module.lds -o -name Kbuild.platforms -o -name Platform) >>${src_hdr_files} > + (cd ${S}; find $(find arch/${ARCH} -name include -o -name scripts -type d) -type f) >>${src_hdr_files} > + > + (cd ${O}; find arch/${ARCH}/include Module.symvers include scripts -type f) >>${obj_hdr_files} > + if [ -n "${CONFIG_STACK_VALIDATION}" ]; then > + (cd ${O}; find tools/objtool -type f -executable) >>${obj_hdr_files} > + fi > + if [ -n "${CONFIG_GCC_PLUGINS}" ]; then > + (cd ${O}; find scripts/gcc-plugins -name *.so -o -name gcc-common.h) >>${obj_hdr_files} > + fi > + > + # deploy files that were matched above > + tar -C ${S} -cf - -T - <${src_hdr_files} | tar -C ${destdir} -xf - > + tar -C ${O} -cf - -T - <${obj_hdr_files} | tar -C ${destdir} -xf - > + > + # add the kernel config > + cp ${O}/${KCONF} ${destdir}/.config > + > + # handle kernel development tools > + kernel_tools > + > + # create symlinks > + ln -sf /${KERNEL_HEADERS_DIR} ${deb_kern_hdr_dir}/lib/modules/${krel}/build > +} > + > +install_headers() { > + headers_check > + [ -z ${kern_pkgs["linux-libc-headers"]} ] || libc_headers > + kernel_headers > +} > + > +main install ${*} > diff --git a/meta/recipes-kernel/linux/files/debian/isar/version.cfg.tmpl b/meta/recipes-kernel/linux/files/debian/isar/version.cfg.tmpl > new file mode 100644 > index 0000000..09fa364 > --- /dev/null > +++ b/meta/recipes-kernel/linux/files/debian/isar/version.cfg.tmpl > @@ -0,0 +1,2 @@ > +CONFIG_LOCALVERSION="-${KERNEL_LOCALVERSION}" > +# CONFIG_LOCALVERSION_AUTO is not set > diff --git a/meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl b/meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl > new file mode 100644 > index 0000000..b373801 > --- /dev/null > +++ b/meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl > @@ -0,0 +1,29 @@ > +#!/bin/sh > +# based on https://salsa.debian.org/kernel-team/linux/blob/master/debian/templates/image.postinst.in > + > +# Tell initramfs builder whether it's wanted > +export INITRD=Yes > + > +version=${PV}-${KERNEL_LOCALVERSION} > +image_path=/boot/${KERNEL_FILE}-${version} > + > +if [ "$1" != configure ]; then > + exit 0 > +fi > + > +depmod $version > + > +if [ -f /lib/modules/$version/.fresh-install ]; then > + change=install > +else > + change=upgrade > +fi > +linux-update-symlinks $change $version $image_path > +rm -f /lib/modules/$version/.fresh-install > + > +if [ -d /etc/kernel/postinst.d ]; then > + DEB_MAINT_PARAMS="$*" run-parts --report --exit-on-error --arg=$version \ > + --arg=$image_path /etc/kernel/postinst.d > +fi > + > +exit 0 > diff --git a/meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl b/meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl > new file mode 100644 > index 0000000..1b5dbbe > --- /dev/null > +++ b/meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl > @@ -0,0 +1,35 @@ > +#!/bin/sh -e > +# based on https://salsa.debian.org/kernel-team/linux/blob/master/debian/templates/image.postrm.in > + > +# Tell initramfs builder whether it's wanted > +export INITRD=Yes > + > +version=${PV}-${KERNEL_LOCALVERSION} > +image_path=/boot/${KERNEL_FILE}-${version} > + > +rm -f /lib/modules/$version/.fresh-install > + > +if [ "$1" != upgrade ] && command -v linux-update-symlinks >/dev/null; then > + linux-update-symlinks remove $version $image_path > +fi > + > +if [ -d /etc/kernel/postrm.d ]; then > + DEB_MAINT_PARAMS="$*" run-parts --report --exit-on-error --arg=$version \ > + --arg=$image_path /etc/kernel/postrm.d > +fi > + > +if [ "$1" = purge ]; then > + for extra_file in modules.dep modules.isapnpmap modules.pcimap \ > + modules.usbmap modules.parportmap \ > + modules.generic_string modules.ieee1394map \ > + modules.ieee1394map modules.pnpbiosmap \ > + modules.alias modules.ccwmap modules.inputmap \ > + modules.symbols modules.ofmap \ > + modules.seriomap modules.\*.bin \ > + modules.softdep modules.devname; do > + eval rm -f /lib/modules/$version/$extra_file > + done > + rmdir /lib/modules/$version || true > +fi > + > +exit 0 > diff --git a/meta/recipes-kernel/linux/files/debian/linux-image.preinst.tmpl b/meta/recipes-kernel/linux/files/debian/linux-image.preinst.tmpl > new file mode 100644 > index 0000000..4f24019 > --- /dev/null > +++ b/meta/recipes-kernel/linux/files/debian/linux-image.preinst.tmpl > @@ -0,0 +1,25 @@ > +#!/bin/sh -e > +# based on https://salsa.debian.org/kernel-team/linux/blob/master/debian/templates/image.preinst.in > + > +# Tell initramfs builder whether it's wanted > +export INITRD=Yes > + > +version=${PV}-${KERNEL_LOCALVERSION} > +image_path=/boot/${KERNEL_FILE}-${version} > + > +if [ "$1" = abort-upgrade ]; then > + exit 0 > +fi > + > +if [ "$1" = install ]; then > + # Create a flag file for postinst > + mkdir -p /lib/modules/$version > + touch /lib/modules/$version/.fresh-install > +fi > + > +if [ -d /etc/kernel/preinst.d ]; then > + DEB_MAINT_PARAMS="$*" run-parts --report --exit-on-error --arg=$version \ > + --arg=$image_path /etc/kernel/preinst.d > +fi > + > +exit 0 > diff --git a/meta/recipes-kernel/linux/files/debian/linux-image.prerm.tmpl b/meta/recipes-kernel/linux/files/debian/linux-image.prerm.tmpl > new file mode 100644 > index 0000000..cd697c0 > --- /dev/null > +++ b/meta/recipes-kernel/linux/files/debian/linux-image.prerm.tmpl > @@ -0,0 +1,21 @@ > +#!/bin/sh -e > +# based on https://salsa.debian.org/kernel-team/linux/blob/master/debian/templates/image.prerm.in > + > +# Tell initramfs builder whether it's wanted > +export INITRD=Yes > + > +version=${PV}-${KERNEL_LOCALVERSION} > +image_path=/boot/${KERNEL_FILE}-${version} > + > +if [ "$1" != remove ]; then > + exit 0 > +fi > + > +linux-check-removal $version > + > +if [ -d /etc/kernel/prerm.d ]; then > + DEB_MAINT_PARAMS="$*" run-parts --report --exit-on-error --arg=$version \ > + --arg=$image_path /etc/kernel/prerm.d > +fi > + > +exit 0 > diff --git a/meta/recipes-kernel/linux/files/debian/rules.tmpl b/meta/recipes-kernel/linux/files/debian/rules.tmpl > new file mode 100755 > index 0000000..c024768 > --- /dev/null > +++ b/meta/recipes-kernel/linux/files/debian/rules.tmpl > @@ -0,0 +1,39 @@ > +#!/usr/bin/make -f > + > +CROSS_COMPILE:=$(DEB_HOST_GNU_TYPE)- > + > +O:=$(CURDIR)/${KERNEL_BUILD_DIR} > +S:=$(CURDIR) > +deb_top_dir:=$(S)/debian > + > +# Dynamic variables to be passed to Isar build scripts > +isar_env=$(strip \ > + export CROSS_COMPILE='$(CROSS_COMPILE)' && \ > + export DEB_BUILD_PROFILES='$(DEB_BUILD_PROFILES)' && \ > + export DEB_HOST_GNU_TYPE='$(DEB_HOST_GNU_TYPE)' && \ > + export MAKE='$(MAKE)' && \ > + export O='${O}' \ > + export S='${S}' \ > +) > + > +%: > + dh $(@) > + > +.PHONY: override_dh_auto_clean > +override_dh_auto_clean: > + $(isar_env) && bash $(deb_top_dir)/isar/clean > + > +.PHONY: override_dh_auto_build > +override_dh_auto_build: > + $(isar_env) && bash $(deb_top_dir)/isar/build > + > +override_dh_auto_install: > + $(isar_env) && bash $(deb_top_dir)/isar/install > + > +.PHONY: override_dh_auto_test > +override_dh_auto_test: > + true > + > +PHONY: override_dh_strip > +override_dh_strip: > + dh_strip -Xvmlinux --no-automatic-dbgsym > diff --git a/meta/recipes-kernel/linux/linux-custom.inc b/meta/recipes-kernel/linux/linux-custom.inc > index 712f52b..bf1646a 100644 > --- a/meta/recipes-kernel/linux/linux-custom.inc > +++ b/meta/recipes-kernel/linux/linux-custom.inc > @@ -2,89 +2,183 @@ > # > # This software is a part of ISAR. > # Copyright (c) Siemens AG, 2018 > +# Copyright (c) Mentor Graphics, a Siemens business, 2019 > # > # SPDX-License-Identifier: MIT > > -FILESEXTRAPATHS_prepend := "${FILE_DIRNAME}/files:" > +# Settings that would typically be done from the custom kernel recipe > +# ------------------------------------------------------------------- > > +CHANGELOG_V = "${PV}+${PR}" > DESCRIPTION ?= "Custom kernel" > +MAINTAINER ?= "isar-users <isar-users@googlegroups.com>" > + > +KBUILD_DEPENDS ?= "build-essential:native, \ > + libelf-dev:native, \ > + libncurses-dev:native, \ > + libssl-dev:native, \ > + bc, \ > + bison, \ > + cpio, \ > + flex, \ > + git, \ > + kmod, \ > + libssl-dev," > + > +KERNEL_DEBIAN_DEPENDS ?= "initramfs-tools | linux-initramfs-tool, \ > + kmod, \ > + linux-base (>= 4.3~)," > + > +KERNEL_HEADERS_DEBIAN_DEPENDS ?= "libc6, \ > + libssl1.1," > + > +KERNEL_LIBC_DEV_DEPLOY ?= "0" > + > +# Settings that may be changed on a per distro, machine or layer basis > +# -------------------------------------------------------------------- > + > +KERNEL_LOCALVERSION ?= "-${KERNEL_NAME_PROVIDED}-${PR}" > +KERNEL_DTB_DIR ?= "usr/lib/linux-image-${PV}-${KERNEL_LOCALVERSION}" > +KERNEL_HEADERS_DIR ?= "usr/src/linux-headers-${PV}-${KERNEL_LOCALVERSION}" > > -KERNEL_NAME_PROVIDED ?= "${@ d.getVar('PN', True).partition('linux-')[2]}" > KERNEL_DEFCONFIG ?= "" > > +# Add our template meta-data to the sources > +FILESEXTRAPATHS_prepend := "${FILE_DIRNAME}/files:" > +SRC_URI += "file://debian" > + > +# Variables and files that make our templates > +# ------------------------------------------- > + > +TEMPLATE_FILES += " \ > + debian/control.tmpl \ > + debian/isar/build.tmpl \ > + debian/isar/clean.tmpl \ > + debian/isar/common.tmpl \ > + debian/isar/install.tmpl \ > + debian/isar/version.cfg.tmpl \ > + debian/linux-image.postinst.tmpl \ > + debian/linux-image.postrm.tmpl \ > + debian/linux-image.preinst.tmpl \ > + debian/linux-image.prerm.tmpl \ > + debian/rules.tmpl \ > +" > + > +TEMPLATE_VARS += " \ > + HOST_ARCH \ > + KBUILD_DEPENDS \ > + KERNEL_ARCH \ > + KERNEL_DEBIAN_DEPENDS \ > + KERNEL_BUILD_DIR \ > + KERNEL_DTB_DIR \ > + KERNEL_FILE \ > + KERNEL_HEADERS_DEBIAN_DEPENDS \ > + KERNEL_HEADERS_DIR \ > + KERNEL_LOCALVERSION \ > + KERNEL_NAME_PROVIDED \ > +" > + > +inherit dpkg > +inherit template > + > +# Derive name of the kernel packages from the name of this recipe > +KERNEL_NAME_PROVIDED ?= "${@ d.getVar('PN', True).partition('linux-')[2]}" > + > +# Make bitbake know we will be producing linux-image and linux-headers packages > python() { > kernel_name = d.getVar("KERNEL_NAME_PROVIDED", True) > d.setVar('PROVIDES', 'linux-image-' + kernel_name + ' ' + \ > 'linux-headers-' + kernel_name) > } > > -inherit dpkg-base > - > -SRC_URI += "file://build-kernel.sh" > - > -KBUILD_DEPENDS ?= " \ > - build-essential:native \ > - libssl-dev \ > - libelf-dev \ > - bc \ > - git \ > - kmod \ > - bison \ > - flex \ > - cpio \ > - libncurses-dev" > -KERNEL_DEBIAN_DEPENDS ?= "initramfs-tools | linux-initramfs-tool, kmod, linux-base (>= 4.3~)" > -KERNEL_HEADERS_DEBIAN_DEPENDS ?= "libc6, libssl1.1" > - > -KERNEL_LIBC_DEV_DEPLOY ?= "0" > - > -do_install_builddeps() { > - dpkg_do_mounts > - E="${@ isar_export_proxies(d)}" > - sudo -E chroot ${BUILDCHROOT_DIR} \ > - apt-get update \ > - -o Dir::Etc::SourceList="sources.list.d/isar-apt.list" \ > - -o Dir::Etc::SourceParts="-" \ > - -o APT::Get::List-Cleanup="0" > - sudo -E chroot ${BUILDCHROOT_DIR} \ > - apt-get install \ > - -y -o Debug::pkgProblemResolver=yes \ > - --no-install-recommends ${KBUILD_DEPENDS} > - dpkg_undo_mounts > +def get_kernel_arch(d): > + distro_arch = d.getVar("DISTRO_ARCH") > + if distro_arch == "amd64": > + kernel_arch = "x86" > + elif distro_arch == "arm64": > + kernel_arch = "arm64" > + elif distro_arch == "armhf": > + kernel_arch = "arm" > + elif distro_arch == "mipsel": > + kernel_arch = "mips" > + else: > + kernel_arch = "" > + return kernel_arch > + > +KERNEL_ARCH ??= "${@get_kernel_arch(d)}" > + > +def config_fragments(d): > + fragments = [] > + sources = d.getVar("SRC_URI").split() > + for s in sources: > + _, _, local, _, _, parm = bb.fetch.decodeurl(s) > + apply = parm.get("apply") > + if apply == "no": > + continue > + base, ext = os.path.splitext(os.path.basename(local)) > + if ext and ext in (".cfg"): > + fragments.append(local) > + return fragments > + > +do_prepare_build_prepend() { > + # copy meta-data over to source tree > + rm -rf ${S}/debian > + cp -r ${WORKDIR}/debian ${S}/ > + > + # remove templates from the source tree > + find ${S}/debian -name *.tmpl | xargs rm -f > + > + # rename install/remove hooks to match user-specified name for our linux-image package > + mv ${S}/debian/linux-image.postinst ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.postinst > + mv ${S}/debian/linux-image.postrm ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.postrm > + mv ${S}/debian/linux-image.preinst ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.preinst > + mv ${S}/debian/linux-image.prerm ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.prerm > + > + # produce a changelog for our kernel build > + deb_add_changelog > } > > -addtask install_builddeps after do_prepare_build before do_dpkg_build > -# apt and reprepro may not run in parallel, acquire the Isar lock > -do_install_builddeps[lockfiles] += "${REPO_ISAR_DIR}/isar.lock" > - > -addtask devshell after do_install_builddeps > +# build directory for our "full" kernel build > +KERNEL_BUILD_DIR = "build-full" > > -dpkg_runbuild() { > - chmod +x ${WORKDIR}/build-kernel.sh > - KERNEL_CONFIG_TARGET="${KERNEL_DEFCONFIG}" > +dpkg_configure_kernel() { > + config_target="${KERNEL_DEFCONFIG}" > + rm -rf ${S}/${KERNEL_BUILD_DIR} && mkdir -p ${S}/${KERNEL_BUILD_DIR} > if [ -n "${KERNEL_DEFCONFIG}" ]; then > if [ -e "${WORKDIR}/${KERNEL_DEFCONFIG}" ]; then > - cp ${WORKDIR}/${KERNEL_DEFCONFIG} ${S}/.config > - KERNEL_CONFIG_TARGET="olddefconfig" > + cp ${WORKDIR}/${KERNEL_DEFCONFIG} ${S}/${KERNEL_BUILD_DIR}/.config > + config_target="olddefconfig" > fi > else > - KERNEL_CONFIG_TARGET="defconfig" > + config_target="defconfig" > fi > > - E="${@ isar_export_proxies(d)}" > - > - export PV=${PV} > - export KERNEL_NAME=${KERNEL_NAME_PROVIDED} > - > - export KBUILD_DEPENDS="${KBUILD_DEPENDS}" > - export KERNEL_CONFIG_TARGET > - export KERNEL_DEBIAN_DEPENDS="${KERNEL_DEBIAN_DEPENDS}" > - export KERNEL_FILE="${KERNEL_FILE}" > - export KERNEL_HEADERS_DEBIAN_DEPENDS="${KERNEL_HEADERS_DEBIAN_DEPENDS}" > + # copy config fragments over to the kernel tree > + src_frags="${@ " ".join(config_fragments(d)) }" > + out_frags="${@ " ".join(map(lambda frag: 'debian/fragments/' + frag, config_fragments(d))) }" > + rm -rf ${S}/debian/fragments > + if [ -n "${src_frags}" ]; then > + mkdir -p ${S}/debian/fragments > + (cd ${WORKDIR} && cp ${src_frags} ${S}/debian/fragments/) > + fi > > - sudo -E chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} ${PP}/build-kernel.sh ${PP}/${PPS} ${DISTRO_ARCH} > + sudo -E chroot --userspec=$(id -u):$(id -g) ${BUILDCHROOT_DIR} sh -c " \ > + export ARCH=${KERNEL_ARCH} && \ > + cd ${PP}/${PPS} && \ > + make O=${KERNEL_BUILD_DIR} ${config_target} && \ > + ./scripts/kconfig/merge_config.sh \ > + -O ${KERNEL_BUILD_DIR}/ \ > + ${KERNEL_BUILD_DIR}/.config \ > + ${out_frags} \ > + debian/isar/version.cfg \ > + " > +} > > +dpkg_runbuild_prepend() { > + profiles="${BASE_DISTRO}" > if [ "${KERNEL_LIBC_DEV_DEPLOY}" != "1" ]; then > - rm -f ${WORKDIR}/linux-libc-dev_${PV}*.deb > + profiles="${profiles} nolibcdev" > fi > + export DEB_BUILD_PROFILES="${profiles}" > + dpkg_configure_kernel > } > -- Quirin Gylstorff Siemens AG Corporate Technology Research in Digitalization and Automation Smart Embedded Systems CT RDA IOT SES-DE Otto-Hahn-Ring 6 81739 Muenchen, Germany Mobile: +49 173 3746683 mailto:quirin.gylstorff@siemens.com www.siemens.com/ingenuityforlife Siemens Aktiengesellschaft: Chairman of the Supervisory Board: Jim Hagemann Snabe; Managing Board: Joe Kaeser, Chairman, President and Chief Executive Officer; Roland Busch, Lisa Davis, Klaus Helmrich, Janina Kugel, Cedrik Neike, Michael Sen, Ralf P. Thomas; Registered offices: Berlin and Munich, Germany; Commercial registries: Berlin Charlottenburg, HRB 12300, Munich, HRB 6684; WEEE-Reg.-No. DE 23691322 Important notice: This e-mail and any attachment thereof contain corporate proprietary information. If you have received it by mistake, please notify us immediately by reply e-mail and delete this e-mail and its attachments from your system. Thank you. ^ permalink raw reply [flat|nested] 83+ messages in thread
* Re: [PATCH L-C v4 0/5] linux-custom recipe rework 2019-11-25 16:04 ` [PATCH L-C v4 0/5] linux-custom recipe rework Cedric Hombourger ` (4 preceding siblings ...) 2019-11-25 16:06 ` [PATCH L-C v4 5/5] linux-custom: rewrite to no longer depend on the kernel's builddeb Cedric Hombourger @ 2019-11-25 16:16 ` Jan Kiszka 2019-11-27 15:49 ` Gylstorff Quirin 2019-12-05 17:01 ` Baurzhan Ismagulov 7 siblings, 0 replies; 83+ messages in thread From: Jan Kiszka @ 2019-11-25 16:16 UTC (permalink / raw) To: Cedric Hombourger, isar-users On 25.11.19 17:04, Cedric Hombourger wrote: > The Linux kernel is packaged a bit differently between Debian variants > and in particular for the perf package(s) in Debian vs Ubuntu. Since > there is no way the kernel build scripts would ever get distro specific > support for building more than the kernel and headers, we need our own > build/packaging scripts. > > Version 4 of this changeset was tested against ci_build.sh and some images were > booted on hardware available (including the de0-nano-soc). No regressions were > detected (to date). > > This version is still relying on qemu-user-static being in the buildchroot-host > I am unfortunetely getting little time to work on removing this dependency (for > the avoidance of doubt, I understand the views of the maintainers). The purpose > of this submission is to share progress. Thanks for the update, much appreciated! Will put this into testing ASAP. But I will also look into splitting the queue so that patch 4 (with the code that uses it) can come later. Again, I see no urgent need for linux-headers on the target so far, thus we can wait for a proper solution. > > Changes compared to v3: > Ship device trees where they can be found (checked on de0-nano-soc) > Initialize LOCAL_VERSION from the recipe PV & PR You also dropped the kernel config fragment patches (4 & 5) of v3. Intentionally? Jan > > Changes compared to v2: > Rework custom_kernel.md > Compare PV+/-LOCAL_VERSION and kernelrelease string > > Changes compared to v1: > Add missing reference to KERNEL_DEFCONFIG in requirements list > Amend postinst/postrm scripts according to CONFIG_BLK_DEV_INITRD > Remove (unused) pre/post host hooks from debian/isar scripts > Add preinst/prerm scripts shipped by the kernel > Align {pre,post}{inst,rm} scripts with upstream > Add ${BASE_DISTRO} to Build-Profiles > Add nolibcdev to Build-Profiles if KERNEL_LIBC_DEV_DEPLOY != 1 > Only build the libc-dev packages if it is needed > > > Cedric Hombourger (5): > recipes-kernel/linux: make KERNEL_DEFCONFIG support in-tree defconfigs > linux-mainline: fix stripping of .0 from the kernel version > linux-mainline: update from 4.19.0 to 4.19.80 > buildchroot-host: install qemu-static to support hybrid cross-compiles > linux-custom: rewrite to no longer depend on the kernel's builddeb > > doc/custom_kernel.md | 80 +++++++ > ...ne_4.19.0.bb => linux-mainline_4.19.80.bb} | 4 +- > .../buildchroot/buildchroot-host.bb | 6 + > .../linux/files/build-kernel.sh | 129 ----------- > meta/recipes-kernel/linux/files/debian/compat | 1 + > .../linux/files/debian/control.tmpl | 37 +++ > .../linux/files/debian/isar/build.tmpl | 45 ++++ > .../linux/files/debian/isar/clean.tmpl | 21 ++ > .../linux/files/debian/isar/common.tmpl | 50 ++++ > .../linux/files/debian/isar/install.tmpl | 213 ++++++++++++++++++ > .../linux/files/debian/isar/version.cfg.tmpl | 2 + > .../files/debian/linux-image.postinst.tmpl | 29 +++ > .../files/debian/linux-image.postrm.tmpl | 35 +++ > .../files/debian/linux-image.preinst.tmpl | 25 ++ > .../linux/files/debian/linux-image.prerm.tmpl | 21 ++ > .../linux/files/debian/rules.tmpl | 39 ++++ > meta/recipes-kernel/linux/linux-custom.inc | 209 ++++++++++++----- > 17 files changed, 761 insertions(+), 185 deletions(-) > create mode 100644 doc/custom_kernel.md > rename meta-isar/recipes-kernel/linux/{linux-mainline_4.19.0.bb => linux-mainline_4.19.80.bb} (73%) > delete mode 100644 meta/recipes-kernel/linux/files/build-kernel.sh > create mode 100644 meta/recipes-kernel/linux/files/debian/compat > create mode 100644 meta/recipes-kernel/linux/files/debian/control.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/isar/build.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/isar/clean.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/isar/common.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/isar/install.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/isar/version.cfg.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.preinst.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.prerm.tmpl > create mode 100755 meta/recipes-kernel/linux/files/debian/rules.tmpl > -- Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 83+ messages in thread
* Re: [PATCH L-C v4 0/5] linux-custom recipe rework 2019-11-25 16:04 ` [PATCH L-C v4 0/5] linux-custom recipe rework Cedric Hombourger ` (5 preceding siblings ...) 2019-11-25 16:16 ` [PATCH L-C v4 0/5] linux-custom recipe rework Jan Kiszka @ 2019-11-27 15:49 ` Gylstorff Quirin 2019-12-05 17:01 ` Baurzhan Ismagulov 7 siblings, 0 replies; 83+ messages in thread From: Gylstorff Quirin @ 2019-11-27 15:49 UTC (permalink / raw) To: isar-users, Cedric Hombourger On 11/25/19 5:04 PM, Cedric Hombourger wrote: > The Linux kernel is packaged a bit differently between Debian variants > and in particular for the perf package(s) in Debian vs Ubuntu. Since > there is no way the kernel build scripts would ever get distro specific > support for building more than the kernel and headers, we need our own > build/packaging scripts. > > Version 4 of this changeset was tested against ci_build.sh and some images were > booted on hardware available (including the de0-nano-soc). No regressions were > detected (to date). > > This version is still relying on qemu-user-static being in the buildchroot-host > I am unfortunetely getting little time to work on removing this dependency (for > the avoidance of doubt, I understand the views of the maintainers). The purpose > of this submission is to share progress. > > Changes compared to v3: > Ship device trees where they can be found (checked on de0-nano-soc) > Initialize LOCAL_VERSION from the recipe PV & PR > > Changes compared to v2: > Rework custom_kernel.md > Compare PV+/-LOCAL_VERSION and kernelrelease string > > Changes compared to v1: > Add missing reference to KERNEL_DEFCONFIG in requirements list > Amend postinst/postrm scripts according to CONFIG_BLK_DEV_INITRD > Remove (unused) pre/post host hooks from debian/isar scripts > Add preinst/prerm scripts shipped by the kernel > Align {pre,post}{inst,rm} scripts with upstream > Add ${BASE_DISTRO} to Build-Profiles > Add nolibcdev to Build-Profiles if KERNEL_LIBC_DEV_DEPLOY != 1 > Only build the libc-dev packages if it is needed > > > Cedric Hombourger (5): > recipes-kernel/linux: make KERNEL_DEFCONFIG support in-tree defconfigs > linux-mainline: fix stripping of .0 from the kernel version > linux-mainline: update from 4.19.0 to 4.19.80 > buildchroot-host: install qemu-static to support hybrid cross-compiles > linux-custom: rewrite to no longer depend on the kernel's builddeb > > doc/custom_kernel.md | 80 +++++++ > ...ne_4.19.0.bb => linux-mainline_4.19.80.bb} | 4 +- > .../buildchroot/buildchroot-host.bb | 6 + > .../linux/files/build-kernel.sh | 129 ----------- > meta/recipes-kernel/linux/files/debian/compat | 1 + > .../linux/files/debian/control.tmpl | 37 +++ > .../linux/files/debian/isar/build.tmpl | 45 ++++ > .../linux/files/debian/isar/clean.tmpl | 21 ++ > .../linux/files/debian/isar/common.tmpl | 50 ++++ > .../linux/files/debian/isar/install.tmpl | 213 ++++++++++++++++++ > .../linux/files/debian/isar/version.cfg.tmpl | 2 + > .../files/debian/linux-image.postinst.tmpl | 29 +++ > .../files/debian/linux-image.postrm.tmpl | 35 +++ > .../files/debian/linux-image.preinst.tmpl | 25 ++ > .../linux/files/debian/linux-image.prerm.tmpl | 21 ++ > .../linux/files/debian/rules.tmpl | 39 ++++ > meta/recipes-kernel/linux/linux-custom.inc | 209 ++++++++++++----- > 17 files changed, 761 insertions(+), 185 deletions(-) > create mode 100644 doc/custom_kernel.md > rename meta-isar/recipes-kernel/linux/{linux-mainline_4.19.0.bb => linux-mainline_4.19.80.bb} (73%) > delete mode 100644 meta/recipes-kernel/linux/files/build-kernel.sh > create mode 100644 meta/recipes-kernel/linux/files/debian/compat > create mode 100644 meta/recipes-kernel/linux/files/debian/control.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/isar/build.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/isar/clean.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/isar/common.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/isar/install.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/isar/version.cfg.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.preinst.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.prerm.tmpl > create mode 100755 meta/recipes-kernel/linux/files/debian/rules.tmpl > I tested this build with isar-cip-core[1] and the iwg20m build fails as it uses a 4.4.X kernel. ARM builds with Kernel version 4.4 don't copy the zImage to the build directory(build-full/.). The image stays in arch/arm/boot/. [1]: https://gitlab.com/cip-project/cip-core/isar-cip-core Kind regards Quirin ^ permalink raw reply [flat|nested] 83+ messages in thread
* Re: [PATCH L-C v4 0/5] linux-custom recipe rework 2019-11-25 16:04 ` [PATCH L-C v4 0/5] linux-custom recipe rework Cedric Hombourger ` (6 preceding siblings ...) 2019-11-27 15:49 ` Gylstorff Quirin @ 2019-12-05 17:01 ` Baurzhan Ismagulov 2019-12-09 7:29 ` Jan Kiszka 7 siblings, 1 reply; 83+ messages in thread From: Baurzhan Ismagulov @ 2019-12-05 17:01 UTC (permalink / raw) To: isar-users Hello Cedric, On Mon, Nov 25, 2019 at 05:04:49PM +0100, Cedric Hombourger wrote: > The Linux kernel is packaged a bit differently between Debian variants > and in particular for the perf package(s) in Debian vs Ubuntu. Since > there is no way the kernel build scripts would ever get distro specific > support for building more than the kernel and headers, we need our own > build/packaging scripts. > > Version 4 of this changeset was tested against ci_build.sh and some images were > booted on hardware available (including the de0-nano-soc). No regressions were > detected (to date). > > This version is still relying on qemu-user-static being in the buildchroot-host > I am unfortunetely getting little time to work on removing this dependency (for > the avoidance of doubt, I understand the views of the maintainers). The purpose > of this submission is to share progress. In general, looks good to me. A couple of thoughts: * Could you provide simple test cases for the "apply" parameter and dts in this or some later series? * Echoing Jan: Why did you drop config fragment patches? * The current usage of copying qemu-*-static in the mainline is racy (a recipe may delete it before chroot needs it). It should be centralized in the future. With kind regards, Baurzhan. ^ permalink raw reply [flat|nested] 83+ messages in thread
* Re: [PATCH L-C v4 0/5] linux-custom recipe rework 2019-12-05 17:01 ` Baurzhan Ismagulov @ 2019-12-09 7:29 ` Jan Kiszka 2019-12-09 7:40 ` Cedric Hombourger 0 siblings, 1 reply; 83+ messages in thread From: Jan Kiszka @ 2019-12-09 7:29 UTC (permalink / raw) To: isar-users, Cedric Hombourger On 05.12.19 18:01, Baurzhan Ismagulov wrote: > Hello Cedric, > > On Mon, Nov 25, 2019 at 05:04:49PM +0100, Cedric Hombourger wrote: >> The Linux kernel is packaged a bit differently between Debian variants >> and in particular for the perf package(s) in Debian vs Ubuntu. Since >> there is no way the kernel build scripts would ever get distro specific >> support for building more than the kernel and headers, we need our own >> build/packaging scripts. >> >> Version 4 of this changeset was tested against ci_build.sh and some images were >> booted on hardware available (including the de0-nano-soc). No regressions were >> detected (to date). >> >> This version is still relying on qemu-user-static being in the buildchroot-host >> I am unfortunetely getting little time to work on removing this dependency (for >> the avoidance of doubt, I understand the views of the maintainers). The purpose >> of this submission is to share progress. > > In general, looks good to me. A couple of thoughts: > > * Could you provide simple test cases for the "apply" parameter and dts in this > or some later series? > > * Echoing Jan: Why did you drop config fragment patches? Looking at this again, I suspect you just wanted to avoid adding that features first for the old recipe and then implementing it again for the new one. I can see it in the new recipe, but the example got lost. > > * The current usage of copying qemu-*-static in the mainline is racy (a recipe > may delete it before chroot needs it). It should be centralized in the > future. Another reason to skip that for now. Cedric, I can take over if you lack the time, just let me know. Jan -- Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 83+ messages in thread
* Re: [PATCH L-C v4 0/5] linux-custom recipe rework 2019-12-09 7:29 ` Jan Kiszka @ 2019-12-09 7:40 ` Cedric Hombourger 0 siblings, 0 replies; 83+ messages in thread From: Cedric Hombourger @ 2019-12-09 7:40 UTC (permalink / raw) To: Jan Kiszka, isar-users On 12/9/2019 8:29 AM, Jan Kiszka wrote: > On 05.12.19 18:01, Baurzhan Ismagulov wrote: >> Hello Cedric, >> >> On Mon, Nov 25, 2019 at 05:04:49PM +0100, Cedric Hombourger wrote: >>> The Linux kernel is packaged a bit differently between Debian variants >>> and in particular for the perf package(s) in Debian vs Ubuntu. Since >>> there is no way the kernel build scripts would ever get distro specific >>> support for building more than the kernel and headers, we need our own >>> build/packaging scripts. >>> >>> Version 4 of this changeset was tested against ci_build.sh and some images were >>> booted on hardware available (including the de0-nano-soc). No regressions were >>> detected (to date). >>> >>> This version is still relying on qemu-user-static being in the buildchroot-host >>> I am unfortunetely getting little time to work on removing this dependency (for >>> the avoidance of doubt, I understand the views of the maintainers). The purpose >>> of this submission is to share progress. >> In general, looks good to me. A couple of thoughts: >> >> * Could you provide simple test cases for the "apply" parameter and dts in this >> or some later series? >> >> * Echoing Jan: Why did you drop config fragment patches? > Looking at this again, I suspect you just wanted to avoid adding that > features first for the old recipe and then implementing it again for the > new one. I can see it in the new recipe, but the example got lost. > >> * The current usage of copying qemu-*-static in the mainline is racy (a recipe >> may delete it before chroot needs it). It should be centralized in the >> future. > Another reason to skip that for now. > > Cedric, I can take over if you lack the time, just let me know. Hi Jan, I could indeed use your help if you have some cycles this week. I was on the road last week and have a bit of catch-up to do on various fronts > > Jan > ^ permalink raw reply [flat|nested] 83+ messages in thread
* Re: [PATCH L-C v2 0/7] linux-custom recipe rework 2019-11-07 9:23 ` [PATCH L-C v2 " Cedric Hombourger ` (6 preceding siblings ...) 2019-11-07 9:23 ` [PATCH L-C v2 7/7] linux-custom: rewrite to no longer depend on the kernel's builddeb Cedric Hombourger @ 2019-11-07 11:39 ` Jan Kiszka 2019-11-07 11:47 ` chombourger 2019-11-07 11:48 ` Jan Kiszka 2019-11-08 7:07 ` [PATCH L-C v3 " Cedric Hombourger 8 siblings, 2 replies; 83+ messages in thread From: Jan Kiszka @ 2019-11-07 11:39 UTC (permalink / raw) To: Cedric Hombourger, isar-users On 07.11.19 10:23, Cedric Hombourger wrote: > The Linux kernel is packaged a bit differently between Debian variants > and in particular for the perf package(s) in Debian vs Ubuntu. Since > there is no way the kernel build scripts would ever get distro specific > support for building more than the kernel and headers, we need our own > build/packaging scripts. The approach is documented in the custom_kernel.md > file included in this patch series. > > Version 2 of this changeset was tested against ci_build.sh and some images were > booted on hardware available here. No regressions were detected (to date). This > was also tested on the creator-ci40-marduk machine (subject of another patch > series). > > Changes compared to v1: > Add missing reference to KERNEL_DEFCONFIG in requirements list > Amend postinst/postrm scripts according to CONFIG_BLK_DEV_INITRD > Remove (unused) pre/post host hooks from debian/isar scripts > Add preinst/prerm scripts shipped by the kernel > Align {pre,post}{inst,rm} scripts with upstream > Add ${BASE_DISTRO} to Build-Profiles > Add nolibcdev to Build-Profiles if KERNEL_LIBC_DEV_DEPLOY != 1 > Only build the libc-dev packages if it is needed > > Cedric Hombourger (7): > recipes-kernel/linux: make KERNEL_DEFCONFIG support in-tree defconfigs > linux-mainline: fix stripping of .0 from the kernel version > linux-mainline: update from 4.19.0 to 4.19.80 > linux-custom: add support for kernel config fragments > linux-mainline: disable support for HFS to demonstrate use of config fragments > buildchroot-host: install qemu-static to support hybrid cross-compiles > linux-custom: rewrite to no longer depend on the kernel's builddeb > > doc/custom_kernel.md | 181 ++++++++++++++++ > .../recipes-kernel/linux/files/no-fs-hfs.cfg | 2 + > ...ne_4.19.0.bb => linux-mainline_4.19.80.bb} | 8 +- > .../buildchroot/buildchroot-host.bb | 6 + > .../linux/files/build-kernel.sh | 129 ------------ > meta/recipes-kernel/linux/files/debian/compat | 1 + > .../linux/files/debian/control.tmpl | 36 ++++ > meta/recipes-kernel/linux/files/debian/files | 5 + > .../linux/files/debian/isar/build.tmpl | 36 ++++ > .../linux/files/debian/isar/clean.tmpl | 20 ++ > .../linux/files/debian/isar/common.tmpl | 60 ++++++ > .../linux/files/debian/isar/install.tmpl | 197 ++++++++++++++++++ > .../files/debian/linux-image.postinst.tmpl | 22 ++ > .../files/debian/linux-image.postrm.tmpl | 18 ++ > .../linux/files/debian/rules.tmpl | 39 ++++ > meta/recipes-kernel/linux/linux-custom.inc | 196 ++++++++++++----- > 16 files changed, 772 insertions(+), 184 deletions(-) > create mode 100644 doc/custom_kernel.md > create mode 100644 meta-isar/recipes-kernel/linux/files/no-fs-hfs.cfg > rename meta-isar/recipes-kernel/linux/{linux-mainline_4.19.0.bb => linux-mainline_4.19.80.bb} (67%) > delete mode 100644 meta/recipes-kernel/linux/files/build-kernel.sh > create mode 100644 meta/recipes-kernel/linux/files/debian/compat > create mode 100644 meta/recipes-kernel/linux/files/debian/control.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/files > create mode 100644 meta/recipes-kernel/linux/files/debian/isar/build.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/isar/clean.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/isar/common.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/isar/install.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl > create mode 100755 meta/recipes-kernel/linux/files/debian/rules.tmpl > What's the baseline of this series? It's not next. Jan -- Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 83+ messages in thread
* Re: [PATCH L-C v2 0/7] linux-custom recipe rework 2019-11-07 11:39 ` [PATCH L-C v2 0/7] " Jan Kiszka @ 2019-11-07 11:47 ` chombourger 2019-11-07 11:48 ` Jan Kiszka 1 sibling, 0 replies; 83+ messages in thread From: chombourger @ 2019-11-07 11:47 UTC (permalink / raw) To: isar-users [-- Attachment #1.1: Type: text/plain, Size: 4361 bytes --] On Thursday, November 7, 2019 at 12:39:29 PM UTC+1, Jan Kiszka wrote: > > On 07.11.19 10:23, Cedric Hombourger wrote: > > The Linux kernel is packaged a bit differently between Debian variants > > and in particular for the perf package(s) in Debian vs Ubuntu. Since > > there is no way the kernel build scripts would ever get distro specific > > support for building more than the kernel and headers, we need our own > > build/packaging scripts. The approach is documented in the > custom_kernel.md > > file included in this patch series. > > > > Version 2 of this changeset was tested against ci_build.sh and some > images were > > booted on hardware available here. No regressions were detected (to > date). This > > was also tested on the creator-ci40-marduk machine (subject of another > patch > > series). > > > > Changes compared to v1: > > Add missing reference to KERNEL_DEFCONFIG in requirements list > > Amend postinst/postrm scripts according to CONFIG_BLK_DEV_INITRD > > Remove (unused) pre/post host hooks from debian/isar scripts > > Add preinst/prerm scripts shipped by the kernel > > Align {pre,post}{inst,rm} scripts with upstream > > Add ${BASE_DISTRO} to Build-Profiles > > Add nolibcdev to Build-Profiles if KERNEL_LIBC_DEV_DEPLOY != 1 > > Only build the libc-dev packages if it is needed > > > > Cedric Hombourger (7): > > recipes-kernel/linux: make KERNEL_DEFCONFIG support in-tree defconfigs > > linux-mainline: fix stripping of .0 from the kernel version > > linux-mainline: update from 4.19.0 to 4.19.80 > > linux-custom: add support for kernel config fragments > > linux-mainline: disable support for HFS to demonstrate use of config > fragments > > buildchroot-host: install qemu-static to support hybrid cross-compiles > > linux-custom: rewrite to no longer depend on the kernel's builddeb > > > > doc/custom_kernel.md | 181 ++++++++++++++++ > > .../recipes-kernel/linux/files/no-fs-hfs.cfg | 2 + > > ...ne_4.19.0.bb => linux-mainline_4.19.80.bb} | 8 +- > > .../buildchroot/buildchroot-host.bb | 6 + > > .../linux/files/build-kernel.sh | 129 ------------ > > meta/recipes-kernel/linux/files/debian/compat | 1 + > > .../linux/files/debian/control.tmpl | 36 ++++ > > meta/recipes-kernel/linux/files/debian/files | 5 + > > .../linux/files/debian/isar/build.tmpl | 36 ++++ > > .../linux/files/debian/isar/clean.tmpl | 20 ++ > > .../linux/files/debian/isar/common.tmpl | 60 ++++++ > > .../linux/files/debian/isar/install.tmpl | 197 ++++++++++++++++++ > > .../files/debian/linux-image.postinst.tmpl | 22 ++ > > .../files/debian/linux-image.postrm.tmpl | 18 ++ > > .../linux/files/debian/rules.tmpl | 39 ++++ > > meta/recipes-kernel/linux/linux-custom.inc | 196 ++++++++++++----- > > 16 files changed, 772 insertions(+), 184 deletions(-) > > create mode 100644 doc/custom_kernel.md > > create mode 100644 meta-isar/recipes-kernel/linux/files/no-fs-hfs.cfg > > rename meta-isar/recipes-kernel/linux/{linux-mainline_4.19.0.bb => > linux-mainline_4.19.80.bb} (67%) > > delete mode 100644 meta/recipes-kernel/linux/files/build-kernel.sh > > create mode 100644 meta/recipes-kernel/linux/files/debian/compat > > create mode 100644 meta/recipes-kernel/linux/files/debian/control.tmpl > > create mode 100644 meta/recipes-kernel/linux/files/debian/files > > create mode 100644 > meta/recipes-kernel/linux/files/debian/isar/build.tmpl > > create mode 100644 > meta/recipes-kernel/linux/files/debian/isar/clean.tmpl > > create mode 100644 > meta/recipes-kernel/linux/files/debian/isar/common.tmpl > > create mode 100644 > meta/recipes-kernel/linux/files/debian/isar/install.tmpl > > create mode 100644 > meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl > > create mode 100644 > meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl > > create mode 100755 meta/recipes-kernel/linux/files/debian/rules.tmpl > > > > What's the baseline of this series? It's not next. > next + https://groups.google.com/forum/#!topic/isar-users/mz1g6elBs74 > > Jan > > -- > Siemens AG, Corporate Technology, CT RDA IOT SES-DE > Corporate Competence Center Embedded Linux > [-- Attachment #1.2: Type: text/html, Size: 7515 bytes --] ^ permalink raw reply [flat|nested] 83+ messages in thread
* Re: [PATCH L-C v2 0/7] linux-custom recipe rework 2019-11-07 11:39 ` [PATCH L-C v2 0/7] " Jan Kiszka 2019-11-07 11:47 ` chombourger @ 2019-11-07 11:48 ` Jan Kiszka 1 sibling, 0 replies; 83+ messages in thread From: Jan Kiszka @ 2019-11-07 11:48 UTC (permalink / raw) To: Cedric Hombourger, isar-users On 07.11.19 12:39, [ext] Jan Kiszka wrote: > On 07.11.19 10:23, Cedric Hombourger wrote: >> The Linux kernel is packaged a bit differently between Debian variants >> and in particular for the perf package(s) in Debian vs Ubuntu. Since >> there is no way the kernel build scripts would ever get distro specific >> support for building more than the kernel and headers, we need our own >> build/packaging scripts. The approach is documented in the custom_kernel.md >> file included in this patch series. >> >> Version 2 of this changeset was tested against ci_build.sh and some images were >> booted on hardware available here. No regressions were detected (to date). This >> was also tested on the creator-ci40-marduk machine (subject of another patch >> series). >> >> Changes compared to v1: >> Add missing reference to KERNEL_DEFCONFIG in requirements list >> Amend postinst/postrm scripts according to CONFIG_BLK_DEV_INITRD >> Remove (unused) pre/post host hooks from debian/isar scripts >> Add preinst/prerm scripts shipped by the kernel >> Align {pre,post}{inst,rm} scripts with upstream >> Add ${BASE_DISTRO} to Build-Profiles >> Add nolibcdev to Build-Profiles if KERNEL_LIBC_DEV_DEPLOY != 1 >> Only build the libc-dev packages if it is needed >> >> Cedric Hombourger (7): >> recipes-kernel/linux: make KERNEL_DEFCONFIG support in-tree defconfigs >> linux-mainline: fix stripping of .0 from the kernel version >> linux-mainline: update from 4.19.0 to 4.19.80 >> linux-custom: add support for kernel config fragments >> linux-mainline: disable support for HFS to demonstrate use of config fragments >> buildchroot-host: install qemu-static to support hybrid cross-compiles >> linux-custom: rewrite to no longer depend on the kernel's builddeb >> >> doc/custom_kernel.md | 181 ++++++++++++++++ >> .../recipes-kernel/linux/files/no-fs-hfs.cfg | 2 + >> ...ne_4.19.0.bb => linux-mainline_4.19.80.bb} | 8 +- >> .../buildchroot/buildchroot-host.bb | 6 + >> .../linux/files/build-kernel.sh | 129 ------------ >> meta/recipes-kernel/linux/files/debian/compat | 1 + >> .../linux/files/debian/control.tmpl | 36 ++++ >> meta/recipes-kernel/linux/files/debian/files | 5 + >> .../linux/files/debian/isar/build.tmpl | 36 ++++ >> .../linux/files/debian/isar/clean.tmpl | 20 ++ >> .../linux/files/debian/isar/common.tmpl | 60 ++++++ >> .../linux/files/debian/isar/install.tmpl | 197 ++++++++++++++++++ >> .../files/debian/linux-image.postinst.tmpl | 22 ++ >> .../files/debian/linux-image.postrm.tmpl | 18 ++ >> .../linux/files/debian/rules.tmpl | 39 ++++ >> meta/recipes-kernel/linux/linux-custom.inc | 196 ++++++++++++----- >> 16 files changed, 772 insertions(+), 184 deletions(-) >> create mode 100644 doc/custom_kernel.md >> create mode 100644 meta-isar/recipes-kernel/linux/files/no-fs-hfs.cfg >> rename meta-isar/recipes-kernel/linux/{linux-mainline_4.19.0.bb => linux-mainline_4.19.80.bb} (67%) >> delete mode 100644 meta/recipes-kernel/linux/files/build-kernel.sh >> create mode 100644 meta/recipes-kernel/linux/files/debian/compat >> create mode 100644 meta/recipes-kernel/linux/files/debian/control.tmpl >> create mode 100644 meta/recipes-kernel/linux/files/debian/files >> create mode 100644 meta/recipes-kernel/linux/files/debian/isar/build.tmpl >> create mode 100644 meta/recipes-kernel/linux/files/debian/isar/clean.tmpl >> create mode 100644 meta/recipes-kernel/linux/files/debian/isar/common.tmpl >> create mode 100644 meta/recipes-kernel/linux/files/debian/isar/install.tmpl >> create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl >> create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl >> create mode 100755 meta/recipes-kernel/linux/files/debian/rules.tmpl >> > > What's the baseline of this series? It's not next. > Found it, it's your "misc" series. Please point that out in the future. Jan -- Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH L-C v3 0/7] linux-custom recipe rework 2019-11-07 9:23 ` [PATCH L-C v2 " Cedric Hombourger ` (7 preceding siblings ...) 2019-11-07 11:39 ` [PATCH L-C v2 0/7] " Jan Kiszka @ 2019-11-08 7:07 ` Cedric Hombourger 2019-11-08 7:07 ` [PATCH L-C v3 1/7] recipes-kernel/linux: make KERNEL_DEFCONFIG support in-tree defconfigs Cedric Hombourger ` (7 more replies) 8 siblings, 8 replies; 83+ messages in thread From: Cedric Hombourger @ 2019-11-08 7:07 UTC (permalink / raw) To: isar-users; +Cc: Cedric Hombourger The Linux kernel is packaged a bit differently between Debian variants and in particular for the perf package(s) in Debian vs Ubuntu. Since there is no way the kernel build scripts would ever get distro specific support for building more than the kernel and headers, we need our own build/packaging scripts. Version 3 of this changeset was tested against ci_build.sh and some images were booted on hardware available here. No regressions were detected (to date). This was also tested on the creator-ci40-marduk machine (subject of another patch series). Changes compared to v2: Rework custom_kernel.md Compare PV+/-LOCAL_VERSION and kernelrelease string Changes compared to v1: Add missing reference to KERNEL_DEFCONFIG in requirements list Amend postinst/postrm scripts according to CONFIG_BLK_DEV_INITRD Remove (unused) pre/post host hooks from debian/isar scripts Add preinst/prerm scripts shipped by the kernel Align {pre,post}{inst,rm} scripts with upstream Add ${BASE_DISTRO} to Build-Profiles Add nolibcdev to Build-Profiles if KERNEL_LIBC_DEV_DEPLOY != 1 Only build the libc-dev packages if it is needed Cedric Hombourger (7): recipes-kernel/linux: make KERNEL_DEFCONFIG support in-tree defconfigs linux-mainline: fix stripping of .0 from the kernel version linux-mainline: update from 4.19.0 to 4.19.80 linux-custom: add support for kernel config fragments linux-mainline: disable support for HFS to demonstrate use of config fragments buildchroot-host: install qemu-static to support hybrid cross-compiles linux-custom: rewrite to no longer depend on the kernel's builddeb doc/custom_kernel.md | 80 +++++++ .../recipes-kernel/linux/files/no-fs-hfs.cfg | 2 + ...ne_4.19.0.bb => linux-mainline_4.19.80.bb} | 8 +- .../buildchroot/buildchroot-host.bb | 6 + .../linux/files/build-kernel.sh | 129 ----------- meta/recipes-kernel/linux/files/debian/compat | 1 + .../linux/files/debian/control.tmpl | 37 ++++ .../linux/files/debian/isar/build.tmpl | 45 ++++ .../linux/files/debian/isar/clean.tmpl | 21 ++ .../linux/files/debian/isar/common.tmpl | 50 +++++ .../linux/files/debian/isar/install.tmpl | 206 ++++++++++++++++++ .../files/debian/linux-image.postinst.tmpl | 29 +++ .../files/debian/linux-image.postrm.tmpl | 35 +++ .../files/debian/linux-image.preinst.tmpl | 25 +++ .../linux/files/debian/linux-image.prerm.tmpl | 21 ++ .../linux/files/debian/rules.tmpl | 39 ++++ meta/recipes-kernel/linux/linux-custom.inc | 204 ++++++++++++----- 17 files changed, 752 insertions(+), 186 deletions(-) create mode 100644 doc/custom_kernel.md create mode 100644 meta-isar/recipes-kernel/linux/files/no-fs-hfs.cfg rename meta-isar/recipes-kernel/linux/{linux-mainline_4.19.0.bb => linux-mainline_4.19.80.bb} (67%) delete mode 100644 meta/recipes-kernel/linux/files/build-kernel.sh create mode 100644 meta/recipes-kernel/linux/files/debian/compat create mode 100644 meta/recipes-kernel/linux/files/debian/control.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/isar/build.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/isar/clean.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/isar/common.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/isar/install.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.preinst.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.prerm.tmpl create mode 100755 meta/recipes-kernel/linux/files/debian/rules.tmpl -- 2.20.1 ^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH L-C v3 1/7] recipes-kernel/linux: make KERNEL_DEFCONFIG support in-tree defconfigs 2019-11-08 7:07 ` [PATCH L-C v3 " Cedric Hombourger @ 2019-11-08 7:07 ` Cedric Hombourger 2019-11-08 7:07 ` [PATCH L-C v3 2/7] linux-mainline: fix stripping of .0 from the kernel version Cedric Hombourger ` (6 subsequent siblings) 7 siblings, 0 replies; 83+ messages in thread From: Cedric Hombourger @ 2019-11-08 7:07 UTC (permalink / raw) To: isar-users; +Cc: Cedric Hombourger The recipe expected the specified KERNEL_DEFCONFIG to be provided as a SRC_URI while the kernel already provides many useful defconfigs. The recipe will now check if the specified configuration can be found in WORKDIR. If it is found, it will then be used via "make olddefconfig" as before. It will otherwise assume that it is an in-tree defconfig and attempt a "make foobar_defconfig" (where foobar_defconfig is the user-provided KERNEL_DEFCONFIG setting). To support this change, a KERNEL_CONFIG_TARGET variable is passed to the build-kernel.sh script via the environment. Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com> --- meta/recipes-kernel/linux/files/build-kernel.sh | 6 +----- meta/recipes-kernel/linux/linux-custom.inc | 9 ++++++++- 2 files changed, 9 insertions(+), 6 deletions(-) diff --git a/meta/recipes-kernel/linux/files/build-kernel.sh b/meta/recipes-kernel/linux/files/build-kernel.sh index b46cefa..386e58b 100644 --- a/meta/recipes-kernel/linux/files/build-kernel.sh +++ b/meta/recipes-kernel/linux/files/build-kernel.sh @@ -36,11 +36,7 @@ REPACK_DIR="$1/../repack" REPACK_LINUX_IMAGE_DIR="${REPACK_DIR}/linux-image" REPACK_LINUX_HEADERS_DIR="${REPACK_DIR}/linux-headers" -if [ -e .config ]; then - make olddefconfig -else - make defconfig -fi +make ${KERNEL_CONFIG_TARGET} || exit ${?} KV=$( make -s kernelrelease ) if [ "${KV}" != "${PV}" ]; then diff --git a/meta/recipes-kernel/linux/linux-custom.inc b/meta/recipes-kernel/linux/linux-custom.inc index b597e25..0c185d8 100644 --- a/meta/recipes-kernel/linux/linux-custom.inc +++ b/meta/recipes-kernel/linux/linux-custom.inc @@ -61,8 +61,14 @@ addtask devshell after do_install_builddeps dpkg_runbuild() { chmod +x ${WORKDIR}/build-kernel.sh + KERNEL_CONFIG_TARGET="${KERNEL_DEFCONFIG}" if [ -n "${KERNEL_DEFCONFIG}" ]; then - cp ${WORKDIR}/${KERNEL_DEFCONFIG} ${S}/.config + if [ -e "${WORKDIR}/${KERNEL_DEFCONFIG}" ]; then + cp ${WORKDIR}/${KERNEL_DEFCONFIG} ${S}/.config + KERNEL_CONFIG_TARGET="olddefconfig" + fi + else + KERNEL_CONFIG_TARGET="defconfig" fi E="${@ bb.utils.export_proxies(d)}" @@ -71,6 +77,7 @@ dpkg_runbuild() { export KERNEL_NAME=${KERNEL_NAME_PROVIDED} export KBUILD_DEPENDS="${KBUILD_DEPENDS}" + export KERNEL_CONFIG_TARGET export KERNEL_DEBIAN_DEPENDS="${KERNEL_DEBIAN_DEPENDS}" export KERNEL_FILE="${KERNEL_FILE}" export KERNEL_HEADERS_DEBIAN_DEPENDS="${KERNEL_HEADERS_DEBIAN_DEPENDS}" -- 2.20.1 ^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH L-C v3 2/7] linux-mainline: fix stripping of .0 from the kernel version 2019-11-08 7:07 ` [PATCH L-C v3 " Cedric Hombourger 2019-11-08 7:07 ` [PATCH L-C v3 1/7] recipes-kernel/linux: make KERNEL_DEFCONFIG support in-tree defconfigs Cedric Hombourger @ 2019-11-08 7:07 ` Cedric Hombourger 2019-11-08 7:07 ` [PATCH L-C v3 3/7] linux-mainline: update from 4.19.0 to 4.19.80 Cedric Hombourger ` (5 subsequent siblings) 7 siblings, 0 replies; 83+ messages in thread From: Cedric Hombourger @ 2019-11-08 7:07 UTC (permalink / raw) To: isar-users; +Cc: Cedric Hombourger The strip() function was used to strip the trailing .0 from the kernel version since it is omitted in the upstream archive file names. That function is actually taking a character set as argument not a substring. This was causing ARCHIVE_VERSION to expand to "4.19.8" when the user provided version was "4.19.80". The inline python code was changed to check if the version ends with ".0" and only then remove the last two characters. Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com> --- meta-isar/recipes-kernel/linux/linux-mainline_4.19.0.bb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/meta-isar/recipes-kernel/linux/linux-mainline_4.19.0.bb b/meta-isar/recipes-kernel/linux/linux-mainline_4.19.0.bb index e50e149..127f6e2 100644 --- a/meta-isar/recipes-kernel/linux/linux-mainline_4.19.0.bb +++ b/meta-isar/recipes-kernel/linux/linux-mainline_4.19.0.bb @@ -7,7 +7,7 @@ require recipes-kernel/linux/linux-custom.inc -ARCHIVE_VERSION = "${@d.getVar('PV').strip('.0')}" +ARCHIVE_VERSION = "${@ d.getVar('PV')[:-2] if d.getVar('PV').endswith('.0') else d.getVar('PV') }" SRC_URI += " \ https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-${ARCHIVE_VERSION}.tar.xz \ -- 2.20.1 ^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH L-C v3 3/7] linux-mainline: update from 4.19.0 to 4.19.80 2019-11-08 7:07 ` [PATCH L-C v3 " Cedric Hombourger 2019-11-08 7:07 ` [PATCH L-C v3 1/7] recipes-kernel/linux: make KERNEL_DEFCONFIG support in-tree defconfigs Cedric Hombourger 2019-11-08 7:07 ` [PATCH L-C v3 2/7] linux-mainline: fix stripping of .0 from the kernel version Cedric Hombourger @ 2019-11-08 7:07 ` Cedric Hombourger 2019-11-08 7:07 ` [PATCH L-C v3 4/7] linux-custom: add support for kernel config fragments Cedric Hombourger ` (4 subsequent siblings) 7 siblings, 0 replies; 83+ messages in thread From: Cedric Hombourger @ 2019-11-08 7:07 UTC (permalink / raw) To: isar-users; +Cc: Cedric Hombourger In preparation for a new machine addition, update the linux-mainline kernel from 4.19.0 to 4.19.80. This is needed to get some of the mips fixes that were integrated in this longterm kernel version (and in particular: MIPS: have "plain" make calls build dtbs for selected platforms, i.e. 637dfa0fad6d91a9a709dc70549a6d20fa77f615 upstream) Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com> --- .../{linux-mainline_4.19.0.bb => linux-mainline_4.19.80.bb} | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) rename meta-isar/recipes-kernel/linux/{linux-mainline_4.19.0.bb => linux-mainline_4.19.80.bb} (87%) diff --git a/meta-isar/recipes-kernel/linux/linux-mainline_4.19.0.bb b/meta-isar/recipes-kernel/linux/linux-mainline_4.19.80.bb similarity index 87% rename from meta-isar/recipes-kernel/linux/linux-mainline_4.19.0.bb rename to meta-isar/recipes-kernel/linux/linux-mainline_4.19.80.bb index 127f6e2..b41d1a8 100644 --- a/meta-isar/recipes-kernel/linux/linux-mainline_4.19.0.bb +++ b/meta-isar/recipes-kernel/linux/linux-mainline_4.19.80.bb @@ -12,7 +12,7 @@ ARCHIVE_VERSION = "${@ d.getVar('PV')[:-2] if d.getVar('PV').endswith('.0') else SRC_URI += " \ https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-${ARCHIVE_VERSION}.tar.xz \ file://x86_64_defconfig" -SRC_URI[sha256sum] = "0c68f5655528aed4f99dae71a5b259edc93239fa899e2df79c055275c21749a1" +SRC_URI[sha256sum] = "80a9ba764e088aa7fddfef5a97c0236905e291468a37832243b6f3828d36e7ec" SRC_URI_append_de0-nano-soc = " \ file://0001-ARM-dts-socfpga-Rename-socfpga_cyclone5_de0_-sockit-.patch" -- 2.20.1 ^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH L-C v3 4/7] linux-custom: add support for kernel config fragments 2019-11-08 7:07 ` [PATCH L-C v3 " Cedric Hombourger ` (2 preceding siblings ...) 2019-11-08 7:07 ` [PATCH L-C v3 3/7] linux-mainline: update from 4.19.0 to 4.19.80 Cedric Hombourger @ 2019-11-08 7:07 ` Cedric Hombourger 2019-11-08 7:07 ` [PATCH L-C v3 5/7] linux-mainline: disable support for HFS to demonstrate use of " Cedric Hombourger ` (3 subsequent siblings) 7 siblings, 0 replies; 83+ messages in thread From: Cedric Hombourger @ 2019-11-08 7:07 UTC (permalink / raw) To: isar-users; +Cc: Cedric Hombourger Check for .cfg files listed in SRC_URI and merge them with the main defconfig using the kernel merge_config script. The optional "apply" parameter may be used for the specified fragment to be ignore (set to "no"). Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com> --- .../linux/files/build-kernel.sh | 3 +++ meta/recipes-kernel/linux/linux-custom.inc | 23 +++++++++++++++++++ 2 files changed, 26 insertions(+) diff --git a/meta/recipes-kernel/linux/files/build-kernel.sh b/meta/recipes-kernel/linux/files/build-kernel.sh index 386e58b..f56e96c 100644 --- a/meta/recipes-kernel/linux/files/build-kernel.sh +++ b/meta/recipes-kernel/linux/files/build-kernel.sh @@ -37,6 +37,9 @@ REPACK_LINUX_IMAGE_DIR="${REPACK_DIR}/linux-image" REPACK_LINUX_HEADERS_DIR="${REPACK_DIR}/linux-headers" make ${KERNEL_CONFIG_TARGET} || exit ${?} +if [ -n "${KERNEL_FRAGMENTS}" ]; then + scripts/kconfig/merge_config.sh -m .config ${KERNEL_FRAGMENTS} +fi KV=$( make -s kernelrelease ) if [ "${KV}" != "${PV}" ]; then diff --git a/meta/recipes-kernel/linux/linux-custom.inc b/meta/recipes-kernel/linux/linux-custom.inc index 0c185d8..ca91f64 100644 --- a/meta/recipes-kernel/linux/linux-custom.inc +++ b/meta/recipes-kernel/linux/linux-custom.inc @@ -18,6 +18,19 @@ python() { 'linux-headers-' + kernel_name) } +def config_fragments(d): + fragments = [] + sources = d.getVar("SRC_URI").split() + for s in sources: + _, _, local, _, _, parm = bb.fetch.decodeurl(s) + apply = parm.get("apply") + if apply == "no": + continue + base, ext = os.path.splitext(os.path.basename(local)) + if ext and ext in (".cfg"): + fragments.append(local) + return fragments + inherit dpkg-base SRC_URI += "file://build-kernel.sh" @@ -71,6 +84,16 @@ dpkg_runbuild() { KERNEL_CONFIG_TARGET="defconfig" fi + # copy config fragments over to the kernel tree + src_frags="${@ " ".join(config_fragments(d)) }" + out_frags="${@ " ".join(map(lambda frag: 'debian/fragments/' + frag, config_fragments(d))) }" + rm -rf ${S}/debian/fragments + if [ -n "${src_frags}" ]; then + mkdir -p ${S}/debian/fragments + (cd ${WORKDIR} && cp ${src_frags} ${S}/debian/fragments/) + fi + export KERNEL_FRAGMENTS="${out_frags}" + E="${@ bb.utils.export_proxies(d)}" export PV=${PV} -- 2.20.1 ^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH L-C v3 5/7] linux-mainline: disable support for HFS to demonstrate use of config fragments 2019-11-08 7:07 ` [PATCH L-C v3 " Cedric Hombourger ` (3 preceding siblings ...) 2019-11-08 7:07 ` [PATCH L-C v3 4/7] linux-custom: add support for kernel config fragments Cedric Hombourger @ 2019-11-08 7:07 ` Cedric Hombourger 2019-11-08 7:07 ` [PATCH L-C v3 6/7] buildchroot-host: install qemu-static to support hybrid cross-compiles Cedric Hombourger ` (2 subsequent siblings) 7 siblings, 0 replies; 83+ messages in thread From: Cedric Hombourger @ 2019-11-08 7:07 UTC (permalink / raw) To: isar-users; +Cc: Cedric Hombourger Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com> --- meta-isar/recipes-kernel/linux/files/no-fs-hfs.cfg | 2 ++ meta-isar/recipes-kernel/linux/linux-mainline_4.19.80.bb | 4 +++- 2 files changed, 5 insertions(+), 1 deletion(-) create mode 100644 meta-isar/recipes-kernel/linux/files/no-fs-hfs.cfg diff --git a/meta-isar/recipes-kernel/linux/files/no-fs-hfs.cfg b/meta-isar/recipes-kernel/linux/files/no-fs-hfs.cfg new file mode 100644 index 0000000..5dbfadb --- /dev/null +++ b/meta-isar/recipes-kernel/linux/files/no-fs-hfs.cfg @@ -0,0 +1,2 @@ +# CONFIG_FS_HFS is not set +# CONFIG_HFSPLUS_FS is not set diff --git a/meta-isar/recipes-kernel/linux/linux-mainline_4.19.80.bb b/meta-isar/recipes-kernel/linux/linux-mainline_4.19.80.bb index b41d1a8..83038f0 100644 --- a/meta-isar/recipes-kernel/linux/linux-mainline_4.19.80.bb +++ b/meta-isar/recipes-kernel/linux/linux-mainline_4.19.80.bb @@ -11,7 +11,9 @@ ARCHIVE_VERSION = "${@ d.getVar('PV')[:-2] if d.getVar('PV').endswith('.0') else SRC_URI += " \ https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-${ARCHIVE_VERSION}.tar.xz \ - file://x86_64_defconfig" + file://x86_64_defconfig \ + file://no-fs-hfs.cfg" + SRC_URI[sha256sum] = "80a9ba764e088aa7fddfef5a97c0236905e291468a37832243b6f3828d36e7ec" SRC_URI_append_de0-nano-soc = " \ -- 2.20.1 ^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH L-C v3 6/7] buildchroot-host: install qemu-static to support hybrid cross-compiles 2019-11-08 7:07 ` [PATCH L-C v3 " Cedric Hombourger ` (4 preceding siblings ...) 2019-11-08 7:07 ` [PATCH L-C v3 5/7] linux-mainline: disable support for HFS to demonstrate use of " Cedric Hombourger @ 2019-11-08 7:07 ` Cedric Hombourger 2019-11-08 7:14 ` Jan Kiszka 2019-11-08 7:07 ` [PATCH L-C v3 7/7] linux-custom: rewrite to no longer depend on the kernel's builddeb Cedric Hombourger 2019-11-08 7:12 ` [PATCH L-C v3 0/7] linux-custom recipe rework Jan Kiszka 7 siblings, 1 reply; 83+ messages in thread From: Cedric Hombourger @ 2019-11-08 7:07 UTC (permalink / raw) To: isar-users; +Cc: Cedric Hombourger The Linux kernel has great support for cross-compiling the kernel image and modules. There is however no support/mechanism for cross-compiling the build tools from its "scripts" directory. While HOSTCC may be set to use our cross-compiler, the kernel build infrasture would then try to run foreign-arch binaries such as fixdep. The easiest and least intrusive way to support this is to enable execution of such binaries via binfmt/qemu like we do in the buildchroot-target environment. Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com> --- meta/recipes-devtools/buildchroot/buildchroot-host.bb | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/meta/recipes-devtools/buildchroot/buildchroot-host.bb b/meta/recipes-devtools/buildchroot/buildchroot-host.bb index 408ad39..2e76acb 100644 --- a/meta/recipes-devtools/buildchroot/buildchroot-host.bb +++ b/meta/recipes-devtools/buildchroot/buildchroot-host.bb @@ -15,3 +15,9 @@ BUILDCHROOT_PREINSTALL ?= " \ ${BUILDCHROOT_PREINSTALL_COMMON} \ libc6:${DISTRO_ARCH} \ crossbuild-essential-${DISTRO_ARCH}" + +buildchroot_install_files_append() { + if [ -e '/usr/bin/qemu-${QEMU_ARCH}-static' ]; then + sudo cp '/usr/bin/qemu-${QEMU_ARCH}-static' '${BUILDCHROOT_DIR}/usr/bin/qemu-${QEMU_ARCH}-static' + fi +} -- 2.20.1 ^ permalink raw reply [flat|nested] 83+ messages in thread
* Re: [PATCH L-C v3 6/7] buildchroot-host: install qemu-static to support hybrid cross-compiles 2019-11-08 7:07 ` [PATCH L-C v3 6/7] buildchroot-host: install qemu-static to support hybrid cross-compiles Cedric Hombourger @ 2019-11-08 7:14 ` Jan Kiszka 2019-11-08 7:22 ` Jan Kiszka 2019-11-08 7:22 ` Cedric Hombourger 0 siblings, 2 replies; 83+ messages in thread From: Jan Kiszka @ 2019-11-08 7:14 UTC (permalink / raw) To: Cedric Hombourger, isar-users On 08.11.19 08:07, Cedric Hombourger wrote: > The Linux kernel has great support for cross-compiling the kernel image > and modules. There is however no support/mechanism for cross-compiling the > build tools from its "scripts" directory. While HOSTCC may be set to use > our cross-compiler, the kernel build infrasture would then try to run > foreign-arch binaries such as fixdep. The easiest and least intrusive way > to support this is to enable execution of such binaries via binfmt/qemu > like we do in the buildchroot-target environment. > > Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com> > --- > meta/recipes-devtools/buildchroot/buildchroot-host.bb | 6 ++++++ > 1 file changed, 6 insertions(+) > > diff --git a/meta/recipes-devtools/buildchroot/buildchroot-host.bb b/meta/recipes-devtools/buildchroot/buildchroot-host.bb > index 408ad39..2e76acb 100644 > --- a/meta/recipes-devtools/buildchroot/buildchroot-host.bb > +++ b/meta/recipes-devtools/buildchroot/buildchroot-host.bb > @@ -15,3 +15,9 @@ BUILDCHROOT_PREINSTALL ?= " \ > ${BUILDCHROOT_PREINSTALL_COMMON} \ > libc6:${DISTRO_ARCH} \ > crossbuild-essential-${DISTRO_ARCH}" > + > +buildchroot_install_files_append() { > + if [ -e '/usr/bin/qemu-${QEMU_ARCH}-static' ]; then > + sudo cp '/usr/bin/qemu-${QEMU_ARCH}-static' '${BUILDCHROOT_DIR}/usr/bin/qemu-${QEMU_ARCH}-static' > + fi > +} > You didn't address my comment on this one. I doubt it's the right approach. Jan -- Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 83+ messages in thread
* Re: [PATCH L-C v3 6/7] buildchroot-host: install qemu-static to support hybrid cross-compiles 2019-11-08 7:14 ` Jan Kiszka @ 2019-11-08 7:22 ` Jan Kiszka 2019-11-08 7:23 ` Cedric Hombourger 2019-11-08 7:22 ` Cedric Hombourger 1 sibling, 1 reply; 83+ messages in thread From: Jan Kiszka @ 2019-11-08 7:22 UTC (permalink / raw) To: Cedric Hombourger, isar-users On 08.11.19 08:14, [ext] Jan Kiszka wrote: > On 08.11.19 08:07, Cedric Hombourger wrote: >> The Linux kernel has great support for cross-compiling the kernel image >> and modules. There is however no support/mechanism for cross-compiling >> the >> build tools from its "scripts" directory. While HOSTCC may be set to use >> our cross-compiler, the kernel build infrasture would then try to run >> foreign-arch binaries such as fixdep. The easiest and least intrusive way >> to support this is to enable execution of such binaries via binfmt/qemu >> like we do in the buildchroot-target environment. >> >> Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com> >> --- >> meta/recipes-devtools/buildchroot/buildchroot-host.bb | 6 ++++++ >> 1 file changed, 6 insertions(+) >> >> diff --git a/meta/recipes-devtools/buildchroot/buildchroot-host.bb >> b/meta/recipes-devtools/buildchroot/buildchroot-host.bb >> index 408ad39..2e76acb 100644 >> --- a/meta/recipes-devtools/buildchroot/buildchroot-host.bb >> +++ b/meta/recipes-devtools/buildchroot/buildchroot-host.bb >> @@ -15,3 +15,9 @@ BUILDCHROOT_PREINSTALL ?= " \ >> ${BUILDCHROOT_PREINSTALL_COMMON} \ >> libc6:${DISTRO_ARCH} \ >> crossbuild-essential-${DISTRO_ARCH}" >> + >> +buildchroot_install_files_append() { >> + if [ -e '/usr/bin/qemu-${QEMU_ARCH}-static' ]; then >> + sudo cp '/usr/bin/qemu-${QEMU_ARCH}-static' >> '${BUILDCHROOT_DIR}/usr/bin/qemu-${QEMU_ARCH}-static' >> + fi >> +} >> > > You didn't address my comment on this one. I doubt it's the right approach. > IOW: When we cross-compile linux-headers, the output is for the host architecture only, not for the target. If the resulting package suggest something different, that must be fixed. Jan -- Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 83+ messages in thread
* Re: [PATCH L-C v3 6/7] buildchroot-host: install qemu-static to support hybrid cross-compiles 2019-11-08 7:22 ` Jan Kiszka @ 2019-11-08 7:23 ` Cedric Hombourger 0 siblings, 0 replies; 83+ messages in thread From: Cedric Hombourger @ 2019-11-08 7:23 UTC (permalink / raw) To: Jan Kiszka, isar-users On 11/8/19 8:22 AM, Jan Kiszka wrote: > On 08.11.19 08:14, [ext] Jan Kiszka wrote: >> On 08.11.19 08:07, Cedric Hombourger wrote: >>> The Linux kernel has great support for cross-compiling the kernel image >>> and modules. There is however no support/mechanism for >>> cross-compiling the >>> build tools from its "scripts" directory. While HOSTCC may be set to >>> use >>> our cross-compiler, the kernel build infrasture would then try to run >>> foreign-arch binaries such as fixdep. The easiest and least >>> intrusive way >>> to support this is to enable execution of such binaries via binfmt/qemu >>> like we do in the buildchroot-target environment. >>> >>> Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com> >>> --- >>> meta/recipes-devtools/buildchroot/buildchroot-host.bb | 6 ++++++ >>> 1 file changed, 6 insertions(+) >>> >>> diff --git a/meta/recipes-devtools/buildchroot/buildchroot-host.bb >>> b/meta/recipes-devtools/buildchroot/buildchroot-host.bb >>> index 408ad39..2e76acb 100644 >>> --- a/meta/recipes-devtools/buildchroot/buildchroot-host.bb >>> +++ b/meta/recipes-devtools/buildchroot/buildchroot-host.bb >>> @@ -15,3 +15,9 @@ BUILDCHROOT_PREINSTALL ?= " \ >>> ${BUILDCHROOT_PREINSTALL_COMMON} \ >>> libc6:${DISTRO_ARCH} \ >>> crossbuild-essential-${DISTRO_ARCH}" >>> + >>> +buildchroot_install_files_append() { >>> + if [ -e '/usr/bin/qemu-${QEMU_ARCH}-static' ]; then >>> + sudo cp '/usr/bin/qemu-${QEMU_ARCH}-static' >>> '${BUILDCHROOT_DIR}/usr/bin/qemu-${QEMU_ARCH}-static' >>> + fi >>> +} >>> >> >> You didn't address my comment on this one. I doubt it's the right >> approach. >> > > IOW: When we cross-compile linux-headers, the output is for the host > architecture only, not for the target. If the resulting package > suggest something different, that must be fixed. see the response I just posted. I am afraid this is incorrect but again feel free to suggest otherwise with a bit more concrete meat and a look upstream... > > Jan > ^ permalink raw reply [flat|nested] 83+ messages in thread
* Re: [PATCH L-C v3 6/7] buildchroot-host: install qemu-static to support hybrid cross-compiles 2019-11-08 7:14 ` Jan Kiszka 2019-11-08 7:22 ` Jan Kiszka @ 2019-11-08 7:22 ` Cedric Hombourger 2019-11-08 7:35 ` Jan Kiszka 1 sibling, 1 reply; 83+ messages in thread From: Cedric Hombourger @ 2019-11-08 7:22 UTC (permalink / raw) To: Jan Kiszka, isar-users On 11/8/19 8:14 AM, Jan Kiszka wrote: > On 08.11.19 08:07, Cedric Hombourger wrote: >> The Linux kernel has great support for cross-compiling the kernel image >> and modules. There is however no support/mechanism for >> cross-compiling the >> build tools from its "scripts" directory. While HOSTCC may be set to use >> our cross-compiler, the kernel build infrasture would then try to run >> foreign-arch binaries such as fixdep. The easiest and least intrusive >> way >> to support this is to enable execution of such binaries via binfmt/qemu >> like we do in the buildchroot-target environment. >> >> Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com> >> --- >> meta/recipes-devtools/buildchroot/buildchroot-host.bb | 6 ++++++ >> 1 file changed, 6 insertions(+) >> >> diff --git a/meta/recipes-devtools/buildchroot/buildchroot-host.bb >> b/meta/recipes-devtools/buildchroot/buildchroot-host.bb >> index 408ad39..2e76acb 100644 >> --- a/meta/recipes-devtools/buildchroot/buildchroot-host.bb >> +++ b/meta/recipes-devtools/buildchroot/buildchroot-host.bb >> @@ -15,3 +15,9 @@ BUILDCHROOT_PREINSTALL ?= " \ >> ${BUILDCHROOT_PREINSTALL_COMMON} \ >> libc6:${DISTRO_ARCH} \ >> crossbuild-essential-${DISTRO_ARCH}" >> + >> +buildchroot_install_files_append() { >> + if [ -e '/usr/bin/qemu-${QEMU_ARCH}-static' ]; then >> + sudo cp '/usr/bin/qemu-${QEMU_ARCH}-static' >> '${BUILDCHROOT_DIR}/usr/bin/qemu-${QEMU_ARCH}-static' >> + fi >> +} >> > > You didn't address my comment on this one. I doubt it's the right > approach. More specifically? I do have something in the works but we *still* need to build a linux-kernel-headers package for the target. there are two approaches: (1) use the Makefiles from the kernel and "simply" override HOSTCC but you then need qemu-static since the kernel will need to run some of its tools from "scripts/" (fixdep in particular) when it builds the other tools (e.g modpost) or (2) do like upstream manually cross-compile the tools. The downside of (2) is that you need to know what to compile (list of .c files) and how to compile them. That "knowledge" may vary from one kernel to another. I have local patches to produce a linux-kernel-headers-cross-amd64 package which will ship headers in /usr/${DEB_HOST_GNU_TYPE} (i.e. next to libc headers) as well as kernel build tools for the build machine (typically amd64). This will allow us to cross-compile out-of-tree kernel modules without running any tools with a foreign-arch (for the avoidance of doubt, we still need qemu-user-static in buildchroot-host while building the kernel) So with all that said, you are more than welcome to suggest a better approach.... > > Jan > ^ permalink raw reply [flat|nested] 83+ messages in thread
* Re: [PATCH L-C v3 6/7] buildchroot-host: install qemu-static to support hybrid cross-compiles 2019-11-08 7:22 ` Cedric Hombourger @ 2019-11-08 7:35 ` Jan Kiszka 2019-11-08 7:59 ` Cedric Hombourger 0 siblings, 1 reply; 83+ messages in thread From: Jan Kiszka @ 2019-11-08 7:35 UTC (permalink / raw) To: Cedric Hombourger, isar-users On 08.11.19 08:22, Cedric Hombourger wrote: > > On 11/8/19 8:14 AM, Jan Kiszka wrote: >> On 08.11.19 08:07, Cedric Hombourger wrote: >>> The Linux kernel has great support for cross-compiling the kernel image >>> and modules. There is however no support/mechanism for >>> cross-compiling the >>> build tools from its "scripts" directory. While HOSTCC may be set to use >>> our cross-compiler, the kernel build infrasture would then try to run >>> foreign-arch binaries such as fixdep. The easiest and least intrusive >>> way >>> to support this is to enable execution of such binaries via binfmt/qemu >>> like we do in the buildchroot-target environment. >>> >>> Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com> >>> --- >>> meta/recipes-devtools/buildchroot/buildchroot-host.bb | 6 ++++++ >>> 1 file changed, 6 insertions(+) >>> >>> diff --git a/meta/recipes-devtools/buildchroot/buildchroot-host.bb >>> b/meta/recipes-devtools/buildchroot/buildchroot-host.bb >>> index 408ad39..2e76acb 100644 >>> --- a/meta/recipes-devtools/buildchroot/buildchroot-host.bb >>> +++ b/meta/recipes-devtools/buildchroot/buildchroot-host.bb >>> @@ -15,3 +15,9 @@ BUILDCHROOT_PREINSTALL ?= " \ >>> ${BUILDCHROOT_PREINSTALL_COMMON} \ >>> libc6:${DISTRO_ARCH} \ >>> crossbuild-essential-${DISTRO_ARCH}" >>> + >>> +buildchroot_install_files_append() { >>> + if [ -e '/usr/bin/qemu-${QEMU_ARCH}-static' ]; then >>> + sudo cp '/usr/bin/qemu-${QEMU_ARCH}-static' >>> '${BUILDCHROOT_DIR}/usr/bin/qemu-${QEMU_ARCH}-static' >>> + fi >>> +} >>> >> >> You didn't address my comment on this one. I doubt it's the right >> approach. > > More specifically? > > I do have something in the works but we *still* need to build a > linux-kernel-headers package for the target. there are two approaches: Currently, if you need the package for the target, you need to build it natively. The cases of requiring such packages is so far rare enough (none in all the many cases I saw so far) to deal with the slowdown. > (1) use the Makefiles from the kernel and "simply" override HOSTCC but > you then need qemu-static since the kernel will need to run some of its > tools from "scripts/" (fixdep in particular) when it builds the other > tools (e.g modpost) or (2) do like upstream manually cross-compile the > tools. > > The downside of (2) is that you need to know what to compile (list of .c > files) and how to compile them. That "knowledge" may vary from one > kernel to another. > > I have local patches to produce a linux-kernel-headers-cross-amd64 > package which will ship headers in /usr/${DEB_HOST_GNU_TYPE} (i.e. next > to libc headers) as well as kernel build tools for the build machine > (typically amd64). This will allow us to cross-compile out-of-tree > kernel modules without running any tools with a foreign-arch (for the > avoidance of doubt, we still need qemu-user-static in buildchroot-host > while building the kernel) > > So with all that said, you are more than welcome to suggest a better > approach.... Step 1 is to tag the currently generated header package in a way that it won't be accidentally selected for target installation when it was cross-built for a different host architecture. Step 2 could be factoring out the linux-headers package build so that it can run with ISAR_CROSS_COMPILE = "0" for the target architecture in the target buildchroot - when that special case is actually needed. Alternatively, though more for the long run, the tools cross-build of upstream could be improved. Jan -- Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 83+ messages in thread
* Re: [PATCH L-C v3 6/7] buildchroot-host: install qemu-static to support hybrid cross-compiles 2019-11-08 7:35 ` Jan Kiszka @ 2019-11-08 7:59 ` Cedric Hombourger 2019-11-08 8:28 ` Jan Kiszka 0 siblings, 1 reply; 83+ messages in thread From: Cedric Hombourger @ 2019-11-08 7:59 UTC (permalink / raw) To: Jan Kiszka, isar-users On 11/8/19 8:35 AM, Jan Kiszka wrote: > On 08.11.19 08:22, Cedric Hombourger wrote: >> >> On 11/8/19 8:14 AM, Jan Kiszka wrote: >>> On 08.11.19 08:07, Cedric Hombourger wrote: >>>> The Linux kernel has great support for cross-compiling the kernel >>>> image >>>> and modules. There is however no support/mechanism for >>>> cross-compiling the >>>> build tools from its "scripts" directory. While HOSTCC may be set >>>> to use >>>> our cross-compiler, the kernel build infrasture would then try to run >>>> foreign-arch binaries such as fixdep. The easiest and least >>>> intrusive way >>>> to support this is to enable execution of such binaries via >>>> binfmt/qemu >>>> like we do in the buildchroot-target environment. >>>> >>>> Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com> >>>> --- >>>> meta/recipes-devtools/buildchroot/buildchroot-host.bb | 6 ++++++ >>>> 1 file changed, 6 insertions(+) >>>> >>>> diff --git a/meta/recipes-devtools/buildchroot/buildchroot-host.bb >>>> b/meta/recipes-devtools/buildchroot/buildchroot-host.bb >>>> index 408ad39..2e76acb 100644 >>>> --- a/meta/recipes-devtools/buildchroot/buildchroot-host.bb >>>> +++ b/meta/recipes-devtools/buildchroot/buildchroot-host.bb >>>> @@ -15,3 +15,9 @@ BUILDCHROOT_PREINSTALL ?= " \ >>>> ${BUILDCHROOT_PREINSTALL_COMMON} \ >>>> libc6:${DISTRO_ARCH} \ >>>> crossbuild-essential-${DISTRO_ARCH}" >>>> + >>>> +buildchroot_install_files_append() { >>>> + if [ -e '/usr/bin/qemu-${QEMU_ARCH}-static' ]; then >>>> + sudo cp '/usr/bin/qemu-${QEMU_ARCH}-static' >>>> '${BUILDCHROOT_DIR}/usr/bin/qemu-${QEMU_ARCH}-static' >>>> + fi >>>> +} >>>> >>> >>> You didn't address my comment on this one. I doubt it's the right >>> approach. >> >> More specifically? >> >> I do have something in the works but we *still* need to build a >> linux-kernel-headers package for the target. there are two approaches: > > Currently, if you need the package for the target, you need to build > it natively. The cases of requiring such packages is so far rare > enough (none in all the many cases I saw so far) to deal with the > slowdown. > >> (1) use the Makefiles from the kernel and "simply" override HOSTCC >> but you then need qemu-static since the kernel will need to run some >> of its tools from "scripts/" (fixdep in particular) when it builds >> the other tools (e.g modpost) or (2) do like upstream manually >> cross-compile the tools. >> >> The downside of (2) is that you need to know what to compile (list of >> .c files) and how to compile them. That "knowledge" may vary from one >> kernel to another. >> >> I have local patches to produce a linux-kernel-headers-cross-amd64 >> package which will ship headers in /usr/${DEB_HOST_GNU_TYPE} (i.e. >> next to libc headers) as well as kernel build tools for the build >> machine (typically amd64). This will allow us to cross-compile >> out-of-tree kernel modules without running any tools with a >> foreign-arch (for the avoidance of doubt, we still need >> qemu-user-static in buildchroot-host while building the kernel) >> >> So with all that said, you are more than welcome to suggest a better >> approach.... > > Step 1 is to tag the currently generated header package in a way that > it won't be accidentally selected for target installation when it was > cross-built for a different host architecture. Step 2 could be > factoring out the linux-headers package build so that it can run with > ISAR_CROSS_COMPILE = "0" for the target architecture in the target > buildchroot - when that special case is actually needed. If in step1, you are suggesting that our recipe generates packages for the following "Architecture": ${HOST_ARCH}, all, ${DISTRO_ARCH} then to the best of my knowledge that isn't possible! I think you can only generate packages for all, ${DISTRO_ARCH} from a dpkg-buildpackage run but again, let me know if you think otherwise (could not find anything in the Debian Standards document suggesting so but I may have overlooked) If we agree that using debhelper and therefore our generic dpkg class is a good improvement (compared to kernel's builddeb script crafting packages on its own and having to specify dependencies manually), I would suggest we stick with the qemu-user-static approach for now (it is only needed for a fraction of seconds during the build so to me that is a non-issue). As hinted, I am working on a linux-kernel-headers-cross package generation. I am currently considering running build.sh a second time but this time targeting HOST_ARCH and with a control file only carrying that -cross package (it would use the same kernel sources/tree as input) but I'd like to make this a second step since it needs more work and I a genuinely believe that the current solution is something we can live with for now > Alternatively, though more for the long run, the tools cross-build of > upstream could be improved. yes that would be good (if you are referring to the ability to do something like make scripts O=build-host-tools HOST_CROSS_COMPILE=${DEB_HOST_GNU_TYPE}- in the kernel) > > Jan > ^ permalink raw reply [flat|nested] 83+ messages in thread
* Re: [PATCH L-C v3 6/7] buildchroot-host: install qemu-static to support hybrid cross-compiles 2019-11-08 7:59 ` Cedric Hombourger @ 2019-11-08 8:28 ` Jan Kiszka 0 siblings, 0 replies; 83+ messages in thread From: Jan Kiszka @ 2019-11-08 8:28 UTC (permalink / raw) To: Cedric Hombourger, isar-users On 08.11.19 08:59, Cedric Hombourger wrote: > > On 11/8/19 8:35 AM, Jan Kiszka wrote: >> On 08.11.19 08:22, Cedric Hombourger wrote: >>> >>> On 11/8/19 8:14 AM, Jan Kiszka wrote: >>>> On 08.11.19 08:07, Cedric Hombourger wrote: >>>>> The Linux kernel has great support for cross-compiling the kernel >>>>> image >>>>> and modules. There is however no support/mechanism for >>>>> cross-compiling the >>>>> build tools from its "scripts" directory. While HOSTCC may be set >>>>> to use >>>>> our cross-compiler, the kernel build infrasture would then try to run >>>>> foreign-arch binaries such as fixdep. The easiest and least >>>>> intrusive way >>>>> to support this is to enable execution of such binaries via >>>>> binfmt/qemu >>>>> like we do in the buildchroot-target environment. >>>>> >>>>> Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com> >>>>> --- >>>>> meta/recipes-devtools/buildchroot/buildchroot-host.bb | 6 ++++++ >>>>> 1 file changed, 6 insertions(+) >>>>> >>>>> diff --git a/meta/recipes-devtools/buildchroot/buildchroot-host.bb >>>>> b/meta/recipes-devtools/buildchroot/buildchroot-host.bb >>>>> index 408ad39..2e76acb 100644 >>>>> --- a/meta/recipes-devtools/buildchroot/buildchroot-host.bb >>>>> +++ b/meta/recipes-devtools/buildchroot/buildchroot-host.bb >>>>> @@ -15,3 +15,9 @@ BUILDCHROOT_PREINSTALL ?= " \ >>>>> ${BUILDCHROOT_PREINSTALL_COMMON} \ >>>>> libc6:${DISTRO_ARCH} \ >>>>> crossbuild-essential-${DISTRO_ARCH}" >>>>> + >>>>> +buildchroot_install_files_append() { >>>>> + if [ -e '/usr/bin/qemu-${QEMU_ARCH}-static' ]; then >>>>> + sudo cp '/usr/bin/qemu-${QEMU_ARCH}-static' >>>>> '${BUILDCHROOT_DIR}/usr/bin/qemu-${QEMU_ARCH}-static' >>>>> + fi >>>>> +} >>>>> >>>> >>>> You didn't address my comment on this one. I doubt it's the right >>>> approach. >>> >>> More specifically? >>> >>> I do have something in the works but we *still* need to build a >>> linux-kernel-headers package for the target. there are two approaches: >> >> Currently, if you need the package for the target, you need to build >> it natively. The cases of requiring such packages is so far rare >> enough (none in all the many cases I saw so far) to deal with the >> slowdown. >> >>> (1) use the Makefiles from the kernel and "simply" override HOSTCC >>> but you then need qemu-static since the kernel will need to run some >>> of its tools from "scripts/" (fixdep in particular) when it builds >>> the other tools (e.g modpost) or (2) do like upstream manually >>> cross-compile the tools. >>> >>> The downside of (2) is that you need to know what to compile (list of >>> .c files) and how to compile them. That "knowledge" may vary from one >>> kernel to another. >>> >>> I have local patches to produce a linux-kernel-headers-cross-amd64 >>> package which will ship headers in /usr/${DEB_HOST_GNU_TYPE} (i.e. >>> next to libc headers) as well as kernel build tools for the build >>> machine (typically amd64). This will allow us to cross-compile >>> out-of-tree kernel modules without running any tools with a >>> foreign-arch (for the avoidance of doubt, we still need >>> qemu-user-static in buildchroot-host while building the kernel) >>> >>> So with all that said, you are more than welcome to suggest a better >>> approach.... >> >> Step 1 is to tag the currently generated header package in a way that >> it won't be accidentally selected for target installation when it was >> cross-built for a different host architecture. Step 2 could be >> factoring out the linux-headers package build so that it can run with >> ISAR_CROSS_COMPILE = "0" for the target architecture in the target >> buildchroot - when that special case is actually needed. > > If in step1, you are suggesting that our recipe generates packages for > the following "Architecture": ${HOST_ARCH}, all, ${DISTRO_ARCH} then to > the best of my knowledge that isn't possible! I think you can only > generate packages for all, ${DISTRO_ARCH} from a dpkg-buildpackage run > but again, let me know if you think otherwise (could not find anything > in the Debian Standards document suggesting so but I may have overlooked) Hmm, the problem is that we only build our packages with -a$target_arch. We would actually have to build them twice when doing cross build, once for the target and once for the host. Then the right architecture would be chosen when installing a self-built build dependency into the buildchroot of the next package, I would assume. > > If we agree that using debhelper and therefore our generic dpkg class is > a good improvement (compared to kernel's builddeb script crafting > packages on its own and having to specify dependencies manually), I > would suggest we stick with the qemu-user-static approach for now (it is > only needed for a fraction of seconds during the build so to me that is > a non-issue). I'm concerned that adding qemu-user-static generally will paper over other issues. The buildchroot-host is not supposed to handle target binaries. We would be off now for this single package. > > As hinted, I am working on a linux-kernel-headers-cross package > generation. I am currently considering running build.sh a second time > but this time targeting HOST_ARCH and with a control file only carrying > that -cross package (it would use the same kernel sources/tree as input) > but I'd like to make this a second step since it needs more work and I a > genuinely believe that the current solution is something we can live > with for now As I said, we have no urgent need for the current solution because you can generate a target-compatible header package by not using cross-build (AKA using production build). You could support that path by introducing a mode where linux-custom only builds the linux-headers package. So I would rather use the time and teach Isar in general about building correct host-vs-target devel packages, including a second build run when cross-building is active. We had similar issues earlier, e.g. in the openssl area. So better fix them conceptually, rather than patching on thing only. > >> Alternatively, though more for the long run, the tools cross-build of >> upstream could be improved. > yes that would be good (if you are referring to the ability to do > something like make scripts O=build-host-tools > HOST_CROSS_COMPILE=${DEB_HOST_GNU_TYPE}- in the kernel) Yes, something in this direction. Jan -- Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 83+ messages in thread
* [PATCH L-C v3 7/7] linux-custom: rewrite to no longer depend on the kernel's builddeb 2019-11-08 7:07 ` [PATCH L-C v3 " Cedric Hombourger ` (5 preceding siblings ...) 2019-11-08 7:07 ` [PATCH L-C v3 6/7] buildchroot-host: install qemu-static to support hybrid cross-compiles Cedric Hombourger @ 2019-11-08 7:07 ` Cedric Hombourger 2019-11-08 7:19 ` Jan Kiszka 2019-11-08 7:12 ` [PATCH L-C v3 0/7] linux-custom recipe rework Jan Kiszka 7 siblings, 1 reply; 83+ messages in thread From: Cedric Hombourger @ 2019-11-08 7:07 UTC (permalink / raw) To: isar-users; +Cc: Cedric Hombourger Regain control over the packaging of the Linux kernel by providing our own debian recipes (debian,{control,rules} files and friends) instead of using the kernel's builddeb script. This will allow generation of packages for "perf" (not included in this changeset) with per distro scheme (Debian and Ubuntu have differences) and maintain compatibility with upstream packages generated from different source packages but dependent on a certain deployment scheme (e.g. linux-base for Debian) Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com> --- doc/custom_kernel.md | 80 +++++++ .../linux/files/build-kernel.sh | 128 ----------- meta/recipes-kernel/linux/files/debian/compat | 1 + .../linux/files/debian/control.tmpl | 37 ++++ .../linux/files/debian/isar/build.tmpl | 45 ++++ .../linux/files/debian/isar/clean.tmpl | 21 ++ .../linux/files/debian/isar/common.tmpl | 50 +++++ .../linux/files/debian/isar/install.tmpl | 206 ++++++++++++++++++ .../files/debian/linux-image.postinst.tmpl | 29 +++ .../files/debian/linux-image.postrm.tmpl | 35 +++ .../files/debian/linux-image.preinst.tmpl | 25 +++ .../linux/files/debian/linux-image.prerm.tmpl | 21 ++ .../linux/files/debian/rules.tmpl | 39 ++++ meta/recipes-kernel/linux/linux-custom.inc | 180 ++++++++++----- 14 files changed, 712 insertions(+), 185 deletions(-) create mode 100644 doc/custom_kernel.md delete mode 100644 meta/recipes-kernel/linux/files/build-kernel.sh create mode 100644 meta/recipes-kernel/linux/files/debian/compat create mode 100644 meta/recipes-kernel/linux/files/debian/control.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/isar/build.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/isar/clean.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/isar/common.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/isar/install.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.preinst.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.prerm.tmpl create mode 100755 meta/recipes-kernel/linux/files/debian/rules.tmpl diff --git a/doc/custom_kernel.md b/doc/custom_kernel.md new file mode 100644 index 0000000..31319d4 --- /dev/null +++ b/doc/custom_kernel.md @@ -0,0 +1,80 @@ +# Custom kernel recipe for Isar + +## Contents + + - [Summary](#summary) + - [Features](#features) + - [Future](#future) + - [Examples](#examples) + +## Summary + +Isar provides a recipe to build custom kernels for Debian-based distributions. +It uses templates to generate the debian meta-data (such as debian/control) and +Debian's [BuildProfiles](https://wiki.debian.org/BuildProfileSpec) to handle +some of the distro specific variations. It should be noted that Isar has moved +away from using the kernel's builddeb script since it would not generate all +the packages we need (and in particular perf). + +## Features + +The linux-custom recipe provides support for: + + 1. Sources to the custom Linux kernel may be specified via `SRC_URI` + + 2. Configure the kernel via an in-tree or an external `defconfig` via + `KERNEL_DEFCONFIG` + + 3. Integrate kernel configuration tweaks via configuration fragments (`.cfg` + files) + + 4. Patches to the linux kernel may be specified via `SRC_URI` + + 5. Ensure that the Isar recipe `PV` matches the kernel release identifier + (`KERNEL_RELEASE`) + + 6. Produce a `linux-image` package that ships the kernel image and modules + + 7. Allow the name of the kernel image to be changed via `KERNEL_FILE` (defaults + to `vmlinuz`) + + 8. Produce a `linux-headers` package which includes kernel headers and kbuild + scripts/tools + + 9. The `linux-headers` package shall support native and cross compiles of + out-of-tree kernel modules + + 10. Produce a `linux-libc-dev` package to support user-land builds + + 11. Only build/ship the `linux-libc-dev` package if instructed to + (`KERNEL_LIBC_DEV_DEPLOY` equals to `"1"`) + + 12. Support both native and cross compiles (`ISAR_CROSS_COMPILE`) + + 13. Support for the following kernel architectures: + + * arm + * arm64 + * mips + * x86 + + 14. Support `devshell` (kernel configuration shall be applied) + +## Future + +In the future, the recipe may be extended to: + + 1. Package perf + + 2. Support inclusion/build of dts files listed in `SRC_URI` + + 3. Be compatible with Ubuntu + +## Examples + +The linux-custom recipe is currently used by the linux-mainline package and is +used mainline recipe may be used for some basic testing. This recipe is being +used by the following machines: + + * de0-nano-soc + * qemumipsel diff --git a/meta/recipes-kernel/linux/files/build-kernel.sh b/meta/recipes-kernel/linux/files/build-kernel.sh deleted file mode 100644 index f56e96c..0000000 --- a/meta/recipes-kernel/linux/files/build-kernel.sh +++ /dev/null @@ -1,128 +0,0 @@ -#!/bin/bash -# -# Custom kernel build -# -# This software is a part of ISAR. -# Copyright (c) Siemens AG, 2018 -# -# SPDX-License-Identifier: MIT - -source /isar/common.sh - -host_arch=$(dpkg --print-architecture) - -if [ "$host_arch" != "$target_arch" ]; then - case $target_arch in - armhf) - export ARCH=arm - export CROSS_COMPILE="arm-linux-gnueabihf-" - ;; - arm64) - export ARCH=arm64 - export CROSS_COMPILE="aarch64-linux-gnu-" - ;; - mipsel) - export ARCH=mips - export CROSS_COMPILE="mipsel-linux-gnu-" - ;; - *) - echo "error: unsupported architecture ($target_arch)" - exit 1 - ;; - esac -fi - -REPACK_DIR="$1/../repack" -REPACK_LINUX_IMAGE_DIR="${REPACK_DIR}/linux-image" -REPACK_LINUX_HEADERS_DIR="${REPACK_DIR}/linux-headers" - -make ${KERNEL_CONFIG_TARGET} || exit ${?} -if [ -n "${KERNEL_FRAGMENTS}" ]; then - scripts/kconfig/merge_config.sh -m .config ${KERNEL_FRAGMENTS} -fi - -KV=$( make -s kernelrelease ) -if [ "${KV}" != "${PV}" ]; then - echo "ERROR: Recipe PV is \"${PV}\" but should be \"${KV}\"" 1>&2 - echo "ERROR: Probably due to CONFIG_LOCALVERSION" 1>&2 - exit 1 -fi - -rm -f .version -KBUILD_DEBARCH=$target_arch make -j $(($(nproc) * 2)) deb-pkg - -rm -rf "${REPACK_DIR}" -mkdir -p "${REPACK_DIR}" -mkdir -p "${REPACK_LINUX_IMAGE_DIR}" -mkdir -p "${REPACK_LINUX_HEADERS_DIR}" - -cp -a debian "${REPACK_DIR}" - -# dpkg-gencontrol performs cross-incompatible checks on the -# Architecture field; trick it to accept the control file -sed -i "s/Architecture: .*/Architecture: any/" "${REPACK_DIR}/debian/control" - -cd .. - -dpkg-deb -R linux-image-${PV}_${PV}-1_*.deb "${REPACK_LINUX_IMAGE_DIR}" -dpkg-deb -R linux-headers-${PV}_${PV}-1_*.deb "${REPACK_LINUX_HEADERS_DIR}" - -dpkg-gencontrol -crepack/debian/control \ - -lrepack/debian/changelog \ - -frepack/debian/files \ - -plinux-image-${PV} \ - -P"${REPACK_LINUX_IMAGE_DIR}" \ - -DPackage="linux-image-${KERNEL_NAME}" \ - -DSection=kernel \ - -DPriority=required \ - -DDepends="${KERNEL_DEBIAN_DEPENDS}" \ - -DArchitecture=$target_arch - -# Add Debian-like link installation to postinst -mkdir -p ${REPACK_LINUX_IMAGE_DIR}/lib/modules/${PV} -touch "${REPACK_LINUX_IMAGE_DIR}/lib/modules/${PV}/.fresh-install" -sed -i "${REPACK_LINUX_IMAGE_DIR}/DEBIAN/postinst" \ - -e "/^set -e$/a\\ -\\ -if [ -f /lib/modules/${PV}/.fresh-install ]; then\\ - change=install\\ -else\\ - change=upgrade\\ -fi\\ -linux-update-symlinks \$change ${PV} /boot/${KERNEL_FILE}-${PV}\\ -rm -f /lib/modules/${PV}/.fresh-install" - -# Add Debian-like link removal to postrm -sed -i "${REPACK_LINUX_IMAGE_DIR}/DEBIAN/postrm" \ - -e "/^set -e$/a\\ -\\ -rm -f /lib/modules/${PV}/.fresh-install\\ -\\ -if [ \"\$1\" != upgrade ] && command -v linux-update-symlinks >/dev/null; then\\ - linux-update-symlinks remove ${PV} /boot/${KERNEL_FILE}-${PV}\\ -fi" - -# Make sure arm64 kernels are decompressed -if [ "$target_arch" = "arm64" ]; then - kernel_file="${REPACK_LINUX_IMAGE_DIR}/boot/${KERNEL_FILE}-${PV}" - mv "${kernel_file}" "${kernel_file}.gz" - gunzip "${kernel_file}.gz" -fi - -dpkg-gencontrol -crepack/debian/control \ - -lrepack/debian/changelog \ - -frepack/debian/files \ - -plinux-headers-${PV} \ - -P"${REPACK_LINUX_HEADERS_DIR}" \ - -Vkernel:debarch="${KERNEL_NAME}" \ - -DPackage="linux-headers-${KERNEL_NAME}" \ - -DSection=kernel \ - -DDepends="${KERNEL_HEADERS_DEBIAN_DEPENDS}" \ - -DArchitecture=$target_arch - -fakeroot dpkg-deb -b "${REPACK_LINUX_IMAGE_DIR}" \ - linux-image-${KERNEL_NAME}_${PV}-1_${KERNEL_NAME}.deb -rm -f linux-image-${PV}_${PV}-1_*.deb -fakeroot dpkg-deb -b "${REPACK_LINUX_HEADERS_DIR}" \ - linux-headers-${KERNEL_NAME}_${PV}-1_${KERNEL_NAME}.deb -rm -f linux-headers-${PV}_${PV}-1_*.deb diff --git a/meta/recipes-kernel/linux/files/debian/compat b/meta/recipes-kernel/linux/files/debian/compat new file mode 100644 index 0000000..ec63514 --- /dev/null +++ b/meta/recipes-kernel/linux/files/debian/compat @@ -0,0 +1 @@ +9 diff --git a/meta/recipes-kernel/linux/files/debian/control.tmpl b/meta/recipes-kernel/linux/files/debian/control.tmpl new file mode 100644 index 0000000..d55096b --- /dev/null +++ b/meta/recipes-kernel/linux/files/debian/control.tmpl @@ -0,0 +1,37 @@ +Source: linux-${KERNEL_NAME_PROVIDED} +Section: kernel +Priority: optional +Maintainer: ${MAINTAINER} +Build-Depends: bc, kmod, cpio, ${KBUILD_DEPENDS} +Homepage: http://www.kernel.org/ + +Package: linux-image-${KERNEL_NAME_PROVIDED} +Architecture: any +Depends: ${KERNEL_DEBIAN_DEPENDS} +Description: ${KERNEL_NAME_PROVIDED} Linux kernel, version ${PV} + This package contains the Linux kernel, modules and corresponding other + files, version: ${PV}. + +Package: linux-headers-${KERNEL_NAME_PROVIDED} +Architecture: any +Depends: ${KERNEL_HEADERS_DEBIAN_DEPENDS}, ${perl:Depends}, ${shlib:Depends} +Description: ${KERNEL_NAME_PROVIDED} Linux kernel headers for ${PV} + This package provides kernel header files for ${PV} on ${DISTRO_ARCH} + . + This is useful for people who need to build external modules + +Package: linux-libc-dev +Build-Profiles: <!nolibcdev> +Section: devel +Provides: linux-kernel-headers +Architecture: any +Description: Linux support headers for userspace development + This package provides userspaces headers from the Linux kernel. These headers + are used by the installed headers for GNU glibc and other system libraries. + +Package: linux-image-${KERNEL_NAME_PROVIDED}-dbg +Section: debug +Architecture: any +Description: Linux kernel debugging symbols for ${PV} + This package will come in handy if you need to debug the kernel. It provides + all the necessary debug symbols for the kernel and its modules. diff --git a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl new file mode 100644 index 0000000..4ea18d5 --- /dev/null +++ b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl @@ -0,0 +1,45 @@ +#!/bin/bash +# Copyright (c) Mentor Graphics, a Siemens business, 2019 +# SPDX-License-Identifier: MIT + +# Load common stuff +. ${S}/debian/isar/common || exit ${?} + +do_build() { + + # Print a few things that are of particular interest + print_settings + + # Process existing kernel configuration to make sure it is complete + # (use defaults for options that were not specified) + ${MAKE} O=${KERNEL_BUILD_DIR} olddefconfig prepare || exit ${?} + + # Check if the recipe's PV makes sense + KR=$(${MAKE} O=${KERNEL_BUILD_DIR} -s --no-print-directory kernelrelease) + eval $(grep ^CONFIG_LOCALVERSION= ${KERNEL_BUILD_DIR}/${KCONF} || true) + if [ "${PV}" != "${KR}" -a "${PV}${CONFIG_LOCALVERSION}" != "${KR}" ]; then + echo "ERROR: Recipe PV (${PV}) does not seem to match the kernelrelease (${KR})!" 1>&2 + echo "ERROR: Make sure the kernel version in your PV setting and/or CONFIG_LOCALVERSION are aligned" 1>&2 + exit 1 + fi + + # Trace what we do here + set -x + + # Build the Linux kernel + ${MAKE} O=${KERNEL_BUILD_DIR} -j $(nproc) || exit ${?} + + # Stop tracing + set +x +} + +print_settings() { + cat <<EOF +# Build settings: +# --------------- +# ARCH=${ARCH} +# CROSS_COMPILE=${CROSS_COMPILE} +EOF +} + +main build ${*} diff --git a/meta/recipes-kernel/linux/files/debian/isar/clean.tmpl b/meta/recipes-kernel/linux/files/debian/isar/clean.tmpl new file mode 100644 index 0000000..f903987 --- /dev/null +++ b/meta/recipes-kernel/linux/files/debian/isar/clean.tmpl @@ -0,0 +1,21 @@ +#!/bin/bash +# Copyright (c) Mentor Graphics, a Siemens business, 2019 +# SPDX-License-Identifier: MIT + +# Load common stuff +. ${S}/debian/isar/common || exit ${?} + +do_clean() { + + # Trace what we do here + set -x + + rm -rf ${deb_img_dir} ${deb_dbg_dir} + rm -rf ${deb_kern_hdrdir} + rm -rf ${deb_libc_hdr_dir} + + # Stop tracing + set +x +} + +main clean ${*} diff --git a/meta/recipes-kernel/linux/files/debian/isar/common.tmpl b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl new file mode 100644 index 0000000..44f67b3 --- /dev/null +++ b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl @@ -0,0 +1,50 @@ +#!/bin/bash +# Copyright (c) Mentor Graphics, a Siemens business, 2019 +# SPDX-License-Identifier: MIT + +# Isar settings +ARCH=${KERNEL_ARCH} +KERNEL_PKG_IMAGE=linux-image-${KERNEL_NAME_PROVIDED} +KERNEL_PKG_KERN_HEADERS=linux-headers-${KERNEL_NAME_PROVIDED} +KERNEL_PKG_LIBC_HEADERS=linux-libc-dev + +# Constants +KCONF=.config + +# Target directories +deb_top_dir=${S}/debian +deb_img_dir=${deb_top_dir}/${KERNEL_PKG_IMAGE} +deb_dbg_dir=${deb_img_dir}-dbg +deb_dtb_dir=${deb_img_dir}/${KERNEL_DTB_DIR} +deb_kern_hdr_dir=${deb_top_dir}/${KERNEL_PKG_KERN_HEADERS} +deb_libc_hdr_dir=${deb_top_dir}/${KERNEL_PKG_LIBC_HEADERS} + +# Array of packages to be generated +declare -A kern_pkgs + +main() { + local target=${1} + + if [ ! -f ${S}/debian/isar/${target} ]; then + echo "error: ${target} is not a supported build target!" >&2 + return 1 + fi + + # create do_<x> variables for each package to be generated + for p in $(unset DEB_HOST_ARCH; dh_listpackages); do + kern_pkgs["${p}"]="1" + done + + # variables to be exported + export ARCH + + # are we cross-compiling? + BUILD_ARCH=$(dpkg-architecture -qDEB_BUILD_ARCH) + if [ "${BUILD_ARCH}" = "${DISTRO_ARCH}" ]; then + # no, make sure CROSS_COMPILE isn't set + unset CROSS_COMPILE + fi + + # call the actual target script + do_${target} || return ${?} +} diff --git a/meta/recipes-kernel/linux/files/debian/isar/install.tmpl b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl new file mode 100644 index 0000000..8ccda68 --- /dev/null +++ b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl @@ -0,0 +1,206 @@ +#!/bin/bash +# Copyright (c) Mentor Graphics, a Siemens business, 2019 +# SPDX-License-Identifier: MIT + +# Load common stuff +. ${S}/debian/isar/common || exit ${?} + +# Stop on error +set -e + +do_install() { + + # check if our kernel was configured + if [ ! -f "${O}/.config" ]; then + echo "error: kernel not configured!" >&2 + return 1 + fi + + # load its configuration + . ${O}/.config + + kimage="$(${MAKE} O=${O} -s --no-print-directory image_name)" + krel="$(${MAKE} O=${O} -s --no-print-directory kernelrelease)" + case "${ARCH}" in + mips|powerpc) kimage_path="boot/vmlinux-${krel}" ;; + um) kimage_path="usr/bin/vmlinux-${krel}" ;; + *) kimage_path="boot/vmlinuz-${krel}" ;; + esac + + print_settings + + # Trace what we do here + set -x + + install_image + install_hooks + install_dtbs + install_kmods + install_headers + + # Stop tracing + set +x +} + +print_settings() { + cat <<EOF +Install settings: +----------------- +deb_dtb_dir=${deb_dtb_dir} +deb_hdr_dir=${deb_hdr_dir} +kimage=${kimage} +kimage_path=${kimage_path} + +EOF +} + +install_image() { + install -m 755 -d ${deb_img_dir}/$(dirname ${kimage_path}) + cp ${O}/${kimage} ${deb_img_dir}/${kimage_path} + install_image_debug +} + +install_image_debug() { + # Different tools want the image in different locations + # perf + mkdir -p ${deb_dbg_dir}/usr/lib/debug/lib/modules/${krel}/ + cp ${O}/vmlinux ${deb_dbg_dir}/usr/lib/debug/lib/modules/${krel}/ + # systemtap + mkdir -p ${deb_dbg_dir}/usr/lib/debug/boot/ + ln -s ../lib/modules/$version/vmlinux ${deb_dbg_dir}/usr/lib/debug/boot/vmlinux-${krel} + # kdump-tools + ln -s lib/modules/${krel}/vmlinux ${deb_dbg_dir}/usr/lib/debug/vmlinux-${krel} +} + +install_hooks() { + install -m 755 -d ${deb_img_dir}/etc/kernel/install.d + install -m 755 -d ${deb_img_dir}/etc/kernel/postinst.d + install -m 755 -d ${deb_img_dir}/etc/kernel/postrm.d + install -m 755 -d ${deb_img_dir}/etc/kernel/prerm.d + + initrd="No" + [ -z "${CONFIG_BLK_DEV_INITRD}" ] || initrd="Yes" + + for script in postinst postrm preinst prerm; do + sed -i -e "s,INITRD=[A-Za-z0-9]*,INITRD=${initrd},g" ${S}/debian/linux-image*.${script} + sed -i -e "s,version=.*,version=${krel},g" ${S}/debian/linux-image*.${script} + done +} + +install_dtbs() { + [ -n "${CONFIG_OF}" ] || return 0 + ${MAKE} O=${O} INSTALL_DTBS_PATH=${deb_dtb_dir} dtbs_install +} + +install_kmods() { + [ -n "${CONFIG_MODULES}" ] || return 0 + ${MAKE} O=${O} INSTALL_MOD_PATH=${deb_img_dir} modules_install + touch ${deb_img_dir}/lib/modules/${krel}/.fresh-install + rm -fv ${deb_img_dir}/lib/modules/${krel}/build + rm -fv ${deb_img_dir}/lib/modules/${krel}/source + install_kmods_debug +} + +kmods_sign() { + [ -n "${CONFIG_MODULE_SIG_ALL}" ] || return 0 + ${MAKE} O=${O} INSTALL_MOD_PATH=${deb_img_dir} modules_sign +} + +install_kmods_debug() { + [ -n "${CONFIG_DEBUG_INFO}" ] || return 0 + + kmod_inst_dir=${deb_img_dir}/lib/modules + kmod_debug_dir=${deb_dbg_dir}/usr/lib/debug + + # copy kernels modules to usr/lib/debug + mkdir -p ${kmod_debug_dir} + tar -C ${kmod_inst_dir}/ -cO --exclude='modules.*' . | tar -C ${kmod_debug_dir}/ -xf - + # strip everything but debug sections for modules in usr/lib/debug + find ${kmod_debug_dir} -name *.ko -exec ${CROSS_COMPILE}objcopy --only-keep-debug {} \; + # and strip debug sections from modules in lib/modules + find ${kmod_inst_dir} -name *.ko -exec ${CROSS_COMPILE}objcopy --strip-debug {} \; + + # re-sign stripped kernel modules + kmods_sign +} + +headers_check() { + ${MAKE} O=${O} headers_check +} + +libc_headers() { + mkdir -p ${deb_libc_hdr_dir} + ${MAKE} O=${O} headers_install INSTALL_HDR_PATH=${deb_libc_hdr_dir}/usr + host_arch=$(dpkg-architecture -a${DISTRO_ARCH} -qDEB_HOST_MULTIARCH) + mkdir ${deb_libc_hdr_dir}/usr/include/${host_arch} + mv ${deb_libc_hdr_dir}/usr/include/asm ${deb_libc_hdr_dir}/usr/include/${host_arch}/ +} + +# the kernel does not seem to provide a way to cross-compile its tools, we however +# have qemu-user-static binaries in our buildchroot. Create a new build tree for the +# scripts and override HOSTCC to force a cross-compile +kernel_tools_build() { + odir="${S}/build-scripts" + mflags="HOSTCC=${CROSS_COMPILE}gcc O=${odir} NOSTDINC_FLAGS=" + rm -rf ${odir} && mkdir -p ${odir} && cp ${S}/${KERNEL_BUILD_DIR}/.config ${odir} + (unset LD_PRELOAD; cd ${odir} && make -C ${S} ${mflags} olddefconfig scripts) + (cd ${odir}; find scripts -type f -executable -exec file {} \;|grep ELF|cut -d: -f1) >${src_hdr_files} + tar -C ${odir} -cf - -T - <${src_hdr_files} | tar -C ${destdir} -xf - +} + +kernel_tools() { + # remove object files + find ${destdir}/scripts -type f -name '*.o' |xargs rm -f + + # we're all done if we aren't cross-compiling + [ -n "${CROSS_COMPILE}" ] || return 0 + + # remove ELF executables from the linux-headers package + find ${destdir}/scripts -type f -exec file {} \;|grep ELF|cut -d: -f1|xargs rm -fv + + # cross-compile kernel tools to be shipped with linux-headers + kernel_tools_build +} + +kernel_headers() { + destdir=${deb_kern_hdr_dir}/${KERNEL_HEADERS_DIR} + src_hdr_files=$(mktemp) + obj_hdr_files=$(mktemp) + + mkdir -p ${destdir} + mkdir -p ${deb_kern_hdr_dir}/lib/modules/${krel} + + (cd ${S}; find . -name 'Makefile*' -o -name 'Kconfig*' -o -name '*.pl') >>${src_hdr_files} + (cd ${S}; find arch/*/include include scripts -type f -o -type l) >>${src_hdr_files} + (cd ${S}; find arch/${ARCH} -name module.lds -o -name Kbuild.platforms -o -name Platform) >>${src_hdr_files} + (cd ${S}; find $(find arch/${ARCH} -name include -o -name scripts -type d) -type f) >>${src_hdr_files} + + (cd ${O}; find arch/${ARCH}/include Module.symvers include scripts -type f) >>${obj_hdr_files} + if [ -n "${CONFIG_STACK_VALIDATION}" ]; then + (cd ${O}; find tools/objtool -type f -executable) >>${obj_hdr_files} + fi + if [ -n "${CONFIG_GCC_PLUGINS}" ]; then + (cd ${O}; find scripts/gcc-plugins -name *.so -o -name gcc-common.h) >>${obj_hdr_files} + fi + + # deploy files that were matched above + tar -C ${S} -cf - -T - <${src_hdr_files} | tar -C ${destdir} -xf - + tar -C ${O} -cf - -T - <${obj_hdr_files} | tar -C ${destdir} -xf - + + # add the kernel config + cp ${O}/${KCONF} ${destdir}/.config + + # handle kernel development tools + kernel_tools + + # create symlinks + ln -sf /${KERNEL_HEADERS_DIR} ${deb_kern_hdr_dir}/lib/modules/${krel}/build +} + +install_headers() { + headers_check + [ -z ${kern_pkgs["linux-libc-headers"]} ] || libc_headers + kernel_headers +} + +main install ${*} diff --git a/meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl b/meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl new file mode 100644 index 0000000..8e8636a --- /dev/null +++ b/meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl @@ -0,0 +1,29 @@ +#!/bin/sh +# based on https://salsa.debian.org/kernel-team/linux/blob/master/debian/templates/image.postinst.in + +# Tell initramfs builder whether it's wanted +export INITRD=Yes + +version=${PV} +image_path=/boot/${KERNEL_FILE}-${version} + +if [ "$1" != configure ]; then + exit 0 +fi + +depmod $version + +if [ -f /lib/modules/$version/.fresh-install ]; then + change=install +else + change=upgrade +fi +linux-update-symlinks $change $version $image_path +rm -f /lib/modules/$version/.fresh-install + +if [ -d /etc/kernel/postinst.d ]; then + DEB_MAINT_PARAMS="$*" run-parts --report --exit-on-error --arg=$version \ + --arg=$image_path /etc/kernel/postinst.d +fi + +exit 0 diff --git a/meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl b/meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl new file mode 100644 index 0000000..4d8a7d9 --- /dev/null +++ b/meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl @@ -0,0 +1,35 @@ +#!/bin/sh -e +# based on https://salsa.debian.org/kernel-team/linux/blob/master/debian/templates/image.postrm.in + +# Tell initramfs builder whether it's wanted +export INITRD=Yes + +version=${PV} +image_path=/boot/${KERNEL_FILE}-${version} + +rm -f /lib/modules/$version/.fresh-install + +if [ "$1" != upgrade ] && command -v linux-update-symlinks >/dev/null; then + linux-update-symlinks remove $version $image_path +fi + +if [ -d /etc/kernel/postrm.d ]; then + DEB_MAINT_PARAMS="$*" run-parts --report --exit-on-error --arg=$version \ + --arg=$image_path /etc/kernel/postrm.d +fi + +if [ "$1" = purge ]; then + for extra_file in modules.dep modules.isapnpmap modules.pcimap \ + modules.usbmap modules.parportmap \ + modules.generic_string modules.ieee1394map \ + modules.ieee1394map modules.pnpbiosmap \ + modules.alias modules.ccwmap modules.inputmap \ + modules.symbols modules.ofmap \ + modules.seriomap modules.\*.bin \ + modules.softdep modules.devname; do + eval rm -f /lib/modules/$version/$extra_file + done + rmdir /lib/modules/$version || true +fi + +exit 0 diff --git a/meta/recipes-kernel/linux/files/debian/linux-image.preinst.tmpl b/meta/recipes-kernel/linux/files/debian/linux-image.preinst.tmpl new file mode 100644 index 0000000..7465197 --- /dev/null +++ b/meta/recipes-kernel/linux/files/debian/linux-image.preinst.tmpl @@ -0,0 +1,25 @@ +#!/bin/sh -e +# based on https://salsa.debian.org/kernel-team/linux/blob/master/debian/templates/image.preinst.in + +# Tell initramfs builder whether it's wanted +export INITRD=Yes + +version=${PV} +image_path=/boot/${KERNEL_FILE}-${version} + +if [ "$1" = abort-upgrade ]; then + exit 0 +fi + +if [ "$1" = install ]; then + # Create a flag file for postinst + mkdir -p /lib/modules/$version + touch /lib/modules/$version/.fresh-install +fi + +if [ -d /etc/kernel/preinst.d ]; then + DEB_MAINT_PARAMS="$*" run-parts --report --exit-on-error --arg=$version \ + --arg=$image_path /etc/kernel/preinst.d +fi + +exit 0 diff --git a/meta/recipes-kernel/linux/files/debian/linux-image.prerm.tmpl b/meta/recipes-kernel/linux/files/debian/linux-image.prerm.tmpl new file mode 100644 index 0000000..1992b32 --- /dev/null +++ b/meta/recipes-kernel/linux/files/debian/linux-image.prerm.tmpl @@ -0,0 +1,21 @@ +#!/bin/sh -e +# based on https://salsa.debian.org/kernel-team/linux/blob/master/debian/templates/image.prerm.in + +# Tell initramfs builder whether it's wanted +export INITRD=Yes + +version=${PV} +image_path=/boot/${KERNEL_FILE}-${version} + +if [ "$1" != remove ]; then + exit 0 +fi + +linux-check-removal $version + +if [ -d /etc/kernel/prerm.d ]; then + DEB_MAINT_PARAMS="$*" run-parts --report --exit-on-error --arg=$version \ + --arg=$image_path /etc/kernel/prerm.d +fi + +exit 0 diff --git a/meta/recipes-kernel/linux/files/debian/rules.tmpl b/meta/recipes-kernel/linux/files/debian/rules.tmpl new file mode 100755 index 0000000..c024768 --- /dev/null +++ b/meta/recipes-kernel/linux/files/debian/rules.tmpl @@ -0,0 +1,39 @@ +#!/usr/bin/make -f + +CROSS_COMPILE:=$(DEB_HOST_GNU_TYPE)- + +O:=$(CURDIR)/${KERNEL_BUILD_DIR} +S:=$(CURDIR) +deb_top_dir:=$(S)/debian + +# Dynamic variables to be passed to Isar build scripts +isar_env=$(strip \ + export CROSS_COMPILE='$(CROSS_COMPILE)' && \ + export DEB_BUILD_PROFILES='$(DEB_BUILD_PROFILES)' && \ + export DEB_HOST_GNU_TYPE='$(DEB_HOST_GNU_TYPE)' && \ + export MAKE='$(MAKE)' && \ + export O='${O}' \ + export S='${S}' \ +) + +%: + dh $(@) + +.PHONY: override_dh_auto_clean +override_dh_auto_clean: + $(isar_env) && bash $(deb_top_dir)/isar/clean + +.PHONY: override_dh_auto_build +override_dh_auto_build: + $(isar_env) && bash $(deb_top_dir)/isar/build + +override_dh_auto_install: + $(isar_env) && bash $(deb_top_dir)/isar/install + +.PHONY: override_dh_auto_test +override_dh_auto_test: + true + +PHONY: override_dh_strip +override_dh_strip: + dh_strip -Xvmlinux --no-automatic-dbgsym diff --git a/meta/recipes-kernel/linux/linux-custom.inc b/meta/recipes-kernel/linux/linux-custom.inc index ca91f64..ecf0e66 100644 --- a/meta/recipes-kernel/linux/linux-custom.inc +++ b/meta/recipes-kernel/linux/linux-custom.inc @@ -2,22 +2,107 @@ # # This software is a part of ISAR. # Copyright (c) Siemens AG, 2018 +# Copyright (c) Mentor Graphics, a Siemens business, 2019 # # SPDX-License-Identifier: MIT -FILESEXTRAPATHS_prepend := "${FILE_DIRNAME}/files:" +# Settings that would typically be done from the custom kernel recipe +# ------------------------------------------------------------------- DESCRIPTION ?= "Custom kernel" +MAINTAINER ?= "isar-users <isar-users@googlegroups.com>" + +KBUILD_DEPENDS ?= "build-essential:native, \ + libelf-dev:native, \ + libncurses-dev:native, \ + libssl-dev:native, \ + bc, \ + bison, \ + cpio, \ + flex, \ + git, \ + kmod, \ + libssl-dev," + +KERNEL_DEBIAN_DEPENDS ?= "initramfs-tools | linux-initramfs-tool, \ + kmod, \ + linux-base (>= 4.3~)," + +KERNEL_HEADERS_DEBIAN_DEPENDS ?= "libc6, \ + libssl1.1," + +KERNEL_LIBC_DEV_DEPLOY ?= "0" + +# Settings that may be changed on a per distro, machine or layer basis +# -------------------------------------------------------------------- + +KERNEL_DTB_DIR ?= "usr/lib/linux-${KERNEL_NAME_PROVIDED}-${PV}" +KERNEL_HEADERS_DIR ?= "usr/src/linux-headers-${PV}" -KERNEL_NAME_PROVIDED ?= "${@ d.getVar('PN', True).partition('linux-')[2]}" KERNEL_DEFCONFIG ?= "" +# Add our template meta-data to the sources +FILESEXTRAPATHS_prepend := "${FILE_DIRNAME}/files:" +SRC_URI += "file://debian" + +# Variables and files that make our templates +# ------------------------------------------- + +TEMPLATE_FILES += " \ + debian/control.tmpl \ + debian/isar/build.tmpl \ + debian/isar/clean.tmpl \ + debian/isar/common.tmpl \ + debian/isar/install.tmpl \ + debian/linux-image.postinst.tmpl \ + debian/linux-image.postrm.tmpl \ + debian/linux-image.preinst.tmpl \ + debian/linux-image.prerm.tmpl \ + debian/rules.tmpl \ +" + +TEMPLATE_VARS += " \ + HOST_ARCH \ + KBUILD_DEPENDS \ + KERNEL_ARCH \ + KERNEL_DEBIAN_DEPENDS \ + KERNEL_BUILD_DIR \ + KERNEL_DTB_DIR \ + KERNEL_FILE \ + KERNEL_HEADERS_DEBIAN_DEPENDS \ + KERNEL_HEADERS_DIR \ + KERNEL_NAME_PROVIDED \ +" + +inherit dpkg +inherit template + +# Derive name of the kernel packages from the name of this recipe +KERNEL_NAME_PROVIDED ?= "${@ d.getVar('PN', True).partition('linux-')[2]}" + +# Make bitbake know we will be producing linux-image and linux-headers packages python() { kernel_name = d.getVar("KERNEL_NAME_PROVIDED", True) d.setVar('PROVIDES', 'linux-image-' + kernel_name + ' ' + \ 'linux-headers-' + kernel_name) } +def get_kernel_arch(d): + distro_arch = d.getVar("DISTRO_ARCH") + if distro_arch == "amd64": + kernel_arch = "x86" + elif distro_arch == "arm64": + kernel_arch = "arm64" + elif distro_arch == "armhf": + kernel_arch = "arm" + elif distro_arch == "mipsel": + kernel_arch = "mips" + else: + kernel_arch = "" + return kernel_arch + +KERNEL_ARCH ??= "${@get_kernel_arch(d)}" + def config_fragments(d): fragments = [] sources = d.getVar("SRC_URI").split() @@ -31,57 +116,37 @@ def config_fragments(d): fragments.append(local) return fragments -inherit dpkg-base - -SRC_URI += "file://build-kernel.sh" +do_prepare_build_prepend() { + # copy meta-data over to source tree + rm -rf ${S}/debian + cp -r ${WORKDIR}/debian ${S}/ -KBUILD_DEPENDS ?= " \ - build-essential:native \ - libssl-dev \ - libelf-dev \ - bc \ - git \ - kmod \ - bison \ - flex \ - cpio \ - libncurses-dev" -KERNEL_DEBIAN_DEPENDS ?= "initramfs-tools | linux-initramfs-tool, kmod, linux-base (>= 4.3~)" -KERNEL_HEADERS_DEBIAN_DEPENDS ?= "libc6, libssl1.1" + # remove templates from the source tree + find ${S}/debian -name *.tmpl | xargs rm -f -KERNEL_LIBC_DEV_DEPLOY ?= "0" + # rename install/remove hooks to match user-specified name for our linux-image package + mv ${S}/debian/linux-image.postinst ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.postinst + mv ${S}/debian/linux-image.postrm ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.postrm + mv ${S}/debian/linux-image.preinst ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.preinst + mv ${S}/debian/linux-image.prerm ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.prerm -do_install_builddeps() { - dpkg_do_mounts - E="${@ bb.utils.export_proxies(d)}" - sudo -E chroot ${BUILDCHROOT_DIR} \ - apt-get update \ - -o Dir::Etc::SourceList="sources.list.d/isar-apt.list" \ - -o Dir::Etc::SourceParts="-" \ - -o APT::Get::List-Cleanup="0" - sudo -E chroot ${BUILDCHROOT_DIR} \ - apt-get install \ - -y -o Debug::pkgProblemResolver=yes \ - --no-install-recommends ${KBUILD_DEPENDS} - dpkg_undo_mounts + # produce a changelog for our kernel build + deb_add_changelog } -addtask install_builddeps after do_prepare_build before do_dpkg_build -# apt and reprepro may not run in parallel, acquire the Isar lock -do_install_builddeps[lockfiles] += "${REPO_ISAR_DIR}/isar.lock" - -addtask devshell after do_install_builddeps +# build directory for our "full" kernel build +KERNEL_BUILD_DIR = "build-full" -dpkg_runbuild() { - chmod +x ${WORKDIR}/build-kernel.sh - KERNEL_CONFIG_TARGET="${KERNEL_DEFCONFIG}" +dpkg_configure_kernel() { + config_target="${KERNEL_DEFCONFIG}" + rm -rf ${S}/${KERNEL_BUILD_DIR} && mkdir -p ${S}/${KERNEL_BUILD_DIR} if [ -n "${KERNEL_DEFCONFIG}" ]; then if [ -e "${WORKDIR}/${KERNEL_DEFCONFIG}" ]; then - cp ${WORKDIR}/${KERNEL_DEFCONFIG} ${S}/.config - KERNEL_CONFIG_TARGET="olddefconfig" + cp ${WORKDIR}/${KERNEL_DEFCONFIG} ${S}/${KERNEL_BUILD_DIR}/.config + config_target="olddefconfig" fi else - KERNEL_CONFIG_TARGET="defconfig" + config_target="defconfig" fi # copy config fragments over to the kernel tree @@ -92,22 +157,23 @@ dpkg_runbuild() { mkdir -p ${S}/debian/fragments (cd ${WORKDIR} && cp ${src_frags} ${S}/debian/fragments/) fi - export KERNEL_FRAGMENTS="${out_frags}" - E="${@ bb.utils.export_proxies(d)}" - - export PV=${PV} - export KERNEL_NAME=${KERNEL_NAME_PROVIDED} - - export KBUILD_DEPENDS="${KBUILD_DEPENDS}" - export KERNEL_CONFIG_TARGET - export KERNEL_DEBIAN_DEPENDS="${KERNEL_DEBIAN_DEPENDS}" - export KERNEL_FILE="${KERNEL_FILE}" - export KERNEL_HEADERS_DEBIAN_DEPENDS="${KERNEL_HEADERS_DEBIAN_DEPENDS}" - - sudo -E chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} ${PP}/build-kernel.sh ${PP}/${PPS} ${DISTRO_ARCH} + sudo -E chroot --userspec=$(id -u):$(id -g) ${BUILDCHROOT_DIR} sh -c " \ + export ARCH=${KERNEL_ARCH} && \ + cd ${PP}/${PPS} && \ + make O=${KERNEL_BUILD_DIR} ${config_target} && \ + ./scripts/kconfig/merge_config.sh \ + -O ${KERNEL_BUILD_DIR}/ \ + ${KERNEL_BUILD_DIR}/.config \ + ${out_frags} \ + " +} +dpkg_runbuild_prepend() { + profiles="${BASE_DISTRO}" if [ "${KERNEL_LIBC_DEV_DEPLOY}" != "1" ]; then - rm -f ${WORKDIR}/linux-libc-dev_${PV}*.deb + profiles="${profiles} nolibcdev" fi + export DEB_BUILD_PROFILES="${profiles}" + dpkg_configure_kernel } -- 2.20.1 ^ permalink raw reply [flat|nested] 83+ messages in thread
* Re: [PATCH L-C v3 7/7] linux-custom: rewrite to no longer depend on the kernel's builddeb 2019-11-08 7:07 ` [PATCH L-C v3 7/7] linux-custom: rewrite to no longer depend on the kernel's builddeb Cedric Hombourger @ 2019-11-08 7:19 ` Jan Kiszka 2019-11-08 7:36 ` Cedric Hombourger 0 siblings, 1 reply; 83+ messages in thread From: Jan Kiszka @ 2019-11-08 7:19 UTC (permalink / raw) To: Cedric Hombourger, isar-users On 08.11.19 08:07, Cedric Hombourger wrote: > Regain control over the packaging of the Linux kernel by providing our > own debian recipes (debian,{control,rules} files and friends) instead > of using the kernel's builddeb script. This will allow generation of > packages for "perf" (not included in this changeset) with per distro > scheme (Debian and Ubuntu have differences) and maintain compatibility > with upstream packages generated from different source packages but > dependent on a certain deployment scheme (e.g. linux-base for Debian) > > Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com> > --- > doc/custom_kernel.md | 80 +++++++ > .../linux/files/build-kernel.sh | 128 ----------- > meta/recipes-kernel/linux/files/debian/compat | 1 + > .../linux/files/debian/control.tmpl | 37 ++++ > .../linux/files/debian/isar/build.tmpl | 45 ++++ > .../linux/files/debian/isar/clean.tmpl | 21 ++ > .../linux/files/debian/isar/common.tmpl | 50 +++++ > .../linux/files/debian/isar/install.tmpl | 206 ++++++++++++++++++ > .../files/debian/linux-image.postinst.tmpl | 29 +++ > .../files/debian/linux-image.postrm.tmpl | 35 +++ > .../files/debian/linux-image.preinst.tmpl | 25 +++ > .../linux/files/debian/linux-image.prerm.tmpl | 21 ++ > .../linux/files/debian/rules.tmpl | 39 ++++ > meta/recipes-kernel/linux/linux-custom.inc | 180 ++++++++++----- > 14 files changed, 712 insertions(+), 185 deletions(-) > create mode 100644 doc/custom_kernel.md > delete mode 100644 meta/recipes-kernel/linux/files/build-kernel.sh > create mode 100644 meta/recipes-kernel/linux/files/debian/compat > create mode 100644 meta/recipes-kernel/linux/files/debian/control.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/isar/build.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/isar/clean.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/isar/common.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/isar/install.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.preinst.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.prerm.tmpl > create mode 100755 meta/recipes-kernel/linux/files/debian/rules.tmpl > > diff --git a/doc/custom_kernel.md b/doc/custom_kernel.md > new file mode 100644 > index 0000000..31319d4 > --- /dev/null > +++ b/doc/custom_kernel.md > @@ -0,0 +1,80 @@ > +# Custom kernel recipe for Isar > + > +## Contents > + > + - [Summary](#summary) > + - [Features](#features) > + - [Future](#future) > + - [Examples](#examples) > + > +## Summary > + > +Isar provides a recipe to build custom kernels for Debian-based distributions. > +It uses templates to generate the debian meta-data (such as debian/control) and > +Debian's [BuildProfiles](https://wiki.debian.org/BuildProfileSpec) to handle > +some of the distro specific variations. It should be noted that Isar has moved > +away from using the kernel's builddeb script since it would not generate all > +the packages we need (and in particular perf). > + > +## Features > + > +The linux-custom recipe provides support for: > + > + 1. Sources to the custom Linux kernel may be specified via `SRC_URI` > + > + 2. Configure the kernel via an in-tree or an external `defconfig` via > + `KERNEL_DEFCONFIG` > + > + 3. Integrate kernel configuration tweaks via configuration fragments (`.cfg` > + files) > + > + 4. Patches to the linux kernel may be specified via `SRC_URI` > + > + 5. Ensure that the Isar recipe `PV` matches the kernel release identifier > + (`KERNEL_RELEASE`) > + > + 6. Produce a `linux-image` package that ships the kernel image and modules > + > + 7. Allow the name of the kernel image to be changed via `KERNEL_FILE` (defaults > + to `vmlinuz`) > + > + 8. Produce a `linux-headers` package which includes kernel headers and kbuild > + scripts/tools > + > + 9. The `linux-headers` package shall support native and cross compiles of > + out-of-tree kernel modules > + > + 10. Produce a `linux-libc-dev` package to support user-land builds > + > + 11. Only build/ship the `linux-libc-dev` package if instructed to > + (`KERNEL_LIBC_DEV_DEPLOY` equals to `"1"`) > + > + 12. Support both native and cross compiles (`ISAR_CROSS_COMPILE`) > + > + 13. Support for the following kernel architectures: > + > + * arm > + * arm64 > + * mips > + * x86 > + > + 14. Support `devshell` (kernel configuration shall be applied) > + > +## Future > + > +In the future, the recipe may be extended to: > + > + 1. Package perf > + > + 2. Support inclusion/build of dts files listed in `SRC_URI` > + > + 3. Be compatible with Ubuntu > + > +## Examples > + > +The linux-custom recipe is currently used by the linux-mainline package and is > +used mainline recipe may be used for some basic testing. This recipe is being > +used by the following machines: > + > + * de0-nano-soc > + * qemumipsel > diff --git a/meta/recipes-kernel/linux/files/build-kernel.sh b/meta/recipes-kernel/linux/files/build-kernel.sh > deleted file mode 100644 > index f56e96c..0000000 > --- a/meta/recipes-kernel/linux/files/build-kernel.sh > +++ /dev/null > @@ -1,128 +0,0 @@ > -#!/bin/bash > -# > -# Custom kernel build > -# > -# This software is a part of ISAR. > -# Copyright (c) Siemens AG, 2018 > -# > -# SPDX-License-Identifier: MIT > - > -source /isar/common.sh > - > -host_arch=$(dpkg --print-architecture) > - > -if [ "$host_arch" != "$target_arch" ]; then > - case $target_arch in > - armhf) > - export ARCH=arm > - export CROSS_COMPILE="arm-linux-gnueabihf-" > - ;; > - arm64) > - export ARCH=arm64 > - export CROSS_COMPILE="aarch64-linux-gnu-" > - ;; > - mipsel) > - export ARCH=mips > - export CROSS_COMPILE="mipsel-linux-gnu-" > - ;; > - *) > - echo "error: unsupported architecture ($target_arch)" > - exit 1 > - ;; > - esac > -fi > - > -REPACK_DIR="$1/../repack" > -REPACK_LINUX_IMAGE_DIR="${REPACK_DIR}/linux-image" > -REPACK_LINUX_HEADERS_DIR="${REPACK_DIR}/linux-headers" > - > -make ${KERNEL_CONFIG_TARGET} || exit ${?} > -if [ -n "${KERNEL_FRAGMENTS}" ]; then > - scripts/kconfig/merge_config.sh -m .config ${KERNEL_FRAGMENTS} > -fi > - > -KV=$( make -s kernelrelease ) > -if [ "${KV}" != "${PV}" ]; then > - echo "ERROR: Recipe PV is \"${PV}\" but should be \"${KV}\"" 1>&2 > - echo "ERROR: Probably due to CONFIG_LOCALVERSION" 1>&2 > - exit 1 > -fi > - > -rm -f .version > -KBUILD_DEBARCH=$target_arch make -j $(($(nproc) * 2)) deb-pkg > - > -rm -rf "${REPACK_DIR}" > -mkdir -p "${REPACK_DIR}" > -mkdir -p "${REPACK_LINUX_IMAGE_DIR}" > -mkdir -p "${REPACK_LINUX_HEADERS_DIR}" > - > -cp -a debian "${REPACK_DIR}" > - > -# dpkg-gencontrol performs cross-incompatible checks on the > -# Architecture field; trick it to accept the control file > -sed -i "s/Architecture: .*/Architecture: any/" "${REPACK_DIR}/debian/control" > - > -cd .. > - > -dpkg-deb -R linux-image-${PV}_${PV}-1_*.deb "${REPACK_LINUX_IMAGE_DIR}" > -dpkg-deb -R linux-headers-${PV}_${PV}-1_*.deb "${REPACK_LINUX_HEADERS_DIR}" > - > -dpkg-gencontrol -crepack/debian/control \ > - -lrepack/debian/changelog \ > - -frepack/debian/files \ > - -plinux-image-${PV} \ > - -P"${REPACK_LINUX_IMAGE_DIR}" \ > - -DPackage="linux-image-${KERNEL_NAME}" \ > - -DSection=kernel \ > - -DPriority=required \ > - -DDepends="${KERNEL_DEBIAN_DEPENDS}" \ > - -DArchitecture=$target_arch > - > -# Add Debian-like link installation to postinst > -mkdir -p ${REPACK_LINUX_IMAGE_DIR}/lib/modules/${PV} > -touch "${REPACK_LINUX_IMAGE_DIR}/lib/modules/${PV}/.fresh-install" > -sed -i "${REPACK_LINUX_IMAGE_DIR}/DEBIAN/postinst" \ > - -e "/^set -e$/a\\ > -\\ > -if [ -f /lib/modules/${PV}/.fresh-install ]; then\\ > - change=install\\ > -else\\ > - change=upgrade\\ > -fi\\ > -linux-update-symlinks \$change ${PV} /boot/${KERNEL_FILE}-${PV}\\ > -rm -f /lib/modules/${PV}/.fresh-install" > - > -# Add Debian-like link removal to postrm > -sed -i "${REPACK_LINUX_IMAGE_DIR}/DEBIAN/postrm" \ > - -e "/^set -e$/a\\ > -\\ > -rm -f /lib/modules/${PV}/.fresh-install\\ > -\\ > -if [ \"\$1\" != upgrade ] && command -v linux-update-symlinks >/dev/null; then\\ > - linux-update-symlinks remove ${PV} /boot/${KERNEL_FILE}-${PV}\\ > -fi" > - > -# Make sure arm64 kernels are decompressed > -if [ "$target_arch" = "arm64" ]; then > - kernel_file="${REPACK_LINUX_IMAGE_DIR}/boot/${KERNEL_FILE}-${PV}" > - mv "${kernel_file}" "${kernel_file}.gz" > - gunzip "${kernel_file}.gz" > -fi > - > -dpkg-gencontrol -crepack/debian/control \ > - -lrepack/debian/changelog \ > - -frepack/debian/files \ > - -plinux-headers-${PV} \ > - -P"${REPACK_LINUX_HEADERS_DIR}" \ > - -Vkernel:debarch="${KERNEL_NAME}" \ > - -DPackage="linux-headers-${KERNEL_NAME}" \ > - -DSection=kernel \ > - -DDepends="${KERNEL_HEADERS_DEBIAN_DEPENDS}" \ > - -DArchitecture=$target_arch > - > -fakeroot dpkg-deb -b "${REPACK_LINUX_IMAGE_DIR}" \ > - linux-image-${KERNEL_NAME}_${PV}-1_${KERNEL_NAME}.deb > -rm -f linux-image-${PV}_${PV}-1_*.deb > -fakeroot dpkg-deb -b "${REPACK_LINUX_HEADERS_DIR}" \ > - linux-headers-${KERNEL_NAME}_${PV}-1_${KERNEL_NAME}.deb > -rm -f linux-headers-${PV}_${PV}-1_*.deb > diff --git a/meta/recipes-kernel/linux/files/debian/compat b/meta/recipes-kernel/linux/files/debian/compat > new file mode 100644 > index 0000000..ec63514 > --- /dev/null > +++ b/meta/recipes-kernel/linux/files/debian/compat > @@ -0,0 +1 @@ > +9 > diff --git a/meta/recipes-kernel/linux/files/debian/control.tmpl b/meta/recipes-kernel/linux/files/debian/control.tmpl > new file mode 100644 > index 0000000..d55096b > --- /dev/null > +++ b/meta/recipes-kernel/linux/files/debian/control.tmpl > @@ -0,0 +1,37 @@ > +Source: linux-${KERNEL_NAME_PROVIDED} > +Section: kernel > +Priority: optional > +Maintainer: ${MAINTAINER} > +Build-Depends: bc, kmod, cpio, ${KBUILD_DEPENDS} > +Homepage: http://www.kernel.org/ > + > +Package: linux-image-${KERNEL_NAME_PROVIDED} > +Architecture: any > +Depends: ${KERNEL_DEBIAN_DEPENDS} > +Description: ${KERNEL_NAME_PROVIDED} Linux kernel, version ${PV} > + This package contains the Linux kernel, modules and corresponding other > + files, version: ${PV}. > + > +Package: linux-headers-${KERNEL_NAME_PROVIDED} > +Architecture: any > +Depends: ${KERNEL_HEADERS_DEBIAN_DEPENDS}, ${perl:Depends}, ${shlib:Depends} > +Description: ${KERNEL_NAME_PROVIDED} Linux kernel headers for ${PV} > + This package provides kernel header files for ${PV} on ${DISTRO_ARCH} > + . > + This is useful for people who need to build external modules > + > +Package: linux-libc-dev > +Build-Profiles: <!nolibcdev> > +Section: devel > +Provides: linux-kernel-headers > +Architecture: any > +Description: Linux support headers for userspace development > + This package provides userspaces headers from the Linux kernel. These headers > + are used by the installed headers for GNU glibc and other system libraries. > + > +Package: linux-image-${KERNEL_NAME_PROVIDED}-dbg > +Section: debug > +Architecture: any > +Description: Linux kernel debugging symbols for ${PV} > + This package will come in handy if you need to debug the kernel. It provides > + all the necessary debug symbols for the kernel and its modules. > diff --git a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl > new file mode 100644 > index 0000000..4ea18d5 > --- /dev/null > +++ b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl > @@ -0,0 +1,45 @@ > +#!/bin/bash > +# Copyright (c) Mentor Graphics, a Siemens business, 2019 > +# SPDX-License-Identifier: MIT > + > +# Load common stuff > +. ${S}/debian/isar/common || exit ${?} > + > +do_build() { > + > + # Print a few things that are of particular interest > + print_settings > + > + # Process existing kernel configuration to make sure it is complete > + # (use defaults for options that were not specified) > + ${MAKE} O=${KERNEL_BUILD_DIR} olddefconfig prepare || exit ${?} > + > + # Check if the recipe's PV makes sense > + KR=$(${MAKE} O=${KERNEL_BUILD_DIR} -s --no-print-directory kernelrelease) > + eval $(grep ^CONFIG_LOCALVERSION= ${KERNEL_BUILD_DIR}/${KCONF} || true) > + if [ "${PV}" != "${KR}" -a "${PV}${CONFIG_LOCALVERSION}" != "${KR}" ]; then > + echo "ERROR: Recipe PV (${PV}) does not seem to match the kernelrelease (${KR})!" 1>&2 > + echo "ERROR: Make sure the kernel version in your PV setting and/or CONFIG_LOCALVERSION are aligned" 1>&2 > + exit 1 > + fi > + > + # Trace what we do here > + set -x > + > + # Build the Linux kernel > + ${MAKE} O=${KERNEL_BUILD_DIR} -j $(nproc) || exit ${?} > + > + # Stop tracing > + set +x > +} > + > +print_settings() { > + cat <<EOF > +# Build settings: > +# --------------- > +# ARCH=${ARCH} > +# CROSS_COMPILE=${CROSS_COMPILE} > +EOF > +} > + > +main build ${*} > diff --git a/meta/recipes-kernel/linux/files/debian/isar/clean.tmpl b/meta/recipes-kernel/linux/files/debian/isar/clean.tmpl > new file mode 100644 > index 0000000..f903987 > --- /dev/null > +++ b/meta/recipes-kernel/linux/files/debian/isar/clean.tmpl > @@ -0,0 +1,21 @@ > +#!/bin/bash > +# Copyright (c) Mentor Graphics, a Siemens business, 2019 > +# SPDX-License-Identifier: MIT > + > +# Load common stuff > +. ${S}/debian/isar/common || exit ${?} > + > +do_clean() { > + > + # Trace what we do here > + set -x > + > + rm -rf ${deb_img_dir} ${deb_dbg_dir} > + rm -rf ${deb_kern_hdrdir} > + rm -rf ${deb_libc_hdr_dir} > + > + # Stop tracing > + set +x > +} > + > +main clean ${*} > diff --git a/meta/recipes-kernel/linux/files/debian/isar/common.tmpl b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl > new file mode 100644 > index 0000000..44f67b3 > --- /dev/null > +++ b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl > @@ -0,0 +1,50 @@ > +#!/bin/bash > +# Copyright (c) Mentor Graphics, a Siemens business, 2019 > +# SPDX-License-Identifier: MIT > + > +# Isar settings > +ARCH=${KERNEL_ARCH} > +KERNEL_PKG_IMAGE=linux-image-${KERNEL_NAME_PROVIDED} > +KERNEL_PKG_KERN_HEADERS=linux-headers-${KERNEL_NAME_PROVIDED} > +KERNEL_PKG_LIBC_HEADERS=linux-libc-dev > + > +# Constants > +KCONF=.config > + > +# Target directories > +deb_top_dir=${S}/debian > +deb_img_dir=${deb_top_dir}/${KERNEL_PKG_IMAGE} > +deb_dbg_dir=${deb_img_dir}-dbg > +deb_dtb_dir=${deb_img_dir}/${KERNEL_DTB_DIR} > +deb_kern_hdr_dir=${deb_top_dir}/${KERNEL_PKG_KERN_HEADERS} > +deb_libc_hdr_dir=${deb_top_dir}/${KERNEL_PKG_LIBC_HEADERS} > + > +# Array of packages to be generated > +declare -A kern_pkgs > + > +main() { > + local target=${1} > + > + if [ ! -f ${S}/debian/isar/${target} ]; then > + echo "error: ${target} is not a supported build target!" >&2 > + return 1 > + fi > + > + # create do_<x> variables for each package to be generated > + for p in $(unset DEB_HOST_ARCH; dh_listpackages); do > + kern_pkgs["${p}"]="1" > + done > + > + # variables to be exported > + export ARCH > + > + # are we cross-compiling? > + BUILD_ARCH=$(dpkg-architecture -qDEB_BUILD_ARCH) > + if [ "${BUILD_ARCH}" = "${DISTRO_ARCH}" ]; then > + # no, make sure CROSS_COMPILE isn't set > + unset CROSS_COMPILE > + fi > + > + # call the actual target script > + do_${target} || return ${?} > +} > diff --git a/meta/recipes-kernel/linux/files/debian/isar/install.tmpl b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl > new file mode 100644 > index 0000000..8ccda68 > --- /dev/null > +++ b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl > @@ -0,0 +1,206 @@ > +#!/bin/bash > +# Copyright (c) Mentor Graphics, a Siemens business, 2019 > +# SPDX-License-Identifier: MIT > + > +# Load common stuff > +. ${S}/debian/isar/common || exit ${?} > + > +# Stop on error > +set -e > + > +do_install() { > + > + # check if our kernel was configured > + if [ ! -f "${O}/.config" ]; then > + echo "error: kernel not configured!" >&2 > + return 1 > + fi > + > + # load its configuration > + . ${O}/.config > + > + kimage="$(${MAKE} O=${O} -s --no-print-directory image_name)" > + krel="$(${MAKE} O=${O} -s --no-print-directory kernelrelease)" > + case "${ARCH}" in > + mips|powerpc) kimage_path="boot/vmlinux-${krel}" ;; > + um) kimage_path="usr/bin/vmlinux-${krel}" ;; > + *) kimage_path="boot/vmlinuz-${krel}" ;; > + esac > + > + print_settings > + > + # Trace what we do here > + set -x > + > + install_image > + install_hooks > + install_dtbs > + install_kmods > + install_headers > + > + # Stop tracing > + set +x > +} > + > +print_settings() { > + cat <<EOF > +Install settings: > +----------------- > +deb_dtb_dir=${deb_dtb_dir} > +deb_hdr_dir=${deb_hdr_dir} > +kimage=${kimage} > +kimage_path=${kimage_path} > + > +EOF > +} > + > +install_image() { > + install -m 755 -d ${deb_img_dir}/$(dirname ${kimage_path}) > + cp ${O}/${kimage} ${deb_img_dir}/${kimage_path} > + install_image_debug > +} > + > +install_image_debug() { > + # Different tools want the image in different locations > + # perf > + mkdir -p ${deb_dbg_dir}/usr/lib/debug/lib/modules/${krel}/ > + cp ${O}/vmlinux ${deb_dbg_dir}/usr/lib/debug/lib/modules/${krel}/ > + # systemtap > + mkdir -p ${deb_dbg_dir}/usr/lib/debug/boot/ > + ln -s ../lib/modules/$version/vmlinux ${deb_dbg_dir}/usr/lib/debug/boot/vmlinux-${krel} > + # kdump-tools > + ln -s lib/modules/${krel}/vmlinux ${deb_dbg_dir}/usr/lib/debug/vmlinux-${krel} > +} > + > +install_hooks() { > + install -m 755 -d ${deb_img_dir}/etc/kernel/install.d > + install -m 755 -d ${deb_img_dir}/etc/kernel/postinst.d > + install -m 755 -d ${deb_img_dir}/etc/kernel/postrm.d > + install -m 755 -d ${deb_img_dir}/etc/kernel/prerm.d > + > + initrd="No" > + [ -z "${CONFIG_BLK_DEV_INITRD}" ] || initrd="Yes" > + > + for script in postinst postrm preinst prerm; do > + sed -i -e "s,INITRD=[A-Za-z0-9]*,INITRD=${initrd},g" ${S}/debian/linux-image*.${script} > + sed -i -e "s,version=.*,version=${krel},g" ${S}/debian/linux-image*.${script} > + done > +} > + > +install_dtbs() { > + [ -n "${CONFIG_OF}" ] || return 0 > + ${MAKE} O=${O} INSTALL_DTBS_PATH=${deb_dtb_dir} dtbs_install > +} > + > +install_kmods() { > + [ -n "${CONFIG_MODULES}" ] || return 0 > + ${MAKE} O=${O} INSTALL_MOD_PATH=${deb_img_dir} modules_install > + touch ${deb_img_dir}/lib/modules/${krel}/.fresh-install > + rm -fv ${deb_img_dir}/lib/modules/${krel}/build > + rm -fv ${deb_img_dir}/lib/modules/${krel}/source > + install_kmods_debug > +} > + > +kmods_sign() { > + [ -n "${CONFIG_MODULE_SIG_ALL}" ] || return 0 > + ${MAKE} O=${O} INSTALL_MOD_PATH=${deb_img_dir} modules_sign > +} > + > +install_kmods_debug() { > + [ -n "${CONFIG_DEBUG_INFO}" ] || return 0 > + > + kmod_inst_dir=${deb_img_dir}/lib/modules > + kmod_debug_dir=${deb_dbg_dir}/usr/lib/debug > + > + # copy kernels modules to usr/lib/debug > + mkdir -p ${kmod_debug_dir} > + tar -C ${kmod_inst_dir}/ -cO --exclude='modules.*' . | tar -C ${kmod_debug_dir}/ -xf - > + # strip everything but debug sections for modules in usr/lib/debug > + find ${kmod_debug_dir} -name *.ko -exec ${CROSS_COMPILE}objcopy --only-keep-debug {} \; > + # and strip debug sections from modules in lib/modules > + find ${kmod_inst_dir} -name *.ko -exec ${CROSS_COMPILE}objcopy --strip-debug {} \; > + > + # re-sign stripped kernel modules > + kmods_sign > +} > + > +headers_check() { > + ${MAKE} O=${O} headers_check > +} > + > +libc_headers() { > + mkdir -p ${deb_libc_hdr_dir} > + ${MAKE} O=${O} headers_install INSTALL_HDR_PATH=${deb_libc_hdr_dir}/usr > + host_arch=$(dpkg-architecture -a${DISTRO_ARCH} -qDEB_HOST_MULTIARCH) > + mkdir ${deb_libc_hdr_dir}/usr/include/${host_arch} > + mv ${deb_libc_hdr_dir}/usr/include/asm ${deb_libc_hdr_dir}/usr/include/${host_arch}/ > +} > + > +# the kernel does not seem to provide a way to cross-compile its tools, we however > +# have qemu-user-static binaries in our buildchroot. Create a new build tree for the > +# scripts and override HOSTCC to force a cross-compile > +kernel_tools_build() { > + odir="${S}/build-scripts" > + mflags="HOSTCC=${CROSS_COMPILE}gcc O=${odir} NOSTDINC_FLAGS=" > + rm -rf ${odir} && mkdir -p ${odir} && cp ${S}/${KERNEL_BUILD_DIR}/.config ${odir} > + (unset LD_PRELOAD; cd ${odir} && make -C ${S} ${mflags} olddefconfig scripts) > + (cd ${odir}; find scripts -type f -executable -exec file {} \;|grep ELF|cut -d: -f1) >${src_hdr_files} > + tar -C ${odir} -cf - -T - <${src_hdr_files} | tar -C ${destdir} -xf - > +} > + > +kernel_tools() { > + # remove object files > + find ${destdir}/scripts -type f -name '*.o' |xargs rm -f > + > + # we're all done if we aren't cross-compiling > + [ -n "${CROSS_COMPILE}" ] || return 0 > + > + # remove ELF executables from the linux-headers package > + find ${destdir}/scripts -type f -exec file {} \;|grep ELF|cut -d: -f1|xargs rm -fv > + > + # cross-compile kernel tools to be shipped with linux-headers > + kernel_tools_build > +} > + > +kernel_headers() { > + destdir=${deb_kern_hdr_dir}/${KERNEL_HEADERS_DIR} > + src_hdr_files=$(mktemp) > + obj_hdr_files=$(mktemp) > + > + mkdir -p ${destdir} > + mkdir -p ${deb_kern_hdr_dir}/lib/modules/${krel} > + > + (cd ${S}; find . -name 'Makefile*' -o -name 'Kconfig*' -o -name '*.pl') >>${src_hdr_files} > + (cd ${S}; find arch/*/include include scripts -type f -o -type l) >>${src_hdr_files} > + (cd ${S}; find arch/${ARCH} -name module.lds -o -name Kbuild.platforms -o -name Platform) >>${src_hdr_files} > + (cd ${S}; find $(find arch/${ARCH} -name include -o -name scripts -type d) -type f) >>${src_hdr_files} > + > + (cd ${O}; find arch/${ARCH}/include Module.symvers include scripts -type f) >>${obj_hdr_files} > + if [ -n "${CONFIG_STACK_VALIDATION}" ]; then > + (cd ${O}; find tools/objtool -type f -executable) >>${obj_hdr_files} > + fi > + if [ -n "${CONFIG_GCC_PLUGINS}" ]; then > + (cd ${O}; find scripts/gcc-plugins -name *.so -o -name gcc-common.h) >>${obj_hdr_files} > + fi > + > + # deploy files that were matched above > + tar -C ${S} -cf - -T - <${src_hdr_files} | tar -C ${destdir} -xf - > + tar -C ${O} -cf - -T - <${obj_hdr_files} | tar -C ${destdir} -xf - > + > + # add the kernel config > + cp ${O}/${KCONF} ${destdir}/.config > + > + # handle kernel development tools > + kernel_tools > + > + # create symlinks > + ln -sf /${KERNEL_HEADERS_DIR} ${deb_kern_hdr_dir}/lib/modules/${krel}/build > +} > + > +install_headers() { > + headers_check > + [ -z ${kern_pkgs["linux-libc-headers"]} ] || libc_headers > + kernel_headers > +} > + > +main install ${*} > diff --git a/meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl b/meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl > new file mode 100644 > index 0000000..8e8636a > --- /dev/null > +++ b/meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl > @@ -0,0 +1,29 @@ > +#!/bin/sh > +# based on https://salsa.debian.org/kernel-team/linux/blob/master/debian/templates/image.postinst.in > + > +# Tell initramfs builder whether it's wanted > +export INITRD=Yes > + > +version=${PV} > +image_path=/boot/${KERNEL_FILE}-${version} > + > +if [ "$1" != configure ]; then > + exit 0 > +fi > + > +depmod $version > + > +if [ -f /lib/modules/$version/.fresh-install ]; then > + change=install > +else > + change=upgrade > +fi > +linux-update-symlinks $change $version $image_path > +rm -f /lib/modules/$version/.fresh-install > + > +if [ -d /etc/kernel/postinst.d ]; then > + DEB_MAINT_PARAMS="$*" run-parts --report --exit-on-error --arg=$version \ > + --arg=$image_path /etc/kernel/postinst.d > +fi > + > +exit 0 > diff --git a/meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl b/meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl > new file mode 100644 > index 0000000..4d8a7d9 > --- /dev/null > +++ b/meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl > @@ -0,0 +1,35 @@ > +#!/bin/sh -e > +# based on https://salsa.debian.org/kernel-team/linux/blob/master/debian/templates/image.postrm.in > + > +# Tell initramfs builder whether it's wanted > +export INITRD=Yes > + > +version=${PV} > +image_path=/boot/${KERNEL_FILE}-${version} > + > +rm -f /lib/modules/$version/.fresh-install > + > +if [ "$1" != upgrade ] && command -v linux-update-symlinks >/dev/null; then > + linux-update-symlinks remove $version $image_path > +fi > + > +if [ -d /etc/kernel/postrm.d ]; then > + DEB_MAINT_PARAMS="$*" run-parts --report --exit-on-error --arg=$version \ > + --arg=$image_path /etc/kernel/postrm.d > +fi > + > +if [ "$1" = purge ]; then > + for extra_file in modules.dep modules.isapnpmap modules.pcimap \ > + modules.usbmap modules.parportmap \ > + modules.generic_string modules.ieee1394map \ > + modules.ieee1394map modules.pnpbiosmap \ > + modules.alias modules.ccwmap modules.inputmap \ > + modules.symbols modules.ofmap \ > + modules.seriomap modules.\*.bin \ > + modules.softdep modules.devname; do > + eval rm -f /lib/modules/$version/$extra_file > + done > + rmdir /lib/modules/$version || true > +fi > + > +exit 0 > diff --git a/meta/recipes-kernel/linux/files/debian/linux-image.preinst.tmpl b/meta/recipes-kernel/linux/files/debian/linux-image.preinst.tmpl > new file mode 100644 > index 0000000..7465197 > --- /dev/null > +++ b/meta/recipes-kernel/linux/files/debian/linux-image.preinst.tmpl > @@ -0,0 +1,25 @@ > +#!/bin/sh -e > +# based on https://salsa.debian.org/kernel-team/linux/blob/master/debian/templates/image.preinst.in > + > +# Tell initramfs builder whether it's wanted > +export INITRD=Yes > + > +version=${PV} > +image_path=/boot/${KERNEL_FILE}-${version} > + > +if [ "$1" = abort-upgrade ]; then > + exit 0 > +fi > + > +if [ "$1" = install ]; then > + # Create a flag file for postinst > + mkdir -p /lib/modules/$version > + touch /lib/modules/$version/.fresh-install > +fi > + > +if [ -d /etc/kernel/preinst.d ]; then > + DEB_MAINT_PARAMS="$*" run-parts --report --exit-on-error --arg=$version \ > + --arg=$image_path /etc/kernel/preinst.d > +fi > + > +exit 0 > diff --git a/meta/recipes-kernel/linux/files/debian/linux-image.prerm.tmpl b/meta/recipes-kernel/linux/files/debian/linux-image.prerm.tmpl > new file mode 100644 > index 0000000..1992b32 > --- /dev/null > +++ b/meta/recipes-kernel/linux/files/debian/linux-image.prerm.tmpl > @@ -0,0 +1,21 @@ > +#!/bin/sh -e > +# based on https://salsa.debian.org/kernel-team/linux/blob/master/debian/templates/image.prerm.in > + > +# Tell initramfs builder whether it's wanted > +export INITRD=Yes > + > +version=${PV} > +image_path=/boot/${KERNEL_FILE}-${version} > + > +if [ "$1" != remove ]; then > + exit 0 > +fi > + > +linux-check-removal $version > + > +if [ -d /etc/kernel/prerm.d ]; then > + DEB_MAINT_PARAMS="$*" run-parts --report --exit-on-error --arg=$version \ > + --arg=$image_path /etc/kernel/prerm.d > +fi > + > +exit 0 > diff --git a/meta/recipes-kernel/linux/files/debian/rules.tmpl b/meta/recipes-kernel/linux/files/debian/rules.tmpl > new file mode 100755 > index 0000000..c024768 > --- /dev/null > +++ b/meta/recipes-kernel/linux/files/debian/rules.tmpl > @@ -0,0 +1,39 @@ > +#!/usr/bin/make -f > + > +CROSS_COMPILE:=$(DEB_HOST_GNU_TYPE)- > + > +O:=$(CURDIR)/${KERNEL_BUILD_DIR} > +S:=$(CURDIR) > +deb_top_dir:=$(S)/debian > + > +# Dynamic variables to be passed to Isar build scripts > +isar_env=$(strip \ > + export CROSS_COMPILE='$(CROSS_COMPILE)' && \ > + export DEB_BUILD_PROFILES='$(DEB_BUILD_PROFILES)' && \ > + export DEB_HOST_GNU_TYPE='$(DEB_HOST_GNU_TYPE)' && \ > + export MAKE='$(MAKE)' && \ > + export O='${O}' \ > + export S='${S}' \ > +) > + > +%: > + dh $(@) > + > +.PHONY: override_dh_auto_clean > +override_dh_auto_clean: > + $(isar_env) && bash $(deb_top_dir)/isar/clean > + > +.PHONY: override_dh_auto_build > +override_dh_auto_build: > + $(isar_env) && bash $(deb_top_dir)/isar/build > + > +override_dh_auto_install: > + $(isar_env) && bash $(deb_top_dir)/isar/install > + > +.PHONY: override_dh_auto_test > +override_dh_auto_test: > + true > + > +PHONY: override_dh_strip > +override_dh_strip: > + dh_strip -Xvmlinux --no-automatic-dbgsym > diff --git a/meta/recipes-kernel/linux/linux-custom.inc b/meta/recipes-kernel/linux/linux-custom.inc > index ca91f64..ecf0e66 100644 > --- a/meta/recipes-kernel/linux/linux-custom.inc > +++ b/meta/recipes-kernel/linux/linux-custom.inc > @@ -2,22 +2,107 @@ > # > # This software is a part of ISAR. > # Copyright (c) Siemens AG, 2018 > +# Copyright (c) Mentor Graphics, a Siemens business, 2019 > # > # SPDX-License-Identifier: MIT > > -FILESEXTRAPATHS_prepend := "${FILE_DIRNAME}/files:" > +# Settings that would typically be done from the custom kernel recipe > +# ------------------------------------------------------------------- > > DESCRIPTION ?= "Custom kernel" > +MAINTAINER ?= "isar-users <isar-users@googlegroups.com>" > + > +KBUILD_DEPENDS ?= "build-essential:native, \ > + libelf-dev:native, \ > + libncurses-dev:native, \ > + libssl-dev:native, \ > + bc, \ > + bison, \ > + cpio, \ > + flex, \ > + git, \ > + kmod, \ > + libssl-dev," > + > +KERNEL_DEBIAN_DEPENDS ?= "initramfs-tools | linux-initramfs-tool, \ > + kmod, \ > + linux-base (>= 4.3~)," > + > +KERNEL_HEADERS_DEBIAN_DEPENDS ?= "libc6, \ > + libssl1.1," > + > +KERNEL_LIBC_DEV_DEPLOY ?= "0" > + > +# Settings that may be changed on a per distro, machine or layer basis > +# -------------------------------------------------------------------- > + > +KERNEL_DTB_DIR ?= "usr/lib/linux-${KERNEL_NAME_PROVIDED}-${PV}" Here is the bug that breaks U-boot setups here: Must be linux-image-${PV} in order to stay compatible with Debian. Did you compare some content of the new packages against that of the original package build? Jan > +KERNEL_HEADERS_DIR ?= "usr/src/linux-headers-${PV}" > > -KERNEL_NAME_PROVIDED ?= "${@ d.getVar('PN', True).partition('linux-')[2]}" > KERNEL_DEFCONFIG ?= "" > > +# Add our template meta-data to the sources > +FILESEXTRAPATHS_prepend := "${FILE_DIRNAME}/files:" > +SRC_URI += "file://debian" > + > +# Variables and files that make our templates > +# ------------------------------------------- > + > +TEMPLATE_FILES += " \ > + debian/control.tmpl \ > + debian/isar/build.tmpl \ > + debian/isar/clean.tmpl \ > + debian/isar/common.tmpl \ > + debian/isar/install.tmpl \ > + debian/linux-image.postinst.tmpl \ > + debian/linux-image.postrm.tmpl \ > + debian/linux-image.preinst.tmpl \ > + debian/linux-image.prerm.tmpl \ > + debian/rules.tmpl \ > +" > + > +TEMPLATE_VARS += " \ > + HOST_ARCH \ > + KBUILD_DEPENDS \ > + KERNEL_ARCH \ > + KERNEL_DEBIAN_DEPENDS \ > + KERNEL_BUILD_DIR \ > + KERNEL_DTB_DIR \ > + KERNEL_FILE \ > + KERNEL_HEADERS_DEBIAN_DEPENDS \ > + KERNEL_HEADERS_DIR \ > + KERNEL_NAME_PROVIDED \ > +" > + > +inherit dpkg > +inherit template > + > +# Derive name of the kernel packages from the name of this recipe > +KERNEL_NAME_PROVIDED ?= "${@ d.getVar('PN', True).partition('linux-')[2]}" > + > +# Make bitbake know we will be producing linux-image and linux-headers packages > python() { > kernel_name = d.getVar("KERNEL_NAME_PROVIDED", True) > d.setVar('PROVIDES', 'linux-image-' + kernel_name + ' ' + \ > 'linux-headers-' + kernel_name) > } > > +def get_kernel_arch(d): > + distro_arch = d.getVar("DISTRO_ARCH") > + if distro_arch == "amd64": > + kernel_arch = "x86" > + elif distro_arch == "arm64": > + kernel_arch = "arm64" > + elif distro_arch == "armhf": > + kernel_arch = "arm" > + elif distro_arch == "mipsel": > + kernel_arch = "mips" > + else: > + kernel_arch = "" > + return kernel_arch > + > +KERNEL_ARCH ??= "${@get_kernel_arch(d)}" > + > def config_fragments(d): > fragments = [] > sources = d.getVar("SRC_URI").split() > @@ -31,57 +116,37 @@ def config_fragments(d): > fragments.append(local) > return fragments > > -inherit dpkg-base > - > -SRC_URI += "file://build-kernel.sh" > +do_prepare_build_prepend() { > + # copy meta-data over to source tree > + rm -rf ${S}/debian > + cp -r ${WORKDIR}/debian ${S}/ > > -KBUILD_DEPENDS ?= " \ > - build-essential:native \ > - libssl-dev \ > - libelf-dev \ > - bc \ > - git \ > - kmod \ > - bison \ > - flex \ > - cpio \ > - libncurses-dev" > -KERNEL_DEBIAN_DEPENDS ?= "initramfs-tools | linux-initramfs-tool, kmod, linux-base (>= 4.3~)" > -KERNEL_HEADERS_DEBIAN_DEPENDS ?= "libc6, libssl1.1" > + # remove templates from the source tree > + find ${S}/debian -name *.tmpl | xargs rm -f > > -KERNEL_LIBC_DEV_DEPLOY ?= "0" > + # rename install/remove hooks to match user-specified name for our linux-image package > + mv ${S}/debian/linux-image.postinst ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.postinst > + mv ${S}/debian/linux-image.postrm ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.postrm > + mv ${S}/debian/linux-image.preinst ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.preinst > + mv ${S}/debian/linux-image.prerm ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.prerm > > -do_install_builddeps() { > - dpkg_do_mounts > - E="${@ bb.utils.export_proxies(d)}" > - sudo -E chroot ${BUILDCHROOT_DIR} \ > - apt-get update \ > - -o Dir::Etc::SourceList="sources.list.d/isar-apt.list" \ > - -o Dir::Etc::SourceParts="-" \ > - -o APT::Get::List-Cleanup="0" > - sudo -E chroot ${BUILDCHROOT_DIR} \ > - apt-get install \ > - -y -o Debug::pkgProblemResolver=yes \ > - --no-install-recommends ${KBUILD_DEPENDS} > - dpkg_undo_mounts > + # produce a changelog for our kernel build > + deb_add_changelog > } > > -addtask install_builddeps after do_prepare_build before do_dpkg_build > -# apt and reprepro may not run in parallel, acquire the Isar lock > -do_install_builddeps[lockfiles] += "${REPO_ISAR_DIR}/isar.lock" > - > -addtask devshell after do_install_builddeps > +# build directory for our "full" kernel build > +KERNEL_BUILD_DIR = "build-full" > > -dpkg_runbuild() { > - chmod +x ${WORKDIR}/build-kernel.sh > - KERNEL_CONFIG_TARGET="${KERNEL_DEFCONFIG}" > +dpkg_configure_kernel() { > + config_target="${KERNEL_DEFCONFIG}" > + rm -rf ${S}/${KERNEL_BUILD_DIR} && mkdir -p ${S}/${KERNEL_BUILD_DIR} > if [ -n "${KERNEL_DEFCONFIG}" ]; then > if [ -e "${WORKDIR}/${KERNEL_DEFCONFIG}" ]; then > - cp ${WORKDIR}/${KERNEL_DEFCONFIG} ${S}/.config > - KERNEL_CONFIG_TARGET="olddefconfig" > + cp ${WORKDIR}/${KERNEL_DEFCONFIG} ${S}/${KERNEL_BUILD_DIR}/.config > + config_target="olddefconfig" > fi > else > - KERNEL_CONFIG_TARGET="defconfig" > + config_target="defconfig" > fi > > # copy config fragments over to the kernel tree > @@ -92,22 +157,23 @@ dpkg_runbuild() { > mkdir -p ${S}/debian/fragments > (cd ${WORKDIR} && cp ${src_frags} ${S}/debian/fragments/) > fi > - export KERNEL_FRAGMENTS="${out_frags}" > > - E="${@ bb.utils.export_proxies(d)}" > - > - export PV=${PV} > - export KERNEL_NAME=${KERNEL_NAME_PROVIDED} > - > - export KBUILD_DEPENDS="${KBUILD_DEPENDS}" > - export KERNEL_CONFIG_TARGET > - export KERNEL_DEBIAN_DEPENDS="${KERNEL_DEBIAN_DEPENDS}" > - export KERNEL_FILE="${KERNEL_FILE}" > - export KERNEL_HEADERS_DEBIAN_DEPENDS="${KERNEL_HEADERS_DEBIAN_DEPENDS}" > - > - sudo -E chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} ${PP}/build-kernel.sh ${PP}/${PPS} ${DISTRO_ARCH} > + sudo -E chroot --userspec=$(id -u):$(id -g) ${BUILDCHROOT_DIR} sh -c " \ > + export ARCH=${KERNEL_ARCH} && \ > + cd ${PP}/${PPS} && \ > + make O=${KERNEL_BUILD_DIR} ${config_target} && \ > + ./scripts/kconfig/merge_config.sh \ > + -O ${KERNEL_BUILD_DIR}/ \ > + ${KERNEL_BUILD_DIR}/.config \ > + ${out_frags} \ > + " > +} > > +dpkg_runbuild_prepend() { > + profiles="${BASE_DISTRO}" > if [ "${KERNEL_LIBC_DEV_DEPLOY}" != "1" ]; then > - rm -f ${WORKDIR}/linux-libc-dev_${PV}*.deb > + profiles="${profiles} nolibcdev" > fi > + export DEB_BUILD_PROFILES="${profiles}" > + dpkg_configure_kernel > } > -- Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 83+ messages in thread
* Re: [PATCH L-C v3 7/7] linux-custom: rewrite to no longer depend on the kernel's builddeb 2019-11-08 7:19 ` Jan Kiszka @ 2019-11-08 7:36 ` Cedric Hombourger 2019-11-08 7:41 ` Jan Kiszka 0 siblings, 1 reply; 83+ messages in thread From: Cedric Hombourger @ 2019-11-08 7:36 UTC (permalink / raw) To: Jan Kiszka, isar-users On 11/8/19 8:19 AM, Jan Kiszka wrote: > On 08.11.19 08:07, Cedric Hombourger wrote: >> Regain control over the packaging of the Linux kernel by providing our >> own debian recipes (debian,{control,rules} files and friends) instead >> of using the kernel's builddeb script. This will allow generation of >> packages for "perf" (not included in this changeset) with per distro >> scheme (Debian and Ubuntu have differences) and maintain compatibility >> with upstream packages generated from different source packages but >> dependent on a certain deployment scheme (e.g. linux-base for Debian) >> >> Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com> >> --- >> doc/custom_kernel.md | 80 +++++++ >> .../linux/files/build-kernel.sh | 128 ----------- >> meta/recipes-kernel/linux/files/debian/compat | 1 + >> .../linux/files/debian/control.tmpl | 37 ++++ >> .../linux/files/debian/isar/build.tmpl | 45 ++++ >> .../linux/files/debian/isar/clean.tmpl | 21 ++ >> .../linux/files/debian/isar/common.tmpl | 50 +++++ >> .../linux/files/debian/isar/install.tmpl | 206 ++++++++++++++++++ >> .../files/debian/linux-image.postinst.tmpl | 29 +++ >> .../files/debian/linux-image.postrm.tmpl | 35 +++ >> .../files/debian/linux-image.preinst.tmpl | 25 +++ >> .../linux/files/debian/linux-image.prerm.tmpl | 21 ++ >> .../linux/files/debian/rules.tmpl | 39 ++++ >> meta/recipes-kernel/linux/linux-custom.inc | 180 ++++++++++----- >> 14 files changed, 712 insertions(+), 185 deletions(-) >> create mode 100644 doc/custom_kernel.md >> delete mode 100644 meta/recipes-kernel/linux/files/build-kernel.sh >> create mode 100644 meta/recipes-kernel/linux/files/debian/compat >> create mode 100644 meta/recipes-kernel/linux/files/debian/control.tmpl >> create mode 100644 >> meta/recipes-kernel/linux/files/debian/isar/build.tmpl >> create mode 100644 >> meta/recipes-kernel/linux/files/debian/isar/clean.tmpl >> create mode 100644 >> meta/recipes-kernel/linux/files/debian/isar/common.tmpl >> create mode 100644 >> meta/recipes-kernel/linux/files/debian/isar/install.tmpl >> create mode 100644 >> meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl >> create mode 100644 >> meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl >> create mode 100644 >> meta/recipes-kernel/linux/files/debian/linux-image.preinst.tmpl >> create mode 100644 >> meta/recipes-kernel/linux/files/debian/linux-image.prerm.tmpl >> create mode 100755 meta/recipes-kernel/linux/files/debian/rules.tmpl >> >> diff --git a/doc/custom_kernel.md b/doc/custom_kernel.md >> new file mode 100644 >> index 0000000..31319d4 >> --- /dev/null >> +++ b/doc/custom_kernel.md >> @@ -0,0 +1,80 @@ >> +# Custom kernel recipe for Isar >> + >> +## Contents >> + >> + - [Summary](#summary) >> + - [Features](#features) >> + - [Future](#future) >> + - [Examples](#examples) >> + >> +## Summary >> + >> +Isar provides a recipe to build custom kernels for Debian-based >> distributions. >> +It uses templates to generate the debian meta-data (such as >> debian/control) and >> +Debian's [BuildProfiles](https://wiki.debian.org/BuildProfileSpec) >> to handle >> +some of the distro specific variations. It should be noted that Isar >> has moved >> +away from using the kernel's builddeb script since it would not >> generate all >> +the packages we need (and in particular perf). >> + >> +## Features >> + >> +The linux-custom recipe provides support for: >> + >> + 1. Sources to the custom Linux kernel may be specified via `SRC_URI` >> + >> + 2. Configure the kernel via an in-tree or an external `defconfig` via >> + `KERNEL_DEFCONFIG` >> + >> + 3. Integrate kernel configuration tweaks via configuration >> fragments (`.cfg` >> + files) >> + >> + 4. Patches to the linux kernel may be specified via `SRC_URI` >> + >> + 5. Ensure that the Isar recipe `PV` matches the kernel release >> identifier >> + (`KERNEL_RELEASE`) >> + >> + 6. Produce a `linux-image` package that ships the kernel image and >> modules >> + >> + 7. Allow the name of the kernel image to be changed via >> `KERNEL_FILE` (defaults >> + to `vmlinuz`) >> + >> + 8. Produce a `linux-headers` package which includes kernel headers >> and kbuild >> + scripts/tools >> + >> + 9. The `linux-headers` package shall support native and cross >> compiles of >> + out-of-tree kernel modules >> + >> + 10. Produce a `linux-libc-dev` package to support user-land builds >> + >> + 11. Only build/ship the `linux-libc-dev` package if instructed to >> + (`KERNEL_LIBC_DEV_DEPLOY` equals to `"1"`) >> + >> + 12. Support both native and cross compiles (`ISAR_CROSS_COMPILE`) >> + >> + 13. Support for the following kernel architectures: >> + >> + * arm >> + * arm64 >> + * mips >> + * x86 >> + >> + 14. Support `devshell` (kernel configuration shall be applied) >> + >> +## Future >> + >> +In the future, the recipe may be extended to: >> + >> + 1. Package perf >> + >> + 2. Support inclusion/build of dts files listed in `SRC_URI` >> + >> + 3. Be compatible with Ubuntu >> + >> +## Examples >> + >> +The linux-custom recipe is currently used by the linux-mainline >> package and is >> +used mainline recipe may be used for some basic testing. This recipe >> is being >> +used by the following machines: >> + >> + * de0-nano-soc >> + * qemumipsel >> diff --git a/meta/recipes-kernel/linux/files/build-kernel.sh >> b/meta/recipes-kernel/linux/files/build-kernel.sh >> deleted file mode 100644 >> index f56e96c..0000000 >> --- a/meta/recipes-kernel/linux/files/build-kernel.sh >> +++ /dev/null >> @@ -1,128 +0,0 @@ >> -#!/bin/bash >> -# >> -# Custom kernel build >> -# >> -# This software is a part of ISAR. >> -# Copyright (c) Siemens AG, 2018 >> -# >> -# SPDX-License-Identifier: MIT >> - >> -source /isar/common.sh >> - >> -host_arch=$(dpkg --print-architecture) >> - >> -if [ "$host_arch" != "$target_arch" ]; then >> - case $target_arch in >> - armhf) >> - export ARCH=arm >> - export CROSS_COMPILE="arm-linux-gnueabihf-" >> - ;; >> - arm64) >> - export ARCH=arm64 >> - export CROSS_COMPILE="aarch64-linux-gnu-" >> - ;; >> - mipsel) >> - export ARCH=mips >> - export CROSS_COMPILE="mipsel-linux-gnu-" >> - ;; >> - *) >> - echo "error: unsupported architecture ($target_arch)" >> - exit 1 >> - ;; >> - esac >> -fi >> - >> -REPACK_DIR="$1/../repack" >> -REPACK_LINUX_IMAGE_DIR="${REPACK_DIR}/linux-image" >> -REPACK_LINUX_HEADERS_DIR="${REPACK_DIR}/linux-headers" >> - >> -make ${KERNEL_CONFIG_TARGET} || exit ${?} >> -if [ -n "${KERNEL_FRAGMENTS}" ]; then >> - scripts/kconfig/merge_config.sh -m .config ${KERNEL_FRAGMENTS} >> -fi >> - >> -KV=$( make -s kernelrelease ) >> -if [ "${KV}" != "${PV}" ]; then >> - echo "ERROR: Recipe PV is \"${PV}\" but should be \"${KV}\"" 1>&2 >> - echo "ERROR: Probably due to CONFIG_LOCALVERSION" 1>&2 >> - exit 1 >> -fi >> - >> -rm -f .version >> -KBUILD_DEBARCH=$target_arch make -j $(($(nproc) * 2)) deb-pkg >> - >> -rm -rf "${REPACK_DIR}" >> -mkdir -p "${REPACK_DIR}" >> -mkdir -p "${REPACK_LINUX_IMAGE_DIR}" >> -mkdir -p "${REPACK_LINUX_HEADERS_DIR}" >> - >> -cp -a debian "${REPACK_DIR}" >> - >> -# dpkg-gencontrol performs cross-incompatible checks on the >> -# Architecture field; trick it to accept the control file >> -sed -i "s/Architecture: .*/Architecture: any/" >> "${REPACK_DIR}/debian/control" >> - >> -cd .. >> - >> -dpkg-deb -R linux-image-${PV}_${PV}-1_*.deb "${REPACK_LINUX_IMAGE_DIR}" >> -dpkg-deb -R linux-headers-${PV}_${PV}-1_*.deb >> "${REPACK_LINUX_HEADERS_DIR}" >> - >> -dpkg-gencontrol -crepack/debian/control \ >> - -lrepack/debian/changelog \ >> - -frepack/debian/files \ >> - -plinux-image-${PV} \ >> - -P"${REPACK_LINUX_IMAGE_DIR}" \ >> - -DPackage="linux-image-${KERNEL_NAME}" \ >> - -DSection=kernel \ >> - -DPriority=required \ >> - -DDepends="${KERNEL_DEBIAN_DEPENDS}" \ >> - -DArchitecture=$target_arch >> - >> -# Add Debian-like link installation to postinst >> -mkdir -p ${REPACK_LINUX_IMAGE_DIR}/lib/modules/${PV} >> -touch "${REPACK_LINUX_IMAGE_DIR}/lib/modules/${PV}/.fresh-install" >> -sed -i "${REPACK_LINUX_IMAGE_DIR}/DEBIAN/postinst" \ >> - -e "/^set -e$/a\\ >> -\\ >> -if [ -f /lib/modules/${PV}/.fresh-install ]; then\\ >> - change=install\\ >> -else\\ >> - change=upgrade\\ >> -fi\\ >> -linux-update-symlinks \$change ${PV} /boot/${KERNEL_FILE}-${PV}\\ >> -rm -f /lib/modules/${PV}/.fresh-install" >> - >> -# Add Debian-like link removal to postrm >> -sed -i "${REPACK_LINUX_IMAGE_DIR}/DEBIAN/postrm" \ >> - -e "/^set -e$/a\\ >> -\\ >> -rm -f /lib/modules/${PV}/.fresh-install\\ >> -\\ >> -if [ \"\$1\" != upgrade ] && command -v linux-update-symlinks >> >/dev/null; then\\ >> - linux-update-symlinks remove ${PV} /boot/${KERNEL_FILE}-${PV}\\ >> -fi" >> - >> -# Make sure arm64 kernels are decompressed >> -if [ "$target_arch" = "arm64" ]; then >> - kernel_file="${REPACK_LINUX_IMAGE_DIR}/boot/${KERNEL_FILE}-${PV}" >> - mv "${kernel_file}" "${kernel_file}.gz" >> - gunzip "${kernel_file}.gz" >> -fi >> - >> -dpkg-gencontrol -crepack/debian/control \ >> - -lrepack/debian/changelog \ >> - -frepack/debian/files \ >> - -plinux-headers-${PV} \ >> - -P"${REPACK_LINUX_HEADERS_DIR}" \ >> - -Vkernel:debarch="${KERNEL_NAME}" \ >> - -DPackage="linux-headers-${KERNEL_NAME}" \ >> - -DSection=kernel \ >> - -DDepends="${KERNEL_HEADERS_DEBIAN_DEPENDS}" \ >> - -DArchitecture=$target_arch >> - >> -fakeroot dpkg-deb -b "${REPACK_LINUX_IMAGE_DIR}" \ >> - linux-image-${KERNEL_NAME}_${PV}-1_${KERNEL_NAME}.deb >> -rm -f linux-image-${PV}_${PV}-1_*.deb >> -fakeroot dpkg-deb -b "${REPACK_LINUX_HEADERS_DIR}" \ >> - linux-headers-${KERNEL_NAME}_${PV}-1_${KERNEL_NAME}.deb >> -rm -f linux-headers-${PV}_${PV}-1_*.deb >> diff --git a/meta/recipes-kernel/linux/files/debian/compat >> b/meta/recipes-kernel/linux/files/debian/compat >> new file mode 100644 >> index 0000000..ec63514 >> --- /dev/null >> +++ b/meta/recipes-kernel/linux/files/debian/compat >> @@ -0,0 +1 @@ >> +9 >> diff --git a/meta/recipes-kernel/linux/files/debian/control.tmpl >> b/meta/recipes-kernel/linux/files/debian/control.tmpl >> new file mode 100644 >> index 0000000..d55096b >> --- /dev/null >> +++ b/meta/recipes-kernel/linux/files/debian/control.tmpl >> @@ -0,0 +1,37 @@ >> +Source: linux-${KERNEL_NAME_PROVIDED} >> +Section: kernel >> +Priority: optional >> +Maintainer: ${MAINTAINER} >> +Build-Depends: bc, kmod, cpio, ${KBUILD_DEPENDS} >> +Homepage: http://www.kernel.org/ >> + >> +Package: linux-image-${KERNEL_NAME_PROVIDED} >> +Architecture: any >> +Depends: ${KERNEL_DEBIAN_DEPENDS} >> +Description: ${KERNEL_NAME_PROVIDED} Linux kernel, version ${PV} >> + This package contains the Linux kernel, modules and corresponding >> other >> + files, version: ${PV}. >> + >> +Package: linux-headers-${KERNEL_NAME_PROVIDED} >> +Architecture: any >> +Depends: ${KERNEL_HEADERS_DEBIAN_DEPENDS}, ${perl:Depends}, >> ${shlib:Depends} >> +Description: ${KERNEL_NAME_PROVIDED} Linux kernel headers for ${PV} >> + This package provides kernel header files for ${PV} on ${DISTRO_ARCH} >> + . >> + This is useful for people who need to build external modules >> + >> +Package: linux-libc-dev >> +Build-Profiles: <!nolibcdev> >> +Section: devel >> +Provides: linux-kernel-headers >> +Architecture: any >> +Description: Linux support headers for userspace development >> + This package provides userspaces headers from the Linux kernel. >> These headers >> + are used by the installed headers for GNU glibc and other system >> libraries. >> + >> +Package: linux-image-${KERNEL_NAME_PROVIDED}-dbg >> +Section: debug >> +Architecture: any >> +Description: Linux kernel debugging symbols for ${PV} >> + This package will come in handy if you need to debug the kernel. It >> provides >> + all the necessary debug symbols for the kernel and its modules. >> diff --git a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl >> b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl >> new file mode 100644 >> index 0000000..4ea18d5 >> --- /dev/null >> +++ b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl >> @@ -0,0 +1,45 @@ >> +#!/bin/bash >> +# Copyright (c) Mentor Graphics, a Siemens business, 2019 >> +# SPDX-License-Identifier: MIT >> + >> +# Load common stuff >> +. ${S}/debian/isar/common || exit ${?} >> + >> +do_build() { >> + >> + # Print a few things that are of particular interest >> + print_settings >> + >> + # Process existing kernel configuration to make sure it is complete >> + # (use defaults for options that were not specified) >> + ${MAKE} O=${KERNEL_BUILD_DIR} olddefconfig prepare || exit ${?} >> + >> + # Check if the recipe's PV makes sense >> + KR=$(${MAKE} O=${KERNEL_BUILD_DIR} -s --no-print-directory >> kernelrelease) >> + eval $(grep ^CONFIG_LOCALVERSION= ${KERNEL_BUILD_DIR}/${KCONF} >> || true) >> + if [ "${PV}" != "${KR}" -a "${PV}${CONFIG_LOCALVERSION}" != >> "${KR}" ]; then >> + echo "ERROR: Recipe PV (${PV}) does not seem to match the >> kernelrelease (${KR})!" 1>&2 >> + echo "ERROR: Make sure the kernel version in your PV setting >> and/or CONFIG_LOCALVERSION are aligned" 1>&2 >> + exit 1 >> + fi >> + >> + # Trace what we do here >> + set -x >> + >> + # Build the Linux kernel >> + ${MAKE} O=${KERNEL_BUILD_DIR} -j $(nproc) || exit ${?} >> + >> + # Stop tracing >> + set +x >> +} >> + >> +print_settings() { >> + cat <<EOF >> +# Build settings: >> +# --------------- >> +# ARCH=${ARCH} >> +# CROSS_COMPILE=${CROSS_COMPILE} >> +EOF >> +} >> + >> +main build ${*} >> diff --git a/meta/recipes-kernel/linux/files/debian/isar/clean.tmpl >> b/meta/recipes-kernel/linux/files/debian/isar/clean.tmpl >> new file mode 100644 >> index 0000000..f903987 >> --- /dev/null >> +++ b/meta/recipes-kernel/linux/files/debian/isar/clean.tmpl >> @@ -0,0 +1,21 @@ >> +#!/bin/bash >> +# Copyright (c) Mentor Graphics, a Siemens business, 2019 >> +# SPDX-License-Identifier: MIT >> + >> +# Load common stuff >> +. ${S}/debian/isar/common || exit ${?} >> + >> +do_clean() { >> + >> + # Trace what we do here >> + set -x >> + >> + rm -rf ${deb_img_dir} ${deb_dbg_dir} >> + rm -rf ${deb_kern_hdrdir} >> + rm -rf ${deb_libc_hdr_dir} >> + >> + # Stop tracing >> + set +x >> +} >> + >> +main clean ${*} >> diff --git a/meta/recipes-kernel/linux/files/debian/isar/common.tmpl >> b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl >> new file mode 100644 >> index 0000000..44f67b3 >> --- /dev/null >> +++ b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl >> @@ -0,0 +1,50 @@ >> +#!/bin/bash >> +# Copyright (c) Mentor Graphics, a Siemens business, 2019 >> +# SPDX-License-Identifier: MIT >> + >> +# Isar settings >> +ARCH=${KERNEL_ARCH} >> +KERNEL_PKG_IMAGE=linux-image-${KERNEL_NAME_PROVIDED} >> +KERNEL_PKG_KERN_HEADERS=linux-headers-${KERNEL_NAME_PROVIDED} >> +KERNEL_PKG_LIBC_HEADERS=linux-libc-dev >> + >> +# Constants >> +KCONF=.config >> + >> +# Target directories >> +deb_top_dir=${S}/debian >> +deb_img_dir=${deb_top_dir}/${KERNEL_PKG_IMAGE} >> +deb_dbg_dir=${deb_img_dir}-dbg >> +deb_dtb_dir=${deb_img_dir}/${KERNEL_DTB_DIR} >> +deb_kern_hdr_dir=${deb_top_dir}/${KERNEL_PKG_KERN_HEADERS} >> +deb_libc_hdr_dir=${deb_top_dir}/${KERNEL_PKG_LIBC_HEADERS} >> + >> +# Array of packages to be generated >> +declare -A kern_pkgs >> + >> +main() { >> + local target=${1} >> + >> + if [ ! -f ${S}/debian/isar/${target} ]; then >> + echo "error: ${target} is not a supported build target!" >&2 >> + return 1 >> + fi >> + >> + # create do_<x> variables for each package to be generated >> + for p in $(unset DEB_HOST_ARCH; dh_listpackages); do >> + kern_pkgs["${p}"]="1" >> + done >> + >> + # variables to be exported >> + export ARCH >> + >> + # are we cross-compiling? >> + BUILD_ARCH=$(dpkg-architecture -qDEB_BUILD_ARCH) >> + if [ "${BUILD_ARCH}" = "${DISTRO_ARCH}" ]; then >> + # no, make sure CROSS_COMPILE isn't set >> + unset CROSS_COMPILE >> + fi >> + >> + # call the actual target script >> + do_${target} || return ${?} >> +} >> diff --git a/meta/recipes-kernel/linux/files/debian/isar/install.tmpl >> b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl >> new file mode 100644 >> index 0000000..8ccda68 >> --- /dev/null >> +++ b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl >> @@ -0,0 +1,206 @@ >> +#!/bin/bash >> +# Copyright (c) Mentor Graphics, a Siemens business, 2019 >> +# SPDX-License-Identifier: MIT >> + >> +# Load common stuff >> +. ${S}/debian/isar/common || exit ${?} >> + >> +# Stop on error >> +set -e >> + >> +do_install() { >> + >> + # check if our kernel was configured >> + if [ ! -f "${O}/.config" ]; then >> + echo "error: kernel not configured!" >&2 >> + return 1 >> + fi >> + >> + # load its configuration >> + . ${O}/.config >> + >> + kimage="$(${MAKE} O=${O} -s --no-print-directory image_name)" >> + krel="$(${MAKE} O=${O} -s --no-print-directory kernelrelease)" >> + case "${ARCH}" in >> + mips|powerpc) kimage_path="boot/vmlinux-${krel}" ;; >> + um) kimage_path="usr/bin/vmlinux-${krel}" ;; >> + *) kimage_path="boot/vmlinuz-${krel}" ;; >> + esac >> + >> + print_settings >> + >> + # Trace what we do here >> + set -x >> + >> + install_image >> + install_hooks >> + install_dtbs >> + install_kmods >> + install_headers >> + >> + # Stop tracing >> + set +x >> +} >> + >> +print_settings() { >> + cat <<EOF >> +Install settings: >> +----------------- >> +deb_dtb_dir=${deb_dtb_dir} >> +deb_hdr_dir=${deb_hdr_dir} >> +kimage=${kimage} >> +kimage_path=${kimage_path} >> + >> +EOF >> +} >> + >> +install_image() { >> + install -m 755 -d ${deb_img_dir}/$(dirname ${kimage_path}) >> + cp ${O}/${kimage} ${deb_img_dir}/${kimage_path} >> + install_image_debug >> +} >> + >> +install_image_debug() { >> + # Different tools want the image in different locations >> + # perf >> + mkdir -p ${deb_dbg_dir}/usr/lib/debug/lib/modules/${krel}/ >> + cp ${O}/vmlinux ${deb_dbg_dir}/usr/lib/debug/lib/modules/${krel}/ >> + # systemtap >> + mkdir -p ${deb_dbg_dir}/usr/lib/debug/boot/ >> + ln -s ../lib/modules/$version/vmlinux >> ${deb_dbg_dir}/usr/lib/debug/boot/vmlinux-${krel} >> + # kdump-tools >> + ln -s lib/modules/${krel}/vmlinux >> ${deb_dbg_dir}/usr/lib/debug/vmlinux-${krel} >> +} >> + >> +install_hooks() { >> + install -m 755 -d ${deb_img_dir}/etc/kernel/install.d >> + install -m 755 -d ${deb_img_dir}/etc/kernel/postinst.d >> + install -m 755 -d ${deb_img_dir}/etc/kernel/postrm.d >> + install -m 755 -d ${deb_img_dir}/etc/kernel/prerm.d >> + >> + initrd="No" >> + [ -z "${CONFIG_BLK_DEV_INITRD}" ] || initrd="Yes" >> + >> + for script in postinst postrm preinst prerm; do >> + sed -i -e "s,INITRD=[A-Za-z0-9]*,INITRD=${initrd},g" >> ${S}/debian/linux-image*.${script} >> + sed -i -e "s,version=.*,version=${krel},g" >> ${S}/debian/linux-image*.${script} >> + done >> +} >> + >> +install_dtbs() { >> + [ -n "${CONFIG_OF}" ] || return 0 >> + ${MAKE} O=${O} INSTALL_DTBS_PATH=${deb_dtb_dir} dtbs_install >> +} >> + >> +install_kmods() { >> + [ -n "${CONFIG_MODULES}" ] || return 0 >> + ${MAKE} O=${O} INSTALL_MOD_PATH=${deb_img_dir} modules_install >> + touch ${deb_img_dir}/lib/modules/${krel}/.fresh-install >> + rm -fv ${deb_img_dir}/lib/modules/${krel}/build >> + rm -fv ${deb_img_dir}/lib/modules/${krel}/source >> + install_kmods_debug >> +} >> + >> +kmods_sign() { >> + [ -n "${CONFIG_MODULE_SIG_ALL}" ] || return 0 >> + ${MAKE} O=${O} INSTALL_MOD_PATH=${deb_img_dir} modules_sign >> +} >> + >> +install_kmods_debug() { >> + [ -n "${CONFIG_DEBUG_INFO}" ] || return 0 >> + >> + kmod_inst_dir=${deb_img_dir}/lib/modules >> + kmod_debug_dir=${deb_dbg_dir}/usr/lib/debug >> + >> + # copy kernels modules to usr/lib/debug >> + mkdir -p ${kmod_debug_dir} >> + tar -C ${kmod_inst_dir}/ -cO --exclude='modules.*' . | tar -C >> ${kmod_debug_dir}/ -xf - >> + # strip everything but debug sections for modules in usr/lib/debug >> + find ${kmod_debug_dir} -name *.ko -exec ${CROSS_COMPILE}objcopy >> --only-keep-debug {} \; >> + # and strip debug sections from modules in lib/modules >> + find ${kmod_inst_dir} -name *.ko -exec ${CROSS_COMPILE}objcopy >> --strip-debug {} \; >> + >> + # re-sign stripped kernel modules >> + kmods_sign >> +} >> + >> +headers_check() { >> + ${MAKE} O=${O} headers_check >> +} >> + >> +libc_headers() { >> + mkdir -p ${deb_libc_hdr_dir} >> + ${MAKE} O=${O} headers_install >> INSTALL_HDR_PATH=${deb_libc_hdr_dir}/usr >> + host_arch=$(dpkg-architecture -a${DISTRO_ARCH} >> -qDEB_HOST_MULTIARCH) >> + mkdir ${deb_libc_hdr_dir}/usr/include/${host_arch} >> + mv ${deb_libc_hdr_dir}/usr/include/asm >> ${deb_libc_hdr_dir}/usr/include/${host_arch}/ >> +} >> + >> +# the kernel does not seem to provide a way to cross-compile its >> tools, we however >> +# have qemu-user-static binaries in our buildchroot. Create a new >> build tree for the >> +# scripts and override HOSTCC to force a cross-compile >> +kernel_tools_build() { >> + odir="${S}/build-scripts" >> + mflags="HOSTCC=${CROSS_COMPILE}gcc O=${odir} NOSTDINC_FLAGS=" >> + rm -rf ${odir} && mkdir -p ${odir} && cp >> ${S}/${KERNEL_BUILD_DIR}/.config ${odir} >> + (unset LD_PRELOAD; cd ${odir} && make -C ${S} ${mflags} >> olddefconfig scripts) >> + (cd ${odir}; find scripts -type f -executable -exec file {} >> \;|grep ELF|cut -d: -f1) >${src_hdr_files} >> + tar -C ${odir} -cf - -T - <${src_hdr_files} | tar -C ${destdir} >> -xf - >> +} >> + >> +kernel_tools() { >> + # remove object files >> + find ${destdir}/scripts -type f -name '*.o' |xargs rm -f >> + >> + # we're all done if we aren't cross-compiling >> + [ -n "${CROSS_COMPILE}" ] || return 0 >> + >> + # remove ELF executables from the linux-headers package >> + find ${destdir}/scripts -type f -exec file {} \;|grep ELF|cut >> -d: -f1|xargs rm -fv >> + >> + # cross-compile kernel tools to be shipped with linux-headers >> + kernel_tools_build >> +} >> + >> +kernel_headers() { >> + destdir=${deb_kern_hdr_dir}/${KERNEL_HEADERS_DIR} >> + src_hdr_files=$(mktemp) >> + obj_hdr_files=$(mktemp) >> + >> + mkdir -p ${destdir} >> + mkdir -p ${deb_kern_hdr_dir}/lib/modules/${krel} >> + >> + (cd ${S}; find . -name 'Makefile*' -o -name 'Kconfig*' -o -name >> '*.pl') >>${src_hdr_files} >> + (cd ${S}; find arch/*/include include scripts -type f -o -type >> l) >>${src_hdr_files} >> + (cd ${S}; find arch/${ARCH} -name module.lds -o -name >> Kbuild.platforms -o -name Platform) >>${src_hdr_files} >> + (cd ${S}; find $(find arch/${ARCH} -name include -o -name >> scripts -type d) -type f) >>${src_hdr_files} >> + >> + (cd ${O}; find arch/${ARCH}/include Module.symvers include >> scripts -type f) >>${obj_hdr_files} >> + if [ -n "${CONFIG_STACK_VALIDATION}" ]; then >> + (cd ${O}; find tools/objtool -type f -executable) >> >>${obj_hdr_files} >> + fi >> + if [ -n "${CONFIG_GCC_PLUGINS}" ]; then >> + (cd ${O}; find scripts/gcc-plugins -name *.so -o -name >> gcc-common.h) >>${obj_hdr_files} >> + fi >> + >> + # deploy files that were matched above >> + tar -C ${S} -cf - -T - <${src_hdr_files} | tar -C ${destdir} -xf - >> + tar -C ${O} -cf - -T - <${obj_hdr_files} | tar -C ${destdir} -xf - >> + >> + # add the kernel config >> + cp ${O}/${KCONF} ${destdir}/.config >> + >> + # handle kernel development tools >> + kernel_tools >> + >> + # create symlinks >> + ln -sf /${KERNEL_HEADERS_DIR} >> ${deb_kern_hdr_dir}/lib/modules/${krel}/build >> +} >> + >> +install_headers() { >> + headers_check >> + [ -z ${kern_pkgs["linux-libc-headers"]} ] || libc_headers >> + kernel_headers >> +} >> + >> +main install ${*} >> diff --git >> a/meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl >> b/meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl >> new file mode 100644 >> index 0000000..8e8636a >> --- /dev/null >> +++ b/meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl >> @@ -0,0 +1,29 @@ >> +#!/bin/sh >> +# based on >> https://salsa.debian.org/kernel-team/linux/blob/master/debian/templates/image.postinst.in >> + >> +# Tell initramfs builder whether it's wanted >> +export INITRD=Yes >> + >> +version=${PV} >> +image_path=/boot/${KERNEL_FILE}-${version} >> + >> +if [ "$1" != configure ]; then >> + exit 0 >> +fi >> + >> +depmod $version >> + >> +if [ -f /lib/modules/$version/.fresh-install ]; then >> + change=install >> +else >> + change=upgrade >> +fi >> +linux-update-symlinks $change $version $image_path >> +rm -f /lib/modules/$version/.fresh-install >> + >> +if [ -d /etc/kernel/postinst.d ]; then >> + DEB_MAINT_PARAMS="$*" run-parts --report --exit-on-error >> --arg=$version \ >> + --arg=$image_path /etc/kernel/postinst.d >> +fi >> + >> +exit 0 >> diff --git >> a/meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl >> b/meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl >> new file mode 100644 >> index 0000000..4d8a7d9 >> --- /dev/null >> +++ b/meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl >> @@ -0,0 +1,35 @@ >> +#!/bin/sh -e >> +# based on >> https://salsa.debian.org/kernel-team/linux/blob/master/debian/templates/image.postrm.in >> + >> +# Tell initramfs builder whether it's wanted >> +export INITRD=Yes >> + >> +version=${PV} >> +image_path=/boot/${KERNEL_FILE}-${version} >> + >> +rm -f /lib/modules/$version/.fresh-install >> + >> +if [ "$1" != upgrade ] && command -v linux-update-symlinks >> >/dev/null; then >> + linux-update-symlinks remove $version $image_path >> +fi >> + >> +if [ -d /etc/kernel/postrm.d ]; then >> + DEB_MAINT_PARAMS="$*" run-parts --report --exit-on-error >> --arg=$version \ >> + --arg=$image_path /etc/kernel/postrm.d >> +fi >> + >> +if [ "$1" = purge ]; then >> + for extra_file in modules.dep modules.isapnpmap modules.pcimap \ >> + modules.usbmap modules.parportmap \ >> + modules.generic_string modules.ieee1394map \ >> + modules.ieee1394map modules.pnpbiosmap \ >> + modules.alias modules.ccwmap modules.inputmap \ >> + modules.symbols modules.ofmap \ >> + modules.seriomap modules.\*.bin \ >> + modules.softdep modules.devname; do >> + eval rm -f /lib/modules/$version/$extra_file >> + done >> + rmdir /lib/modules/$version || true >> +fi >> + >> +exit 0 >> diff --git >> a/meta/recipes-kernel/linux/files/debian/linux-image.preinst.tmpl >> b/meta/recipes-kernel/linux/files/debian/linux-image.preinst.tmpl >> new file mode 100644 >> index 0000000..7465197 >> --- /dev/null >> +++ b/meta/recipes-kernel/linux/files/debian/linux-image.preinst.tmpl >> @@ -0,0 +1,25 @@ >> +#!/bin/sh -e >> +# based on >> https://salsa.debian.org/kernel-team/linux/blob/master/debian/templates/image.preinst.in >> + >> +# Tell initramfs builder whether it's wanted >> +export INITRD=Yes >> + >> +version=${PV} >> +image_path=/boot/${KERNEL_FILE}-${version} >> + >> +if [ "$1" = abort-upgrade ]; then >> + exit 0 >> +fi >> + >> +if [ "$1" = install ]; then >> + # Create a flag file for postinst >> + mkdir -p /lib/modules/$version >> + touch /lib/modules/$version/.fresh-install >> +fi >> + >> +if [ -d /etc/kernel/preinst.d ]; then >> + DEB_MAINT_PARAMS="$*" run-parts --report --exit-on-error >> --arg=$version \ >> + --arg=$image_path /etc/kernel/preinst.d >> +fi >> + >> +exit 0 >> diff --git >> a/meta/recipes-kernel/linux/files/debian/linux-image.prerm.tmpl >> b/meta/recipes-kernel/linux/files/debian/linux-image.prerm.tmpl >> new file mode 100644 >> index 0000000..1992b32 >> --- /dev/null >> +++ b/meta/recipes-kernel/linux/files/debian/linux-image.prerm.tmpl >> @@ -0,0 +1,21 @@ >> +#!/bin/sh -e >> +# based on >> https://salsa.debian.org/kernel-team/linux/blob/master/debian/templates/image.prerm.in >> + >> +# Tell initramfs builder whether it's wanted >> +export INITRD=Yes >> + >> +version=${PV} >> +image_path=/boot/${KERNEL_FILE}-${version} >> + >> +if [ "$1" != remove ]; then >> + exit 0 >> +fi >> + >> +linux-check-removal $version >> + >> +if [ -d /etc/kernel/prerm.d ]; then >> + DEB_MAINT_PARAMS="$*" run-parts --report --exit-on-error >> --arg=$version \ >> + --arg=$image_path /etc/kernel/prerm.d >> +fi >> + >> +exit 0 >> diff --git a/meta/recipes-kernel/linux/files/debian/rules.tmpl >> b/meta/recipes-kernel/linux/files/debian/rules.tmpl >> new file mode 100755 >> index 0000000..c024768 >> --- /dev/null >> +++ b/meta/recipes-kernel/linux/files/debian/rules.tmpl >> @@ -0,0 +1,39 @@ >> +#!/usr/bin/make -f >> + >> +CROSS_COMPILE:=$(DEB_HOST_GNU_TYPE)- >> + >> +O:=$(CURDIR)/${KERNEL_BUILD_DIR} >> +S:=$(CURDIR) >> +deb_top_dir:=$(S)/debian >> + >> +# Dynamic variables to be passed to Isar build scripts >> +isar_env=$(strip \ >> + export CROSS_COMPILE='$(CROSS_COMPILE)' && \ >> + export DEB_BUILD_PROFILES='$(DEB_BUILD_PROFILES)' && \ >> + export DEB_HOST_GNU_TYPE='$(DEB_HOST_GNU_TYPE)' && \ >> + export MAKE='$(MAKE)' && \ >> + export O='${O}' \ >> + export S='${S}' \ >> +) >> + >> +%: >> + dh $(@) >> + >> +.PHONY: override_dh_auto_clean >> +override_dh_auto_clean: >> + $(isar_env) && bash $(deb_top_dir)/isar/clean >> + >> +.PHONY: override_dh_auto_build >> +override_dh_auto_build: >> + $(isar_env) && bash $(deb_top_dir)/isar/build >> + >> +override_dh_auto_install: >> + $(isar_env) && bash $(deb_top_dir)/isar/install >> + >> +.PHONY: override_dh_auto_test >> +override_dh_auto_test: >> + true >> + >> +PHONY: override_dh_strip >> +override_dh_strip: >> + dh_strip -Xvmlinux --no-automatic-dbgsym >> diff --git a/meta/recipes-kernel/linux/linux-custom.inc >> b/meta/recipes-kernel/linux/linux-custom.inc >> index ca91f64..ecf0e66 100644 >> --- a/meta/recipes-kernel/linux/linux-custom.inc >> +++ b/meta/recipes-kernel/linux/linux-custom.inc >> @@ -2,22 +2,107 @@ >> # >> # This software is a part of ISAR. >> # Copyright (c) Siemens AG, 2018 >> +# Copyright (c) Mentor Graphics, a Siemens business, 2019 >> # >> # SPDX-License-Identifier: MIT >> -FILESEXTRAPATHS_prepend := "${FILE_DIRNAME}/files:" >> +# Settings that would typically be done from the custom kernel recipe >> +# ------------------------------------------------------------------- >> DESCRIPTION ?= "Custom kernel" >> +MAINTAINER ?= "isar-users <isar-users@googlegroups.com>" >> + >> +KBUILD_DEPENDS ?= "build-essential:native, \ >> + libelf-dev:native, \ >> + libncurses-dev:native, \ >> + libssl-dev:native, \ >> + bc, \ >> + bison, \ >> + cpio, \ >> + flex, \ >> + git, \ >> + kmod, \ >> + libssl-dev," >> + >> +KERNEL_DEBIAN_DEPENDS ?= "initramfs-tools | linux-initramfs-tool, \ >> + kmod, \ >> + linux-base (>= 4.3~)," >> + >> +KERNEL_HEADERS_DEBIAN_DEPENDS ?= "libc6, \ >> + libssl1.1," >> + >> +KERNEL_LIBC_DEV_DEPLOY ?= "0" >> + >> +# Settings that may be changed on a per distro, machine or layer basis >> +# -------------------------------------------------------------------- >> + >> +KERNEL_DTB_DIR ?= "usr/lib/linux-${KERNEL_NAME_PROVIDED}-${PV}" > > Here is the bug that breaks U-boot setups here: Must be > linux-image-${PV} in order to stay compatible with Debian. > > Did you compare some content of the new packages against that of the > original package build? > Yes but I had not anticipated this one as being an issue (so thanks for pointing that out). I have limited hardware available here (mostly Siemens IPCs or non supported boards such as Xilinx ref boards). will check if I can get remote access to some of the boards supported by Isar. I have made the change locally and started a build. I hope I can check it out on the de0-nano-soc board that we have in meta-isar > Jan > >> +KERNEL_HEADERS_DIR ?= "usr/src/linux-headers-${PV}" >> -KERNEL_NAME_PROVIDED ?= "${@ d.getVar('PN', >> True).partition('linux-')[2]}" >> KERNEL_DEFCONFIG ?= "" >> +# Add our template meta-data to the sources >> +FILESEXTRAPATHS_prepend := "${FILE_DIRNAME}/files:" >> +SRC_URI += "file://debian" >> + >> +# Variables and files that make our templates >> +# ------------------------------------------- >> + >> +TEMPLATE_FILES += " \ >> + debian/control.tmpl \ >> + debian/isar/build.tmpl \ >> + debian/isar/clean.tmpl \ >> + debian/isar/common.tmpl \ >> + debian/isar/install.tmpl \ >> + debian/linux-image.postinst.tmpl \ >> + debian/linux-image.postrm.tmpl \ >> + debian/linux-image.preinst.tmpl \ >> + debian/linux-image.prerm.tmpl \ >> + debian/rules.tmpl \ >> +" >> + >> +TEMPLATE_VARS += " \ >> + HOST_ARCH \ >> + KBUILD_DEPENDS \ >> + KERNEL_ARCH \ >> + KERNEL_DEBIAN_DEPENDS \ >> + KERNEL_BUILD_DIR \ >> + KERNEL_DTB_DIR \ >> + KERNEL_FILE \ >> + KERNEL_HEADERS_DEBIAN_DEPENDS \ >> + KERNEL_HEADERS_DIR \ >> + KERNEL_NAME_PROVIDED \ >> +" >> + >> +inherit dpkg >> +inherit template >> + >> +# Derive name of the kernel packages from the name of this recipe >> +KERNEL_NAME_PROVIDED ?= "${@ d.getVar('PN', >> True).partition('linux-')[2]}" >> + >> +# Make bitbake know we will be producing linux-image and >> linux-headers packages >> python() { >> kernel_name = d.getVar("KERNEL_NAME_PROVIDED", True) >> d.setVar('PROVIDES', 'linux-image-' + kernel_name + ' ' + \ >> 'linux-headers-' + kernel_name) >> } >> +def get_kernel_arch(d): >> + distro_arch = d.getVar("DISTRO_ARCH") >> + if distro_arch == "amd64": >> + kernel_arch = "x86" >> + elif distro_arch == "arm64": >> + kernel_arch = "arm64" >> + elif distro_arch == "armhf": >> + kernel_arch = "arm" >> + elif distro_arch == "mipsel": >> + kernel_arch = "mips" >> + else: >> + kernel_arch = "" >> + return kernel_arch >> + >> +KERNEL_ARCH ??= "${@get_kernel_arch(d)}" >> + >> def config_fragments(d): >> fragments = [] >> sources = d.getVar("SRC_URI").split() >> @@ -31,57 +116,37 @@ def config_fragments(d): >> fragments.append(local) >> return fragments >> -inherit dpkg-base >> - >> -SRC_URI += "file://build-kernel.sh" >> +do_prepare_build_prepend() { >> + # copy meta-data over to source tree >> + rm -rf ${S}/debian >> + cp -r ${WORKDIR}/debian ${S}/ >> -KBUILD_DEPENDS ?= " \ >> - build-essential:native \ >> - libssl-dev \ >> - libelf-dev \ >> - bc \ >> - git \ >> - kmod \ >> - bison \ >> - flex \ >> - cpio \ >> - libncurses-dev" >> -KERNEL_DEBIAN_DEPENDS ?= "initramfs-tools | linux-initramfs-tool, >> kmod, linux-base (>= 4.3~)" >> -KERNEL_HEADERS_DEBIAN_DEPENDS ?= "libc6, libssl1.1" >> + # remove templates from the source tree >> + find ${S}/debian -name *.tmpl | xargs rm -f >> -KERNEL_LIBC_DEV_DEPLOY ?= "0" >> + # rename install/remove hooks to match user-specified name for >> our linux-image package >> + mv ${S}/debian/linux-image.postinst >> ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.postinst >> + mv ${S}/debian/linux-image.postrm >> ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.postrm >> + mv ${S}/debian/linux-image.preinst >> ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.preinst >> + mv ${S}/debian/linux-image.prerm >> ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.prerm >> -do_install_builddeps() { >> - dpkg_do_mounts >> - E="${@ bb.utils.export_proxies(d)}" >> - sudo -E chroot ${BUILDCHROOT_DIR} \ >> - apt-get update \ >> - -o Dir::Etc::SourceList="sources.list.d/isar-apt.list" \ >> - -o Dir::Etc::SourceParts="-" \ >> - -o APT::Get::List-Cleanup="0" >> - sudo -E chroot ${BUILDCHROOT_DIR} \ >> - apt-get install \ >> - -y -o Debug::pkgProblemResolver=yes \ >> - --no-install-recommends ${KBUILD_DEPENDS} >> - dpkg_undo_mounts >> + # produce a changelog for our kernel build >> + deb_add_changelog >> } >> -addtask install_builddeps after do_prepare_build before do_dpkg_build >> -# apt and reprepro may not run in parallel, acquire the Isar lock >> -do_install_builddeps[lockfiles] += "${REPO_ISAR_DIR}/isar.lock" >> - >> -addtask devshell after do_install_builddeps >> +# build directory for our "full" kernel build >> +KERNEL_BUILD_DIR = "build-full" >> -dpkg_runbuild() { >> - chmod +x ${WORKDIR}/build-kernel.sh >> - KERNEL_CONFIG_TARGET="${KERNEL_DEFCONFIG}" >> +dpkg_configure_kernel() { >> + config_target="${KERNEL_DEFCONFIG}" >> + rm -rf ${S}/${KERNEL_BUILD_DIR} && mkdir -p >> ${S}/${KERNEL_BUILD_DIR} >> if [ -n "${KERNEL_DEFCONFIG}" ]; then >> if [ -e "${WORKDIR}/${KERNEL_DEFCONFIG}" ]; then >> - cp ${WORKDIR}/${KERNEL_DEFCONFIG} ${S}/.config >> - KERNEL_CONFIG_TARGET="olddefconfig" >> + cp ${WORKDIR}/${KERNEL_DEFCONFIG} >> ${S}/${KERNEL_BUILD_DIR}/.config >> + config_target="olddefconfig" >> fi >> else >> - KERNEL_CONFIG_TARGET="defconfig" >> + config_target="defconfig" >> fi >> # copy config fragments over to the kernel tree >> @@ -92,22 +157,23 @@ dpkg_runbuild() { >> mkdir -p ${S}/debian/fragments >> (cd ${WORKDIR} && cp ${src_frags} ${S}/debian/fragments/) >> fi >> - export KERNEL_FRAGMENTS="${out_frags}" >> - E="${@ bb.utils.export_proxies(d)}" >> - >> - export PV=${PV} >> - export KERNEL_NAME=${KERNEL_NAME_PROVIDED} >> - >> - export KBUILD_DEPENDS="${KBUILD_DEPENDS}" >> - export KERNEL_CONFIG_TARGET >> - export KERNEL_DEBIAN_DEPENDS="${KERNEL_DEBIAN_DEPENDS}" >> - export KERNEL_FILE="${KERNEL_FILE}" >> - export >> KERNEL_HEADERS_DEBIAN_DEPENDS="${KERNEL_HEADERS_DEBIAN_DEPENDS}" >> - >> - sudo -E chroot --userspec=$( id -u ):$( id -g ) >> ${BUILDCHROOT_DIR} ${PP}/build-kernel.sh ${PP}/${PPS} ${DISTRO_ARCH} >> + sudo -E chroot --userspec=$(id -u):$(id -g) ${BUILDCHROOT_DIR} >> sh -c " \ >> + export ARCH=${KERNEL_ARCH} >> && \ >> + cd ${PP}/${PPS} >> && \ >> + make O=${KERNEL_BUILD_DIR} ${config_target} >> && \ >> + ./scripts/kconfig/merge_config.sh \ >> + -O ${KERNEL_BUILD_DIR}/ \ >> + ${KERNEL_BUILD_DIR}/.config \ >> + ${out_frags} \ >> + " >> +} >> +dpkg_runbuild_prepend() { >> + profiles="${BASE_DISTRO}" >> if [ "${KERNEL_LIBC_DEV_DEPLOY}" != "1" ]; then >> - rm -f ${WORKDIR}/linux-libc-dev_${PV}*.deb >> + profiles="${profiles} nolibcdev" >> fi >> + export DEB_BUILD_PROFILES="${profiles}" >> + dpkg_configure_kernel >> } >> > > ^ permalink raw reply [flat|nested] 83+ messages in thread
* Re: [PATCH L-C v3 7/7] linux-custom: rewrite to no longer depend on the kernel's builddeb 2019-11-08 7:36 ` Cedric Hombourger @ 2019-11-08 7:41 ` Jan Kiszka 0 siblings, 0 replies; 83+ messages in thread From: Jan Kiszka @ 2019-11-08 7:41 UTC (permalink / raw) To: Cedric Hombourger, isar-users On 08.11.19 08:36, Cedric Hombourger wrote: > > On 11/8/19 8:19 AM, Jan Kiszka wrote: >> On 08.11.19 08:07, Cedric Hombourger wrote: >>> Regain control over the packaging of the Linux kernel by providing our >>> own debian recipes (debian,{control,rules} files and friends) instead >>> of using the kernel's builddeb script. This will allow generation of >>> packages for "perf" (not included in this changeset) with per distro >>> scheme (Debian and Ubuntu have differences) and maintain compatibility >>> with upstream packages generated from different source packages but >>> dependent on a certain deployment scheme (e.g. linux-base for Debian) >>> >>> Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com> >>> --- >>> doc/custom_kernel.md | 80 +++++++ >>> .../linux/files/build-kernel.sh | 128 ----------- >>> meta/recipes-kernel/linux/files/debian/compat | 1 + >>> .../linux/files/debian/control.tmpl | 37 ++++ >>> .../linux/files/debian/isar/build.tmpl | 45 ++++ >>> .../linux/files/debian/isar/clean.tmpl | 21 ++ >>> .../linux/files/debian/isar/common.tmpl | 50 +++++ >>> .../linux/files/debian/isar/install.tmpl | 206 ++++++++++++++++++ >>> .../files/debian/linux-image.postinst.tmpl | 29 +++ >>> .../files/debian/linux-image.postrm.tmpl | 35 +++ >>> .../files/debian/linux-image.preinst.tmpl | 25 +++ >>> .../linux/files/debian/linux-image.prerm.tmpl | 21 ++ >>> .../linux/files/debian/rules.tmpl | 39 ++++ >>> meta/recipes-kernel/linux/linux-custom.inc | 180 ++++++++++----- >>> 14 files changed, 712 insertions(+), 185 deletions(-) >>> create mode 100644 doc/custom_kernel.md >>> delete mode 100644 meta/recipes-kernel/linux/files/build-kernel.sh >>> create mode 100644 meta/recipes-kernel/linux/files/debian/compat >>> create mode 100644 meta/recipes-kernel/linux/files/debian/control.tmpl >>> create mode 100644 >>> meta/recipes-kernel/linux/files/debian/isar/build.tmpl >>> create mode 100644 >>> meta/recipes-kernel/linux/files/debian/isar/clean.tmpl >>> create mode 100644 >>> meta/recipes-kernel/linux/files/debian/isar/common.tmpl >>> create mode 100644 >>> meta/recipes-kernel/linux/files/debian/isar/install.tmpl >>> create mode 100644 >>> meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl >>> create mode 100644 >>> meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl >>> create mode 100644 >>> meta/recipes-kernel/linux/files/debian/linux-image.preinst.tmpl >>> create mode 100644 >>> meta/recipes-kernel/linux/files/debian/linux-image.prerm.tmpl >>> create mode 100755 meta/recipes-kernel/linux/files/debian/rules.tmpl >>> >>> diff --git a/doc/custom_kernel.md b/doc/custom_kernel.md >>> new file mode 100644 >>> index 0000000..31319d4 >>> --- /dev/null >>> +++ b/doc/custom_kernel.md >>> @@ -0,0 +1,80 @@ >>> +# Custom kernel recipe for Isar >>> + >>> +## Contents >>> + >>> + - [Summary](#summary) >>> + - [Features](#features) >>> + - [Future](#future) >>> + - [Examples](#examples) >>> + >>> +## Summary >>> + >>> +Isar provides a recipe to build custom kernels for Debian-based >>> distributions. >>> +It uses templates to generate the debian meta-data (such as >>> debian/control) and >>> +Debian's [BuildProfiles](https://wiki.debian.org/BuildProfileSpec) >>> to handle >>> +some of the distro specific variations. It should be noted that Isar >>> has moved >>> +away from using the kernel's builddeb script since it would not >>> generate all >>> +the packages we need (and in particular perf). >>> + >>> +## Features >>> + >>> +The linux-custom recipe provides support for: >>> + >>> + 1. Sources to the custom Linux kernel may be specified via `SRC_URI` >>> + >>> + 2. Configure the kernel via an in-tree or an external `defconfig` via >>> + `KERNEL_DEFCONFIG` >>> + >>> + 3. Integrate kernel configuration tweaks via configuration >>> fragments (`.cfg` >>> + files) >>> + >>> + 4. Patches to the linux kernel may be specified via `SRC_URI` >>> + >>> + 5. Ensure that the Isar recipe `PV` matches the kernel release >>> identifier >>> + (`KERNEL_RELEASE`) >>> + >>> + 6. Produce a `linux-image` package that ships the kernel image and >>> modules >>> + >>> + 7. Allow the name of the kernel image to be changed via >>> `KERNEL_FILE` (defaults >>> + to `vmlinuz`) >>> + >>> + 8. Produce a `linux-headers` package which includes kernel headers >>> and kbuild >>> + scripts/tools >>> + >>> + 9. The `linux-headers` package shall support native and cross >>> compiles of >>> + out-of-tree kernel modules >>> + >>> + 10. Produce a `linux-libc-dev` package to support user-land builds >>> + >>> + 11. Only build/ship the `linux-libc-dev` package if instructed to >>> + (`KERNEL_LIBC_DEV_DEPLOY` equals to `"1"`) >>> + >>> + 12. Support both native and cross compiles (`ISAR_CROSS_COMPILE`) >>> + >>> + 13. Support for the following kernel architectures: >>> + >>> + * arm >>> + * arm64 >>> + * mips >>> + * x86 >>> + >>> + 14. Support `devshell` (kernel configuration shall be applied) >>> + >>> +## Future >>> + >>> +In the future, the recipe may be extended to: >>> + >>> + 1. Package perf >>> + >>> + 2. Support inclusion/build of dts files listed in `SRC_URI` >>> + >>> + 3. Be compatible with Ubuntu >>> + >>> +## Examples >>> + >>> +The linux-custom recipe is currently used by the linux-mainline >>> package and is >>> +used mainline recipe may be used for some basic testing. This recipe >>> is being >>> +used by the following machines: >>> + >>> + * de0-nano-soc >>> + * qemumipsel >>> diff --git a/meta/recipes-kernel/linux/files/build-kernel.sh >>> b/meta/recipes-kernel/linux/files/build-kernel.sh >>> deleted file mode 100644 >>> index f56e96c..0000000 >>> --- a/meta/recipes-kernel/linux/files/build-kernel.sh >>> +++ /dev/null >>> @@ -1,128 +0,0 @@ >>> -#!/bin/bash >>> -# >>> -# Custom kernel build >>> -# >>> -# This software is a part of ISAR. >>> -# Copyright (c) Siemens AG, 2018 >>> -# >>> -# SPDX-License-Identifier: MIT >>> - >>> -source /isar/common.sh >>> - >>> -host_arch=$(dpkg --print-architecture) >>> - >>> -if [ "$host_arch" != "$target_arch" ]; then >>> - case $target_arch in >>> - armhf) >>> - export ARCH=arm >>> - export CROSS_COMPILE="arm-linux-gnueabihf-" >>> - ;; >>> - arm64) >>> - export ARCH=arm64 >>> - export CROSS_COMPILE="aarch64-linux-gnu-" >>> - ;; >>> - mipsel) >>> - export ARCH=mips >>> - export CROSS_COMPILE="mipsel-linux-gnu-" >>> - ;; >>> - *) >>> - echo "error: unsupported architecture ($target_arch)" >>> - exit 1 >>> - ;; >>> - esac >>> -fi >>> - >>> -REPACK_DIR="$1/../repack" >>> -REPACK_LINUX_IMAGE_DIR="${REPACK_DIR}/linux-image" >>> -REPACK_LINUX_HEADERS_DIR="${REPACK_DIR}/linux-headers" >>> - >>> -make ${KERNEL_CONFIG_TARGET} || exit ${?} >>> -if [ -n "${KERNEL_FRAGMENTS}" ]; then >>> - scripts/kconfig/merge_config.sh -m .config ${KERNEL_FRAGMENTS} >>> -fi >>> - >>> -KV=$( make -s kernelrelease ) >>> -if [ "${KV}" != "${PV}" ]; then >>> - echo "ERROR: Recipe PV is \"${PV}\" but should be \"${KV}\"" 1>&2 >>> - echo "ERROR: Probably due to CONFIG_LOCALVERSION" 1>&2 >>> - exit 1 >>> -fi >>> - >>> -rm -f .version >>> -KBUILD_DEBARCH=$target_arch make -j $(($(nproc) * 2)) deb-pkg >>> - >>> -rm -rf "${REPACK_DIR}" >>> -mkdir -p "${REPACK_DIR}" >>> -mkdir -p "${REPACK_LINUX_IMAGE_DIR}" >>> -mkdir -p "${REPACK_LINUX_HEADERS_DIR}" >>> - >>> -cp -a debian "${REPACK_DIR}" >>> - >>> -# dpkg-gencontrol performs cross-incompatible checks on the >>> -# Architecture field; trick it to accept the control file >>> -sed -i "s/Architecture: .*/Architecture: any/" >>> "${REPACK_DIR}/debian/control" >>> - >>> -cd .. >>> - >>> -dpkg-deb -R linux-image-${PV}_${PV}-1_*.deb "${REPACK_LINUX_IMAGE_DIR}" >>> -dpkg-deb -R linux-headers-${PV}_${PV}-1_*.deb >>> "${REPACK_LINUX_HEADERS_DIR}" >>> - >>> -dpkg-gencontrol -crepack/debian/control \ >>> - -lrepack/debian/changelog \ >>> - -frepack/debian/files \ >>> - -plinux-image-${PV} \ >>> - -P"${REPACK_LINUX_IMAGE_DIR}" \ >>> - -DPackage="linux-image-${KERNEL_NAME}" \ >>> - -DSection=kernel \ >>> - -DPriority=required \ >>> - -DDepends="${KERNEL_DEBIAN_DEPENDS}" \ >>> - -DArchitecture=$target_arch >>> - >>> -# Add Debian-like link installation to postinst >>> -mkdir -p ${REPACK_LINUX_IMAGE_DIR}/lib/modules/${PV} >>> -touch "${REPACK_LINUX_IMAGE_DIR}/lib/modules/${PV}/.fresh-install" >>> -sed -i "${REPACK_LINUX_IMAGE_DIR}/DEBIAN/postinst" \ >>> - -e "/^set -e$/a\\ >>> -\\ >>> -if [ -f /lib/modules/${PV}/.fresh-install ]; then\\ >>> - change=install\\ >>> -else\\ >>> - change=upgrade\\ >>> -fi\\ >>> -linux-update-symlinks \$change ${PV} /boot/${KERNEL_FILE}-${PV}\\ >>> -rm -f /lib/modules/${PV}/.fresh-install" >>> - >>> -# Add Debian-like link removal to postrm >>> -sed -i "${REPACK_LINUX_IMAGE_DIR}/DEBIAN/postrm" \ >>> - -e "/^set -e$/a\\ >>> -\\ >>> -rm -f /lib/modules/${PV}/.fresh-install\\ >>> -\\ >>> -if [ \"\$1\" != upgrade ] && command -v linux-update-symlinks >>> >/dev/null; then\\ >>> - linux-update-symlinks remove ${PV} /boot/${KERNEL_FILE}-${PV}\\ >>> -fi" >>> - >>> -# Make sure arm64 kernels are decompressed >>> -if [ "$target_arch" = "arm64" ]; then >>> - kernel_file="${REPACK_LINUX_IMAGE_DIR}/boot/${KERNEL_FILE}-${PV}" >>> - mv "${kernel_file}" "${kernel_file}.gz" >>> - gunzip "${kernel_file}.gz" >>> -fi >>> - >>> -dpkg-gencontrol -crepack/debian/control \ >>> - -lrepack/debian/changelog \ >>> - -frepack/debian/files \ >>> - -plinux-headers-${PV} \ >>> - -P"${REPACK_LINUX_HEADERS_DIR}" \ >>> - -Vkernel:debarch="${KERNEL_NAME}" \ >>> - -DPackage="linux-headers-${KERNEL_NAME}" \ >>> - -DSection=kernel \ >>> - -DDepends="${KERNEL_HEADERS_DEBIAN_DEPENDS}" \ >>> - -DArchitecture=$target_arch >>> - >>> -fakeroot dpkg-deb -b "${REPACK_LINUX_IMAGE_DIR}" \ >>> - linux-image-${KERNEL_NAME}_${PV}-1_${KERNEL_NAME}.deb >>> -rm -f linux-image-${PV}_${PV}-1_*.deb >>> -fakeroot dpkg-deb -b "${REPACK_LINUX_HEADERS_DIR}" \ >>> - linux-headers-${KERNEL_NAME}_${PV}-1_${KERNEL_NAME}.deb >>> -rm -f linux-headers-${PV}_${PV}-1_*.deb >>> diff --git a/meta/recipes-kernel/linux/files/debian/compat >>> b/meta/recipes-kernel/linux/files/debian/compat >>> new file mode 100644 >>> index 0000000..ec63514 >>> --- /dev/null >>> +++ b/meta/recipes-kernel/linux/files/debian/compat >>> @@ -0,0 +1 @@ >>> +9 >>> diff --git a/meta/recipes-kernel/linux/files/debian/control.tmpl >>> b/meta/recipes-kernel/linux/files/debian/control.tmpl >>> new file mode 100644 >>> index 0000000..d55096b >>> --- /dev/null >>> +++ b/meta/recipes-kernel/linux/files/debian/control.tmpl >>> @@ -0,0 +1,37 @@ >>> +Source: linux-${KERNEL_NAME_PROVIDED} >>> +Section: kernel >>> +Priority: optional >>> +Maintainer: ${MAINTAINER} >>> +Build-Depends: bc, kmod, cpio, ${KBUILD_DEPENDS} >>> +Homepage: http://www.kernel.org/ >>> + >>> +Package: linux-image-${KERNEL_NAME_PROVIDED} >>> +Architecture: any >>> +Depends: ${KERNEL_DEBIAN_DEPENDS} >>> +Description: ${KERNEL_NAME_PROVIDED} Linux kernel, version ${PV} >>> + This package contains the Linux kernel, modules and corresponding >>> other >>> + files, version: ${PV}. >>> + >>> +Package: linux-headers-${KERNEL_NAME_PROVIDED} >>> +Architecture: any >>> +Depends: ${KERNEL_HEADERS_DEBIAN_DEPENDS}, ${perl:Depends}, >>> ${shlib:Depends} >>> +Description: ${KERNEL_NAME_PROVIDED} Linux kernel headers for ${PV} >>> + This package provides kernel header files for ${PV} on ${DISTRO_ARCH} >>> + . >>> + This is useful for people who need to build external modules >>> + >>> +Package: linux-libc-dev >>> +Build-Profiles: <!nolibcdev> >>> +Section: devel >>> +Provides: linux-kernel-headers >>> +Architecture: any >>> +Description: Linux support headers for userspace development >>> + This package provides userspaces headers from the Linux kernel. >>> These headers >>> + are used by the installed headers for GNU glibc and other system >>> libraries. >>> + >>> +Package: linux-image-${KERNEL_NAME_PROVIDED}-dbg >>> +Section: debug >>> +Architecture: any >>> +Description: Linux kernel debugging symbols for ${PV} >>> + This package will come in handy if you need to debug the kernel. It >>> provides >>> + all the necessary debug symbols for the kernel and its modules. >>> diff --git a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl >>> b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl >>> new file mode 100644 >>> index 0000000..4ea18d5 >>> --- /dev/null >>> +++ b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl >>> @@ -0,0 +1,45 @@ >>> +#!/bin/bash >>> +# Copyright (c) Mentor Graphics, a Siemens business, 2019 >>> +# SPDX-License-Identifier: MIT >>> + >>> +# Load common stuff >>> +. ${S}/debian/isar/common || exit ${?} >>> + >>> +do_build() { >>> + >>> + # Print a few things that are of particular interest >>> + print_settings >>> + >>> + # Process existing kernel configuration to make sure it is complete >>> + # (use defaults for options that were not specified) >>> + ${MAKE} O=${KERNEL_BUILD_DIR} olddefconfig prepare || exit ${?} >>> + >>> + # Check if the recipe's PV makes sense >>> + KR=$(${MAKE} O=${KERNEL_BUILD_DIR} -s --no-print-directory >>> kernelrelease) >>> + eval $(grep ^CONFIG_LOCALVERSION= ${KERNEL_BUILD_DIR}/${KCONF} >>> || true) >>> + if [ "${PV}" != "${KR}" -a "${PV}${CONFIG_LOCALVERSION}" != >>> "${KR}" ]; then >>> + echo "ERROR: Recipe PV (${PV}) does not seem to match the >>> kernelrelease (${KR})!" 1>&2 >>> + echo "ERROR: Make sure the kernel version in your PV setting >>> and/or CONFIG_LOCALVERSION are aligned" 1>&2 >>> + exit 1 >>> + fi >>> + >>> + # Trace what we do here >>> + set -x >>> + >>> + # Build the Linux kernel >>> + ${MAKE} O=${KERNEL_BUILD_DIR} -j $(nproc) || exit ${?} >>> + >>> + # Stop tracing >>> + set +x >>> +} >>> + >>> +print_settings() { >>> + cat <<EOF >>> +# Build settings: >>> +# --------------- >>> +# ARCH=${ARCH} >>> +# CROSS_COMPILE=${CROSS_COMPILE} >>> +EOF >>> +} >>> + >>> +main build ${*} >>> diff --git a/meta/recipes-kernel/linux/files/debian/isar/clean.tmpl >>> b/meta/recipes-kernel/linux/files/debian/isar/clean.tmpl >>> new file mode 100644 >>> index 0000000..f903987 >>> --- /dev/null >>> +++ b/meta/recipes-kernel/linux/files/debian/isar/clean.tmpl >>> @@ -0,0 +1,21 @@ >>> +#!/bin/bash >>> +# Copyright (c) Mentor Graphics, a Siemens business, 2019 >>> +# SPDX-License-Identifier: MIT >>> + >>> +# Load common stuff >>> +. ${S}/debian/isar/common || exit ${?} >>> + >>> +do_clean() { >>> + >>> + # Trace what we do here >>> + set -x >>> + >>> + rm -rf ${deb_img_dir} ${deb_dbg_dir} >>> + rm -rf ${deb_kern_hdrdir} >>> + rm -rf ${deb_libc_hdr_dir} >>> + >>> + # Stop tracing >>> + set +x >>> +} >>> + >>> +main clean ${*} >>> diff --git a/meta/recipes-kernel/linux/files/debian/isar/common.tmpl >>> b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl >>> new file mode 100644 >>> index 0000000..44f67b3 >>> --- /dev/null >>> +++ b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl >>> @@ -0,0 +1,50 @@ >>> +#!/bin/bash >>> +# Copyright (c) Mentor Graphics, a Siemens business, 2019 >>> +# SPDX-License-Identifier: MIT >>> + >>> +# Isar settings >>> +ARCH=${KERNEL_ARCH} >>> +KERNEL_PKG_IMAGE=linux-image-${KERNEL_NAME_PROVIDED} >>> +KERNEL_PKG_KERN_HEADERS=linux-headers-${KERNEL_NAME_PROVIDED} >>> +KERNEL_PKG_LIBC_HEADERS=linux-libc-dev >>> + >>> +# Constants >>> +KCONF=.config >>> + >>> +# Target directories >>> +deb_top_dir=${S}/debian >>> +deb_img_dir=${deb_top_dir}/${KERNEL_PKG_IMAGE} >>> +deb_dbg_dir=${deb_img_dir}-dbg >>> +deb_dtb_dir=${deb_img_dir}/${KERNEL_DTB_DIR} >>> +deb_kern_hdr_dir=${deb_top_dir}/${KERNEL_PKG_KERN_HEADERS} >>> +deb_libc_hdr_dir=${deb_top_dir}/${KERNEL_PKG_LIBC_HEADERS} >>> + >>> +# Array of packages to be generated >>> +declare -A kern_pkgs >>> + >>> +main() { >>> + local target=${1} >>> + >>> + if [ ! -f ${S}/debian/isar/${target} ]; then >>> + echo "error: ${target} is not a supported build target!" >&2 >>> + return 1 >>> + fi >>> + >>> + # create do_<x> variables for each package to be generated >>> + for p in $(unset DEB_HOST_ARCH; dh_listpackages); do >>> + kern_pkgs["${p}"]="1" >>> + done >>> + >>> + # variables to be exported >>> + export ARCH >>> + >>> + # are we cross-compiling? >>> + BUILD_ARCH=$(dpkg-architecture -qDEB_BUILD_ARCH) >>> + if [ "${BUILD_ARCH}" = "${DISTRO_ARCH}" ]; then >>> + # no, make sure CROSS_COMPILE isn't set >>> + unset CROSS_COMPILE >>> + fi >>> + >>> + # call the actual target script >>> + do_${target} || return ${?} >>> +} >>> diff --git a/meta/recipes-kernel/linux/files/debian/isar/install.tmpl >>> b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl >>> new file mode 100644 >>> index 0000000..8ccda68 >>> --- /dev/null >>> +++ b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl >>> @@ -0,0 +1,206 @@ >>> +#!/bin/bash >>> +# Copyright (c) Mentor Graphics, a Siemens business, 2019 >>> +# SPDX-License-Identifier: MIT >>> + >>> +# Load common stuff >>> +. ${S}/debian/isar/common || exit ${?} >>> + >>> +# Stop on error >>> +set -e >>> + >>> +do_install() { >>> + >>> + # check if our kernel was configured >>> + if [ ! -f "${O}/.config" ]; then >>> + echo "error: kernel not configured!" >&2 >>> + return 1 >>> + fi >>> + >>> + # load its configuration >>> + . ${O}/.config >>> + >>> + kimage="$(${MAKE} O=${O} -s --no-print-directory image_name)" >>> + krel="$(${MAKE} O=${O} -s --no-print-directory kernelrelease)" >>> + case "${ARCH}" in >>> + mips|powerpc) kimage_path="boot/vmlinux-${krel}" ;; >>> + um) kimage_path="usr/bin/vmlinux-${krel}" ;; >>> + *) kimage_path="boot/vmlinuz-${krel}" ;; >>> + esac >>> + >>> + print_settings >>> + >>> + # Trace what we do here >>> + set -x >>> + >>> + install_image >>> + install_hooks >>> + install_dtbs >>> + install_kmods >>> + install_headers >>> + >>> + # Stop tracing >>> + set +x >>> +} >>> + >>> +print_settings() { >>> + cat <<EOF >>> +Install settings: >>> +----------------- >>> +deb_dtb_dir=${deb_dtb_dir} >>> +deb_hdr_dir=${deb_hdr_dir} >>> +kimage=${kimage} >>> +kimage_path=${kimage_path} >>> + >>> +EOF >>> +} >>> + >>> +install_image() { >>> + install -m 755 -d ${deb_img_dir}/$(dirname ${kimage_path}) >>> + cp ${O}/${kimage} ${deb_img_dir}/${kimage_path} >>> + install_image_debug >>> +} >>> + >>> +install_image_debug() { >>> + # Different tools want the image in different locations >>> + # perf >>> + mkdir -p ${deb_dbg_dir}/usr/lib/debug/lib/modules/${krel}/ >>> + cp ${O}/vmlinux ${deb_dbg_dir}/usr/lib/debug/lib/modules/${krel}/ >>> + # systemtap >>> + mkdir -p ${deb_dbg_dir}/usr/lib/debug/boot/ >>> + ln -s ../lib/modules/$version/vmlinux >>> ${deb_dbg_dir}/usr/lib/debug/boot/vmlinux-${krel} >>> + # kdump-tools >>> + ln -s lib/modules/${krel}/vmlinux >>> ${deb_dbg_dir}/usr/lib/debug/vmlinux-${krel} >>> +} >>> + >>> +install_hooks() { >>> + install -m 755 -d ${deb_img_dir}/etc/kernel/install.d >>> + install -m 755 -d ${deb_img_dir}/etc/kernel/postinst.d >>> + install -m 755 -d ${deb_img_dir}/etc/kernel/postrm.d >>> + install -m 755 -d ${deb_img_dir}/etc/kernel/prerm.d >>> + >>> + initrd="No" >>> + [ -z "${CONFIG_BLK_DEV_INITRD}" ] || initrd="Yes" >>> + >>> + for script in postinst postrm preinst prerm; do >>> + sed -i -e "s,INITRD=[A-Za-z0-9]*,INITRD=${initrd},g" >>> ${S}/debian/linux-image*.${script} >>> + sed -i -e "s,version=.*,version=${krel},g" >>> ${S}/debian/linux-image*.${script} >>> + done >>> +} >>> + >>> +install_dtbs() { >>> + [ -n "${CONFIG_OF}" ] || return 0 >>> + ${MAKE} O=${O} INSTALL_DTBS_PATH=${deb_dtb_dir} dtbs_install >>> +} >>> + >>> +install_kmods() { >>> + [ -n "${CONFIG_MODULES}" ] || return 0 >>> + ${MAKE} O=${O} INSTALL_MOD_PATH=${deb_img_dir} modules_install >>> + touch ${deb_img_dir}/lib/modules/${krel}/.fresh-install >>> + rm -fv ${deb_img_dir}/lib/modules/${krel}/build >>> + rm -fv ${deb_img_dir}/lib/modules/${krel}/source >>> + install_kmods_debug >>> +} >>> + >>> +kmods_sign() { >>> + [ -n "${CONFIG_MODULE_SIG_ALL}" ] || return 0 >>> + ${MAKE} O=${O} INSTALL_MOD_PATH=${deb_img_dir} modules_sign >>> +} >>> + >>> +install_kmods_debug() { >>> + [ -n "${CONFIG_DEBUG_INFO}" ] || return 0 >>> + >>> + kmod_inst_dir=${deb_img_dir}/lib/modules >>> + kmod_debug_dir=${deb_dbg_dir}/usr/lib/debug >>> + >>> + # copy kernels modules to usr/lib/debug >>> + mkdir -p ${kmod_debug_dir} >>> + tar -C ${kmod_inst_dir}/ -cO --exclude='modules.*' . | tar -C >>> ${kmod_debug_dir}/ -xf - >>> + # strip everything but debug sections for modules in usr/lib/debug >>> + find ${kmod_debug_dir} -name *.ko -exec ${CROSS_COMPILE}objcopy >>> --only-keep-debug {} \; >>> + # and strip debug sections from modules in lib/modules >>> + find ${kmod_inst_dir} -name *.ko -exec ${CROSS_COMPILE}objcopy >>> --strip-debug {} \; >>> + >>> + # re-sign stripped kernel modules >>> + kmods_sign >>> +} >>> + >>> +headers_check() { >>> + ${MAKE} O=${O} headers_check >>> +} >>> + >>> +libc_headers() { >>> + mkdir -p ${deb_libc_hdr_dir} >>> + ${MAKE} O=${O} headers_install >>> INSTALL_HDR_PATH=${deb_libc_hdr_dir}/usr >>> + host_arch=$(dpkg-architecture -a${DISTRO_ARCH} >>> -qDEB_HOST_MULTIARCH) >>> + mkdir ${deb_libc_hdr_dir}/usr/include/${host_arch} >>> + mv ${deb_libc_hdr_dir}/usr/include/asm >>> ${deb_libc_hdr_dir}/usr/include/${host_arch}/ >>> +} >>> + >>> +# the kernel does not seem to provide a way to cross-compile its >>> tools, we however >>> +# have qemu-user-static binaries in our buildchroot. Create a new >>> build tree for the >>> +# scripts and override HOSTCC to force a cross-compile >>> +kernel_tools_build() { >>> + odir="${S}/build-scripts" >>> + mflags="HOSTCC=${CROSS_COMPILE}gcc O=${odir} NOSTDINC_FLAGS=" >>> + rm -rf ${odir} && mkdir -p ${odir} && cp >>> ${S}/${KERNEL_BUILD_DIR}/.config ${odir} >>> + (unset LD_PRELOAD; cd ${odir} && make -C ${S} ${mflags} >>> olddefconfig scripts) >>> + (cd ${odir}; find scripts -type f -executable -exec file {} >>> \;|grep ELF|cut -d: -f1) >${src_hdr_files} >>> + tar -C ${odir} -cf - -T - <${src_hdr_files} | tar -C ${destdir} >>> -xf - >>> +} >>> + >>> +kernel_tools() { >>> + # remove object files >>> + find ${destdir}/scripts -type f -name '*.o' |xargs rm -f >>> + >>> + # we're all done if we aren't cross-compiling >>> + [ -n "${CROSS_COMPILE}" ] || return 0 >>> + >>> + # remove ELF executables from the linux-headers package >>> + find ${destdir}/scripts -type f -exec file {} \;|grep ELF|cut >>> -d: -f1|xargs rm -fv >>> + >>> + # cross-compile kernel tools to be shipped with linux-headers >>> + kernel_tools_build >>> +} >>> + >>> +kernel_headers() { >>> + destdir=${deb_kern_hdr_dir}/${KERNEL_HEADERS_DIR} >>> + src_hdr_files=$(mktemp) >>> + obj_hdr_files=$(mktemp) >>> + >>> + mkdir -p ${destdir} >>> + mkdir -p ${deb_kern_hdr_dir}/lib/modules/${krel} >>> + >>> + (cd ${S}; find . -name 'Makefile*' -o -name 'Kconfig*' -o -name >>> '*.pl') >>${src_hdr_files} >>> + (cd ${S}; find arch/*/include include scripts -type f -o -type >>> l) >>${src_hdr_files} >>> + (cd ${S}; find arch/${ARCH} -name module.lds -o -name >>> Kbuild.platforms -o -name Platform) >>${src_hdr_files} >>> + (cd ${S}; find $(find arch/${ARCH} -name include -o -name >>> scripts -type d) -type f) >>${src_hdr_files} >>> + >>> + (cd ${O}; find arch/${ARCH}/include Module.symvers include >>> scripts -type f) >>${obj_hdr_files} >>> + if [ -n "${CONFIG_STACK_VALIDATION}" ]; then >>> + (cd ${O}; find tools/objtool -type f -executable) >>> >>${obj_hdr_files} >>> + fi >>> + if [ -n "${CONFIG_GCC_PLUGINS}" ]; then >>> + (cd ${O}; find scripts/gcc-plugins -name *.so -o -name >>> gcc-common.h) >>${obj_hdr_files} >>> + fi >>> + >>> + # deploy files that were matched above >>> + tar -C ${S} -cf - -T - <${src_hdr_files} | tar -C ${destdir} -xf - >>> + tar -C ${O} -cf - -T - <${obj_hdr_files} | tar -C ${destdir} -xf - >>> + >>> + # add the kernel config >>> + cp ${O}/${KCONF} ${destdir}/.config >>> + >>> + # handle kernel development tools >>> + kernel_tools >>> + >>> + # create symlinks >>> + ln -sf /${KERNEL_HEADERS_DIR} >>> ${deb_kern_hdr_dir}/lib/modules/${krel}/build >>> +} >>> + >>> +install_headers() { >>> + headers_check >>> + [ -z ${kern_pkgs["linux-libc-headers"]} ] || libc_headers >>> + kernel_headers >>> +} >>> + >>> +main install ${*} >>> diff --git >>> a/meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl >>> b/meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl >>> new file mode 100644 >>> index 0000000..8e8636a >>> --- /dev/null >>> +++ b/meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl >>> @@ -0,0 +1,29 @@ >>> +#!/bin/sh >>> +# based on >>> https://salsa.debian.org/kernel-team/linux/blob/master/debian/templates/image.postinst.in >>> >>> + >>> +# Tell initramfs builder whether it's wanted >>> +export INITRD=Yes >>> + >>> +version=${PV} >>> +image_path=/boot/${KERNEL_FILE}-${version} >>> + >>> +if [ "$1" != configure ]; then >>> + exit 0 >>> +fi >>> + >>> +depmod $version >>> + >>> +if [ -f /lib/modules/$version/.fresh-install ]; then >>> + change=install >>> +else >>> + change=upgrade >>> +fi >>> +linux-update-symlinks $change $version $image_path >>> +rm -f /lib/modules/$version/.fresh-install >>> + >>> +if [ -d /etc/kernel/postinst.d ]; then >>> + DEB_MAINT_PARAMS="$*" run-parts --report --exit-on-error >>> --arg=$version \ >>> + --arg=$image_path /etc/kernel/postinst.d >>> +fi >>> + >>> +exit 0 >>> diff --git >>> a/meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl >>> b/meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl >>> new file mode 100644 >>> index 0000000..4d8a7d9 >>> --- /dev/null >>> +++ b/meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl >>> @@ -0,0 +1,35 @@ >>> +#!/bin/sh -e >>> +# based on >>> https://salsa.debian.org/kernel-team/linux/blob/master/debian/templates/image.postrm.in >>> >>> + >>> +# Tell initramfs builder whether it's wanted >>> +export INITRD=Yes >>> + >>> +version=${PV} >>> +image_path=/boot/${KERNEL_FILE}-${version} >>> + >>> +rm -f /lib/modules/$version/.fresh-install >>> + >>> +if [ "$1" != upgrade ] && command -v linux-update-symlinks >>> >/dev/null; then >>> + linux-update-symlinks remove $version $image_path >>> +fi >>> + >>> +if [ -d /etc/kernel/postrm.d ]; then >>> + DEB_MAINT_PARAMS="$*" run-parts --report --exit-on-error >>> --arg=$version \ >>> + --arg=$image_path /etc/kernel/postrm.d >>> +fi >>> + >>> +if [ "$1" = purge ]; then >>> + for extra_file in modules.dep modules.isapnpmap modules.pcimap \ >>> + modules.usbmap modules.parportmap \ >>> + modules.generic_string modules.ieee1394map \ >>> + modules.ieee1394map modules.pnpbiosmap \ >>> + modules.alias modules.ccwmap modules.inputmap \ >>> + modules.symbols modules.ofmap \ >>> + modules.seriomap modules.\*.bin \ >>> + modules.softdep modules.devname; do >>> + eval rm -f /lib/modules/$version/$extra_file >>> + done >>> + rmdir /lib/modules/$version || true >>> +fi >>> + >>> +exit 0 >>> diff --git >>> a/meta/recipes-kernel/linux/files/debian/linux-image.preinst.tmpl >>> b/meta/recipes-kernel/linux/files/debian/linux-image.preinst.tmpl >>> new file mode 100644 >>> index 0000000..7465197 >>> --- /dev/null >>> +++ b/meta/recipes-kernel/linux/files/debian/linux-image.preinst.tmpl >>> @@ -0,0 +1,25 @@ >>> +#!/bin/sh -e >>> +# based on >>> https://salsa.debian.org/kernel-team/linux/blob/master/debian/templates/image.preinst.in >>> >>> + >>> +# Tell initramfs builder whether it's wanted >>> +export INITRD=Yes >>> + >>> +version=${PV} >>> +image_path=/boot/${KERNEL_FILE}-${version} >>> + >>> +if [ "$1" = abort-upgrade ]; then >>> + exit 0 >>> +fi >>> + >>> +if [ "$1" = install ]; then >>> + # Create a flag file for postinst >>> + mkdir -p /lib/modules/$version >>> + touch /lib/modules/$version/.fresh-install >>> +fi >>> + >>> +if [ -d /etc/kernel/preinst.d ]; then >>> + DEB_MAINT_PARAMS="$*" run-parts --report --exit-on-error >>> --arg=$version \ >>> + --arg=$image_path /etc/kernel/preinst.d >>> +fi >>> + >>> +exit 0 >>> diff --git >>> a/meta/recipes-kernel/linux/files/debian/linux-image.prerm.tmpl >>> b/meta/recipes-kernel/linux/files/debian/linux-image.prerm.tmpl >>> new file mode 100644 >>> index 0000000..1992b32 >>> --- /dev/null >>> +++ b/meta/recipes-kernel/linux/files/debian/linux-image.prerm.tmpl >>> @@ -0,0 +1,21 @@ >>> +#!/bin/sh -e >>> +# based on >>> https://salsa.debian.org/kernel-team/linux/blob/master/debian/templates/image.prerm.in >>> >>> + >>> +# Tell initramfs builder whether it's wanted >>> +export INITRD=Yes >>> + >>> +version=${PV} >>> +image_path=/boot/${KERNEL_FILE}-${version} >>> + >>> +if [ "$1" != remove ]; then >>> + exit 0 >>> +fi >>> + >>> +linux-check-removal $version >>> + >>> +if [ -d /etc/kernel/prerm.d ]; then >>> + DEB_MAINT_PARAMS="$*" run-parts --report --exit-on-error >>> --arg=$version \ >>> + --arg=$image_path /etc/kernel/prerm.d >>> +fi >>> + >>> +exit 0 >>> diff --git a/meta/recipes-kernel/linux/files/debian/rules.tmpl >>> b/meta/recipes-kernel/linux/files/debian/rules.tmpl >>> new file mode 100755 >>> index 0000000..c024768 >>> --- /dev/null >>> +++ b/meta/recipes-kernel/linux/files/debian/rules.tmpl >>> @@ -0,0 +1,39 @@ >>> +#!/usr/bin/make -f >>> + >>> +CROSS_COMPILE:=$(DEB_HOST_GNU_TYPE)- >>> + >>> +O:=$(CURDIR)/${KERNEL_BUILD_DIR} >>> +S:=$(CURDIR) >>> +deb_top_dir:=$(S)/debian >>> + >>> +# Dynamic variables to be passed to Isar build scripts >>> +isar_env=$(strip \ >>> + export CROSS_COMPILE='$(CROSS_COMPILE)' && \ >>> + export DEB_BUILD_PROFILES='$(DEB_BUILD_PROFILES)' && \ >>> + export DEB_HOST_GNU_TYPE='$(DEB_HOST_GNU_TYPE)' && \ >>> + export MAKE='$(MAKE)' && \ >>> + export O='${O}' \ >>> + export S='${S}' \ >>> +) >>> + >>> +%: >>> + dh $(@) >>> + >>> +.PHONY: override_dh_auto_clean >>> +override_dh_auto_clean: >>> + $(isar_env) && bash $(deb_top_dir)/isar/clean >>> + >>> +.PHONY: override_dh_auto_build >>> +override_dh_auto_build: >>> + $(isar_env) && bash $(deb_top_dir)/isar/build >>> + >>> +override_dh_auto_install: >>> + $(isar_env) && bash $(deb_top_dir)/isar/install >>> + >>> +.PHONY: override_dh_auto_test >>> +override_dh_auto_test: >>> + true >>> + >>> +PHONY: override_dh_strip >>> +override_dh_strip: >>> + dh_strip -Xvmlinux --no-automatic-dbgsym >>> diff --git a/meta/recipes-kernel/linux/linux-custom.inc >>> b/meta/recipes-kernel/linux/linux-custom.inc >>> index ca91f64..ecf0e66 100644 >>> --- a/meta/recipes-kernel/linux/linux-custom.inc >>> +++ b/meta/recipes-kernel/linux/linux-custom.inc >>> @@ -2,22 +2,107 @@ >>> # >>> # This software is a part of ISAR. >>> # Copyright (c) Siemens AG, 2018 >>> +# Copyright (c) Mentor Graphics, a Siemens business, 2019 >>> # >>> # SPDX-License-Identifier: MIT >>> -FILESEXTRAPATHS_prepend := "${FILE_DIRNAME}/files:" >>> +# Settings that would typically be done from the custom kernel recipe >>> +# ------------------------------------------------------------------- >>> DESCRIPTION ?= "Custom kernel" >>> +MAINTAINER ?= "isar-users <isar-users@googlegroups.com>" >>> + >>> +KBUILD_DEPENDS ?= "build-essential:native, \ >>> + libelf-dev:native, \ >>> + libncurses-dev:native, \ >>> + libssl-dev:native, \ >>> + bc, \ >>> + bison, \ >>> + cpio, \ >>> + flex, \ >>> + git, \ >>> + kmod, \ >>> + libssl-dev," >>> + >>> +KERNEL_DEBIAN_DEPENDS ?= "initramfs-tools | linux-initramfs-tool, \ >>> + kmod, \ >>> + linux-base (>= 4.3~)," >>> + >>> +KERNEL_HEADERS_DEBIAN_DEPENDS ?= "libc6, \ >>> + libssl1.1," >>> + >>> +KERNEL_LIBC_DEV_DEPLOY ?= "0" >>> + >>> +# Settings that may be changed on a per distro, machine or layer basis >>> +# -------------------------------------------------------------------- >>> + >>> +KERNEL_DTB_DIR ?= "usr/lib/linux-${KERNEL_NAME_PROVIDED}-${PV}" >> >> Here is the bug that breaks U-boot setups here: Must be >> linux-image-${PV} in order to stay compatible with Debian. >> >> Did you compare some content of the new packages against that of the >> original package build? >> > Yes but I had not anticipated this one as being an issue (so thanks for > pointing that out). I have limited hardware available here (mostly > Siemens IPCs or non supported boards such as Xilinx ref boards). will > check if I can get remote access to some of the boards supported by > Isar. I have made the change locally and started a build. I hope I can > check it out on the de0-nano-soc board that we have in meta-isar There are plenty of fully enabled test cases of Isar publicly available, all being affected in some way: - https://github.com/siemens/jailhouse-images (recommended test case for a broad coverages of cases) - https://gitlab.denx.de/Xenomai/xenomai-images - https://gitlab.com/cip-project/cip-core/isar-cip-core In addition, our isar-siemens should be broken for the de0-nano-soc, and I think that board is also in Isar upstream, and your lab. Jan -- Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 83+ messages in thread
* Re: [PATCH L-C v3 0/7] linux-custom recipe rework 2019-11-08 7:07 ` [PATCH L-C v3 " Cedric Hombourger ` (6 preceding siblings ...) 2019-11-08 7:07 ` [PATCH L-C v3 7/7] linux-custom: rewrite to no longer depend on the kernel's builddeb Cedric Hombourger @ 2019-11-08 7:12 ` Jan Kiszka 2019-11-08 7:27 ` Cedric Hombourger 7 siblings, 1 reply; 83+ messages in thread From: Jan Kiszka @ 2019-11-08 7:12 UTC (permalink / raw) To: Cedric Hombourger, isar-users On 08.11.19 08:07, Cedric Hombourger wrote: > The Linux kernel is packaged a bit differently between Debian variants > and in particular for the perf package(s) in Debian vs Ubuntu. Since > there is no way the kernel build scripts would ever get distro specific > support for building more than the kernel and headers, we need our own > build/packaging scripts. > > Version 3 of this changeset was tested against ci_build.sh and some images were > booted on hardware available here. No regressions were detected (to date). This > was also tested on the creator-ci40-marduk machine (subject of another patch > series). dts files weren't tested so far. They end up in the wrong directory on the target and, thus, will not be found by the boot loader. I can likely tell you in a moment which part exactly is wrong. Jan > > Changes compared to v2: > Rework custom_kernel.md > Compare PV+/-LOCAL_VERSION and kernelrelease string > > Changes compared to v1: > Add missing reference to KERNEL_DEFCONFIG in requirements list > Amend postinst/postrm scripts according to CONFIG_BLK_DEV_INITRD > Remove (unused) pre/post host hooks from debian/isar scripts > Add preinst/prerm scripts shipped by the kernel > Align {pre,post}{inst,rm} scripts with upstream > Add ${BASE_DISTRO} to Build-Profiles > Add nolibcdev to Build-Profiles if KERNEL_LIBC_DEV_DEPLOY != 1 > Only build the libc-dev packages if it is needed > > > Cedric Hombourger (7): > recipes-kernel/linux: make KERNEL_DEFCONFIG support in-tree defconfigs > linux-mainline: fix stripping of .0 from the kernel version > linux-mainline: update from 4.19.0 to 4.19.80 > linux-custom: add support for kernel config fragments > linux-mainline: disable support for HFS to demonstrate use of config fragments > buildchroot-host: install qemu-static to support hybrid cross-compiles > linux-custom: rewrite to no longer depend on the kernel's builddeb > > doc/custom_kernel.md | 80 +++++++ > .../recipes-kernel/linux/files/no-fs-hfs.cfg | 2 + > ...ne_4.19.0.bb => linux-mainline_4.19.80.bb} | 8 +- > .../buildchroot/buildchroot-host.bb | 6 + > .../linux/files/build-kernel.sh | 129 ----------- > meta/recipes-kernel/linux/files/debian/compat | 1 + > .../linux/files/debian/control.tmpl | 37 ++++ > .../linux/files/debian/isar/build.tmpl | 45 ++++ > .../linux/files/debian/isar/clean.tmpl | 21 ++ > .../linux/files/debian/isar/common.tmpl | 50 +++++ > .../linux/files/debian/isar/install.tmpl | 206 ++++++++++++++++++ > .../files/debian/linux-image.postinst.tmpl | 29 +++ > .../files/debian/linux-image.postrm.tmpl | 35 +++ > .../files/debian/linux-image.preinst.tmpl | 25 +++ > .../linux/files/debian/linux-image.prerm.tmpl | 21 ++ > .../linux/files/debian/rules.tmpl | 39 ++++ > meta/recipes-kernel/linux/linux-custom.inc | 204 ++++++++++++----- > 17 files changed, 752 insertions(+), 186 deletions(-) > create mode 100644 doc/custom_kernel.md > create mode 100644 meta-isar/recipes-kernel/linux/files/no-fs-hfs.cfg > rename meta-isar/recipes-kernel/linux/{linux-mainline_4.19.0.bb => linux-mainline_4.19.80.bb} (67%) > delete mode 100644 meta/recipes-kernel/linux/files/build-kernel.sh > create mode 100644 meta/recipes-kernel/linux/files/debian/compat > create mode 100644 meta/recipes-kernel/linux/files/debian/control.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/isar/build.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/isar/clean.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/isar/common.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/isar/install.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.preinst.tmpl > create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.prerm.tmpl > create mode 100755 meta/recipes-kernel/linux/files/debian/rules.tmpl > -- Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 83+ messages in thread
* Re: [PATCH L-C v3 0/7] linux-custom recipe rework 2019-11-08 7:12 ` [PATCH L-C v3 0/7] linux-custom recipe rework Jan Kiszka @ 2019-11-08 7:27 ` Cedric Hombourger 2019-11-08 7:37 ` Jan Kiszka 0 siblings, 1 reply; 83+ messages in thread From: Cedric Hombourger @ 2019-11-08 7:27 UTC (permalink / raw) To: Jan Kiszka, isar-users On 11/8/19 8:12 AM, Jan Kiszka wrote: > On 08.11.19 08:07, Cedric Hombourger wrote: >> The Linux kernel is packaged a bit differently between Debian variants >> and in particular for the perf package(s) in Debian vs Ubuntu. Since >> there is no way the kernel build scripts would ever get distro specific >> support for building more than the kernel and headers, we need our own >> build/packaging scripts. >> >> Version 3 of this changeset was tested against ci_build.sh and some >> images were >> booted on hardware available here. No regressions were detected (to >> date). This >> was also tested on the creator-ci40-marduk machine (subject of >> another patch >> series). > > dts files weren't tested so far. They end up in the wrong directory on > the target and, thus, will not be found by the boot loader. I can > likely tell you in a moment which part exactly is wrong. ok thanks. the boards I tested either do not use dtb files or require a custom dtb path setting anyway > > Jan > >> >> Changes compared to v2: >> Rework custom_kernel.md >> Compare PV+/-LOCAL_VERSION and kernelrelease string >> >> Changes compared to v1: >> Add missing reference to KERNEL_DEFCONFIG in requirements list >> Amend postinst/postrm scripts according to CONFIG_BLK_DEV_INITRD >> Remove (unused) pre/post host hooks from debian/isar scripts >> Add preinst/prerm scripts shipped by the kernel >> Align {pre,post}{inst,rm} scripts with upstream >> Add ${BASE_DISTRO} to Build-Profiles >> Add nolibcdev to Build-Profiles if KERNEL_LIBC_DEV_DEPLOY != 1 >> Only build the libc-dev packages if it is needed >> >> >> Cedric Hombourger (7): >> recipes-kernel/linux: make KERNEL_DEFCONFIG support in-tree >> defconfigs >> linux-mainline: fix stripping of .0 from the kernel version >> linux-mainline: update from 4.19.0 to 4.19.80 >> linux-custom: add support for kernel config fragments >> linux-mainline: disable support for HFS to demonstrate use of >> config fragments >> buildchroot-host: install qemu-static to support hybrid >> cross-compiles >> linux-custom: rewrite to no longer depend on the kernel's builddeb >> >> doc/custom_kernel.md | 80 +++++++ >> .../recipes-kernel/linux/files/no-fs-hfs.cfg | 2 + >> ...ne_4.19.0.bb => linux-mainline_4.19.80.bb} | 8 +- >> .../buildchroot/buildchroot-host.bb | 6 + >> .../linux/files/build-kernel.sh | 129 ----------- >> meta/recipes-kernel/linux/files/debian/compat | 1 + >> .../linux/files/debian/control.tmpl | 37 ++++ >> .../linux/files/debian/isar/build.tmpl | 45 ++++ >> .../linux/files/debian/isar/clean.tmpl | 21 ++ >> .../linux/files/debian/isar/common.tmpl | 50 +++++ >> .../linux/files/debian/isar/install.tmpl | 206 ++++++++++++++++++ >> .../files/debian/linux-image.postinst.tmpl | 29 +++ >> .../files/debian/linux-image.postrm.tmpl | 35 +++ >> .../files/debian/linux-image.preinst.tmpl | 25 +++ >> .../linux/files/debian/linux-image.prerm.tmpl | 21 ++ >> .../linux/files/debian/rules.tmpl | 39 ++++ >> meta/recipes-kernel/linux/linux-custom.inc | 204 ++++++++++++----- >> 17 files changed, 752 insertions(+), 186 deletions(-) >> create mode 100644 doc/custom_kernel.md >> create mode 100644 meta-isar/recipes-kernel/linux/files/no-fs-hfs.cfg >> rename meta-isar/recipes-kernel/linux/{linux-mainline_4.19.0.bb => >> linux-mainline_4.19.80.bb} (67%) >> delete mode 100644 meta/recipes-kernel/linux/files/build-kernel.sh >> create mode 100644 meta/recipes-kernel/linux/files/debian/compat >> create mode 100644 meta/recipes-kernel/linux/files/debian/control.tmpl >> create mode 100644 >> meta/recipes-kernel/linux/files/debian/isar/build.tmpl >> create mode 100644 >> meta/recipes-kernel/linux/files/debian/isar/clean.tmpl >> create mode 100644 >> meta/recipes-kernel/linux/files/debian/isar/common.tmpl >> create mode 100644 >> meta/recipes-kernel/linux/files/debian/isar/install.tmpl >> create mode 100644 >> meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl >> create mode 100644 >> meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl >> create mode 100644 >> meta/recipes-kernel/linux/files/debian/linux-image.preinst.tmpl >> create mode 100644 >> meta/recipes-kernel/linux/files/debian/linux-image.prerm.tmpl >> create mode 100755 meta/recipes-kernel/linux/files/debian/rules.tmpl >> ^ permalink raw reply [flat|nested] 83+ messages in thread
* Re: [PATCH L-C v3 0/7] linux-custom recipe rework 2019-11-08 7:27 ` Cedric Hombourger @ 2019-11-08 7:37 ` Jan Kiszka 2019-11-08 7:42 ` Cedric Hombourger 0 siblings, 1 reply; 83+ messages in thread From: Jan Kiszka @ 2019-11-08 7:37 UTC (permalink / raw) To: Cedric Hombourger, isar-users On 08.11.19 08:27, Cedric Hombourger wrote: > > On 11/8/19 8:12 AM, Jan Kiszka wrote: >> On 08.11.19 08:07, Cedric Hombourger wrote: >>> The Linux kernel is packaged a bit differently between Debian variants >>> and in particular for the perf package(s) in Debian vs Ubuntu. Since >>> there is no way the kernel build scripts would ever get distro specific >>> support for building more than the kernel and headers, we need our own >>> build/packaging scripts. >>> >>> Version 3 of this changeset was tested against ci_build.sh and some >>> images were >>> booted on hardware available here. No regressions were detected (to >>> date). This >>> was also tested on the creator-ci40-marduk machine (subject of >>> another patch >>> series). >> >> dts files weren't tested so far. They end up in the wrong directory on >> the target and, thus, will not be found by the boot loader. I can >> likely tell you in a moment which part exactly is wrong. > ok thanks. the boards I tested either do not use dtb files or require a > custom dtb path setting anyway I strongly recommend to switch everything to ditro boot. That's usually just a simple patch to the default u-boot env. More and more vendors support that, and it's the unified way of booting Isar. Jan ^ permalink raw reply [flat|nested] 83+ messages in thread
* Re: [PATCH L-C v3 0/7] linux-custom recipe rework 2019-11-08 7:37 ` Jan Kiszka @ 2019-11-08 7:42 ` Cedric Hombourger 2019-11-08 7:54 ` Jan Kiszka 0 siblings, 1 reply; 83+ messages in thread From: Cedric Hombourger @ 2019-11-08 7:42 UTC (permalink / raw) To: Jan Kiszka, isar-users On 11/8/19 8:37 AM, Jan Kiszka wrote: > On 08.11.19 08:27, Cedric Hombourger wrote: >> >> On 11/8/19 8:12 AM, Jan Kiszka wrote: >>> On 08.11.19 08:07, Cedric Hombourger wrote: >>>> The Linux kernel is packaged a bit differently between Debian variants >>>> and in particular for the perf package(s) in Debian vs Ubuntu. Since >>>> there is no way the kernel build scripts would ever get distro >>>> specific >>>> support for building more than the kernel and headers, we need our own >>>> build/packaging scripts. >>>> >>>> Version 3 of this changeset was tested against ci_build.sh and some >>>> images were >>>> booted on hardware available here. No regressions were detected (to >>>> date). This >>>> was also tested on the creator-ci40-marduk machine (subject of >>>> another patch >>>> series). >>> >>> dts files weren't tested so far. They end up in the wrong directory >>> on the target and, thus, will not be found by the boot loader. I can >>> likely tell you in a moment which part exactly is wrong. >> ok thanks. the boards I tested either do not use dtb files or require >> a custom dtb path setting anyway > > I strongly recommend to switch everything to ditro boot. That's > usually just a simple patch to the default u-boot env. More and more > vendors support that, and it's the unified way of booting Isar. I can't agree more (need to work with the vendor of the board I have to update their insanely old u-boot, it is insane to have it supported with mainline kernels but keep a 4-years old u-boot). > > Jan ^ permalink raw reply [flat|nested] 83+ messages in thread
* Re: [PATCH L-C v3 0/7] linux-custom recipe rework 2019-11-08 7:42 ` Cedric Hombourger @ 2019-11-08 7:54 ` Jan Kiszka 2019-11-08 8:01 ` Cedric Hombourger 0 siblings, 1 reply; 83+ messages in thread From: Jan Kiszka @ 2019-11-08 7:54 UTC (permalink / raw) To: Cedric Hombourger, isar-users On 08.11.19 08:42, Cedric Hombourger wrote: > > On 11/8/19 8:37 AM, Jan Kiszka wrote: >> On 08.11.19 08:27, Cedric Hombourger wrote: >>> >>> On 11/8/19 8:12 AM, Jan Kiszka wrote: >>>> On 08.11.19 08:07, Cedric Hombourger wrote: >>>>> The Linux kernel is packaged a bit differently between Debian variants >>>>> and in particular for the perf package(s) in Debian vs Ubuntu. Since >>>>> there is no way the kernel build scripts would ever get distro >>>>> specific >>>>> support for building more than the kernel and headers, we need our own >>>>> build/packaging scripts. >>>>> >>>>> Version 3 of this changeset was tested against ci_build.sh and some >>>>> images were >>>>> booted on hardware available here. No regressions were detected (to >>>>> date). This >>>>> was also tested on the creator-ci40-marduk machine (subject of >>>>> another patch >>>>> series). >>>> >>>> dts files weren't tested so far. They end up in the wrong directory >>>> on the target and, thus, will not be found by the boot loader. I can >>>> likely tell you in a moment which part exactly is wrong. >>> ok thanks. the boards I tested either do not use dtb files or require >>> a custom dtb path setting anyway >> >> I strongly recommend to switch everything to ditro boot. That's >> usually just a simple patch to the default u-boot env. More and more >> vendors support that, and it's the unified way of booting Isar. > I can't agree more (need to work with the vendor of the board I have to > update their insanely old u-boot, it is insane to have it supported with > mainline kernels but keep a 4-years old u-boot). Ah, another "we are done" BSP. Yeah, never use BSPs... BTW, another fallout from the full nightly build: NOTE: recipe isar-image-base-1.0-r0: task do_copy_boot_files: Started ERROR: mc:qemumipsel-bullseye:isar-image-base-1.0-r0 do_copy_boot_files: Execution of '/builds/ebsy/debian/isar/build/tmp/work/debian-bullseye-mipsel/isar-image-base-qemumipsel-ext4-img/1.0-r0/temp/run.do_copy_boot_files.105304' failed with exit code 1: cp: cannot stat '/builds/ebsy/debian/isar/build/tmp/work/debian-bullseye-mipsel/isar-image-base-qemumipsel-ext4-img/1.0-r0/rootfs/vmlinuz': No such file or directory WARNING: exit code 1 from a shell command. ERROR: Logfile of failure stored in: /builds/ebsy/debian/isar/build/tmp/work/debian-bullseye-mipsel/isar-image-base-qemumipsel-ext4-img/1.0-r0/temp/log.do_copy_boot_files.105304 Log data follows: | DEBUG: Executing shell function do_copy_boot_files | cp: cannot stat '/builds/ebsy/debian/isar/build/tmp/work/debian-bullseye-mipsel/isar-image-base-qemumipsel-ext4-img/1.0-r0/rootfs/vmlinuz': No such file or directory | WARNING: exit code 1 from a shell command. | ERROR: Execution of '/builds/ebsy/debian/isar/build/tmp/work/debian-bullseye-mipsel/isar-image-base-qemumipsel-ext4-img/1.0-r0/temp/run.do_copy_boot_files.105304' failed with exit code 1: | cp: cannot stat '/builds/ebsy/debian/isar/build/tmp/work/debian-bullseye-mipsel/isar-image-base-qemumipsel-ext4-img/1.0-r0/rootfs/vmlinuz': No such file or directory | WARNING: exit code 1 from a shell command. | NOTE: recipe isar-image-base-1.0-r0: task do_copy_boot_files: Failed ERROR: Task (mc:qemumipsel-bullseye:/builds/ebsy/debian/isar/meta-isar/recipes-core/images/isar-image-base.bb:do_copy_boot_files) failed with exit code '1' Can send you the link the full log on our internal CI on code.siemens.com. Jan -- Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 83+ messages in thread
* Re: [PATCH L-C v3 0/7] linux-custom recipe rework 2019-11-08 7:54 ` Jan Kiszka @ 2019-11-08 8:01 ` Cedric Hombourger 2019-11-08 8:16 ` Cedric Hombourger 0 siblings, 1 reply; 83+ messages in thread From: Cedric Hombourger @ 2019-11-08 8:01 UTC (permalink / raw) To: Jan Kiszka, isar-users On 11/8/19 8:54 AM, Jan Kiszka wrote: > On 08.11.19 08:42, Cedric Hombourger wrote: >> On 11/8/19 8:37 AM, Jan Kiszka wrote: >>> On 08.11.19 08:27, Cedric Hombourger wrote: >>>> On 11/8/19 8:12 AM, Jan Kiszka wrote: >>>>> On 08.11.19 08:07, Cedric Hombourger wrote: >>>>>> The Linux kernel is packaged a bit differently between Debian variants >>>>>> and in particular for the perf package(s) in Debian vs Ubuntu. Since >>>>>> there is no way the kernel build scripts would ever get distro >>>>>> specific >>>>>> support for building more than the kernel and headers, we need our own >>>>>> build/packaging scripts. >>>>>> >>>>>> Version 3 of this changeset was tested against ci_build.sh and some >>>>>> images were >>>>>> booted on hardware available here. No regressions were detected (to >>>>>> date). This >>>>>> was also tested on the creator-ci40-marduk machine (subject of >>>>>> another patch >>>>>> series). >>>>> dts files weren't tested so far. They end up in the wrong directory >>>>> on the target and, thus, will not be found by the boot loader. I can >>>>> likely tell you in a moment which part exactly is wrong. >>>> ok thanks. the boards I tested either do not use dtb files or require >>>> a custom dtb path setting anyway >>> I strongly recommend to switch everything to ditro boot. That's >>> usually just a simple patch to the default u-boot env. More and more >>> vendors support that, and it's the unified way of booting Isar. >> I can't agree more (need to work with the vendor of the board I have to >> update their insanely old u-boot, it is insane to have it supported with >> mainline kernels but keep a 4-years old u-boot). > Ah, another "we are done" BSP. Yeah, never use BSPs... > > > BTW, another fallout from the full nightly build: > > NOTE: recipe isar-image-base-1.0-r0: task do_copy_boot_files: Started > ERROR: mc:qemumipsel-bullseye:isar-image-base-1.0-r0 do_copy_boot_files: Execution of '/builds/ebsy/debian/isar/build/tmp/work/debian-bullseye-mipsel/isar-image-base-qemumipsel-ext4-img/1.0-r0/temp/run.do_copy_boot_files.105304' failed with exit code 1: > cp: cannot stat '/builds/ebsy/debian/isar/build/tmp/work/debian-bullseye-mipsel/isar-image-base-qemumipsel-ext4-img/1.0-r0/rootfs/vmlinuz': No such file or directory > WARNING: exit code 1 from a shell command. > > ERROR: Logfile of failure stored in: /builds/ebsy/debian/isar/build/tmp/work/debian-bullseye-mipsel/isar-image-base-qemumipsel-ext4-img/1.0-r0/temp/log.do_copy_boot_files.105304 > Log data follows: > | DEBUG: Executing shell function do_copy_boot_files > | cp: cannot stat '/builds/ebsy/debian/isar/build/tmp/work/debian-bullseye-mipsel/isar-image-base-qemumipsel-ext4-img/1.0-r0/rootfs/vmlinuz': No such file or directory > | WARNING: exit code 1 from a shell command. > | ERROR: Execution of '/builds/ebsy/debian/isar/build/tmp/work/debian-bullseye-mipsel/isar-image-base-qemumipsel-ext4-img/1.0-r0/temp/run.do_copy_boot_files.105304' failed with exit code 1: > | cp: cannot stat '/builds/ebsy/debian/isar/build/tmp/work/debian-bullseye-mipsel/isar-image-base-qemumipsel-ext4-img/1.0-r0/rootfs/vmlinuz': No such file or directory > | WARNING: exit code 1 from a shell command. > | > NOTE: recipe isar-image-base-1.0-r0: task do_copy_boot_files: Failed > ERROR: Task (mc:qemumipsel-bullseye:/builds/ebsy/debian/isar/meta-isar/recipes-core/images/isar-image-base.bb:do_copy_boot_files) failed with exit code '1' > > Can send you the link the full log on our internal CI on > code.siemens.com. shouldn't be needed. looks like KERNEL_FILE setting isn't honored for this target. should be easy to determine why that isn't the case > > Jan > ^ permalink raw reply [flat|nested] 83+ messages in thread
* Re: [PATCH L-C v3 0/7] linux-custom recipe rework 2019-11-08 8:01 ` Cedric Hombourger @ 2019-11-08 8:16 ` Cedric Hombourger 0 siblings, 0 replies; 83+ messages in thread From: Cedric Hombourger @ 2019-11-08 8:16 UTC (permalink / raw) To: Jan Kiszka, isar-users On 11/8/19 9:01 AM, Cedric Hombourger wrote: > > On 11/8/19 8:54 AM, Jan Kiszka wrote: >> On 08.11.19 08:42, Cedric Hombourger wrote: >>> On 11/8/19 8:37 AM, Jan Kiszka wrote: >>>> On 08.11.19 08:27, Cedric Hombourger wrote: >>>>> On 11/8/19 8:12 AM, Jan Kiszka wrote: >>>>>> On 08.11.19 08:07, Cedric Hombourger wrote: >>>>>>> The Linux kernel is packaged a bit differently between Debian >>>>>>> variants >>>>>>> and in particular for the perf package(s) in Debian vs Ubuntu. >>>>>>> Since >>>>>>> there is no way the kernel build scripts would ever get distro >>>>>>> specific >>>>>>> support for building more than the kernel and headers, we need >>>>>>> our own >>>>>>> build/packaging scripts. >>>>>>> >>>>>>> Version 3 of this changeset was tested against ci_build.sh and some >>>>>>> images were >>>>>>> booted on hardware available here. No regressions were detected (to >>>>>>> date). This >>>>>>> was also tested on the creator-ci40-marduk machine (subject of >>>>>>> another patch >>>>>>> series). >>>>>> dts files weren't tested so far. They end up in the wrong directory >>>>>> on the target and, thus, will not be found by the boot loader. I can >>>>>> likely tell you in a moment which part exactly is wrong. >>>>> ok thanks. the boards I tested either do not use dtb files or require >>>>> a custom dtb path setting anyway >>>> I strongly recommend to switch everything to ditro boot. That's >>>> usually just a simple patch to the default u-boot env. More and more >>>> vendors support that, and it's the unified way of booting Isar. >>> I can't agree more (need to work with the vendor of the board I have to >>> update their insanely old u-boot, it is insane to have it supported >>> with >>> mainline kernels but keep a 4-years old u-boot). >> Ah, another "we are done" BSP. Yeah, never use BSPs... >> >> >> BTW, another fallout from the full nightly build: >> >> NOTE: recipe isar-image-base-1.0-r0: task do_copy_boot_files: Started >> ERROR: mc:qemumipsel-bullseye:isar-image-base-1.0-r0 >> do_copy_boot_files: Execution of >> '/builds/ebsy/debian/isar/build/tmp/work/debian-bullseye-mipsel/isar-image-base-qemumipsel-ext4-img/1.0-r0/temp/run.do_copy_boot_files.105304' >> failed with exit code 1: >> cp: cannot stat >> '/builds/ebsy/debian/isar/build/tmp/work/debian-bullseye-mipsel/isar-image-base-qemumipsel-ext4-img/1.0-r0/rootfs/vmlinuz': >> No such file or directory >> WARNING: exit code 1 from a shell command. >> >> ERROR: Logfile of failure stored in: >> /builds/ebsy/debian/isar/build/tmp/work/debian-bullseye-mipsel/isar-image-base-qemumipsel-ext4-img/1.0-r0/temp/log.do_copy_boot_files.105304 >> Log data follows: >> | DEBUG: Executing shell function do_copy_boot_files >> | cp: cannot stat >> '/builds/ebsy/debian/isar/build/tmp/work/debian-bullseye-mipsel/isar-image-base-qemumipsel-ext4-img/1.0-r0/rootfs/vmlinuz': >> No such file or directory >> | WARNING: exit code 1 from a shell command. >> | ERROR: Execution of >> '/builds/ebsy/debian/isar/build/tmp/work/debian-bullseye-mipsel/isar-image-base-qemumipsel-ext4-img/1.0-r0/temp/run.do_copy_boot_files.105304' >> failed with exit code 1: >> | cp: cannot stat >> '/builds/ebsy/debian/isar/build/tmp/work/debian-bullseye-mipsel/isar-image-base-qemumipsel-ext4-img/1.0-r0/rootfs/vmlinuz': >> No such file or directory >> | WARNING: exit code 1 from a shell command. >> | >> NOTE: recipe isar-image-base-1.0-r0: task do_copy_boot_files: Failed >> ERROR: Task >> (mc:qemumipsel-bullseye:/builds/ebsy/debian/isar/meta-isar/recipes-core/images/isar-image-base.bb:do_copy_boot_files) >> failed with exit code '1' >> >> Can send you the link the full log on our internal CI on >> code.siemens.com. > shouldn't be needed. looks like KERNEL_FILE setting isn't honored for > this target. should be easy to determine why that isn't the case from a quick code inspection: meta-isar/conf/machine/qemumipsel.conf:KERNEL_FILE ?= "vmlinux" sounds like it should be = not ?= but we could also provide a better default so we don't have to do this for every single MIPS machine our global def config has: KERNEL_FILE ?= "vmlinuz" we could add KERNEL_FILE_mipsel ?= "vmlinux" will add this change to the relevant patch series >> >> Jan >> ^ permalink raw reply [flat|nested] 83+ messages in thread
end of thread, other threads:[~2019-12-09 7:40 UTC | newest] Thread overview: 83+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2019-11-05 13:55 [PATCH 0/7] linux-custom recipe rework Cedric Hombourger 2019-11-05 13:55 ` [PATCH 1/7] recipes-kernel/linux: make KERNEL_DEFCONFIG support in-tree defconfigs Cedric Hombourger 2019-11-05 13:55 ` [PATCH 2/7] linux-mainline: fix stripping of .0 from the kernel version Cedric Hombourger 2019-11-05 13:55 ` [PATCH 3/7] linux-mainline: update from 4.19.0 to 4.19.80 Cedric Hombourger 2019-11-05 13:55 ` [PATCH 4/7] linux-custom: add support for kernel config fragments Cedric Hombourger 2019-11-05 13:57 ` [PATCH 5/7] linux-mainline: disable support for HFS to demonstrate use of " Cedric Hombourger 2019-11-05 13:57 ` [PATCH 6/7] buildchroot-host: install qemu-static to support hybrid cross-compiles Cedric Hombourger 2019-11-05 14:52 ` Jan Kiszka 2019-11-05 15:00 ` Cedric Hombourger 2019-11-05 15:27 ` Jan Kiszka 2019-11-05 13:57 ` [PATCH 7/7] linux-custom: rewrite to no longer depend on the kernel's builddeb Cedric Hombourger 2019-11-05 15:26 ` Jan Kiszka 2019-11-05 16:20 ` Cedric Hombourger 2019-11-05 16:22 ` Jan Kiszka 2019-11-05 16:25 ` Jan Kiszka 2019-11-05 20:16 ` chombourger 2019-11-05 20:24 ` Jan Kiszka 2019-11-07 11:44 ` Gylstorff Quirin 2019-11-07 11:55 ` Cedric Hombourger 2019-11-07 13:20 ` Cedric Hombourger 2019-11-07 15:43 ` Gylstorff Quirin 2019-11-05 15:32 ` [PATCH 0/7] linux-custom recipe rework Jan Kiszka 2019-11-05 15:55 ` Henning Schild 2019-11-05 16:00 ` Cedric Hombourger 2019-11-05 16:19 ` Henning Schild 2019-11-07 9:23 ` [PATCH L-C v2 " Cedric Hombourger 2019-11-07 9:23 ` [PATCH L-C v2 1/7] recipes-kernel/linux: make KERNEL_DEFCONFIG support in-tree defconfigs Cedric Hombourger 2019-11-07 9:23 ` [PATCH L-C v2 2/7] linux-mainline: fix stripping of .0 from the kernel version Cedric Hombourger 2019-11-07 9:23 ` [PATCH L-C v2 3/7] linux-mainline: update from 4.19.0 to 4.19.80 Cedric Hombourger 2019-11-07 9:23 ` [PATCH L-C v2 4/7] linux-custom: add support for kernel config fragments Cedric Hombourger 2019-11-07 9:23 ` [PATCH L-C v2 5/7] linux-mainline: disable support for HFS to demonstrate use of " Cedric Hombourger 2019-11-07 9:23 ` [PATCH L-C v2 6/7] buildchroot-host: install qemu-static to support hybrid cross-compiles Cedric Hombourger 2019-11-07 9:23 ` [PATCH L-C v2 7/7] linux-custom: rewrite to no longer depend on the kernel's builddeb Cedric Hombourger 2019-11-07 11:50 ` Jan Kiszka 2019-11-07 11:52 ` Jan Kiszka 2019-11-07 11:53 ` Cedric Hombourger 2019-11-07 12:14 ` Jan Kiszka 2019-11-09 8:51 ` Jan Kiszka 2019-11-09 13:43 ` Cedric Hombourger 2019-11-22 18:58 ` Jan Kiszka 2019-11-25 16:04 ` [PATCH L-C v4 0/5] linux-custom recipe rework Cedric Hombourger 2019-11-25 16:04 ` [PATCH L-C v4 1/5] recipes-kernel/linux: make KERNEL_DEFCONFIG support in-tree defconfigs Cedric Hombourger 2019-11-25 16:04 ` [PATCH L-C v4 2/5] linux-mainline: fix stripping of .0 from the kernel version Cedric Hombourger 2019-11-25 16:04 ` [PATCH L-C v4 3/5] linux-mainline: update from 4.19.0 to 4.19.80 Cedric Hombourger 2019-11-25 16:04 ` [PATCH L-C v4 4/5] buildchroot-host: install qemu-static to support hybrid cross-compiles Cedric Hombourger 2019-11-25 16:06 ` [PATCH L-C v4 5/5] linux-custom: rewrite to no longer depend on the kernel's builddeb Cedric Hombourger 2019-11-25 19:55 ` Jan Kiszka 2019-11-26 7:42 ` Cedric Hombourger 2019-11-26 9:13 ` Jan Kiszka 2019-11-26 17:10 ` Gylstorff Quirin 2019-11-25 16:16 ` [PATCH L-C v4 0/5] linux-custom recipe rework Jan Kiszka 2019-11-27 15:49 ` Gylstorff Quirin 2019-12-05 17:01 ` Baurzhan Ismagulov 2019-12-09 7:29 ` Jan Kiszka 2019-12-09 7:40 ` Cedric Hombourger 2019-11-07 11:39 ` [PATCH L-C v2 0/7] " Jan Kiszka 2019-11-07 11:47 ` chombourger 2019-11-07 11:48 ` Jan Kiszka 2019-11-08 7:07 ` [PATCH L-C v3 " Cedric Hombourger 2019-11-08 7:07 ` [PATCH L-C v3 1/7] recipes-kernel/linux: make KERNEL_DEFCONFIG support in-tree defconfigs Cedric Hombourger 2019-11-08 7:07 ` [PATCH L-C v3 2/7] linux-mainline: fix stripping of .0 from the kernel version Cedric Hombourger 2019-11-08 7:07 ` [PATCH L-C v3 3/7] linux-mainline: update from 4.19.0 to 4.19.80 Cedric Hombourger 2019-11-08 7:07 ` [PATCH L-C v3 4/7] linux-custom: add support for kernel config fragments Cedric Hombourger 2019-11-08 7:07 ` [PATCH L-C v3 5/7] linux-mainline: disable support for HFS to demonstrate use of " Cedric Hombourger 2019-11-08 7:07 ` [PATCH L-C v3 6/7] buildchroot-host: install qemu-static to support hybrid cross-compiles Cedric Hombourger 2019-11-08 7:14 ` Jan Kiszka 2019-11-08 7:22 ` Jan Kiszka 2019-11-08 7:23 ` Cedric Hombourger 2019-11-08 7:22 ` Cedric Hombourger 2019-11-08 7:35 ` Jan Kiszka 2019-11-08 7:59 ` Cedric Hombourger 2019-11-08 8:28 ` Jan Kiszka 2019-11-08 7:07 ` [PATCH L-C v3 7/7] linux-custom: rewrite to no longer depend on the kernel's builddeb Cedric Hombourger 2019-11-08 7:19 ` Jan Kiszka 2019-11-08 7:36 ` Cedric Hombourger 2019-11-08 7:41 ` Jan Kiszka 2019-11-08 7:12 ` [PATCH L-C v3 0/7] linux-custom recipe rework Jan Kiszka 2019-11-08 7:27 ` Cedric Hombourger 2019-11-08 7:37 ` Jan Kiszka 2019-11-08 7:42 ` Cedric Hombourger 2019-11-08 7:54 ` Jan Kiszka 2019-11-08 8:01 ` Cedric Hombourger 2019-11-08 8:16 ` Cedric Hombourger
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox