On Tuesday, November 5, 2019 at 4:26:33 PM UTC+1, Jan Kiszka wrote: > > On 05.11.19 14:57, Cedric Hombourger wrote: > > Regain control over the packaging of the Linux kernel by providing our > > own debian recipes (debian,{control,rules} files and friends) instead > > of using the kernel's builddeb script. This will allow generation of > > packages for "perf" (not included in this changeset) with per distro > > scheme (Debian and Ubuntu have differences) and maintain compatibility > > with upstream packages generated from different source packages but > > dependent on a certain deployment scheme (e.g. linux-base for Debian) > > > > Signed-off-by: Cedric Hombourger > > > --- > > doc/custom_kernel.md | 181 ++++++++++++++++ > > .../linux/files/build-kernel.sh | 128 ------------ > > meta/recipes-kernel/linux/files/debian/compat | 1 + > > .../linux/files/debian/control.tmpl | 36 ++++ > > meta/recipes-kernel/linux/files/debian/files | 5 + > > .../linux/files/debian/isar/build.tmpl | 36 ++++ > > .../linux/files/debian/isar/clean.tmpl | 20 ++ > > .../linux/files/debian/isar/common.tmpl | 60 ++++++ > > .../linux/files/debian/isar/install.tmpl | 197 ++++++++++++++++++ > > .../files/debian/linux-image.postinst.tmpl | 22 ++ > > .../files/debian/linux-image.postrm.tmpl | 18 ++ > > .../linux/files/debian/rules.tmpl | 39 ++++ > > meta/recipes-kernel/linux/linux-custom.inc | 172 ++++++++++----- > > 13 files changed, 732 insertions(+), 183 deletions(-) > > create mode 100644 doc/custom_kernel.md > > delete mode 100644 meta/recipes-kernel/linux/files/build-kernel.sh > > create mode 100644 meta/recipes-kernel/linux/files/debian/compat > > create mode 100644 meta/recipes-kernel/linux/files/debian/control.tmpl > > create mode 100644 meta/recipes-kernel/linux/files/debian/files > > create mode 100644 > meta/recipes-kernel/linux/files/debian/isar/build.tmpl > > create mode 100644 > meta/recipes-kernel/linux/files/debian/isar/clean.tmpl > > create mode 100644 > meta/recipes-kernel/linux/files/debian/isar/common.tmpl > > create mode 100644 > meta/recipes-kernel/linux/files/debian/isar/install.tmpl > > create mode 100644 > meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl > > create mode 100644 > meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl > > create mode 100755 meta/recipes-kernel/linux/files/debian/rules.tmpl > > > > diff --git a/doc/custom_kernel.md b/doc/custom_kernel.md > > new file mode 100644 > > index 0000000..5ab2599 > > --- /dev/null > > +++ b/doc/custom_kernel.md > > @@ -0,0 +1,181 @@ > > +# Custom kernel recipe for Isar > > + > > +## Contents > > + > > + - [Summary](#summary) > > + - [Proposal owners](#proposal-owners) > > + - [Current status](#current-status) > > + - [Detailed description](#detailed-description) > > + - [Benefit to Isar](#benefit-to-isar) > > + - [How can I help?](#how-can-i-help) > > + - [Scope](#scope) > > + - [How to test](#how-to-test) > > + - [Dependencies](#dependencies) > > + - [Documentation](#documentation) > > + - [Questions and answers](#questions-and-answers) > > + > > +## Summary > > + > > +Isar currently uses the Linux kernel's builddeb script to generate > linux-image, > > +linux-headers and linux-libc-dev packages for your custom kernel > sources. The > > +main benefit of this approach was that the low level details of how to > package > > +the Linux kernel image, modules and headers were left in the kernel. > There are > > +however some drawbacks: Isar had to repack generated packages to > introduce some > > +of the changes it needs such as suffixing the kernel packages with the > name of > > +the custom kernel (e.g. with "mainline" for the "linux-mainline" > recipe), lack > > +of automatic dependencies (the builddeb script from the kernel does not > use > > +debhelper), it was difficult to add packages especially distribution > specific > > +packages (maintain version specific patch in Isar vs being able to > upstream > > +distribution specific changes upstream), etc. This change is about > being less > > +dependent on the kernel packaging scripts (which aren't used by major > distros > > +anyway) and regain control on how we want or need the Linux kernel > packaged. > > + > > +## Proposal owners > > + > > + * name: [Cedric Hombourger](https://github.com/chombourger) > > + > > +## Current status > > + > > +### Tests > > + > > +The following were recently checked: > > + > > + * Custom kernels in meta-isar build without (known) failures > > + * initrd image gets generated when the kernel is added to the rootfs > > + * `ISAR_CROSS_COMPILE` is supported > > + * `KERNEL_FILE` is honored > > + * `KERNEL_DEFCONFIG` may either be a file (specified via `SRC_URI`) or > a > > + kernel-provided defconfig > > + * Config fragments get merged with the user-specified kernel > configuration > > + * Path to installed DTBs may be configured (new) > > + * Configuration and build from a `devshell` > > + * Support for the `KERNEL_LIBC_DEV_DEPLOY` directive > > + > > +while the following needs work: > > + > > + * Check if custom-kernel builds are reproducible > > + > > +### Remaining work items > > + > > + * Check if there is a better way to cross compile kernel "scripts" > such as > > + `fixdep` or `modpost` than manually compiling them (they are > compiled for > > + the build machine (amd64) and not for the target (e.g. armhf) even > though > > + we are cross-compiling - yet the linux-headers package should be > shipping > > + "scripts" for the target machine) > > + > > +## Detailed description > > + > > +### Requirements > > + > > +#### Current > > + > > +Isar has had a `linux-custom` recipe for a long time and counts many > users. It is therefore > > +paramount to maintain features that it has today with no or little > changes to the APIs. The > > +following requirements were identified and will be maintained: > > + > > + 1. Sources to the custom Linux kernel may be specified via `SRC_URI` > > + > > + 2. Configure the kernel via an in-tree or an external `defconfig` > > Reference to KERNEL_DEFCONFIG is missing. > > > + > > + 3. Support kernel configuration tweaks via configuration fragments > (`.cfg` files) > > + > > + 4. Patches to the linux kernel may be specified via `SRC_URI` > > + > > + 5. Ensure that the Isar recipe `PV` matches the kernel release > identifier (`KERNEL_RELEASE`) > > + > > + 6. Produce a `linux-image` package that ships the kernel image and > modules > > + > > + 7. Allow the name of the kernel image to be changed via `KERNEL_FILE` > (defaults to `vmlinuz`) > > + > > + 8. Produce a `linux-headers` package which includes kernel headers and > kbuild scripts/tools > > + > > + 9. The `linux-headers` package shall support native and cross compiles > of out-of-tree kernel modules > > One package to serve them all? Is that the use case for patch 6? IOW, we > are now building linux-headers for the target, not the host OS anymore? > What's the slowdown compare to cross-linux-headers? > > The current policy is that, when cross-building, our -dev packagages > (which is what linux-headers falls under) will work natively with the > buildchroot. The future path is clearly generating fitting packages for > both host and target. Or is there anything in the linux-headers that > prevents providing both? > > > + > > + 10. Produce a `linux-libc-dev` package to support user-land builds > > This should be opt-in only: No need to build it when we don't ship it. > Just add the related fragement to debian/control when > KERNEL_LIBC_DEV_DEPLOY is set. > > > + > > + 11. Only ship the `linux-libc-dev` package if instructed to > (`KERNEL_LIBC_DEV_DEPLOY` equals to `"1"`) > > + > > + 12. Support both native and cross compiles (`ISAR_CROSS_COMPILE`) > > + > > + 13. Support for the following kernel architectures: > > + > > + * arm > > + * arm64 > > + * mips > > + * x86 > > + > > + 14. Support `devshell` (kernel configuration shall be applied) > > + > > +#### New requirements (proposed) > > + > > +This proposal includes the following new requirements: > > + > > + 1. Support creation of additional packages > > + > > + 2. Allow per-distro packaging tweaks > > + > > +### Approach > > + > > +The new implementation will have the `builddeb` code ported from the > Linux kernel included to the > > +Isar environment. It will be refactor to support pre and post hooks. > Hooks may be used by external > > +layers to extend the kernel packaging instructions provided by "stock" > Isar or by Isar itself to > > +implement distro-specific packages or tweaks. > > + > > +Hooks need the ability to add `build` and `install` steps which can be > achieved by having the main > > +`build` and `install` steps call pre and post scripts. This will > however solve only one side of the > > +problem: new packages should be declared in the `debian/control` file. > The packaging process > > +(initiated by `debian/rules`) will use a baseline `control` file when > started and a mechanism will > > +be provided for hooks to append `control` blocks as they are processed. > > Is this a section addressing users or reviewers? The latter audience is > better addressed via the commit log. > > > + > > +### Future work > > + > > +In the event where this proposal is accepted, here are a few future > projects that are being > > +considered: > > + > > + 1. Package `perf` as Debian does > > + > > + 2. Add `ubuntu` as an Isar supported target distro and validate > `linux-custom` recipe > > + > > +## Benefit to Isar > > + > > + * Control the kernel build and packaging process > > + > > + * No longer need to repack packages > > + > > + * Provide hooks for custom kernel recipes to add build/install > directives > > Careful with hooks! They tend to pile up mess. Where exactly do you want > them? What for? > > > + > > +## How can I help? > > + > > + * Check if you are able to build your own linux-custom recipes with > this change > > + * Check if packages generated by this recipe are compatible with your > Debian-based distro > > + * Review and provide comments on this changeset > > + * Create templates for distro-specific packages (e.g. perf) > > + > > +## Scope > > + > > + * Proposal owners: proposal owners are to communicate the idea to > isar-users and make sure > > + that all of their requirements are met (to the extent they are > already supported with the > > + current solution) or may be met in the future. > > + > > + * Other developers: port your out-of-tree linux-custom recipes to this > new solution and > > + report issues. > > + > > +## How to test? > > + > > +The linux-mainline recipe may be used for some basic testing. This > recipe is being used by the > > +following machines: > > + > > + * de0-nano-soc > > + * qemumipsel > > + > > +## Dependencies > > + > > + * None > > + > > +## Documentation > > + > > + * Document user-visible variables added by this proposal > > + > > +## Questions and answers > > + > > + * None > > diff --git a/meta/recipes-kernel/linux/files/build-kernel.sh > b/meta/recipes-kernel/linux/files/build-kernel.sh > > deleted file mode 100644 > > index f56e96c..0000000 > > --- a/meta/recipes-kernel/linux/files/build-kernel.sh > > +++ /dev/null > > @@ -1,128 +0,0 @@ > > -#!/bin/bash > > -# > > -# Custom kernel build > > -# > > -# This software is a part of ISAR. > > -# Copyright (c) Siemens AG, 2018 > > -# > > -# SPDX-License-Identifier: MIT > > - > > -source /isar/common.sh > > - > > -host_arch=$(dpkg --print-architecture) > > - > > -if [ "$host_arch" != "$target_arch" ]; then > > - case $target_arch in > > - armhf) > > - export ARCH=arm > > - export CROSS_COMPILE="arm-linux-gnueabihf-" > > - ;; > > - arm64) > > - export ARCH=arm64 > > - export CROSS_COMPILE="aarch64-linux-gnu-" > > - ;; > > - mipsel) > > - export ARCH=mips > > - export CROSS_COMPILE="mipsel-linux-gnu-" > > - ;; > > - *) > > - echo "error: unsupported architecture ($target_arch)" > > - exit 1 > > - ;; > > - esac > > -fi > > - > > -REPACK_DIR="$1/../repack" > > -REPACK_LINUX_IMAGE_DIR="${REPACK_DIR}/linux-image" > > -REPACK_LINUX_HEADERS_DIR="${REPACK_DIR}/linux-headers" > > - > > -make ${KERNEL_CONFIG_TARGET} || exit ${?} > > -if [ -n "${KERNEL_FRAGMENTS}" ]; then > > - scripts/kconfig/merge_config.sh -m .config ${KERNEL_FRAGMENTS} > > -fi > > - > > -KV=$( make -s kernelrelease ) > > -if [ "${KV}" != "${PV}" ]; then > > - echo "ERROR: Recipe PV is \"${PV}\" but should be \"${KV}\"" > 1>&2 > > - echo "ERROR: Probably due to CONFIG_LOCALVERSION" 1>&2 > > - exit 1 > > -fi > > - > > -rm -f .version > > -KBUILD_DEBARCH=$target_arch make -j $(($(nproc) * 2)) deb-pkg > > - > > -rm -rf "${REPACK_DIR}" > > -mkdir -p "${REPACK_DIR}" > > -mkdir -p "${REPACK_LINUX_IMAGE_DIR}" > > -mkdir -p "${REPACK_LINUX_HEADERS_DIR}" > > - > > -cp -a debian "${REPACK_DIR}" > > - > > -# dpkg-gencontrol performs cross-incompatible checks on the > > -# Architecture field; trick it to accept the control file > > -sed -i "s/Architecture: .*/Architecture: any/" > "${REPACK_DIR}/debian/control" > > - > > -cd .. > > - > > -dpkg-deb -R linux-image-${PV}_${PV}-1_*.deb "${REPACK_LINUX_IMAGE_DIR}" > > -dpkg-deb -R linux-headers-${PV}_${PV}-1_*.deb > "${REPACK_LINUX_HEADERS_DIR}" > > - > > -dpkg-gencontrol -crepack/debian/control \ > > - -lrepack/debian/changelog \ > > - -frepack/debian/files \ > > - -plinux-image-${PV} \ > > - -P"${REPACK_LINUX_IMAGE_DIR}" \ > > - -DPackage="linux-image-${KERNEL_NAME}" \ > > - -DSection=kernel \ > > - -DPriority=required \ > > - -DDepends="${KERNEL_DEBIAN_DEPENDS}" \ > > - -DArchitecture=$target_arch > > - > > -# Add Debian-like link installation to postinst > > -mkdir -p ${REPACK_LINUX_IMAGE_DIR}/lib/modules/${PV} > > -touch "${REPACK_LINUX_IMAGE_DIR}/lib/modules/${PV}/.fresh-install" > > -sed -i "${REPACK_LINUX_IMAGE_DIR}/DEBIAN/postinst" \ > > - -e "/^set -e$/a\\ > > -\\ > > -if [ -f /lib/modules/${PV}/.fresh-install ]; then\\ > > - change=install\\ > > -else\\ > > - change=upgrade\\ > > -fi\\ > > -linux-update-symlinks \$change ${PV} /boot/${KERNEL_FILE}-${PV}\\ > > -rm -f /lib/modules/${PV}/.fresh-install" > > - > > -# Add Debian-like link removal to postrm > > -sed -i "${REPACK_LINUX_IMAGE_DIR}/DEBIAN/postrm" \ > > - -e "/^set -e$/a\\ > > -\\ > > -rm -f /lib/modules/${PV}/.fresh-install\\ > > -\\ > > -if [ \"\$1\" != upgrade ] && command -v linux-update-symlinks > >/dev/null; then\\ > > - linux-update-symlinks remove ${PV} > /boot/${KERNEL_FILE}-${PV}\\ > > -fi" > > - > > -# Make sure arm64 kernels are decompressed > > -if [ "$target_arch" = "arm64" ]; then > > > - kernel_file="${REPACK_LINUX_IMAGE_DIR}/boot/${KERNEL_FILE}-${PV}" > > - mv "${kernel_file}" "${kernel_file}.gz" > > - gunzip "${kernel_file}.gz" > > -fi > > - > > -dpkg-gencontrol -crepack/debian/control \ > > - -lrepack/debian/changelog \ > > - -frepack/debian/files \ > > - -plinux-headers-${PV} \ > > - -P"${REPACK_LINUX_HEADERS_DIR}" \ > > - -Vkernel:debarch="${KERNEL_NAME}" \ > > - -DPackage="linux-headers-${KERNEL_NAME}" \ > > - -DSection=kernel \ > > - -DDepends="${KERNEL_HEADERS_DEBIAN_DEPENDS}" \ > > - -DArchitecture=$target_arch > > - > > -fakeroot dpkg-deb -b "${REPACK_LINUX_IMAGE_DIR}" \ > > - linux-image-${KERNEL_NAME}_${PV}-1_${KERNEL_NAME}.deb > > -rm -f linux-image-${PV}_${PV}-1_*.deb > > -fakeroot dpkg-deb -b "${REPACK_LINUX_HEADERS_DIR}" \ > > - linux-headers-${KERNEL_NAME}_${PV}-1_${KERNEL_NAME}.deb > > -rm -f linux-headers-${PV}_${PV}-1_*.deb > > diff --git a/meta/recipes-kernel/linux/files/debian/compat > b/meta/recipes-kernel/linux/files/debian/compat > > new file mode 100644 > > index 0000000..ec63514 > > --- /dev/null > > +++ b/meta/recipes-kernel/linux/files/debian/compat > > @@ -0,0 +1 @@ > > +9 > > diff --git a/meta/recipes-kernel/linux/files/debian/control.tmpl > b/meta/recipes-kernel/linux/files/debian/control.tmpl > > new file mode 100644 > > index 0000000..1295466 > > --- /dev/null > > +++ b/meta/recipes-kernel/linux/files/debian/control.tmpl > > @@ -0,0 +1,36 @@ > > +Source: linux-${KERNEL_NAME_PROVIDED} > > +Section: kernel > > +Priority: optional > > +Maintainer: ${MAINTAINER} > > +Build-Depends: bc, kmod, cpio, ${KBUILD_DEPENDS} > > +Homepage: http://www.kernel.org/ > > Standard-Version? > > > + > > +Package: linux-image-${KERNEL_NAME_PROVIDED} > > +Architecture: any > > +Depends: ${KERNEL_DEBIAN_DEPENDS} > > +Description: ${KERNEL_NAME_PROVIDED} Linux kernel, version ${PV} > > + This package contains the Linux kernel, modules and corresponding > other > > + files, version: ${PV}. > > + > > +Package: linux-headers-${KERNEL_NAME_PROVIDED} > > +Architecture: any > > +Depends: ${KERNEL_HEADERS_DEBIAN_DEPENDS}, ${perl:Depends}, > ${shlib:Depends} > > +Description: ${KERNEL_NAME_PROVIDED} Linux kernel headers for ${PV} > > + This package provides kernel header files for ${PV} on ${DISTRO_ARCH} > > + . > > + This is useful for people who need to build external modules > > + > > +Package: linux-libc-dev > > See above, should not be here by default. > > > +Section: devel > > +Provides: linux-kernel-headers > > +Architecture: any > > +Description: Linux support headers for userspace development > > + This package provides userspaces headers from the Linux kernel. These > headers > > + are used by the installed headers for GNU glibc and other system > libraries. > > + > > +Package: linux-image-${KERNEL_NAME_PROVIDED}-dbg > > +Section: debug > > +Architecture: any > > +Description: Linux kernel debugging symbols for ${PV} > > + This package will come in handy if you need to debug the kernel. It > provides > > + all the necessary debug symbols for the kernel and its modules. > > diff --git a/meta/recipes-kernel/linux/files/debian/files > b/meta/recipes-kernel/linux/files/debian/files > > new file mode 100644 > > index 0000000..a8ace32 > > --- /dev/null > > +++ b/meta/recipes-kernel/linux/files/debian/files > > @@ -0,0 +1,5 @@ > > +linux-headers_4.19.80-1_mipsel.deb kernel optional > > +linux-image-dbg_4.19.80-1_mipsel.deb debug optional > > +linux-image_4.19.80-1_mipsel.deb kernel optional > > +linux-libc-dev_4.19.80-1_mipsel.deb devel optional > > +linux_4.19.80-1_mipsel.buildinfo kernel optional > > Looks like an accidentally added file. > > > diff --git a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl > b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl > > new file mode 100644 > > index 0000000..ce79d56 > > --- /dev/null > > +++ b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl > > @@ -0,0 +1,36 @@ > > +#!/bin/sh > > +# Copyright (c) Mentor Graphics, a Siemens business, 2019 > > +# SPDX-License-Identifier: MIT > > + > > +# Load common stuff > > +. ${S}/debian/isar/common || exit ${?} > > + > > +do_build() { > > + > > + # Print a few things that are of particular interest > > + print_settings > > + > > + # Trace what we do here > > + set -x > > + > > + # Process existing kernel configuration to make sure it is complete > > + # (use defaults for options that were not specified) > > + ${MAKE} O=${KERNEL_BUILD_DIR} olddefconfig || exit ${?} > > + > > + # Build the Linux kernel > > + ${MAKE} O=${KERNEL_BUILD_DIR} -j $(nproc) || exit ${?} > > + > > + # Stop tracing > > + set +x > > +} > > + > > +print_settings() { > > + cat < > +# Build settings: > > +# --------------- > > +# ARCH=${ARCH} > > +# CROSS_COMPILE=${CROSS_COMPILE} > > +EOF > > +} > > + > > +main build ${*} > > diff --git a/meta/recipes-kernel/linux/files/debian/isar/clean.tmpl > b/meta/recipes-kernel/linux/files/debian/isar/clean.tmpl > > new file mode 100644 > > index 0000000..2aa3742 > > --- /dev/null > > +++ b/meta/recipes-kernel/linux/files/debian/isar/clean.tmpl > > @@ -0,0 +1,20 @@ > > +#!/bin/sh > > +# Copyright (c) Mentor Graphics, a Siemens business, 2019 > > +# SPDX-License-Identifier: MIT > > + > > +# Load common stuff > > +. ${S}/debian/isar/common || exit ${?} > > + > > +do_clean() { > > + > > + # Trace what we do here > > + set -x > > + > > + rm -rf ${deb_img_dir} ${deb_dbg_dir} > > + rm -rf ${deb_kern_hdrdir} ${deb_libc_hdr_dir} > > I suppose we rather (or also?) want "make clean" here for the build > artifacts. Just make sure to save/restore the debian folder across that > step. > > > + > > + # Stop tracing > > + set +x > > +} > > + > > +main clean ${*} > > diff --git a/meta/recipes-kernel/linux/files/debian/isar/common.tmpl > b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl > > new file mode 100644 > > index 0000000..b392b46 > > --- /dev/null > > +++ b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl > > @@ -0,0 +1,60 @@ > > +#!/bin/sh > > +# Copyright (c) Mentor Graphics, a Siemens business, 2019 > > +# SPDX-License-Identifier: MIT > > + > > +# Isar settings > > +ARCH=${KERNEL_ARCH} > > +KERNEL_PKG_IMAGE=linux-image-${KERNEL_NAME_PROVIDED} > > +KERNEL_PKG_KERN_HEADERS=linux-headers-${KERNEL_NAME_PROVIDED} > > +KERNEL_PKG_LIBC_HEADERS=linux-libc-dev > > + > > +# Constants > > +KCONF=.config > > + > > +# Target directories > > +deb_top_dir=${S}/debian > > +deb_img_dir=${deb_top_dir}/${KERNEL_PKG_IMAGE} > > +deb_dbg_dir=${deb_img_dir}-dbg > > +deb_dtb_dir=${deb_img_dir}/${KERNEL_DTB_DIR} > > +deb_kern_hdr_dir=${deb_top_dir}/${KERNEL_PKG_KERN_HEADERS} > > +deb_libc_hdr_dir=${deb_top_dir}/${KERNEL_PKG_LIBC_HEADERS} > > + > > +main() { > > + target=${1} > > + > > + if [ ! -f ${S}/debian/isar/${target} ]; then > > + echo "error: ${target} is not a supported build target!" >&2 > > + return 1 > > + fi > > + > > + # check for scripts to be sourced > > + for f in ${S}/debian/isar/defaults.d/${target}/*; do > > + [ -f ${f} ] || continue > > + . ${f} || return ${?} > > + done > > + > > + # variables to be exported > > + export ARCH > > + > > + # are we cross-compiling? > > + BUILD_ARCH=$(dpkg-architecture -qDEB_BUILD_ARCH) > > + if [ "${BUILD_ARCH}" = "${DISTRO_ARCH}" ]; then > > + # no, make sure CROSS_COMPILE isn't set > > + unset CROSS_COMPILE > > + fi > > + > > + # check for pre-target scripts > > + for f in ${S}/debian/isar/pre.d/${target}/*; do > > + [ -f ${f} ] || continue > > + sh ${f} || return ${?} > > + done > > + > > + # call the actual target script > > + do_${target} || return ${?} > > + > > + # check for post-target scripts > > + for f in ${S}/debian/isar/post.d/${target}/*; do > > + [ -f ${f} ] || continue > > + sh ${f} || return ${?} > > + done > > +} > > diff --git a/meta/recipes-kernel/linux/files/debian/isar/install.tmpl > b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl > > new file mode 100644 > > index 0000000..2055669 > > --- /dev/null > > +++ b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl > > @@ -0,0 +1,197 @@ > > +#!/bin/sh > > +# Copyright (c) Mentor Graphics, a Siemens business, 2019 > > +# SPDX-License-Identifier: MIT > > + > > +# Load common stuff > > +. ${S}/debian/isar/common || exit ${?} > > + > > +do_install() { > > + > > + # check if our kernel was configured > > + if [ ! -f "${O}/.config" ]; then > > + echo "error: kernel not configured!" >&2 > > + return 1 > > + fi > > + > > + # load its configuration > > + . ${O}/.config > > + > > + kimage="$(${MAKE} O=${O} -s --no-print-directory image_name)" > > + case "${ARCH}" in > > + mips|powerpc) kimage_path="boot/vmlinux-${PV}" ;; > > + um) kimage_path="usr/bin/vmlinux-${PV}" ;; > > + *) kimage_path="boot/vmlinuz-${PV}" ;; > > + esac > > + > > + print_settings > > + > > + # Stop on error > > + set -e > > + > > + # Trace what we do here > > + set -x > > + > > + install_image > > + install_hooks > > + install_dtbs > > + install_kmods > > + install_headers > > + > > + # Stop tracing > > + set +x > > +} > > + > > +print_settings() { > > + cat < > +Install settings: > > +----------------- > > +deb_dtb_dir=${deb_dtb_dir} > > +deb_hdr_dir=${deb_hdr_dir} > > +kimage=${kimage} > > +kimage_path=${kimage_path} > > + > > +EOF > > +} > > + > > +install_image() { > > + install -m 755 -d ${deb_img_dir}/$(dirname ${kimage_path}) > > + cp ${O}/${kimage} ${deb_img_dir}/${kimage_path} > > + install_image_debug > > +} > > + > > +install_image_debug() { > > + # Different tools want the image in different locations > > + # perf > > + mkdir -p ${deb_dbg_dir}/usr/lib/debug/lib/modules/${PV}/ > > + cp vmlinux ${deb_dbg_dir}/usr/lib/debug/lib/modules/${PV}/ > > + # systemtap > > + mkdir -p ${deb_dbg_dir}/usr/lib/debug/boot/ > > + ln -s ../lib/modules/$version/vmlinux > ${deb_dbg_dir}/usr/lib/debug/boot/vmlinux-${PV} > > + # kdump-tools > > + ln -s lib/modules/${PV}/vmlinux > ${deb_dbg_dir}/usr/lib/debug/vmlinux-${PV} > > +} > > + > > +install_hooks() { > > + install -m 755 -d ${deb_img_dir}/etc/kernel/install.d > > + install -m 755 -d ${deb_img_dir}/etc/kernel/postinst.d > > + install -m 755 -d ${deb_img_dir}/etc/kernel/postrm.d > > + install -m 755 -d ${deb_img_dir}/etc/kernel/prerm.d > > +} > > + > > +install_dtbs() { > > + [ -n "${CONFIG_OF}" ] || return 0 > > + ${MAKE} O=${O} INSTALL_DTBS_PATH=${deb_dtb_dir} dtbs_install > > +} > > + > > +install_kmods() { > > + [ -n "${CONFIG_MODULES}" ] || return 0 > > + ${MAKE} O=${O} INSTALL_MOD_PATH=${deb_img_dir} modules_install > > + touch ${deb_img_dir}/lib/modules/${PV}/.fresh-install > > + rm -fv ${deb_img_dir}/lib/modules/${PV}/build > > + rm -fv ${deb_img_dir}/lib/modules/${PV}/source > > + install_kmods_debug > > +} > > + > > +kmods_sign() { > > + [ -n "${CONFIG_MODULE_SIG_ALL}" ] || return 0 > > + ${MAKE} O=${O} INSTALL_MOD_PATH=${deb_img_dir} modules_sign > > +} > > + > > +install_kmods_debug() { > > + [ -n "${CONFIG_DEBUG_INFO}" ] || return 0 > > + > > + kmod_inst_dir=${deb_img_dir}/lib/modules > > + kmod_debug_dir=${deb_dbg_dir}/usr/lib/debug > > + > > + # copy kernels modules to usr/lib/debug > > + mkdir -p ${kmod_debug_dir} > > + tar -C ${kmod_inst_dir}/ -cO --exclude='modules.*' . | tar -C > ${kmod_debug_dir}/ -xf - > > + # strip everything but debug sections for modules in usr/lib/debug > > + find ${kmod_debug_dir} -name *.ko -exec ${CROSS_COMPILE}objcopy > --only-keep-debug {} \; > > + # and strip debug sections from modules in lib/modules > > + find ${kmod_inst_dir} -name *.ko -exec ${CROSS_COMPILE}objcopy > --strip-debug {} \; > > + > > + # re-sign stripped kernel modules > > + kmods_sign > > +} > > + > > +headers_check() { > > + ${MAKE} O=${O} headers_check > > +} > > + > > +libc_headers() { > > + mkdir -p ${deb_libc_hdr_dir} > > + ${MAKE} O=${O} headers_install > INSTALL_HDR_PATH=${deb_libc_hdr_dir}/usr > > + host_arch=$(dpkg-architecture -a${DISTRO_ARCH} > -qDEB_HOST_MULTIARCH) > > + mkdir ${deb_libc_hdr_dir}/usr/include/${host_arch} > > + mv ${deb_libc_hdr_dir}/usr/include/asm > ${deb_libc_hdr_dir}/usr/include/${host_arch}/ > > +} > > + > > +# the kernel does not seem to provide a way to cross-compile its tools, > we however > > +# have qemu-user-static binaries in our buildchroot. Create a new build > tree for the > > +# scripts and override HOSTCC to force a cross-compile > > +kernel_tools_build() { > > + odir="${S}/build-scripts" > > + mflags="HOSTCC=${CROSS_COMPILE}gcc O=${odir} NOSTDINC_FLAGS=" > > + rm -rf ${odir} && mkdir -p ${odir} && cp > ${S}/${KERNEL_BUILD_DIR}/.config ${odir} > > + (unset LD_PRELOAD; cd ${odir} && make -C ${S} ${mflags} > olddefconfig scripts) > > + (cd ${odir}; find scripts -type f -executable -exec file {} \;|grep > ELF|cut -d: -f1) >${src_hdr_files} > > + tar -C ${odir} -cf - -T - <${src_hdr_files} | tar -C ${destdir} -xf > - > > +} > > + > > +kernel_tools() { > > + # remove object files > > + find ${destdir}/scripts -type f -name '*.o' |xargs rm -f > > + > > + # we're all done if we aren't cross-compiling > > + [ -n "${CROSS_COMPILE}" ] || return 0 > > + > > + # remove ELF executables from the linux-headers package > > + find ${destdir}/scripts -type f -exec file {} \;|grep ELF|cut -d: > -f1|xargs rm -fv > > + > > + # cross-compile kernel tools to be shipped with linux-headers > > + kernel_tools_build > > +} > > + > > +kernel_headers() { > > + destdir=${deb_kern_hdr_dir}/${KERNEL_HEADERS_DIR} > > + src_hdr_files=$(mktemp) > > + obj_hdr_files=$(mktemp) > > + > > + mkdir -p ${destdir} > > + mkdir -p ${deb_kern_hdr_dir}/lib/modules/${PV} > > + > > + (cd ${S}; find . -name 'Makefile*' -o -name 'Kconfig*' -o -name > '*.pl') >>${src_hdr_files} > > + (cd ${S}; find arch/*/include include scripts -type f -o -type l) > >>${src_hdr_files} > > + (cd ${S}; find arch/${ARCH} -name module.lds -o -name > Kbuild.platforms -o -name Platform) >>${src_hdr_files} > > + (cd ${S}; find $(find arch/${ARCH} -name include -o -name scripts > -type d) -type f) >>${src_hdr_files} > > + > > + (cd ${O}; find arch/${ARCH}/include Module.symvers include scripts > -type f) >>${obj_hdr_files} > > + if [ -n "${CONFIG_STACK_VALIDATION}" ]; then > > + (cd ${O}; find tools/objtool -type f -executable) > >>${obj_hdr_files} > > + fi > > + if [ -n "${CONFIG_GCC_PLUGINS}" ]; then > > + (cd ${O}; find scripts/gcc-plugins -name *.so -o -name > gcc-common.h) >>${obj_hdr_files} > > + fi > > + > > + # deploy files that were matched above > > + tar -C ${S} -cf - -T - <${src_hdr_files} | tar -C ${destdir} -xf - > > + tar -C ${O} -cf - -T - <${obj_hdr_files} | tar -C ${destdir} -xf - > > + > > + # add the kernel config > > + cp ${O}/${KCONF} ${destdir}/.config > > + > > + # handle kernel development tools > > + kernel_tools > > + > > + # create symlinks > > + ln -sf /${KERNEL_HEADERS_DIR} > ${deb_kern_hdr_dir}/lib/modules/${PV}/build > > +} > > + > > +install_headers() { > > + headers_check > > + libc_headers > > + kernel_headers > > +} > > + > > +main install ${*} > > diff --git > a/meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl > b/meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl > > new file mode 100644 > > index 0000000..fd89121 > > --- /dev/null > > +++ b/meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl > > @@ -0,0 +1,22 @@ > > +#!/bin/sh > > + > > +set -e > > +set -x > > + > > +if [ -f /lib/modules/${PV}/.fresh-install ]; then > > + change=install > > +else > > + change=upgrade > > +fi > > + > > +linux-update-symlinks $change ${PV} /boot/${KERNEL_FILE}-${PV} > > +rm -f /lib/modules/${PV}/.fresh-install > > + > > +# Pass maintainer script parameters to hook scripts > > +export DEB_MAINT_PARAMS="$*" > > + > > +# Tell initramfs builder whether it's wanted > > +export INITRD=Yes > > I've noticed while hacking my version that this should follow > CONFIG_BLK_DEV_INITRD. > > > + > > +test -d /etc/kernel/postinst.d && run-parts --arg="${PV}" > --arg="/boot/${KERNEL_FILE}-${PV}" /etc/kernel/postinst.d > > +exit 0 > > diff --git > a/meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl > b/meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl > > new file mode 100644 > > index 0000000..9d88218 > > --- /dev/null > > +++ b/meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl > > @@ -0,0 +1,18 @@ > > +#!/bin/sh > > + > > +set -e > > + > > +rm -f /lib/modules/${PV}/.fresh-install > > + > > +if [ "$1" != upgrade ] && command -v linux-update-symlinks >/dev/null; > then > > + linux-update-symlinks remove ${PV} /boot/${KERNEL_FILE}-${PV} > > +fi > > + > > +# Pass maintainer script parameters to hook scripts > > +export DEB_MAINT_PARAMS="$*" > > + > > +# Tell initramfs builder whether it's wanted > > +export INITRD=Yes > > + > > +test -d /etc/kernel/postrm.d && run-parts --arg="${PV}" > --arg="/boot/${KERNEL_FILE}-${PV}" /etc/kernel/postrm.d > > +exit 0 > > diff --git a/meta/recipes-kernel/linux/files/debian/rules.tmpl > b/meta/recipes-kernel/linux/files/debian/rules.tmpl > > There is quite some duplication between postinst and postrm already. > Moreover, I think you are missing preinst and prerm. I think all four > should probably be generated from one template. > Ok I have checked upstream scripts and it seems that they all differ see https://salsa.debian.org/kernel-team/linux/tree/master/debian/templates I would recommend we just copy over their scripts but use our templating system instead of theirs (use ${var} instead of @var@) > > > new file mode 100755 > > index 0000000..93c8a5b > > --- /dev/null > > +++ b/meta/recipes-kernel/linux/files/debian/rules.tmpl > > @@ -0,0 +1,39 @@ > > +#!/usr/bin/make -f > > + > > +export CROSS_COMPILE=$(DEB_HOST_GNU_TYPE)- > > + > > +O:=$(CURDIR)/${KERNEL_BUILD_DIR} > > +S:=$(CURDIR) > > +deb_top_dir:=$(S)/debian > > + > > +DH_VERBOSE=1 > > + > > +# Dynamic variables to be passed to Isar build scripts > > +isar_env=$(strip \ > > + export CROSS_COMPILE='$(CROSS_COMPILE)' && \ > > + export MAKE='$(MAKE)' && \ > > + export O='${O}' \ > > + export S='${S}' \ > > +) > > + > > +%: > > + dh $(@) > > + > > +.PHONY: override_dh_auto_clean > > +override_dh_auto_clean: > > + $(isar_env) && sh $(deb_top_dir)/isar/clean > > + > > +.PHONY: override_dh_auto_build > > +override_dh_auto_build: > > + $(isar_env) && sh $(deb_top_dir)/isar/build > > + > > +override_dh_auto_install: > > + $(isar_env) && sh $(deb_top_dir)/isar/install > > + > > +.PHONY: override_dh_auto_test > > +override_dh_auto_test: > > + true > > + > > +PHONY: override_dh_strip > > +override_dh_strip: > > + dh_strip -Xvmlinux --no-automatic-dbgsym > > diff --git a/meta/recipes-kernel/linux/linux-custom.inc > b/meta/recipes-kernel/linux/linux-custom.inc > > index ca91f64..36086be 100644 > > --- a/meta/recipes-kernel/linux/linux-custom.inc > > +++ b/meta/recipes-kernel/linux/linux-custom.inc > > @@ -2,22 +2,104 @@ > > # > > # This software is a part of ISAR. > > # Copyright (c) Siemens AG, 2018 > > +# Copyright (c) Mentor Graphics, a Siemens business, 2019 > > # > > # SPDX-License-Identifier: MIT > > > > -FILESEXTRAPATHS_prepend := "${FILE_DIRNAME}/files:" > > +# Settings that would typically be done from the custom kernel recipe > > +# ------------------------------------------------------------------- > > > > DESCRIPTION ?= "Custom kernel" > > +MAINTAINER ?= "isar-users >" > > + > > +KBUILD_DEPENDS ?= "build-essential:native, \ > > + libelf-dev:native, \ > > + libncurses-dev:native, \ > > + libssl-dev:native, \ > > + bc, \ > > + bison, \ > > + cpio, \ > > + flex, \ > > + git, \ > > + kmod, \ > > + libssl-dev," > > + > > +KERNEL_DEBIAN_DEPENDS ?= "initramfs-tools | linux-initramfs-tool, \ > > + kmod, \ > > + linux-base (>= 4.3~)," > > + > > +KERNEL_HEADERS_DEBIAN_DEPENDS ?= "libc6, \ > > + libssl1.1," > > + > > +KERNEL_LIBC_DEV_DEPLOY ?= "0" > > + > > +# Settings that may be changed on a per distro, machine or layer basis > > +# -------------------------------------------------------------------- > > + > > +KERNEL_DTB_DIR ?= "usr/lib/linux-${KERNEL_NAME_PROVIDED}-${PV}" > > +KERNEL_HEADERS_DIR ?= "usr/src/linux-headers-${PV}" > > > > -KERNEL_NAME_PROVIDED ?= "${@ d.getVar('PN', > True).partition('linux-')[2]}" > > KERNEL_DEFCONFIG ?= "" > > > > +# Add our template meta-data to the sources > > +FILESEXTRAPATHS_prepend := "${FILE_DIRNAME}/files:" > > +SRC_URI += "file://debian" > > + > > +# Variables and files that make our templates > > +# ------------------------------------------- > > + > > +TEMPLATE_FILES += " \ > > + debian/control.tmpl \ > > + debian/isar/build.tmpl \ > > + debian/isar/clean.tmpl \ > > + debian/isar/common.tmpl \ > > + debian/isar/install.tmpl \ > > + debian/linux-image.postinst.tmpl \ > > + debian/linux-image.postrm.tmpl \ > > + debian/rules.tmpl \ > > +" > > + > > +TEMPLATE_VARS += " \ > > + KBUILD_DEPENDS \ > > + KERNEL_ARCH \ > > + KERNEL_DEBIAN_DEPENDS \ > > + KERNEL_BUILD_DIR \ > > + KERNEL_DTB_DIR \ > > + KERNEL_FILE \ > > + KERNEL_HEADERS_DEBIAN_DEPENDS \ > > + KERNEL_HEADERS_DIR \ > > + KERNEL_NAME_PROVIDED \ > > +" > > + > > +inherit dpkg > > +inherit template > > + > > +# Derive name of the kernel packages from the name of this recipe > > +KERNEL_NAME_PROVIDED ?= "${@ d.getVar('PN', > True).partition('linux-')[2]}" > > + > > +# Make bitbake know we will be producing linux-image and linux-headers > packages > > python() { > > kernel_name = d.getVar("KERNEL_NAME_PROVIDED", True) > > d.setVar('PROVIDES', 'linux-image-' + kernel_name + ' ' + \ > > 'linux-headers-' + kernel_name) > > } > > > > +def get_kernel_arch(d): > > + distro_arch = d.getVar("DISTRO_ARCH") > > + if distro_arch == "amd64": > > + kernel_arch = "x86" > > + elif distro_arch == "arm64": > > + kernel_arch = "arm64" > > + elif distro_arch == "armhf": > > + kernel_arch = "arm" > > + elif distro_arch == "mipsel": > > + kernel_arch = "mips" > > + else: > > + kernel_arch = "" > > + return kernel_arch > > + > > +KERNEL_ARCH ??= "${@get_kernel_arch(d)}" > > + > > def config_fragments(d): > > fragments = [] > > sources = d.getVar("SRC_URI").split() > > @@ -31,57 +113,35 @@ def config_fragments(d): > > fragments.append(local) > > return fragments > > > > -inherit dpkg-base > > - > > -SRC_URI += "file://build-kernel.sh" > > +do_prepare_build_prepend() { > > + # copy meta-data over to source tree > > + rm -rf ${S}/debian > > + cp -r ${WORKDIR}/debian ${S}/ > > > > -KBUILD_DEPENDS ?= " \ > > - build-essential:native \ > > - libssl-dev \ > > - libelf-dev \ > > - bc \ > > - git \ > > - kmod \ > > - bison \ > > - flex \ > > - cpio \ > > - libncurses-dev" > > -KERNEL_DEBIAN_DEPENDS ?= "initramfs-tools | linux-initramfs-tool, kmod, > linux-base (>= 4.3~)" > > -KERNEL_HEADERS_DEBIAN_DEPENDS ?= "libc6, libssl1.1" > > + # remove templates from the source tree > > + find ${S}/debian -name *.tmpl | xargs rm -f > > > > -KERNEL_LIBC_DEV_DEPLOY ?= "0" > > + # rename install/remove hooks to match user-specified name for > our linux-image package > > + mv ${S}/debian/linux-image.postinst > ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.postinst > > + mv ${S}/debian/linux-image.postrm > ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.postrm > > > > -do_install_builddeps() { > > - dpkg_do_mounts > > - E="${@ bb.utils.export_proxies(d)}" > > - sudo -E chroot ${BUILDCHROOT_DIR} \ > > - apt-get update \ > > - -o > Dir::Etc::SourceList="sources.list.d/isar-apt.list" \ > > - -o Dir::Etc::SourceParts="-" \ > > - -o APT::Get::List-Cleanup="0" > > - sudo -E chroot ${BUILDCHROOT_DIR} \ > > - apt-get install \ > > - -y -o Debug::pkgProblemResolver=yes \ > > - --no-install-recommends ${KBUILD_DEPENDS} > > - dpkg_undo_mounts > > + # produce a changelog for our kernel build > > + deb_add_changelog > > } > > > > -addtask install_builddeps after do_prepare_build before do_dpkg_build > > -# apt and reprepro may not run in parallel, acquire the Isar lock > > -do_install_builddeps[lockfiles] += "${REPO_ISAR_DIR}/isar.lock" > > - > > -addtask devshell after do_install_builddeps > > +# build directory for our "full" kernel build > > +KERNEL_BUILD_DIR = "build-full" > > > > -dpkg_runbuild() { > > - chmod +x ${WORKDIR}/build-kernel.sh > > - KERNEL_CONFIG_TARGET="${KERNEL_DEFCONFIG}" > > +dpkg_configure_kernel() { > > + config_target="${KERNEL_DEFCONFIG}" > > + rm -rf ${S}/${KERNEL_BUILD_DIR} && mkdir -p > ${S}/${KERNEL_BUILD_DIR} > > if [ -n "${KERNEL_DEFCONFIG}" ]; then > > if [ -e "${WORKDIR}/${KERNEL_DEFCONFIG}" ]; then > > - cp ${WORKDIR}/${KERNEL_DEFCONFIG} ${S}/.config > > - KERNEL_CONFIG_TARGET="olddefconfig" > > + cp ${WORKDIR}/${KERNEL_DEFCONFIG} > ${S}/${KERNEL_BUILD_DIR}/.config > > + config_target="olddefconfig" > > fi > > else > > - KERNEL_CONFIG_TARGET="defconfig" > > + config_target="defconfig" > > fi > > > > # copy config fragments over to the kernel tree > > @@ -92,21 +152,23 @@ dpkg_runbuild() { > > mkdir -p ${S}/debian/fragments > > (cd ${WORKDIR} && cp ${src_frags} > ${S}/debian/fragments/) > > fi > > - export KERNEL_FRAGMENTS="${out_frags}" > > > > - E="${@ bb.utils.export_proxies(d)}" > > - > > - export PV=${PV} > > - export KERNEL_NAME=${KERNEL_NAME_PROVIDED} > > - > > - export KBUILD_DEPENDS="${KBUILD_DEPENDS}" > > - export KERNEL_CONFIG_TARGET > > - export KERNEL_DEBIAN_DEPENDS="${KERNEL_DEBIAN_DEPENDS}" > > - export KERNEL_FILE="${KERNEL_FILE}" > > - export > KERNEL_HEADERS_DEBIAN_DEPENDS="${KERNEL_HEADERS_DEBIAN_DEPENDS}" > > + sudo -E chroot --userspec=$(id -u):$(id -g) ${BUILDCHROOT_DIR} > sh -c " \ > > + export ARCH=${KERNEL_ARCH} && > \ > > + cd ${PP}/${PPS} && > \ > > + make O=${KERNEL_BUILD_DIR} ${config_target} && > \ > > + ./scripts/kconfig/merge_config.sh > \ > > + -O ${KERNEL_BUILD_DIR}/ > \ > > + ${KERNEL_BUILD_DIR}/.config > \ > > + ${out_frags} > \ > > + " > > +} > > > > - sudo -E chroot --userspec=$( id -u ):$( id -g ) > ${BUILDCHROOT_DIR} ${PP}/build-kernel.sh ${PP}/${PPS} ${DISTRO_ARCH} > > +dpkg_runbuild_prepend() { > > + dpkg_configure_kernel > > +} > > > > +dpkg_runbuild_append() { > > if [ "${KERNEL_LIBC_DEV_DEPLOY}" != "1" ]; then > > rm -f ${WORKDIR}/linux-libc-dev_${PV}*.deb > > fi > > > > Valuable step forward! I still need to check some more details against > my unfinished prototype, but it seems it generally a superset of it. > > Jan > > -- > Siemens AG, Corporate Technology, CT RDA IOT SES-DE > Corporate Competence Center Embedded Linux >