From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6755821036247187456 X-Received: by 2002:ad4:5441:: with SMTP id h1mr2051182qvt.120.1573307046819; Sat, 09 Nov 2019 05:44:06 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:ac8:2fc8:: with SMTP id m8ls3923722qta.9.gmail; Sat, 09 Nov 2019 05:44:05 -0800 (PST) X-Google-Smtp-Source: APXvYqzJtANLexG7xENzXRJk+NHKk/f1j0ki/ts2Ttv76PmgppMzbqe+8RDTcjDH8FqDWpMtVaf8 X-Received: by 2002:ac8:118d:: with SMTP id d13mr16844166qtj.249.1573307045830; Sat, 09 Nov 2019 05:44:05 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1573307045; cv=none; d=google.com; s=arc-20160816; b=vjLBH/vNZdoTk2pq0rjj3ZzfyE7VkjBGWdiG+0Dmej5/sQW+kQYIY9x+T41sMFwb66 n56UlbT0lpBIRnOZit8BPR7oHFpRH7ArgrzuNi0eicC77kWwIbBri9F2XmojdN7d5MFQ 9YzLU/2sZ5eYOivtcXARzU/itYp6jPWYbgIbTG2q0T5FevpVCAnZdg4TN1E/2WgNS+WS BRKHdALo/8Gtpvjid/beWjOZhDXiuymtaYVagJ6hBJVhGS/ivoCr6hE2SWfCwSnC4N9G IAGcbpkgX2FINVBgeo7VfVGBsrvbFZfU0QtUck9q9jm5evaKh7Fc9dx8RkJ6KEugw22h 2l6Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:dkim-signature; bh=iEz+drOaEvKqvA+G+UovSIxQXe+y7pr0Qtn8Ull94yE=; b=V5sKXjXqqcRqOJ6et9Rqx7xTq5kvocY8JPu1Bw/KjwuLehZQBNAps04zJzKOzN312v ktFDjNxoRFpIjMoythEagXHdb1/UtUg7AEOHn3a/2/AGyxkSnP3w6Ye61eyBRExp+qqC Q8GaoubvdBlNeKq1m6R9XPtcCTixTL4BXIDY/VESq36LN/tL0sL+/YiTE80LGI2VdRLw Wh+GLCBDGkvNdxPydwtZxttlsGBZHIRPUxij+kbQGGwQLi4NZVWjASK80GYK5QkbPH29 RZkWt76nVgt2/ndGU3yOpmLqEbBVK4roXBBKQ7YrjruxApRlUU3L+4tYuaRwJPXSaMjL 78sA== ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=RfkmIT8v; spf=pass (google.com: domain of chombourger@gmail.com designates 2607:f8b0:4864:20::842 as permitted sender) smtp.mailfrom=chombourger@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from mail-qt1-x842.google.com (mail-qt1-x842.google.com. [2607:f8b0:4864:20::842]) by gmr-mx.google.com with ESMTPS id z90si511423qtc.3.2019.11.09.05.44.05 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sat, 09 Nov 2019 05:44:05 -0800 (PST) Received-SPF: pass (google.com: domain of chombourger@gmail.com designates 2607:f8b0:4864:20::842 as permitted sender) client-ip=2607:f8b0:4864:20::842; Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=RfkmIT8v; spf=pass (google.com: domain of chombourger@gmail.com designates 2607:f8b0:4864:20::842 as permitted sender) smtp.mailfrom=chombourger@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: by mail-qt1-x842.google.com with SMTP id u22so10096528qtq.13 for ; Sat, 09 Nov 2019 05:44:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=iEz+drOaEvKqvA+G+UovSIxQXe+y7pr0Qtn8Ull94yE=; b=RfkmIT8v5zRZPczT4xG0gwU+yBWNIiY0j1yrvC6mhJg3OzS7qotb7izjcA3xgX42Ss TxXoTzzEZVcry4UXSL/4gxM6qPy6jlj1yH+ELYbOIhOGG7dBLv6eOCxFwu11fwCVfI9I +9cRcqb4Lbv8TbXNfBCsvvx91zBgi+iEmwLIzWHhwKG+GhzNOIpJZ4gUkhiRhKZ6jVDm gIR67acuwAvCX6PRAGinBc+LJ52WgE7rP0wBOQT7+IMAZfBCedljbm2ES3MQ74XWGrgS RsczifyuK/ekNsqFYE4o6UYlV7v5+ieqIqRi5BnuZoUj5XRXiunHblH2KKJdBNwLXdkC BpaQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=iEz+drOaEvKqvA+G+UovSIxQXe+y7pr0Qtn8Ull94yE=; b=E4Sv2yEUv/kfOfLG06PKpIBf2qPQIvzb9iISW6CY2pn0iypeDjzFuGmzwzqZWa9RhB umZ9G2HZgPsOcerbqLMzI2rcWXAJWafQZ/fHiIjlGIG+bLaMbEEvSPG3/Rwphr1R9bNg BwCSbMZx2cEQ8CvEHbgatayilT3HwbNIOq5TCn0pvp/4XzFDICCVjnrD5ajxbMf+sKOG x6QvJ+L5iaKJJBm92vC/aj4jDc0MH5ssZAJvoKOD1E8wWjr57OfoGUHMX2SMyF/OvPXO DSr2bszlzIL1eGms7xCR36VX5t9FfIQnAJMv5i+fKARog+sIeH5JNccN7i+CBH5H33GA nDEw== X-Gm-Message-State: APjAAAVMr9RJLpqoPLfFNBe2/2kZT3Nc+X2KXxwMoeaCEcCYscW6pLKR JYhDUlBp9uvmamD/2GhwVuoUrs9VQnGXNzk51qQ= X-Received: by 2002:aed:3ef2:: with SMTP id o47mr17346992qtf.107.1573307045380; Sat, 09 Nov 2019 05:44:05 -0800 (PST) MIME-Version: 1.0 References: <1572962145-318-1-git-send-email-Cedric_Hombourger@mentor.com> <1573118604-909-1-git-send-email-Cedric_Hombourger@mentor.com> <1573118604-909-8-git-send-email-Cedric_Hombourger@mentor.com> In-Reply-To: From: Cedric Hombourger Date: Sat, 9 Nov 2019 14:43:53 +0100 Message-ID: Subject: Re: [PATCH L-C v2 7/7] linux-custom: rewrite to no longer depend on the kernel's builddeb To: Jan Kiszka Cc: Cedric Hombourger , isar-users Content-Type: multipart/alternative; boundary="0000000000009009c50596ea173f" X-TUID: 3tdOQlf+GDjj --0000000000009009c50596ea173f Content-Type: text/plain; charset="UTF-8" On Sat, 9 Nov 2019 at 09:51, Jan Kiszka wrote: > On 07.11.19 10:23, Cedric Hombourger wrote: > > Regain control over the packaging of the Linux kernel by providing our > > own debian recipes (debian,{control,rules} files and friends) instead > > of using the kernel's builddeb script. This will allow generation of > > packages for "perf" (not included in this changeset) with per distro > > scheme (Debian and Ubuntu have differences) and maintain compatibility > > with upstream packages generated from different source packages but > > dependent on a certain deployment scheme (e.g. linux-base for Debian) > > > > Signed-off-by: Cedric Hombourger > > --- > > doc/custom_kernel.md | 181 ++++++++++++++++ > > .../linux/files/build-kernel.sh | 128 ----------- > > meta/recipes-kernel/linux/files/debian/compat | 1 + > > .../linux/files/debian/control.tmpl | 37 ++++ > > .../linux/files/debian/isar/build.tmpl | 36 ++++ > > .../linux/files/debian/isar/clean.tmpl | 20 ++ > > .../linux/files/debian/isar/common.tmpl | 48 +++++ > > .../linux/files/debian/isar/install.tmpl | 204 ++++++++++++++++++ > > .../files/debian/linux-image.postinst.tmpl | 29 +++ > > .../files/debian/linux-image.postrm.tmpl | 35 +++ > > .../files/debian/linux-image.preinst.tmpl | 25 +++ > > .../linux/files/debian/linux-image.prerm.tmpl | 21 ++ > > .../linux/files/debian/rules.tmpl | 41 ++++ > > meta/recipes-kernel/linux/linux-custom.inc | 179 ++++++++++----- > > 14 files changed, 800 insertions(+), 185 deletions(-) > > create mode 100644 doc/custom_kernel.md > > delete mode 100644 meta/recipes-kernel/linux/files/build-kernel.sh > > create mode 100644 meta/recipes-kernel/linux/files/debian/compat > > create mode 100644 meta/recipes-kernel/linux/files/debian/control.tmpl > > create mode 100644 > meta/recipes-kernel/linux/files/debian/isar/build.tmpl > > create mode 100644 > meta/recipes-kernel/linux/files/debian/isar/clean.tmpl > > create mode 100644 > meta/recipes-kernel/linux/files/debian/isar/common.tmpl > > create mode 100644 > meta/recipes-kernel/linux/files/debian/isar/install.tmpl > > create mode 100644 > meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl > > create mode 100644 > meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl > > create mode 100644 > meta/recipes-kernel/linux/files/debian/linux-image.preinst.tmpl > > create mode 100644 > meta/recipes-kernel/linux/files/debian/linux-image.prerm.tmpl > > create mode 100755 meta/recipes-kernel/linux/files/debian/rules.tmpl > > > > diff --git a/doc/custom_kernel.md b/doc/custom_kernel.md > > new file mode 100644 > > index 0000000..934325a > > --- /dev/null > > +++ b/doc/custom_kernel.md > > @@ -0,0 +1,181 @@ > > +# Custom kernel recipe for Isar > > + > > +## Contents > > + > > + - [Summary](#summary) > > + - [Proposal owners](#proposal-owners) > > + - [Current status](#current-status) > > + - [Detailed description](#detailed-description) > > + - [Benefit to Isar](#benefit-to-isar) > > + - [How can I help?](#how-can-i-help) > > + - [Scope](#scope) > > + - [How to test](#how-to-test) > > + - [Dependencies](#dependencies) > > + - [Documentation](#documentation) > > + - [Questions and answers](#questions-and-answers) > > + > > +## Summary > > + > > +Isar currently uses the Linux kernel's builddeb script to generate > linux-image, > > +linux-headers and linux-libc-dev packages for your custom kernel > sources. The > > +main benefit of this approach was that the low level details of how to > package > > +the Linux kernel image, modules and headers were left in the kernel. > There are > > +however some drawbacks: Isar had to repack generated packages to > introduce some > > +of the changes it needs such as suffixing the kernel packages with the > name of > > +the custom kernel (e.g. with "mainline" for the "linux-mainline" > recipe), lack > > +of automatic dependencies (the builddeb script from the kernel does not > use > > +debhelper), it was difficult to add packages especially distribution > specific > > +packages (maintain version specific patch in Isar vs being able to > upstream > > +distribution specific changes upstream), etc. This change is about > being less > > +dependent on the kernel packaging scripts (which aren't used by major > distros > > +anyway) and regain control on how we want or need the Linux kernel > packaged. > > + > > +## Proposal owners > > + > > + * name: [Cedric Hombourger](https://github.com/chombourger) > > + > > +## Current status > > + > > +### Tests > > + > > +The following were recently checked: > > + > > + * Custom kernels in meta-isar build without (known) failures > > + * initrd image gets generated when the kernel is added to the rootfs > > + * `ISAR_CROSS_COMPILE` is supported > > + * `KERNEL_FILE` is honored > > + * `KERNEL_DEFCONFIG` may either be a file (specified via `SRC_URI`) or > a > > + kernel-provided defconfig > > + * Config fragments get merged with the user-specified kernel > configuration > > + * Path to installed DTBs may be configured (new) > > + * Configuration and build from a `devshell` > > + * Support for the `KERNEL_LIBC_DEV_DEPLOY` directive > > + > > +while the following needs work: > > + > > + * Check if custom-kernel builds are reproducible > > + > > +### Remaining work items > > + > > + * Check if there is a better way to cross compile kernel "scripts" > such as > > + `fixdep` or `modpost` than manually compiling them (they are > compiled for > > + the build machine (amd64) and not for the target (e.g. armhf) even > though > > + we are cross-compiling - yet the linux-headers package should be > shipping > > + "scripts" for the target machine) > > + > > +## Detailed description > > + > > +### Requirements > > + > > +#### Current > > + > > +Isar has had a `linux-custom` recipe for a long time and counts many > users. It is therefore > > +paramount to maintain features that it has today with no or little > changes to the APIs. The > > +following requirements were identified and will be maintained: > > + > > + 1. Sources to the custom Linux kernel may be specified via `SRC_URI` > > + > > + 2. Configure the kernel via an in-tree or an external `defconfig` via > `KERNEL_DEFCONFIG` > > + > > + 3. Support kernel configuration tweaks via configuration fragments > (`.cfg` files) > > + > > + 4. Patches to the linux kernel may be specified via `SRC_URI` > > + > > + 5. Ensure that the Isar recipe `PV` matches the kernel release > identifier (`KERNEL_RELEASE`) > > + > > + 6. Produce a `linux-image` package that ships the kernel image and > modules > > + > > + 7. Allow the name of the kernel image to be changed via `KERNEL_FILE` > (defaults to `vmlinuz`) > > + > > + 8. Produce a `linux-headers` package which includes kernel headers and > kbuild scripts/tools > > + > > + 9. The `linux-headers` package shall support native and cross compiles > of out-of-tree kernel modules > > + > > + 10. Produce a `linux-libc-dev` package to support user-land builds > > + > > + 11. Only ship the `linux-libc-dev` package if instructed to > (`KERNEL_LIBC_DEV_DEPLOY` equals to `"1"`) > > + > > + 12. Support both native and cross compiles (`ISAR_CROSS_COMPILE`) > > + > > + 13. Support for the following kernel architectures: > > + > > + * arm > > + * arm64 > > + * mips > > + * x86 > > + > > + 14. Support `devshell` (kernel configuration shall be applied) > > + > > +#### New requirements (proposed) > > + > > +This proposal includes the following new requirements: > > + > > + 1. Support creation of additional packages > > + > > + 2. Allow per-distro packaging tweaks > > + > > +### Approach > > + > > +The new implementation will have the `builddeb` code ported from the > Linux kernel included to the > > +Isar environment. It will be refactor to support pre and post hooks. > Hooks may be used by external > > +layers to extend the kernel packaging instructions provided by "stock" > Isar or by Isar itself to > > +implement distro-specific packages or tweaks. > > + > > +Hooks need the ability to add `build` and `install` steps which can be > achieved by having the main > > +`build` and `install` steps call pre and post scripts. This will > however solve only one side of the > > +problem: new packages should be declared in the `debian/control` file. > The packaging process > > +(initiated by `debian/rules`) will use a baseline `control` file when > started and a mechanism will > > +be provided for hooks to append `control` blocks as they are processed. > > + > > +### Future work > > + > > +In the event where this proposal is accepted, here are a few future > projects that are being > > +considered: > > + > > + 1. Package `perf` as Debian does > > + > > + 2. Add `ubuntu` as an Isar supported target distro and validate > `linux-custom` recipe > > + > > +## Benefit to Isar > > + > > + * Control the kernel build and packaging process > > + > > + * No longer need to repack packages > > + > > + * Provide hooks for custom kernel recipes to add build/install > directives > > + > > +## How can I help? > > + > > + * Check if you are able to build your own linux-custom recipes with > this change > > + * Check if packages generated by this recipe are compatible with your > Debian-based distro > > + * Review and provide comments on this changeset > > + * Create templates for distro-specific packages (e.g. perf) > > + > > +## Scope > > + > > + * Proposal owners: proposal owners are to communicate the idea to > isar-users and make sure > > + that all of their requirements are met (to the extent they are > already supported with the > > + current solution) or may be met in the future. > > + > > + * Other developers: port your out-of-tree linux-custom recipes to this > new solution and > > + report issues. > > + > > +## How to test? > > + > > +The linux-mainline recipe may be used for some basic testing. This > recipe is being used by the > > +following machines: > > + > > + * de0-nano-soc > > + * qemumipsel > > + > > +## Dependencies > > + > > + * None > > + > > +## Documentation > > + > > + * Document user-visible variables added by this proposal > > + > > +## Questions and answers > > + > > + * None > > diff --git a/meta/recipes-kernel/linux/files/build-kernel.sh > b/meta/recipes-kernel/linux/files/build-kernel.sh > > deleted file mode 100644 > > index f56e96c..0000000 > > --- a/meta/recipes-kernel/linux/files/build-kernel.sh > > +++ /dev/null > > @@ -1,128 +0,0 @@ > > -#!/bin/bash > > -# > > -# Custom kernel build > > -# > > -# This software is a part of ISAR. > > -# Copyright (c) Siemens AG, 2018 > > -# > > -# SPDX-License-Identifier: MIT > > - > > -source /isar/common.sh > > - > > -host_arch=$(dpkg --print-architecture) > > - > > -if [ "$host_arch" != "$target_arch" ]; then > > - case $target_arch in > > - armhf) > > - export ARCH=arm > > - export CROSS_COMPILE="arm-linux-gnueabihf-" > > - ;; > > - arm64) > > - export ARCH=arm64 > > - export CROSS_COMPILE="aarch64-linux-gnu-" > > - ;; > > - mipsel) > > - export ARCH=mips > > - export CROSS_COMPILE="mipsel-linux-gnu-" > > - ;; > > - *) > > - echo "error: unsupported architecture ($target_arch)" > > - exit 1 > > - ;; > > - esac > > -fi > > - > > -REPACK_DIR="$1/../repack" > > -REPACK_LINUX_IMAGE_DIR="${REPACK_DIR}/linux-image" > > -REPACK_LINUX_HEADERS_DIR="${REPACK_DIR}/linux-headers" > > - > > -make ${KERNEL_CONFIG_TARGET} || exit ${?} > > -if [ -n "${KERNEL_FRAGMENTS}" ]; then > > - scripts/kconfig/merge_config.sh -m .config ${KERNEL_FRAGMENTS} > > -fi > > - > > -KV=$( make -s kernelrelease ) > > -if [ "${KV}" != "${PV}" ]; then > > - echo "ERROR: Recipe PV is \"${PV}\" but should be \"${KV}\"" 1>&2 > > - echo "ERROR: Probably due to CONFIG_LOCALVERSION" 1>&2 > > - exit 1 > > -fi > > - > > -rm -f .version > > -KBUILD_DEBARCH=$target_arch make -j $(($(nproc) * 2)) deb-pkg > > - > > -rm -rf "${REPACK_DIR}" > > -mkdir -p "${REPACK_DIR}" > > -mkdir -p "${REPACK_LINUX_IMAGE_DIR}" > > -mkdir -p "${REPACK_LINUX_HEADERS_DIR}" > > - > > -cp -a debian "${REPACK_DIR}" > > - > > -# dpkg-gencontrol performs cross-incompatible checks on the > > -# Architecture field; trick it to accept the control file > > -sed -i "s/Architecture: .*/Architecture: any/" > "${REPACK_DIR}/debian/control" > > - > > -cd .. > > - > > -dpkg-deb -R linux-image-${PV}_${PV}-1_*.deb "${REPACK_LINUX_IMAGE_DIR}" > > -dpkg-deb -R linux-headers-${PV}_${PV}-1_*.deb > "${REPACK_LINUX_HEADERS_DIR}" > > - > > -dpkg-gencontrol -crepack/debian/control \ > > - -lrepack/debian/changelog \ > > - -frepack/debian/files \ > > - -plinux-image-${PV} \ > > - -P"${REPACK_LINUX_IMAGE_DIR}" \ > > - -DPackage="linux-image-${KERNEL_NAME}" \ > > - -DSection=kernel \ > > - -DPriority=required \ > > - -DDepends="${KERNEL_DEBIAN_DEPENDS}" \ > > - -DArchitecture=$target_arch > > - > > -# Add Debian-like link installation to postinst > > -mkdir -p ${REPACK_LINUX_IMAGE_DIR}/lib/modules/${PV} > > -touch "${REPACK_LINUX_IMAGE_DIR}/lib/modules/${PV}/.fresh-install" > > -sed -i "${REPACK_LINUX_IMAGE_DIR}/DEBIAN/postinst" \ > > - -e "/^set -e$/a\\ > > -\\ > > -if [ -f /lib/modules/${PV}/.fresh-install ]; then\\ > > - change=install\\ > > -else\\ > > - change=upgrade\\ > > -fi\\ > > -linux-update-symlinks \$change ${PV} /boot/${KERNEL_FILE}-${PV}\\ > > -rm -f /lib/modules/${PV}/.fresh-install" > > - > > -# Add Debian-like link removal to postrm > > -sed -i "${REPACK_LINUX_IMAGE_DIR}/DEBIAN/postrm" \ > > - -e "/^set -e$/a\\ > > -\\ > > -rm -f /lib/modules/${PV}/.fresh-install\\ > > -\\ > > -if [ \"\$1\" != upgrade ] && command -v linux-update-symlinks > >/dev/null; then\\ > > - linux-update-symlinks remove ${PV} /boot/${KERNEL_FILE}-${PV}\\ > > -fi" > > - > > -# Make sure arm64 kernels are decompressed > > -if [ "$target_arch" = "arm64" ]; then > > - kernel_file="${REPACK_LINUX_IMAGE_DIR}/boot/${KERNEL_FILE}-${PV}" > > - mv "${kernel_file}" "${kernel_file}.gz" > > - gunzip "${kernel_file}.gz" > > -fi > > Next regression: This got lost. > Fixed locally ( https://github.com/chombourger/isar-fork/blob/cedric/next/meta/recipes-kernel/linux/files/debian/isar/install.tmpl#L62 ) Started a test build to check Thanks for catching that subtle one > > Jan > > -- > Siemens AG, Corporate Technology, CT RDA IOT SES-DE > Corporate Competence Center Embedded Linux > > -- > You received this message because you are subscribed to a topic in the > Google Groups "isar-users" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/isar-users/XDeqow7oo9I/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > isar-users+unsubscribe@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/isar-users/d177ccc9-c18c-15c7-a356-92dd2c9fd8e6%40siemens.com > . > --0000000000009009c50596ea173f Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Sat, 9 Nov 2019 at 09:51, Jan Kisz= ka <jan.kiszka@siemens.com= > wrote:
On 0= 7.11.19 10:23, Cedric Hombourger wrote:
> Regain control over the packaging of the Linux kernel by providing our=
> own debian recipes (debian,{control,rules} files and friends) instead<= br> > of using the kernel's builddeb script. This will allow generation = of
> packages for "perf" (not included in this changeset) with pe= r distro
> scheme (Debian and Ubuntu have differences) and maintain compatibility=
> with upstream packages generated from different source packages but > dependent on a certain deployment scheme (e.g. linux-base for Debian)<= br> >
> Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com>
> ---
>=C2=A0 =C2=A0doc/custom_kernel.md=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 | 181 ++++++++++++++++=
>=C2=A0 =C2=A0.../linux/files/build-kernel.sh=C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0| 128 -----------
>=C2=A0 =C2=A0meta/recipes-kernel/linux/files/debian/compat |=C2=A0 =C2= =A01 +
>=C2=A0 =C2=A0.../linux/files/debian/control.tmpl=C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0|=C2=A0 37 ++++
>=C2=A0 =C2=A0.../linux/files/debian/isar/build.tmpl=C2=A0 =C2=A0 =C2=A0= =C2=A0 |=C2=A0 36 ++++
>=C2=A0 =C2=A0.../linux/files/debian/isar/clean.tmpl=C2=A0 =C2=A0 =C2=A0= =C2=A0 |=C2=A0 20 ++
>=C2=A0 =C2=A0.../linux/files/debian/isar/common.tmpl=C2=A0 =C2=A0 =C2= =A0 =C2=A0|=C2=A0 48 +++++
>=C2=A0 =C2=A0.../linux/files/debian/isar/install.tmpl=C2=A0 =C2=A0 =C2= =A0 | 204 ++++++++++++++++++
>=C2=A0 =C2=A0.../files/debian/linux-image.postinst.tmpl=C2=A0 =C2=A0 |= =C2=A0 29 +++
>=C2=A0 =C2=A0.../files/debian/linux-image.postrm.tmpl=C2=A0 =C2=A0 =C2= =A0 |=C2=A0 35 +++
>=C2=A0 =C2=A0.../files/debian/linux-image.preinst.tmpl=C2=A0 =C2=A0 =C2= =A0|=C2=A0 25 +++
>=C2=A0 =C2=A0.../linux/files/debian/linux-image.prerm.tmpl |=C2=A0 21 += +
>=C2=A0 =C2=A0.../linux/files/debian/rules.tmpl=C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0|=C2=A0 41 ++++
>=C2=A0 =C2=A0meta/recipes-kernel/linux/linux-custom.inc=C2=A0 =C2=A0 | = 179 ++++++++++-----
>=C2=A0 =C2=A014 files changed, 800 insertions(+), 185 deletions(-)
>=C2=A0 =C2=A0create mode 100644 doc/custom_kernel.md
>=C2=A0 =C2=A0delete mode 100644 meta/recipes-kernel/linux/files/build-k= ernel.sh
>=C2=A0 =C2=A0create mode 100644 meta/recipes-kernel/linux/files/debian/= compat
>=C2=A0 =C2=A0create mode 100644 meta/recipes-kernel/linux/files/debian/= control.tmpl
>=C2=A0 =C2=A0create mode 100644 meta/recipes-kernel/linux/files/debian/= isar/build.tmpl
>=C2=A0 =C2=A0create mode 100644 meta/recipes-kernel/linux/files/debian/= isar/clean.tmpl
>=C2=A0 =C2=A0create mode 100644 meta/recipes-kernel/linux/files/debian/= isar/common.tmpl
>=C2=A0 =C2=A0create mode 100644 meta/recipes-kernel/linux/files/debian/= isar/install.tmpl
>=C2=A0 =C2=A0create mode 100644 meta/recipes-kernel/linux/files/debian/= linux-image.postinst.tmpl
>=C2=A0 =C2=A0create mode 100644 meta/recipes-kernel/linux/files/debian/= linux-image.postrm.tmpl
>=C2=A0 =C2=A0create mode 100644 meta/recipes-kernel/linux/files/debian/= linux-image.preinst.tmpl
>=C2=A0 =C2=A0create mode 100644 meta/recipes-kernel/linux/files/debian/= linux-image.prerm.tmpl
>=C2=A0 =C2=A0create mode 100755 meta/recipes-kernel/linux/files/debian/= rules.tmpl
>
> diff --git a/doc/custom_kernel.md b/doc/custom_kernel.md
> new file mode 100644
> index 0000000..934325a
> --- /dev/null
> +++ b/doc/custom_kernel.md
> @@ -0,0 +1,181 @@
> +# Custom kernel recipe for Isar
> +
> +## Contents
> +
> + - [Summary](#summary)
> + - [Proposal owners](#proposal-owners)
> + - [Current status](#current-status)
> + - [Detailed description](#detailed-description)
> + - [Benefit to Isar](#benefit-to-isar)
> + - [How can I help?](#how-can-i-help)
> + - [Scope](#scope)
> + - [How to test](#how-to-test)
> + - [Dependencies](#dependencies)
> + - [Documentation](#documentation)
> + - [Questions and answers](#questions-and-answers)
> +
> +## Summary
> +
> +Isar currently uses the Linux kernel's builddeb script to generat= e linux-image,
> +linux-headers and linux-libc-dev packages for your custom kernel sour= ces. The
> +main benefit of this approach was that the low level details of how t= o package
> +the Linux kernel image, modules and headers were left in the kernel. = There are
> +however some drawbacks: Isar had to repack generated packages to intr= oduce some
> +of the changes it needs such as suffixing the kernel packages with th= e name of
> +the custom kernel (e.g. with "mainline" for the "linux= -mainline" recipe), lack
> +of automatic dependencies (the builddeb script from the kernel does n= ot use
> +debhelper), it was difficult to add packages especially distribution = specific
> +packages (maintain version specific patch in Isar vs being able to up= stream
> +distribution specific changes upstream), etc. This change is about be= ing less
> +dependent on the kernel packaging scripts (which aren't used by m= ajor distros
> +anyway) and regain control on how we want or need the Linux kernel pa= ckaged.
> +
> +## Proposal owners
> +
> + * name: [Cedric Hombourger](https://github.com/chombourger)=
> +
> +## Current status
> +
> +### Tests
> +
> +The following were recently checked:
> +
> + * Custom kernels in meta-isar build without (known) failures
> + * initrd image gets generated when the kernel is added to the rootfs=
> + * `ISAR_CROSS_COMPILE` is supported
> + * `KERNEL_FILE` is honored
> + * `KERNEL_DEFCONFIG` may either be a file (specified via `SRC_URI`) = or a
> +=C2=A0 =C2=A0kernel-provided defconfig
> + * Config fragments get merged with the user-specified kernel configu= ration
> + * Path to installed DTBs may be configured (new)
> + * Configuration and build from a `devshell`
> + * Support for the `KERNEL_LIBC_DEV_DEPLOY` directive
> +
> +while the following needs work:
> +
> + * Check if custom-kernel builds are reproducible
> +
> +### Remaining work items
> +
> + * Check if there is a better way to cross compile kernel "scrip= ts" such as
> +=C2=A0 =C2=A0`fixdep` or `modpost` than manually compiling them (they= are compiled for
> +=C2=A0 =C2=A0the build machine (amd64) and not for the target (e.g. a= rmhf) even though
> +=C2=A0 =C2=A0we are cross-compiling - yet the linux-headers package s= hould be shipping
> +=C2=A0 =C2=A0"scripts" for the target machine)
> +
> +## Detailed description
> +
> +### Requirements
> +
> +#### Current
> +
> +Isar has had a `linux-custom` recipe for a long time and counts many = users. It is therefore
> +paramount to maintain features that it has today with no or little ch= anges to the APIs. The
> +following requirements were identified and will be maintained:
> +
> + 1. Sources to the custom Linux kernel may be specified via `SRC_URI`=
> +
> + 2. Configure the kernel via an in-tree or an external `defconfig` vi= a `KERNEL_DEFCONFIG`
> +
> + 3. Support kernel configuration tweaks via configuration fragments (= `.cfg` files)
> +
> + 4. Patches to the linux kernel may be specified via `SRC_URI`
> +
> + 5. Ensure that the Isar recipe `PV` matches the kernel release ident= ifier (`KERNEL_RELEASE`)
> +
> + 6. Produce a `linux-image` package that ships the kernel image and m= odules
> +
> + 7. Allow the name of the kernel image to be changed via `KERNEL_FILE= ` (defaults to `vmlinuz`)
> +
> + 8. Produce a `linux-headers` package which includes kernel headers a= nd kbuild scripts/tools
> +
> + 9. The `linux-headers` package shall support native and cross compil= es of out-of-tree kernel modules
> +
> + 10. Produce a `linux-libc-dev` package to support user-land builds > +
> + 11. Only ship the `linux-libc-dev` package if instructed to (`KERNEL= _LIBC_DEV_DEPLOY` equals to `"1"`)
> +
> + 12. Support both native and cross compiles (`ISAR_CROSS_COMPILE`) > +
> + 13. Support for the following kernel architectures:
> +
> +=C2=A0 =C2=A0* arm
> +=C2=A0 =C2=A0* arm64
> +=C2=A0 =C2=A0* mips
> +=C2=A0 =C2=A0* x86
> +
> + 14. Support `devshell` (kernel configuration shall be applied)
> +
> +#### New requirements (proposed)
> +
> +This proposal includes the following new requirements:
> +
> + 1. Support creation of additional packages
> +
> + 2. Allow per-distro packaging tweaks
> +
> +### Approach
> +
> +The new implementation will have the `builddeb` code ported from the = Linux kernel included to the
> +Isar environment. It will be refactor to support pre and post hooks. = Hooks may be used by external
> +layers to extend the kernel packaging instructions provided by "= stock" Isar or by Isar itself to
> +implement distro-specific packages or tweaks.
> +
> +Hooks need the ability to add `build` and `install` steps which can b= e achieved by having the main
> +`build` and `install` steps call pre and post scripts. This will howe= ver solve only one side of the
> +problem: new packages should be declared in the `debian/control` file= . The packaging process
> +(initiated by `debian/rules`) will use a baseline `control` file when= started and a mechanism will
> +be provided for hooks to append `control` blocks as they are processe= d.
> +
> +### Future work
> +
> +In the event where this proposal is accepted, here are a few future p= rojects that are being
> +considered:
> +
> + 1. Package `perf` as Debian does
> +
> + 2. Add `ubuntu` as an Isar supported target distro and validate `lin= ux-custom` recipe
> +
> +## Benefit to Isar
> +
> + * Control the kernel build and packaging process
> +
> + * No longer need to repack packages
> +
> + * Provide hooks for custom kernel recipes to add build/install direc= tives
> +
> +## How can I help?
> +
> + * Check if you are able to build your own linux-custom recipes with = this change
> + * Check if packages generated by this recipe are compatible with you= r Debian-based distro
> + * Review and provide comments on this changeset
> + * Create templates for distro-specific packages (e.g. perf)
> +
> +## Scope
> +
> + * Proposal owners: proposal owners are to communicate the idea to is= ar-users and make sure
> +=C2=A0 =C2=A0that all of their requirements are met (to the extent th= ey are already supported with the
> +=C2=A0 =C2=A0current solution) or may be met in the future.
> +
> + * Other developers: port your out-of-tree linux-custom recipes to th= is new solution and
> +=C2=A0 =C2=A0report issues.
> +
> +## How to test?
> +
> +The linux-mainline recipe may be used for some basic testing. This re= cipe is being used by the
> +following machines:
> +
> + * de0-nano-soc
> + * qemumipsel
> +
> +## Dependencies
> +
> + * None
> +
> +## Documentation
> +
> + * Document user-visible variables added by this proposal
> +
> +## Questions and answers
> +
> + * None
> diff --git a/meta/recipes-kernel/linux/files/build-kernel.sh b/meta/re= cipes-kernel/linux/files/build-kernel.sh
> deleted file mode 100644
> index f56e96c..0000000
> --- a/meta/recipes-kernel/linux/files/build-kernel.sh
> +++ /dev/null
> @@ -1,128 +0,0 @@
> -#!/bin/bash
> -#
> -# Custom kernel build
> -#
> -# This software is a part of ISAR.
> -# Copyright (c) Siemens AG, 2018
> -#
> -# SPDX-License-Identifier: MIT
> -
> -source /isar/common.sh
> -
> -host_arch=3D$(dpkg --print-architecture)
> -
> -if [ "$host_arch" !=3D "$target_arch" ]; then
> -=C2=A0 =C2=A0 case $target_arch in
> -=C2=A0 =C2=A0 armhf)
> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 export ARCH=3Darm
> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 export CROSS_COMPILE=3D"arm-linux-gn= ueabihf-"
> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 ;;
> -=C2=A0 =C2=A0 arm64)
> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 export ARCH=3Darm64
> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 export CROSS_COMPILE=3D"aarch64-linu= x-gnu-"
> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 ;;
> -=C2=A0 =C2=A0 mipsel)
> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 export ARCH=3Dmips
> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 export CROSS_COMPILE=3D"mipsel-linux= -gnu-"
> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 ;;
> -=C2=A0 =C2=A0 *)
> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 echo "error: unsupported architectur= e ($target_arch)"
> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 exit 1
> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 ;;
> -=C2=A0 =C2=A0 esac
> -fi
> -
> -REPACK_DIR=3D"$1/../repack"
> -REPACK_LINUX_IMAGE_DIR=3D"${REPACK_DIR}/linux-image"
> -REPACK_LINUX_HEADERS_DIR=3D"${REPACK_DIR}/linux-headers" > -
> -make ${KERNEL_CONFIG_TARGET} || exit ${?}
> -if [ -n "${KERNEL_FRAGMENTS}" ]; then
> -=C2=A0 =C2=A0 scripts/kconfig/merge_config.sh -m .config ${KERNEL_FRA= GMENTS}
> -fi
> -
> -KV=3D$( make -s kernelrelease )
> -if [ "${KV}" !=3D "${PV}" ]; then
> -=C2=A0 =C2=A0 =C2=A0echo "ERROR: Recipe PV is \"${PV}\"= ; but should be \"${KV}\"" 1>&2
> -=C2=A0 =C2=A0 =C2=A0echo "ERROR: Probably due to CONFIG_LOCALVER= SION" 1>&2
> -=C2=A0 =C2=A0 =C2=A0exit 1
> -fi
> -
> -rm -f .version
> -KBUILD_DEBARCH=3D$target_arch make -j $(($(nproc) * 2)) deb-pkg
> -
> -rm -rf "${REPACK_DIR}"
> -mkdir -p "${REPACK_DIR}"
> -mkdir -p "${REPACK_LINUX_IMAGE_DIR}"
> -mkdir -p "${REPACK_LINUX_HEADERS_DIR}"
> -
> -cp -a debian "${REPACK_DIR}"
> -
> -# dpkg-gencontrol performs cross-incompatible checks on the
> -# Architecture field; trick it to accept the control file
> -sed -i "s/Architecture: .*/Architecture: any/" "${REPA= CK_DIR}/debian/control"
> -
> -cd ..
> -
> -dpkg-deb -R linux-image-${PV}_${PV}-1_*.deb "${REPACK_LINUX_IMAG= E_DIR}"
> -dpkg-deb -R linux-headers-${PV}_${PV}-1_*.deb "${REPACK_LINUX_HE= ADERS_DIR}"
> -
> -dpkg-gencontrol -crepack/debian/control \
> -=C2=A0 =C2=A0 =C2=A0-lrepack/debian/changelog \
> -=C2=A0 =C2=A0 =C2=A0-frepack/debian/files \
> -=C2=A0 =C2=A0 =C2=A0-plinux-image-${PV} \
> -=C2=A0 =C2=A0 =C2=A0-P"${REPACK_LINUX_IMAGE_DIR}" \
> -=C2=A0 =C2=A0 =C2=A0-DPackage=3D"linux-image-${KERNEL_NAME}"= ; \
> -=C2=A0 =C2=A0 =C2=A0-DSection=3Dkernel \
> -=C2=A0 =C2=A0 =C2=A0-DPriority=3Drequired \
> -=C2=A0 =C2=A0 =C2=A0-DDepends=3D"${KERNEL_DEBIAN_DEPENDS}" = \
> -=C2=A0 =C2=A0 =C2=A0-DArchitecture=3D$target_arch
> -
> -# Add Debian-like link installation to postinst
> -mkdir -p ${REPACK_LINUX_IMAGE_DIR}/lib/modules/${PV}
> -touch "${REPACK_LINUX_IMAGE_DIR}/lib/modules/${PV}/.fresh-instal= l"
> -sed -i "${REPACK_LINUX_IMAGE_DIR}/DEBIAN/postinst" \
> -=C2=A0 =C2=A0 -e "/^set -e$/a\\
> -\\
> -if [ -f /lib/modules/${PV}/.fresh-install ]; then\\
> -=C2=A0 =C2=A0 =C2=A0change=3Dinstall\\
> -else\\
> -=C2=A0 =C2=A0 =C2=A0change=3Dupgrade\\
> -fi\\
> -linux-update-symlinks \$change ${PV} /boot/${KERNEL_FILE}-${PV}\\
> -rm -f /lib/modules/${PV}/.fresh-install"
> -
> -# Add Debian-like link removal to postrm
> -sed -i "${REPACK_LINUX_IMAGE_DIR}/DEBIAN/postrm" \
> -=C2=A0 =C2=A0 -e "/^set -e$/a\\
> -\\
> -rm -f /lib/modules/${PV}/.fresh-install\\
> -\\
> -if [ \"\$1\" !=3D upgrade ] && command -v linux-upd= ate-symlinks >/dev/null; then\\
> -=C2=A0 =C2=A0 =C2=A0linux-update-symlinks remove ${PV}=C2=A0 /boot/${= KERNEL_FILE}-${PV}\\
> -fi"
> -
> -# Make sure arm64 kernels are decompressed
> -if [ "$target_arch" =3D "arm64" ]; then
> -=C2=A0 =C2=A0 =C2=A0kernel_file=3D"${REPACK_LINUX_IMAGE_DIR}/boo= t/${KERNEL_FILE}-${PV}"
> -=C2=A0 =C2=A0 =C2=A0mv "${kernel_file}" "${kernel_file= }.gz"
> -=C2=A0 =C2=A0 =C2=A0gunzip "${kernel_file}.gz"
> -fi

Next regression: This got lost.

Started a test build to che= ck

Thanks for catching that subtle one
= =C2=A0

Jan

--
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

--
You received this message because you are subscribed to a topic in the Goog= le Groups "isar-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/isar-users/XDeqow7oo9I/unsubscribe<= /a>.
To unsubscribe from this group and all its topics, send an email to isa= r-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/isar-= users/d177ccc9-c18c-15c7-a356-92dd2c9fd8e6%40siemens.com.
--0000000000009009c50596ea173f--