From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6755821036247187456 X-Received: by 2002:a63:d10c:: with SMTP id k12mr3348759pgg.344.1573118733620; Thu, 07 Nov 2019 01:25:33 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a62:f201:: with SMTP id m1ls1360731pfh.1.gmail; Thu, 07 Nov 2019 01:25:33 -0800 (PST) X-Google-Smtp-Source: APXvYqzoMMdiIaJ+d/JLfaYpx2XaOHUmLNUYL1kjYffyVT8dCONX4I2rcis8UIT4iCgODBMjQvxw X-Received: by 2002:a63:ec50:: with SMTP id r16mr793991pgj.284.1573118732940; Thu, 07 Nov 2019 01:25:32 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1573118732; cv=none; d=google.com; s=arc-20160816; b=odYi2+aS1ahPr42gsoOUb/AWhVYG6f0UFmUIi1Mrxl4bT1WTsKh/+niPtvXA5d/cks GcpbbJYTpArwdzT6Qi7RHXP2a95NPB5sfkmCkoSMmVdCduzWtbg1yt+JCg89IFC3yQED lrwStObcR/8LpE6+++2yc2O1Bx4adG8xKHEYtSlOG1vA4HdSiigd7kbMJfvoI1SGBr/x hGoMogFPQ2oa5MeSqODYiEAVPAis1u0T+3jlJ+hq1RkS76OujW7sNel1IJc2WstocSX5 aIL5XeAc5Nghe4vB/5athZV8neorGDoXLo0S/FuWxyjTv4dQw3ipNpUVuKbby0BoWD/P icsQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:ironport-sdr:ironport-sdr; bh=AphCBXjaX83SQKPruth+5ASgxoWX/7B/xn4sEWKdBDE=; b=UbdEGXnWTZaLC9SipXnX2sw6wsl20KQfZ7DLBgPr1ZDoHlqDQ6wgNdc8rbU6AprqBI ZcT03L8BM2Z64n5q8GFYRM7/HN2ZNGvrlBskIRaVChaevZmk4zVb6t1752b29rlAaTx0 17jA90R++bAhIf+xFO4kuzIxXfEmzllc4JwTyzZiQiDmPHlMS6M75iTXxhV7Oyaw/DUe HslWFdhHW4uDZtRW61M0BcgsLu/s/hWrV2DFo/3EcOnT0ayI9rIj6TunJra3Z0Hyq3hA pFaK81C/F/Ow1hYm7XcHKwrbELVyS9Muse5paa0dmFgRo5Cx0jP7ktV2p1tX/ny44tz1 9TXw== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of cedric_hombourger@mentor.com designates 68.232.129.153 as permitted sender) smtp.mailfrom=Cedric_Hombourger@mentor.com Return-Path: Received: from esa1.mentor.iphmx.com (esa1.mentor.iphmx.com. [68.232.129.153]) by gmr-mx.google.com with ESMTPS id t12si47795pjv.3.2019.11.07.01.25.32 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 07 Nov 2019 01:25:32 -0800 (PST) Received-SPF: pass (google.com: domain of cedric_hombourger@mentor.com designates 68.232.129.153 as permitted sender) client-ip=68.232.129.153; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of cedric_hombourger@mentor.com designates 68.232.129.153 as permitted sender) smtp.mailfrom=Cedric_Hombourger@mentor.com IronPort-SDR: JANhu42FFAED76D8JMIW3u/k/A4GH2dyzUXJXeZzikPEwBQtR9utqTzo4rz13GSjYbfuzVoSV8 u6qbgWq9FalBwQe7JkJVOocqf2ulWpNjIDUt2Plo7uv9nGr9lMb9zkbsvQpHMC8bvlVaCFL7Dl Yr9UUoPI75BnCpjTCfFMNu7/Y+60RFp/tT70Jzo7W5GJ4mgI77cydh67iTAbliIPrLFwk6QUZj 8yELxZRaDBxe+FKaCN6ksTAFEUeZf1w3R5hYwlpuhO2hhB/9YjlqWLAylMAzq2E5kX/VZh/Rzi kF0= X-IronPort-AV: E=Sophos;i="5.68,277,1569312000"; d="scan'208";a="44801822" Received: from orw-gwy-01-in.mentorg.com ([192.94.38.165]) by esa1.mentor.iphmx.com with ESMTP; 07 Nov 2019 01:25:30 -0800 IronPort-SDR: dPekTRVEgMkdYi/+TbWIoPHk6cA3iO7MxoKMz8spb5XPNlKjighZu7u61lH/h4juWKCyHb1kYj RvtAJXnw/3SntDBf5iERg0HPU98MPWSF48gg2Ax2ogXdBw8bhWcY8VQtbQmnnRdca/Gh2fdIt6 fkn7ebHaEGiEVeT2vKfMLjkif+RSDC5gXJUnso7Ltaqx+NrHUqEdOC81cruJQmkj40dAg94/y4 5m3JFgsje9vSHVTHiu+5iOdnAISoN5/+qhxyrUqBAxZ2OIzXqMAVxhTsOt//kqWOVrTBkxAewS MZ4= From: Cedric Hombourger To: CC: Cedric Hombourger Subject: [PATCH L-C v2 7/7] linux-custom: rewrite to no longer depend on the kernel's builddeb Date: Thu, 7 Nov 2019 10:23:24 +0100 Message-ID: <1573118604-909-8-git-send-email-Cedric_Hombourger@mentor.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1573118604-909-1-git-send-email-Cedric_Hombourger@mentor.com> References: <1572962145-318-1-git-send-email-Cedric_Hombourger@mentor.com> <1573118604-909-1-git-send-email-Cedric_Hombourger@mentor.com> MIME-Version: 1.0 Content-Type: text/plain Return-Path: Cedric_Hombourger@mentor.com X-Originating-IP: [137.202.0.90] X-ClientProxiedBy: svr-ies-mbx-02.mgc.mentorg.com (139.181.222.2) To svr-ies-mbx-02.mgc.mentorg.com (139.181.222.2) X-TUID: wj5kCJagXi9n Regain control over the packaging of the Linux kernel by providing our own debian recipes (debian,{control,rules} files and friends) instead of using the kernel's builddeb script. This will allow generation of packages for "perf" (not included in this changeset) with per distro scheme (Debian and Ubuntu have differences) and maintain compatibility with upstream packages generated from different source packages but dependent on a certain deployment scheme (e.g. linux-base for Debian) Signed-off-by: Cedric Hombourger --- doc/custom_kernel.md | 181 ++++++++++++++++ .../linux/files/build-kernel.sh | 128 ----------- meta/recipes-kernel/linux/files/debian/compat | 1 + .../linux/files/debian/control.tmpl | 37 ++++ .../linux/files/debian/isar/build.tmpl | 36 ++++ .../linux/files/debian/isar/clean.tmpl | 20 ++ .../linux/files/debian/isar/common.tmpl | 48 +++++ .../linux/files/debian/isar/install.tmpl | 204 ++++++++++++++++++ .../files/debian/linux-image.postinst.tmpl | 29 +++ .../files/debian/linux-image.postrm.tmpl | 35 +++ .../files/debian/linux-image.preinst.tmpl | 25 +++ .../linux/files/debian/linux-image.prerm.tmpl | 21 ++ .../linux/files/debian/rules.tmpl | 41 ++++ meta/recipes-kernel/linux/linux-custom.inc | 179 ++++++++++----- 14 files changed, 800 insertions(+), 185 deletions(-) create mode 100644 doc/custom_kernel.md delete mode 100644 meta/recipes-kernel/linux/files/build-kernel.sh create mode 100644 meta/recipes-kernel/linux/files/debian/compat create mode 100644 meta/recipes-kernel/linux/files/debian/control.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/isar/build.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/isar/clean.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/isar/common.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/isar/install.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.preinst.tmpl create mode 100644 meta/recipes-kernel/linux/files/debian/linux-image.prerm.tmpl create mode 100755 meta/recipes-kernel/linux/files/debian/rules.tmpl diff --git a/doc/custom_kernel.md b/doc/custom_kernel.md new file mode 100644 index 0000000..934325a --- /dev/null +++ b/doc/custom_kernel.md @@ -0,0 +1,181 @@ +# Custom kernel recipe for Isar + +## Contents + + - [Summary](#summary) + - [Proposal owners](#proposal-owners) + - [Current status](#current-status) + - [Detailed description](#detailed-description) + - [Benefit to Isar](#benefit-to-isar) + - [How can I help?](#how-can-i-help) + - [Scope](#scope) + - [How to test](#how-to-test) + - [Dependencies](#dependencies) + - [Documentation](#documentation) + - [Questions and answers](#questions-and-answers) + +## Summary + +Isar currently uses the Linux kernel's builddeb script to generate linux-image, +linux-headers and linux-libc-dev packages for your custom kernel sources. The +main benefit of this approach was that the low level details of how to package +the Linux kernel image, modules and headers were left in the kernel. There are +however some drawbacks: Isar had to repack generated packages to introduce some +of the changes it needs such as suffixing the kernel packages with the name of +the custom kernel (e.g. with "mainline" for the "linux-mainline" recipe), lack +of automatic dependencies (the builddeb script from the kernel does not use +debhelper), it was difficult to add packages especially distribution specific +packages (maintain version specific patch in Isar vs being able to upstream +distribution specific changes upstream), etc. This change is about being less +dependent on the kernel packaging scripts (which aren't used by major distros +anyway) and regain control on how we want or need the Linux kernel packaged. + +## Proposal owners + + * name: [Cedric Hombourger](https://github.com/chombourger) + +## Current status + +### Tests + +The following were recently checked: + + * Custom kernels in meta-isar build without (known) failures + * initrd image gets generated when the kernel is added to the rootfs + * `ISAR_CROSS_COMPILE` is supported + * `KERNEL_FILE` is honored + * `KERNEL_DEFCONFIG` may either be a file (specified via `SRC_URI`) or a + kernel-provided defconfig + * Config fragments get merged with the user-specified kernel configuration + * Path to installed DTBs may be configured (new) + * Configuration and build from a `devshell` + * Support for the `KERNEL_LIBC_DEV_DEPLOY` directive + +while the following needs work: + + * Check if custom-kernel builds are reproducible + +### Remaining work items + + * Check if there is a better way to cross compile kernel "scripts" such as + `fixdep` or `modpost` than manually compiling them (they are compiled for + the build machine (amd64) and not for the target (e.g. armhf) even though + we are cross-compiling - yet the linux-headers package should be shipping + "scripts" for the target machine) + +## Detailed description + +### Requirements + +#### Current + +Isar has had a `linux-custom` recipe for a long time and counts many users. It is therefore +paramount to maintain features that it has today with no or little changes to the APIs. The +following requirements were identified and will be maintained: + + 1. Sources to the custom Linux kernel may be specified via `SRC_URI` + + 2. Configure the kernel via an in-tree or an external `defconfig` via `KERNEL_DEFCONFIG` + + 3. Support kernel configuration tweaks via configuration fragments (`.cfg` files) + + 4. Patches to the linux kernel may be specified via `SRC_URI` + + 5. Ensure that the Isar recipe `PV` matches the kernel release identifier (`KERNEL_RELEASE`) + + 6. Produce a `linux-image` package that ships the kernel image and modules + + 7. Allow the name of the kernel image to be changed via `KERNEL_FILE` (defaults to `vmlinuz`) + + 8. Produce a `linux-headers` package which includes kernel headers and kbuild scripts/tools + + 9. The `linux-headers` package shall support native and cross compiles of out-of-tree kernel modules + + 10. Produce a `linux-libc-dev` package to support user-land builds + + 11. Only ship the `linux-libc-dev` package if instructed to (`KERNEL_LIBC_DEV_DEPLOY` equals to `"1"`) + + 12. Support both native and cross compiles (`ISAR_CROSS_COMPILE`) + + 13. Support for the following kernel architectures: + + * arm + * arm64 + * mips + * x86 + + 14. Support `devshell` (kernel configuration shall be applied) + +#### New requirements (proposed) + +This proposal includes the following new requirements: + + 1. Support creation of additional packages + + 2. Allow per-distro packaging tweaks + +### Approach + +The new implementation will have the `builddeb` code ported from the Linux kernel included to the +Isar environment. It will be refactor to support pre and post hooks. Hooks may be used by external +layers to extend the kernel packaging instructions provided by "stock" Isar or by Isar itself to +implement distro-specific packages or tweaks. + +Hooks need the ability to add `build` and `install` steps which can be achieved by having the main +`build` and `install` steps call pre and post scripts. This will however solve only one side of the +problem: new packages should be declared in the `debian/control` file. The packaging process +(initiated by `debian/rules`) will use a baseline `control` file when started and a mechanism will +be provided for hooks to append `control` blocks as they are processed. + +### Future work + +In the event where this proposal is accepted, here are a few future projects that are being +considered: + + 1. Package `perf` as Debian does + + 2. Add `ubuntu` as an Isar supported target distro and validate `linux-custom` recipe + +## Benefit to Isar + + * Control the kernel build and packaging process + + * No longer need to repack packages + + * Provide hooks for custom kernel recipes to add build/install directives + +## How can I help? + + * Check if you are able to build your own linux-custom recipes with this change + * Check if packages generated by this recipe are compatible with your Debian-based distro + * Review and provide comments on this changeset + * Create templates for distro-specific packages (e.g. perf) + +## Scope + + * Proposal owners: proposal owners are to communicate the idea to isar-users and make sure + that all of their requirements are met (to the extent they are already supported with the + current solution) or may be met in the future. + + * Other developers: port your out-of-tree linux-custom recipes to this new solution and + report issues. + +## How to test? + +The linux-mainline recipe may be used for some basic testing. This recipe is being used by the +following machines: + + * de0-nano-soc + * qemumipsel + +## Dependencies + + * None + +## Documentation + + * Document user-visible variables added by this proposal + +## Questions and answers + + * None diff --git a/meta/recipes-kernel/linux/files/build-kernel.sh b/meta/recipes-kernel/linux/files/build-kernel.sh deleted file mode 100644 index f56e96c..0000000 --- a/meta/recipes-kernel/linux/files/build-kernel.sh +++ /dev/null @@ -1,128 +0,0 @@ -#!/bin/bash -# -# Custom kernel build -# -# This software is a part of ISAR. -# Copyright (c) Siemens AG, 2018 -# -# SPDX-License-Identifier: MIT - -source /isar/common.sh - -host_arch=$(dpkg --print-architecture) - -if [ "$host_arch" != "$target_arch" ]; then - case $target_arch in - armhf) - export ARCH=arm - export CROSS_COMPILE="arm-linux-gnueabihf-" - ;; - arm64) - export ARCH=arm64 - export CROSS_COMPILE="aarch64-linux-gnu-" - ;; - mipsel) - export ARCH=mips - export CROSS_COMPILE="mipsel-linux-gnu-" - ;; - *) - echo "error: unsupported architecture ($target_arch)" - exit 1 - ;; - esac -fi - -REPACK_DIR="$1/../repack" -REPACK_LINUX_IMAGE_DIR="${REPACK_DIR}/linux-image" -REPACK_LINUX_HEADERS_DIR="${REPACK_DIR}/linux-headers" - -make ${KERNEL_CONFIG_TARGET} || exit ${?} -if [ -n "${KERNEL_FRAGMENTS}" ]; then - scripts/kconfig/merge_config.sh -m .config ${KERNEL_FRAGMENTS} -fi - -KV=$( make -s kernelrelease ) -if [ "${KV}" != "${PV}" ]; then - echo "ERROR: Recipe PV is \"${PV}\" but should be \"${KV}\"" 1>&2 - echo "ERROR: Probably due to CONFIG_LOCALVERSION" 1>&2 - exit 1 -fi - -rm -f .version -KBUILD_DEBARCH=$target_arch make -j $(($(nproc) * 2)) deb-pkg - -rm -rf "${REPACK_DIR}" -mkdir -p "${REPACK_DIR}" -mkdir -p "${REPACK_LINUX_IMAGE_DIR}" -mkdir -p "${REPACK_LINUX_HEADERS_DIR}" - -cp -a debian "${REPACK_DIR}" - -# dpkg-gencontrol performs cross-incompatible checks on the -# Architecture field; trick it to accept the control file -sed -i "s/Architecture: .*/Architecture: any/" "${REPACK_DIR}/debian/control" - -cd .. - -dpkg-deb -R linux-image-${PV}_${PV}-1_*.deb "${REPACK_LINUX_IMAGE_DIR}" -dpkg-deb -R linux-headers-${PV}_${PV}-1_*.deb "${REPACK_LINUX_HEADERS_DIR}" - -dpkg-gencontrol -crepack/debian/control \ - -lrepack/debian/changelog \ - -frepack/debian/files \ - -plinux-image-${PV} \ - -P"${REPACK_LINUX_IMAGE_DIR}" \ - -DPackage="linux-image-${KERNEL_NAME}" \ - -DSection=kernel \ - -DPriority=required \ - -DDepends="${KERNEL_DEBIAN_DEPENDS}" \ - -DArchitecture=$target_arch - -# Add Debian-like link installation to postinst -mkdir -p ${REPACK_LINUX_IMAGE_DIR}/lib/modules/${PV} -touch "${REPACK_LINUX_IMAGE_DIR}/lib/modules/${PV}/.fresh-install" -sed -i "${REPACK_LINUX_IMAGE_DIR}/DEBIAN/postinst" \ - -e "/^set -e$/a\\ -\\ -if [ -f /lib/modules/${PV}/.fresh-install ]; then\\ - change=install\\ -else\\ - change=upgrade\\ -fi\\ -linux-update-symlinks \$change ${PV} /boot/${KERNEL_FILE}-${PV}\\ -rm -f /lib/modules/${PV}/.fresh-install" - -# Add Debian-like link removal to postrm -sed -i "${REPACK_LINUX_IMAGE_DIR}/DEBIAN/postrm" \ - -e "/^set -e$/a\\ -\\ -rm -f /lib/modules/${PV}/.fresh-install\\ -\\ -if [ \"\$1\" != upgrade ] && command -v linux-update-symlinks >/dev/null; then\\ - linux-update-symlinks remove ${PV} /boot/${KERNEL_FILE}-${PV}\\ -fi" - -# Make sure arm64 kernels are decompressed -if [ "$target_arch" = "arm64" ]; then - kernel_file="${REPACK_LINUX_IMAGE_DIR}/boot/${KERNEL_FILE}-${PV}" - mv "${kernel_file}" "${kernel_file}.gz" - gunzip "${kernel_file}.gz" -fi - -dpkg-gencontrol -crepack/debian/control \ - -lrepack/debian/changelog \ - -frepack/debian/files \ - -plinux-headers-${PV} \ - -P"${REPACK_LINUX_HEADERS_DIR}" \ - -Vkernel:debarch="${KERNEL_NAME}" \ - -DPackage="linux-headers-${KERNEL_NAME}" \ - -DSection=kernel \ - -DDepends="${KERNEL_HEADERS_DEBIAN_DEPENDS}" \ - -DArchitecture=$target_arch - -fakeroot dpkg-deb -b "${REPACK_LINUX_IMAGE_DIR}" \ - linux-image-${KERNEL_NAME}_${PV}-1_${KERNEL_NAME}.deb -rm -f linux-image-${PV}_${PV}-1_*.deb -fakeroot dpkg-deb -b "${REPACK_LINUX_HEADERS_DIR}" \ - linux-headers-${KERNEL_NAME}_${PV}-1_${KERNEL_NAME}.deb -rm -f linux-headers-${PV}_${PV}-1_*.deb diff --git a/meta/recipes-kernel/linux/files/debian/compat b/meta/recipes-kernel/linux/files/debian/compat new file mode 100644 index 0000000..ec63514 --- /dev/null +++ b/meta/recipes-kernel/linux/files/debian/compat @@ -0,0 +1 @@ +9 diff --git a/meta/recipes-kernel/linux/files/debian/control.tmpl b/meta/recipes-kernel/linux/files/debian/control.tmpl new file mode 100644 index 0000000..d55096b --- /dev/null +++ b/meta/recipes-kernel/linux/files/debian/control.tmpl @@ -0,0 +1,37 @@ +Source: linux-${KERNEL_NAME_PROVIDED} +Section: kernel +Priority: optional +Maintainer: ${MAINTAINER} +Build-Depends: bc, kmod, cpio, ${KBUILD_DEPENDS} +Homepage: http://www.kernel.org/ + +Package: linux-image-${KERNEL_NAME_PROVIDED} +Architecture: any +Depends: ${KERNEL_DEBIAN_DEPENDS} +Description: ${KERNEL_NAME_PROVIDED} Linux kernel, version ${PV} + This package contains the Linux kernel, modules and corresponding other + files, version: ${PV}. + +Package: linux-headers-${KERNEL_NAME_PROVIDED} +Architecture: any +Depends: ${KERNEL_HEADERS_DEBIAN_DEPENDS}, ${perl:Depends}, ${shlib:Depends} +Description: ${KERNEL_NAME_PROVIDED} Linux kernel headers for ${PV} + This package provides kernel header files for ${PV} on ${DISTRO_ARCH} + . + This is useful for people who need to build external modules + +Package: linux-libc-dev +Build-Profiles: +Section: devel +Provides: linux-kernel-headers +Architecture: any +Description: Linux support headers for userspace development + This package provides userspaces headers from the Linux kernel. These headers + are used by the installed headers for GNU glibc and other system libraries. + +Package: linux-image-${KERNEL_NAME_PROVIDED}-dbg +Section: debug +Architecture: any +Description: Linux kernel debugging symbols for ${PV} + This package will come in handy if you need to debug the kernel. It provides + all the necessary debug symbols for the kernel and its modules. diff --git a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl new file mode 100644 index 0000000..e4e75e3 --- /dev/null +++ b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl @@ -0,0 +1,36 @@ +#!/bin/bash +# Copyright (c) Mentor Graphics, a Siemens business, 2019 +# SPDX-License-Identifier: MIT + +# Load common stuff +. ${S}/debian/isar/common || exit ${?} + +do_build() { + + # Print a few things that are of particular interest + print_settings + + # Trace what we do here + set -x + + # Process existing kernel configuration to make sure it is complete + # (use defaults for options that were not specified) + ${MAKE} O=${KERNEL_BUILD_DIR} olddefconfig || exit ${?} + + # Build the Linux kernel + ${MAKE} O=${KERNEL_BUILD_DIR} -j $(nproc) || exit ${?} + + # Stop tracing + set +x +} + +print_settings() { + cat <&2 + return 1 + fi + + # create do_ variables for each package to be generated + for p in ${DEB_PACKAGES}; do + p="${p//-/_}" + eval do_${p}="1" + done + + # variables to be exported + export ARCH + + # are we cross-compiling? + BUILD_ARCH=$(dpkg-architecture -qDEB_BUILD_ARCH) + if [ "${BUILD_ARCH}" = "${DISTRO_ARCH}" ]; then + # no, make sure CROSS_COMPILE isn't set + unset CROSS_COMPILE + fi + + # call the actual target script + do_${target} || return ${?} +} diff --git a/meta/recipes-kernel/linux/files/debian/isar/install.tmpl b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl new file mode 100644 index 0000000..d586392 --- /dev/null +++ b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl @@ -0,0 +1,204 @@ +#!/bin/bash +# Copyright (c) Mentor Graphics, a Siemens business, 2019 +# SPDX-License-Identifier: MIT + +# Load common stuff +. ${S}/debian/isar/common || exit ${?} + +# Stop on error +set -e + +do_install() { + + # check if our kernel was configured + if [ ! -f "${O}/.config" ]; then + echo "error: kernel not configured!" >&2 + return 1 + fi + + # load its configuration + . ${O}/.config + + kimage="$(${MAKE} O=${O} -s --no-print-directory image_name)" + case "${ARCH}" in + mips|powerpc) kimage_path="boot/vmlinux-${PV}" ;; + um) kimage_path="usr/bin/vmlinux-${PV}" ;; + *) kimage_path="boot/vmlinuz-${PV}" ;; + esac + + print_settings + + # Trace what we do here + set -x + + install_image + install_hooks + install_dtbs + install_kmods + install_headers + + # Stop tracing + set +x +} + +print_settings() { + cat <${src_hdr_files} + tar -C ${odir} -cf - -T - <${src_hdr_files} | tar -C ${destdir} -xf - +} + +kernel_tools() { + # remove object files + find ${destdir}/scripts -type f -name '*.o' |xargs rm -f + + # we're all done if we aren't cross-compiling + [ -n "${CROSS_COMPILE}" ] || return 0 + + # remove ELF executables from the linux-headers package + find ${destdir}/scripts -type f -exec file {} \;|grep ELF|cut -d: -f1|xargs rm -fv + + # cross-compile kernel tools to be shipped with linux-headers + kernel_tools_build +} + +kernel_headers() { + destdir=${deb_kern_hdr_dir}/${KERNEL_HEADERS_DIR} + src_hdr_files=$(mktemp) + obj_hdr_files=$(mktemp) + + mkdir -p ${destdir} + mkdir -p ${deb_kern_hdr_dir}/lib/modules/${PV} + + (cd ${S}; find . -name 'Makefile*' -o -name 'Kconfig*' -o -name '*.pl') >>${src_hdr_files} + (cd ${S}; find arch/*/include include scripts -type f -o -type l) >>${src_hdr_files} + (cd ${S}; find arch/${ARCH} -name module.lds -o -name Kbuild.platforms -o -name Platform) >>${src_hdr_files} + (cd ${S}; find $(find arch/${ARCH} -name include -o -name scripts -type d) -type f) >>${src_hdr_files} + + (cd ${O}; find arch/${ARCH}/include Module.symvers include scripts -type f) >>${obj_hdr_files} + if [ -n "${CONFIG_STACK_VALIDATION}" ]; then + (cd ${O}; find tools/objtool -type f -executable) >>${obj_hdr_files} + fi + if [ -n "${CONFIG_GCC_PLUGINS}" ]; then + (cd ${O}; find scripts/gcc-plugins -name *.so -o -name gcc-common.h) >>${obj_hdr_files} + fi + + # deploy files that were matched above + tar -C ${S} -cf - -T - <${src_hdr_files} | tar -C ${destdir} -xf - + tar -C ${O} -cf - -T - <${obj_hdr_files} | tar -C ${destdir} -xf - + + # add the kernel config + cp ${O}/${KCONF} ${destdir}/.config + + # handle kernel development tools + kernel_tools + + # create symlinks + ln -sf /${KERNEL_HEADERS_DIR} ${deb_kern_hdr_dir}/lib/modules/${PV}/build +} + +install_headers() { + headers_check + [ -z "${do_linux_libc_headers}" ] || libc_headers + kernel_headers +} + +main install ${*} diff --git a/meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl b/meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl new file mode 100644 index 0000000..58fb4a1 --- /dev/null +++ b/meta/recipes-kernel/linux/files/debian/linux-image.postinst.tmpl @@ -0,0 +1,29 @@ +#!/bin/sh +# based on https://salsa.debian.org/kernel-team/linux/blob/master/debian/templates/image.postinst.in + +# Tell initramfs builder whether it's wanted +export INITRD=Yes + +version=${PV} +image_path=/boot/${KERNEL_FILE}-${PV} + +if [ "$1" != configure ]; then + exit 0 +fi + +depmod $version + +if [ -f /lib/modules/$version/.fresh-install ]; then + change=install +else + change=upgrade +fi +linux-update-symlinks $change $version $image_path +rm -f /lib/modules/$version/.fresh-install + +if [ -d /etc/kernel/postinst.d ]; then + DEB_MAINT_PARAMS="$*" run-parts --report --exit-on-error --arg=$version \ + --arg=$image_path /etc/kernel/postinst.d +fi + +exit 0 diff --git a/meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl b/meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl new file mode 100644 index 0000000..56325cf --- /dev/null +++ b/meta/recipes-kernel/linux/files/debian/linux-image.postrm.tmpl @@ -0,0 +1,35 @@ +#!/bin/sh -e +# based on https://salsa.debian.org/kernel-team/linux/blob/master/debian/templates/image.postrm.in + +# Tell initramfs builder whether it's wanted +export INITRD=Yes + +version=${PV} +image_path=/boot/${KERNEL_FILE}/${PV} + +rm -f /lib/modules/$version/.fresh-install + +if [ "$1" != upgrade ] && command -v linux-update-symlinks >/dev/null; then + linux-update-symlinks remove $version $image_path +fi + +if [ -d /etc/kernel/postrm.d ]; then + DEB_MAINT_PARAMS="$*" run-parts --report --exit-on-error --arg=$version \ + --arg=$image_path /etc/kernel/postrm.d +fi + +if [ "$1" = purge ]; then + for extra_file in modules.dep modules.isapnpmap modules.pcimap \ + modules.usbmap modules.parportmap \ + modules.generic_string modules.ieee1394map \ + modules.ieee1394map modules.pnpbiosmap \ + modules.alias modules.ccwmap modules.inputmap \ + modules.symbols modules.ofmap \ + modules.seriomap modules.\*.bin \ + modules.softdep modules.devname; do + eval rm -f /lib/modules/$version/$extra_file + done + rmdir /lib/modules/$version || true +fi + +exit 0 diff --git a/meta/recipes-kernel/linux/files/debian/linux-image.preinst.tmpl b/meta/recipes-kernel/linux/files/debian/linux-image.preinst.tmpl new file mode 100644 index 0000000..4440893 --- /dev/null +++ b/meta/recipes-kernel/linux/files/debian/linux-image.preinst.tmpl @@ -0,0 +1,25 @@ +#!/bin/sh -e +# based on https://salsa.debian.org/kernel-team/linux/blob/master/debian/templates/image.preinst.in + +# Tell initramfs builder whether it's wanted +export INITRD=Yes + +version=${PV} +image_path=/boot/${KERNEL_FILE}-${PV} + +if [ "$1" = abort-upgrade ]; then + exit 0 +fi + +if [ "$1" = install ]; then + # Create a flag file for postinst + mkdir -p /lib/modules/$version + touch /lib/modules/$version/.fresh-install +fi + +if [ -d /etc/kernel/preinst.d ]; then + DEB_MAINT_PARAMS="$*" run-parts --report --exit-on-error --arg=$version \ + --arg=$image_path /etc/kernel/preinst.d +fi + +exit 0 diff --git a/meta/recipes-kernel/linux/files/debian/linux-image.prerm.tmpl b/meta/recipes-kernel/linux/files/debian/linux-image.prerm.tmpl new file mode 100644 index 0000000..d5d5ccb --- /dev/null +++ b/meta/recipes-kernel/linux/files/debian/linux-image.prerm.tmpl @@ -0,0 +1,21 @@ +#!/bin/sh -e +# based on https://salsa.debian.org/kernel-team/linux/blob/master/debian/templates/image.prerm.in + +# Tell initramfs builder whether it's wanted +export INITRD=Yes + +version=${PV} +image_path=/boot/${KERNEL_FILE}-${PV} + +if [ "$1" != remove ]; then + exit 0 +fi + +linux-check-removal $version + +if [ -d /etc/kernel/prerm.d ]; then + DEB_MAINT_PARAMS="$*" run-parts --report --exit-on-error --arg=$version \ + --arg=$image_path /etc/kernel/prerm.d +fi + +exit 0 diff --git a/meta/recipes-kernel/linux/files/debian/rules.tmpl b/meta/recipes-kernel/linux/files/debian/rules.tmpl new file mode 100755 index 0000000..1728dda --- /dev/null +++ b/meta/recipes-kernel/linux/files/debian/rules.tmpl @@ -0,0 +1,41 @@ +#!/usr/bin/make -f + +export CROSS_COMPILE:=$(DEB_HOST_GNU_TYPE)- +export DEB_PACKAGES:=$(shell dh_listpackages) + +O:=$(CURDIR)/${KERNEL_BUILD_DIR} +S:=$(CURDIR) +deb_top_dir:=$(S)/debian + +DH_VERBOSE=1 + +# Dynamic variables to be passed to Isar build scripts +isar_env=$(strip \ + export CROSS_COMPILE='$(CROSS_COMPILE)' && \ + export DEB_PACKAGES='$(DEB_PACKAGES)' && \ + export MAKE='$(MAKE)' && \ + export O='${O}' \ + export S='${S}' \ +) + +%: + dh $(@) + +.PHONY: override_dh_auto_clean +override_dh_auto_clean: + $(isar_env) && bash $(deb_top_dir)/isar/clean + +.PHONY: override_dh_auto_build +override_dh_auto_build: + $(isar_env) && bash $(deb_top_dir)/isar/build + +override_dh_auto_install: + $(isar_env) && bash $(deb_top_dir)/isar/install + +.PHONY: override_dh_auto_test +override_dh_auto_test: + true + +PHONY: override_dh_strip +override_dh_strip: + dh_strip -Xvmlinux --no-automatic-dbgsym diff --git a/meta/recipes-kernel/linux/linux-custom.inc b/meta/recipes-kernel/linux/linux-custom.inc index ca91f64..00e90d0 100644 --- a/meta/recipes-kernel/linux/linux-custom.inc +++ b/meta/recipes-kernel/linux/linux-custom.inc @@ -2,22 +2,106 @@ # # This software is a part of ISAR. # Copyright (c) Siemens AG, 2018 +# Copyright (c) Mentor Graphics, a Siemens business, 2019 # # SPDX-License-Identifier: MIT -FILESEXTRAPATHS_prepend := "${FILE_DIRNAME}/files:" +# Settings that would typically be done from the custom kernel recipe +# ------------------------------------------------------------------- DESCRIPTION ?= "Custom kernel" +MAINTAINER ?= "isar-users " + +KBUILD_DEPENDS ?= "build-essential:native, \ + libelf-dev:native, \ + libncurses-dev:native, \ + libssl-dev:native, \ + bc, \ + bison, \ + cpio, \ + flex, \ + git, \ + kmod, \ + libssl-dev," + +KERNEL_DEBIAN_DEPENDS ?= "initramfs-tools | linux-initramfs-tool, \ + kmod, \ + linux-base (>= 4.3~)," + +KERNEL_HEADERS_DEBIAN_DEPENDS ?= "libc6, \ + libssl1.1," + +KERNEL_LIBC_DEV_DEPLOY ?= "0" + +# Settings that may be changed on a per distro, machine or layer basis +# -------------------------------------------------------------------- + +KERNEL_DTB_DIR ?= "usr/lib/linux-${KERNEL_NAME_PROVIDED}-${PV}" +KERNEL_HEADERS_DIR ?= "usr/src/linux-headers-${PV}" -KERNEL_NAME_PROVIDED ?= "${@ d.getVar('PN', True).partition('linux-')[2]}" KERNEL_DEFCONFIG ?= "" +# Add our template meta-data to the sources +FILESEXTRAPATHS_prepend := "${FILE_DIRNAME}/files:" +SRC_URI += "file://debian" + +# Variables and files that make our templates +# ------------------------------------------- + +TEMPLATE_FILES += " \ + debian/control.tmpl \ + debian/isar/build.tmpl \ + debian/isar/clean.tmpl \ + debian/isar/common.tmpl \ + debian/isar/install.tmpl \ + debian/linux-image.postinst.tmpl \ + debian/linux-image.postrm.tmpl \ + debian/linux-image.preinst.tmpl \ + debian/linux-image.prerm.tmpl \ + debian/rules.tmpl \ +" + +TEMPLATE_VARS += " \ + KBUILD_DEPENDS \ + KERNEL_ARCH \ + KERNEL_DEBIAN_DEPENDS \ + KERNEL_BUILD_DIR \ + KERNEL_DTB_DIR \ + KERNEL_FILE \ + KERNEL_HEADERS_DEBIAN_DEPENDS \ + KERNEL_HEADERS_DIR \ + KERNEL_NAME_PROVIDED \ +" + +inherit dpkg +inherit template + +# Derive name of the kernel packages from the name of this recipe +KERNEL_NAME_PROVIDED ?= "${@ d.getVar('PN', True).partition('linux-')[2]}" + +# Make bitbake know we will be producing linux-image and linux-headers packages python() { kernel_name = d.getVar("KERNEL_NAME_PROVIDED", True) d.setVar('PROVIDES', 'linux-image-' + kernel_name + ' ' + \ 'linux-headers-' + kernel_name) } +def get_kernel_arch(d): + distro_arch = d.getVar("DISTRO_ARCH") + if distro_arch == "amd64": + kernel_arch = "x86" + elif distro_arch == "arm64": + kernel_arch = "arm64" + elif distro_arch == "armhf": + kernel_arch = "arm" + elif distro_arch == "mipsel": + kernel_arch = "mips" + else: + kernel_arch = "" + return kernel_arch + +KERNEL_ARCH ??= "${@get_kernel_arch(d)}" + def config_fragments(d): fragments = [] sources = d.getVar("SRC_URI").split() @@ -31,57 +115,37 @@ def config_fragments(d): fragments.append(local) return fragments -inherit dpkg-base - -SRC_URI += "file://build-kernel.sh" +do_prepare_build_prepend() { + # copy meta-data over to source tree + rm -rf ${S}/debian + cp -r ${WORKDIR}/debian ${S}/ -KBUILD_DEPENDS ?= " \ - build-essential:native \ - libssl-dev \ - libelf-dev \ - bc \ - git \ - kmod \ - bison \ - flex \ - cpio \ - libncurses-dev" -KERNEL_DEBIAN_DEPENDS ?= "initramfs-tools | linux-initramfs-tool, kmod, linux-base (>= 4.3~)" -KERNEL_HEADERS_DEBIAN_DEPENDS ?= "libc6, libssl1.1" + # remove templates from the source tree + find ${S}/debian -name *.tmpl | xargs rm -f -KERNEL_LIBC_DEV_DEPLOY ?= "0" + # rename install/remove hooks to match user-specified name for our linux-image package + mv ${S}/debian/linux-image.postinst ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.postinst + mv ${S}/debian/linux-image.postrm ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.postrm + mv ${S}/debian/linux-image.preinst ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.preinst + mv ${S}/debian/linux-image.prerm ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.prerm -do_install_builddeps() { - dpkg_do_mounts - E="${@ bb.utils.export_proxies(d)}" - sudo -E chroot ${BUILDCHROOT_DIR} \ - apt-get update \ - -o Dir::Etc::SourceList="sources.list.d/isar-apt.list" \ - -o Dir::Etc::SourceParts="-" \ - -o APT::Get::List-Cleanup="0" - sudo -E chroot ${BUILDCHROOT_DIR} \ - apt-get install \ - -y -o Debug::pkgProblemResolver=yes \ - --no-install-recommends ${KBUILD_DEPENDS} - dpkg_undo_mounts + # produce a changelog for our kernel build + deb_add_changelog } -addtask install_builddeps after do_prepare_build before do_dpkg_build -# apt and reprepro may not run in parallel, acquire the Isar lock -do_install_builddeps[lockfiles] += "${REPO_ISAR_DIR}/isar.lock" - -addtask devshell after do_install_builddeps +# build directory for our "full" kernel build +KERNEL_BUILD_DIR = "build-full" -dpkg_runbuild() { - chmod +x ${WORKDIR}/build-kernel.sh - KERNEL_CONFIG_TARGET="${KERNEL_DEFCONFIG}" +dpkg_configure_kernel() { + config_target="${KERNEL_DEFCONFIG}" + rm -rf ${S}/${KERNEL_BUILD_DIR} && mkdir -p ${S}/${KERNEL_BUILD_DIR} if [ -n "${KERNEL_DEFCONFIG}" ]; then if [ -e "${WORKDIR}/${KERNEL_DEFCONFIG}" ]; then - cp ${WORKDIR}/${KERNEL_DEFCONFIG} ${S}/.config - KERNEL_CONFIG_TARGET="olddefconfig" + cp ${WORKDIR}/${KERNEL_DEFCONFIG} ${S}/${KERNEL_BUILD_DIR}/.config + config_target="olddefconfig" fi else - KERNEL_CONFIG_TARGET="defconfig" + config_target="defconfig" fi # copy config fragments over to the kernel tree @@ -92,22 +156,23 @@ dpkg_runbuild() { mkdir -p ${S}/debian/fragments (cd ${WORKDIR} && cp ${src_frags} ${S}/debian/fragments/) fi - export KERNEL_FRAGMENTS="${out_frags}" - E="${@ bb.utils.export_proxies(d)}" - - export PV=${PV} - export KERNEL_NAME=${KERNEL_NAME_PROVIDED} - - export KBUILD_DEPENDS="${KBUILD_DEPENDS}" - export KERNEL_CONFIG_TARGET - export KERNEL_DEBIAN_DEPENDS="${KERNEL_DEBIAN_DEPENDS}" - export KERNEL_FILE="${KERNEL_FILE}" - export KERNEL_HEADERS_DEBIAN_DEPENDS="${KERNEL_HEADERS_DEBIAN_DEPENDS}" - - sudo -E chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} ${PP}/build-kernel.sh ${PP}/${PPS} ${DISTRO_ARCH} + sudo -E chroot --userspec=$(id -u):$(id -g) ${BUILDCHROOT_DIR} sh -c " \ + export ARCH=${KERNEL_ARCH} && \ + cd ${PP}/${PPS} && \ + make O=${KERNEL_BUILD_DIR} ${config_target} && \ + ./scripts/kconfig/merge_config.sh \ + -O ${KERNEL_BUILD_DIR}/ \ + ${KERNEL_BUILD_DIR}/.config \ + ${out_frags} \ + " +} +dpkg_runbuild_prepend() { + profiles="${BASE_DISTRO}" if [ "${KERNEL_LIBC_DEV_DEPLOY}" != "1" ]; then - rm -f ${WORKDIR}/linux-libc-dev_${PV}*.deb + profiles="${profiles} nolibcdev" fi + export DEB_BUILD_PROFILES="${profiles}" + dpkg_configure_kernel } -- 2.20.1