From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6520199916203016192 X-Received: by 10.25.160.195 with SMTP id j186mr54711lfe.44.1518517262361; Tue, 13 Feb 2018 02:21:02 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 10.46.104.25 with SMTP id c25ls1467639lja.7.gmail; Tue, 13 Feb 2018 02:21:01 -0800 (PST) X-Google-Smtp-Source: AH8x227FoONEIwLoJHWTfpaAL7PkbRWIdJxHUL6M8nZB7nRk8szbvXuyjcWUvTw8HwsAWD0WRYpC X-Received: by 10.46.89.91 with SMTP id n88mr41450ljb.35.1518517261697; Tue, 13 Feb 2018 02:21:01 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1518517261; cv=none; d=google.com; s=arc-20160816; b=k+DEVQxIOFV8eeMMZls9meMZdPBtL4FKZojlDzRv9aK8+ybT/8UfnElIZAy4P0+3sF xfuLg2NEdlxl81k/F2QLP/c/XPV546/DwtRauhntAhn9KpNI0JV33hLB7jtGkeCl2M9V 2nJDH2vIgCavGIuiuvyzy7tJz9saX084/iSUcJ/XfmGgzo33mDm3ADCy2okU0ViF8sBG XwZADDqJ5xA6R94LplER3JucGaGz7USBgCPEgWiTkhYp2K1sTzgZPyIQZrfZuJw7tBXm Vzq7KVHdzHRRmgcicnmyobS3+koHr7NnK7bx/qgH4w3H7olTAFVxiINqlkaODR7QsX3n d31Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:cc:to:subject :arc-authentication-results; bh=kK/ZVwCyaymiBsaXy2XNJSjqcx52eGrcWCiRab2NCd8=; b=kqWJ5F7k221oq9FJVzNazoRni+5Y6EzvT+QkaC+ZSkoLnaYwqyBjaTjwXG+8L5nQCY FZjoNg2Nq3aYkKuYlHUuzLjHaepgHtj0noyZ8gNpCW4NMXU4swWSvCw5cjB7V7tAqq/9 VpvSJ5u2Hr3joSEupsSGhKfW2nUWPLs46qtmsO+8rLjvT2R+bNST7MyprMY95THCV0mi hPC8QS+1lrEt80XhFMDpoZ1B9lh+xLVhMVS50qwvlvHAlRizN4SCwNTVzMQ8dvJZcjXC SktrPfBHJehP9OXax2yxABV9bgaXm3Hh9Y1YWR7mVBkixZgL9GI5g6+AP4nqUQYEaMEM xmcg== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.28 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com Return-Path: Received: from goliath.siemens.de (goliath.siemens.de. [192.35.17.28]) by gmr-mx.google.com with ESMTPS id w7si81886ljw.4.2018.02.13.02.21.01 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 13 Feb 2018 02:21:01 -0800 (PST) Received-SPF: pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.28 as permitted sender) client-ip=192.35.17.28; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.28 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com Received: from mail2.siemens.de (mail2.siemens.de [139.25.208.11]) by goliath.siemens.de (8.15.2/8.15.2) with ESMTPS id w1DAL0CY010144 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2018 11:21:01 +0100 Received: from [139.25.68.37] (md1q0hnc.ad001.siemens.net [139.25.68.37] (may be forged)) by mail2.siemens.de (8.15.2/8.15.2) with ESMTP id w1DAL0Np017680; Tue, 13 Feb 2018 11:21:00 +0100 Subject: Re: [PATCH v4 5/8] Provide class for easy custom kernel builds To: Henning Schild Cc: Alexander Smirnov , isar-users References: <10f71292d5c06de6909594c891f2478334106135.1518362719.git.jan.kiszka@siemens.com> <5771c955-dec7-8347-123d-d4b3f08e751a@siemens.com> <20180213105349.05baf3a0@mmd1pvb1c.ad001.siemens.net> From: Jan Kiszka Message-ID: <3e4b153a-f82f-4d41-bc99-eb8f396be945@siemens.com> Date: Tue, 13 Feb 2018 11:21:00 +0100 User-Agent: Mozilla/5.0 (X11; U; Linux i686 (x86_64); de; rv:1.8.1.12) Gecko/20080226 SUSE/2.0.0.12-1.1 Thunderbird/2.0.0.12 Mnenhy/0.7.5.666 MIME-Version: 1.0 In-Reply-To: <20180213105349.05baf3a0@mmd1pvb1c.ad001.siemens.net> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-TUID: 29e7xIlrs1/q On 2018-02-13 10:53, Henning Schild wrote: > Am Tue, 13 Feb 2018 08:03:12 +0100 > schrieb "[ext] Jan Kiszka" : > >> On 2018-02-12 19:56, Alexander Smirnov wrote: >>> On 02/11/2018 06:25 PM, Jan Kiszka wrote: >>>> From: Jan Kiszka >>>> >>>> With this class, it becomes almost trivial to replace the distro >>>> kernel with a custom build. You just need to inherit linux-kernel, >>>> specify the source URI, define via S where the source is unpacked >>>> to and provide a defconfig. To switch to a custom kernel recipe, >>>> PREFERRED_PROVIDER_virtual/kernel has to be adjusted in local.conf >>>> or the distro conf. >>>> >>>> The approach works internally by first running "make deb-pkg" on >>>> the kernel and the repackages the output to make the binary >>>> linux-image and linux-header debs act as replacement of their >>>> distro packages. This results in a suboptimal technical >>>> implementation which may eventually be replaced by an >>>> isar-implemented deb-pkg build process. However, this is not >>>> expected to affect the user-visible interface of this class. >>>> >>>> Signed-off-by: Jan Kiszka >>>> --- >>>>   meta/classes/linux-kernel.bbclass | 98 >>>> +++++++++++++++++++++++++++++++++++++++ >>>>   1 file changed, 98 insertions(+) >>>>   create mode 100644 meta/classes/linux-kernel.bbclass >>>> >>>> diff --git a/meta/classes/linux-kernel.bbclass >>>> b/meta/classes/linux-kernel.bbclass >>>> new file mode 100644 >>>> index 0000000..5f4df3f >>>> --- /dev/null >>>> +++ b/meta/classes/linux-kernel.bbclass >>>> @@ -0,0 +1,98 @@ >>>> +# Custom kernel build >>>> +# >>>> +# This software is a part of ISAR. >>>> +# Copyright (c) Siemens AG, 2018 >>>> +# >>>> +# SPDX-License-Identifier: MIT >>>> + >>>> +DESCRIPTION ?= "Custom kernel" >>>> +PROVIDES = "virtual/kernel" >>>> + >>>> +inherit dpkg-base >>>> + >>>> +KERNEL_DEBIAN_DEPENDS ?= "initramfs-tools | linux-initramfs-tool, >>>> kmod, linux-base (>= 4.3~)" >>>> +KERNEL_HEADERS_DEBIAN_DEPENDS ?= "libc6, libssl1.1, gcc" >>>> +KBUILD_DEPENDS ?= "libssl-dev libelf-dev bc" >>>> + >>>> +REPACK_DIR = "${PP}/repack" >>>> +REPACK_LINUX_IMAGE_DIR = "${REPACK_DIR}/linux-image" >>>> +REPACK_LINUX_HEADERS_DIR = "${REPACK_DIR}/linux-headers" >>>> + >>>> +dpkg_runbuild() { >>>> +    E="${@ bb.utils.export_proxies(d)}" >>>> +    sudo -E chroot ${BUILDCHROOT_DIR} sh -c ' >>>> +        set -e >>>> + >>>> +        apt-get install -y -o Debug::pkgProblemResolver=yes \ >>>> +            --no-install-recommends ${KBUILD_DEPENDS} >>>> + >>>> +        cd ${PP}/${S} >>>> +        cp ../defconfig .config >>>> +        make olddefconfig >>>> + >>>> +        make -j ${@bb.utils.cpu_count() * 2} deb-pkg >>>> + >>>> +        rm -rf ${REPACK_DIR} >>>> +        mkdir -p ${REPACK_DIR} >>>> +        mkdir -p ${REPACK_LINUX_IMAGE_DIR} >>>> +        mkdir -p ${REPACK_LINUX_HEADERS_DIR} >>>> + >>>> +        cd ${PP} >>>> +        tar xzf linux-${PV}_${PV}-1.debian.tar.gz -C ${REPACK_DIR} >>>> +        dpkg-deb -R linux-image-${PV}_${PV}-1_${KERNEL_ARCH}.deb \ >>>> +            ${REPACK_LINUX_IMAGE_DIR} >>>> +        dpkg-deb -R >>>> linux-headers-${PV}_${PV}-1_${KERNEL_ARCH}.deb \ >>>> +            ${REPACK_LINUX_HEADERS_DIR} >>>> + >>>> +        dpkg-gencontrol -crepack/debian/control \ >>>> +            -lrepack/debian/changelog \ >>>> +            -frepack/debian/files \ >>>> +            -plinux-image-${PV} \ >>>> +            -P${REPACK_LINUX_IMAGE_DIR} \ >>>> +            -DPackage="linux-image-${KERNEL_ARCH}" \ >>>> +            -DSection=kernel \ >>>> +            -DPriority=required \ >>>> +            -DProvides="${PN}" \ >>>> +            -DDepends="${KERNEL_DEBIAN_DEPENDS}" >>>> + >>>> +        # Add Debian-like link installation to postinst >>>> +        touch >>>> ${REPACK_LINUX_IMAGE_DIR}/lib/modules/${PV}/.fresh-install >>>> +        sed -i ${REPACK_LINUX_IMAGE_DIR}/DEBIAN/postinst \ >>>> +            -e "/^set -e$/a\\ >>>> +\\ >>>> +if [ -f /lib/modules/${PV}/.fresh-install ]; then\\ >>>> +    change=install\\ >>>> +else\\ >>>> +    change=upgrade\\ >>>> +fi\\ >>>> +linux-update-symlinks \$change ${PV} /boot/vmlinuz-${PV}\\ >>>> +rm -f /lib/modules/${PV}/.fresh-install" >>>> + >>>> +        # Add Debian-like link removal to postrm >>>> +        sed -i ${REPACK_LINUX_IMAGE_DIR}/DEBIAN/postrm \ >>>> +            -e "/^set -e$/a\\ >>>> +\\ >>>> +rm -f /lib/modules/${PV}/.fresh-install\\ >>>> +\\ >>>> +if [ \"\$1\" != upgrade ] && command -v linux-update-symlinks >>>>> /dev/null; then\\ >>>> +    linux-update-symlinks remove ${PV}  /boot/vmlinuz-${PV}\\ >>>> +fi" >>>> + >>>> +        dpkg-gencontrol -crepack/debian/control \ >>>> +            -lrepack/debian/changelog \ >>>> +            -frepack/debian/files \ >>>> +            -plinux-headers-${PV} \ >>>> +            -P${REPACK_LINUX_HEADERS_DIR} \ >>>> +            -Vkernel:debarch="${KERNEL_ARCH}" \ >>>> +            -DPackage="linux-headers-${KERNEL_ARCH}" \ >>>> +            -DSection=kernel \ >>>> +            -DDepends="${KERNEL_HEADERS_DEBIAN_DEPENDS}" >>>> + >>>> +        dpkg-deb -b ${REPACK_LINUX_IMAGE_DIR} \ >>>> +            linux-image-${KERNEL_ARCH}_${PV}-1_${KERNEL_ARCH}.deb >>>> +        rm -f linux-image-${PV}_${PV}-1_${KERNEL_ARCH}.deb >>>> +        dpkg-deb -b ${REPACK_LINUX_HEADERS_DIR} \ >>>> + >>>> linux-headers-${KERNEL_ARCH}_${PV}-1_${KERNEL_ARCH}.deb >>>> +        rm -f linux-headers-${PV}_${PV}-1_${KERNEL_ARCH}.deb >>>> +    ' >>>> +} >>>> >>> >>> What is the benefit of having this script in the class? I see that >>> we have separate files for: >>> >>>  - buildchroot >>>  - kernel modules >>> >>> Why this chroot script should be a part of bitbake class? This makes >>> debugging much more complicated, because I have no possibility to >>> run this script manually from buildchroot (like I could do with >>> build.sh) >> >> I played with it, and it "just" took 6 exports to set up the interface >> to script-based build. That will likely not be convenient for >> debugging as well (and I never had to call things manually that way, >> I got all information from logs), but I can keep that pattern. Will >> also mean: back to an include, rather than using a class (no big >> issue). > > You could write all these exports into a file that the scripts sources. > Given that file exists, one could run manually and the recipe would not > contain all that logic. That sounds like overkill. And there is already a file containing that information (run.do_build). I'm still looking for a concrete case where local invocation is needed, even more as we are now able to restart builds easily. > >>> >>> Also formatting style looks nasty here. >> >> That's not going to change unless you suggest some alternative for the >> sed-based code injection into postinst/postrm. > > Can that code be appended with echo >> ? The patching looks pretty > unconditional already. I need to inject the code at a specific place in to the script, so it's not about appending or prepending. At the same time, I'm concerned that a real patch would break too often. Jan -- Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux