Re: [PATCH v4 5/8] Provide class for easy custom kernel builds

[Jan Kiszka <jan.kiszka@siemens.com>]

On 2018-02-13 10:53, Henning Schild wrote:
> Am Tue, 13 Feb 2018 08:03:12 +0100
> schrieb "[ext] Jan Kiszka" <jan.kiszka@siemens.com>:
> 
>> On 2018-02-12 19:56, Alexander Smirnov wrote:
>>> On 02/11/2018 06:25 PM, Jan Kiszka wrote:  
>>>> From: Jan Kiszka <jan.kiszka@siemens.com>
>>>>
>>>> With this class, it becomes almost trivial to replace the distro
>>>> kernel with a custom build. You just need to inherit linux-kernel,
>>>> specify the source URI, define via S where the source is unpacked
>>>> to and provide a defconfig. To switch to a custom kernel recipe,
>>>> PREFERRED_PROVIDER_virtual/kernel has to be adjusted in local.conf
>>>> or the distro conf.
>>>>
>>>> The approach works internally by first running "make deb-pkg" on
>>>> the kernel and the repackages the output to make the binary
>>>> linux-image and linux-header debs act as replacement of their
>>>> distro packages. This results in a suboptimal technical
>>>> implementation which may eventually be replaced by an
>>>> isar-implemented deb-pkg build process. However, this is not
>>>> expected to affect the user-visible interface of this class.
>>>>
>>>> Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
>>>> ---
>>>>   meta/classes/linux-kernel.bbclass | 98
>>>> +++++++++++++++++++++++++++++++++++++++
>>>>   1 file changed, 98 insertions(+)
>>>>   create mode 100644 meta/classes/linux-kernel.bbclass
>>>>
>>>> diff --git a/meta/classes/linux-kernel.bbclass
>>>> b/meta/classes/linux-kernel.bbclass
>>>> new file mode 100644
>>>> index 0000000..5f4df3f
>>>> --- /dev/null
>>>> +++ b/meta/classes/linux-kernel.bbclass
>>>> @@ -0,0 +1,98 @@
>>>> +# Custom kernel build
>>>> +#
>>>> +# This software is a part of ISAR.
>>>> +# Copyright (c) Siemens AG, 2018
>>>> +#
>>>> +# SPDX-License-Identifier: MIT
>>>> +
>>>> +DESCRIPTION ?= "Custom kernel"
>>>> +PROVIDES = "virtual/kernel"
>>>> +
>>>> +inherit dpkg-base
>>>> +
>>>> +KERNEL_DEBIAN_DEPENDS ?= "initramfs-tools | linux-initramfs-tool,
>>>> kmod, linux-base (>= 4.3~)"
>>>> +KERNEL_HEADERS_DEBIAN_DEPENDS ?= "libc6, libssl1.1, gcc"
>>>> +KBUILD_DEPENDS ?= "libssl-dev libelf-dev bc"
>>>> +
>>>> +REPACK_DIR = "${PP}/repack"
>>>> +REPACK_LINUX_IMAGE_DIR = "${REPACK_DIR}/linux-image"
>>>> +REPACK_LINUX_HEADERS_DIR = "${REPACK_DIR}/linux-headers"
>>>> +
>>>> +dpkg_runbuild() {
>>>> +    E="${@ bb.utils.export_proxies(d)}"
>>>> +    sudo -E chroot ${BUILDCHROOT_DIR} sh -c '
>>>> +        set -e
>>>> +
>>>> +        apt-get install -y -o Debug::pkgProblemResolver=yes \
>>>> +            --no-install-recommends ${KBUILD_DEPENDS}
>>>> +
>>>> +        cd ${PP}/${S}
>>>> +        cp ../defconfig .config
>>>> +        make olddefconfig
>>>> +
>>>> +        make -j ${@bb.utils.cpu_count() * 2} deb-pkg
>>>> +
>>>> +        rm -rf ${REPACK_DIR}
>>>> +        mkdir -p ${REPACK_DIR}
>>>> +        mkdir -p ${REPACK_LINUX_IMAGE_DIR}
>>>> +        mkdir -p ${REPACK_LINUX_HEADERS_DIR}
>>>> +
>>>> +        cd ${PP}
>>>> +        tar xzf linux-${PV}_${PV}-1.debian.tar.gz -C ${REPACK_DIR}
>>>> +        dpkg-deb -R linux-image-${PV}_${PV}-1_${KERNEL_ARCH}.deb \
>>>> +            ${REPACK_LINUX_IMAGE_DIR}
>>>> +        dpkg-deb -R
>>>> linux-headers-${PV}_${PV}-1_${KERNEL_ARCH}.deb \
>>>> +            ${REPACK_LINUX_HEADERS_DIR}
>>>> +
>>>> +        dpkg-gencontrol -crepack/debian/control \
>>>> +            -lrepack/debian/changelog \
>>>> +            -frepack/debian/files \
>>>> +            -plinux-image-${PV} \
>>>> +            -P${REPACK_LINUX_IMAGE_DIR} \
>>>> +            -DPackage="linux-image-${KERNEL_ARCH}" \
>>>> +            -DSection=kernel \
>>>> +            -DPriority=required \
>>>> +            -DProvides="${PN}" \
>>>> +            -DDepends="${KERNEL_DEBIAN_DEPENDS}"
>>>> +
>>>> +        # Add Debian-like link installation to postinst
>>>> +        touch
>>>> ${REPACK_LINUX_IMAGE_DIR}/lib/modules/${PV}/.fresh-install
>>>> +        sed -i ${REPACK_LINUX_IMAGE_DIR}/DEBIAN/postinst \
>>>> +            -e "/^set -e$/a\\
>>>> +\\
>>>> +if [ -f /lib/modules/${PV}/.fresh-install ]; then\\
>>>> +    change=install\\
>>>> +else\\
>>>> +    change=upgrade\\
>>>> +fi\\
>>>> +linux-update-symlinks \$change ${PV} /boot/vmlinuz-${PV}\\
>>>> +rm -f /lib/modules/${PV}/.fresh-install"
>>>> +
>>>> +        # Add Debian-like link removal to postrm
>>>> +        sed -i ${REPACK_LINUX_IMAGE_DIR}/DEBIAN/postrm \
>>>> +            -e "/^set -e$/a\\
>>>> +\\
>>>> +rm -f /lib/modules/${PV}/.fresh-install\\
>>>> +\\
>>>> +if [ \"\$1\" != upgrade ] && command -v linux-update-symlinks  
>>>>> /dev/null; then\\  
>>>> +    linux-update-symlinks remove ${PV}  /boot/vmlinuz-${PV}\\
>>>> +fi"
>>>> +
>>>> +        dpkg-gencontrol -crepack/debian/control \
>>>> +            -lrepack/debian/changelog \
>>>> +            -frepack/debian/files \
>>>> +            -plinux-headers-${PV} \
>>>> +            -P${REPACK_LINUX_HEADERS_DIR} \
>>>> +            -Vkernel:debarch="${KERNEL_ARCH}" \
>>>> +            -DPackage="linux-headers-${KERNEL_ARCH}" \
>>>> +            -DSection=kernel \
>>>> +            -DDepends="${KERNEL_HEADERS_DEBIAN_DEPENDS}"
>>>> +
>>>> +        dpkg-deb -b ${REPACK_LINUX_IMAGE_DIR} \
>>>> +            linux-image-${KERNEL_ARCH}_${PV}-1_${KERNEL_ARCH}.deb
>>>> +        rm -f linux-image-${PV}_${PV}-1_${KERNEL_ARCH}.deb
>>>> +        dpkg-deb -b ${REPACK_LINUX_HEADERS_DIR} \
>>>> +
>>>> linux-headers-${KERNEL_ARCH}_${PV}-1_${KERNEL_ARCH}.deb
>>>> +        rm -f linux-headers-${PV}_${PV}-1_${KERNEL_ARCH}.deb
>>>> +    '
>>>> +}
>>>>  
>>>
>>> What is the benefit of having this script in the class? I see that
>>> we have separate files for:
>>>
>>>  - buildchroot
>>>  - kernel modules
>>>
>>> Why this chroot script should be a part of bitbake class? This makes
>>> debugging much more complicated, because I have no possibility to
>>> run this script manually from buildchroot (like I could do with
>>> build.sh)  
>>
>> I played with it, and it "just" took 6 exports to set up the interface
>> to script-based build. That will likely not be convenient for
>> debugging as well (and I never had to call things manually that way,
>> I got all information from logs), but I can keep that pattern. Will
>> also mean: back to an include, rather than using a class (no big
>> issue).
> 
> You could write all these exports into a file that the scripts sources.
> Given that file exists, one could run manually and the recipe would not
> contain all that logic.

That sounds like overkill. And there is already a file containing that
information (run.do_build).

I'm still looking for a concrete case where local invocation is needed,
even more as we are now able to restart builds easily.

> 
>>>
>>> Also formatting style looks nasty here.  
>>
>> That's not going to change unless you suggest some alternative for the
>> sed-based code injection into postinst/postrm.
> 
> Can that code be appended with echo >> ? The patching looks pretty
> unconditional already.

I need to inject the code at a specific place in to the script, so it's
not about appending or prepending. At the same time, I'm concerned that
a real patch would break too often.

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux