RE: [PATCH 1/1] Add support to build binary version of DKMS kernel modules

["Moessbauer, Felix" <felix.moessbauer@siemens.com>]

Hi,

I just added it to the CI and did some early tests with the v4l2loopback module.
For x86 and arm64, things run smooth, but for armv7 the module cannot be build (even after disabling crossbuilds).

This seems to be not an ISAR issue, but an upstream bug in the module DKMS itself.

I'll have a closer look at that and push a v2 soon.

Best regards,
Felix

> -----Original Message-----
> From: Jan Kiszka <jan.kiszka@siemens.com>
> Sent: Thursday, April 15, 2021 2:53 PM
> To: Schild, Henning (T RDA IOT SES-DE) <henning.schild@siemens.com>
> Cc: Moessbauer, Felix (T RDA IOT SES-DE) <felix.moessbauer@siemens.com>;
> isar-users@googlegroups.com
> Subject: Re: [PATCH 1/1] Add support to build binary version of DKMS kernel
> modules
> 
> On 15.04.21 14:26, Henning Schild wrote:
> > Am Wed, 14 Apr 2021 11:03:50 +0200
> > schrieb Jan Kiszka <jan.kiszka@siemens.com>:
> >
> >> On 14.04.21 10:52, Henning Schild wrote:
> >>> Hi Felix,
> >>>
> >>> thanks for pushing this one upstream.
> >>>
> >>> We will need a testcase for it, so make sure to enter into
> >>> "scripts/ci_build.sh"
> >>>
> >>> I would suggest the virtualbox modules or maybe vmware modules
> >>> instead. This way we would probably have more distro coverage and in
> >>> fact have a useful example for people doing virtualbox (the one i
> >>> would prefer). Not sure that is possible or why you picked dpdk.
> >>
> >> None of those crappy modules will build on non-x86. We should look
> >> for a portable example.
> >
> > Yes ideally we find a dkms package that exists on all distros and
> > arches, not sure that exists. Would also allow testing the Isar
> > crossbuild on that feature.
> >
> > If cross causes issues ISAR_CROSS_COMPILE = "0" in the class might be
> > acceptable, would be to me.
> >
> 
> Would enforce CROSS = "0" on a potential custom kernel as well, just to remind
> this.
> 
> Jan
> 
> > If we do not find such a generic module, all i said is that vbox is
> > probably more useful than igb_uio. And probably more available across
> > distros.
> >
> > Henning
> >
> >
> >> Jan
> >>
> >>>
> >>> Am Wed, 14 Apr 2021 10:36:17 +0200
> >>> schrieb Felix Moessbauer <felix.moessbauer@siemens.com>:
> >>>
> >>>> This patch adds support to build and install a kernel module that
> >>>> is available via a debian DKMS package.
> >>>> As it is hard to directly build and install the package we create
> >>>> and distribute a meta package that is independent of the kernel
> >>>> version. In that package, we depend on the versioned prebuild
> >>>> kernel module package.
> >>>>
> >>>> To build the dkms module, we add two tasks and directly use dkms to
> >>>> build the debian package containing the binary module Then we just
> >>>> copy the binary-module debian package to the workdir,
> >>>>
> >>>> Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
> >>>> ---
> >>>>  .../example-dkms-module/igb-uio_20.11.bb      | 14 ++++
> >>>>  meta/classes/dkms-module.bbclass              | 68
> >>>> +++++++++++++++++++ 2 files changed, 82 insertions(+)
> >>>>  create mode 100644
> >>>> meta-isar/recipes-kernel/example-dkms-module/igb-uio_20.11.bb
> >>>> create mode 100644 meta/classes/dkms-module.bbclass
> >>>>
> >>>> diff --git
> >>>> a/meta-isar/recipes-kernel/example-dkms-module/igb-uio_20.11.bb
> >>>> b/meta-isar/recipes-kernel/example-dkms-module/igb-uio_20.11.bb new
> >>>> file mode 100644 index 0000000..abb9922 --- /dev/null
> >>>> +++ b/meta-isar/recipes-kernel/example-dkms-module/igb-uio_20.11.bb
> >>>> @@ -0,0 +1,14 @@
> >>>> +# Example recipe for building the binary version of a DKMS module
> >>>> +# # This software is a part of ISAR.
> >>>> +# Copyright (c) Siemens AG, 2018
> >>>> +#
> >>>> +# SPDX-License-Identifier: MIT
> >>>> +
> >>>> +inherit dkms-module
> >>>> +
> >>>> +PN .= "-${KERNEL_NAME}"
> >>>> +
> >>>> +#package name (without -dkms. E.g "dpdk-kmods" for package
> >>>> "dpdk-kmods-dkms") +DKMS_PACKAGE_NAME = "dpdk-kmods"
> >>>> +AUTOLOAD += "igb_uio"
> >>>> diff --git a/meta/classes/dkms-module.bbclass
> >>>> b/meta/classes/dkms-module.bbclass new file mode 100644 index
> >>>> 0000000..d1dbba9
> >>>> --- /dev/null
> >>>> +++ b/meta/classes/dkms-module.bbclass
> >>>> @@ -0,0 +1,68 @@
> >>>> +# This software is a part of ISAR.
> >>>> +# Copyright (C) 2021 Siemens AG
> >>>> +#
> >>>> +# SPDX-License-Identifier: MIT
> >>>> +
> >>>> +inherit dpkg-raw
> >>>> +
> >>>> +# Build and install a kernel module that is available via a debian
> >>>> DKMS package. +# As it is hard to directly build and install the
> >>>> package we create and distribute +# a meta package that is
> >>>> independent of the kernel version. +# In that package, we depend on
> >>>> the versioned prebuild kernel module package. +#
> >>>> +# To build the dkms module, we add two tasks and directly # use
> >>>> +dkms to build the debian package containing the binary
> >>>> module +# Then we just copy the binary-module debian package to the
> >>>> workdir, +# so ISAR's do_deploy_deb picks it up.
> >>>> +
> >>>> +#package name (without -dkms. E.g "dpdk-kmods" for package
> >>>> "dpdk-kmods-dkms") +DKMS_PACKAGE_NAME ?= ""
> >>>> +AUTOLOAD ?= ""
> >>>> +
> >>>> +DESCRIPTION ?= "Kernel module from DKMS package for ${PN}"
> >>>> +DEPENDS += "linux-headers-${KERNEL_NAME}"
> >>>> +DEBIAN_DEPENDS += "${DKMS_PACKAGE_NAME}-modules,"
> >>>> +DEBIAN_BUILD_DEPENDS += "linux-headers-${KERNEL_NAME},
> >>>> ${DKMS_PACKAGE_NAME}-dkms," +
> >>>> +# install configuration to auto-load the modules in ${AUTOLOAD}
> >>>> +do_install() {
> >>>> +    # auto load the module
> >>>> +    install -v -d ${D}/etc/modules-load.d
> >>>> +    for module in "${AUTOLOAD}"; do
> >>>> +        echo $module > ${D}/etc/modules-load.d/${PN}.conf
> >>>> +    done
> >>>
> >>> I think this might need an "update-initramfs -u" in postinst, but i
> >>> am not sure, check the other autoloader code and see if you can
> >>> share. Such modules probably do not need early loading in initrd,
> >>> but "/etc/modules-load.d" is mirrored into the initrd so we should
> >>> not get out of sync.
> >>> It is important that both autoloaders do the same or similar things,
> >>> initrd could stay open if the other one also does not care.
> >>>
> >>> Henning
> >>>
> >>>> +}
> >>>> +
> >>>> +# build the binary kernel module and package as debian package
> >>>> (versioned) +do_module_build() {
> >>>> +    # we have to find out the module version, e.g.
> >>>> dpdk-kmods/0~20201113+git -k 5.10.0-3-rt-amd64/x86_64
> >>>> +    REVISION=$(find
> >>>> ${BUILDCHROOT_DIR}/usr/src/${DKMS_PACKAGE_NAME}-* -type d -exec
> >>>> basename {} + | sed 's/${DKMS_PACKAGE_NAME}-//g')
> >>>> +    if ! dpkg -s --root=${BUILDCHROOT_DIR}
> >>>> linux-headers-${KERNEL_NAME} | grep "Depends:.*linux-headers"; then
> >>>> +        # custom kernels directly place their files in
> >>>> linux-image-KERNEL-NAME, instead
> >>>> +        # of using a meta package + a versioned package with the
> >>>> resources
> >>>> +        # The prebuild DKMS binary package depends on the
> >>>> versioned kernel package,
> >>>> +        # but that is not available on custom kernels. Hence, we
> >>>> just remove the dependency.
> >>>> +        bbnote "Building ${DKMS_PACKAGE_NAME} for custom kernel
> >>>> ${KERNEL_NAME}"
> >>>> +        cp
> >>>> ${BUILDCHROOT_DIR}/etc/dkms/template-dkms-
> mkbmdeb/debian/control
> >>>> ${WORKDIR}/control.dkms.orig
> >>>> +        sudo sed -i 's/, linux-image-KERNEL_VERSION//'
> >>>> ${BUILDCHROOT_DIR}/etc/dkms/template-dkms-
> mkbmdeb/debian/control
> >>>> +    fi
> >>>> +    # build the module for all installed kernels (should be just
> >>>> one)
> >>>> +    sudo -E chroot ${BUILDCHROOT_DIR} dkms mkbmdeb
> >>>> ${DKMS_PACKAGE_NAME}/${REVISION} --all
> >>>> +    DKMS_PACKAGE_VERSIONED=$(find
> >>>> ${BUILDCHROOT_DIR}/var/lib/dkms/${DKMS_PACKAGE_NAME}/ -name
> >>>> "${DKMS_PACKAGE_NAME}-modules*.deb" -exec dpkg -I {} + | grep
> >>>> "Package:" | awk '{print $2}')
> >>>> +    if [ -z "$DKMS_PACKAGE_VERSIONED" ]; then
> >>>> +        bberror "No prebuild dkms module found"
> >>>> +        exit 1
> >>>> +    fi
> >>>> +    # restore dkms template (if any)
> >>>> +    cp ${WORKDIR}/control.dkms.orig
> >>>> ${BUILDCHROOT_DIR}/etc/dkms/template-dkms-
> mkbmdeb/debian/control ||
> >>>> true +} +
> >>>> +# simply copy our module from the build tree to the expected
> >>>> output location +# of this recipe. Then, do_deploy_dep finds it and
> >>>> adds it to the +# debian isar repo
> >>>> +do_module_deploy() {
> >>>> +    cp
> >>>>
> ${BUILDCHROOT_DIR}/var/lib/dkms/${DKMS_PACKAGE_NAME}/*/bmdeb/*.de
> b
> >>>> ${S}/../ +} +
> >>>> +addtask module_build after do_install_builddeps before
> >>>> do_dpkg_build +addtask module_deploy after do_module_build before
> >>>> do_deploy_deb
> >>>
> >>
> >
> 
> --
> Siemens AG, T RDA IOT
> Corporate Competence Center Embedded Linux