* [PATCH 0/3] linux-custom: Split up binaries from kernel headers to kbuild packages
@ 2022-11-09 10:32 Koch, Stefan
2022-11-09 10:32 ` [PATCH 1/3] sbuild: Support overwriting configured schroot dir Koch, Stefan
` (4 more replies)
0 siblings, 5 replies; 18+ messages in thread
From: Koch, Stefan @ 2022-11-09 10:32 UTC (permalink / raw)
To: isar-users
Cc: Kiszka, Jan, Storm, Christian, Adler, Michael, Sudler, Simon,
Koch, Stefan
Hi
This patchset is a set of three patches:
- Support overwriting configured schroot dir
- Split up binaries from kernel headers to kbuild packages
- Update custom_kernel docs for split up of kernel scripts and tools
The main use-case was to swap out the binaries
from the kernel headers package.
They will be swapped out into host and target
specific kernel kbuild packages.
The main development goals were these:
1. Solve already known isar custom kernel
limitations from doc/custom_kernel.inc
- kernel headers package does not support both native
and cross compilation of kernel modules when cross built
2. Honor recommendations for future from doc/custom_kernel.inc
- Generate kernel headers packages for both host and target
when using cross build
3. Add extensions known from debian kernel packages structure
- Generate a kernel headers package without binaries
- Create specific kernel kbuild packages that
will ship the "scripts" and "tools" binaries
- Use symlinks to point to the "scripts" and "tools" binaries
4. Be user friendly
- Avoid usage of separate kbuild bitbake recipe that may enforce
redundant configuration of kernel source definitions with user
actions to enable kbuild package generation
- Use already known way to include linux-custom.inc in just one
own bitbake recipe that provides the kernel source definitions
- Keep known user behavior for existing build configurations:
just update isar, and kbuild packages will be created automatically
Best regards
Stefan
Stefan Koch (3):
sbuild: Support overwriting configured schroot dir
linux-custom: Split up binaries from kernel headers to kbuild packages
docs: Update custom_kernel docs for split up of kernel scripts and
tools
doc/custom_kernel.md | 13 +--
meta/classes/sbuild.bbclass | 9 +-
.../linux/files/debian/control.tmpl | 18 ++-
.../linux/files/debian/isar/build.tmpl | 9 +-
.../linux/files/debian/isar/common.tmpl | 7 +-
.../linux/files/debian/isar/install.tmpl | 64 +++++++++--
.../linux/files/debian/rules.tmpl | 5 +-
meta/recipes-kernel/linux/linux-custom.inc | 104 ++++++++++++++++--
8 files changed, 191 insertions(+), 38 deletions(-)
--
2.30.2
^ permalink raw reply [flat|nested] 18+ messages in thread
* [PATCH 1/3] sbuild: Support overwriting configured schroot dir
2022-11-09 10:32 [PATCH 0/3] linux-custom: Split up binaries from kernel headers to kbuild packages Koch, Stefan
@ 2022-11-09 10:32 ` Koch, Stefan
2022-11-09 10:32 ` [PATCH 2/3] linux-custom: Split up binaries from kernel headers to kbuild packages Koch, Stefan
` (3 subsequent siblings)
4 siblings, 0 replies; 18+ messages in thread
From: Koch, Stefan @ 2022-11-09 10:32 UTC (permalink / raw)
To: isar-users
Cc: Kiszka, Jan, Storm, Christian, Adler, Michael, Sudler, Simon,
Koch, Stefan
This brings support to specify an other than the default schroot dir
as argument when creating the schroot configs.
Signed-off-by: Stefan Koch <stefan-koch@siemens.com>
---
meta/classes/sbuild.bbclass | 9 +++++++--
1 file changed, 7 insertions(+), 2 deletions(-)
diff --git a/meta/classes/sbuild.bbclass b/meta/classes/sbuild.bbclass
index a29b745..ad9f72f 100644
--- a/meta/classes/sbuild.bbclass
+++ b/meta/classes/sbuild.bbclass
@@ -31,14 +31,19 @@ SCHROOT_CONF_FILE ?= "${SCHROOT_CONF}/chroot.d/${SBUILD_CHROOT}"
SBUILD_CONFIG="${WORKDIR}/sbuild.conf"
schroot_create_configs() {
+ schroot_dir="${SCHROOT_DIR}"
+ if [ -n "${1}" ]; then
+ schroot_dir="${1}"
+ fi
+
mkdir -p "${TMPDIR}/schroot-overlay"
- sudo -s <<'EOSUDO'
+ schroot_dir="${schroot_dir}" sudo --preserve-env=schroot_dir -s <<'EOSUDO'
set -e
cat << EOF > "${SCHROOT_CONF_FILE}"
[${SBUILD_CHROOT}]
type=directory
-directory=${SCHROOT_DIR}
+directory=${schroot_dir}
profile=${SBUILD_CHROOT}
users=${SCHROOT_USER}
groups=root,sbuild
--
2.30.2
^ permalink raw reply [flat|nested] 18+ messages in thread
* [PATCH 2/3] linux-custom: Split up binaries from kernel headers to kbuild packages
2022-11-09 10:32 [PATCH 0/3] linux-custom: Split up binaries from kernel headers to kbuild packages Koch, Stefan
2022-11-09 10:32 ` [PATCH 1/3] sbuild: Support overwriting configured schroot dir Koch, Stefan
@ 2022-11-09 10:32 ` Koch, Stefan
2022-11-11 5:34 ` Uladzimir Bely
2022-11-09 10:32 ` [PATCH 3/3] docs: Update custom_kernel docs for split up of kernel scripts and tools Koch, Stefan
` (2 subsequent siblings)
4 siblings, 1 reply; 18+ messages in thread
From: Koch, Stefan @ 2022-11-09 10:32 UTC (permalink / raw)
To: isar-users
Cc: Kiszka, Jan, Storm, Christian, Adler, Michael, Sudler, Simon,
Koch, Stefan
When using a cross build this patch does introduce
host and target specific kernel kbuild packages that
ship the "scripts" and "tools" binaries.
The kernel headers fulfill this using symlinks to point
to the "scripts" and "tools" of the respective kernel kbuild package.
Known from doc/custom_kernel.inc:
- The kernel headers package has not supported both native
and cross compilation of kernel modules when itself was cross built
- Future roadmap: Generate kernel headers package for both host
and target when using a cross build
Known from debian kernel packages structure:
- Generate a kernel headers package without binaries
- Create specific kernel kbuild packages that
ship the "scripts" and "tools" binaries
- Using symlinks to point to the "scripts"
and "tools" binaries
Signed-off-by: Stefan Koch <stefan-koch@siemens.com>
---
.../linux/files/debian/control.tmpl | 18 ++-
.../linux/files/debian/isar/build.tmpl | 9 +-
.../linux/files/debian/isar/common.tmpl | 7 +-
.../linux/files/debian/isar/install.tmpl | 64 +++++++++--
.../linux/files/debian/rules.tmpl | 5 +-
meta/recipes-kernel/linux/linux-custom.inc | 104 ++++++++++++++++--
6 files changed, 178 insertions(+), 29 deletions(-)
diff --git a/meta/recipes-kernel/linux/files/debian/control.tmpl b/meta/recipes-kernel/linux/files/debian/control.tmpl
index dd0b624..adac4ef 100644
--- a/meta/recipes-kernel/linux/files/debian/control.tmpl
+++ b/meta/recipes-kernel/linux/files/debian/control.tmpl
@@ -26,7 +26,7 @@ Section: devel
Provides: linux-kernel-headers
Architecture: any
Description: Linux support headers for userspace development
- This package provides userspaces headers from the Linux kernel. These headers
+ This package provides userspaces headers from the Linux kernel. These headers
are used by the installed headers for GNU glibc and other system libraries.
Package: linux-image-${KERNEL_NAME_PROVIDED}-dbg
@@ -35,3 +35,19 @@ Architecture: any
Description: Linux kernel debugging symbols for @KR@
This package will come in handy if you need to debug the kernel. It provides
all the necessary debug symbols for the kernel and its modules.
+
+Package: linux-kbuild-${KERNEL_NAME_PROVIDED}
+Architecture: any
+Depends: ${KERNEL_HEADERS_DEBIAN_DEPENDS}, ${perl:Depends}, ${shlib:Depends}
+Description: ${KERNEL_NAME_PROVIDED} Linux kbuild scripts and tools for @KR@
+ This package provides kernel kbuild scripts and tools for @KR@
+ This is useful for people who need to build external modules
+
+Package: linux-kbuild-${KERNEL_NAME_PROVIDED}-cross
+Architecture: ${CONTROL_TARGET_ARCH}
+Depends: ${KERNEL_HEADERS_DEBIAN_DEPENDS}, ${perl:Depends}, ${shlib:Depends}
+Description: ${KERNEL_NAME_PROVIDED} Linux kbuild scripts and tools for @KR@
+ This package provides kernel kbuild scripts and tools
+ as ${HOST_ARCH} cross binaries for @KR@
+ This is useful for those who need to cross build
+ external modules using ISAR's sbuild-chroot-host
diff --git a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl
index 94cfbe0..7e2daad 100644
--- a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl
+++ b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl
@@ -13,16 +13,19 @@ do_build() {
# Trace what we do here
set -x
+ # Set correct build directory for kernel or kbuild (kernel scripts and tools) cases
+ build_dir=${KERNEL_BUILD_DIR}
+
# Process existing kernel configuration to make sure it is complete
# (use defaults for options that were not specified)
- ${MAKE} O=${KERNEL_BUILD_DIR} olddefconfig prepare
+ ${MAKE} O=${build_dir} olddefconfig prepare
# Transfer effective kernel version into control file and scripts
- KR=$(${MAKE} O=${KERNEL_BUILD_DIR} -s --no-print-directory kernelrelease)
+ KR=$(${MAKE} O=${build_dir} -s --no-print-directory kernelrelease)
sed -i "s/@KR@/${KR}/g" ${S}/debian/control ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.*
# Build the Linux kernel
- ${MAKE} O=${KERNEL_BUILD_DIR} ${PARALLEL_MAKE} KCFLAGS="${KCFLAGS}"
+ ${MAKE} O=${build_dir} ${PARALLEL_MAKE} KCFLAGS="${KCFLAGS}"
# Stop tracing
set +x
diff --git a/meta/recipes-kernel/linux/files/debian/isar/common.tmpl b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl
index 52ebebb..55d6123 100644
--- a/meta/recipes-kernel/linux/files/debian/isar/common.tmpl
+++ b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl
@@ -5,10 +5,14 @@
set -e
# Isar settings
-ARCH=${KERNEL_ARCH}
KERNEL_PKG_IMAGE=linux-image-${KERNEL_NAME_PROVIDED}
KERNEL_PKG_KERN_HEADERS=linux-headers-${KERNEL_NAME_PROVIDED}
KERNEL_PKG_LIBC_HEADERS=linux-libc-dev
+KERNEL_PKG_KERN_KBUILD=linux-kbuild-${KERNEL_NAME_PROVIDED}
+
+if [ -z "${ARCH}" ]; then
+ ARCH=${KERNEL_ARCH}
+fi
# Constants
KCONF=.config
@@ -19,6 +23,7 @@ deb_img_dir=${deb_top_dir}/${KERNEL_PKG_IMAGE}
deb_dbg_dir=${deb_img_dir}-dbg
deb_kern_hdr_dir=${deb_top_dir}/${KERNEL_PKG_KERN_HEADERS}
deb_libc_hdr_dir=${deb_top_dir}/${KERNEL_PKG_LIBC_HEADERS}
+deb_kern_kbuild_dir=${deb_top_dir}/${KERNEL_PKG_KERN_KBUILD}
# Array of packages to be generated
declare -A kern_pkgs
diff --git a/meta/recipes-kernel/linux/files/debian/isar/install.tmpl b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl
index 0a8645d..890a996 100644
--- a/meta/recipes-kernel/linux/files/debian/isar/install.tmpl
+++ b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl
@@ -33,6 +33,7 @@ do_install() {
# Trace what we do here
set -x
+ # Run the install steps
install_image
if [ "${ARCH}" != "um" ]; then
install_config
@@ -43,6 +44,19 @@ do_install() {
install_kmods
install_headers
+ if [ -d "$(dirname ${O})/build-full-kbuild-target" ]; then
+ # Install cross kernel scripts and tools
+ install_kbuild ${deb_kern_kbuild_dir}-cross build-full
+
+ # Cleanup and install kernel scripts and tools for target arch
+ rm -rf ${deb_kern_kbuild_dir}
+ install_kbuild ${deb_kern_kbuild_dir} build-full-kbuild-target
+ else
+ # Cleanup and install kernel scripts and tools
+ rm -rf ${deb_kern_kbuild_dir}
+ install_kbuild ${deb_kern_kbuild_dir} build-full
+ fi
+
# Stop tracing
set +x
}
@@ -168,21 +182,15 @@ kernel_headers() {
mkdir -p ${destdir}
mkdir -p ${deb_kern_hdr_dir}/lib/modules/${krel}
- (cd ${S}; find . -name 'Makefile*' -o -name 'Kconfig*' -o -name '*.pl') >>${src_hdr_files}
- (cd ${S}; find arch/*/include include scripts -type f -o -type l) >>${src_hdr_files}
+ (cd ${S}; find . -not -path './scripts/*' -a -not -path './tools/*' -a \( -name 'Makefile*' -o -name 'Kconfig*' -o -name '*.pl' \)) >>${src_hdr_files}
+ (cd ${S}; find arch/*/include include -type f -o -type l) >>${src_hdr_files}
(cd ${S}; find arch/${ARCH} -name module.lds -o -name Kbuild.platforms -o -name Platform) >>${src_hdr_files}
(cd ${S}; find $(find arch/${ARCH} -name include -o -name scripts -type d) -type f) >>${src_hdr_files}
if [ -n "${CONFIG_MODULES}" ]; then
echo Module.symvers >> ${obj_hdr_files}
fi
- (cd ${O}; find arch/${ARCH}/include include scripts -type f) >>${obj_hdr_files}
- if [ -n "${CONFIG_STACK_VALIDATION}" ]; then
- (cd ${O}; find tools/objtool -type f -executable) >>${obj_hdr_files}
- fi
- if [ -n "${CONFIG_GCC_PLUGINS}" ]; then
- (cd ${O}; find scripts/gcc-plugins -name *.so -o -name gcc-common.h) >>${obj_hdr_files}
- fi
+ (cd ${O}; find arch/${ARCH}/include include -type f) >>${obj_hdr_files}
# deploy files that were matched above
tar -C ${S} -cf - -T - <${src_hdr_files} | tar -C ${destdir} -xf -
@@ -191,8 +199,11 @@ kernel_headers() {
# add the kernel config
cp ${O}/${KCONF} ${destdir}/.config
- # handle kernel development tools
- kernel_tools
+ # add symlink to scripts and tools directories
+ ln -sf ../../lib/linux-kbuild-${krel}/scripts ${destdir}/scripts
+ if [ -n "${CONFIG_STACK_VALIDATION}" ]; then
+ ln -sf ../../lib/linux-kbuild-${krel}/tools ${destdir}/tools
+ fi
# create symlinks
ln -sf /${kernel_headers_dir} ${deb_kern_hdr_dir}/lib/modules/${krel}/build
@@ -206,4 +217,35 @@ install_headers() {
kernel_headers
}
+install_kbuild() {
+ kernel_kbuild_dir=usr/lib/linux-kbuild-${krel}
+ destdir=${1}/${kernel_kbuild_dir}
+ src_kbuild_files=$(mktemp)
+ obj_kbuild_files=$(mktemp)
+
+ if [ -n "${2}" ]; then
+ O=$(dirname ${O})/${2}
+ fi
+
+ mkdir -p ${destdir}
+
+ (cd ${S}; find . -path './scripts/*' -a -path './tools/*' -a \( -name 'Makefile*' -o -name 'Kconfig*' -o -name '*.pl' \)) >>${src_kbuild_files}
+ (cd ${S}; find scripts -type f -o -type l) >>${src_kbuild_files}
+
+ (cd ${O}; find scripts -type f) >>${obj_kbuild_files}
+ if [ -n "${CONFIG_STACK_VALIDATION}" ]; then
+ (cd ${O}; find tools/objtool -type f -executable) >>${obj_kbuild_files}
+ fi
+ if [ -n "${CONFIG_GCC_PLUGINS}" ]; then
+ (cd ${O}; find scripts/gcc-plugins -name *.so -o -name gcc-common.h) >>${obj_kbuild_files}
+ fi
+
+ # deploy files that were matched above
+ tar -C ${S} -cf - -T - <${src_kbuild_files} | tar -C ${destdir} -xf -
+ tar -C ${O} -cf - -T - <${obj_kbuild_files} | tar -C ${destdir} -xf -
+
+ # handle kernel development tools
+ kernel_tools
+}
+
main install ${*}
diff --git a/meta/recipes-kernel/linux/files/debian/rules.tmpl b/meta/recipes-kernel/linux/files/debian/rules.tmpl
index 8063c49..e131288 100755
--- a/meta/recipes-kernel/linux/files/debian/rules.tmpl
+++ b/meta/recipes-kernel/linux/files/debian/rules.tmpl
@@ -36,4 +36,7 @@ override_dh_auto_test:
true
override_dh_strip:
- unset DEB_HOST_GNU_TYPE && dh_strip -Xvmlinu --no-automatic-dbgsym
+ dh_strip -Xvmlinu -Xlinux-kbuild-${KERNEL_NAME_PROVIDED}-cross --no-automatic-dbgsym
+
+override_dh_shlibdeps:
+ dh_shlibdeps -Xlinux-kbuild-${KERNEL_NAME_PROVIDED}
diff --git a/meta/recipes-kernel/linux/linux-custom.inc b/meta/recipes-kernel/linux/linux-custom.inc
index 96f0afc..0f8ba50 100644
--- a/meta/recipes-kernel/linux/linux-custom.inc
+++ b/meta/recipes-kernel/linux/linux-custom.inc
@@ -74,12 +74,14 @@ TEMPLATE_VARS += " \
KERNEL_ARCH \
KERNEL_DEBIAN_DEPENDS \
KERNEL_BUILD_DIR \
+ KERNEL_KBUILD_DIR \
KERNEL_FILE \
KERNEL_HEADERS_DEBIAN_DEPENDS \
LINUX_VERSION_EXTENSION \
KERNEL_NAME_PROVIDED \
KERNEL_CONFIG_FRAGMENTS \
KCFLAGS \
+ CONTROL_TARGET_ARCH \
"
inherit dpkg
@@ -91,30 +93,54 @@ KCFLAGS ?= "-fdebug-prefix-map=${CURDIR}=."
# Derive name of the kernel packages from the name of this recipe
KERNEL_NAME_PROVIDED ?= "${@ d.getVar('PN', True).partition('linux-')[2]}"
-# Make bitbake know we will be producing linux-image and linux-headers packages
python() {
kernel_name = d.getVar("KERNEL_NAME_PROVIDED", True)
- d.setVar('PROVIDES', 'linux-image-' + kernel_name + ' ' + \
- 'linux-headers-' + kernel_name)
+ distro_arch = d.getVar("DISTRO_ARCH", True)
+ host_arch = d.getVar("HOST_ARCH", True)
+ target_arch = ""
+ kbuild_depends = "linux-kbuild-" + kernel_name
+
+ # For different distro and host archs
+ # - Set target arch (empty string disables package build)
+ # - Add dependency for sbuild-chroot-target
+ # to allow building arch specific kbuild scripts and tools
+ # - Set correct name of kbuild package dependency
+ if distro_arch != host_arch:
+ d.appendVar("SCHROOT_DEP", " sbuild-chroot-target:do_build")
+ if d.getVar("ISAR_CROSS_COMPILE", True) == "1":
+ target_arch = distro_arch
+ kbuild_depends = kbuild_depends + "-cross | " + kbuild_depends
+
+ # Set CONTROL_TARGET_ARCH
+ d.setVar("CONTROL_TARGET_ARCH", target_arch)
+
+ # Make bitbake know we will be producing
+ # linux-image and linux-headers packages
+ d.setVar("PROVIDES", "linux-image-" + kernel_name + " " + \
+ "linux-headers-" + kernel_name)
+
+ # Set dependency for kbuild package
+ d.appendVar("KERNEL_HEADERS_DEBIAN_DEPENDS", kbuild_depends)
}
-def get_kernel_arch(d):
- distro_arch = d.getVar("DISTRO_ARCH")
- if distro_arch in ["amd64", "i386"]:
+def get_kernel_arch(d, arch="DISTRO_ARCH"):
+ arch = d.getVar(arch)
+ if arch in ["amd64", "i386"]:
kernel_arch = "x86"
- elif distro_arch == "arm64":
+ elif arch == "arm64":
kernel_arch = "arm64"
- elif distro_arch == "armhf":
+ elif arch == "armhf":
kernel_arch = "arm"
- elif distro_arch == "mipsel":
+ elif arch == "mipsel":
kernel_arch = "mips"
- elif distro_arch == "riscv64":
+ elif arch == "riscv64":
kernel_arch = "riscv"
else:
kernel_arch = ""
return kernel_arch
KERNEL_ARCH ??= "${@get_kernel_arch(d)}"
+KERNEL_HOST_ARCH ??= "${@get_kernel_arch(d, 'HOST_ARCH')}"
def config_fragments(d):
fragments = []
@@ -157,6 +183,7 @@ do_prepare_build_prepend() {
# build directory for our "full" kernel build
KERNEL_BUILD_DIR = "build-full"
+KERNEL_KBUILD_DIR = "${KERNEL_BUILD_DIR}-kbuild"
def get_kernel_config_target(d):
kernel_defconfig = d.getVar('KERNEL_DEFCONFIG', True)
@@ -187,15 +214,19 @@ def get_kernel_config_fragments(d):
KERNEL_CONFIG_FRAGMENTS = "${@get_kernel_config_fragments(d)}"
dpkg_configure_kernel() {
+ build_dir="${KERNEL_BUILD_DIR}"
+ if [ -n "${1}" ]; then
+ build_dir="${1}"
+ fi
grep -q "KERNEL_CONFIG_TARGET=" ${S}/debian/isar/configure ||
cat << EOF | sed -i '/^do_configure() {/ r /dev/stdin' ${S}/debian/isar/configure
KERNEL_CONFIG_TARGET="${@get_kernel_config_target(d)}"
EOF
- rm -rf ${S}/${KERNEL_BUILD_DIR} && mkdir -p ${S}/${KERNEL_BUILD_DIR}
+ rm -rf ${S}/${build_dir} && mkdir -p ${S}/${build_dir}
if [ -n "${KERNEL_DEFCONFIG}" ]; then
if [ -e "${WORKDIR}/${KERNEL_DEFCONFIG}" ]; then
- cp ${WORKDIR}/${KERNEL_DEFCONFIG} ${S}/${KERNEL_BUILD_DIR}/.config
+ cp ${WORKDIR}/${KERNEL_DEFCONFIG} ${S}/${build_dir}/.config
fi
fi
@@ -210,4 +241,53 @@ EOF
dpkg_runbuild_prepend() {
dpkg_configure_kernel
+
+ if [ "${ISAR_CROSS_COMPILE}" = "1" ] && [ "${HOST_ARCH}" != "${DISTRO_ARCH}" ]; then
+ # Create temporarily kernel schroot-target configuration
+ schroot_create_configs ${SCHROOT_TARGET_DIR}
+
+ # Begin kernel schroot-target session
+ schroot_session=$(schroot -b -c ${SBUILD_CHROOT})
+
+ # Install build deps for schroot session
+ schroot -u root -r -c $schroot_session -- sh -c " \
+ cd ${PP}/${PPS}; \
+ mk-build-deps -i -r --host-arch ${DISTRO_ARCH} -t \"apt-get -y --no-install-recommends --allow-downgrades -o Debug::pkgProblemResolver=yes\" debian/control; \
+ "
+
+ # Create kernel configuration for kbuild
+ dpkg_configure_kernel ${KERNEL_KBUILD_DIR}-target
+ KERNEL_CONFIG_TARGET="${@get_kernel_config_target(d)}"
+
+ schroot -r -c $schroot_session -- sh -c " \
+ cd ${PP}/${PPS}; \
+ ARCH=${KERNEL_ARCH} make O=${KERNEL_KBUILD_DIR}-target ${KERNEL_CONFIG_TARGET}; \
+ "
+
+ cd ${S}
+ ARCH=${KERNEL_ARCH} ./scripts/kconfig/merge_config.sh -m -O ${KERNEL_KBUILD_DIR}-target/ ${KERNEL_KBUILD_DIR}-target/.config ${KERNEL_CONFIG_FRAGMENTS}
+ cd -
+
+ # Build kernel scripts and tools for target arch in a non-cross way
+ # Cross building kernel scripts and tools would not provide them in target architecture
+ schroot -r -c $schroot_session -- sh -c " \
+ cd ${PP}/${PPS}; \
+ ARCH=${KERNEL_ARCH} make O=${KERNEL_KBUILD_DIR}-target olddefconfig; \
+ ARCH=${KERNEL_ARCH} make O=${KERNEL_KBUILD_DIR}-target ${PARALLEL_MAKE} scripts; \
+ if [ -d "tools/objtool" ]; then \
+ echo ARCH=${KERNEL_ARCH} make O=${KERNEL_KBUILD_DIR}-target ${PARALLEL_MAKE} tools/objtool; \
+ ARCH=${KERNEL_ARCH} make O=${KERNEL_KBUILD_DIR}-target ${PARALLEL_MAKE} tools/objtool; \
+ fi; \
+ if grep -q "CONFIG_MODULES=y" ${KERNEL_KBUILD_DIR}-target/.config; then \
+ echo ARCH=${KERNEL_ARCH} make O=${KERNEL_KBUILD_DIR}-target ${PARALLEL_MAKE} modules_prepare; \
+ ARCH=${KERNEL_ARCH} make O=${KERNEL_KBUILD_DIR}-target ${PARALLEL_MAKE} modules_prepare; \
+ fi; \
+ "
+
+ # End kernel schroot-target session
+ schroot -e -c $schroot_session
+
+ # Restore kernel schroot-host configuration
+ schroot_create_configs
+ fi
}
--
2.30.2
^ permalink raw reply [flat|nested] 18+ messages in thread
* [PATCH 3/3] docs: Update custom_kernel docs for split up of kernel scripts and tools
2022-11-09 10:32 [PATCH 0/3] linux-custom: Split up binaries from kernel headers to kbuild packages Koch, Stefan
2022-11-09 10:32 ` [PATCH 1/3] sbuild: Support overwriting configured schroot dir Koch, Stefan
2022-11-09 10:32 ` [PATCH 2/3] linux-custom: Split up binaries from kernel headers to kbuild packages Koch, Stefan
@ 2022-11-09 10:32 ` Koch, Stefan
2022-11-09 11:50 ` [PATCH 0/3] linux-custom: Split up binaries from kernel headers to kbuild packages Jan Kiszka
2022-11-15 13:44 ` Uladzimir Bely
4 siblings, 0 replies; 18+ messages in thread
From: Koch, Stefan @ 2022-11-09 10:32 UTC (permalink / raw)
To: isar-users
Cc: Kiszka, Jan, Storm, Christian, Adler, Michael, Sudler, Simon,
Koch, Stefan
Signed-off-by: Stefan Koch <stefan-koch@siemens.com>
---
doc/custom_kernel.md | 13 ++++++-------
1 file changed, 6 insertions(+), 7 deletions(-)
diff --git a/doc/custom_kernel.md b/doc/custom_kernel.md
index ffeaddc..083c143 100644
--- a/doc/custom_kernel.md
+++ b/doc/custom_kernel.md
@@ -38,12 +38,13 @@ The linux-custom recipe provides support for:
7. Allow the name of the kernel image to be changed via `KERNEL_FILE` (defaults
to `vmlinuz`)
- 8. Produce a `linux-headers` package which includes kernel headers and kbuild
- scripts/tools
+ 8. Produce a `linux-headers` package which includes kernel headers
- 9. The `linux-headers` package shall support native and cross compiles of
- out-of-tree kernel modules. However, when built in cross-compilation mode,
- it cannot be used on the target so far.
+ 9. Produce a `linux-kbuild` package for both host and target
+ which includes kbuild scripts/tools.
+ So the `linux-headers` package supports native and cross compiles of
+ out-of-tree kernel modules. Even, when built in cross-compilation mode,
+ it can be used on the target using the `linux-kbuild` package.
10. Produce a `linux-libc-dev` package to support user-land builds
@@ -71,8 +72,6 @@ In the future, the recipe may be extended to:
3. Be compatible with Ubuntu
- 4. When cross-building, generate kernel-headers for both host and target
-
## Examples
The linux-custom recipe is currently used by the linux-mainline package and is
--
2.30.2
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 0/3] linux-custom: Split up binaries from kernel headers to kbuild packages
2022-11-09 10:32 [PATCH 0/3] linux-custom: Split up binaries from kernel headers to kbuild packages Koch, Stefan
` (2 preceding siblings ...)
2022-11-09 10:32 ` [PATCH 3/3] docs: Update custom_kernel docs for split up of kernel scripts and tools Koch, Stefan
@ 2022-11-09 11:50 ` Jan Kiszka
2022-11-09 15:06 ` Koch, Stefan
2022-11-15 13:44 ` Uladzimir Bely
4 siblings, 1 reply; 18+ messages in thread
From: Jan Kiszka @ 2022-11-09 11:50 UTC (permalink / raw)
To: Koch, Stefan (DI PA DCP R&D 3), isar-users
Cc: Storm, Christian (T CED SES-DE), Adler, Michael (T CED SES-DE),
Sudler, Simon (DI PA DCP TI)
On 09.11.22 11:32, Koch, Stefan (DI PA DCP R&D 3) wrote:
> Hi
>
> This patchset is a set of three patches:
> - Support overwriting configured schroot dir
> - Split up binaries from kernel headers to kbuild packages
> - Update custom_kernel docs for split up of kernel scripts and tools
>
> The main use-case was to swap out the binaries
> from the kernel headers package.
> They will be swapped out into host and target
> specific kernel kbuild packages.
>
> The main development goals were these:
>
> 1. Solve already known isar custom kernel
> limitations from doc/custom_kernel.inc
> - kernel headers package does not support both native
> and cross compilation of kernel modules when cross built
Right, that requires building packages for multiple architectures. This
is the feature we need, and it has nothing to do with the kernel
packages per se.
Jan
>
> 2. Honor recommendations for future from doc/custom_kernel.inc
> - Generate kernel headers packages for both host and target
> when using cross build
>
> 3. Add extensions known from debian kernel packages structure
> - Generate a kernel headers package without binaries
> - Create specific kernel kbuild packages that
> will ship the "scripts" and "tools" binaries
> - Use symlinks to point to the "scripts" and "tools" binaries
Keep in mind that we intend to remain compatible with the layout of
Debian. Self-built kernels should remain drop-in replacements of the
Debian kernel packages. And that is true for development packages as well.
So, are your changes working against that, or are they improving
existing incompatibility with the debian packages?
--
Siemens AG, Technology
Competence Center Embedded Linux
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 0/3] linux-custom: Split up binaries from kernel headers to kbuild packages
2022-11-09 11:50 ` [PATCH 0/3] linux-custom: Split up binaries from kernel headers to kbuild packages Jan Kiszka
@ 2022-11-09 15:06 ` Koch, Stefan
2022-11-09 16:00 ` Jan Kiszka
0 siblings, 1 reply; 18+ messages in thread
From: Koch, Stefan @ 2022-11-09 15:06 UTC (permalink / raw)
To: isar-users, Kiszka, Jan; +Cc: Sudler, Simon, Storm, Christian, Adler, Michael
On Wed, 2022-11-09 at 12:50 +0100, Jan Kiszka wrote:
> On 09.11.22 11:32, Koch, Stefan (DI PA DCP R&D 3) wrote:
> > Hi
> >
> > This patchset is a set of three patches:
> > - Support overwriting configured schroot dir
> > - Split up binaries from kernel headers to kbuild packages
> > - Update custom_kernel docs for split up of kernel scripts and
> > tools
> >
> > The main use-case was to swap out the binaries
> > from the kernel headers package.
> > They will be swapped out into host and target
> > specific kernel kbuild packages.
> >
> > The main development goals were these:
> >
> > 1. Solve already known isar custom kernel
> > limitations from doc/custom_kernel.inc
> > - kernel headers package does not support both native
> > and cross compilation of kernel modules when cross built
>
> Right, that requires building packages for multiple architectures.
Important feature... Maybe it should be added to the TODO list in the
docs?
> This
> is the feature we need, and it has nothing to do with the kernel
> packages per se.
The way that I assume debian will do building the kernel is that they
use a generic debian defconfig for each architecture.
E. g. linux-config-5.10/config.amd64_none_amd64.xz, linux-config-
5.10/config.arm64_none_arm64.xz
ISAR builds are often using specific defconfigs honored by ISAR's
linux-custom for the target build.
Debian specific architecture builds (e. g. host and target, maybe with
sbuild) for one target device, using target specific stripped-down
defconfigs, may not working properly. The kernel build for a foreign
architecture (e. g. host) may fail because of an incompatible
defconfig.
Debian provide a generic very similar defconfig for different
architectures, and they build the linux-image, linux-headers and linux-
kbuild for each architecture:
- So I assume the linux-kbuild:amd64 was build with linux-config-
5.10/config.amd64_none_amd64.xz
- So I assume the linux-kbuild:arm64 was build with linux-config-
5.10/config.arm64_none_arm64.xz
The debian generic defconfig may differ in some arch specific points
but you can use the linux-kbuild:amd64 for a cross-build of linux-
image:arm64 because the resulting scripts and tools are from the
identical set. So mixing linux-headers with different archictecture
specific builds from linux-kbuild should work since the defconfigs have
a "shared base" (e. g. both define CONFIG_HAVE_C_RECORDMCOUNT thats
needed for recordmcount utility as example).
> Jan
>
> >
> > 2. Honor recommendations for future from doc/custom_kernel.inc
> > - Generate kernel headers packages for both host and target
> > when using cross build
> >
> > 3. Add extensions known from debian kernel packages structure
> > - Generate a kernel headers package without binaries
> > - Create specific kernel kbuild packages that
> > will ship the "scripts" and "tools" binaries
> > - Use symlinks to point to the "scripts" and "tools" binaries
>
> Keep in mind that we intend to remain compatible with the layout of
> Debian. Self-built kernels should remain drop-in replacements of the
> Debian kernel packages. And that is true for development packages as
> well.
>
> So, are your changes working against that, or are they improving
> existing incompatibility with the debian packages?
The approach from this patchset does build the kernel in the known ISAR
specific way. Debian itself patches the kernel Makefiles to modularize
the kernel build what ISAR doesn't do. With this patch the packages
linux-kbuild and linux-kbuild-cross are the only new packages that the
ISAR build will provide. That means ISAR will provide then linux-
headers, linux-image and linux-kbuild. They match on the target with
the package name of the respective debian packages. So the existing
drop-in should not be affected.
In short this patch brings the ISAR linux-custom mechanism a bit nearer
to the debian package structure. Since linux-headers behaves then more
like the original debian linux-heades to ship the binaries within a
separate package -> linux-kbuild.
The trick is to build only the scripts and tools with schroot-target
(QEMU) in a non-cross way. Since the kernel does not support building
the scripts and tools for target architecture when do a cross build
without patching the kernel Makefiles. I think that patch was never
merged into the kernel:
https://patchwork.kernel.org/project/linux-kbuild/patch/1376046432-12588-1-git-send-email-broonie@kernel.org/
--
Stefan Koch
Siemens AG
www.siemens.com
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 0/3] linux-custom: Split up binaries from kernel headers to kbuild packages
2022-11-09 15:06 ` Koch, Stefan
@ 2022-11-09 16:00 ` Jan Kiszka
2022-11-10 17:49 ` Koch, Stefan
0 siblings, 1 reply; 18+ messages in thread
From: Jan Kiszka @ 2022-11-09 16:00 UTC (permalink / raw)
To: Koch, Stefan (DI PA DCP R&D 3), isar-users
Cc: Sudler, Simon (DI PA DCP TI), Storm, Christian (T CED SES-DE),
Adler, Michael (T CED SES-DE)
On 09.11.22 16:06, Koch, Stefan (DI PA DCP R&D 3) wrote:
> On Wed, 2022-11-09 at 12:50 +0100, Jan Kiszka wrote:
>> On 09.11.22 11:32, Koch, Stefan (DI PA DCP R&D 3) wrote:
>>> Hi
>>>
>>> This patchset is a set of three patches:
>>> - Support overwriting configured schroot dir
>>> - Split up binaries from kernel headers to kbuild packages
>>> - Update custom_kernel docs for split up of kernel scripts and
>>> tools
>>>
>>> The main use-case was to swap out the binaries
>>> from the kernel headers package.
>>> They will be swapped out into host and target
>>> specific kernel kbuild packages.
>>>
>>> The main development goals were these:
>>>
>>> 1. Solve already known isar custom kernel
>>> limitations from doc/custom_kernel.inc
>>> - kernel headers package does not support both native
>>> and cross compilation of kernel modules when cross built
>>
>> Right, that requires building packages for multiple architectures.
>
> Important feature... Maybe it should be added to the TODO list in the
> docs?
>
>> This
>> is the feature we need, and it has nothing to do with the kernel
>> packages per se.
>
> The way that I assume debian will do building the kernel is that they
> use a generic debian defconfig for each architecture.
> E. g. linux-config-5.10/config.amd64_none_amd64.xz, linux-config-
> 5.10/config.arm64_none_arm64.xz
>
> ISAR builds are often using specific defconfigs honored by ISAR's
> linux-custom for the target build.
>
> Debian specific architecture builds (e. g. host and target, maybe with
> sbuild) for one target device, using target specific stripped-down
> defconfigs, may not working properly. The kernel build for a foreign
> architecture (e. g. host) may fail because of an incompatible
> defconfig.
>
> Debian provide a generic very similar defconfig for different
> architectures, and they build the linux-image, linux-headers and linux-
> kbuild for each architecture:
> - So I assume the linux-kbuild:amd64 was build with linux-config-
> 5.10/config.amd64_none_amd64.xz
> - So I assume the linux-kbuild:arm64 was build with linux-config-
> 5.10/config.arm64_none_arm64.xz
>
> The debian generic defconfig may differ in some arch specific points
> but you can use the linux-kbuild:amd64 for a cross-build of linux-
> image:arm64 because the resulting scripts and tools are from the
> identical set. So mixing linux-headers with different archictecture
> specific builds from linux-kbuild should work since the defconfigs have
> a "shared base" (e. g. both define CONFIG_HAVE_C_RECORDMCOUNT thats
> needed for recordmcount utility as example).
Sorry, I don't understand what role the config plays here. Isar is used
to compile (usually) one specific kernel, Debian compiles a few generic
ones. But the fact that Debian compile everything natively (for
production) and we do cross-compilation is not related to different configs.
The answer to the problem "I need kernel headers on the target" right
now is in Isar to build natively the kernel and, thus, the header
package. Not optimal, but we still need this case too rarely to have
cared about it more so far.
>
>> Jan
>>
>>>
>>> 2. Honor recommendations for future from doc/custom_kernel.inc
>>> - Generate kernel headers packages for both host and target
>>> when using cross build
>>>
>>> 3. Add extensions known from debian kernel packages structure
>>> - Generate a kernel headers package without binaries
>>> - Create specific kernel kbuild packages that
>>> will ship the "scripts" and "tools" binaries
>>> - Use symlinks to point to the "scripts" and "tools" binaries
>>
>> Keep in mind that we intend to remain compatible with the layout of
>> Debian. Self-built kernels should remain drop-in replacements of the
>> Debian kernel packages. And that is true for development packages as
>> well.
>>
>> So, are your changes working against that, or are they improving
>> existing incompatibility with the debian packages?
>
> The approach from this patchset does build the kernel in the known ISAR
> specific way. Debian itself patches the kernel Makefiles to modularize
> the kernel build what ISAR doesn't do. With this patch the packages
> linux-kbuild and linux-kbuild-cross are the only new packages that the
> ISAR build will provide. That means ISAR will provide then linux-
> headers, linux-image and linux-kbuild. They match on the target with
> the package name of the respective debian packages. So the existing
> drop-in should not be affected.
>
> In short this patch brings the ISAR linux-custom mechanism a bit nearer
> to the debian package structure. Since linux-headers behaves then more
> like the original debian linux-heades to ship the binaries within a
> separate package -> linux-kbuild.
>
> The trick is to build only the scripts and tools with schroot-target
> (QEMU) in a non-cross way. Since the kernel does not support building
> the scripts and tools for target architecture when do a cross build
> without patching the kernel Makefiles. I think that patch was never
> merged into the kernel:
> https://patchwork.kernel.org/project/linux-kbuild/patch/1376046432-12588-1-git-send-email-broonie@kernel.org/
>
The problem is that patch might have to not provide that use case I
wrote about above. It is a rare use case, a non-distro use case, but I
wouldn't call it an invalid one. If we can address it by splitting up
the Isar kernel build into different packages and then only compile the
binary bits once again, now for the target architecture, that would be a
nice solution.
On top, we should resolve the Isar issue that you can only build one
package in one build for one architecture, not multiples. But I don't
want special-casing the kernel build beyond that because the problem - I
need dev tools both for my Isar build and later also for the target - is
generic, even though still rare in your deployments.
Jan
--
Siemens AG, Technology
Competence Center Embedded Linux
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 0/3] linux-custom: Split up binaries from kernel headers to kbuild packages
2022-11-09 16:00 ` Jan Kiszka
@ 2022-11-10 17:49 ` Koch, Stefan
2022-11-10 18:33 ` Jan Kiszka
0 siblings, 1 reply; 18+ messages in thread
From: Koch, Stefan @ 2022-11-10 17:49 UTC (permalink / raw)
To: isar-users, Kiszka, Jan; +Cc: Sudler, Simon, Storm, Christian, Adler, Michael
On Wed, 2022-11-09 at 17:00 +0100, Jan Kiszka wrote:
> On 09.11.22 16:06, Koch, Stefan (DI PA DCP R&D 3) wrote:
> > On Wed, 2022-11-09 at 12:50 +0100, Jan Kiszka wrote:
> > > On 09.11.22 11:32, Koch, Stefan (DI PA DCP R&D 3) wrote:
> > > > Hi
> > > >
> > > > This patchset is a set of three patches:
> > > > - Support overwriting configured schroot dir
> > > > - Split up binaries from kernel headers to kbuild packages
> > > > - Update custom_kernel docs for split up of kernel scripts and
> > > > tools
> > > >
> > > > The main use-case was to swap out the binaries
> > > > from the kernel headers package.
> > > > They will be swapped out into host and target
> > > > specific kernel kbuild packages.
> > > >
> > > > The main development goals were these:
> > > >
> > > > 1. Solve already known isar custom kernel
> > > > limitations from doc/custom_kernel.inc
> > > > - kernel headers package does not support both native
> > > > and cross compilation of kernel modules when cross built
> > >
> > > Right, that requires building packages for multiple
> > > architectures.
> >
> > Important feature... Maybe it should be added to the TODO list in
> > the
> > docs?
> >
> > > This
> > > is the feature we need, and it has nothing to do with the kernel
> > > packages per se.
> >
> > The way that I assume debian will do building the kernel is that
> > they
> > use a generic debian defconfig for each architecture.
> > E. g. linux-config-5.10/config.amd64_none_amd64.xz, linux-config-
> > 5.10/config.arm64_none_arm64.xz
> >
> > ISAR builds are often using specific defconfigs honored by ISAR's
> > linux-custom for the target build.
> >
> > Debian specific architecture builds (e. g. host and target, maybe
> > with
> > sbuild) for one target device, using target specific stripped-down
> > defconfigs, may not working properly. The kernel build for a
> > foreign
> > architecture (e. g. host) may fail because of an incompatible
> > defconfig.
> >
> > Debian provide a generic very similar defconfig for different
> > architectures, and they build the linux-image, linux-headers and
> > linux-
> > kbuild for each architecture:
> > - So I assume the linux-kbuild:amd64 was build with linux-config-
> > 5.10/config.amd64_none_amd64.xz
> > - So I assume the linux-kbuild:arm64 was build with linux-config-
> > 5.10/config.arm64_none_arm64.xz
> >
> > The debian generic defconfig may differ in some arch specific
> > points
> > but you can use the linux-kbuild:amd64 for a cross-build of linux-
> > image:arm64 because the resulting scripts and tools are from the
> > identical set. So mixing linux-headers with different archictecture
> > specific builds from linux-kbuild should work since the defconfigs
> > have
> > a "shared base" (e. g. both define CONFIG_HAVE_C_RECORDMCOUNT thats
> > needed for recordmcount utility as example).
>
> Sorry, I don't understand what role the config plays here. Isar is
> used
> to compile (usually) one specific kernel, Debian compiles a few
> generic
> ones. But the fact that Debian compile everything natively (for
> production) and we do cross-compilation is not related to different
> configs.
>
I just wanted to point out for the future case when ISAR would support
a generic way to build packages for multiple architectures: This might
not be applicable to all kernel builds. E. g. because of the fact that
the used defconfig for a "specific kernel" is not compatible for a
multi architecture build, only for the architecture that the defconfig
was made for.
Yes, it may work if you could specify different defconfigs for each
architecture. One goal of this patch was to provide the kbuild scripts
and tools just for host and target in a transparent way - without any
changes to the build process of existing ISAR users.
> The answer to the problem "I need kernel headers on the target" right
> now is in Isar to build natively the kernel and, thus, the header
> package. Not optimal, but we still need this case too rarely to have
> cared about it more so far.
>
> >
> > > Jan
> > >
> > > >
> > > > 2. Honor recommendations for future from doc/custom_kernel.inc
> > > > - Generate kernel headers packages for both host and target
> > > > when using cross build
> > > >
> > > > 3. Add extensions known from debian kernel packages structure
> > > > - Generate a kernel headers package without binaries
> > > > - Create specific kernel kbuild packages that
> > > > will ship the "scripts" and "tools" binaries
> > > > - Use symlinks to point to the "scripts" and "tools" binaries
> > >
> > > Keep in mind that we intend to remain compatible with the layout
> > > of
> > > Debian. Self-built kernels should remain drop-in replacements of
> > > the
> > > Debian kernel packages. And that is true for development packages
> > > as
> > > well.
> > >
> > > So, are your changes working against that, or are they improving
> > > existing incompatibility with the debian packages?
> >
> > The approach from this patchset does build the kernel in the known
> > ISAR
> > specific way. Debian itself patches the kernel Makefiles to
> > modularize
> > the kernel build what ISAR doesn't do. With this patch the packages
> > linux-kbuild and linux-kbuild-cross are the only new packages that
> > the
> > ISAR build will provide. That means ISAR will provide then linux-
> > headers, linux-image and linux-kbuild. They match on the target
> > with
> > the package name of the respective debian packages. So the existing
> > drop-in should not be affected.
> >
> > In short this patch brings the ISAR linux-custom mechanism a bit
> > nearer
> > to the debian package structure. Since linux-headers behaves then
> > more
> > like the original debian linux-heades to ship the binaries within a
> > separate package -> linux-kbuild.
> >
> > The trick is to build only the scripts and tools with schroot-
> > target
> > (QEMU) in a non-cross way. Since the kernel does not support
> > building
> > the scripts and tools for target architecture when do a cross build
> > without patching the kernel Makefiles. I think that patch was never
> > merged into the kernel:
> >
> > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpatchwork.kernel.org%2Fproject%2Flinux-kbuild%2Fpatch%2F1376046432-12588-1-git-send-email-broonie%40kernel.org%2F&data=05%7C01%7Cstefan-koch%40siemens.com%7C5054591149e44251f36d08dac26b7b61%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C638036064125400893%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=iPeO%2FkFjrLNSGQJ83fNsifWNI0wbPzFsWTIi%2B1fZ9PI%3D&reserved=0
> >
>
> The problem is that patch might have to not provide that use case I
> wrote about above. It is a rare use case, a non-distro use case, but
> I
> wouldn't call it an invalid one. If we can address it by splitting up
> the Isar kernel build into different packages and then only compile
> the
> binary bits once again, now for the target architecture, that would
> be a
> nice solution.
That's what the patch do:
- Move the binary scripts and tools from linux-headers to linux-kbuild
as debian does it for their kernels
- Compile only the scripts and tools for the target using schroot-
target (QEMU) and package it as linux-kbuild
- Package the binary scripts and tools from the full cross kernel build
to linux-kbuild-cross using schroot-host
>
> On top, we should resolve the Isar issue that you can only build one
> package in one build for one architecture, not multiples. But I don't
> want special-casing the kernel build beyond that because the problem
> - I
> need dev tools both for my Isar build and later also for the target -
> is
> generic, even though still rare in your deployments.
This is a nice and important goal. Maybe ISAR is now nearer to this
since it has changed to use sbuild now.
>
> Jan
>
--
Stefan Koch
Siemens AG
www.siemens.com
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 0/3] linux-custom: Split up binaries from kernel headers to kbuild packages
2022-11-10 17:49 ` Koch, Stefan
@ 2022-11-10 18:33 ` Jan Kiszka
2022-11-10 18:36 ` Jan Kiszka
0 siblings, 1 reply; 18+ messages in thread
From: Jan Kiszka @ 2022-11-10 18:33 UTC (permalink / raw)
To: Koch, Stefan (DI PA DCP R&D 3), isar-users
Cc: Sudler, Simon (DI PA DCP TI), Storm, Christian (T CED SES-DE),
Adler, Michael (T CED SES-DE)
On 10.11.22 18:49, Koch, Stefan (DI PA DCP R&D 3) wrote:
> On Wed, 2022-11-09 at 17:00 +0100, Jan Kiszka wrote:
>> On 09.11.22 16:06, Koch, Stefan (DI PA DCP R&D 3) wrote:
>>> On Wed, 2022-11-09 at 12:50 +0100, Jan Kiszka wrote:
>>>> On 09.11.22 11:32, Koch, Stefan (DI PA DCP R&D 3) wrote:
>>>>> Hi
>>>>>
>>>>> This patchset is a set of three patches:
>>>>> - Support overwriting configured schroot dir
>>>>> - Split up binaries from kernel headers to kbuild packages
>>>>> - Update custom_kernel docs for split up of kernel scripts and
>>>>> tools
>>>>>
>>>>> The main use-case was to swap out the binaries
>>>>> from the kernel headers package.
>>>>> They will be swapped out into host and target
>>>>> specific kernel kbuild packages.
>>>>>
>>>>> The main development goals were these:
>>>>>
>>>>> 1. Solve already known isar custom kernel
>>>>> limitations from doc/custom_kernel.inc
>>>>> - kernel headers package does not support both native
>>>>> and cross compilation of kernel modules when cross built
>>>>
>>>> Right, that requires building packages for multiple
>>>> architectures.
>>>
>>> Important feature... Maybe it should be added to the TODO list in
>>> the
>>> docs?
>>>
>>>> This
>>>> is the feature we need, and it has nothing to do with the kernel
>>>> packages per se.
>>>
>>> The way that I assume debian will do building the kernel is that
>>> they
>>> use a generic debian defconfig for each architecture.
>>> E. g. linux-config-5.10/config.amd64_none_amd64.xz, linux-config-
>>> 5.10/config.arm64_none_arm64.xz
>>>
>>> ISAR builds are often using specific defconfigs honored by ISAR's
>>> linux-custom for the target build.
>>>
>>> Debian specific architecture builds (e. g. host and target, maybe
>>> with
>>> sbuild) for one target device, using target specific stripped-down
>>> defconfigs, may not working properly. The kernel build for a
>>> foreign
>>> architecture (e. g. host) may fail because of an incompatible
>>> defconfig.
>>>
>>> Debian provide a generic very similar defconfig for different
>>> architectures, and they build the linux-image, linux-headers and
>>> linux-
>>> kbuild for each architecture:
>>> - So I assume the linux-kbuild:amd64 was build with linux-config-
>>> 5.10/config.amd64_none_amd64.xz
>>> - So I assume the linux-kbuild:arm64 was build with linux-config-
>>> 5.10/config.arm64_none_arm64.xz
>>>
>>> The debian generic defconfig may differ in some arch specific
>>> points
>>> but you can use the linux-kbuild:amd64 for a cross-build of linux-
>>> image:arm64 because the resulting scripts and tools are from the
>>> identical set. So mixing linux-headers with different archictecture
>>> specific builds from linux-kbuild should work since the defconfigs
>>> have
>>> a "shared base" (e. g. both define CONFIG_HAVE_C_RECORDMCOUNT thats
>>> needed for recordmcount utility as example).
>>
>> Sorry, I don't understand what role the config plays here. Isar is
>> used
>> to compile (usually) one specific kernel, Debian compiles a few
>> generic
>> ones. But the fact that Debian compile everything natively (for
>> production) and we do cross-compilation is not related to different
>> configs.
>>
>
> I just wanted to point out for the future case when ISAR would support
> a generic way to build packages for multiple architectures: This might
> not be applicable to all kernel builds. E. g. because of the fact that
> the used defconfig for a "specific kernel" is not compatible for a
> multi architecture build, only for the architecture that the defconfig
> was made for.
>
> Yes, it may work if you could specify different defconfigs for each
> architecture. One goal of this patch was to provide the kbuild scripts
> and tools just for host and target in a transparent way - without any
> changes to the build process of existing ISAR users.
Misunderstanding: I didn't meant to propose building the kernel-image
package for multiple arch, only the then factored-out kbuild packages
with the host tool (once for the build env, once for the target).
>
>> The answer to the problem "I need kernel headers on the target" right
>> now is in Isar to build natively the kernel and, thus, the header
>> package. Not optimal, but we still need this case too rarely to have
>> cared about it more so far.
>>
>>>
>>>> Jan
>>>>
>>>>>
>>>>> 2. Honor recommendations for future from doc/custom_kernel.inc
>>>>> - Generate kernel headers packages for both host and target
>>>>> when using cross build
>>>>>
>>>>> 3. Add extensions known from debian kernel packages structure
>>>>> - Generate a kernel headers package without binaries
>>>>> - Create specific kernel kbuild packages that
>>>>> will ship the "scripts" and "tools" binaries
>>>>> - Use symlinks to point to the "scripts" and "tools" binaries
>>>>
>>>> Keep in mind that we intend to remain compatible with the layout
>>>> of
>>>> Debian. Self-built kernels should remain drop-in replacements of
>>>> the
>>>> Debian kernel packages. And that is true for development packages
>>>> as
>>>> well.
>>>>
>>>> So, are your changes working against that, or are they improving
>>>> existing incompatibility with the debian packages?
>>>
>>> The approach from this patchset does build the kernel in the known
>>> ISAR
>>> specific way. Debian itself patches the kernel Makefiles to
>>> modularize
>>> the kernel build what ISAR doesn't do. With this patch the packages
>>> linux-kbuild and linux-kbuild-cross are the only new packages that
>>> the
>>> ISAR build will provide. That means ISAR will provide then linux-
>>> headers, linux-image and linux-kbuild. They match on the target
>>> with
>>> the package name of the respective debian packages. So the existing
>>> drop-in should not be affected.
>>>
>>> In short this patch brings the ISAR linux-custom mechanism a bit
>>> nearer
>>> to the debian package structure. Since linux-headers behaves then
>>> more
>>> like the original debian linux-heades to ship the binaries within a
>>> separate package -> linux-kbuild.
>>>
>>> The trick is to build only the scripts and tools with schroot-
>>> target
>>> (QEMU) in a non-cross way. Since the kernel does not support
>>> building
>>> the scripts and tools for target architecture when do a cross build
>>> without patching the kernel Makefiles. I think that patch was never
>>> merged into the kernel:
>>>
>>> https://patchwork.kernel.org/project/linux-kbuild/patch/1376046432-12588-1-git-send-email-broonie@kernel.org/
>>>
>>
>> The problem is that patch might have to not provide that use case I
>> wrote about above. It is a rare use case, a non-distro use case, but
>> I
>> wouldn't call it an invalid one. If we can address it by splitting up
>> the Isar kernel build into different packages and then only compile
>> the
>> binary bits once again, now for the target architecture, that would
>> be a
>> nice solution.
>
> That's what the patch do:
> - Move the binary scripts and tools from linux-headers to linux-kbuild
> as debian does it for their kernels
> - Compile only the scripts and tools for the target using schroot-
> target (QEMU) and package it as linux-kbuild
> - Package the binary scripts and tools from the full cross kernel build
> to linux-kbuild-cross using schroot-host
It's moving in a good direction - but why linux-kbuild-cross? Nothing
depends on it in your patches, thus module builds will be broken. I
suspect we rather need
KERNEL_PKG_KERN_KBUILD=linux-kbuild-${KERNEL_NAME_PROVIDED}:<native>
and therefore (on amd64) linux-kbuild-${KERNEL_NAME_PROVIDED}:amd64 to
be built and then installed for module builds.
Jan
--
Siemens AG, Technology
Competence Center Embedded Linux
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 0/3] linux-custom: Split up binaries from kernel headers to kbuild packages
2022-11-10 18:33 ` Jan Kiszka
@ 2022-11-10 18:36 ` Jan Kiszka
2022-11-11 9:47 ` Koch, Stefan
0 siblings, 1 reply; 18+ messages in thread
From: Jan Kiszka @ 2022-11-10 18:36 UTC (permalink / raw)
To: Koch, Stefan (DI PA DCP R&D 3), isar-users
Cc: Sudler, Simon (DI PA DCP TI), Storm, Christian (T CED SES-DE),
Adler, Michael (T CED SES-DE)
On 10.11.22 19:33, Jan Kiszka wrote:
> On 10.11.22 18:49, Koch, Stefan (DI PA DCP R&D 3) wrote:
>> On Wed, 2022-11-09 at 17:00 +0100, Jan Kiszka wrote:
>>> On 09.11.22 16:06, Koch, Stefan (DI PA DCP R&D 3) wrote:
>>>> On Wed, 2022-11-09 at 12:50 +0100, Jan Kiszka wrote:
>>>>> On 09.11.22 11:32, Koch, Stefan (DI PA DCP R&D 3) wrote:
>>>>>> Hi
>>>>>>
>>>>>> This patchset is a set of three patches:
>>>>>> - Support overwriting configured schroot dir
>>>>>> - Split up binaries from kernel headers to kbuild packages
>>>>>> - Update custom_kernel docs for split up of kernel scripts and
>>>>>> tools
>>>>>>
>>>>>> The main use-case was to swap out the binaries
>>>>>> from the kernel headers package.
>>>>>> They will be swapped out into host and target
>>>>>> specific kernel kbuild packages.
>>>>>>
>>>>>> The main development goals were these:
>>>>>>
>>>>>> 1. Solve already known isar custom kernel
>>>>>> limitations from doc/custom_kernel.inc
>>>>>> - kernel headers package does not support both native
>>>>>> and cross compilation of kernel modules when cross built
>>>>>
>>>>> Right, that requires building packages for multiple
>>>>> architectures.
>>>>
>>>> Important feature... Maybe it should be added to the TODO list in
>>>> the
>>>> docs?
>>>>
>>>>> This
>>>>> is the feature we need, and it has nothing to do with the kernel
>>>>> packages per se.
>>>>
>>>> The way that I assume debian will do building the kernel is that
>>>> they
>>>> use a generic debian defconfig for each architecture.
>>>> E. g. linux-config-5.10/config.amd64_none_amd64.xz, linux-config-
>>>> 5.10/config.arm64_none_arm64.xz
>>>>
>>>> ISAR builds are often using specific defconfigs honored by ISAR's
>>>> linux-custom for the target build.
>>>>
>>>> Debian specific architecture builds (e. g. host and target, maybe
>>>> with
>>>> sbuild) for one target device, using target specific stripped-down
>>>> defconfigs, may not working properly. The kernel build for a
>>>> foreign
>>>> architecture (e. g. host) may fail because of an incompatible
>>>> defconfig.
>>>>
>>>> Debian provide a generic very similar defconfig for different
>>>> architectures, and they build the linux-image, linux-headers and
>>>> linux-
>>>> kbuild for each architecture:
>>>> - So I assume the linux-kbuild:amd64 was build with linux-config-
>>>> 5.10/config.amd64_none_amd64.xz
>>>> - So I assume the linux-kbuild:arm64 was build with linux-config-
>>>> 5.10/config.arm64_none_arm64.xz
>>>>
>>>> The debian generic defconfig may differ in some arch specific
>>>> points
>>>> but you can use the linux-kbuild:amd64 for a cross-build of linux-
>>>> image:arm64 because the resulting scripts and tools are from the
>>>> identical set. So mixing linux-headers with different archictecture
>>>> specific builds from linux-kbuild should work since the defconfigs
>>>> have
>>>> a "shared base" (e. g. both define CONFIG_HAVE_C_RECORDMCOUNT thats
>>>> needed for recordmcount utility as example).
>>>
>>> Sorry, I don't understand what role the config plays here. Isar is
>>> used
>>> to compile (usually) one specific kernel, Debian compiles a few
>>> generic
>>> ones. But the fact that Debian compile everything natively (for
>>> production) and we do cross-compilation is not related to different
>>> configs.
>>>
>>
>> I just wanted to point out for the future case when ISAR would support
>> a generic way to build packages for multiple architectures: This might
>> not be applicable to all kernel builds. E. g. because of the fact that
>> the used defconfig for a "specific kernel" is not compatible for a
>> multi architecture build, only for the architecture that the defconfig
>> was made for.
>>
>> Yes, it may work if you could specify different defconfigs for each
>> architecture. One goal of this patch was to provide the kbuild scripts
>> and tools just for host and target in a transparent way - without any
>> changes to the build process of existing ISAR users.
>
> Misunderstanding: I didn't meant to propose building the kernel-image
> package for multiple arch, only the then factored-out kbuild packages
> with the host tool (once for the build env, once for the target).
>
>>
>>> The answer to the problem "I need kernel headers on the target" right
>>> now is in Isar to build natively the kernel and, thus, the header
>>> package. Not optimal, but we still need this case too rarely to have
>>> cared about it more so far.
>>>
>>>>
>>>>> Jan
>>>>>
>>>>>>
>>>>>> 2. Honor recommendations for future from doc/custom_kernel.inc
>>>>>> - Generate kernel headers packages for both host and target
>>>>>> when using cross build
>>>>>>
>>>>>> 3. Add extensions known from debian kernel packages structure
>>>>>> - Generate a kernel headers package without binaries
>>>>>> - Create specific kernel kbuild packages that
>>>>>> will ship the "scripts" and "tools" binaries
>>>>>> - Use symlinks to point to the "scripts" and "tools" binaries
>>>>>
>>>>> Keep in mind that we intend to remain compatible with the layout
>>>>> of
>>>>> Debian. Self-built kernels should remain drop-in replacements of
>>>>> the
>>>>> Debian kernel packages. And that is true for development packages
>>>>> as
>>>>> well.
>>>>>
>>>>> So, are your changes working against that, or are they improving
>>>>> existing incompatibility with the debian packages?
>>>>
>>>> The approach from this patchset does build the kernel in the known
>>>> ISAR
>>>> specific way. Debian itself patches the kernel Makefiles to
>>>> modularize
>>>> the kernel build what ISAR doesn't do. With this patch the packages
>>>> linux-kbuild and linux-kbuild-cross are the only new packages that
>>>> the
>>>> ISAR build will provide. That means ISAR will provide then linux-
>>>> headers, linux-image and linux-kbuild. They match on the target
>>>> with
>>>> the package name of the respective debian packages. So the existing
>>>> drop-in should not be affected.
>>>>
>>>> In short this patch brings the ISAR linux-custom mechanism a bit
>>>> nearer
>>>> to the debian package structure. Since linux-headers behaves then
>>>> more
>>>> like the original debian linux-heades to ship the binaries within a
>>>> separate package -> linux-kbuild.
>>>>
>>>> The trick is to build only the scripts and tools with schroot-
>>>> target
>>>> (QEMU) in a non-cross way. Since the kernel does not support
>>>> building
>>>> the scripts and tools for target architecture when do a cross build
>>>> without patching the kernel Makefiles. I think that patch was never
>>>> merged into the kernel:
>>>>
>>>> https://patchwork.kernel.org/project/linux-kbuild/patch/1376046432-12588-1-git-send-email-broonie@kernel.org/
>>>>
>>>
>>> The problem is that patch might have to not provide that use case I
>>> wrote about above. It is a rare use case, a non-distro use case, but
>>> I
>>> wouldn't call it an invalid one. If we can address it by splitting up
>>> the Isar kernel build into different packages and then only compile
>>> the
>>> binary bits once again, now for the target architecture, that would
>>> be a
>>> nice solution.
>>
>> That's what the patch do:
>> - Move the binary scripts and tools from linux-headers to linux-kbuild
>> as debian does it for their kernels
>> - Compile only the scripts and tools for the target using schroot-
>> target (QEMU) and package it as linux-kbuild
>> - Package the binary scripts and tools from the full cross kernel build
>> to linux-kbuild-cross using schroot-host
>
> It's moving in a good direction - but why linux-kbuild-cross? Nothing
> depends on it in your patches, thus module builds will be broken. I
> suspect we rather need
>
> KERNEL_PKG_KERN_KBUILD=linux-kbuild-${KERNEL_NAME_PROVIDED}:<native>
>
> and therefore (on amd64) linux-kbuild-${KERNEL_NAME_PROVIDED}:amd64 to
> be built and then installed for module builds.
>
> Jan
>
And regarding structure of the series: please split up the creation of
kbuild package from also building it for the target arch - if possible.
Jan
--
Siemens AG, Technology
Competence Center Embedded Linux
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 2/3] linux-custom: Split up binaries from kernel headers to kbuild packages
2022-11-09 10:32 ` [PATCH 2/3] linux-custom: Split up binaries from kernel headers to kbuild packages Koch, Stefan
@ 2022-11-11 5:34 ` Uladzimir Bely
2022-11-11 9:03 ` Koch, Stefan
0 siblings, 1 reply; 18+ messages in thread
From: Uladzimir Bely @ 2022-11-11 5:34 UTC (permalink / raw)
To: isar-users; +Cc: Koch, Stefan
In mail from среда, 9 ноября 2022 г. 13:32:48 +03 user Koch, Stefan wrote:
> When using a cross build this patch does introduce
> host and target specific kernel kbuild packages that
> ship the "scripts" and "tools" binaries.
> The kernel headers fulfill this using symlinks to point
> to the "scripts" and "tools" of the respective kernel kbuild package.
>
> Known from doc/custom_kernel.inc:
> - The kernel headers package has not supported both native
> and cross compilation of kernel modules when itself was cross built
> - Future roadmap: Generate kernel headers package for both host
> and target when using a cross build
>
> Known from debian kernel packages structure:
> - Generate a kernel headers package without binaries
> - Create specific kernel kbuild packages that
> ship the "scripts" and "tools" binaries
> - Using symlinks to point to the "scripts"
> and "tools" binaries
>
> Signed-off-by: Stefan Koch <stefan-koch@siemens.com>
> ---
> .../linux/files/debian/control.tmpl | 18 ++-
> .../linux/files/debian/isar/build.tmpl | 9 +-
> .../linux/files/debian/isar/common.tmpl | 7 +-
> .../linux/files/debian/isar/install.tmpl | 64 +++++++++--
> .../linux/files/debian/rules.tmpl | 5 +-
> meta/recipes-kernel/linux/linux-custom.inc | 104 ++++++++++++++++--
> 6 files changed, 178 insertions(+), 29 deletions(-)
>
> diff --git a/meta/recipes-kernel/linux/files/debian/control.tmpl
> b/meta/recipes-kernel/linux/files/debian/control.tmpl index
> dd0b624..adac4ef 100644
> --- a/meta/recipes-kernel/linux/files/debian/control.tmpl
> +++ b/meta/recipes-kernel/linux/files/debian/control.tmpl
> @@ -26,7 +26,7 @@ Section: devel
> Provides: linux-kernel-headers
> Architecture: any
> Description: Linux support headers for userspace development
> - This package provides userspaces headers from the Linux kernel. These
> headers + This package provides userspaces headers from the Linux kernel.
> These headers are used by the installed headers for GNU glibc and other
> system libraries.
>
> Package: linux-image-${KERNEL_NAME_PROVIDED}-dbg
> @@ -35,3 +35,19 @@ Architecture: any
> Description: Linux kernel debugging symbols for @KR@
> This package will come in handy if you need to debug the kernel. It
> provides all the necessary debug symbols for the kernel and its modules.
> +
> +Package: linux-kbuild-${KERNEL_NAME_PROVIDED}
> +Architecture: any
> +Depends: ${KERNEL_HEADERS_DEBIAN_DEPENDS}, ${perl:Depends},
> ${shlib:Depends} +Description: ${KERNEL_NAME_PROVIDED} Linux kbuild scripts
> and tools for @KR@ + This package provides kernel kbuild scripts and tools
> for @KR@
> + This is useful for people who need to build external modules
> +
> +Package: linux-kbuild-${KERNEL_NAME_PROVIDED}-cross
> +Architecture: ${CONTROL_TARGET_ARCH}
Is this architecture correct? Below, it's set to target_arch = distro_arch for
cross-compile case.
I looked into `linux-kbuild-mainline-cross_5.4.203+r0_armhf.deb` and there are
some binaries inside, "ELF 64-bit LSB pie executable, x86-64", so it looks for
me that the package is supposed to be installed on host machine and to have
the host architecture instead.
> +Depends: ${KERNEL_HEADERS_DEBIAN_DEPENDS}, ${perl:Depends},
> ${shlib:Depends} +Description: ${KERNEL_NAME_PROVIDED} Linux kbuild scripts
> and tools for @KR@ + This package provides kernel kbuild scripts and tools
> + as ${HOST_ARCH} cross binaries for @KR@
> + This is useful for those who need to cross build
> + external modules using ISAR's sbuild-chroot-host
> diff --git a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl
> b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl index
> 94cfbe0..7e2daad 100644
> --- a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl
> +++ b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl
> @@ -13,16 +13,19 @@ do_build() {
> # Trace what we do here
> set -x
>
> + # Set correct build directory for kernel or kbuild (kernel scripts and
> tools) cases + build_dir=${KERNEL_BUILD_DIR}
> +
> # Process existing kernel configuration to make sure it is complete
> # (use defaults for options that were not specified)
> - ${MAKE} O=${KERNEL_BUILD_DIR} olddefconfig prepare
> + ${MAKE} O=${build_dir} olddefconfig prepare
>
> # Transfer effective kernel version into control file and scripts
> - KR=$(${MAKE} O=${KERNEL_BUILD_DIR} -s --no-print-directory
> kernelrelease) + KR=$(${MAKE} O=${build_dir} -s --no-print-directory
> kernelrelease) sed -i "s/@KR@/${KR}/g" ${S}/debian/control
> ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.*
>
> # Build the Linux kernel
> - ${MAKE} O=${KERNEL_BUILD_DIR} ${PARALLEL_MAKE} KCFLAGS="${KCFLAGS}"
> + ${MAKE} O=${build_dir} ${PARALLEL_MAKE} KCFLAGS="${KCFLAGS}"
>
> # Stop tracing
> set +x
> diff --git a/meta/recipes-kernel/linux/files/debian/isar/common.tmpl
> b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl index
> 52ebebb..55d6123 100644
> --- a/meta/recipes-kernel/linux/files/debian/isar/common.tmpl
> +++ b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl
> @@ -5,10 +5,14 @@
> set -e
>
> # Isar settings
> -ARCH=${KERNEL_ARCH}
> KERNEL_PKG_IMAGE=linux-image-${KERNEL_NAME_PROVIDED}
> KERNEL_PKG_KERN_HEADERS=linux-headers-${KERNEL_NAME_PROVIDED}
> KERNEL_PKG_LIBC_HEADERS=linux-libc-dev
> +KERNEL_PKG_KERN_KBUILD=linux-kbuild-${KERNEL_NAME_PROVIDED}
> +
> +if [ -z "${ARCH}" ]; then
> + ARCH=${KERNEL_ARCH}
> +fi
>
> # Constants
> KCONF=.config
> @@ -19,6 +23,7 @@ deb_img_dir=${deb_top_dir}/${KERNEL_PKG_IMAGE}
> deb_dbg_dir=${deb_img_dir}-dbg
> deb_kern_hdr_dir=${deb_top_dir}/${KERNEL_PKG_KERN_HEADERS}
> deb_libc_hdr_dir=${deb_top_dir}/${KERNEL_PKG_LIBC_HEADERS}
> +deb_kern_kbuild_dir=${deb_top_dir}/${KERNEL_PKG_KERN_KBUILD}
>
> # Array of packages to be generated
> declare -A kern_pkgs
> diff --git a/meta/recipes-kernel/linux/files/debian/isar/install.tmpl
> b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl index
> 0a8645d..890a996 100644
> --- a/meta/recipes-kernel/linux/files/debian/isar/install.tmpl
> +++ b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl
> @@ -33,6 +33,7 @@ do_install() {
> # Trace what we do here
> set -x
>
> + # Run the install steps
> install_image
> if [ "${ARCH}" != "um" ]; then
> install_config
> @@ -43,6 +44,19 @@ do_install() {
> install_kmods
> install_headers
>
> + if [ -d "$(dirname ${O})/build-full-kbuild-target" ]; then
> + # Install cross kernel scripts and tools
> + install_kbuild ${deb_kern_kbuild_dir}-cross build-full
> +
> + # Cleanup and install kernel scripts and tools for target arch
> + rm -rf ${deb_kern_kbuild_dir}
> + install_kbuild ${deb_kern_kbuild_dir} build-full-kbuild-target
> + else
> + # Cleanup and install kernel scripts and tools
> + rm -rf ${deb_kern_kbuild_dir}
> + install_kbuild ${deb_kern_kbuild_dir} build-full
> + fi
> +
> # Stop tracing
> set +x
> }
> @@ -168,21 +182,15 @@ kernel_headers() {
> mkdir -p ${destdir}
> mkdir -p ${deb_kern_hdr_dir}/lib/modules/${krel}
>
> - (cd ${S}; find . -name 'Makefile*' -o -name 'Kconfig*' -o -name '*.pl')
> >>${src_hdr_files} - (cd ${S}; find arch/*/include include scripts -type
> f -o -type l) >>${src_hdr_files} + (cd ${S}; find . -not -path
> './scripts/*' -a -not -path './tools/*' -a \( -name 'Makefile*' -o -name
> 'Kconfig*' -o -name '*.pl' \)) >>${src_hdr_files} + (cd ${S}; find
> arch/*/include include -type f -o -type l) >>${src_hdr_files} (cd ${S};
> find arch/${ARCH} -name module.lds -o -name Kbuild.platforms -o -name
> Platform) >>${src_hdr_files} (cd ${S}; find $(find arch/${ARCH} -name
> include -o -name scripts -type d) -type f) >>${src_hdr_files}
>
> if [ -n "${CONFIG_MODULES}" ]; then
> echo Module.symvers >> ${obj_hdr_files}
> fi
> - (cd ${O}; find arch/${ARCH}/include include scripts -type f)
> >>${obj_hdr_files} - if [ -n "${CONFIG_STACK_VALIDATION}" ]; then
> - (cd ${O}; find tools/objtool -type f -executable)
> >>${obj_hdr_files} - fi
> - if [ -n "${CONFIG_GCC_PLUGINS}" ]; then
> - (cd ${O}; find scripts/gcc-plugins -name *.so -o -name
> gcc-common.h) >>${obj_hdr_files} - fi
> + (cd ${O}; find arch/${ARCH}/include include -type f) >>${obj_hdr_files}
>
> # deploy files that were matched above
> tar -C ${S} -cf - -T - <${src_hdr_files} | tar -C ${destdir} -xf -
> @@ -191,8 +199,11 @@ kernel_headers() {
> # add the kernel config
> cp ${O}/${KCONF} ${destdir}/.config
>
> - # handle kernel development tools
> - kernel_tools
> + # add symlink to scripts and tools directories
> + ln -sf ../../lib/linux-kbuild-${krel}/scripts ${destdir}/scripts
> + if [ -n "${CONFIG_STACK_VALIDATION}" ]; then
> + ln -sf ../../lib/linux-kbuild-${krel}/tools ${destdir}/tools
> + fi
>
> # create symlinks
> ln -sf /${kernel_headers_dir}
> ${deb_kern_hdr_dir}/lib/modules/${krel}/build @@ -206,4 +217,35 @@
> install_headers() {
> kernel_headers
> }
>
> +install_kbuild() {
> + kernel_kbuild_dir=usr/lib/linux-kbuild-${krel}
> + destdir=${1}/${kernel_kbuild_dir}
> + src_kbuild_files=$(mktemp)
> + obj_kbuild_files=$(mktemp)
> +
> + if [ -n "${2}" ]; then
> + O=$(dirname ${O})/${2}
> + fi
> +
> + mkdir -p ${destdir}
> +
> + (cd ${S}; find . -path './scripts/*' -a -path './tools/*' -a \( -name
> 'Makefile*' -o -name 'Kconfig*' -o -name '*.pl' \)) >>${src_kbuild_files} +
> (cd ${S}; find scripts -type f -o -type l) >>${src_kbuild_files} +
> + (cd ${O}; find scripts -type f) >>${obj_kbuild_files}
> + if [ -n "${CONFIG_STACK_VALIDATION}" ]; then
> + (cd ${O}; find tools/objtool -type f -executable)
> >>${obj_kbuild_files} + fi
> + if [ -n "${CONFIG_GCC_PLUGINS}" ]; then
> + (cd ${O}; find scripts/gcc-plugins -name *.so -o -name
> gcc-common.h) >>${obj_kbuild_files} + fi
> +
> + # deploy files that were matched above
> + tar -C ${S} -cf - -T - <${src_kbuild_files} | tar -C ${destdir} -xf -
> + tar -C ${O} -cf - -T - <${obj_kbuild_files} | tar -C ${destdir} -xf -
> +
> + # handle kernel development tools
> + kernel_tools
> +}
> +
> main install ${*}
> diff --git a/meta/recipes-kernel/linux/files/debian/rules.tmpl
> b/meta/recipes-kernel/linux/files/debian/rules.tmpl index 8063c49..e131288
> 100755
> --- a/meta/recipes-kernel/linux/files/debian/rules.tmpl
> +++ b/meta/recipes-kernel/linux/files/debian/rules.tmpl
> @@ -36,4 +36,7 @@ override_dh_auto_test:
> true
>
> override_dh_strip:
> - unset DEB_HOST_GNU_TYPE && dh_strip -Xvmlinu --no-automatic-dbgsym
> + dh_strip -Xvmlinu -Xlinux-kbuild-${KERNEL_NAME_PROVIDED}-cross
> --no-automatic-dbgsym +
> +override_dh_shlibdeps:
> + dh_shlibdeps -Xlinux-kbuild-${KERNEL_NAME_PROVIDED}
> diff --git a/meta/recipes-kernel/linux/linux-custom.inc
> b/meta/recipes-kernel/linux/linux-custom.inc index 96f0afc..0f8ba50 100644
> --- a/meta/recipes-kernel/linux/linux-custom.inc
> +++ b/meta/recipes-kernel/linux/linux-custom.inc
> @@ -74,12 +74,14 @@ TEMPLATE_VARS += " \
> KERNEL_ARCH \
> KERNEL_DEBIAN_DEPENDS \
> KERNEL_BUILD_DIR \
> + KERNEL_KBUILD_DIR \
> KERNEL_FILE \
> KERNEL_HEADERS_DEBIAN_DEPENDS \
> LINUX_VERSION_EXTENSION \
> KERNEL_NAME_PROVIDED \
> KERNEL_CONFIG_FRAGMENTS \
> KCFLAGS \
> + CONTROL_TARGET_ARCH \
> "
>
> inherit dpkg
> @@ -91,30 +93,54 @@ KCFLAGS ?= "-fdebug-prefix-map=${CURDIR}=."
> # Derive name of the kernel packages from the name of this recipe
> KERNEL_NAME_PROVIDED ?= "${@ d.getVar('PN', True).partition('linux-')[2]}"
>
> -# Make bitbake know we will be producing linux-image and linux-headers
> packages python() {
> kernel_name = d.getVar("KERNEL_NAME_PROVIDED", True)
> - d.setVar('PROVIDES', 'linux-image-' + kernel_name + ' ' + \
> - 'linux-headers-' + kernel_name)
> + distro_arch = d.getVar("DISTRO_ARCH", True)
> + host_arch = d.getVar("HOST_ARCH", True)
> + target_arch = ""
> + kbuild_depends = "linux-kbuild-" + kernel_name
> +
> + # For different distro and host archs
> + # - Set target arch (empty string disables package build)
> + # - Add dependency for sbuild-chroot-target
> + # to allow building arch specific kbuild scripts and tools
> + # - Set correct name of kbuild package dependency
> + if distro_arch != host_arch:
> + d.appendVar("SCHROOT_DEP", " sbuild-chroot-target:do_build")
> + if d.getVar("ISAR_CROSS_COMPILE", True) == "1":
> + target_arch = distro_arch
> + kbuild_depends = kbuild_depends + "-cross | " + kbuild_depends
> +
> + # Set CONTROL_TARGET_ARCH
> + d.setVar("CONTROL_TARGET_ARCH", target_arch)
> +
> + # Make bitbake know we will be producing
> + # linux-image and linux-headers packages
> + d.setVar("PROVIDES", "linux-image-" + kernel_name + " " + \
> + "linux-headers-" + kernel_name)
> +
> + # Set dependency for kbuild package
> + d.appendVar("KERNEL_HEADERS_DEBIAN_DEPENDS", kbuild_depends)
> }
>
> -def get_kernel_arch(d):
> - distro_arch = d.getVar("DISTRO_ARCH")
> - if distro_arch in ["amd64", "i386"]:
> +def get_kernel_arch(d, arch="DISTRO_ARCH"):
> + arch = d.getVar(arch)
> + if arch in ["amd64", "i386"]:
> kernel_arch = "x86"
> - elif distro_arch == "arm64":
> + elif arch == "arm64":
> kernel_arch = "arm64"
> - elif distro_arch == "armhf":
> + elif arch == "armhf":
> kernel_arch = "arm"
> - elif distro_arch == "mipsel":
> + elif arch == "mipsel":
> kernel_arch = "mips"
> - elif distro_arch == "riscv64":
> + elif arch == "riscv64":
> kernel_arch = "riscv"
> else:
> kernel_arch = ""
> return kernel_arch
>
> KERNEL_ARCH ??= "${@get_kernel_arch(d)}"
> +KERNEL_HOST_ARCH ??= "${@get_kernel_arch(d, 'HOST_ARCH')}"
>
> def config_fragments(d):
> fragments = []
> @@ -157,6 +183,7 @@ do_prepare_build_prepend() {
>
> # build directory for our "full" kernel build
> KERNEL_BUILD_DIR = "build-full"
> +KERNEL_KBUILD_DIR = "${KERNEL_BUILD_DIR}-kbuild"
>
> def get_kernel_config_target(d):
> kernel_defconfig = d.getVar('KERNEL_DEFCONFIG', True)
> @@ -187,15 +214,19 @@ def get_kernel_config_fragments(d):
> KERNEL_CONFIG_FRAGMENTS = "${@get_kernel_config_fragments(d)}"
>
> dpkg_configure_kernel() {
> + build_dir="${KERNEL_BUILD_DIR}"
> + if [ -n "${1}" ]; then
> + build_dir="${1}"
> + fi
> grep -q "KERNEL_CONFIG_TARGET=" ${S}/debian/isar/configure ||
> cat << EOF | sed -i '/^do_configure() {/ r /dev/stdin'
> ${S}/debian/isar/configure
> KERNEL_CONFIG_TARGET="${@get_kernel_config_target(d)}"
> EOF
>
> - rm -rf ${S}/${KERNEL_BUILD_DIR} && mkdir -p ${S}/$
{KERNEL_BUILD_DIR}
> + rm -rf ${S}/${build_dir} && mkdir -p ${S}/${build_dir}
> if [ -n "${KERNEL_DEFCONFIG}" ]; then
> if [ -e "${WORKDIR}/${KERNEL_DEFCONFIG}" ]; then
> - cp ${WORKDIR}/${KERNEL_DEFCONFIG} ${S}/$
{KERNEL_BUILD_DIR}/.config
> + cp ${WORKDIR}/${KERNEL_DEFCONFIG} ${S}/$
{build_dir}/.config
> fi
> fi
>
> @@ -210,4 +241,53 @@ EOF
>
> dpkg_runbuild_prepend() {
> dpkg_configure_kernel
> +
> + if [ "${ISAR_CROSS_COMPILE}" = "1" ] && [ "${HOST_ARCH}" !=
> "${DISTRO_ARCH}" ]; then + # Create temporarily kernel
schroot-target
> configuration
> + schroot_create_configs ${SCHROOT_TARGET_DIR}
> +
> + # Begin kernel schroot-target session
> + schroot_session=$(schroot -b -c ${SBUILD_CHROOT})
> +
> + # Install build deps for schroot session
> + schroot -u root -r -c $schroot_session -- sh -c " \
> + cd ${PP}/${PPS}; \
> + mk-build-deps -i -r --host-arch $
{DISTRO_ARCH} -t \"apt-get -y
> --no-install-recommends --allow-downgrades -o
> Debug::pkgProblemResolver=yes\" debian/control; \ + "
> +
> + # Create kernel configuration for kbuild
> + dpkg_configure_kernel ${KERNEL_KBUILD_DIR}-target
> + KERNEL_CONFIG_TARGET="${@get_kernel_config_target(d)}"
> +
> + schroot -r -c $schroot_session -- sh -c " \
> + cd ${PP}/${PPS}; \
> + ARCH=${KERNEL_ARCH} make O=$
{KERNEL_KBUILD_DIR}-target
> ${KERNEL_CONFIG_TARGET}; \ + "
> +
> + cd ${S}
> + ARCH=${KERNEL_ARCH} ./scripts/kconfig/merge_config.sh -m
-O
> ${KERNEL_KBUILD_DIR}-target/ ${KERNEL_KBUILD_DIR}-target/.config
> ${KERNEL_CONFIG_FRAGMENTS} + cd -
> +
> + # Build kernel scripts and tools for target arch in a non-
cross
> way + # Cross building kernel scripts and tools would not provide
> them in target architecture + schroot -r -c
$schroot_session -- sh -c " \
> + cd ${PP}/${PPS}; \
> + ARCH=${KERNEL_ARCH} make O=$
{KERNEL_KBUILD_DIR}-target olddefconfig; \
> + ARCH=${KERNEL_ARCH} make O=$
{KERNEL_KBUILD_DIR}-target ${PARALLEL_MAKE}
> scripts; \ + if [ -d "tools/objtool" ]; then \
> + echo ARCH=${KERNEL_ARCH} make O=$
{KERNEL_KBUILD_DIR}-target
> ${PARALLEL_MAKE} tools/objtool; \ + ARCH=$
{KERNEL_ARCH} make
> O=${KERNEL_KBUILD_DIR}-target ${PARALLEL_MAKE} tools/objtool; \ +
fi; \
> + if grep -q "CONFIG_MODULES=y" $
{KERNEL_KBUILD_DIR}-target/.config; then
> \ + echo ARCH=${KERNEL_ARCH} make O=$
{KERNEL_KBUILD_DIR}-target
> ${PARALLEL_MAKE} modules_prepare; \ +
ARCH=${KERNEL_ARCH} make
> O=${KERNEL_KBUILD_DIR}-target ${PARALLEL_MAKE} modules_prepare; \ +
fi; \
> + "
> +
> + # End kernel schroot-target session
> + schroot -e -c $schroot_session
> +
> + # Restore kernel schroot-host configuration
> + schroot_create_configs
> + fi
> }
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 2/3] linux-custom: Split up binaries from kernel headers to kbuild packages
2022-11-11 5:34 ` Uladzimir Bely
@ 2022-11-11 9:03 ` Koch, Stefan
2022-11-11 10:50 ` Uladzimir Bely
0 siblings, 1 reply; 18+ messages in thread
From: Koch, Stefan @ 2022-11-11 9:03 UTC (permalink / raw)
To: ubely, isar-users; +Cc: Kiszka, Jan
On Fri, 2022-11-11 at 08:34 +0300, Uladzimir Bely wrote:
> In mail from среда, 9 ноября 2022 г. 13:32:48 +03 user Koch, Stefan
> wrote:
> > When using a cross build this patch does introduce
> > host and target specific kernel kbuild packages that
> > ship the "scripts" and "tools" binaries.
> > The kernel headers fulfill this using symlinks to point
> > to the "scripts" and "tools" of the respective kernel kbuild
> > package.
> >
> > Known from doc/custom_kernel.inc:
> > - The kernel headers package has not supported both native
> > and cross compilation of kernel modules when itself was cross
> > built
> > - Future roadmap: Generate kernel headers package for both host
> > and target when using a cross build
> >
> > Known from debian kernel packages structure:
> > - Generate a kernel headers package without binaries
> > - Create specific kernel kbuild packages that
> > ship the "scripts" and "tools" binaries
> > - Using symlinks to point to the "scripts"
> > and "tools" binaries
> >
> > Signed-off-by: Stefan Koch <stefan-koch@siemens.com>
> > ---
> > .../linux/files/debian/control.tmpl | 18 ++-
> > .../linux/files/debian/isar/build.tmpl | 9 +-
> > .../linux/files/debian/isar/common.tmpl | 7 +-
> > .../linux/files/debian/isar/install.tmpl | 64 +++++++++--
> > .../linux/files/debian/rules.tmpl | 5 +-
> > meta/recipes-kernel/linux/linux-custom.inc | 104
> > ++++++++++++++++--
> > 6 files changed, 178 insertions(+), 29 deletions(-)
> >
> > diff --git a/meta/recipes-kernel/linux/files/debian/control.tmpl
> > b/meta/recipes-kernel/linux/files/debian/control.tmpl index
> > dd0b624..adac4ef 100644
> > --- a/meta/recipes-kernel/linux/files/debian/control.tmpl
> > +++ b/meta/recipes-kernel/linux/files/debian/control.tmpl
> > @@ -26,7 +26,7 @@ Section: devel
> > Provides: linux-kernel-headers
> > Architecture: any
> > Description: Linux support headers for userspace development
> > - This package provides userspaces headers from the Linux kernel.
> > These
> > headers + This package provides userspaces headers from the Linux
> > kernel.
> > These headers are used by the installed headers for GNU glibc and
> > other
> > system libraries.
> >
> > Package: linux-image-${KERNEL_NAME_PROVIDED}-dbg
> > @@ -35,3 +35,19 @@ Architecture: any
> > Description: Linux kernel debugging symbols for @KR@
> > This package will come in handy if you need to debug the kernel.
> > It
> > provides all the necessary debug symbols for the kernel and its
> > modules.
> > +
> > +Package: linux-kbuild-${KERNEL_NAME_PROVIDED}
> > +Architecture: any
> > +Depends: ${KERNEL_HEADERS_DEBIAN_DEPENDS}, ${perl:Depends},
> > ${shlib:Depends} +Description: ${KERNEL_NAME_PROVIDED} Linux kbuild
> > scripts
> > and tools for @KR@ + This package provides kernel kbuild scripts
> > and tools
> > for @KR@
> > + This is useful for people who need to build external modules
> > +
> > +Package: linux-kbuild-${KERNEL_NAME_PROVIDED}-cross
> > +Architecture: ${CONTROL_TARGET_ARCH}
>
> Is this architecture correct? Below, it's set to target_arch =
> distro_arch for
> cross-compile case.
>
> I looked into `linux-kbuild-mainline-cross_5.4.203+r0_armhf.deb` and
> there are
> some binaries inside, "ELF 64-bit LSB pie executable, x86-64", so it
> looks for
> me that the package is supposed to be installed on host machine and
> to have
> the host architecture instead.
I think your `linux-headers-mainline_5.4.203+r0_armhf.deb` is also
armhf? Without the patch there are the same x86-64 binaries... They
have moved from linux-headers:armhf to linux-kbuild-cross:armhf.
Might it be better to have linux-headers:all instead of :armhf?
Since the binaries are out? But debian uses for their header packages
not :all even if they have split out the binaries...
I would be appreciative for a tip to getting the dpkg build not to
complain, when we would change it to linux-kbuild-cross:amd64
Keeping and not fixing the existing limitations was the goal here...
See comment below, too.
>
> > +Depends: ${KERNEL_HEADERS_DEBIAN_DEPENDS}, ${perl:Depends},
> > ${shlib:Depends} +Description: ${KERNEL_NAME_PROVIDED} Linux kbuild
> > scripts
> > and tools for @KR@ + This package provides kernel kbuild scripts
> > and tools
> > + as ${HOST_ARCH} cross binaries for @KR@
> > + This is useful for those who need to cross build
> > + external modules using ISAR's sbuild-chroot-host
> > diff --git a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl
> > b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl index
> > 94cfbe0..7e2daad 100644
> > --- a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl
> > +++ b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl
> > @@ -13,16 +13,19 @@ do_build() {
> > # Trace what we do here
> > set -x
> >
> > + # Set correct build directory for kernel or kbuild (kernel
> > scripts and
> > tools) cases + build_dir=${KERNEL_BUILD_DIR}
> > +
> > # Process existing kernel configuration to make sure it is
> > complete
> > # (use defaults for options that were not specified)
> > - ${MAKE} O=${KERNEL_BUILD_DIR} olddefconfig prepare
> > + ${MAKE} O=${build_dir} olddefconfig prepare
> >
> > # Transfer effective kernel version into control file and
> > scripts
> > - KR=$(${MAKE} O=${KERNEL_BUILD_DIR} -s --no-print-directory
> > kernelrelease) + KR=$(${MAKE} O=${build_dir} -s --no-print-
> > directory
> > kernelrelease) sed -i "s/@KR@/${KR}/g" ${S}/debian/control
> > ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.*
> >
> > # Build the Linux kernel
> > - ${MAKE} O=${KERNEL_BUILD_DIR} ${PARALLEL_MAKE}
> > KCFLAGS="${KCFLAGS}"
> > + ${MAKE} O=${build_dir} ${PARALLEL_MAKE} KCFLAGS="${KCFLAGS}"
> >
> > # Stop tracing
> > set +x
> > diff --git a/meta/recipes-
> > kernel/linux/files/debian/isar/common.tmpl
> > b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl index
> > 52ebebb..55d6123 100644
> > --- a/meta/recipes-kernel/linux/files/debian/isar/common.tmpl
> > +++ b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl
> > @@ -5,10 +5,14 @@
> > set -e
> >
> > # Isar settings
> > -ARCH=${KERNEL_ARCH}
> > KERNEL_PKG_IMAGE=linux-image-${KERNEL_NAME_PROVIDED}
> > KERNEL_PKG_KERN_HEADERS=linux-headers-${KERNEL_NAME_PROVIDED}
> > KERNEL_PKG_LIBC_HEADERS=linux-libc-dev
> > +KERNEL_PKG_KERN_KBUILD=linux-kbuild-${KERNEL_NAME_PROVIDED}
> > +
> > +if [ -z "${ARCH}" ]; then
> > + ARCH=${KERNEL_ARCH}
> > +fi
> >
> > # Constants
> > KCONF=.config
> > @@ -19,6 +23,7 @@ deb_img_dir=${deb_top_dir}/${KERNEL_PKG_IMAGE}
> > deb_dbg_dir=${deb_img_dir}-dbg
> > deb_kern_hdr_dir=${deb_top_dir}/${KERNEL_PKG_KERN_HEADERS}
> > deb_libc_hdr_dir=${deb_top_dir}/${KERNEL_PKG_LIBC_HEADERS}
> > +deb_kern_kbuild_dir=${deb_top_dir}/${KERNEL_PKG_KERN_KBUILD}
> >
> > # Array of packages to be generated
> > declare -A kern_pkgs
> > diff --git a/meta/recipes-
> > kernel/linux/files/debian/isar/install.tmpl
> > b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl index
> > 0a8645d..890a996 100644
> > --- a/meta/recipes-kernel/linux/files/debian/isar/install.tmpl
> > +++ b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl
> > @@ -33,6 +33,7 @@ do_install() {
> > # Trace what we do here
> > set -x
> >
> > + # Run the install steps
> > install_image
> > if [ "${ARCH}" != "um" ]; then
> > install_config
> > @@ -43,6 +44,19 @@ do_install() {
> > install_kmods
> > install_headers
> >
> > + if [ -d "$(dirname ${O})/build-full-kbuild-target" ]; then
> > + # Install cross kernel scripts and tools
> > + install_kbuild ${deb_kern_kbuild_dir}-cross build-full
> > +
> > + # Cleanup and install kernel scripts and tools for target
> > arch
> > + rm -rf ${deb_kern_kbuild_dir}
> > + install_kbuild ${deb_kern_kbuild_dir} build-full-kbuild-
> > target
> > + else
> > + # Cleanup and install kernel scripts and tools
> > + rm -rf ${deb_kern_kbuild_dir}
> > + install_kbuild ${deb_kern_kbuild_dir} build-full
> > + fi
> > +
> > # Stop tracing
> > set +x
> > }
> > @@ -168,21 +182,15 @@ kernel_headers() {
> > mkdir -p ${destdir}
> > mkdir -p ${deb_kern_hdr_dir}/lib/modules/${krel}
> >
> > - (cd ${S}; find . -name 'Makefile*' -o -name 'Kconfig*' -o -
> > name '*.pl')
> > > > ${src_hdr_files} - (cd ${S}; find arch/*/include include
> > > > scripts -type
> > f -o -type l) >>${src_hdr_files} + (cd ${S}; find . -not -path
> > './scripts/*' -a -not -path './tools/*' -a \( -name 'Makefile*' -o
> > -name
> > 'Kconfig*' -o -name '*.pl' \)) >>${src_hdr_files} + (cd ${S};
> > find
> > arch/*/include include -type f -o -type l) >>${src_hdr_files} (cd
> > ${S};
> > find arch/${ARCH} -name module.lds -o -name Kbuild.platforms -o -
> > name
> > Platform) >>${src_hdr_files} (cd ${S}; find $(find arch/${ARCH} -
> > name
> > include -o -name scripts -type d) -type f) >>${src_hdr_files}
> >
> > if [ -n "${CONFIG_MODULES}" ]; then
> > echo Module.symvers >> ${obj_hdr_files}
> > fi
> > - (cd ${O}; find arch/${ARCH}/include include scripts -type f)
> > > > ${obj_hdr_files} - if [ -n "${CONFIG_STACK_VALIDATION}" ];
> > > > then
> > - (cd ${O}; find tools/objtool -type f -executable)
> > > > ${obj_hdr_files} - fi
> > - if [ -n "${CONFIG_GCC_PLUGINS}" ]; then
> > - (cd ${O}; find scripts/gcc-plugins -name *.so -o -name
> > gcc-common.h) >>${obj_hdr_files} - fi
> > + (cd ${O}; find arch/${ARCH}/include include -type f)
> > >>${obj_hdr_files}
> >
> > # deploy files that were matched above
> > tar -C ${S} -cf - -T - <${src_hdr_files} | tar -C ${destdir} -
> > xf -
> > @@ -191,8 +199,11 @@ kernel_headers() {
> > # add the kernel config
> > cp ${O}/${KCONF} ${destdir}/.config
> >
> > - # handle kernel development tools
> > - kernel_tools
> > + # add symlink to scripts and tools directories
> > + ln -sf ../../lib/linux-kbuild-${krel}/scripts
> > ${destdir}/scripts
> > + if [ -n "${CONFIG_STACK_VALIDATION}" ]; then
> > + ln -sf ../../lib/linux-kbuild-${krel}/tools
> > ${destdir}/tools
> > + fi
> >
> > # create symlinks
> > ln -sf /${kernel_headers_dir}
> > ${deb_kern_hdr_dir}/lib/modules/${krel}/build @@ -206,4 +217,35 @@
> > install_headers() {
> > kernel_headers
> > }
> >
> > +install_kbuild() {
> > + kernel_kbuild_dir=usr/lib/linux-kbuild-${krel}
> > + destdir=${1}/${kernel_kbuild_dir}
> > + src_kbuild_files=$(mktemp)
> > + obj_kbuild_files=$(mktemp)
> > +
> > + if [ -n "${2}" ]; then
> > + O=$(dirname ${O})/${2}
> > + fi
> > +
> > + mkdir -p ${destdir}
> > +
> > + (cd ${S}; find . -path './scripts/*' -a -path './tools/*' -a
> > \( -name
> > 'Makefile*' -o -name 'Kconfig*' -o -name '*.pl' \))
> > >>${src_kbuild_files} +
> > (cd ${S}; find scripts -type f -o -type l) >>${src_kbuild_files}
> > +
> > + (cd ${O}; find scripts -type f) >>${obj_kbuild_files}
> > + if [ -n "${CONFIG_STACK_VALIDATION}" ]; then
> > + (cd ${O}; find tools/objtool -type f -executable)
> > > > ${obj_kbuild_files} + fi
> > + if [ -n "${CONFIG_GCC_PLUGINS}" ]; then
> > + (cd ${O}; find scripts/gcc-plugins -name *.so -o -name
> > gcc-common.h) >>${obj_kbuild_files} + fi
> > +
> > + # deploy files that were matched above
> > + tar -C ${S} -cf - -T - <${src_kbuild_files} | tar -C
> > ${destdir} -xf -
> > + tar -C ${O} -cf - -T - <${obj_kbuild_files} | tar -C
> > ${destdir} -xf -
> > +
> > + # handle kernel development tools
> > + kernel_tools
> > +}
> > +
> > main install ${*}
> > diff --git a/meta/recipes-kernel/linux/files/debian/rules.tmpl
> > b/meta/recipes-kernel/linux/files/debian/rules.tmpl index
> > 8063c49..e131288
> > 100755
> > --- a/meta/recipes-kernel/linux/files/debian/rules.tmpl
> > +++ b/meta/recipes-kernel/linux/files/debian/rules.tmpl
> > @@ -36,4 +36,7 @@ override_dh_auto_test:
> > true
> >
> > override_dh_strip:
> > - unset DEB_HOST_GNU_TYPE && dh_strip -Xvmlinu
That line was because `linux-headers-mainline_5.4.203+r0_armhf.deb` is
also armhf and contained x86-64 binaries...
> > --no-automatic-dbgsym
> > + dh_strip -Xvmlinu -Xlinux-kbuild-${KERNEL_NAME_PROVIDED}-
> > cross
> > --no-automatic-dbgsym +
> > +override_dh_shlibdeps:
> > + dh_shlibdeps -Xlinux-kbuild-${KERNEL_NAME_PROVIDED}
> > diff --git a/meta/recipes-kernel/linux/linux-custom.inc
> > b/meta/recipes-kernel/linux/linux-custom.inc index 96f0afc..0f8ba50
> > 100644
> > --- a/meta/recipes-kernel/linux/linux-custom.inc
> > +++ b/meta/recipes-kernel/linux/linux-custom.inc
> > @@ -74,12 +74,14 @@ TEMPLATE_VARS += " \
> > KERNEL_ARCH \
> > KERNEL_DEBIAN_DEPENDS \
> > KERNEL_BUILD_DIR \
> > + KERNEL_KBUILD_DIR \
> > KERNEL_FILE \
> > KERNEL_HEADERS_DEBIAN_DEPENDS \
> > LINUX_VERSION_EXTENSION \
> > KERNEL_NAME_PROVIDED \
> > KERNEL_CONFIG_FRAGMENTS \
> > KCFLAGS \
> > + CONTROL_TARGET_ARCH \
> > "
> >
> > inherit dpkg
> > @@ -91,30 +93,54 @@ KCFLAGS ?= "-fdebug-prefix-map=${CURDIR}=."
> > # Derive name of the kernel packages from the name of this recipe
> > KERNEL_NAME_PROVIDED ?= "${@ d.getVar('PN',
> > True).partition('linux-')[2]}"
> >
> > -# Make bitbake know we will be producing linux-image and linux-
> > headers
> > packages python() {
> > kernel_name = d.getVar("KERNEL_NAME_PROVIDED", True)
> > - d.setVar('PROVIDES', 'linux-image-' + kernel_name + ' ' + \
> > - 'linux-headers-' + kernel_name)
> > + distro_arch = d.getVar("DISTRO_ARCH", True)
> > + host_arch = d.getVar("HOST_ARCH", True)
> > + target_arch = ""
> > + kbuild_depends = "linux-kbuild-" + kernel_name
> > +
> > + # For different distro and host archs
> > + # - Set target arch (empty string disables package build)
> > + # - Add dependency for sbuild-chroot-target
> > + # to allow building arch specific kbuild scripts and tools
> > + # - Set correct name of kbuild package dependency
> > + if distro_arch != host_arch:
> > + d.appendVar("SCHROOT_DEP", " sbuild-chroot-
> > target:do_build")
> > + if d.getVar("ISAR_CROSS_COMPILE", True) == "1":
> > + target_arch = distro_arch
> > + kbuild_depends = kbuild_depends + "-cross | " +
> > kbuild_depends
> > +
> > + # Set CONTROL_TARGET_ARCH
> > + d.setVar("CONTROL_TARGET_ARCH", target_arch)
> > +
> > + # Make bitbake know we will be producing
> > + # linux-image and linux-headers packages
> > + d.setVar("PROVIDES", "linux-image-" + kernel_name + " " + \
> > + "linux-headers-" + kernel_name)
> > +
> > + # Set dependency for kbuild package
> > + d.appendVar("KERNEL_HEADERS_DEBIAN_DEPENDS", kbuild_depends)
> > }
> >
> > -def get_kernel_arch(d):
> > - distro_arch = d.getVar("DISTRO_ARCH")
> > - if distro_arch in ["amd64", "i386"]:
> > +def get_kernel_arch(d, arch="DISTRO_ARCH"):
> > + arch = d.getVar(arch)
> > + if arch in ["amd64", "i386"]:
> > kernel_arch = "x86"
> > - elif distro_arch == "arm64":
> > + elif arch == "arm64":
> > kernel_arch = "arm64"
> > - elif distro_arch == "armhf":
> > + elif arch == "armhf":
> > kernel_arch = "arm"
> > - elif distro_arch == "mipsel":
> > + elif arch == "mipsel":
> > kernel_arch = "mips"
> > - elif distro_arch == "riscv64":
> > + elif arch == "riscv64":
> > kernel_arch = "riscv"
> > else:
> > kernel_arch = ""
> > return kernel_arch
> >
> > KERNEL_ARCH ??= "${@get_kernel_arch(d)}"
> > +KERNEL_HOST_ARCH ??= "${@get_kernel_arch(d, 'HOST_ARCH')}"
> >
> > def config_fragments(d):
> > fragments = []
> > @@ -157,6 +183,7 @@ do_prepare_build_prepend() {
> >
> > # build directory for our "full" kernel build
> > KERNEL_BUILD_DIR = "build-full"
> > +KERNEL_KBUILD_DIR = "${KERNEL_BUILD_DIR}-kbuild"
> >
> > def get_kernel_config_target(d):
> > kernel_defconfig = d.getVar('KERNEL_DEFCONFIG', True)
> > @@ -187,15 +214,19 @@ def get_kernel_config_fragments(d):
> > KERNEL_CONFIG_FRAGMENTS = "${@get_kernel_config_fragments(d)}"
> >
> > dpkg_configure_kernel() {
> > + build_dir="${KERNEL_BUILD_DIR}"
> > + if [ -n "${1}" ]; then
> > + build_dir="${1}"
> > + fi
> > grep -q "KERNEL_CONFIG_TARGET=" ${S}/debian/isar/configure
> > ||
> > cat << EOF | sed -i '/^do_configure() {/ r
> > /dev/stdin'
> > ${S}/debian/isar/configure
> > KERNEL_CONFIG_TARGET="${@get_kernel_config_target(d)}"
> > EOF
> >
> > - rm -rf ${S}/${KERNEL_BUILD_DIR} && mkdir -p ${S}/$
> {KERNEL_BUILD_DIR}
> > + rm -rf ${S}/${build_dir} && mkdir -p ${S}/${build_dir}
> > if [ -n "${KERNEL_DEFCONFIG}" ]; then
> > if [ -e "${WORKDIR}/${KERNEL_DEFCONFIG}" ]; then
> > - cp ${WORKDIR}/${KERNEL_DEFCONFIG} ${S}/$
> {KERNEL_BUILD_DIR}/.config
> > + cp ${WORKDIR}/${KERNEL_DEFCONFIG} ${S}/$
> {build_dir}/.config
> > fi
> > fi
> >
> > @@ -210,4 +241,53 @@ EOF
> >
> > dpkg_runbuild_prepend() {
> > dpkg_configure_kernel
> > +
> > + if [ "${ISAR_CROSS_COMPILE}" = "1" ] && [ "${HOST_ARCH}" !=
> > "${DISTRO_ARCH}" ]; then + # Create temporarily kernel
> schroot-target
> > configuration
> > + schroot_create_configs ${SCHROOT_TARGET_DIR}
> > +
> > + # Begin kernel schroot-target session
> > + schroot_session=$(schroot -b -c ${SBUILD_CHROOT})
> > +
> > + # Install build deps for schroot session
> > + schroot -u root -r -c $schroot_session -- sh -c " \
> > + cd ${PP}/${PPS}; \
> > + mk-build-deps -i -r --host-arch $
> {DISTRO_ARCH} -t \"apt-get -y
> > --no-install-recommends --allow-downgrades -o
> > Debug::pkgProblemResolver=yes\" debian/control; \ + "
> > +
> > + # Create kernel configuration for kbuild
> > + dpkg_configure_kernel ${KERNEL_KBUILD_DIR}-target
> > + KERNEL_CONFIG_TARGET="${@get_kernel_config_target(d
> > )}"
> > +
> > + schroot -r -c $schroot_session -- sh -c " \
> > + cd ${PP}/${PPS}; \
> > + ARCH=${KERNEL_ARCH} make O=$
> {KERNEL_KBUILD_DIR}-target
> > ${KERNEL_CONFIG_TARGET}; \ + "
> > +
> > + cd ${S}
> > + ARCH=${KERNEL_ARCH}
> > ./scripts/kconfig/merge_config.sh -m
> -O
> > ${KERNEL_KBUILD_DIR}-target/ ${KERNEL_KBUILD_DIR}-target/.config
> > ${KERNEL_CONFIG_FRAGMENTS} + cd -
> > +
> > + # Build kernel scripts and tools for target arch in
> > a non-
> cross
> > way + # Cross building kernel scripts and tools would not
> > provide
> > them in target architecture + schroot -r -c
> $schroot_session -- sh -c " \
> > + cd ${PP}/${PPS}; \
> > + ARCH=${KERNEL_ARCH} make O=$
> {KERNEL_KBUILD_DIR}-target olddefconfig; \
> > + ARCH=${KERNEL_ARCH} make O=$
> {KERNEL_KBUILD_DIR}-target ${PARALLEL_MAKE}
> > scripts; \ + if [ -d "tools/objtool" ]; then \
> > + echo ARCH=${KERNEL_ARCH} make O=$
> {KERNEL_KBUILD_DIR}-target
> > ${PARALLEL_MAKE} tools/objtool; \
> > + ARCH=$
> {KERNEL_ARCH} make
> > O=${KERNEL_KBUILD_DIR}-target ${PARALLEL_MAKE} tools/objtool; \
> > +
> fi; \
> > + if grep -q "CONFIG_MODULES=y" $
> {KERNEL_KBUILD_DIR}-target/.config; then
> > \ + echo ARCH=${KERNEL_ARCH} make O=$
> {KERNEL_KBUILD_DIR}-target
> > ${PARALLEL_MAKE} modules_prepare; \ +
> ARCH=${KERNEL_ARCH} make
> > O=${KERNEL_KBUILD_DIR}-target ${PARALLEL_MAKE} modules_prepare; \
> > +
> fi; \
> > + "
> > +
> > + # End kernel schroot-target session
> > + schroot -e -c $schroot_session
> > +
> > + # Restore kernel schroot-host configuration
> > + schroot_create_configs
> > + fi
> > }
>
>
>
>
--
Stefan Koch
Siemens AG
www.siemens.com
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 0/3] linux-custom: Split up binaries from kernel headers to kbuild packages
2022-11-10 18:36 ` Jan Kiszka
@ 2022-11-11 9:47 ` Koch, Stefan
0 siblings, 0 replies; 18+ messages in thread
From: Koch, Stefan @ 2022-11-11 9:47 UTC (permalink / raw)
To: isar-users, Kiszka, Jan
Cc: Sudler, Simon, ubely, Storm, Christian, Adler, Michael
On Thu, 2022-11-10 at 19:36 +0100, Jan Kiszka wrote:
> On 10.11.22 19:33, Jan Kiszka wrote:
> > On 10.11.22 18:49, Koch, Stefan (DI PA DCP R&D 3) wrote:
> > > On Wed, 2022-11-09 at 17:00 +0100, Jan Kiszka wrote:
> > > > On 09.11.22 16:06, Koch, Stefan (DI PA DCP R&D 3) wrote:
> > > > > On Wed, 2022-11-09 at 12:50 +0100, Jan Kiszka wrote:
> > > > > > On 09.11.22 11:32, Koch, Stefan (DI PA DCP R&D 3) wrote:
> > > > > > > Hi
> > > > > > >
> > > > > > > This patchset is a set of three patches:
> > > > > > > - Support overwriting configured schroot dir
> > > > > > > - Split up binaries from kernel headers to kbuild
> > > > > > > packages
> > > > > > > - Update custom_kernel docs for split up of kernel
> > > > > > > scripts and
> > > > > > > tools
> > > > > > >
> > > > > > > The main use-case was to swap out the binaries
> > > > > > > from the kernel headers package.
> > > > > > > They will be swapped out into host and target
> > > > > > > specific kernel kbuild packages.
> > > > > > >
> > > > > > > The main development goals were these:
> > > > > > >
> > > > > > > 1. Solve already known isar custom kernel
> > > > > > > limitations from doc/custom_kernel.inc
> > > > > > > - kernel headers package does not support both native
> > > > > > > and cross compilation of kernel modules when cross
> > > > > > > built
> > > > > >
> > > > > > Right, that requires building packages for multiple
> > > > > > architectures.
> > > > >
> > > > > Important feature... Maybe it should be added to the TODO
> > > > > list in
> > > > > the
> > > > > docs?
> > > > >
> > > > > > This
> > > > > > is the feature we need, and it has nothing to do with the
> > > > > > kernel
> > > > > > packages per se.
> > > > >
> > > > > The way that I assume debian will do building the kernel is
> > > > > that
> > > > > they
> > > > > use a generic debian defconfig for each architecture.
> > > > > E. g. linux-config-5.10/config.amd64_none_amd64.xz, linux-
> > > > > config-
> > > > > 5.10/config.arm64_none_arm64.xz
> > > > >
> > > > > ISAR builds are often using specific defconfigs honored by
> > > > > ISAR's
> > > > > linux-custom for the target build.
> > > > >
> > > > > Debian specific architecture builds (e. g. host and target,
> > > > > maybe
> > > > > with
> > > > > sbuild) for one target device, using target specific
> > > > > stripped-down
> > > > > defconfigs, may not working properly. The kernel build for a
> > > > > foreign
> > > > > architecture (e. g. host) may fail because of an incompatible
> > > > > defconfig.
> > > > >
> > > > > Debian provide a generic very similar defconfig for different
> > > > > architectures, and they build the linux-image, linux-headers
> > > > > and
> > > > > linux-
> > > > > kbuild for each architecture:
> > > > > - So I assume the linux-kbuild:amd64 was build with linux-
> > > > > config-
> > > > > 5.10/config.amd64_none_amd64.xz
> > > > > - So I assume the linux-kbuild:arm64 was build with linux-
> > > > > config-
> > > > > 5.10/config.arm64_none_arm64.xz
> > > > >
> > > > > The debian generic defconfig may differ in some arch specific
> > > > > points
> > > > > but you can use the linux-kbuild:amd64 for a cross-build of
> > > > > linux-
> > > > > image:arm64 because the resulting scripts and tools are from
> > > > > the
> > > > > identical set. So mixing linux-headers with different
> > > > > archictecture
> > > > > specific builds from linux-kbuild should work since the
> > > > > defconfigs
> > > > > have
> > > > > a "shared base" (e. g. both define CONFIG_HAVE_C_RECORDMCOUNT
> > > > > thats
> > > > > needed for recordmcount utility as example).
> > > >
> > > > Sorry, I don't understand what role the config plays here. Isar
> > > > is
> > > > used
> > > > to compile (usually) one specific kernel, Debian compiles a few
> > > > generic
> > > > ones. But the fact that Debian compile everything natively (for
> > > > production) and we do cross-compilation is not related to
> > > > different
> > > > configs.
> > > >
> > >
> > > I just wanted to point out for the future case when ISAR would
> > > support
> > > a generic way to build packages for multiple architectures: This
> > > might
> > > not be applicable to all kernel builds. E. g. because of the fact
> > > that
> > > the used defconfig for a "specific kernel" is not compatible for
> > > a
> > > multi architecture build, only for the architecture that the
> > > defconfig
> > > was made for.
> > >
> > > Yes, it may work if you could specify different defconfigs for
> > > each
> > > architecture. One goal of this patch was to provide the kbuild
> > > scripts
> > > and tools just for host and target in a transparent way - without
> > > any
> > > changes to the build process of existing ISAR users.
> >
> > Misunderstanding: I didn't meant to propose building the kernel-
> > image
> > package for multiple arch, only the then factored-out kbuild
> > packages
> > with the host tool (once for the build env, once for the target).
> >
Maybe then some logic that detects cross-build could use for the target
build with "make scripts tools" only...
> > >
> > > > The answer to the problem "I need kernel headers on the target"
> > > > right
> > > > now is in Isar to build natively the kernel and, thus, the
> > > > header
> > > > package. Not optimal, but we still need this case too rarely to
> > > > have
> > > > cared about it more so far.
> > > >
> > > > >
> > > > > > Jan
> > > > > >
> > > > > > >
> > > > > > > 2. Honor recommendations for future from
> > > > > > > doc/custom_kernel.inc
> > > > > > > - Generate kernel headers packages for both host and
> > > > > > > target
> > > > > > > when using cross build
> > > > > > >
> > > > > > > 3. Add extensions known from debian kernel packages
> > > > > > > structure
> > > > > > > - Generate a kernel headers package without binaries
> > > > > > > - Create specific kernel kbuild packages that
> > > > > > > will ship the "scripts" and "tools" binaries
> > > > > > > - Use symlinks to point to the "scripts" and "tools"
> > > > > > > binaries
> > > > > >
> > > > > > Keep in mind that we intend to remain compatible with the
> > > > > > layout
> > > > > > of
> > > > > > Debian. Self-built kernels should remain drop-in
> > > > > > replacements of
> > > > > > the
> > > > > > Debian kernel packages. And that is true for development
> > > > > > packages
> > > > > > as
> > > > > > well.
> > > > > >
> > > > > > So, are your changes working against that, or are they
> > > > > > improving
> > > > > > existing incompatibility with the debian packages?
> > > > >
> > > > > The approach from this patchset does build the kernel in the
> > > > > known
> > > > > ISAR
> > > > > specific way. Debian itself patches the kernel Makefiles to
> > > > > modularize
> > > > > the kernel build what ISAR doesn't do. With this patch the
> > > > > packages
> > > > > linux-kbuild and linux-kbuild-cross are the only new packages
> > > > > that
> > > > > the
> > > > > ISAR build will provide. That means ISAR will provide then
> > > > > linux-
> > > > > headers, linux-image and linux-kbuild. They match on the
> > > > > target
> > > > > with
> > > > > the package name of the respective debian packages. So the
> > > > > existing
> > > > > drop-in should not be affected.
> > > > >
> > > > > In short this patch brings the ISAR linux-custom mechanism a
> > > > > bit
> > > > > nearer
> > > > > to the debian package structure. Since linux-headers behaves
> > > > > then
> > > > > more
> > > > > like the original debian linux-heades to ship the binaries
> > > > > within a
> > > > > separate package -> linux-kbuild.
> > > > >
> > > > > The trick is to build only the scripts and tools with
> > > > > schroot-
> > > > > target
> > > > > (QEMU) in a non-cross way. Since the kernel does not support
> > > > > building
> > > > > the scripts and tools for target architecture when do a cross
> > > > > build
> > > > > without patching the kernel Makefiles. I think that patch was
> > > > > never
> > > > > merged into the kernel:
> > > > >
> > > > >
> > > > > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpatchwork.kernel.org%2Fproject%2Flinux-kbuild%2Fpatch%2F1376046432-12588-1-git-send-email-broonie%40kernel.org%2F&data=05%7C01%7Cstefan-koch%40siemens.com%7C8ac82ead88ae49e7854308dac34a8659%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C638037022082153110%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=MgLc%2Bh9ygiMGt2sXXl2CT9rPscIaFFr1f4Y2s5xqGIs%3D&reserved=0
> > > > >
> > > >
> > > > The problem is that patch might have to not provide that use
> > > > case I
> > > > wrote about above. It is a rare use case, a non-distro use
> > > > case, but
> > > > I
> > > > wouldn't call it an invalid one. If we can address it by
> > > > splitting up
> > > > the Isar kernel build into different packages and then only
> > > > compile
> > > > the
> > > > binary bits once again, now for the target architecture, that
> > > > would
> > > > be a
> > > > nice solution.
> > >
> > > That's what the patch do:
> > > - Move the binary scripts and tools from linux-headers to linux-
> > > kbuild
> > > as debian does it for their kernels
> > > - Compile only the scripts and tools for the target using
> > > schroot-
> > > target (QEMU) and package it as linux-kbuild
> > > - Package the binary scripts and tools from the full cross kernel
> > > build
> > > to linux-kbuild-cross using schroot-host
> >
> > It's moving in a good direction - but why linux-kbuild-cross?
> > Nothing
> > depends on it in your patches, thus module builds will be broken.
Module build should work since linux-headers package depends on "linux-
kbuild-cross | linux-kbuild"
I have successfully cross build some module using the linux-module
recipe...
> > I
> > suspect we rather need
> >
> > KERNEL_PKG_KERN_KBUILD=linux-kbuild-
> > ${KERNEL_NAME_PROVIDED}:<native>
> >
> > and therefore (on amd64) linux-kbuild-${KERNEL_NAME_PROVIDED}:amd64
> > to
> > be built and then installed for module builds.
> >
See my answer to [PATCH 2/3]...
> > Jan
> >
>
> And regarding structure of the series: please split up the creation
> of
> kbuild package from also building it for the target arch - if
> possible.
- Move the binary scripts and tools from linux-headers to linux-kbuild
as debian does it for their kernels
- Compile only the scripts and tools for the target using schroot-
target (QEMU) and package it as linux-kbuild
I would suggest this structure using two instead of one patch... Should
the target kbuild creation be toggleable? When yes, enabled or disabled
by default?
>
> Jan
>
--
Stefan Koch
Siemens AG
www.siemens.com
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 2/3] linux-custom: Split up binaries from kernel headers to kbuild packages
2022-11-11 9:03 ` Koch, Stefan
@ 2022-11-11 10:50 ` Uladzimir Bely
0 siblings, 0 replies; 18+ messages in thread
From: Uladzimir Bely @ 2022-11-11 10:50 UTC (permalink / raw)
To: Koch, Stefan; +Cc: isar-users
In mail from пятница, 11 ноября 2022 г. 12:03:40 +03 You wrote:
> On Fri, 2022-11-11 at 08:34 +0300, Uladzimir Bely wrote:
>
> > In mail from среда, 9 ноября 2022 г. 13:32:48 +03 user Koch, Stefan
> > wrote:
> >
> > > When using a cross build this patch does introduce
> > > host and target specific kernel kbuild packages that
> > > ship the "scripts" and "tools" binaries.
> > > The kernel headers fulfill this using symlinks to point
> > > to the "scripts" and "tools" of the respective kernel kbuild
> > > package.
> > >
> > > Known from doc/custom_kernel.inc:
> > > - The kernel headers package has not supported both native
> > > and cross compilation of kernel modules when itself was cross
> > > built
> > > - Future roadmap: Generate kernel headers package for both host
> > > and target when using a cross build
> > >
> > > Known from debian kernel packages structure:
> > > - Generate a kernel headers package without binaries
> > > - Create specific kernel kbuild packages that
> > > ship the "scripts" and "tools" binaries
> > > - Using symlinks to point to the "scripts"
> > > and "tools" binaries
> > >
> > > Signed-off-by: Stefan Koch <stefan-koch@siemens.com>
> > > ---
> > > .../linux/files/debian/control.tmpl | 18 ++-
> > > .../linux/files/debian/isar/build.tmpl | 9 +-
> > > .../linux/files/debian/isar/common.tmpl | 7 +-
> > > .../linux/files/debian/isar/install.tmpl | 64 +++++++++--
> > > .../linux/files/debian/rules.tmpl | 5 +-
> > > meta/recipes-kernel/linux/linux-custom.inc | 104
> > > ++++++++++++++++--
> > > 6 files changed, 178 insertions(+), 29 deletions(-)
> > >
> > > diff --git a/meta/recipes-kernel/linux/files/debian/control.tmpl
> > > b/meta/recipes-kernel/linux/files/debian/control.tmpl index
> > > dd0b624..adac4ef 100644
> > > --- a/meta/recipes-kernel/linux/files/debian/control.tmpl
> > > +++ b/meta/recipes-kernel/linux/files/debian/control.tmpl
> > > @@ -26,7 +26,7 @@ Section: devel
> > > Provides: linux-kernel-headers
> > > Architecture: any
> > > Description: Linux support headers for userspace development
> > > - This package provides userspaces headers from the Linux kernel.
> > > These
> > > headers + This package provides userspaces headers from the Linux
> > > kernel.
> > > These headers are used by the installed headers for GNU glibc and
> > > other
> > > system libraries.
> > >
> > > Package: linux-image-${KERNEL_NAME_PROVIDED}-dbg
> > > @@ -35,3 +35,19 @@ Architecture: any
> > > Description: Linux kernel debugging symbols for @KR@
> > > This package will come in handy if you need to debug the kernel.
> > > It
> > > provides all the necessary debug symbols for the kernel and its
> > > modules.
> > > +
> > > +Package: linux-kbuild-${KERNEL_NAME_PROVIDED}
> > > +Architecture: any
> > > +Depends: ${KERNEL_HEADERS_DEBIAN_DEPENDS}, ${perl:Depends},
> > > ${shlib:Depends} +Description: ${KERNEL_NAME_PROVIDED} Linux kbuild
> > > scripts
> > > and tools for @KR@ + This package provides kernel kbuild scripts
> > > and tools
> > > for @KR@
> > > + This is useful for people who need to build external modules
> > > +
> > > +Package: linux-kbuild-${KERNEL_NAME_PROVIDED}-cross
> > > +Architecture: ${CONTROL_TARGET_ARCH}
> >
> >
> > Is this architecture correct? Below, it's set to target_arch =
> > distro_arch for
> > cross-compile case.
> >
> > I looked into `linux-kbuild-mainline-cross_5.4.203+r0_armhf.deb` and
> > there are
> > some binaries inside, "ELF 64-bit LSB pie executable, x86-64", so it
> > looks for
> > me that the package is supposed to be installed on host machine and
> > to have
> > the host architecture instead.
>
>
> I think your `linux-headers-mainline_5.4.203+r0_armhf.deb` is also
> armhf? Without the patch there are the same x86-64 binaries... They
> have moved from linux-headers:armhf to linux-kbuild-cross:armhf.
>
Yes, you are right.
If we compile current `next` natively, we have `linux-headers-
mainline_5.4.203+r0_armhf.deb` package with 32-bit ARM binaries inside the
package (in `scripts` dir). Buf if we crosscompile it, we have 64-bit x86-64
binaries in the package with the same name.
You patches fix this by generating `linux-kbuild-
mainline_5.4.203+r0_armhf.deb` (with 32-bit ARM binaries) and `linux-kbuild-
mainline-cross_5.4.203+r0_armhf.deb` (with 64-bit x86-64 binaries).
I'm just not sure that this "-cross" package should be with `armhf` prefix
(e.g., now it has control file with "Architecture: armhf" field).
> Might it be better to have linux-headers:all instead of :armhf?
> Since the binaries are out? But debian uses for their header packages
> not :all even if they have split out the binaries...
>
> I would be appreciative for a tip to getting the dpkg build not to
> complain, when we would change it to linux-kbuild-cross:amd64
>
> Keeping and not fixing the existing limitations was the goal here...
> See comment below, too.
>
>
> >
> >
> > > +Depends: ${KERNEL_HEADERS_DEBIAN_DEPENDS}, ${perl:Depends},
> > > ${shlib:Depends} +Description: ${KERNEL_NAME_PROVIDED} Linux kbuild
> > > scripts
> > > and tools for @KR@ + This package provides kernel kbuild scripts
> > > and tools
> > > + as ${HOST_ARCH} cross binaries for @KR@
> > > + This is useful for those who need to cross build
> > > + external modules using ISAR's sbuild-chroot-host
> > > diff --git a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl
> > > b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl index
> > > 94cfbe0..7e2daad 100644
> > > --- a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl
> > > +++ b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl
> > > @@ -13,16 +13,19 @@ do_build() {
> > > # Trace what we do here
> > > set -x
> > >
> > > + # Set correct build directory for kernel or kbuild (kernel
> > > scripts and
> > > tools) cases + build_dir=${KERNEL_BUILD_DIR}
> > > +
> > > # Process existing kernel configuration to make sure it is
> > > complete
> > > # (use defaults for options that were not specified)
> > > - ${MAKE} O=${KERNEL_BUILD_DIR} olddefconfig prepare
> > > + ${MAKE} O=${build_dir} olddefconfig prepare
> > >
> > > # Transfer effective kernel version into control file and
> > > scripts
> > > - KR=$(${MAKE} O=${KERNEL_BUILD_DIR} -s --no-print-directory
> > > kernelrelease) + KR=$(${MAKE} O=${build_dir} -s --no-print-
> > > directory
> > > kernelrelease) sed -i "s/@KR@/${KR}/g" ${S}/debian/control
> > > ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.*
> > >
> > > # Build the Linux kernel
> > > - ${MAKE} O=${KERNEL_BUILD_DIR} ${PARALLEL_MAKE}
> > > KCFLAGS="${KCFLAGS}"
> > > + ${MAKE} O=${build_dir} ${PARALLEL_MAKE} KCFLAGS="${KCFLAGS}"
> > >
> > > # Stop tracing
> > > set +x
> > > diff --git a/meta/recipes-
> > > kernel/linux/files/debian/isar/common.tmpl
> > > b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl index
> > > 52ebebb..55d6123 100644
> > > --- a/meta/recipes-kernel/linux/files/debian/isar/common.tmpl
> > > +++ b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl
> > > @@ -5,10 +5,14 @@
> > > set -e
> > >
> > > # Isar settings
> > > -ARCH=${KERNEL_ARCH}
> > > KERNEL_PKG_IMAGE=linux-image-${KERNEL_NAME_PROVIDED}
> > > KERNEL_PKG_KERN_HEADERS=linux-headers-${KERNEL_NAME_PROVIDED}
> > > KERNEL_PKG_LIBC_HEADERS=linux-libc-dev
> > > +KERNEL_PKG_KERN_KBUILD=linux-kbuild-${KERNEL_NAME_PROVIDED}
> > > +
> > > +if [ -z "${ARCH}" ]; then
> > > + ARCH=${KERNEL_ARCH}
> > > +fi
> > >
> > > # Constants
> > > KCONF=.config
> > > @@ -19,6 +23,7 @@ deb_img_dir=${deb_top_dir}/${KERNEL_PKG_IMAGE}
> > > deb_dbg_dir=${deb_img_dir}-dbg
> > > deb_kern_hdr_dir=${deb_top_dir}/${KERNEL_PKG_KERN_HEADERS}
> > > deb_libc_hdr_dir=${deb_top_dir}/${KERNEL_PKG_LIBC_HEADERS}
> > > +deb_kern_kbuild_dir=${deb_top_dir}/${KERNEL_PKG_KERN_KBUILD}
> > >
> > > # Array of packages to be generated
> > > declare -A kern_pkgs
> > > diff --git a/meta/recipes-
> > > kernel/linux/files/debian/isar/install.tmpl
> > > b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl index
> > > 0a8645d..890a996 100644
> > > --- a/meta/recipes-kernel/linux/files/debian/isar/install.tmpl
> > > +++ b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl
> > > @@ -33,6 +33,7 @@ do_install() {
> > > # Trace what we do here
> > > set -x
> > >
> > > + # Run the install steps
> > > install_image
> > > if [ "${ARCH}" != "um" ]; then
> > > install_config
> > > @@ -43,6 +44,19 @@ do_install() {
> > > install_kmods
> > > install_headers
> > >
> > > + if [ -d "$(dirname ${O})/build-full-kbuild-target" ]; then
> > > + # Install cross kernel scripts and tools
> > > + install_kbuild ${deb_kern_kbuild_dir}-cross build-full
> > > +
> > > + # Cleanup and install kernel scripts and tools for target
> > > arch
> > > + rm -rf ${deb_kern_kbuild_dir}
> > > + install_kbuild ${deb_kern_kbuild_dir} build-full-kbuild-
> > > target
> > > + else
> > > + # Cleanup and install kernel scripts and tools
> > > + rm -rf ${deb_kern_kbuild_dir}
> > > + install_kbuild ${deb_kern_kbuild_dir} build-full
> > > + fi
> > > +
> > > # Stop tracing
> > > set +x
> > > }
> > > @@ -168,21 +182,15 @@ kernel_headers() {
> > > mkdir -p ${destdir}
> > > mkdir -p ${deb_kern_hdr_dir}/lib/modules/${krel}
> > >
> > > - (cd ${S}; find . -name 'Makefile*' -o -name 'Kconfig*' -o -
> > > name '*.pl')
> > >
> > > > > ${src_hdr_files} - (cd ${S}; find arch/*/include include
> > > > > scripts -type
> > >
> > > f -o -type l) >>${src_hdr_files} + (cd ${S}; find . -not -path
> > > './scripts/*' -a -not -path './tools/*' -a \( -name 'Makefile*' -o
> > > -name
> > > 'Kconfig*' -o -name '*.pl' \)) >>${src_hdr_files} + (cd ${S};
> > > find
> > > arch/*/include include -type f -o -type l) >>${src_hdr_files} (cd
> > > ${S};
> > > find arch/${ARCH} -name module.lds -o -name Kbuild.platforms -o -
> > > name
> > > Platform) >>${src_hdr_files} (cd ${S}; find $(find arch/${ARCH} -
> > > name
> > > include -o -name scripts -type d) -type f) >>${src_hdr_files}
> > >
> > > if [ -n "${CONFIG_MODULES}" ]; then
> > > echo Module.symvers >> ${obj_hdr_files}
> > > fi
> > > - (cd ${O}; find arch/${ARCH}/include include scripts -type f)
> > >
> > > > > ${obj_hdr_files} - if [ -n "${CONFIG_STACK_VALIDATION}" ];
> > > > > then
> > >
> > > - (cd ${O}; find tools/objtool -type f -executable)
> > >
> > > > > ${obj_hdr_files} - fi
> > >
> > > - if [ -n "${CONFIG_GCC_PLUGINS}" ]; then
> > > - (cd ${O}; find scripts/gcc-plugins -name *.so -o -name
> > > gcc-common.h) >>${obj_hdr_files} - fi
> > > + (cd ${O}; find arch/${ARCH}/include include -type f)
> > >
> > > >>${obj_hdr_files}
> > >
> > >
> > > # deploy files that were matched above
> > > tar -C ${S} -cf - -T - <${src_hdr_files} | tar -C ${destdir} -
> > > xf -
> > > @@ -191,8 +199,11 @@ kernel_headers() {
> > > # add the kernel config
> > > cp ${O}/${KCONF} ${destdir}/.config
> > >
> > > - # handle kernel development tools
> > > - kernel_tools
> > > + # add symlink to scripts and tools directories
> > > + ln -sf ../../lib/linux-kbuild-${krel}/scripts
> > > ${destdir}/scripts
> > > + if [ -n "${CONFIG_STACK_VALIDATION}" ]; then
> > > + ln -sf ../../lib/linux-kbuild-${krel}/tools
> > > ${destdir}/tools
> > > + fi
> > >
> > > # create symlinks
> > > ln -sf /${kernel_headers_dir}
> > > ${deb_kern_hdr_dir}/lib/modules/${krel}/build @@ -206,4 +217,35 @@
> > > install_headers() {
> > > kernel_headers
> > > }
> > >
> > > +install_kbuild() {
> > > + kernel_kbuild_dir=usr/lib/linux-kbuild-${krel}
> > > + destdir=${1}/${kernel_kbuild_dir}
> > > + src_kbuild_files=$(mktemp)
> > > + obj_kbuild_files=$(mktemp)
> > > +
> > > + if [ -n "${2}" ]; then
> > > + O=$(dirname ${O})/${2}
> > > + fi
> > > +
> > > + mkdir -p ${destdir}
> > > +
> > > + (cd ${S}; find . -path './scripts/*' -a -path './tools/*' -a
> > > \( -name
> > > 'Makefile*' -o -name 'Kconfig*' -o -name '*.pl' \))
> > >
> > > >>${src_kbuild_files} +
> > >
> > > (cd ${S}; find scripts -type f -o -type l) >>${src_kbuild_files}
> > > +
> > > + (cd ${O}; find scripts -type f) >>${obj_kbuild_files}
> > > + if [ -n "${CONFIG_STACK_VALIDATION}" ]; then
> > > + (cd ${O}; find tools/objtool -type f -executable)
> > >
> > > > > ${obj_kbuild_files} + fi
> > >
> > > + if [ -n "${CONFIG_GCC_PLUGINS}" ]; then
> > > + (cd ${O}; find scripts/gcc-plugins -name *.so -o -name
> > > gcc-common.h) >>${obj_kbuild_files} + fi
> > > +
> > > + # deploy files that were matched above
> > > + tar -C ${S} -cf - -T - <${src_kbuild_files} | tar -C
> > > ${destdir} -xf -
> > > + tar -C ${O} -cf - -T - <${obj_kbuild_files} | tar -C
> > > ${destdir} -xf -
> > > +
> > > + # handle kernel development tools
> > > + kernel_tools
> > > +}
> > > +
> > > main install ${*}
> > > diff --git a/meta/recipes-kernel/linux/files/debian/rules.tmpl
> > > b/meta/recipes-kernel/linux/files/debian/rules.tmpl index
> > > 8063c49..e131288
> > > 100755
> > > --- a/meta/recipes-kernel/linux/files/debian/rules.tmpl
> > > +++ b/meta/recipes-kernel/linux/files/debian/rules.tmpl
> > > @@ -36,4 +36,7 @@ override_dh_auto_test:
> > > true
> > >
> > > override_dh_strip:
> > > - unset DEB_HOST_GNU_TYPE && dh_strip -Xvmlinu
>
>
> That line was because `linux-headers-mainline_5.4.203+r0_armhf.deb` is
> also armhf and contained x86-64 binaries...
>
>
> > > --no-automatic-dbgsym
> > > + dh_strip -Xvmlinu -Xlinux-kbuild-${KERNEL_NAME_PROVIDED}-
> > > cross
> > > --no-automatic-dbgsym +
> > > +override_dh_shlibdeps:
> > > + dh_shlibdeps -Xlinux-kbuild-${KERNEL_NAME_PROVIDED}
> > > diff --git a/meta/recipes-kernel/linux/linux-custom.inc
> > > b/meta/recipes-kernel/linux/linux-custom.inc index 96f0afc..0f8ba50
> > > 100644
> > > --- a/meta/recipes-kernel/linux/linux-custom.inc
> > > +++ b/meta/recipes-kernel/linux/linux-custom.inc
> > > @@ -74,12 +74,14 @@ TEMPLATE_VARS += " \
> > > KERNEL_ARCH \
> > > KERNEL_DEBIAN_DEPENDS \
> > > KERNEL_BUILD_DIR \
> > > + KERNEL_KBUILD_DIR \
> > > KERNEL_FILE \
> > > KERNEL_HEADERS_DEBIAN_DEPENDS \
> > > LINUX_VERSION_EXTENSION \
> > > KERNEL_NAME_PROVIDED \
> > > KERNEL_CONFIG_FRAGMENTS \
> > > KCFLAGS \
> > > + CONTROL_TARGET_ARCH \
> > > "
> > >
> > > inherit dpkg
> > > @@ -91,30 +93,54 @@ KCFLAGS ?= "-fdebug-prefix-map=${CURDIR}=."
> > > # Derive name of the kernel packages from the name of this recipe
> > > KERNEL_NAME_PROVIDED ?= "${@ d.getVar('PN',
> > > True).partition('linux-')[2]}"
> > >
> > > -# Make bitbake know we will be producing linux-image and linux-
> > > headers
> > > packages python() {
> > > kernel_name = d.getVar("KERNEL_NAME_PROVIDED", True)
> > > - d.setVar('PROVIDES', 'linux-image-' + kernel_name + ' ' + \
> > > - 'linux-headers-' + kernel_name)
> > > + distro_arch = d.getVar("DISTRO_ARCH", True)
> > > + host_arch = d.getVar("HOST_ARCH", True)
> > > + target_arch = ""
> > > + kbuild_depends = "linux-kbuild-" + kernel_name
> > > +
> > > + # For different distro and host archs
> > > + # - Set target arch (empty string disables package build)
> > > + # - Add dependency for sbuild-chroot-target
> > > + # to allow building arch specific kbuild scripts and tools
> > > + # - Set correct name of kbuild package dependency
> > > + if distro_arch != host_arch:
> > > + d.appendVar("SCHROOT_DEP", " sbuild-chroot-
> > > target:do_build")
> > > + if d.getVar("ISAR_CROSS_COMPILE", True) == "1":
> > > + target_arch = distro_arch
> > > + kbuild_depends = kbuild_depends + "-cross | " +
> > > kbuild_depends
> > > +
> > > + # Set CONTROL_TARGET_ARCH
> > > + d.setVar("CONTROL_TARGET_ARCH", target_arch)
> > > +
> > > + # Make bitbake know we will be producing
> > > + # linux-image and linux-headers packages
> > > + d.setVar("PROVIDES", "linux-image-" + kernel_name + " " + \
> > > + "linux-headers-" + kernel_name)
> > > +
> > > + # Set dependency for kbuild package
> > > + d.appendVar("KERNEL_HEADERS_DEBIAN_DEPENDS", kbuild_depends)
> > > }
> > >
> > > -def get_kernel_arch(d):
> > > - distro_arch = d.getVar("DISTRO_ARCH")
> > > - if distro_arch in ["amd64", "i386"]:
> > > +def get_kernel_arch(d, arch="DISTRO_ARCH"):
> > > + arch = d.getVar(arch)
> > > + if arch in ["amd64", "i386"]:
> > > kernel_arch = "x86"
> > > - elif distro_arch == "arm64":
> > > + elif arch == "arm64":
> > > kernel_arch = "arm64"
> > > - elif distro_arch == "armhf":
> > > + elif arch == "armhf":
> > > kernel_arch = "arm"
> > > - elif distro_arch == "mipsel":
> > > + elif arch == "mipsel":
> > > kernel_arch = "mips"
> > > - elif distro_arch == "riscv64":
> > > + elif arch == "riscv64":
> > > kernel_arch = "riscv"
> > > else:
> > > kernel_arch = ""
> > > return kernel_arch
> > >
> > > KERNEL_ARCH ??= "${@get_kernel_arch(d)}"
> > > +KERNEL_HOST_ARCH ??= "${@get_kernel_arch(d, 'HOST_ARCH')}"
> > >
> > > def config_fragments(d):
> > > fragments = []
> > > @@ -157,6 +183,7 @@ do_prepare_build_prepend() {
> > >
> > > # build directory for our "full" kernel build
> > > KERNEL_BUILD_DIR = "build-full"
> > > +KERNEL_KBUILD_DIR = "${KERNEL_BUILD_DIR}-kbuild"
> > >
> > > def get_kernel_config_target(d):
> > > kernel_defconfig = d.getVar('KERNEL_DEFCONFIG', True)
> > > @@ -187,15 +214,19 @@ def get_kernel_config_fragments(d):
> > > KERNEL_CONFIG_FRAGMENTS = "${@get_kernel_config_fragments(d)}"
> > >
> > > dpkg_configure_kernel() {
> > > + build_dir="${KERNEL_BUILD_DIR}"
> > > + if [ -n "${1}" ]; then
> > > + build_dir="${1}"
> > > + fi
> > > grep -q "KERNEL_CONFIG_TARGET=" ${S}/debian/isar/configure
> > >
> > > ||
> > >
> > > cat << EOF | sed -i '/^do_configure() {/ r
> > > /dev/stdin'
> > > ${S}/debian/isar/configure
> > > KERNEL_CONFIG_TARGET="${@get_kernel_config_target(d)}"
> > > EOF
> > >
> > > - rm -rf ${S}/${KERNEL_BUILD_DIR} && mkdir -p ${S}/$
> >
> > {KERNEL_BUILD_DIR}
> >
> > > + rm -rf ${S}/${build_dir} && mkdir -p ${S}/${build_dir}
> > > if [ -n "${KERNEL_DEFCONFIG}" ]; then
> > > if [ -e "${WORKDIR}/${KERNEL_DEFCONFIG}" ]; then
> > > - cp ${WORKDIR}/${KERNEL_DEFCONFIG} ${S}/$
> >
> > {KERNEL_BUILD_DIR}/.config
> >
> > > + cp ${WORKDIR}/${KERNEL_DEFCONFIG} ${S}/$
> >
> > {build_dir}/.config
> >
> > > fi
> > > fi
> > >
> > > @@ -210,4 +241,53 @@ EOF
> > >
> > > dpkg_runbuild_prepend() {
> > > dpkg_configure_kernel
> > > +
> > > + if [ "${ISAR_CROSS_COMPILE}" = "1" ] && [ "${HOST_ARCH}" !=
> > > "${DISTRO_ARCH}" ]; then + # Create temporarily kernel
> >
> > schroot-target
> >
> > > configuration
> > > + schroot_create_configs ${SCHROOT_TARGET_DIR}
> > > +
> > > + # Begin kernel schroot-target session
> > > + schroot_session=$(schroot -b -c ${SBUILD_CHROOT})
> > > +
> > > + # Install build deps for schroot session
> > > + schroot -u root -r -c $schroot_session -- sh -c " \
> > > + cd ${PP}/${PPS}; \
> > > + mk-build-deps -i -r --host-arch $
> >
> > {DISTRO_ARCH} -t \"apt-get -y
> >
> > > --no-install-recommends --allow-downgrades -o
> > > Debug::pkgProblemResolver=yes\" debian/control; \ + "
> > > +
> > > + # Create kernel configuration for kbuild
> > > + dpkg_configure_kernel ${KERNEL_KBUILD_DIR}-target
> > > + KERNEL_CONFIG_TARGET="${@get_kernel_config_target(d
> > > )}"
> > > +
> > > + schroot -r -c $schroot_session -- sh -c " \
> > > + cd ${PP}/${PPS}; \
> > > + ARCH=${KERNEL_ARCH} make O=$
> >
> > {KERNEL_KBUILD_DIR}-target
> >
> > > ${KERNEL_CONFIG_TARGET}; \ + "
> > > +
> > > + cd ${S}
> > > + ARCH=${KERNEL_ARCH}
> > > ./scripts/kconfig/merge_config.sh -m
> >
> > -O
> >
> > > ${KERNEL_KBUILD_DIR}-target/ ${KERNEL_KBUILD_DIR}-target/.config
> > > ${KERNEL_CONFIG_FRAGMENTS} + cd -
> > > +
> > > + # Build kernel scripts and tools for target arch in
> > > a non-
> >
> > cross
> >
> > > way + # Cross building kernel scripts and tools would not
> > > provide
> > > them in target architecture + schroot -r -c
> >
> > $schroot_session -- sh -c " \
> >
> > > + cd ${PP}/${PPS}; \
> > > + ARCH=${KERNEL_ARCH} make O=$
> >
> > {KERNEL_KBUILD_DIR}-target olddefconfig; \
> >
> > > + ARCH=${KERNEL_ARCH} make O=$
> >
> > {KERNEL_KBUILD_DIR}-target ${PARALLEL_MAKE}
> >
> > > scripts; \ + if [ -d "tools/objtool" ]; then \
> > > + echo ARCH=${KERNEL_ARCH} make O=$
> >
> > {KERNEL_KBUILD_DIR}-target
> >
> > > ${PARALLEL_MAKE} tools/objtool; \
> > > + ARCH=$
> >
> > {KERNEL_ARCH} make
> >
> > > O=${KERNEL_KBUILD_DIR}-target ${PARALLEL_MAKE} tools/objtool; \
> > > +
> >
> > fi; \
> >
> > > + if grep -q "CONFIG_MODULES=y" $
> >
> > {KERNEL_KBUILD_DIR}-target/.config; then
> >
> > > \ + echo ARCH=${KERNEL_ARCH} make O=$
> >
> > {KERNEL_KBUILD_DIR}-target
> >
> > > ${PARALLEL_MAKE} modules_prepare; \ +
> >
> > ARCH=${KERNEL_ARCH} make
> >
> > > O=${KERNEL_KBUILD_DIR}-target ${PARALLEL_MAKE} modules_prepare; \
> > > +
> >
> > fi; \
> >
> > > + "
> > > +
> > > + # End kernel schroot-target session
> > > + schroot -e -c $schroot_session
> > > +
> > > + # Restore kernel schroot-host configuration
> > > + schroot_create_configs
> > > + fi
> > > }
> >
> >
> >
> >
> >
>
>
> --
> Stefan Koch
> Siemens AG
> www.siemens.com
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 0/3] linux-custom: Split up binaries from kernel headers to kbuild packages
2022-11-09 10:32 [PATCH 0/3] linux-custom: Split up binaries from kernel headers to kbuild packages Koch, Stefan
` (3 preceding siblings ...)
2022-11-09 11:50 ` [PATCH 0/3] linux-custom: Split up binaries from kernel headers to kbuild packages Jan Kiszka
@ 2022-11-15 13:44 ` Uladzimir Bely
2022-11-15 17:23 ` Jan Kiszka
4 siblings, 1 reply; 18+ messages in thread
From: Uladzimir Bely @ 2022-11-15 13:44 UTC (permalink / raw)
To: isar-users, Koch, Stefan
In mail from среда, 9 ноября 2022 г. 13:32:45 +03 user Koch, Stefan wrote:
> Hi
>
> This patchset is a set of three patches:
> - Support overwriting configured schroot dir
> - Split up binaries from kernel headers to kbuild packages
> - Update custom_kernel docs for split up of kernel scripts and tools
>
Just some thoughts regarding schroot path overwriting...
What if `linux-custom` recipe could disable specific components?
I mean, currently we have this feature only for `linux-libc-dev` package, that
is disabled when "nolibcdev" build profile is selected. If we added similar
build profiles for other components, we could disable also `linux-$
{KERNEL_NAME_PROVIDED}`, linux-image-{KERNEL_NAME_PROVIDED}, linux-headers-
{KERNEL_NAME_PROVIDED} and `linux-image-${KERNEL_NAME_PROVIDED}-dbg`
Then, we make `linux-mainline` build all except `linux-kbuild` (if we don't
need cross version). And we could create the similar recipe `linux-kbuild`
that inherits `linux-custom`, but builds only `linux-kbuild` - with
ISAR_CROSS_COMPILE="0".
Of course, such an approach breaks the feature "build everything from one
recipe", but it doesn't require "schroot" workaround and would make build
process more configurable.
> The main use-case was to swap out the binaries
> from the kernel headers package.
> They will be swapped out into host and target
> specific kernel kbuild packages.
>
> The main development goals were these:
>
> 1. Solve already known isar custom kernel
> limitations from doc/custom_kernel.inc
> - kernel headers package does not support both native
> and cross compilation of kernel modules when cross built
>
> 2. Honor recommendations for future from doc/custom_kernel.inc
> - Generate kernel headers packages for both host and target
> when using cross build
>
> 3. Add extensions known from debian kernel packages structure
> - Generate a kernel headers package without binaries
> - Create specific kernel kbuild packages that
> will ship the "scripts" and "tools" binaries
> - Use symlinks to point to the "scripts" and "tools" binaries
>
> 4. Be user friendly
> - Avoid usage of separate kbuild bitbake recipe that may enforce
> redundant configuration of kernel source definitions with user
> actions to enable kbuild package generation
> - Use already known way to include linux-custom.inc in just one
> own bitbake recipe that provides the kernel source definitions
> - Keep known user behavior for existing build configurations:
> just update isar, and kbuild packages will be created automatically
>
> Best regards
>
> Stefan
>
> Stefan Koch (3):
> sbuild: Support overwriting configured schroot dir
> linux-custom: Split up binaries from kernel headers to kbuild packages
> docs: Update custom_kernel docs for split up of kernel scripts and
> tools
>
> doc/custom_kernel.md | 13 +--
> meta/classes/sbuild.bbclass | 9 +-
> .../linux/files/debian/control.tmpl | 18 ++-
> .../linux/files/debian/isar/build.tmpl | 9 +-
> .../linux/files/debian/isar/common.tmpl | 7 +-
> .../linux/files/debian/isar/install.tmpl | 64 +++++++++--
> .../linux/files/debian/rules.tmpl | 5 +-
> meta/recipes-kernel/linux/linux-custom.inc | 104 ++++++++++++++++--
> 8 files changed, 191 insertions(+), 38 deletions(-)
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 0/3] linux-custom: Split up binaries from kernel headers to kbuild packages
2022-11-15 13:44 ` Uladzimir Bely
@ 2022-11-15 17:23 ` Jan Kiszka
2022-11-18 17:11 ` Koch, Stefan
0 siblings, 1 reply; 18+ messages in thread
From: Jan Kiszka @ 2022-11-15 17:23 UTC (permalink / raw)
To: Uladzimir Bely, isar-users, Koch, Stefan
On 15.11.22 14:44, Uladzimir Bely wrote:
> In mail from среда, 9 ноября 2022 г. 13:32:45 +03 user Koch, Stefan wrote:
>> Hi
>>
>> This patchset is a set of three patches:
>> - Support overwriting configured schroot dir
>> - Split up binaries from kernel headers to kbuild packages
>> - Update custom_kernel docs for split up of kernel scripts and tools
>>
>
> Just some thoughts regarding schroot path overwriting...
>
> What if `linux-custom` recipe could disable specific components?
Right, I sill wanted to follow up on that: Debian build profiles can do
the trick here. Define one for "host tools only", and only build that in
the second step.
>
> I mean, currently we have this feature only for `linux-libc-dev` package, that
> is disabled when "nolibcdev" build profile is selected. If we added similar
> build profiles for other components, we could disable also `linux-$
> {KERNEL_NAME_PROVIDED}`, linux-image-{KERNEL_NAME_PROVIDED}, linux-headers-
> {KERNEL_NAME_PROVIDED} and `linux-image-${KERNEL_NAME_PROVIDED}-dbg`
>
> Then, we make `linux-mainline` build all except `linux-kbuild` (if we don't
> need cross version). And we could create the similar recipe `linux-kbuild`
> that inherits `linux-custom`, but builds only `linux-kbuild` - with
> ISAR_CROSS_COMPILE="0".
>
> Of course, such an approach breaks the feature "build everything from one
> recipe", but it doesn't require "schroot" workaround and would make build
> process more configurable.
On top, we need a loop in the dpkg class the runs the recipe for
multiple archs, not just the target one. Then also drop that special and
ugly "-cross" package and just built the kbuild thing for the target and
the builder arch.
Jan
--
Siemens AG, Technology
Competence Center Embedded Linux
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 0/3] linux-custom: Split up binaries from kernel headers to kbuild packages
2022-11-15 17:23 ` Jan Kiszka
@ 2022-11-18 17:11 ` Koch, Stefan
2022-12-20 16:57 ` Koch, Stefan
0 siblings, 1 reply; 18+ messages in thread
From: Koch, Stefan @ 2022-11-18 17:11 UTC (permalink / raw)
To: ubely, isar-users, Kiszka, Jan
On Tue, 2022-11-15 at 18:23 +0100, Jan Kiszka wrote:
> On 15.11.22 14:44, Uladzimir Bely wrote:
> > In mail from среда, 9 ноября 2022 г. 13:32:45 +03 user Koch, Stefan
> > wrote:
> > > Hi
> > >
> > > This patchset is a set of three patches:
> > > - Support overwriting configured schroot dir
> > > - Split up binaries from kernel headers to kbuild packages
> > > - Update custom_kernel docs for split up of kernel scripts and
> > > tools
> > >
> >
> > Just some thoughts regarding schroot path overwriting...
> >
> > What if `linux-custom` recipe could disable specific components?
>
> Right, I sill wanted to follow up on that: Debian build profiles can
> do
> the trick here. Define one for "host tools only", and only build that
> in
> the second step.
- I have sucessfully set up a prototype of second patchset that uses
the debian build profiles approach.
>
> >
> > I mean, currently we have this feature only for `linux-libc-dev`
> > package, that
> > is disabled when "nolibcdev" build profile is selected. If we added
> > similar
> > build profiles for other components, we could disable also `linux-$
> > {KERNEL_NAME_PROVIDED}`, linux-image-{KERNEL_NAME_PROVIDED}, linux-
> > headers-
> > {KERNEL_NAME_PROVIDED} and `linux-image-${KERNEL_NAME_PROVIDED}-
> > dbg`
> >
> > Then, we make `linux-mainline` build all except `linux-kbuild` (if
> > we don't
> > need cross version). And we could create the similar recipe `linux-
> > kbuild`
> > that inherits `linux-custom`, but builds only `linux-kbuild` - with
> > ISAR_CROSS_COMPILE="0".
> >
@Bely
Your commit fb61feff moves merge_config.sh call from linux-custom.inc
to debian/isar/configure.tmpl
The merge_config.sh is very slow in target schroot (seems slower than
the build of the scripts/tools itself), so maybe it should be moved
back to linux-custom.inc to run in host schroot.
> > Of course, such an approach breaks the feature "build everything
> > from one
> > recipe", but it doesn't require "schroot" workaround and would make
> > build
> > process more configurable.
- Prototype keeps build everythong from one recipe approach.
>
> On top, we need a loop in the dpkg class the runs the recipe for
> multiple archs, not just the target one. Then also drop that special
> and
> ugly "-cross" package and just built the kbuild thing for the target
> and
> the builder arch.
- Prototype build using dpkg class without the most linux-custom
schroot customizations. Improvment is ongoing...
What would a beautiful solution to enable a recipe for multiarch in a
generic way? Any ideas? (Specify that target build should run)
>
> Jan
>
--
Stefan Koch
Siemens AG
www.siemens.com
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 0/3] linux-custom: Split up binaries from kernel headers to kbuild packages
2022-11-18 17:11 ` Koch, Stefan
@ 2022-12-20 16:57 ` Koch, Stefan
0 siblings, 0 replies; 18+ messages in thread
From: Koch, Stefan @ 2022-12-20 16:57 UTC (permalink / raw)
To: ubely, isar-users, Kiszka, Jan
On Fri, 2022-11-18 at 18:11 +0100, Stefan Koch wrote:
> On Tue, 2022-11-15 at 18:23 +0100, Jan Kiszka wrote:
> > On 15.11.22 14:44, Uladzimir Bely wrote:
> > > In mail from среда, 9 ноября 2022 г. 13:32:45 +03 user Koch,
> > > Stefan
> > > wrote:
> > > > Hi
> > > >
> > > > This patchset is a set of three patches:
> > > > - Support overwriting configured schroot dir
> > > > - Split up binaries from kernel headers to kbuild packages
> > > > - Update custom_kernel docs for split up of kernel scripts and
> > > > tools
> > > >
> > >
> > > Just some thoughts regarding schroot path overwriting...
> > >
> > > What if `linux-custom` recipe could disable specific components?
> >
> > Right, I sill wanted to follow up on that: Debian build profiles
> > can
> > do
> > the trick here. Define one for "host tools only", and only build
> > that
> > in
> > the second step.
>
> - I have sucessfully set up a prototype of second patchset that uses
> the debian build profiles approach.
I'll send a v2 patchset later... That uses build profiles, now.
>
> >
> > >
> > > I mean, currently we have this feature only for `linux-libc-dev`
> > > package, that
> > > is disabled when "nolibcdev" build profile is selected. If we
> > > added
> > > similar
> > > build profiles for other components, we could disable also
> > > `linux-$
> > > {KERNEL_NAME_PROVIDED}`, linux-image-{KERNEL_NAME_PROVIDED},
> > > linux-
> > > headers-
> > > {KERNEL_NAME_PROVIDED} and `linux-image-${KERNEL_NAME_PROVIDED}-
> > > dbg`
> > >
> > > Then, we make `linux-mainline` build all except `linux-kbuild`
> > > (if
> > > we don't
> > > need cross version). And we could create the similar recipe
> > > `linux-
> > > kbuild`
> > > that inherits `linux-custom`, but builds only `linux-kbuild` -
> > > with
> > > ISAR_CROSS_COMPILE="0".
> > >
>
> @Bely
> Your commit fb61feff moves merge_config.sh call from linux-custom.inc
> to debian/isar/configure.tmpl
>
> The merge_config.sh is very slow in target schroot (seems slower than
> the build of the scripts/tools itself), so maybe it should be moved
> back to linux-custom.inc to run in host schroot.
The .config is now reused from first build case now. So merge_config.sh
needs not to be rerun anymore.
>
> > > Of course, such an approach breaks the feature "build everything
> > > from one
> > > recipe", but it doesn't require "schroot" workaround and would
> > > make
> > > build
> > > process more configurable.
> - Prototype keeps build everythong from one recipe approach.
> >
> > On top, we need a loop in the dpkg class the runs the recipe for
> > multiple archs, not just the target one. Then also drop that
> > special
> > and
> > ugly "-cross" package and just built the kbuild thing for the
> > target
> > and
> > the builder arch.
>
> - Prototype build using dpkg class without the most linux-custom
> schroot customizations. Improvment is ongoing...
Finished... A loop runs the default build and optionally when appending
"target" it creates the kbuild binaries for the target. When appending
"host", the ugly -cross package with invalid traget arch will not be
created - but instead a regular kbuild package with host architecture
will be created.
Just append "target" and/or "host" to ISAR_BUILDS in the recipe that
includes linux-custom.inc
>
> What would a beautiful solution to enable a recipe for multiarch in a
> generic way? Any ideas? (Specify that target build should run)
>
> >
> > Jan
> >
>
Merry Christmas and a Happy New Year 2023...
^ permalink raw reply [flat|nested] 18+ messages in thread
end of thread, other threads:[~2022-12-20 16:57 UTC | newest]
Thread overview: 18+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-11-09 10:32 [PATCH 0/3] linux-custom: Split up binaries from kernel headers to kbuild packages Koch, Stefan
2022-11-09 10:32 ` [PATCH 1/3] sbuild: Support overwriting configured schroot dir Koch, Stefan
2022-11-09 10:32 ` [PATCH 2/3] linux-custom: Split up binaries from kernel headers to kbuild packages Koch, Stefan
2022-11-11 5:34 ` Uladzimir Bely
2022-11-11 9:03 ` Koch, Stefan
2022-11-11 10:50 ` Uladzimir Bely
2022-11-09 10:32 ` [PATCH 3/3] docs: Update custom_kernel docs for split up of kernel scripts and tools Koch, Stefan
2022-11-09 11:50 ` [PATCH 0/3] linux-custom: Split up binaries from kernel headers to kbuild packages Jan Kiszka
2022-11-09 15:06 ` Koch, Stefan
2022-11-09 16:00 ` Jan Kiszka
2022-11-10 17:49 ` Koch, Stefan
2022-11-10 18:33 ` Jan Kiszka
2022-11-10 18:36 ` Jan Kiszka
2022-11-11 9:47 ` Koch, Stefan
2022-11-15 13:44 ` Uladzimir Bely
2022-11-15 17:23 ` Jan Kiszka
2022-11-18 17:11 ` Koch, Stefan
2022-12-20 16:57 ` Koch, Stefan
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox