* [PATCH v4 0/3] linux-custom: Split up binaries from kernel headers to kbuild packages
@ 2023-08-21 13:44 Stefan Koch
2023-08-21 13:44 ` [PATCH v4 1/3] linux-custom: Split up binaries from kernel headers to kbuild package Stefan Koch
` (6 more replies)
0 siblings, 7 replies; 17+ messages in thread
From: Stefan Koch @ 2023-08-21 13:44 UTC (permalink / raw)
To: isar-users
Cc: stefan-koch, jan.kiszka, christian.storm, michael.adler,
simon.sudler, cedric.hombourger, adriaan.schmidt,
felix.moessbauer, ubely
Hi
This updated v4 patchset implement suggestions from reviewed v3 patchset.
It's now a set of five patches:
- linux-custom: Split up binaries from kernel headers to kbuild package
- linux-custom: Provide host and target specific kernel kbuild packages
- docs: Update custom_kernel docs for split up of kernel scripts and tools
The main use-case was to swap out the binaries
from the kernel headers into kernel kbuild package.
This is introduced by the first commit
"Split up binaries from kernel headers to kbuild package"
The second commit "Provide host and target specific kernel kbuild packages"
introduces that the binaries could be swapped out into host and target
specific kernel kbuild packages.
The main development goals were these:
1. Solve already known isar custom kernel
limitations from doc/custom_kernel.inc
- kernel headers package does not support both native
and cross compilation of kernel modules when cross built
2. Honor recommendations for future from doc/custom_kernel.inc
- Generate kernel headers packages for both host and target
when using cross build
3. Add extensions known from debian kernel packages structure
- Generate a kernel headers package without binaries
- Create specific kernel kbuild packages that
will ship the "scripts" and "tools" binaries
- Use symlinks to point to the "scripts" and "tools" binaries
4. Be user friendly
- Avoid redundant configuration of kernel source definitions with user
actions to enable kbuild package generation
- Use already known way to include linux-custom.inc in just one
own bitbake recipe that provides the kernel source definitions
- Keep known user behavior for existing build configurations:
just update isar, request "-compat" (and "-native") bitbake targets
and kbuild packages for target and host will be created automatically
Best regards
Stefan
Stefan Koch (3):
linux-custom: Split up binaries from kernel headers to kbuild package
linux-custom: Provide host and target specific kernel kbuild packages
docs: Update custom_kernel docs for split up of kernel scripts and
tools
doc/custom_kernel.md | 16 ++--
.../linux/files/debian/control.tmpl | 25 ++++++-
.../linux/files/debian/isar/build.tmpl | 12 ++-
.../linux/files/debian/isar/common.tmpl | 10 +++
.../linux/files/debian/isar/install.tmpl | 73 ++++++++++++++-----
.../linux/files/debian/rules.tmpl | 2 +-
meta/recipes-kernel/linux/linux-custom.inc | 53 ++++++++++++--
7 files changed, 155 insertions(+), 36 deletions(-)
--
2.39.2
^ permalink raw reply [flat|nested] 17+ messages in thread
* [PATCH v4 1/3] linux-custom: Split up binaries from kernel headers to kbuild package
2023-08-21 13:44 [PATCH v4 0/3] linux-custom: Split up binaries from kernel headers to kbuild packages Stefan Koch
@ 2023-08-21 13:44 ` Stefan Koch
2023-08-21 13:45 ` [PATCH v4 2/3] linux-custom: Provide host and target specific kernel kbuild packages Stefan Koch
` (5 subsequent siblings)
6 siblings, 0 replies; 17+ messages in thread
From: Stefan Koch @ 2023-08-21 13:44 UTC (permalink / raw)
To: isar-users
Cc: stefan-koch, jan.kiszka, christian.storm, michael.adler,
simon.sudler, cedric.hombourger, adriaan.schmidt,
felix.moessbauer, ubely
This patch does introduce specific kernel kbuild packages that
ship the "scripts" and "tools" binaries.
The kernel headers fulfill this using symlinks to point
to the "scripts" and "tools" of the kernel kbuild package.
This is already known from debian kernel packages structure:
- Generate a kernel headers package without binaries
- Create kernel kbuild packages that
ship the "scripts" and "tools" binaries
- Using symlinks to point to the "scripts"
and "tools" binaries
Signed-off-by: Stefan Koch <stefan-koch@siemens.com>
---
.../linux/files/debian/control.tmpl | 9 +++-
.../linux/files/debian/isar/common.tmpl | 2 +
.../linux/files/debian/isar/install.tmpl | 51 +++++++++++++++----
meta/recipes-kernel/linux/linux-custom.inc | 4 ++
4 files changed, 54 insertions(+), 12 deletions(-)
diff --git a/meta/recipes-kernel/linux/files/debian/control.tmpl b/meta/recipes-kernel/linux/files/debian/control.tmpl
index dd0b6245..b19ca2c9 100644
--- a/meta/recipes-kernel/linux/files/debian/control.tmpl
+++ b/meta/recipes-kernel/linux/files/debian/control.tmpl
@@ -26,7 +26,7 @@ Section: devel
Provides: linux-kernel-headers
Architecture: any
Description: Linux support headers for userspace development
- This package provides userspaces headers from the Linux kernel. These headers
+ This package provides userspaces headers from the Linux kernel. These headers
are used by the installed headers for GNU glibc and other system libraries.
Package: linux-image-${KERNEL_NAME_PROVIDED}-dbg
@@ -35,3 +35,10 @@ Architecture: any
Description: Linux kernel debugging symbols for @KR@
This package will come in handy if you need to debug the kernel. It provides
all the necessary debug symbols for the kernel and its modules.
+
+Package: linux-kbuild-${KERNEL_NAME_PROVIDED}
+Architecture: any
+Depends: ${perl:Depends}, ${shlib:Depends}
+Description: ${KERNEL_NAME_PROVIDED} Linux kbuild scripts and tools for @KR@
+ This package provides kernel kbuild scripts and tools for @KR@
+ This is useful for people who need to build external modules
diff --git a/meta/recipes-kernel/linux/files/debian/isar/common.tmpl b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl
index 52ebebb0..f4c0519f 100644
--- a/meta/recipes-kernel/linux/files/debian/isar/common.tmpl
+++ b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl
@@ -9,6 +9,7 @@ ARCH=${KERNEL_ARCH}
KERNEL_PKG_IMAGE=linux-image-${KERNEL_NAME_PROVIDED}
KERNEL_PKG_KERN_HEADERS=linux-headers-${KERNEL_NAME_PROVIDED}
KERNEL_PKG_LIBC_HEADERS=linux-libc-dev
+KERNEL_PKG_KERN_KBUILD=linux-kbuild-${KERNEL_NAME_PROVIDED}
# Constants
KCONF=.config
@@ -19,6 +20,7 @@ deb_img_dir=${deb_top_dir}/${KERNEL_PKG_IMAGE}
deb_dbg_dir=${deb_img_dir}-dbg
deb_kern_hdr_dir=${deb_top_dir}/${KERNEL_PKG_KERN_HEADERS}
deb_libc_hdr_dir=${deb_top_dir}/${KERNEL_PKG_LIBC_HEADERS}
+deb_kern_kbuild_dir=${deb_top_dir}/${KERNEL_PKG_KERN_KBUILD}
# Array of packages to be generated
declare -A kern_pkgs
diff --git a/meta/recipes-kernel/linux/files/debian/isar/install.tmpl b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl
index 8a604e45..236b67c4 100644
--- a/meta/recipes-kernel/linux/files/debian/isar/install.tmpl
+++ b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl
@@ -33,6 +33,7 @@ do_install() {
# Trace what we do here
set -x
+ # Run the install steps
install_image
if [ "${ARCH}" != "um" ]; then
install_config
@@ -43,6 +44,10 @@ do_install() {
install_kmods
install_headers
+ # Cleanup and install kernel scripts and tools
+ rm -rf ${deb_kern_kbuild_dir}
+ install_kbuild ${deb_kern_kbuild_dir}
+
# Stop tracing
set +x
}
@@ -168,21 +173,15 @@ kernel_headers() {
mkdir -p ${destdir}
mkdir -p ${deb_kern_hdr_dir}/lib/modules/${krel}
- (cd ${S}; find . -name 'Makefile*' -o -name 'Kconfig*' -o -name '*.pl') >>${src_hdr_files}
- (cd ${S}; find arch/*/include include scripts -type f -o -type l) >>${src_hdr_files}
+ (cd ${S}; find . -not -path './scripts/*' -a -not -path './tools/*' -a \( -name 'Makefile*' -o -name 'Kconfig*' -o -name '*.pl' \)) >>${src_hdr_files}
+ (cd ${S}; find arch/*/include include -type f -o -type l) >>${src_hdr_files}
(cd ${S}; find arch/${ARCH} -name module.lds -o -name Kbuild.platforms -o -name Platform) >>${src_hdr_files}
(cd ${S}; find $(find arch/${ARCH} -name include -o -name scripts -type d) -type f) >>${src_hdr_files}
if [ -n "${CONFIG_MODULES}" ]; then
echo Module.symvers >> ${obj_hdr_files}
fi
- (cd ${O}; find arch/${ARCH}/include include scripts -type f) >>${obj_hdr_files}
- if [ -n "${CONFIG_STACK_VALIDATION}" -o -n "${CONFIG_HAVE_OBJTOOL}" ]; then
- (cd ${O}; find tools/objtool -type f -executable) >>${obj_hdr_files}
- fi
- if [ -n "${CONFIG_GCC_PLUGINS}" ]; then
- (cd ${O}; find scripts/gcc-plugins -name *.so -o -name gcc-common.h) >>${obj_hdr_files}
- fi
+ (cd ${O}; find arch/${ARCH}/include include -type f) >>${obj_hdr_files}
# deploy files that were matched above
tar -C ${S} -cf - -T - <${src_hdr_files} | tar -C ${destdir} -xf -
@@ -191,8 +190,11 @@ kernel_headers() {
# add the kernel config
cp ${O}/${KCONF} ${destdir}/.config
- # handle kernel development tools
- kernel_tools
+ # add symlink to scripts and tools directories
+ ln -sf ../../lib/linux-kbuild-${krel}/scripts ${destdir}/scripts
+ if [ -n "${CONFIG_STACK_VALIDATION}" -o -n "${CONFIG_HAVE_OBJTOOL}" ]; then
+ ln -sf ../../lib/linux-kbuild-${krel}/tools ${destdir}/tools
+ fi
# create symlinks
ln -sf /${kernel_headers_dir} ${deb_kern_hdr_dir}/lib/modules/${krel}/build
@@ -206,4 +208,31 @@ install_headers() {
kernel_headers
}
+install_kbuild() {
+ kernel_kbuild_dir=usr/lib/linux-kbuild-${krel}
+ destdir=${1}/${kernel_kbuild_dir}
+ src_kbuild_files=$(mktemp)
+ obj_kbuild_files=$(mktemp)
+
+ mkdir -p ${destdir}
+
+ (cd ${S}; find . -path './scripts/*' -a -path './tools/*' -a \( -name 'Makefile*' -o -name 'Kconfig*' -o -name '*.pl' \)) >>${src_kbuild_files}
+ (cd ${S}; find scripts -type f -o -type l) >>${src_kbuild_files}
+
+ (cd ${O}; find scripts -type f) >>${obj_kbuild_files}
+ if [ -n "${CONFIG_STACK_VALIDATION}" -o -n "${CONFIG_HAVE_OBJTOOL}" ]; then
+ (cd ${O}; find tools/objtool -type f -executable) >>${obj_kbuild_files}
+ fi
+ if [ -n "${CONFIG_GCC_PLUGINS}" ]; then
+ (cd ${O}; find scripts/gcc-plugins -name *.so -o -name gcc-common.h) >>${obj_kbuild_files}
+ fi
+
+ # deploy files that were matched above
+ tar -C ${S} -cf - -T - <${src_kbuild_files} | tar -C ${destdir} -xf -
+ tar -C ${O} -cf - -T - <${obj_kbuild_files} | tar -C ${destdir} -xf -
+
+ # handle kernel development tools
+ kernel_tools
+}
+
main install ${*}
diff --git a/meta/recipes-kernel/linux/linux-custom.inc b/meta/recipes-kernel/linux/linux-custom.inc
index 8b367a2f..94ed84a2 100644
--- a/meta/recipes-kernel/linux/linux-custom.inc
+++ b/meta/recipes-kernel/linux/linux-custom.inc
@@ -100,6 +100,10 @@ python() {
kernel_name = d.getVar("KERNEL_NAME_PROVIDED")
d.setVar('PROVIDES', 'linux-image-' + kernel_name + ' ' + \
'linux-headers-' + kernel_name)
+ headers_depends = "linux-kbuild-" + kernel_name
+
+ # Set dependency for kernel headers
+ d.appendVar("KERNEL_HEADERS_DEBIAN_DEPENDS", headers_depends)
}
def get_kernel_arch(d):
--
2.39.2
^ permalink raw reply [flat|nested] 17+ messages in thread
* [PATCH v4 2/3] linux-custom: Provide host and target specific kernel kbuild packages
2023-08-21 13:44 [PATCH v4 0/3] linux-custom: Split up binaries from kernel headers to kbuild packages Stefan Koch
2023-08-21 13:44 ` [PATCH v4 1/3] linux-custom: Split up binaries from kernel headers to kbuild package Stefan Koch
@ 2023-08-21 13:45 ` Stefan Koch
2023-09-06 6:38 ` Schmidt, Adriaan
2023-08-21 13:45 ` [PATCH v4 3/3] docs: Update custom_kernel docs for split up of kernel scripts and tools Stefan Koch
` (4 subsequent siblings)
6 siblings, 1 reply; 17+ messages in thread
From: Stefan Koch @ 2023-08-21 13:45 UTC (permalink / raw)
To: isar-users
Cc: stefan-koch, jan.kiszka, christian.storm, michael.adler,
simon.sudler, cedric.hombourger, adriaan.schmidt,
felix.moessbauer, ubely
When using a cross build this patch does introduce optionally
host and target specific kernel kbuild packages that
ship the "scripts" and "tools" binaries.
By default at cross builds a kbuild-cross package will be
generated that provides the host specific kbuild package.
The "-compat" and "-native" multiarch bitbake targets are useable to run
additional target or host specific builds for kbuild scripts and tools.
Using the "-compat" bitbake target enables the build of
a target specific kbuild package at cross builds.
Using the "-native" bitbake target enables the build of
a host specific kbuild package at cross builds.
This solves this from doc/custom_kernel.inc:
- The kernel headers package has not supported both native
and cross compilation of kernel modules when itself was cross built
- Future roadmap: Generate kernel headers package for both host
and target when using a cross build
Signed-off-by: Stefan Koch <stefan-koch@siemens.com>
---
.../linux/files/debian/control.tmpl | 16 +++++-
.../linux/files/debian/isar/build.tmpl | 12 ++++-
.../linux/files/debian/isar/common.tmpl | 8 +++
.../linux/files/debian/isar/install.tmpl | 26 ++++++----
.../linux/files/debian/rules.tmpl | 2 +-
meta/recipes-kernel/linux/linux-custom.inc | 49 ++++++++++++++++---
6 files changed, 94 insertions(+), 19 deletions(-)
diff --git a/meta/recipes-kernel/linux/files/debian/control.tmpl b/meta/recipes-kernel/linux/files/debian/control.tmpl
index b19ca2c9..c012048b 100644
--- a/meta/recipes-kernel/linux/files/debian/control.tmpl
+++ b/meta/recipes-kernel/linux/files/debian/control.tmpl
@@ -6,6 +6,7 @@ Build-Depends: bc, kmod, cpio, ${KBUILD_DEPENDS}
Homepage: http://www.kernel.org/
Package: linux-image-${KERNEL_NAME_PROVIDED}
+Build-Profiles: <!targetbuild !hostbuild>
Architecture: any
Depends: ${KERNEL_DEBIAN_DEPENDS}
Description: ${KERNEL_NAME_PROVIDED} Linux kernel, version @KR@
@@ -13,6 +14,7 @@ Description: ${KERNEL_NAME_PROVIDED} Linux kernel, version @KR@
files, version: @KR@.
Package: linux-headers-${KERNEL_NAME_PROVIDED}
+Build-Profiles: <!targetbuild !hostbuild>
Architecture: any
Depends: ${KERNEL_HEADERS_DEBIAN_DEPENDS}, ${perl:Depends}, ${shlib:Depends}
Description: ${KERNEL_NAME_PROVIDED} Linux kernel headers for @KR@
@@ -21,7 +23,7 @@ Description: ${KERNEL_NAME_PROVIDED} Linux kernel headers for @KR@
This is useful for people who need to build external modules
Package: linux-libc-dev
-Build-Profiles: <!nolibcdev>
+Build-Profiles: <!nolibcdev !targetbuild !hostbuild>
Section: devel
Provides: linux-kernel-headers
Architecture: any
@@ -30,6 +32,7 @@ Description: Linux support headers for userspace development
are used by the installed headers for GNU glibc and other system libraries.
Package: linux-image-${KERNEL_NAME_PROVIDED}-dbg
+Build-Profiles: <!targetbuild !hostbuild>
Section: debug
Architecture: any
Description: Linux kernel debugging symbols for @KR@
@@ -37,8 +40,19 @@ Description: Linux kernel debugging symbols for @KR@
all the necessary debug symbols for the kernel and its modules.
Package: linux-kbuild-${KERNEL_NAME_PROVIDED}
+Build-Profiles: <targetbuild> <hostbuild> <defaultkbuild>
Architecture: any
Depends: ${perl:Depends}, ${shlib:Depends}
Description: ${KERNEL_NAME_PROVIDED} Linux kbuild scripts and tools for @KR@
This package provides kernel kbuild scripts and tools for @KR@
This is useful for people who need to build external modules
+
+Package: linux-kbuild-${KERNEL_NAME_PROVIDED}-cross
+Build-Profiles: <!targetbuild !hostbuild crosskbuild>
+Architecture: any
+Depends: ${KERNEL_HEADERS_DEBIAN_DEPENDS}, ${perl:Depends}, ${shlib:Depends}
+Description: ${KERNEL_NAME_PROVIDED} Linux kbuild scripts and tools for @KR@
+ This package provides kernel kbuild scripts and tools
+ as ${HOST_ARCH} cross binaries for @KR@
+ This is useful for those who need to cross build
+ external modules using ISAR's sbuild-chroot-host
diff --git a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl
index 906dc580..7f293528 100644
--- a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl
+++ b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl
@@ -22,7 +22,17 @@ do_build() {
sed -i "s/@KR@/${KR}/g" ${S}/debian/control ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.*
# Build the Linux kernel
- ${MAKE} O=${KERNEL_BUILD_DIR} ${PARALLEL_MAKE} KCFLAGS="${KCFLAGS}" KAFLAGS="${KAFLAGS}"
+ if echo "${DEB_BUILD_PROFILES}" | grep -q -e "targetbuild" -e "hostbuild"; then # Build kernel scripts and tools
+ ${MAKE} O=${KERNEL_BUILD_DIR} ${PARALLEL_MAKE} KCFLAGS="${KCFLAGS}" KAFLAGS="${KAFLAGS}" scripts
+ if grep -q -E "CONFIG_STACK_VALIDATION=y|CONFIG_HAVE_OBJTOOL=y" ${KERNEL_BUILD_DIR}/.config && [ -d "tools/objtool" ]; then
+ ${MAKE} O=${KERNEL_BUILD_DIR} ${PARALLEL_MAKE} KCFLAGS="${KCFLAGS}" KAFLAGS="${KAFLAGS}" tools/objtool || true
+ fi
+ if grep -q "CONFIG_MODULES=y" ${KERNEL_BUILD_DIR}/.config; then
+ ${MAKE} O=${KERNEL_BUILD_DIR} ${PARALLEL_MAKE} KCFLAGS="${KCFLAGS}" KAFLAGS="${KAFLAGS}" modules_prepare
+ fi
+ else # Build the Linux kernel
+ ${MAKE} O=${KERNEL_BUILD_DIR} ${PARALLEL_MAKE} KCFLAGS="${KCFLAGS}" KAFLAGS="${KAFLAGS}"
+ fi
# Stop tracing
set +x
diff --git a/meta/recipes-kernel/linux/files/debian/isar/common.tmpl b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl
index f4c0519f..36e01691 100644
--- a/meta/recipes-kernel/linux/files/debian/isar/common.tmpl
+++ b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl
@@ -11,6 +11,14 @@ KERNEL_PKG_KERN_HEADERS=linux-headers-${KERNEL_NAME_PROVIDED}
KERNEL_PKG_LIBC_HEADERS=linux-libc-dev
KERNEL_PKG_KERN_KBUILD=linux-kbuild-${KERNEL_NAME_PROVIDED}
+if echo "${DEB_BUILD_PROFILES}" | grep -q "hostbuild"; then
+ # Force creating kernel kbuild debian package with valid host arch
+ # Use a cross build to comply with arch specific kernel defconfigs
+ # The scripts and tools are always created for host arch
+ eval $(dpkg-architecture -f -A ${DISTRO_ARCH})
+ CROSS_COMPILE=${DEB_TARGET_GNU_TYPE}-
+fi
+
# Constants
KCONF=.config
diff --git a/meta/recipes-kernel/linux/files/debian/isar/install.tmpl b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl
index 236b67c4..08b8a9a5 100644
--- a/meta/recipes-kernel/linux/files/debian/isar/install.tmpl
+++ b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl
@@ -34,18 +34,24 @@ do_install() {
set -x
# Run the install steps
- install_image
- if [ "${ARCH}" != "um" ]; then
- install_config
- install_map
+ if ! echo "${DEB_BUILD_PROFILES}" | grep -q -e "targetbuild" -e "hostbuild"; then
+ install_image
+ if [ "${ARCH}" != "um" ]; then
+ install_config
+ install_map
+ fi
+ install_hooks
+ install_dtbs
+ install_kmods
+ install_headers
fi
- install_hooks
- install_dtbs
- install_kmods
- install_headers
- # Cleanup and install kernel scripts and tools
- rm -rf ${deb_kern_kbuild_dir}
+ if ! echo "${DEB_BUILD_PROFILES}" | grep -q -e "targetbuild" -e "hostbuild" && echo "${DEB_BUILD_PROFILES}" | grep -q "crosskbuild"; then
+ # Install cross kernel scripts and tools
+ install_kbuild ${deb_kern_kbuild_dir}-cross
+ fi
+
+ # Install kernel scripts and tools
install_kbuild ${deb_kern_kbuild_dir}
# Stop tracing
diff --git a/meta/recipes-kernel/linux/files/debian/rules.tmpl b/meta/recipes-kernel/linux/files/debian/rules.tmpl
index e8ae3daa..1f838a41 100755
--- a/meta/recipes-kernel/linux/files/debian/rules.tmpl
+++ b/meta/recipes-kernel/linux/files/debian/rules.tmpl
@@ -48,4 +48,4 @@ override_dh_strip_nondeterminism:
true
override_dh_strip:
- unset DEB_HOST_GNU_TYPE && dh_strip -Xvmlinu --no-automatic-dbgsym
+ dh_strip -Xvmlinu -Xlinux-kbuild-${KERNEL_NAME_PROVIDED}-cross --no-automatic-dbgsym
diff --git a/meta/recipes-kernel/linux/linux-custom.inc b/meta/recipes-kernel/linux/linux-custom.inc
index 94ed84a2..85faa2d7 100644
--- a/meta/recipes-kernel/linux/linux-custom.inc
+++ b/meta/recipes-kernel/linux/linux-custom.inc
@@ -85,9 +85,6 @@ TEMPLATE_VARS += " \
DISTRIBUTOR \
"
-inherit dpkg
-inherit template
-
# Add custom cflags to the kernel build
KCFLAGS ?= "-fdebug-prefix-map=${CURDIR}=."
KAFLAGS ?= "-fdebug-prefix-map=${CURDIR}=."
@@ -95,17 +92,57 @@ KAFLAGS ?= "-fdebug-prefix-map=${CURDIR}=."
# Derive name of the kernel packages from the name of this recipe
KERNEL_NAME_PROVIDED ?= "${@ d.getVar('PN').partition('linux-')[2]}"
-# Make bitbake know we will be producing linux-image and linux-headers packages
python() {
kernel_name = d.getVar("KERNEL_NAME_PROVIDED")
- d.setVar('PROVIDES', 'linux-image-' + kernel_name + ' ' + \
- 'linux-headers-' + kernel_name)
+ distro_arch = d.getVar("DISTRO_ARCH")
+ host_arch = d.getVar("HOST_ARCH")
headers_depends = "linux-kbuild-" + kernel_name
+ # The "-compat" and "-native" multiarch bitbake targets are used to run
+ # additional target or host specific builds for kbuild scripts and tools.
+ # The "-compat" bitbake target requests a "target" build
+ # and the "-native" bitbake target requests a "host" build.
+ #
+ # For different distro and host archs
+ # - Add dependency for sbuild-chroot-target
+ # to allow building arch specific kbuild scripts and tools
+ # - Set correct variable overrides
+ is_compat = "compat" in d.getVar("PN", True)
+ is_native = "native" in d.getVar("PN", True)
+
+ # Determine creation of a kbuild-cross or (standard) kbuild package for the host
+ is_cross_profile = distro_arch != host_arch and d.getVar("ISAR_CROSS_COMPILE", True) == "1"
+
+ if is_cross_profile:
+ headers_depends = headers_depends + "-cross | " + headers_depends
+ build_depends = "crossbuild-essential-" + distro_arch + " [" + host_arch + "], "
+ d.appendVar("DEB_BUILD_PROFILES", " crosskbuild")
+ d.prependVar("KBUILD_DEPENDS", build_depends)
+ d.appendVarFlag("do_prepare_build", "depends", " sbuild-chroot-target:do_build")
+ d.setVar("ISAR_ENABLE_COMPAT_ARCH", "1")
+ d.setVar("COMPAT_DISTRO_ARCH", distro_arch)
+
+ if is_compat:
+ d.appendVar("DEB_BUILD_PROFILES", " targetbuild")
+ d.setVar("ISAR_CROSS_COMPILE", "0")
+ elif is_native:
+ d.appendVar("DEB_BUILD_PROFILES", " hostbuild")
+ else:
+ d.appendVar("DEB_BUILD_PROFILES", " defaultkbuild")
+
+ # Make bitbake know we will be producing
+ # linux-image and linux-headers packages
+ if not is_compat and not is_native:
+ d.setVar("PROVIDES", "linux-image-" + kernel_name + " " + \
+ "linux-headers-" + kernel_name)
+
# Set dependency for kernel headers
d.appendVar("KERNEL_HEADERS_DEBIAN_DEPENDS", headers_depends)
}
+inherit dpkg
+inherit template
+
def get_kernel_arch(d):
distro_arch = d.getVar("DISTRO_ARCH")
if distro_arch in ["amd64", "i386"]:
--
2.39.2
^ permalink raw reply [flat|nested] 17+ messages in thread
* [PATCH v4 3/3] docs: Update custom_kernel docs for split up of kernel scripts and tools
2023-08-21 13:44 [PATCH v4 0/3] linux-custom: Split up binaries from kernel headers to kbuild packages Stefan Koch
2023-08-21 13:44 ` [PATCH v4 1/3] linux-custom: Split up binaries from kernel headers to kbuild package Stefan Koch
2023-08-21 13:45 ` [PATCH v4 2/3] linux-custom: Provide host and target specific kernel kbuild packages Stefan Koch
@ 2023-08-21 13:45 ` Stefan Koch
2023-09-01 10:22 ` [PATCH v4 0/3] linux-custom: Split up binaries from kernel headers to kbuild packages Uladzimir Bely
` (3 subsequent siblings)
6 siblings, 0 replies; 17+ messages in thread
From: Stefan Koch @ 2023-08-21 13:45 UTC (permalink / raw)
To: isar-users
Cc: stefan-koch, jan.kiszka, christian.storm, michael.adler,
simon.sudler, cedric.hombourger, adriaan.schmidt,
felix.moessbauer, ubely
Signed-off-by: Stefan Koch <stefan-koch@siemens.com>
---
doc/custom_kernel.md | 16 +++++++++-------
1 file changed, 9 insertions(+), 7 deletions(-)
diff --git a/doc/custom_kernel.md b/doc/custom_kernel.md
index 73987a5b..0b544713 100644
--- a/doc/custom_kernel.md
+++ b/doc/custom_kernel.md
@@ -38,12 +38,16 @@ The linux-custom recipe provides support for:
7. Allow the name of the kernel image to be changed via `KERNEL_FILE` (defaults
to `vmlinuz`)
- 8. Produce a `linux-headers` package which includes kernel headers and kbuild
- scripts/tools
+ 8. Produce a `linux-headers` package which includes kernel headers
- 9. The `linux-headers` package shall support native and cross compiles of
- out-of-tree kernel modules. However, when built in cross-compilation mode,
- it cannot be used on the target so far.
+ 9. Produce a `linux-kbuild` package for both `host` and/or `target` arch
+ which includes kbuild scripts and tools.
+ When cross building the `-compat` and `-native` multiarch bitbake targets
+ can be used to create additional `target` or `host` specific packages.
+
+ The `linux-headers` package supports native and cross compiles of
+ out-of-tree kernel modules. Even, when built in cross-compilation mode,
+ it can be used on the target using the compat `linux-kbuild` package.
10. Produce a `linux-libc-dev` package to support user-land builds
@@ -72,8 +76,6 @@ In the future, the recipe may be extended to:
3. Be compatible with Ubuntu
- 4. When cross-building, generate kernel-headers for both host and target
-
## Examples
The linux-custom recipe is currently used by the linux-mainline package and is
--
2.39.2
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v4 0/3] linux-custom: Split up binaries from kernel headers to kbuild packages
2023-08-21 13:44 [PATCH v4 0/3] linux-custom: Split up binaries from kernel headers to kbuild packages Stefan Koch
` (2 preceding siblings ...)
2023-08-21 13:45 ` [PATCH v4 3/3] docs: Update custom_kernel docs for split up of kernel scripts and tools Stefan Koch
@ 2023-09-01 10:22 ` Uladzimir Bely
2023-09-05 20:18 ` Jan Kiszka
2023-09-05 20:17 ` Jan Kiszka
` (2 subsequent siblings)
6 siblings, 1 reply; 17+ messages in thread
From: Uladzimir Bely @ 2023-09-01 10:22 UTC (permalink / raw)
To: Stefan Koch, isar-users
Cc: jan.kiszka, christian.storm, michael.adler, simon.sudler,
cedric.hombourger, adriaan.schmidt, felix.moessbauer
On Mon, 2023-08-21 at 15:44 +0200, Stefan Koch wrote:
> Hi
>
> This updated v4 patchset implement suggestions from reviewed v3
> patchset.
> It's now a set of five patches:
> - linux-custom: Split up binaries from kernel headers to kbuild
> package
> - linux-custom: Provide host and target specific kernel kbuild
> packages
> - docs: Update custom_kernel docs for split up of kernel scripts and
> tools
>
> The main use-case was to swap out the binaries
> from the kernel headers into kernel kbuild package.
> This is introduced by the first commit
> "Split up binaries from kernel headers to kbuild package"
>
> The second commit "Provide host and target specific kernel kbuild
> packages"
> introduces that the binaries could be swapped out into host and
> target
> specific kernel kbuild packages.
>
> The main development goals were these:
>
> 1. Solve already known isar custom kernel
> limitations from doc/custom_kernel.inc
> - kernel headers package does not support both native
> and cross compilation of kernel modules when cross built
>
> 2. Honor recommendations for future from doc/custom_kernel.inc
> - Generate kernel headers packages for both host and target
> when using cross build
>
> 3. Add extensions known from debian kernel packages structure
> - Generate a kernel headers package without binaries
> - Create specific kernel kbuild packages that
> will ship the "scripts" and "tools" binaries
> - Use symlinks to point to the "scripts" and "tools" binaries
>
> 4. Be user friendly
> - Avoid redundant configuration of kernel source definitions with
> user
> actions to enable kbuild package generation
> - Use already known way to include linux-custom.inc in just one
> own bitbake recipe that provides the kernel source definitions
> - Keep known user behavior for existing build configurations:
> just update isar, request "-compat" (and "-native") bitbake targets
> and kbuild packages for target and host will be created
> automatically
>
> Best regards
>
> Stefan
>
> Stefan Koch (3):
> linux-custom: Split up binaries from kernel headers to kbuild
> package
> linux-custom: Provide host and target specific kernel kbuild
> packages
> docs: Update custom_kernel docs for split up of kernel scripts and
> tools
>
> doc/custom_kernel.md | 16 ++--
> .../linux/files/debian/control.tmpl | 25 ++++++-
> .../linux/files/debian/isar/build.tmpl | 12 ++-
> .../linux/files/debian/isar/common.tmpl | 10 +++
> .../linux/files/debian/isar/install.tmpl | 73 ++++++++++++++---
> --
> .../linux/files/debian/rules.tmpl | 2 +-
> meta/recipes-kernel/linux/linux-custom.inc | 53 ++++++++++++--
> 7 files changed, 155 insertions(+), 36 deletions(-)
>
This passes fast/full CI, so we tend to apply it next week, if there
are no objections.
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v4 0/3] linux-custom: Split up binaries from kernel headers to kbuild packages
2023-08-21 13:44 [PATCH v4 0/3] linux-custom: Split up binaries from kernel headers to kbuild packages Stefan Koch
` (3 preceding siblings ...)
2023-09-01 10:22 ` [PATCH v4 0/3] linux-custom: Split up binaries from kernel headers to kbuild packages Uladzimir Bely
@ 2023-09-05 20:17 ` Jan Kiszka
2023-10-04 15:36 ` Koch, Stefan
2023-11-21 7:26 ` Uladzimir Bely
2023-12-17 16:50 ` Jan Kiszka
6 siblings, 1 reply; 17+ messages in thread
From: Jan Kiszka @ 2023-09-05 20:17 UTC (permalink / raw)
To: Stefan Koch, isar-users
Cc: christian.storm, michael.adler, simon.sudler, cedric.hombourger,
adriaan.schmidt, felix.moessbauer, ubely
On 21.08.23 15:44, Stefan Koch wrote:
> Hi
>
> This updated v4 patchset implement suggestions from reviewed v3 patchset.
> It's now a set of five patches:
> - linux-custom: Split up binaries from kernel headers to kbuild package
> - linux-custom: Provide host and target specific kernel kbuild packages
> - docs: Update custom_kernel docs for split up of kernel scripts and tools
>
> The main use-case was to swap out the binaries
> from the kernel headers into kernel kbuild package.
> This is introduced by the first commit
> "Split up binaries from kernel headers to kbuild package"
>
> The second commit "Provide host and target specific kernel kbuild packages"
> introduces that the binaries could be swapped out into host and target
> specific kernel kbuild packages.
>
> The main development goals were these:
>
> 1. Solve already known isar custom kernel
> limitations from doc/custom_kernel.inc
> - kernel headers package does not support both native
> and cross compilation of kernel modules when cross built
>
> 2. Honor recommendations for future from doc/custom_kernel.inc
> - Generate kernel headers packages for both host and target
> when using cross build
>
> 3. Add extensions known from debian kernel packages structure
> - Generate a kernel headers package without binaries
> - Create specific kernel kbuild packages that
> will ship the "scripts" and "tools" binaries
> - Use symlinks to point to the "scripts" and "tools" binaries
>
> 4. Be user friendly
> - Avoid redundant configuration of kernel source definitions with user
> actions to enable kbuild package generation
> - Use already known way to include linux-custom.inc in just one
> own bitbake recipe that provides the kernel source definitions
> - Keep known user behavior for existing build configurations:
> just update isar, request "-compat" (and "-native") bitbake targets
> and kbuild packages for target and host will be created automatically
Most things look fairly good, however - maybe it's just too late today -
I'm not yet getting the scenario where -compat plays a role. My
impression is right now, you are misusing this for some scenario it is
not designed for (compat = 32-bit package for 64-bit target). Or am I
misinterpreting this?
Jan
--
Siemens AG, Technology
Linux Expert Center
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v4 0/3] linux-custom: Split up binaries from kernel headers to kbuild packages
2023-09-01 10:22 ` [PATCH v4 0/3] linux-custom: Split up binaries from kernel headers to kbuild packages Uladzimir Bely
@ 2023-09-05 20:18 ` Jan Kiszka
0 siblings, 0 replies; 17+ messages in thread
From: Jan Kiszka @ 2023-09-05 20:18 UTC (permalink / raw)
To: Uladzimir Bely, Stefan Koch, isar-users
Cc: christian.storm, michael.adler, simon.sudler, cedric.hombourger,
adriaan.schmidt, felix.moessbauer
On 01.09.23 12:22, Uladzimir Bely wrote:
> On Mon, 2023-08-21 at 15:44 +0200, Stefan Koch wrote:
>> Hi
>>
>> This updated v4 patchset implement suggestions from reviewed v3
>> patchset.
>> It's now a set of five patches:
>> - linux-custom: Split up binaries from kernel headers to kbuild
>> package
>> - linux-custom: Provide host and target specific kernel kbuild
>> packages
>> - docs: Update custom_kernel docs for split up of kernel scripts and
>> tools
>>
>> The main use-case was to swap out the binaries
>> from the kernel headers into kernel kbuild package.
>> This is introduced by the first commit
>> "Split up binaries from kernel headers to kbuild package"
>>
>> The second commit "Provide host and target specific kernel kbuild
>> packages"
>> introduces that the binaries could be swapped out into host and
>> target
>> specific kernel kbuild packages.
>>
>> The main development goals were these:
>>
>> 1. Solve already known isar custom kernel
>> limitations from doc/custom_kernel.inc
>> - kernel headers package does not support both native
>> and cross compilation of kernel modules when cross built
>>
>> 2. Honor recommendations for future from doc/custom_kernel.inc
>> - Generate kernel headers packages for both host and target
>> when using cross build
>>
>> 3. Add extensions known from debian kernel packages structure
>> - Generate a kernel headers package without binaries
>> - Create specific kernel kbuild packages that
>> will ship the "scripts" and "tools" binaries
>> - Use symlinks to point to the "scripts" and "tools" binaries
>>
>> 4. Be user friendly
>> - Avoid redundant configuration of kernel source definitions with
>> user
>> actions to enable kbuild package generation
>> - Use already known way to include linux-custom.inc in just one
>> own bitbake recipe that provides the kernel source definitions
>> - Keep known user behavior for existing build configurations:
>> just update isar, request "-compat" (and "-native") bitbake targets
>> and kbuild packages for target and host will be created
>> automatically
>>
>> Best regards
>>
>> Stefan
>>
>> Stefan Koch (3):
>> linux-custom: Split up binaries from kernel headers to kbuild
>> package
>> linux-custom: Provide host and target specific kernel kbuild
>> packages
>> docs: Update custom_kernel docs for split up of kernel scripts and
>> tools
>>
>> doc/custom_kernel.md | 16 ++--
>> .../linux/files/debian/control.tmpl | 25 ++++++-
>> .../linux/files/debian/isar/build.tmpl | 12 ++-
>> .../linux/files/debian/isar/common.tmpl | 10 +++
>> .../linux/files/debian/isar/install.tmpl | 73 ++++++++++++++---
>> --
>> .../linux/files/debian/rules.tmpl | 2 +-
>> meta/recipes-kernel/linux/linux-custom.inc | 53 ++++++++++++--
>> 7 files changed, 155 insertions(+), 36 deletions(-)
>>
>
> This passes fast/full CI, so we tend to apply it next week, if there
> are no objections.
I have some remaining question on the -compat scenario, but I'm getting
an OOO notice (internally) from Stefan. Unless someone else can answer
that question, we may need to wait until he returned.
Jan
--
Siemens AG, Technology
Linux Expert Center
^ permalink raw reply [flat|nested] 17+ messages in thread
* RE: [PATCH v4 2/3] linux-custom: Provide host and target specific kernel kbuild packages
2023-08-21 13:45 ` [PATCH v4 2/3] linux-custom: Provide host and target specific kernel kbuild packages Stefan Koch
@ 2023-09-06 6:38 ` Schmidt, Adriaan
2023-10-04 15:34 ` Koch, Stefan
0 siblings, 1 reply; 17+ messages in thread
From: Schmidt, Adriaan @ 2023-09-06 6:38 UTC (permalink / raw)
To: Koch, Stefan, isar-users
Cc: Kiszka, Jan, Storm, Christian, Adler, Michael, Sudler, Simon,
cedric.hombourger, MOESSBAUER, FELIX JONATHAN, ubely
Koch, Stefan (DI PA DCP R&D 3) <stefan-koch@siemens.com>, Monday, August 21, 2023 3:45 PM:
> When using a cross build this patch does introduce optionally
> host and target specific kernel kbuild packages that
> ship the "scripts" and "tools" binaries.
>
> By default at cross builds a kbuild-cross package will be
> generated that provides the host specific kbuild package.
>
> The "-compat" and "-native" multiarch bitbake targets are useable to run
> additional target or host specific builds for kbuild scripts and tools.
>
> Using the "-compat" bitbake target enables the build of
> a target specific kbuild package at cross builds.
>
> Using the "-native" bitbake target enables the build of
> a host specific kbuild package at cross builds.
>
> This solves this from doc/custom_kernel.inc:
> - The kernel headers package has not supported both native
> and cross compilation of kernel modules when itself was cross built
> - Future roadmap: Generate kernel headers package for both host
> and target when using a cross build
>
> Signed-off-by: Stefan Koch <stefan-koch@siemens.com>
> ---
> .../linux/files/debian/control.tmpl | 16 +++++-
> .../linux/files/debian/isar/build.tmpl | 12 ++++-
> .../linux/files/debian/isar/common.tmpl | 8 +++
> .../linux/files/debian/isar/install.tmpl | 26 ++++++----
> .../linux/files/debian/rules.tmpl | 2 +-
> meta/recipes-kernel/linux/linux-custom.inc | 49 ++++++++++++++++---
> 6 files changed, 94 insertions(+), 19 deletions(-)
>
> diff --git a/meta/recipes-kernel/linux/files/debian/control.tmpl
> b/meta/recipes-kernel/linux/files/debian/control.tmpl
> index b19ca2c9..c012048b 100644
> --- a/meta/recipes-kernel/linux/files/debian/control.tmpl
> +++ b/meta/recipes-kernel/linux/files/debian/control.tmpl
> @@ -6,6 +6,7 @@ Build-Depends: bc, kmod, cpio, ${KBUILD_DEPENDS}
> Homepage: http://www.kernel.org/
>
> Package: linux-image-${KERNEL_NAME_PROVIDED}
> +Build-Profiles: <!targetbuild !hostbuild>
> Architecture: any
> Depends: ${KERNEL_DEBIAN_DEPENDS}
> Description: ${KERNEL_NAME_PROVIDED} Linux kernel, version @KR@
> @@ -13,6 +14,7 @@ Description: ${KERNEL_NAME_PROVIDED} Linux kernel, version
> @KR@
> files, version: @KR@.
>
> Package: linux-headers-${KERNEL_NAME_PROVIDED}
> +Build-Profiles: <!targetbuild !hostbuild>
> Architecture: any
> Depends: ${KERNEL_HEADERS_DEBIAN_DEPENDS}, ${perl:Depends}, ${shlib:Depends}
> Description: ${KERNEL_NAME_PROVIDED} Linux kernel headers for @KR@
> @@ -21,7 +23,7 @@ Description: ${KERNEL_NAME_PROVIDED} Linux kernel headers
> for @KR@
> This is useful for people who need to build external modules
>
> Package: linux-libc-dev
> -Build-Profiles: <!nolibcdev>
> +Build-Profiles: <!nolibcdev !targetbuild !hostbuild>
> Section: devel
> Provides: linux-kernel-headers
> Architecture: any
> @@ -30,6 +32,7 @@ Description: Linux support headers for userspace
> development
> are used by the installed headers for GNU glibc and other system libraries.
>
> Package: linux-image-${KERNEL_NAME_PROVIDED}-dbg
> +Build-Profiles: <!targetbuild !hostbuild>
> Section: debug
> Architecture: any
> Description: Linux kernel debugging symbols for @KR@
> @@ -37,8 +40,19 @@ Description: Linux kernel debugging symbols for @KR@
> all the necessary debug symbols for the kernel and its modules.
>
> Package: linux-kbuild-${KERNEL_NAME_PROVIDED}
> +Build-Profiles: <targetbuild> <hostbuild> <defaultkbuild>
> Architecture: any
> Depends: ${perl:Depends}, ${shlib:Depends}
> Description: ${KERNEL_NAME_PROVIDED} Linux kbuild scripts and tools for @KR@
> This package provides kernel kbuild scripts and tools for @KR@
> This is useful for people who need to build external modules
> +
> +Package: linux-kbuild-${KERNEL_NAME_PROVIDED}-cross
> +Build-Profiles: <!targetbuild !hostbuild crosskbuild>
> +Architecture: any
> +Depends: ${KERNEL_HEADERS_DEBIAN_DEPENDS}, ${perl:Depends}, ${shlib:Depends}
> +Description: ${KERNEL_NAME_PROVIDED} Linux kbuild scripts and tools for @KR@
> + This package provides kernel kbuild scripts and tools
> + as ${HOST_ARCH} cross binaries for @KR@
> + This is useful for those who need to cross build
> + external modules using ISAR's sbuild-chroot-host
> diff --git a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl
> b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl
> index 906dc580..7f293528 100644
> --- a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl
> +++ b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl
> @@ -22,7 +22,17 @@ do_build() {
> sed -i "s/@KR@/${KR}/g" ${S}/debian/control ${S}/debian/linux-image-
> ${KERNEL_NAME_PROVIDED}.*
>
> # Build the Linux kernel
> - ${MAKE} O=${KERNEL_BUILD_DIR} ${PARALLEL_MAKE} KCFLAGS="${KCFLAGS}"
> KAFLAGS="${KAFLAGS}"
> + if echo "${DEB_BUILD_PROFILES}" | grep -q -e "targetbuild" -e
> "hostbuild"; then # Build kernel scripts and tools
> + ${MAKE} O=${KERNEL_BUILD_DIR} ${PARALLEL_MAKE} KCFLAGS="${KCFLAGS}"
> KAFLAGS="${KAFLAGS}" scripts
> + if grep -q -E "CONFIG_STACK_VALIDATION=y|CONFIG_HAVE_OBJTOOL=y"
> ${KERNEL_BUILD_DIR}/.config && [ -d "tools/objtool" ]; then
> + ${MAKE} O=${KERNEL_BUILD_DIR} ${PARALLEL_MAKE}
> KCFLAGS="${KCFLAGS}" KAFLAGS="${KAFLAGS}" tools/objtool || true
> + fi
> + if grep -q "CONFIG_MODULES=y" ${KERNEL_BUILD_DIR}/.config; then
> + ${MAKE} O=${KERNEL_BUILD_DIR} ${PARALLEL_MAKE}
> KCFLAGS="${KCFLAGS}" KAFLAGS="${KAFLAGS}" modules_prepare
> + fi
> + else # Build the Linux kernel
> + ${MAKE} O=${KERNEL_BUILD_DIR} ${PARALLEL_MAKE} KCFLAGS="${KCFLAGS}"
> KAFLAGS="${KAFLAGS}"
> + fi
>
> # Stop tracing
> set +x
> diff --git a/meta/recipes-kernel/linux/files/debian/isar/common.tmpl
> b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl
> index f4c0519f..36e01691 100644
> --- a/meta/recipes-kernel/linux/files/debian/isar/common.tmpl
> +++ b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl
> @@ -11,6 +11,14 @@ KERNEL_PKG_KERN_HEADERS=linux-headers-
> ${KERNEL_NAME_PROVIDED}
> KERNEL_PKG_LIBC_HEADERS=linux-libc-dev
> KERNEL_PKG_KERN_KBUILD=linux-kbuild-${KERNEL_NAME_PROVIDED}
>
> +if echo "${DEB_BUILD_PROFILES}" | grep -q "hostbuild"; then
> + # Force creating kernel kbuild debian package with valid host arch
> + # Use a cross build to comply with arch specific kernel defconfigs
> + # The scripts and tools are always created for host arch
> + eval $(dpkg-architecture -f -A ${DISTRO_ARCH})
> + CROSS_COMPILE=${DEB_TARGET_GNU_TYPE}-
> +fi
> +
> # Constants
> KCONF=.config
>
> diff --git a/meta/recipes-kernel/linux/files/debian/isar/install.tmpl
> b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl
> index 236b67c4..08b8a9a5 100644
> --- a/meta/recipes-kernel/linux/files/debian/isar/install.tmpl
> +++ b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl
> @@ -34,18 +34,24 @@ do_install() {
> set -x
>
> # Run the install steps
> - install_image
> - if [ "${ARCH}" != "um" ]; then
> - install_config
> - install_map
> + if ! echo "${DEB_BUILD_PROFILES}" | grep -q -e "targetbuild" -e
> "hostbuild"; then
> + install_image
> + if [ "${ARCH}" != "um" ]; then
> + install_config
> + install_map
> + fi
> + install_hooks
> + install_dtbs
> + install_kmods
> + install_headers
> fi
> - install_hooks
> - install_dtbs
> - install_kmods
> - install_headers
>
> - # Cleanup and install kernel scripts and tools
> - rm -rf ${deb_kern_kbuild_dir}
> + if ! echo "${DEB_BUILD_PROFILES}" | grep -q -e "targetbuild" -e
> "hostbuild" && echo "${DEB_BUILD_PROFILES}" | grep -q "crosskbuild"; then
> + # Install cross kernel scripts and tools
> + install_kbuild ${deb_kern_kbuild_dir}-cross
> + fi
> +
> + # Install kernel scripts and tools
> install_kbuild ${deb_kern_kbuild_dir}
>
> # Stop tracing
> diff --git a/meta/recipes-kernel/linux/files/debian/rules.tmpl
> b/meta/recipes-kernel/linux/files/debian/rules.tmpl
> index e8ae3daa..1f838a41 100755
> --- a/meta/recipes-kernel/linux/files/debian/rules.tmpl
> +++ b/meta/recipes-kernel/linux/files/debian/rules.tmpl
> @@ -48,4 +48,4 @@ override_dh_strip_nondeterminism:
> true
>
> override_dh_strip:
> - unset DEB_HOST_GNU_TYPE && dh_strip -Xvmlinu --no-automatic-dbgsym
> + dh_strip -Xvmlinu -Xlinux-kbuild-${KERNEL_NAME_PROVIDED}-cross --no-
> automatic-dbgsym
> diff --git a/meta/recipes-kernel/linux/linux-custom.inc b/meta/recipes-
> kernel/linux/linux-custom.inc
> index 94ed84a2..85faa2d7 100644
> --- a/meta/recipes-kernel/linux/linux-custom.inc
> +++ b/meta/recipes-kernel/linux/linux-custom.inc
> @@ -85,9 +85,6 @@ TEMPLATE_VARS += " \
> DISTRIBUTOR \
> "
>
> -inherit dpkg
> -inherit template
> -
> # Add custom cflags to the kernel build
> KCFLAGS ?= "-fdebug-prefix-map=${CURDIR}=."
> KAFLAGS ?= "-fdebug-prefix-map=${CURDIR}=."
> @@ -95,17 +92,57 @@ KAFLAGS ?= "-fdebug-prefix-map=${CURDIR}=."
> # Derive name of the kernel packages from the name of this recipe
> KERNEL_NAME_PROVIDED ?= "${@ d.getVar('PN').partition('linux-')[2]}"
>
> -# Make bitbake know we will be producing linux-image and linux-headers
> packages
> python() {
> kernel_name = d.getVar("KERNEL_NAME_PROVIDED")
> - d.setVar('PROVIDES', 'linux-image-' + kernel_name + ' ' + \
> - 'linux-headers-' + kernel_name)
> + distro_arch = d.getVar("DISTRO_ARCH")
> + host_arch = d.getVar("HOST_ARCH")
> headers_depends = "linux-kbuild-" + kernel_name
>
> + # The "-compat" and "-native" multiarch bitbake targets are used to run
> + # additional target or host specific builds for kbuild scripts and
> tools.
> + # The "-compat" bitbake target requests a "target" build
> + # and the "-native" bitbake target requests a "host" build.
> + #
> + # For different distro and host archs
> + # - Add dependency for sbuild-chroot-target
> + # to allow building arch specific kbuild scripts and tools
> + # - Set correct variable overrides
> + is_compat = "compat" in d.getVar("PN", True)
> + is_native = "native" in d.getVar("PN", True)
Just a minor thing: The intended/preferred interface for recipes are the
"class-native" and "class-compat" overrides set in multiarch.bbclass.
This is nice for bitbake (just set MY_VAR:class-native = "value"), but
unfortunately not so easy to use from Python code. Still, I'd avoid
looking at PN again in the recipes, and instead go for
is_compat = "class-compat" in d.getVar("OVERRIDES").split(":")
> +
> + # Determine creation of a kbuild-cross or (standard) kbuild package for
> the host
> + is_cross_profile = distro_arch != host_arch and
> d.getVar("ISAR_CROSS_COMPILE", True) == "1"
> +
> + if is_cross_profile:
> + headers_depends = headers_depends + "-cross | " + headers_depends
> + build_depends = "crossbuild-essential-" + distro_arch + " [" +
> host_arch + "], "
> + d.appendVar("DEB_BUILD_PROFILES", " crosskbuild")
> + d.prependVar("KBUILD_DEPENDS", build_depends)
> + d.appendVarFlag("do_prepare_build", "depends", " sbuild-chroot-
> target:do_build")
> + d.setVar("ISAR_ENABLE_COMPAT_ARCH", "1")
> + d.setVar("COMPAT_DISTRO_ARCH", distro_arch)
This feels hacky... it overrides the compat mechanism, resulting in a
"*-compat" package that is not actually for the compat arch.
Adriaan
> +
> + if is_compat:
> + d.appendVar("DEB_BUILD_PROFILES", " targetbuild")
> + d.setVar("ISAR_CROSS_COMPILE", "0")
> + elif is_native:
> + d.appendVar("DEB_BUILD_PROFILES", " hostbuild")
> + else:
> + d.appendVar("DEB_BUILD_PROFILES", " defaultkbuild")
> +
> + # Make bitbake know we will be producing
> + # linux-image and linux-headers packages
> + if not is_compat and not is_native:
> + d.setVar("PROVIDES", "linux-image-" + kernel_name + " " + \
> + "linux-headers-" + kernel_name)
> +
> # Set dependency for kernel headers
> d.appendVar("KERNEL_HEADERS_DEBIAN_DEPENDS", headers_depends)
> }
>
> +inherit dpkg
> +inherit template
> +
> def get_kernel_arch(d):
> distro_arch = d.getVar("DISTRO_ARCH")
> if distro_arch in ["amd64", "i386"]:
> --
> 2.39.2
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v4 2/3] linux-custom: Provide host and target specific kernel kbuild packages
2023-09-06 6:38 ` Schmidt, Adriaan
@ 2023-10-04 15:34 ` Koch, Stefan
2023-10-04 16:15 ` Jan Kiszka
0 siblings, 1 reply; 17+ messages in thread
From: Koch, Stefan @ 2023-10-04 15:34 UTC (permalink / raw)
To: Schmidt, Adriaan, isar-users
Cc: Sudler, Simon, Storm, Christian, ubely, Kiszka, Jan, Adler,
Michael, MOESSBAUER, FELIX JONATHAN, cedric.hombourger
On Wed, 2023-09-06 at 06:38 +0000, Schmidt, Adriaan (T CED SES-DE)
wrote:
> Koch, Stefan (DI PA DCP R&D 3) <stefan-koch@siemens.com>, Monday,
> August 21, 2023 3:45 PM:
> > When using a cross build this patch does introduce optionally
> > host and target specific kernel kbuild packages that
> > ship the "scripts" and "tools" binaries.
> >
> > By default at cross builds a kbuild-cross package will be
> > generated that provides the host specific kbuild package.
> >
> > The "-compat" and "-native" multiarch bitbake targets are useable
> > to run
> > additional target or host specific builds for kbuild scripts and
> > tools.
> >
> > Using the "-compat" bitbake target enables the build of
> > a target specific kbuild package at cross builds.
> >
> > Using the "-native" bitbake target enables the build of
> > a host specific kbuild package at cross builds.
> >
> > This solves this from doc/custom_kernel.inc:
> > - The kernel headers package has not supported both native
> > and cross compilation of kernel modules when itself was cross
> > built
> > - Future roadmap: Generate kernel headers package for both host
> > and target when using a cross build
> >
> > Signed-off-by: Stefan Koch <stefan-koch@siemens.com>
> > ---
> > .../linux/files/debian/control.tmpl | 16 +++++-
> > .../linux/files/debian/isar/build.tmpl | 12 ++++-
> > .../linux/files/debian/isar/common.tmpl | 8 +++
> > .../linux/files/debian/isar/install.tmpl | 26 ++++++----
> > .../linux/files/debian/rules.tmpl | 2 +-
> > meta/recipes-kernel/linux/linux-custom.inc | 49
> > ++++++++++++++++---
> > 6 files changed, 94 insertions(+), 19 deletions(-)
> >
> > diff --git a/meta/recipes-kernel/linux/files/debian/control.tmpl
> > b/meta/recipes-kernel/linux/files/debian/control.tmpl
> > index b19ca2c9..c012048b 100644
> > --- a/meta/recipes-kernel/linux/files/debian/control.tmpl
> > +++ b/meta/recipes-kernel/linux/files/debian/control.tmpl
> > @@ -6,6 +6,7 @@ Build-Depends: bc, kmod, cpio, ${KBUILD_DEPENDS}
> > Homepage:
> > http://www.kernel.org/
> >
> > Package: linux-image-${KERNEL_NAME_PROVIDED}
> > +Build-Profiles: <!targetbuild !hostbuild>
> > Architecture: any
> > Depends: ${KERNEL_DEBIAN_DEPENDS}
> > Description: ${KERNEL_NAME_PROVIDED} Linux kernel, version @KR@
> > @@ -13,6 +14,7 @@ Description: ${KERNEL_NAME_PROVIDED} Linux
> > kernel, version
> > @KR@
> > files, version: @KR@.
> >
> > Package: linux-headers-${KERNEL_NAME_PROVIDED}
> > +Build-Profiles: <!targetbuild !hostbuild>
> > Architecture: any
> > Depends: ${KERNEL_HEADERS_DEBIAN_DEPENDS}, ${perl:Depends},
> > ${shlib:Depends}
> > Description: ${KERNEL_NAME_PROVIDED} Linux kernel headers for @KR@
> > @@ -21,7 +23,7 @@ Description: ${KERNEL_NAME_PROVIDED} Linux kernel
> > headers
> > for @KR@
> > This is useful for people who need to build external modules
> >
> > Package: linux-libc-dev
> > -Build-Profiles: <!nolibcdev>
> > +Build-Profiles: <!nolibcdev !targetbuild !hostbuild>
> > Section: devel
> > Provides: linux-kernel-headers
> > Architecture: any
> > @@ -30,6 +32,7 @@ Description: Linux support headers for userspace
> > development
> > are used by the installed headers for GNU glibc and other system
> > libraries.
> >
> > Package: linux-image-${KERNEL_NAME_PROVIDED}-dbg
> > +Build-Profiles: <!targetbuild !hostbuild>
> > Section: debug
> > Architecture: any
> > Description: Linux kernel debugging symbols for @KR@
> > @@ -37,8 +40,19 @@ Description: Linux kernel debugging symbols for
> > @KR@
> > all the necessary debug symbols for the kernel and its modules.
> >
> > Package: linux-kbuild-${KERNEL_NAME_PROVIDED}
> > +Build-Profiles: <targetbuild> <hostbuild> <defaultkbuild>
> > Architecture: any
> > Depends: ${perl:Depends}, ${shlib:Depends}
> > Description: ${KERNEL_NAME_PROVIDED} Linux kbuild scripts and
> > tools for @KR@
> > This package provides kernel kbuild scripts and tools for @KR@
> > This is useful for people who need to build external modules
> > +
> > +Package: linux-kbuild-${KERNEL_NAME_PROVIDED}-cross
> > +Build-Profiles: <!targetbuild !hostbuild crosskbuild>
> > +Architecture: any
> > +Depends: ${KERNEL_HEADERS_DEBIAN_DEPENDS}, ${perl:Depends},
> > ${shlib:Depends}
> > +Description: ${KERNEL_NAME_PROVIDED} Linux kbuild scripts and
> > tools for @KR@
> > + This package provides kernel kbuild scripts and tools
> > + as ${HOST_ARCH} cross binaries for @KR@
> > + This is useful for those who need to cross build
> > + external modules using ISAR's sbuild-chroot-host
> > diff --git a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl
> > b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl
> > index 906dc580..7f293528 100644
> > --- a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl
> > +++ b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl
> > @@ -22,7 +22,17 @@ do_build() {
> > sed -i "s/@KR@/${KR}/g" ${S}/debian/control ${S}/debian/linux-
> > image-
> > ${KERNEL_NAME_PROVIDED}.*
> >
> > # Build the Linux kernel
> > - ${MAKE} O=${KERNEL_BUILD_DIR} ${PARALLEL_MAKE}
> > KCFLAGS="${KCFLAGS}"
> > KAFLAGS="${KAFLAGS}"
> > + if echo "${DEB_BUILD_PROFILES}" | grep -q -e "targetbuild" -e
> > "hostbuild"; then # Build kernel scripts and tools
> > + ${MAKE} O=${KERNEL_BUILD_DIR} ${PARALLEL_MAKE}
> > KCFLAGS="${KCFLAGS}"
> > KAFLAGS="${KAFLAGS}" scripts
> > + if grep -q -E
> > "CONFIG_STACK_VALIDATION=y|CONFIG_HAVE_OBJTOOL=y"
> > ${KERNEL_BUILD_DIR}/.config && [ -d "tools/objtool" ]; then
> > + ${MAKE} O=${KERNEL_BUILD_DIR} ${PARALLEL_MAKE}
> > KCFLAGS="${KCFLAGS}" KAFLAGS="${KAFLAGS}" tools/objtool || true
> > + fi
> > + if grep -q "CONFIG_MODULES=y" ${KERNEL_BUILD_DIR}/.config;
> > then
> > + ${MAKE} O=${KERNEL_BUILD_DIR} ${PARALLEL_MAKE}
> > KCFLAGS="${KCFLAGS}" KAFLAGS="${KAFLAGS}" modules_prepare
> > + fi
> > + else # Build the Linux kernel
> > + ${MAKE} O=${KERNEL_BUILD_DIR} ${PARALLEL_MAKE}
> > KCFLAGS="${KCFLAGS}"
> > KAFLAGS="${KAFLAGS}"
> > + fi
> >
> > # Stop tracing
> > set +x
> > diff --git a/meta/recipes-
> > kernel/linux/files/debian/isar/common.tmpl
> > b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl
> > index f4c0519f..36e01691 100644
> > --- a/meta/recipes-kernel/linux/files/debian/isar/common.tmpl
> > +++ b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl
> > @@ -11,6 +11,14 @@ KERNEL_PKG_KERN_HEADERS=linux-headers-
> > ${KERNEL_NAME_PROVIDED}
> > KERNEL_PKG_LIBC_HEADERS=linux-libc-dev
> > KERNEL_PKG_KERN_KBUILD=linux-kbuild-${KERNEL_NAME_PROVIDED}
> >
> > +if echo "${DEB_BUILD_PROFILES}" | grep -q "hostbuild"; then
> > + # Force creating kernel kbuild debian package with valid
> > host arch
> > + # Use a cross build to comply with arch specific kernel
> > defconfigs
> > + # The scripts and tools are always created for host arch
> > + eval $(dpkg-architecture -f -A ${DISTRO_ARCH})
> > + CROSS_COMPILE=${DEB_TARGET_GNU_TYPE}-
> > +fi
> > +
> > # Constants
> > KCONF=.config
> >
> > diff --git a/meta/recipes-
> > kernel/linux/files/debian/isar/install.tmpl
> > b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl
> > index 236b67c4..08b8a9a5 100644
> > --- a/meta/recipes-kernel/linux/files/debian/isar/install.tmpl
> > +++ b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl
> > @@ -34,18 +34,24 @@ do_install() {
> > set -x
> >
> > # Run the install steps
> > - install_image
> > - if [ "${ARCH}" != "um" ]; then
> > - install_config
> > - install_map
> > + if ! echo "${DEB_BUILD_PROFILES}" | grep -q -e "targetbuild" -
> > e
> > "hostbuild"; then
> > + install_image
> > + if [ "${ARCH}" != "um" ]; then
> > + install_config
> > + install_map
> > + fi
> > + install_hooks
> > + install_dtbs
> > + install_kmods
> > + install_headers
> > fi
> > - install_hooks
> > - install_dtbs
> > - install_kmods
> > - install_headers
> >
> > - # Cleanup and install kernel scripts and tools
> > - rm -rf ${deb_kern_kbuild_dir}
> > + if ! echo "${DEB_BUILD_PROFILES}" | grep -q -e "targetbuild" -
> > e
> > "hostbuild" && echo "${DEB_BUILD_PROFILES}" | grep -q
> > "crosskbuild"; then
> > + # Install cross kernel scripts and tools
> > + install_kbuild ${deb_kern_kbuild_dir}-cross
> > + fi
> > +
> > + # Install kernel scripts and tools
> > install_kbuild ${deb_kern_kbuild_dir}
> >
> > # Stop tracing
> > diff --git a/meta/recipes-kernel/linux/files/debian/rules.tmpl
> > b/meta/recipes-kernel/linux/files/debian/rules.tmpl
> > index e8ae3daa..1f838a41 100755
> > --- a/meta/recipes-kernel/linux/files/debian/rules.tmpl
> > +++ b/meta/recipes-kernel/linux/files/debian/rules.tmpl
> > @@ -48,4 +48,4 @@ override_dh_strip_nondeterminism:
> > true
> >
> > override_dh_strip:
> > - unset DEB_HOST_GNU_TYPE && dh_strip -Xvmlinu --no-
> > automatic-dbgsym
> > + dh_strip -Xvmlinu -Xlinux-kbuild-${KERNEL_NAME_PROVIDED}-
> > cross --no-
> > automatic-dbgsym
> > diff --git a/meta/recipes-kernel/linux/linux-custom.inc
> > b/meta/recipes-
> > kernel/linux/linux-custom.inc
> > index 94ed84a2..85faa2d7 100644
> > --- a/meta/recipes-kernel/linux/linux-custom.inc
> > +++ b/meta/recipes-kernel/linux/linux-custom.inc
> > @@ -85,9 +85,6 @@ TEMPLATE_VARS += " \
> > DISTRIBUTOR \
> > "
> >
> > -inherit dpkg
> > -inherit template
> > -
> > # Add custom cflags to the kernel build
> > KCFLAGS ?= "-fdebug-prefix-map=${CURDIR}=."
> > KAFLAGS ?= "-fdebug-prefix-map=${CURDIR}=."
> > @@ -95,17 +92,57 @@ KAFLAGS ?= "-fdebug-prefix-map=${CURDIR}=."
> > # Derive name of the kernel packages from the name of this recipe
> > KERNEL_NAME_PROVIDED ?= "${@ d.getVar('PN').partition('linux-
> > ')[2]}"
> >
> > -# Make bitbake know we will be producing linux-image and linux-
> > headers
> > packages
> > python() {
> > kernel_name = d.getVar("KERNEL_NAME_PROVIDED")
> > - d.setVar('PROVIDES', 'linux-image-' + kernel_name + ' ' + \
> > - 'linux-headers-' + kernel_name)
> > + distro_arch = d.getVar("DISTRO_ARCH")
> > + host_arch = d.getVar("HOST_ARCH")
> > headers_depends = "linux-kbuild-" + kernel_name
> >
> > + # The "-compat" and "-native" multiarch bitbake targets are
> > used to run
> > + # additional target or host specific builds for kbuild scripts
> > and
> > tools.
> > + # The "-compat" bitbake target requests a "target" build
> > + # and the "-native" bitbake target requests a "host" build.
> > + #
> > + # For different distro and host archs
> > + # - Add dependency for sbuild-chroot-target
> > + # to allow building arch specific kbuild scripts and tools
> > + # - Set correct variable overrides
> > + is_compat = "compat" in d.getVar("PN", True)
> > + is_native = "native" in d.getVar("PN", True)
>
> Just a minor thing: The intended/preferred interface for recipes are
> the
> "class-native" and "class-compat" overrides set in multiarch.bbclass.
> This is nice for bitbake (just set MY_VAR:class-native = "value"),
> but
> unfortunately not so easy to use from Python code. Still, I'd avoid
> looking at PN again in the recipes, and instead go for
>
> is_compat = "class-compat" in d.getVar("OVERRIDES").split(":")
>
> > +
> > + # Determine creation of a kbuild-cross or (standard) kbuild
> > package for
> > the host
> > + is_cross_profile = distro_arch != host_arch and
> > d.getVar("ISAR_CROSS_COMPILE", True) == "1"
> > +
> > + if is_cross_profile:
> > + headers_depends = headers_depends + "-cross | " +
> > headers_depends
> > + build_depends = "crossbuild-essential-" + distro_arch + "
> > [" +
> > host_arch + "], "
> > + d.appendVar("DEB_BUILD_PROFILES", " crosskbuild")
> > + d.prependVar("KBUILD_DEPENDS", build_depends)
> > + d.appendVarFlag("do_prepare_build", "depends", " sbuild-
> > chroot-
> > target:do_build")
> > + d.setVar("ISAR_ENABLE_COMPAT_ARCH", "1")
> > + d.setVar("COMPAT_DISTRO_ARCH", distro_arch)
>
> This feels hacky... it overrides the compat mechanism, resulting in a
> "*-compat" package that is not actually for the compat arch.
Yes, it seems a bit hacky...
I'm unsure if this special kernel specific approach to rebuild kbuild
tools/scripts, forcing non-cross QEMU build even if ISAR cross-build is
enabled, legimitates adding a new similar approach like the
compat/native mechanism within ISAR.
Reusing the "-compat" bitbake target enables the build of
a target specific kbuild package at cross builds avoiding deeper
changes to ISAR. Forcing a QEMU non-cross build even if
ISAR_CROSS_COMPILE is enabled. This is the important part: Running a
separate QEMU non-cross build for kernel scripts/tools. This is needed
because the tools/scripts kernel build does not support creation for
target arch while cross building.
Let me know if you have suggestions to archieve that without using the
compat mechanism and avoiding deeper changes to ISAR. Important is that
a second dpkg build runs separatly within target's schroot (using
forced non-cross QEMU) while main build runs cross in host's schroot.
Earlier patchsets of this series used a way without using the new
compat/native mechanism but made deeper changes to ISAR.
>
> Adriaan
>
> > +
> > + if is_compat:
> > + d.appendVar("DEB_BUILD_PROFILES", " targetbuild")
> > + d.setVar("ISAR_CROSS_COMPILE", "0")
> > + elif is_native:
> > + d.appendVar("DEB_BUILD_PROFILES", " hostbuild")
> > + else:
> > + d.appendVar("DEB_BUILD_PROFILES", " defaultkbuild")
> > +
> > + # Make bitbake know we will be producing
> > + # linux-image and linux-headers packages
> > + if not is_compat and not is_native:
> > + d.setVar("PROVIDES", "linux-image-" + kernel_name + " " +
> > \
> > + "linux-headers-" + kernel_name)
> > +
> > # Set dependency for kernel headers
> > d.appendVar("KERNEL_HEADERS_DEBIAN_DEPENDS", headers_depends)
> > }
> >
> > +inherit dpkg
> > +inherit template
> > +
> > def get_kernel_arch(d):
> > distro_arch = d.getVar("DISTRO_ARCH")
> > if distro_arch in ["amd64", "i386"]:
> > --
> > 2.39.2
>
--
Stefan Koch
Siemens AG
http://www.siemens.com/
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v4 0/3] linux-custom: Split up binaries from kernel headers to kbuild packages
2023-09-05 20:17 ` Jan Kiszka
@ 2023-10-04 15:36 ` Koch, Stefan
0 siblings, 0 replies; 17+ messages in thread
From: Koch, Stefan @ 2023-10-04 15:36 UTC (permalink / raw)
To: isar-users, Kiszka, Jan
Cc: Sudler, Simon, Storm, Christian, Schmidt, Adriaan,
cedric.hombourger, Adler, Michael, MOESSBAUER, FELIX JONATHAN,
ubely
On Tue, 2023-09-05 at 22:17 +0200, Jan Kiszka wrote:
> On 21.08.23 15:44, Stefan Koch wrote:
> > Hi
> >
> > This updated v4 patchset implement suggestions from reviewed v3
> > patchset.
> > It's now a set of five patches:
> > - linux-custom: Split up binaries from kernel headers to kbuild
> > package
> > - linux-custom: Provide host and target specific kernel kbuild
> > packages
> > - docs: Update custom_kernel docs for split up of kernel scripts
> > and tools
> >
> > The main use-case was to swap out the binaries
> > from the kernel headers into kernel kbuild package.
> > This is introduced by the first commit
> > "Split up binaries from kernel headers to kbuild package"
> >
> > The second commit "Provide host and target specific kernel kbuild
> > packages"
> > introduces that the binaries could be swapped out into host and
> > target
> > specific kernel kbuild packages.
> >
> > The main development goals were these:
> >
> > 1. Solve already known isar custom kernel
> > limitations from doc/custom_kernel.inc
> > - kernel headers package does not support both native
> > and cross compilation of kernel modules when cross built
> >
> > 2. Honor recommendations for future from doc/custom_kernel.inc
> > - Generate kernel headers packages for both host and target
> > when using cross build
> >
> > 3. Add extensions known from debian kernel packages structure
> > - Generate a kernel headers package without binaries
> > - Create specific kernel kbuild packages that
> > will ship the "scripts" and "tools" binaries
> > - Use symlinks to point to the "scripts" and "tools" binaries
> >
> > 4. Be user friendly
> > - Avoid redundant configuration of kernel source definitions with
> > user
> > actions to enable kbuild package generation
> > - Use already known way to include linux-custom.inc in just one
> > own bitbake recipe that provides the kernel source definitions
> > - Keep known user behavior for existing build configurations:
> > just update isar, request "-compat" (and "-native") bitbake
> > targets
> > and kbuild packages for target and host will be created
> > automatically
>
> Most things look fairly good, however - maybe it's just too late
> today -
> I'm not yet getting the scenario where -compat plays a role. My
> impression is right now, you are misusing this for some scenario it
> is
> not designed for (compat = 32-bit package for 64-bit target). Or am I
> misinterpreting this?
Yes, compat is used for the target build. See last answer for [PATCH v4
2/3].
>
> Jan
>
--
Stefan Koch
Siemens AG
www.siemens.com
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v4 2/3] linux-custom: Provide host and target specific kernel kbuild packages
2023-10-04 15:34 ` Koch, Stefan
@ 2023-10-04 16:15 ` Jan Kiszka
2023-10-10 9:12 ` Koch, Stefan
0 siblings, 1 reply; 17+ messages in thread
From: Jan Kiszka @ 2023-10-04 16:15 UTC (permalink / raw)
To: Koch, Stefan (DI PA DCP R&D 3),
Schmidt, Adriaan (T CED SES-DE),
isar-users
Cc: Sudler, Simon (DI PA DCP TI), Storm, Christian (T CED SES-DE),
ubely, Adler, Michael (T CED SES-DE),
MOESSBAUER, Felix (T CED INW-CN),
Hombourger, Cedric (DI SW CAS ES LI)
On 04.10.23 17:34, Koch, Stefan (DI PA DCP R&D 3) wrote:
> On Wed, 2023-09-06 at 06:38 +0000, Schmidt, Adriaan (T CED SES-DE)
> wrote:
>> Koch, Stefan (DI PA DCP R&D 3) <stefan-koch@siemens.com>, Monday,
>> August 21, 2023 3:45 PM:
>>> When using a cross build this patch does introduce optionally
>>> host and target specific kernel kbuild packages that
>>> ship the "scripts" and "tools" binaries.
>>>
>>> By default at cross builds a kbuild-cross package will be
>>> generated that provides the host specific kbuild package.
>>>
>>> The "-compat" and "-native" multiarch bitbake targets are useable
>>> to run
>>> additional target or host specific builds for kbuild scripts and
>>> tools.
>>>
>>> Using the "-compat" bitbake target enables the build of
>>> a target specific kbuild package at cross builds.
>>>
>>> Using the "-native" bitbake target enables the build of
>>> a host specific kbuild package at cross builds.
>>>
>>> This solves this from doc/custom_kernel.inc:
>>> - The kernel headers package has not supported both native
>>> and cross compilation of kernel modules when itself was cross
>>> built
>>> - Future roadmap: Generate kernel headers package for both host
>>> and target when using a cross build
>>>
>>> Signed-off-by: Stefan Koch <stefan-koch@siemens.com>
>>> ---
>>> .../linux/files/debian/control.tmpl | 16 +++++-
>>> .../linux/files/debian/isar/build.tmpl | 12 ++++-
>>> .../linux/files/debian/isar/common.tmpl | 8 +++
>>> .../linux/files/debian/isar/install.tmpl | 26 ++++++----
>>> .../linux/files/debian/rules.tmpl | 2 +-
>>> meta/recipes-kernel/linux/linux-custom.inc | 49
>>> ++++++++++++++++---
>>> 6 files changed, 94 insertions(+), 19 deletions(-)
>>>
>>> diff --git a/meta/recipes-kernel/linux/files/debian/control.tmpl
>>> b/meta/recipes-kernel/linux/files/debian/control.tmpl
>>> index b19ca2c9..c012048b 100644
>>> --- a/meta/recipes-kernel/linux/files/debian/control.tmpl
>>> +++ b/meta/recipes-kernel/linux/files/debian/control.tmpl
>>> @@ -6,6 +6,7 @@ Build-Depends: bc, kmod, cpio, ${KBUILD_DEPENDS}
>>> Homepage:
>>> http://www.kernel.org/
>>>
>>> Package: linux-image-${KERNEL_NAME_PROVIDED}
>>> +Build-Profiles: <!targetbuild !hostbuild>
>>> Architecture: any
>>> Depends: ${KERNEL_DEBIAN_DEPENDS}
>>> Description: ${KERNEL_NAME_PROVIDED} Linux kernel, version @KR@
>>> @@ -13,6 +14,7 @@ Description: ${KERNEL_NAME_PROVIDED} Linux
>>> kernel, version
>>> @KR@
>>> files, version: @KR@.
>>>
>>> Package: linux-headers-${KERNEL_NAME_PROVIDED}
>>> +Build-Profiles: <!targetbuild !hostbuild>
>>> Architecture: any
>>> Depends: ${KERNEL_HEADERS_DEBIAN_DEPENDS}, ${perl:Depends},
>>> ${shlib:Depends}
>>> Description: ${KERNEL_NAME_PROVIDED} Linux kernel headers for @KR@
>>> @@ -21,7 +23,7 @@ Description: ${KERNEL_NAME_PROVIDED} Linux kernel
>>> headers
>>> for @KR@
>>> This is useful for people who need to build external modules
>>>
>>> Package: linux-libc-dev
>>> -Build-Profiles: <!nolibcdev>
>>> +Build-Profiles: <!nolibcdev !targetbuild !hostbuild>
>>> Section: devel
>>> Provides: linux-kernel-headers
>>> Architecture: any
>>> @@ -30,6 +32,7 @@ Description: Linux support headers for userspace
>>> development
>>> are used by the installed headers for GNU glibc and other system
>>> libraries.
>>>
>>> Package: linux-image-${KERNEL_NAME_PROVIDED}-dbg
>>> +Build-Profiles: <!targetbuild !hostbuild>
>>> Section: debug
>>> Architecture: any
>>> Description: Linux kernel debugging symbols for @KR@
>>> @@ -37,8 +40,19 @@ Description: Linux kernel debugging symbols for
>>> @KR@
>>> all the necessary debug symbols for the kernel and its modules.
>>>
>>> Package: linux-kbuild-${KERNEL_NAME_PROVIDED}
>>> +Build-Profiles: <targetbuild> <hostbuild> <defaultkbuild>
>>> Architecture: any
>>> Depends: ${perl:Depends}, ${shlib:Depends}
>>> Description: ${KERNEL_NAME_PROVIDED} Linux kbuild scripts and
>>> tools for @KR@
>>> This package provides kernel kbuild scripts and tools for @KR@
>>> This is useful for people who need to build external modules
>>> +
>>> +Package: linux-kbuild-${KERNEL_NAME_PROVIDED}-cross
>>> +Build-Profiles: <!targetbuild !hostbuild crosskbuild>
>>> +Architecture: any
>>> +Depends: ${KERNEL_HEADERS_DEBIAN_DEPENDS}, ${perl:Depends},
>>> ${shlib:Depends}
>>> +Description: ${KERNEL_NAME_PROVIDED} Linux kbuild scripts and
>>> tools for @KR@
>>> + This package provides kernel kbuild scripts and tools
>>> + as ${HOST_ARCH} cross binaries for @KR@
>>> + This is useful for those who need to cross build
>>> + external modules using ISAR's sbuild-chroot-host
>>> diff --git a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl
>>> b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl
>>> index 906dc580..7f293528 100644
>>> --- a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl
>>> +++ b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl
>>> @@ -22,7 +22,17 @@ do_build() {
>>> sed -i "s/@KR@/${KR}/g" ${S}/debian/control ${S}/debian/linux-
>>> image-
>>> ${KERNEL_NAME_PROVIDED}.*
>>>
>>> # Build the Linux kernel
>>> - ${MAKE} O=${KERNEL_BUILD_DIR} ${PARALLEL_MAKE}
>>> KCFLAGS="${KCFLAGS}"
>>> KAFLAGS="${KAFLAGS}"
>>> + if echo "${DEB_BUILD_PROFILES}" | grep -q -e "targetbuild" -e
>>> "hostbuild"; then # Build kernel scripts and tools
>>> + ${MAKE} O=${KERNEL_BUILD_DIR} ${PARALLEL_MAKE}
>>> KCFLAGS="${KCFLAGS}"
>>> KAFLAGS="${KAFLAGS}" scripts
>>> + if grep -q -E
>>> "CONFIG_STACK_VALIDATION=y|CONFIG_HAVE_OBJTOOL=y"
>>> ${KERNEL_BUILD_DIR}/.config && [ -d "tools/objtool" ]; then
>>> + ${MAKE} O=${KERNEL_BUILD_DIR} ${PARALLEL_MAKE}
>>> KCFLAGS="${KCFLAGS}" KAFLAGS="${KAFLAGS}" tools/objtool || true
>>> + fi
>>> + if grep -q "CONFIG_MODULES=y" ${KERNEL_BUILD_DIR}/.config;
>>> then
>>> + ${MAKE} O=${KERNEL_BUILD_DIR} ${PARALLEL_MAKE}
>>> KCFLAGS="${KCFLAGS}" KAFLAGS="${KAFLAGS}" modules_prepare
>>> + fi
>>> + else # Build the Linux kernel
>>> + ${MAKE} O=${KERNEL_BUILD_DIR} ${PARALLEL_MAKE}
>>> KCFLAGS="${KCFLAGS}"
>>> KAFLAGS="${KAFLAGS}"
>>> + fi
>>>
>>> # Stop tracing
>>> set +x
>>> diff --git a/meta/recipes-
>>> kernel/linux/files/debian/isar/common.tmpl
>>> b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl
>>> index f4c0519f..36e01691 100644
>>> --- a/meta/recipes-kernel/linux/files/debian/isar/common.tmpl
>>> +++ b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl
>>> @@ -11,6 +11,14 @@ KERNEL_PKG_KERN_HEADERS=linux-headers-
>>> ${KERNEL_NAME_PROVIDED}
>>> KERNEL_PKG_LIBC_HEADERS=linux-libc-dev
>>> KERNEL_PKG_KERN_KBUILD=linux-kbuild-${KERNEL_NAME_PROVIDED}
>>>
>>> +if echo "${DEB_BUILD_PROFILES}" | grep -q "hostbuild"; then
>>> + # Force creating kernel kbuild debian package with valid
>>> host arch
>>> + # Use a cross build to comply with arch specific kernel
>>> defconfigs
>>> + # The scripts and tools are always created for host arch
>>> + eval $(dpkg-architecture -f -A ${DISTRO_ARCH})
>>> + CROSS_COMPILE=${DEB_TARGET_GNU_TYPE}-
>>> +fi
>>> +
>>> # Constants
>>> KCONF=.config
>>>
>>> diff --git a/meta/recipes-
>>> kernel/linux/files/debian/isar/install.tmpl
>>> b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl
>>> index 236b67c4..08b8a9a5 100644
>>> --- a/meta/recipes-kernel/linux/files/debian/isar/install.tmpl
>>> +++ b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl
>>> @@ -34,18 +34,24 @@ do_install() {
>>> set -x
>>>
>>> # Run the install steps
>>> - install_image
>>> - if [ "${ARCH}" != "um" ]; then
>>> - install_config
>>> - install_map
>>> + if ! echo "${DEB_BUILD_PROFILES}" | grep -q -e "targetbuild" -
>>> e
>>> "hostbuild"; then
>>> + install_image
>>> + if [ "${ARCH}" != "um" ]; then
>>> + install_config
>>> + install_map
>>> + fi
>>> + install_hooks
>>> + install_dtbs
>>> + install_kmods
>>> + install_headers
>>> fi
>>> - install_hooks
>>> - install_dtbs
>>> - install_kmods
>>> - install_headers
>>>
>>> - # Cleanup and install kernel scripts and tools
>>> - rm -rf ${deb_kern_kbuild_dir}
>>> + if ! echo "${DEB_BUILD_PROFILES}" | grep -q -e "targetbuild" -
>>> e
>>> "hostbuild" && echo "${DEB_BUILD_PROFILES}" | grep -q
>>> "crosskbuild"; then
>>> + # Install cross kernel scripts and tools
>>> + install_kbuild ${deb_kern_kbuild_dir}-cross
>>> + fi
>>> +
>>> + # Install kernel scripts and tools
>>> install_kbuild ${deb_kern_kbuild_dir}
>>>
>>> # Stop tracing
>>> diff --git a/meta/recipes-kernel/linux/files/debian/rules.tmpl
>>> b/meta/recipes-kernel/linux/files/debian/rules.tmpl
>>> index e8ae3daa..1f838a41 100755
>>> --- a/meta/recipes-kernel/linux/files/debian/rules.tmpl
>>> +++ b/meta/recipes-kernel/linux/files/debian/rules.tmpl
>>> @@ -48,4 +48,4 @@ override_dh_strip_nondeterminism:
>>> true
>>>
>>> override_dh_strip:
>>> - unset DEB_HOST_GNU_TYPE && dh_strip -Xvmlinu --no-
>>> automatic-dbgsym
>>> + dh_strip -Xvmlinu -Xlinux-kbuild-${KERNEL_NAME_PROVIDED}-
>>> cross --no-
>>> automatic-dbgsym
>>> diff --git a/meta/recipes-kernel/linux/linux-custom.inc
>>> b/meta/recipes-
>>> kernel/linux/linux-custom.inc
>>> index 94ed84a2..85faa2d7 100644
>>> --- a/meta/recipes-kernel/linux/linux-custom.inc
>>> +++ b/meta/recipes-kernel/linux/linux-custom.inc
>>> @@ -85,9 +85,6 @@ TEMPLATE_VARS += " \
>>> DISTRIBUTOR \
>>> "
>>>
>>> -inherit dpkg
>>> -inherit template
>>> -
>>> # Add custom cflags to the kernel build
>>> KCFLAGS ?= "-fdebug-prefix-map=${CURDIR}=."
>>> KAFLAGS ?= "-fdebug-prefix-map=${CURDIR}=."
>>> @@ -95,17 +92,57 @@ KAFLAGS ?= "-fdebug-prefix-map=${CURDIR}=."
>>> # Derive name of the kernel packages from the name of this recipe
>>> KERNEL_NAME_PROVIDED ?= "${@ d.getVar('PN').partition('linux-
>>> ')[2]}"
>>>
>>> -# Make bitbake know we will be producing linux-image and linux-
>>> headers
>>> packages
>>> python() {
>>> kernel_name = d.getVar("KERNEL_NAME_PROVIDED")
>>> - d.setVar('PROVIDES', 'linux-image-' + kernel_name + ' ' + \
>>> - 'linux-headers-' + kernel_name)
>>> + distro_arch = d.getVar("DISTRO_ARCH")
>>> + host_arch = d.getVar("HOST_ARCH")
>>> headers_depends = "linux-kbuild-" + kernel_name
>>>
>>> + # The "-compat" and "-native" multiarch bitbake targets are
>>> used to run
>>> + # additional target or host specific builds for kbuild scripts
>>> and
>>> tools.
>>> + # The "-compat" bitbake target requests a "target" build
>>> + # and the "-native" bitbake target requests a "host" build.
>>> + #
>>> + # For different distro and host archs
>>> + # - Add dependency for sbuild-chroot-target
>>> + # to allow building arch specific kbuild scripts and tools
>>> + # - Set correct variable overrides
>>> + is_compat = "compat" in d.getVar("PN", True)
>>> + is_native = "native" in d.getVar("PN", True)
>>
>> Just a minor thing: The intended/preferred interface for recipes are
>> the
>> "class-native" and "class-compat" overrides set in multiarch.bbclass.
>> This is nice for bitbake (just set MY_VAR:class-native = "value"),
>> but
>> unfortunately not so easy to use from Python code. Still, I'd avoid
>> looking at PN again in the recipes, and instead go for
>>
>> is_compat = "class-compat" in d.getVar("OVERRIDES").split(":")
>>
>>> +
>>> + # Determine creation of a kbuild-cross or (standard) kbuild
>>> package for
>>> the host
>>> + is_cross_profile = distro_arch != host_arch and
>>> d.getVar("ISAR_CROSS_COMPILE", True) == "1"
>>> +
>>> + if is_cross_profile:
>>> + headers_depends = headers_depends + "-cross | " +
>>> headers_depends
>>> + build_depends = "crossbuild-essential-" + distro_arch + "
>>> [" +
>>> host_arch + "], "
>>> + d.appendVar("DEB_BUILD_PROFILES", " crosskbuild")
>>> + d.prependVar("KBUILD_DEPENDS", build_depends)
>>> + d.appendVarFlag("do_prepare_build", "depends", " sbuild-
>>> chroot-
>>> target:do_build")
>>> + d.setVar("ISAR_ENABLE_COMPAT_ARCH", "1")
>>> + d.setVar("COMPAT_DISTRO_ARCH", distro_arch)
>>
>> This feels hacky... it overrides the compat mechanism, resulting in a
>> "*-compat" package that is not actually for the compat arch.
>
> Yes, it seems a bit hacky...
>
> I'm unsure if this special kernel specific approach to rebuild kbuild
> tools/scripts, forcing non-cross QEMU build even if ISAR cross-build is
> enabled, legimitates adding a new similar approach like the
> compat/native mechanism within ISAR.
>
> Reusing the "-compat" bitbake target enables the build of
> a target specific kbuild package at cross builds avoiding deeper
> changes to ISAR. Forcing a QEMU non-cross build even if
> ISAR_CROSS_COMPILE is enabled. This is the important part: Running a
> separate QEMU non-cross build for kernel scripts/tools. This is needed
> because the tools/scripts kernel build does not support creation for
> target arch while cross building.
This is the property we should change once we have the means to build
for the native arch as well.
>
> Let me know if you have suggestions to archieve that without using the
> compat mechanism and avoiding deeper changes to ISAR. Important is that
> a second dpkg build runs separatly within target's schroot (using
> forced non-cross QEMU) while main build runs cross in host's schroot.
> Earlier patchsets of this series used a way without using the new
> compat/native mechanism but made deeper changes to ISAR.
We will need changes in Isar and possibly also in some downstream
recipes, though I think can be caught by linux-custom-module.inc and its
helpers. This should not be your blocker.
So far: we build tools for the host, even when building the kernel for
the target.
What we need: we build tools for the target, and if some module recipe
wants to have them for cross-building as well, it asks for -native.
Does that make sense?
Jan
--
Siemens AG, Technology
Linux Expert Center
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v4 2/3] linux-custom: Provide host and target specific kernel kbuild packages
2023-10-04 16:15 ` Jan Kiszka
@ 2023-10-10 9:12 ` Koch, Stefan
0 siblings, 0 replies; 17+ messages in thread
From: Koch, Stefan @ 2023-10-10 9:12 UTC (permalink / raw)
To: Schmidt, Adriaan, isar-users, Kiszka, Jan
Cc: Sudler, Simon, Storm, Christian, ubely, cedric.hombourger, Adler,
Michael, MOESSBAUER, FELIX JONATHAN
On Wed, 2023-10-04 at 18:15 +0200, Jan Kiszka wrote:
> On 04.10.23 17:34, Koch, Stefan (DI PA DCP R&D 3) wrote:
> > On Wed, 2023-09-06 at 06:38 +0000, Schmidt, Adriaan (T CED SES-DE)
> > wrote:
> > > Koch, Stefan (DI PA DCP R&D 3) <stefan-koch@siemens.com>, Monday,
> > > August 21, 2023 3:45 PM:
> > > > When using a cross build this patch does introduce optionally
> > > > host and target specific kernel kbuild packages that
> > > > ship the "scripts" and "tools" binaries.
> > > >
> > > > By default at cross builds a kbuild-cross package will be
> > > > generated that provides the host specific kbuild package.
> > > >
> > > > The "-compat" and "-native" multiarch bitbake targets are
> > > > useable
> > > > to run
> > > > additional target or host specific builds for kbuild scripts
> > > > and
> > > > tools.
> > > >
> > > > Using the "-compat" bitbake target enables the build of
> > > > a target specific kbuild package at cross builds.
> > > >
> > > > Using the "-native" bitbake target enables the build of
> > > > a host specific kbuild package at cross builds.
> > > >
> > > > This solves this from doc/custom_kernel.inc:
> > > > - The kernel headers package has not supported both native
> > > > and cross compilation of kernel modules when itself was cross
> > > > built
> > > > - Future roadmap: Generate kernel headers package for both host
> > > > and target when using a cross build
> > > >
> > > > Signed-off-by: Stefan Koch <stefan-koch@siemens.com>
> > > > ---
> > > > .../linux/files/debian/control.tmpl | 16 +++++-
> > > > .../linux/files/debian/isar/build.tmpl | 12 ++++-
> > > > .../linux/files/debian/isar/common.tmpl | 8 +++
> > > > .../linux/files/debian/isar/install.tmpl | 26 ++++++----
> > > > .../linux/files/debian/rules.tmpl | 2 +-
> > > > meta/recipes-kernel/linux/linux-custom.inc | 49
> > > > ++++++++++++++++---
> > > > 6 files changed, 94 insertions(+), 19 deletions(-)
> > > >
> > > > diff --git a/meta/recipes-
> > > > kernel/linux/files/debian/control.tmpl
> > > > b/meta/recipes-kernel/linux/files/debian/control.tmpl
> > > > index b19ca2c9..c012048b 100644
> > > > --- a/meta/recipes-kernel/linux/files/debian/control.tmpl
> > > > +++ b/meta/recipes-kernel/linux/files/debian/control.tmpl
> > > > @@ -6,6 +6,7 @@ Build-Depends: bc, kmod, cpio,
> > > > ${KBUILD_DEPENDS}
> > > > Homepage:
> > > > http://www.kernel.org/
> > > >
> > > > Package: linux-image-${KERNEL_NAME_PROVIDED}
> > > > +Build-Profiles: <!targetbuild !hostbuild>
> > > > Architecture: any
> > > > Depends: ${KERNEL_DEBIAN_DEPENDS}
> > > > Description: ${KERNEL_NAME_PROVIDED} Linux kernel, version
> > > > @KR@
> > > > @@ -13,6 +14,7 @@ Description: ${KERNEL_NAME_PROVIDED} Linux
> > > > kernel, version
> > > > @KR@
> > > > files, version: @KR@.
> > > >
> > > > Package: linux-headers-${KERNEL_NAME_PROVIDED}
> > > > +Build-Profiles: <!targetbuild !hostbuild>
> > > > Architecture: any
> > > > Depends: ${KERNEL_HEADERS_DEBIAN_DEPENDS}, ${perl:Depends},
> > > > ${shlib:Depends}
> > > > Description: ${KERNEL_NAME_PROVIDED} Linux kernel headers for
> > > > @KR@
> > > > @@ -21,7 +23,7 @@ Description: ${KERNEL_NAME_PROVIDED} Linux
> > > > kernel
> > > > headers
> > > > for @KR@
> > > > This is useful for people who need to build external modules
> > > >
> > > > Package: linux-libc-dev
> > > > -Build-Profiles: <!nolibcdev>
> > > > +Build-Profiles: <!nolibcdev !targetbuild !hostbuild>
> > > > Section: devel
> > > > Provides: linux-kernel-headers
> > > > Architecture: any
> > > > @@ -30,6 +32,7 @@ Description: Linux support headers for
> > > > userspace
> > > > development
> > > > are used by the installed headers for GNU glibc and other
> > > > system
> > > > libraries.
> > > >
> > > > Package: linux-image-${KERNEL_NAME_PROVIDED}-dbg
> > > > +Build-Profiles: <!targetbuild !hostbuild>
> > > > Section: debug
> > > > Architecture: any
> > > > Description: Linux kernel debugging symbols for @KR@
> > > > @@ -37,8 +40,19 @@ Description: Linux kernel debugging symbols
> > > > for
> > > > @KR@
> > > > all the necessary debug symbols for the kernel and its
> > > > modules.
> > > >
> > > > Package: linux-kbuild-${KERNEL_NAME_PROVIDED}
> > > > +Build-Profiles: <targetbuild> <hostbuild> <defaultkbuild>
> > > > Architecture: any
> > > > Depends: ${perl:Depends}, ${shlib:Depends}
> > > > Description: ${KERNEL_NAME_PROVIDED} Linux kbuild scripts and
> > > > tools for @KR@
> > > > This package provides kernel kbuild scripts and tools for
> > > > @KR@
> > > > This is useful for people who need to build external modules
> > > > +
> > > > +Package: linux-kbuild-${KERNEL_NAME_PROVIDED}-cross
> > > > +Build-Profiles: <!targetbuild !hostbuild crosskbuild>
> > > > +Architecture: any
> > > > +Depends: ${KERNEL_HEADERS_DEBIAN_DEPENDS}, ${perl:Depends},
> > > > ${shlib:Depends}
> > > > +Description: ${KERNEL_NAME_PROVIDED} Linux kbuild scripts and
> > > > tools for @KR@
> > > > + This package provides kernel kbuild scripts and tools
> > > > + as ${HOST_ARCH} cross binaries for @KR@
> > > > + This is useful for those who need to cross build
> > > > + external modules using ISAR's sbuild-chroot-host
> > > > diff --git a/meta/recipes-
> > > > kernel/linux/files/debian/isar/build.tmpl
> > > > b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl
> > > > index 906dc580..7f293528 100644
> > > > --- a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl
> > > > +++ b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl
> > > > @@ -22,7 +22,17 @@ do_build() {
> > > > sed -i "s/@KR@/${KR}/g" ${S}/debian/control
> > > > ${S}/debian/linux-
> > > > image-
> > > > ${KERNEL_NAME_PROVIDED}.*
> > > >
> > > > # Build the Linux kernel
> > > > - ${MAKE} O=${KERNEL_BUILD_DIR} ${PARALLEL_MAKE}
> > > > KCFLAGS="${KCFLAGS}"
> > > > KAFLAGS="${KAFLAGS}"
> > > > + if echo "${DEB_BUILD_PROFILES}" | grep -q -e "targetbuild"
> > > > -e
> > > > "hostbuild"; then # Build kernel scripts and tools
> > > > + ${MAKE} O=${KERNEL_BUILD_DIR} ${PARALLEL_MAKE}
> > > > KCFLAGS="${KCFLAGS}"
> > > > KAFLAGS="${KAFLAGS}" scripts
> > > > + if grep -q -E
> > > > "CONFIG_STACK_VALIDATION=y|CONFIG_HAVE_OBJTOOL=y"
> > > > ${KERNEL_BUILD_DIR}/.config && [ -d "tools/objtool" ]; then
> > > > + ${MAKE} O=${KERNEL_BUILD_DIR} ${PARALLEL_MAKE}
> > > > KCFLAGS="${KCFLAGS}" KAFLAGS="${KAFLAGS}" tools/objtool || true
> > > > + fi
> > > > + if grep -q "CONFIG_MODULES=y"
> > > > ${KERNEL_BUILD_DIR}/.config;
> > > > then
> > > > + ${MAKE} O=${KERNEL_BUILD_DIR} ${PARALLEL_MAKE}
> > > > KCFLAGS="${KCFLAGS}" KAFLAGS="${KAFLAGS}" modules_prepare
> > > > + fi
> > > > + else # Build the Linux kernel
> > > > + ${MAKE} O=${KERNEL_BUILD_DIR} ${PARALLEL_MAKE}
> > > > KCFLAGS="${KCFLAGS}"
> > > > KAFLAGS="${KAFLAGS}"
> > > > + fi
> > > >
> > > > # Stop tracing
> > > > set +x
> > > > diff --git a/meta/recipes-
> > > > kernel/linux/files/debian/isar/common.tmpl
> > > > b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl
> > > > index f4c0519f..36e01691 100644
> > > > --- a/meta/recipes-kernel/linux/files/debian/isar/common.tmpl
> > > > +++ b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl
> > > > @@ -11,6 +11,14 @@ KERNEL_PKG_KERN_HEADERS=linux-headers-
> > > > ${KERNEL_NAME_PROVIDED}
> > > > KERNEL_PKG_LIBC_HEADERS=linux-libc-dev
> > > > KERNEL_PKG_KERN_KBUILD=linux-kbuild-${KERNEL_NAME_PROVIDED}
> > > >
> > > > +if echo "${DEB_BUILD_PROFILES}" | grep -q "hostbuild"; then
> > > > + # Force creating kernel kbuild debian package with
> > > > valid
> > > > host arch
> > > > + # Use a cross build to comply with arch specific kernel
> > > > defconfigs
> > > > + # The scripts and tools are always created for host
> > > > arch
> > > > + eval $(dpkg-architecture -f -A ${DISTRO_ARCH})
> > > > + CROSS_COMPILE=${DEB_TARGET_GNU_TYPE}-
> > > > +fi
> > > > +
> > > > # Constants
> > > > KCONF=.config
> > > >
> > > > diff --git a/meta/recipes-
> > > > kernel/linux/files/debian/isar/install.tmpl
> > > > b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl
> > > > index 236b67c4..08b8a9a5 100644
> > > > --- a/meta/recipes-kernel/linux/files/debian/isar/install.tmpl
> > > > +++ b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl
> > > > @@ -34,18 +34,24 @@ do_install() {
> > > > set -x
> > > >
> > > > # Run the install steps
> > > > - install_image
> > > > - if [ "${ARCH}" != "um" ]; then
> > > > - install_config
> > > > - install_map
> > > > + if ! echo "${DEB_BUILD_PROFILES}" | grep -q -e
> > > > "targetbuild" -
> > > > e
> > > > "hostbuild"; then
> > > > + install_image
> > > > + if [ "${ARCH}" != "um" ]; then
> > > > + install_config
> > > > + install_map
> > > > + fi
> > > > + install_hooks
> > > > + install_dtbs
> > > > + install_kmods
> > > > + install_headers
> > > > fi
> > > > - install_hooks
> > > > - install_dtbs
> > > > - install_kmods
> > > > - install_headers
> > > >
> > > > - # Cleanup and install kernel scripts and tools
> > > > - rm -rf ${deb_kern_kbuild_dir}
> > > > + if ! echo "${DEB_BUILD_PROFILES}" | grep -q -e
> > > > "targetbuild" -
> > > > e
> > > > "hostbuild" && echo "${DEB_BUILD_PROFILES}" | grep -q
> > > > "crosskbuild"; then
> > > > + # Install cross kernel scripts and tools
> > > > + install_kbuild ${deb_kern_kbuild_dir}-cross
> > > > + fi
> > > > +
> > > > + # Install kernel scripts and tools
> > > > install_kbuild ${deb_kern_kbuild_dir}
> > > >
> > > > # Stop tracing
> > > > diff --git a/meta/recipes-kernel/linux/files/debian/rules.tmpl
> > > > b/meta/recipes-kernel/linux/files/debian/rules.tmpl
> > > > index e8ae3daa..1f838a41 100755
> > > > --- a/meta/recipes-kernel/linux/files/debian/rules.tmpl
> > > > +++ b/meta/recipes-kernel/linux/files/debian/rules.tmpl
> > > > @@ -48,4 +48,4 @@ override_dh_strip_nondeterminism:
> > > > true
> > > >
> > > > override_dh_strip:
> > > > - unset DEB_HOST_GNU_TYPE && dh_strip -Xvmlinu --no-
> > > > automatic-dbgsym
> > > > + dh_strip -Xvmlinu -Xlinux-kbuild-
> > > > ${KERNEL_NAME_PROVIDED}-
> > > > cross --no-
> > > > automatic-dbgsym
> > > > diff --git a/meta/recipes-kernel/linux/linux-custom.inc
> > > > b/meta/recipes-
> > > > kernel/linux/linux-custom.inc
> > > > index 94ed84a2..85faa2d7 100644
> > > > --- a/meta/recipes-kernel/linux/linux-custom.inc
> > > > +++ b/meta/recipes-kernel/linux/linux-custom.inc
> > > > @@ -85,9 +85,6 @@ TEMPLATE_VARS += " \
> > > > DISTRIBUTOR \
> > > > "
> > > >
> > > > -inherit dpkg
> > > > -inherit template
> > > > -
> > > > # Add custom cflags to the kernel build
> > > > KCFLAGS ?= "-fdebug-prefix-map=${CURDIR}=."
> > > > KAFLAGS ?= "-fdebug-prefix-map=${CURDIR}=."
> > > > @@ -95,17 +92,57 @@ KAFLAGS ?= "-fdebug-prefix-map=${CURDIR}=."
> > > > # Derive name of the kernel packages from the name of this
> > > > recipe
> > > > KERNEL_NAME_PROVIDED ?= "${@ d.getVar('PN').partition('linux-
> > > > ')[2]}"
> > > >
> > > > -# Make bitbake know we will be producing linux-image and
> > > > linux-
> > > > headers
> > > > packages
> > > > python() {
> > > > kernel_name = d.getVar("KERNEL_NAME_PROVIDED")
> > > > - d.setVar('PROVIDES', 'linux-image-' + kernel_name + ' ' +
> > > > \
> > > > - 'linux-headers-' + kernel_name)
> > > > + distro_arch = d.getVar("DISTRO_ARCH")
> > > > + host_arch = d.getVar("HOST_ARCH")
> > > > headers_depends = "linux-kbuild-" + kernel_name
> > > >
> > > > + # The "-compat" and "-native" multiarch bitbake targets
> > > > are
> > > > used to run
> > > > + # additional target or host specific builds for kbuild
> > > > scripts
> > > > and
> > > > tools.
> > > > + # The "-compat" bitbake target requests a "target" build
> > > > + # and the "-native" bitbake target requests a "host"
> > > > build.
> > > > + #
> > > > + # For different distro and host archs
> > > > + # - Add dependency for sbuild-chroot-target
> > > > + # to allow building arch specific kbuild scripts and
> > > > tools
> > > > + # - Set correct variable overrides
> > > > + is_compat = "compat" in d.getVar("PN", True)
> > > > + is_native = "native" in d.getVar("PN", True)
> > >
> > > Just a minor thing: The intended/preferred interface for recipes
> > > are
> > > the
> > > "class-native" and "class-compat" overrides set in
> > > multiarch.bbclass.
> > > This is nice for bitbake (just set MY_VAR:class-native =
> > > "value"),
> > > but
> > > unfortunately not so easy to use from Python code. Still, I'd
> > > avoid
> > > looking at PN again in the recipes, and instead go for
> > >
> > > is_compat = "class-compat" in d.getVar("OVERRIDES").split(":")
> > >
> > > > +
> > > > + # Determine creation of a kbuild-cross or (standard)
> > > > kbuild
> > > > package for
> > > > the host
> > > > + is_cross_profile = distro_arch != host_arch and
> > > > d.getVar("ISAR_CROSS_COMPILE", True) == "1"
> > > > +
> > > > + if is_cross_profile:
> > > > + headers_depends = headers_depends + "-cross | " +
> > > > headers_depends
> > > > + build_depends = "crossbuild-essential-" + distro_arch
> > > > + "
> > > > [" +
> > > > host_arch + "], "
> > > > + d.appendVar("DEB_BUILD_PROFILES", " crosskbuild")
> > > > + d.prependVar("KBUILD_DEPENDS", build_depends)
> > > > + d.appendVarFlag("do_prepare_build", "depends", "
> > > > sbuild-
> > > > chroot-
> > > > target:do_build")
> > > > + d.setVar("ISAR_ENABLE_COMPAT_ARCH", "1")
> > > > + d.setVar("COMPAT_DISTRO_ARCH", distro_arch)
> > >
> > > This feels hacky... it overrides the compat mechanism, resulting
> > > in a
> > > "*-compat" package that is not actually for the compat arch.
> >
> > Yes, it seems a bit hacky...
> >
> > I'm unsure if this special kernel specific approach to rebuild
> > kbuild
> > tools/scripts, forcing non-cross QEMU build even if ISAR cross-
> > build is
> > enabled, legimitates adding a new similar approach like the
> > compat/native mechanism within ISAR.
> >
> > Reusing the "-compat" bitbake target enables the build of
> > a target specific kbuild package at cross builds avoiding deeper
> > changes to ISAR. Forcing a QEMU non-cross build even if
> > ISAR_CROSS_COMPILE is enabled. This is the important part: Running
> > a
> > separate QEMU non-cross build for kernel scripts/tools. This is
> > needed
> > because the tools/scripts kernel build does not support creation
> > for
> > target arch while cross building.
>
> This is the property we should change once we have the means to build
> for the native arch as well.
>
> >
> > Let me know if you have suggestions to archieve that without using
> > the
> > compat mechanism and avoiding deeper changes to ISAR. Important is
> > that
> > a second dpkg build runs separatly within target's schroot (using
> > forced non-cross QEMU) while main build runs cross in host's
> > schroot.
> > Earlier patchsets of this series used a way without using the new
> > compat/native mechanism but made deeper changes to ISAR.
>
> We will need changes in Isar and possibly also in some downstream
> recipes, though I think can be caught by linux-custom-module.inc and
> its
> helpers. This should not be your blocker.
In the past linux-module.inc was not sufficient for building the kbuild
tools. linux-custom.inc was the right place.
What changes to the downstream modules are needed for that purpose?
>
> So far: we build tools for the host, even when building the kernel
> for
> the target.
>
> What we need: we build tools for the target, and if some module
> recipe
> wants to have them for cross-building as well, it asks for -native.
The following example uses arm64 as target and amd64 as host.
a) Yes with the patchset, linux-custom-native generates them for the
cross-build host:
- linux-custom-kbuild_amd64.deb that contains binary amd64 tools.
b) But anyway they are build within the default package (non-compat and
non-native) at cross-build, too. In the default case the host packages
are inside the target package with architecture mismatch:
- linux-custom-kbuild-cross_arm64.deb that contains binary amd64 tools.
Without the patchset
- linux-custom-headers_arm64.deb contains binary amd64 tools.
c) And linux-custom-compat:
linux-custom-kbuild_arm64.deb that contains binary arm64 tools.
>
> Does that make sense?
[PATCH v4 1/3] linux-custom: Split up binaries from kernel headers to
kbuild package
This patch could already merged. Since it keeps the old native/compat
behaviour. And moves the tools/scripts from kernel-headers in a
separated package.
>
> Jan
>
--
Stefan Koch
Siemens AG
http://www.siemens.com/
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v4 0/3] linux-custom: Split up binaries from kernel headers to kbuild packages
2023-08-21 13:44 [PATCH v4 0/3] linux-custom: Split up binaries from kernel headers to kbuild packages Stefan Koch
` (4 preceding siblings ...)
2023-09-05 20:17 ` Jan Kiszka
@ 2023-11-21 7:26 ` Uladzimir Bely
2023-12-17 16:50 ` Jan Kiszka
6 siblings, 0 replies; 17+ messages in thread
From: Uladzimir Bely @ 2023-11-21 7:26 UTC (permalink / raw)
To: Stefan Koch, isar-users
Cc: jan.kiszka, christian.storm, michael.adler, simon.sudler,
cedric.hombourger, adriaan.schmidt, felix.moessbauer
On Mon, 2023-08-21 at 15:44 +0200, Stefan Koch wrote:
> Hi
>
> This updated v4 patchset implement suggestions from reviewed v3
> patchset.
> It's now a set of five patches:
> - linux-custom: Split up binaries from kernel headers to kbuild
> package
> - linux-custom: Provide host and target specific kernel kbuild
> packages
> - docs: Update custom_kernel docs for split up of kernel scripts and
> tools
>
> The main use-case was to swap out the binaries
> from the kernel headers into kernel kbuild package.
> This is introduced by the first commit
> "Split up binaries from kernel headers to kbuild package"
>
> The second commit "Provide host and target specific kernel kbuild
> packages"
> introduces that the binaries could be swapped out into host and
> target
> specific kernel kbuild packages.
>
> The main development goals were these:
>
> 1. Solve already known isar custom kernel
> limitations from doc/custom_kernel.inc
> - kernel headers package does not support both native
> and cross compilation of kernel modules when cross built
>
> 2. Honor recommendations for future from doc/custom_kernel.inc
> - Generate kernel headers packages for both host and target
> when using cross build
>
> 3. Add extensions known from debian kernel packages structure
> - Generate a kernel headers package without binaries
> - Create specific kernel kbuild packages that
> will ship the "scripts" and "tools" binaries
> - Use symlinks to point to the "scripts" and "tools" binaries
>
> 4. Be user friendly
> - Avoid redundant configuration of kernel source definitions with
> user
> actions to enable kbuild package generation
> - Use already known way to include linux-custom.inc in just one
> own bitbake recipe that provides the kernel source definitions
> - Keep known user behavior for existing build configurations:
> just update isar, request "-compat" (and "-native") bitbake targets
> and kbuild packages for target and host will be created
> automatically
>
> Best regards
>
> Stefan
>
> Stefan Koch (3):
> linux-custom: Split up binaries from kernel headers to kbuild
> package
> linux-custom: Provide host and target specific kernel kbuild
> packages
> docs: Update custom_kernel docs for split up of kernel scripts and
> tools
>
> doc/custom_kernel.md | 16 ++--
> .../linux/files/debian/control.tmpl | 25 ++++++-
> .../linux/files/debian/isar/build.tmpl | 12 ++-
> .../linux/files/debian/isar/common.tmpl | 10 +++
> .../linux/files/debian/isar/install.tmpl | 73 ++++++++++++++---
> --
> .../linux/files/debian/rules.tmpl | 2 +-
> meta/recipes-kernel/linux/linux-custom.inc | 53 ++++++++++++--
> 7 files changed, 155 insertions(+), 36 deletions(-)
>
Hello.
Patch 2 brough up a discussion, but patch 1 itself looks OK.
According to internal discussion, we could apply patch 1 before "v0.10"
released (with documentation changes from patch 3), if it's not too
risky for being in new release without response from multiple
downstreams...
Anyway, since it's not applicable on top of current 'next', it needs
rebasing.
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v4 0/3] linux-custom: Split up binaries from kernel headers to kbuild packages
2023-08-21 13:44 [PATCH v4 0/3] linux-custom: Split up binaries from kernel headers to kbuild packages Stefan Koch
` (5 preceding siblings ...)
2023-11-21 7:26 ` Uladzimir Bely
@ 2023-12-17 16:50 ` Jan Kiszka
2024-01-08 10:05 ` Koch, Stefan
2024-01-08 12:18 ` Koch, Stefan
6 siblings, 2 replies; 17+ messages in thread
From: Jan Kiszka @ 2023-12-17 16:50 UTC (permalink / raw)
To: Stefan Koch, isar-users, adriaan.schmidt
Cc: christian.storm, michael.adler, simon.sudler, cedric.hombourger,
felix.moessbauer, ubely
On 21.08.23 15:44, Stefan Koch wrote:
> Hi
>
> This updated v4 patchset implement suggestions from reviewed v3 patchset.
> It's now a set of five patches:
> - linux-custom: Split up binaries from kernel headers to kbuild package
> - linux-custom: Provide host and target specific kernel kbuild packages
> - docs: Update custom_kernel docs for split up of kernel scripts and tools
>
> The main use-case was to swap out the binaries
> from the kernel headers into kernel kbuild package.
> This is introduced by the first commit
> "Split up binaries from kernel headers to kbuild package"
>
> The second commit "Provide host and target specific kernel kbuild packages"
> introduces that the binaries could be swapped out into host and target
> specific kernel kbuild packages.
>
> The main development goals were these:
>
> 1. Solve already known isar custom kernel
> limitations from doc/custom_kernel.inc
> - kernel headers package does not support both native
> and cross compilation of kernel modules when cross built
>
> 2. Honor recommendations for future from doc/custom_kernel.inc
> - Generate kernel headers packages for both host and target
> when using cross build
>
> 3. Add extensions known from debian kernel packages structure
> - Generate a kernel headers package without binaries
> - Create specific kernel kbuild packages that
> will ship the "scripts" and "tools" binaries
> - Use symlinks to point to the "scripts" and "tools" binaries
>
> 4. Be user friendly
> - Avoid redundant configuration of kernel source definitions with user
> actions to enable kbuild package generation
> - Use already known way to include linux-custom.inc in just one
> own bitbake recipe that provides the kernel source definitions
> - Keep known user behavior for existing build configurations:
> just update isar, request "-compat" (and "-native") bitbake targets
> and kbuild packages for target and host will be created automatically
>
> Best regards
>
> Stefan
>
> Stefan Koch (3):
> linux-custom: Split up binaries from kernel headers to kbuild package
> linux-custom: Provide host and target specific kernel kbuild packages
> docs: Update custom_kernel docs for split up of kernel scripts and
> tools
>
> doc/custom_kernel.md | 16 ++--
> .../linux/files/debian/control.tmpl | 25 ++++++-
> .../linux/files/debian/isar/build.tmpl | 12 ++-
> .../linux/files/debian/isar/common.tmpl | 10 +++
> .../linux/files/debian/isar/install.tmpl | 73 ++++++++++++++-----
> .../linux/files/debian/rules.tmpl | 2 +-
> meta/recipes-kernel/linux/linux-custom.inc | 53 ++++++++++++--
> 7 files changed, 155 insertions(+), 36 deletions(-)
>
I'd like to complete this rather soon now because it should allow us to
build and actually use a custom linux-libc-dev - currently only the one
of the target architecture is built, and that can't be installed into
sbuild-chroot because it conflicts with linux-libc-dev:native.
What were the plans to revive it? Patch 1 was ready but patch 2 needed
to stay away from misusing -compat.
Jan
--
Siemens AG, Technology
Linux Expert Center
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v4 0/3] linux-custom: Split up binaries from kernel headers to kbuild packages
2023-12-17 16:50 ` Jan Kiszka
@ 2024-01-08 10:05 ` Koch, Stefan
2024-01-08 12:18 ` Koch, Stefan
1 sibling, 0 replies; 17+ messages in thread
From: Koch, Stefan @ 2024-01-08 10:05 UTC (permalink / raw)
To: Schmidt, Adriaan, isar-users, Kiszka, Jan
Cc: Sudler, Simon, Storm, Christian, ubely, cedric.hombourger, Adler,
Michael, MOESSBAUER, Felix
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v4 0/3] linux-custom: Split up binaries from kernel headers to kbuild packages
2023-12-17 16:50 ` Jan Kiszka
2024-01-08 10:05 ` Koch, Stefan
@ 2024-01-08 12:18 ` Koch, Stefan
2024-01-08 12:20 ` Jan Kiszka
1 sibling, 1 reply; 17+ messages in thread
From: Koch, Stefan @ 2024-01-08 12:18 UTC (permalink / raw)
To: Schmidt, Adriaan, isar-users, Kiszka, Jan
Cc: Sudler, Simon, Storm, Christian, ubely, cedric.hombourger, Adler,
Michael, MOESSBAUER, Felix
On Sun, 2023-12-17 at 17:50 +0100, Jan Kiszka wrote:
> On 21.08.23 15:44, Stefan Koch wrote:
> > Hi
> >
> > This updated v4 patchset implement suggestions from reviewed v3
> > patchset.
> > It's now a set of five patches:
> > - linux-custom: Split up binaries from kernel headers to kbuild
> > package
> > - linux-custom: Provide host and target specific kernel kbuild
> > packages
> > - docs: Update custom_kernel docs for split up of kernel scripts
> > and tools
> >
> > The main use-case was to swap out the binaries
> > from the kernel headers into kernel kbuild package.
> > This is introduced by the first commit
> > "Split up binaries from kernel headers to kbuild package"
> >
> > The second commit "Provide host and target specific kernel kbuild
> > packages"
> > introduces that the binaries could be swapped out into host and
> > target
> > specific kernel kbuild packages.
> >
> > The main development goals were these:
> >
> > 1. Solve already known isar custom kernel
> > limitations from doc/custom_kernel.inc
> > - kernel headers package does not support both native
> > and cross compilation of kernel modules when cross built
> >
> > 2. Honor recommendations for future from doc/custom_kernel.inc
> > - Generate kernel headers packages for both host and target
> > when using cross build
> >
> > 3. Add extensions known from debian kernel packages structure
> > - Generate a kernel headers package without binaries
> > - Create specific kernel kbuild packages that
> > will ship the "scripts" and "tools" binaries
> > - Use symlinks to point to the "scripts" and "tools" binaries
> >
> > 4. Be user friendly
> > - Avoid redundant configuration of kernel source definitions with
> > user
> > actions to enable kbuild package generation
> > - Use already known way to include linux-custom.inc in just one
> > own bitbake recipe that provides the kernel source definitions
> > - Keep known user behavior for existing build configurations:
> > just update isar, request "-compat" (and "-native") bitbake
> > targets
> > and kbuild packages for target and host will be created
> > automatically
> >
> > Best regards
> >
> > Stefan
> >
> > Stefan Koch (3):
> > linux-custom: Split up binaries from kernel headers to kbuild
> > package
> > linux-custom: Provide host and target specific kernel kbuild
> > packages
> > docs: Update custom_kernel docs for split up of kernel scripts
> > and
> > tools
> >
> > doc/custom_kernel.md | 16 ++--
> > .../linux/files/debian/control.tmpl | 25 ++++++-
> > .../linux/files/debian/isar/build.tmpl | 12 ++-
> > .../linux/files/debian/isar/common.tmpl | 10 +++
> > .../linux/files/debian/isar/install.tmpl | 73 ++++++++++++++-
> > ----
> > .../linux/files/debian/rules.tmpl | 2 +-
> > meta/recipes-kernel/linux/linux-custom.inc | 53 ++++++++++++--
> > 7 files changed, 155 insertions(+), 36 deletions(-)
> >
>
> I'd like to complete this rather soon now because it should allow us
> to
> build and actually use a custom linux-libc-dev - currently only the
> one
> of the target architecture is built, and that can't be installed into
> sbuild-chroot because it conflicts with linux-libc-dev:native.
>
> What were the plans to revive it? Patch 1 was ready but patch 2
> needed
> to stay away from misusing -compat.
>
> Jan
>
Base implementation of new patchset is already finished.
It will soon be published.
Stefan
--
Stefan Koch
Siemens AG
www.siemens.com
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PATCH v4 0/3] linux-custom: Split up binaries from kernel headers to kbuild packages
2024-01-08 12:18 ` Koch, Stefan
@ 2024-01-08 12:20 ` Jan Kiszka
0 siblings, 0 replies; 17+ messages in thread
From: Jan Kiszka @ 2024-01-08 12:20 UTC (permalink / raw)
To: Koch, Stefan (DI PA DCP R&D 3),
Schmidt, Adriaan (T CED EDC-DE),
isar-users
Cc: Sudler, Simon (DI PA DCP TI), Storm, Christian (T CED OES-DE),
ubely, Hombourger, Cedric (DI CTO FDS CES LX),
Adler, Michael (T CED OES-DE), Moessbauer, Felix (T CED OES-DE)
On 08.01.24 13:18, Koch, Stefan (DI PA DCP R&D 3) wrote:
> On Sun, 2023-12-17 at 17:50 +0100, Jan Kiszka wrote:
>> On 21.08.23 15:44, Stefan Koch wrote:
>>> Hi
>>>
>>> This updated v4 patchset implement suggestions from reviewed v3
>>> patchset.
>>> It's now a set of five patches:
>>> - linux-custom: Split up binaries from kernel headers to kbuild
>>> package
>>> - linux-custom: Provide host and target specific kernel kbuild
>>> packages
>>> - docs: Update custom_kernel docs for split up of kernel scripts
>>> and tools
>>>
>>> The main use-case was to swap out the binaries
>>> from the kernel headers into kernel kbuild package.
>>> This is introduced by the first commit
>>> "Split up binaries from kernel headers to kbuild package"
>>>
>>> The second commit "Provide host and target specific kernel kbuild
>>> packages"
>>> introduces that the binaries could be swapped out into host and
>>> target
>>> specific kernel kbuild packages.
>>>
>>> The main development goals were these:
>>>
>>> 1. Solve already known isar custom kernel
>>> limitations from doc/custom_kernel.inc
>>> - kernel headers package does not support both native
>>> and cross compilation of kernel modules when cross built
>>>
>>> 2. Honor recommendations for future from doc/custom_kernel.inc
>>> - Generate kernel headers packages for both host and target
>>> when using cross build
>>>
>>> 3. Add extensions known from debian kernel packages structure
>>> - Generate a kernel headers package without binaries
>>> - Create specific kernel kbuild packages that
>>> will ship the "scripts" and "tools" binaries
>>> - Use symlinks to point to the "scripts" and "tools" binaries
>>>
>>> 4. Be user friendly
>>> - Avoid redundant configuration of kernel source definitions with
>>> user
>>> actions to enable kbuild package generation
>>> - Use already known way to include linux-custom.inc in just one
>>> own bitbake recipe that provides the kernel source definitions
>>> - Keep known user behavior for existing build configurations:
>>> just update isar, request "-compat" (and "-native") bitbake
>>> targets
>>> and kbuild packages for target and host will be created
>>> automatically
>>>
>>> Best regards
>>>
>>> Stefan
>>>
>>> Stefan Koch (3):
>>> linux-custom: Split up binaries from kernel headers to kbuild
>>> package
>>> linux-custom: Provide host and target specific kernel kbuild
>>> packages
>>> docs: Update custom_kernel docs for split up of kernel scripts
>>> and
>>> tools
>>>
>>> doc/custom_kernel.md | 16 ++--
>>> .../linux/files/debian/control.tmpl | 25 ++++++-
>>> .../linux/files/debian/isar/build.tmpl | 12 ++-
>>> .../linux/files/debian/isar/common.tmpl | 10 +++
>>> .../linux/files/debian/isar/install.tmpl | 73 ++++++++++++++-
>>> ----
>>> .../linux/files/debian/rules.tmpl | 2 +-
>>> meta/recipes-kernel/linux/linux-custom.inc | 53 ++++++++++++--
>>> 7 files changed, 155 insertions(+), 36 deletions(-)
>>>
>>
>> I'd like to complete this rather soon now because it should allow us
>> to
>> build and actually use a custom linux-libc-dev - currently only the
>> one
>> of the target architecture is built, and that can't be installed into
>> sbuild-chroot because it conflicts with linux-libc-dev:native.
>>
>> What were the plans to revive it? Patch 1 was ready but patch 2
>> needed
>> to stay away from misusing -compat.
>>
>> Jan
>>
>
> Base implementation of new patchset is already finished.
> It will soon be published.
>
Looking forward!
Fortunately, my own demand was a misunderstanding - Isar already
delivers all I needed (linux-libc-dev-arm64-cross).
Jan
--
Siemens AG, Technology
Linux Expert Center
^ permalink raw reply [flat|nested] 17+ messages in thread
end of thread, other threads:[~2024-01-08 12:20 UTC | newest]
Thread overview: 17+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-08-21 13:44 [PATCH v4 0/3] linux-custom: Split up binaries from kernel headers to kbuild packages Stefan Koch
2023-08-21 13:44 ` [PATCH v4 1/3] linux-custom: Split up binaries from kernel headers to kbuild package Stefan Koch
2023-08-21 13:45 ` [PATCH v4 2/3] linux-custom: Provide host and target specific kernel kbuild packages Stefan Koch
2023-09-06 6:38 ` Schmidt, Adriaan
2023-10-04 15:34 ` Koch, Stefan
2023-10-04 16:15 ` Jan Kiszka
2023-10-10 9:12 ` Koch, Stefan
2023-08-21 13:45 ` [PATCH v4 3/3] docs: Update custom_kernel docs for split up of kernel scripts and tools Stefan Koch
2023-09-01 10:22 ` [PATCH v4 0/3] linux-custom: Split up binaries from kernel headers to kbuild packages Uladzimir Bely
2023-09-05 20:18 ` Jan Kiszka
2023-09-05 20:17 ` Jan Kiszka
2023-10-04 15:36 ` Koch, Stefan
2023-11-21 7:26 ` Uladzimir Bely
2023-12-17 16:50 ` Jan Kiszka
2024-01-08 10:05 ` Koch, Stefan
2024-01-08 12:18 ` Koch, Stefan
2024-01-08 12:20 ` Jan Kiszka
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox