[PATCH] linux-custom: introduce Specific Target Build and some fixes

[PATCH] linux-custom: introduce Specific Target Build and some fixes

['Nicusor Huhulea' via isar-users]

Hello,


These series primarily are offering an alternative to the Full Build(default build) compilation of the kernel 
by introducing the Specific Target Build. An issue from a Full Build is the compilation of all dtbs in the case where
just one or a few is needed i.e. DTB_FILES
This automatic compilation is due to the kernel configuration that automatically includes the compilation of dtbs 
in the default build process and the makefiles rules in the bsp directories that builds DTBs along with the kernel by default.
Another important aspect is that Isar currently is missing this feature forcing the user to use only the Full Build approach.


Compiling the linux kernel using a specific target stops the automatic build of the dtbs, explicitly
specifying dtbs files as make targets will instruct the build process to compile only those files.
Note: Because it seems that there is no project where a .dtbo file is used, this case was not handled. But that doesn't mean
it won't work anymore, we'll still have the Full Build alternative until the .dtbo will be handled when using Targeted Build.


Pros of using a Specific Target Build:
* Efficiency, compiling only what is required(e.g. only the required dtbs) is faster and consumes fewer resources. Some analysis
will follow.
* Focus, makes it easier to focus only on the target hardware reducing the issues with unsuported or bad configurations
* Particularly for the ARM architecture it's better to compile only the specific dtbs needed for the target hardware and
a Full Build should be reserved for the whole range of target devices from the bsp directory. If there are multiple kernel image types 
present in the configuration(Image, zImage, lzImage, Image.gz) you can specify KERNEL_IMAGETYPE to indicate the image type you want to be built.(the same as in OE)
* Good to use for the development and deploying, focusing only on that target hardware and not all the targets from the arch/arm/boot/dts/bsp_dir/*
* The ability to build multiple targets is on Full Build side, but in production usually only one specific image format is deployed therefore the need
for the Specific Target Build


Certain speed and disk space usage tests were made. These tests targeted the phyboard-mira(on Kernel Post-6.x
and Kernel Pre-6.x spcifically on 4.19.88) and iot2050(Kernel Post-6.x):


specific target: Build needed 00:02:08, 2219688k disk space - linux-phy kernel 6.x
full build:      Build needed 00:02:14, 2271528k disk space - linux-phy kernel 6.x

specific build : Build needed 00:02:19, 2355400k disk space - linux-iot2050 kernel 6.x
full build:      Build needed 00:02:23, 2356828k disk space - liux-iot2050 kernel 6.x


On linux-phy Kernel 6.x the full build took 6 seconds longer than the specific target build and
used approximately 50.6 MB more disk space than the specific target build. The iot2050 data is somewhat similar.
Based on the data from the dpkg_build.log the benefits of using specific targets reduces build times
and resource efficiency. However there can be some potential pitfalls associated with this approach. When the engineer
is not intimate with the bsp certain kernel configurations or some dependencies may not be satisfied. And this is
why this is enabled only when KERNEL_IMAGETYPE is present.

Reference:
https://groups.google.com/g/isar-users/c/2Gjupa2m0OA

Nicu

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/isar-users/20240814173524.383600-1-nicusor.huhulea%40siemens.com.

[PATCH 1/3] linux-custom: refactor kernel's make command

['Nicusor Huhulea' via isar-users]

Simplify the kernel's make command by improving readability/structure.

Signed-off-by: Nicusor Huhulea <nicusor.huhulea@siemens.com>
---
 .../recipes-kernel/linux/files/debian/isar/build.tmpl | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl
index bafc5ca4..1fd6f948 100644
--- a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl
+++ b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl
@@ -21,15 +21,16 @@ do_build() {
     KR=$(${MAKE} O=${KERNEL_BUILD_DIR} -s --no-print-directory kernelrelease)
     sed -i "s/@KR@/${KR}/g" ${S}/debian/control ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.*
 
-    if echo "${DEB_BUILD_PROFILES}" | grep -q "kernel"; then # Build kernel scripts and tools
-        ${MAKE} O=${KERNEL_BUILD_DIR} ${PARALLEL_MAKE} ${KERNEL_EXTRA_BUILDARGS} KCFLAGS="${KCFLAGS}" KAFLAGS="${KAFLAGS}"
+    MAKE_COMMON_ARGS="O=${KERNEL_BUILD_DIR} ${PARALLEL_MAKE} KCFLAGS=${KCFLAGS} KAFLAGS=${KAFLAGS}"
+    if echo "${DEB_BUILD_PROFILES}" | grep -q "kernel"; then # Full build
+        ${MAKE} ${MAKE_COMMON_ARGS} ${KERNEL_EXTRA_BUILDARGS}
     elif echo "${DEB_BUILD_PROFILES}" | grep -q "kbuild"; then # Build kernel scripts and tools
-        ${MAKE} O=${KERNEL_BUILD_DIR} ${PARALLEL_MAKE} ${KERNEL_EXTRA_BUILDARGS} KCFLAGS="${KCFLAGS}" KAFLAGS="${KAFLAGS}" scripts
+        ${MAKE} ${MAKE_COMMON_ARGS} ${KERNEL_EXTRA_BUILDARGS} scripts
         if grep -q -E "CONFIG_STACK_VALIDATION=y|CONFIG_HAVE_OBJTOOL=y" ${KERNEL_BUILD_DIR}/.config && [ -d "tools/objtool" ]; then
-            ${MAKE} O=${KERNEL_BUILD_DIR} ${PARALLEL_MAKE} KCFLAGS="${KCFLAGS}" KAFLAGS="${KAFLAGS}" tools/objtool || true
+            ${MAKE} ${MAKE_COMMON_ARGS} tools/objtool || true
         fi
         if grep -q "CONFIG_MODULES=y" ${KERNEL_BUILD_DIR}/.config; then
-            ${MAKE} O=${KERNEL_BUILD_DIR} ${PARALLEL_MAKE} KCFLAGS="${KCFLAGS}" KAFLAGS="${KAFLAGS}" modules_prepare
+            ${MAKE} ${MAKE_COMMON_ARGS} modules_prepare
         fi
     fi
 
-- 
2.39.2

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/isar-users/20240814173524.383600-2-nicusor.huhulea%40siemens.com.

[PATCH 2/3] linux-custom: compile only the specified dtbs by using specific target build

['Nicusor Huhulea' via isar-users]

The current implementation makes a full kernel build and that means it will
compile all dtbs specified by the bsp makefile e.g arch/arm64/boot/dts/ti/Makefile
Currently the required dtbs are given by the DTB_FILES.

These changes are checking if the DTB_FILES is specified, compile only those
and avoids compiling all the dtbs from the bsp directory. Otherwise build all dtbs
specified by the kernel source.
Preventing the automatic dtb builds is made by using target specific build
commands e.g KERNEL_IMAGETYPE
The compilation of the dtbs accepts the following formats e.g:
Kernel Post 6.x:
a) full path from the architecture directory:
   arch/${ARCH}/boot/dts/bsp_dir/test.dts
   arch/${ARCH}/boot/dts/bsp_dir/test.dtb
b) relative path from the dts directory: bsp_dir/test.dts
c) device tree blob file: bsp_dir/test.dtb
Kernel Pre 6.x:
The same as on Kernel Post 6.x but without the bsp_dir

The differences when building a full build and when using a specific target
build are:(tests were made on the machine phyboard-mira on 6.x and 4.x versions)
specific target: Build needed 00:02:08, 2219688k disk space linux-phy 6.x
full build:      Build needed 00:02:14, 2271528k disk space linux-phy 6.x
The time difference: the full build took 6s longer than the specific target build
The disk space difference: the full build used 50.6MB more disk space than
the specific target build.
In conclusion the specific target build is slightly faster and uses less
disk space.

For now the DTB_FILES variable should accept only Device Tree Blobs(.dtb)
because some areas still depends on it e.g: do_copy_boot_files, plugins

Signed-off-by: Nicusor Huhulea <nicusor.huhulea@siemens.com>
---
 .../linux/files/debian/isar/build.tmpl        | 56 ++++++++++++++++++-
 .../linux/files/debian/isar/install.tmpl      | 12 +++-
 meta/recipes-kernel/linux/linux-custom.inc    |  6 ++
 3 files changed, 71 insertions(+), 3 deletions(-)

diff --git a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl
index 1fd6f948..6f186e02 100644
--- a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl
+++ b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl
@@ -22,8 +22,13 @@ do_build() {
     sed -i "s/@KR@/${KR}/g" ${S}/debian/control ${S}/debian/linux-image-${KERNEL_NAME_PROVIDED}.*
 
     MAKE_COMMON_ARGS="O=${KERNEL_BUILD_DIR} ${PARALLEL_MAKE} KCFLAGS=${KCFLAGS} KAFLAGS=${KAFLAGS}"
-    if echo "${DEB_BUILD_PROFILES}" | grep -q "kernel"; then # Full build
-        ${MAKE} ${MAKE_COMMON_ARGS} ${KERNEL_EXTRA_BUILDARGS}
+    if echo "${DEB_BUILD_PROFILES}" | grep -q "kernel"; then
+        if [ -n "${KERNEL_IMAGETYPE}" ]; then # Override the default compilation
+            ${MAKE} ${MAKE_COMMON_ARGS} ${KERNEL_EXTRA_BUILDARGS} ${KERNEL_IMAGETYPE} modules
+            compile_dtbs
+        else $ Full build
+            ${MAKE} ${MAKE_COMMON_ARGS} ${KERNEL_EXTRA_BUILDARGS}
+        fi
     elif echo "${DEB_BUILD_PROFILES}" | grep -q "kbuild"; then # Build kernel scripts and tools
         ${MAKE} ${MAKE_COMMON_ARGS} ${KERNEL_EXTRA_BUILDARGS} scripts
         if grep -q -E "CONFIG_STACK_VALIDATION=y|CONFIG_HAVE_OBJTOOL=y" ${KERNEL_BUILD_DIR}/.config && [ -d "tools/objtool" ]; then
@@ -38,6 +43,53 @@ do_build() {
     set +x
 }
 
+compile_dtbs() {
+    local prefix="arch/${ARCH}/boot/dts/"
+
+    for dtb in ${DTB_FILES}; do
+        # Handle the case where a standalone(with no path .dtb/.dts) file may reside within
+        # a specific BSP directory or not(kernel pre-6.x)
+        if [[ ("${dtb}" == *.dtb || "${dtb}" == *.dts) && "${dtb}" != */* ]]; then
+            bsp_dir=""
+            dts_path=""
+            if [[ "${dtb}" == *.dtb ]]; then
+                dts="${dtb%.dtb}.dts"
+            else
+                dts="${dtb}"
+            fi
+
+            # recursively search for the dts file in all possible BSP directories
+            while IFS= read -r dts_path; do
+                if [ -n "${dts_path}" ]; then
+                    # Eliminate the prefix and dts to get only the bsp_dir
+                    bsp_dir="${dts_path#${prefix}}"
+                    bsp_dir="${bsp_dir%${dts}}"
+
+                    relative_dtb_path="${bsp_dir}${dtb}"
+                else
+                    relative_dtb_path="${prefix}${dtb}"
+                fi
+            done < <(find "${prefix}" -type f -name "${dts}")
+
+        # Check if the path is relative (starts with arch/${ARCH}/)
+        # and if it does then extract the relative path from the architecture specific path
+        elif [[ "${dtb}" == "${prefix}"* ]]; then
+            relative_dtb_path="${dtb#${prefix}}"
+        else
+            # directly use the given dtb
+            relative_dtb_path="${dtb}"
+        fi
+
+        # Check if it's a .dts file
+        if [[ "${relative_dtb_path}" == *.dts ]]; then
+            #  Change .dts to .dtb
+            relative_dtb_path="${relative_dtb_path%.dts}.dtb"
+        fi
+
+        ${MAKE} ${MAKE_COMMON_ARGS} ${relative_dtb_path}
+    done
+}
+
 print_settings() {
     cat <<EOF
 # Build settings:
diff --git a/meta/recipes-kernel/linux/files/debian/isar/install.tmpl b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl
index 00011517..c0a60f4e 100644
--- a/meta/recipes-kernel/linux/files/debian/isar/install.tmpl
+++ b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl
@@ -124,7 +124,17 @@ install_hooks() {
 
 install_dtbs() {
     [ -n "${CONFIG_OF}" ] || return 0
-    ${MAKE} O=${O} INSTALL_DTBS_PATH=${deb_img_dir}/usr/lib/linux-image-${krel} dtbs_install
+    prefix="arch/${ARCH}/boot/dts/"
+    src_dir="${O}/${prefix}"
+    dst_dir=""
+
+    dst_dir="${deb_img_dir}/usr/lib/linux-image-${krel}/"
+    find "${src_dir}" -type f -name '*.dtb' | while IFS= read -r dtb_file; do
+        local relative_path="${dtb_file#"${src_dir}"}"
+        mkdir -p "${dst_dir}$(dirname "${relative_path}")"
+        cp "${dtb_file}" "${dst_dir}/${relative_path}"
+        echo "Copying ${dtb_file} to ${dest_dir}/${relative_path}"
+    done
 }
 
 install_kmods() {
diff --git a/meta/recipes-kernel/linux/linux-custom.inc b/meta/recipes-kernel/linux/linux-custom.inc
index 6aa70fd3..31cc480b 100644
--- a/meta/recipes-kernel/linux/linux-custom.inc
+++ b/meta/recipes-kernel/linux/linux-custom.inc
@@ -76,6 +76,7 @@ TEMPLATE_VARS += "                \
     KERNEL_DEBIAN_DEPENDS         \
     KERNEL_BUILD_DIR              \
     KERNEL_FILE                   \
+    KERNEL_IMAGETYPE              \
     KERNEL_HEADERS_DEBIAN_DEPENDS \
     LINUX_VERSION_EXTENSION       \
     KERNEL_NAME_PROVIDED          \
@@ -84,12 +85,17 @@ TEMPLATE_VARS += "                \
     KAFLAGS                       \
     DISTRIBUTOR                   \
     KERNEL_EXTRA_BUILDARGS        \
+    DTB_FILES                     \
 "
 
 inherit dpkg
 inherit template
 inherit kbuildtarget
 
+DTB_FILES ?= ""
+
+KERNEL_IMAGETYPE ?= ""
+
 # Add custom cflags to the kernel build
 KCFLAGS ?= "-fdebug-prefix-map=${CURDIR}=."
 KAFLAGS ?= "-fdebug-prefix-map=${CURDIR}=."
-- 
2.39.2

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/isar-users/20240814173524.383600-3-nicusor.huhulea%40siemens.com.

[PATCH 3/3] phyboard-mira.conf: use a specific target build

['Nicusor Huhulea' via isar-users]

The reason for this change is that phyboard-mira full build produces
several tens of dtbs from the nxp bsp dir, therefore use a target build
to get only the desired dtb
It's also needed so that we have at least one test in-tree.


Signed-off-by: Nicusor Huhulea <nicusor.huhulea@siemens.com>
---
 meta-isar/conf/machine/phyboard-mira.conf | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/meta-isar/conf/machine/phyboard-mira.conf b/meta-isar/conf/machine/phyboard-mira.conf
index feec4e54..1d2b3e1c 100644
--- a/meta-isar/conf/machine/phyboard-mira.conf
+++ b/meta-isar/conf/machine/phyboard-mira.conf
@@ -20,6 +20,8 @@ DTB_FILES = "imx6q-phytec-mira-rdk-nand.dtb"
 
 IMAGE_INSTALL += "barebox"
 
+KERNEL_IMAGETYPE = "zImage"
+
 BAREBOX_CONFIG = "imx_v7_defconfig"
 BAREBOX_ENV = "phytec-mira-env"
 BAREBOX_BASE_BIN = "barebox-phytec-phycore-imx6q-som-nand-1gib"
-- 
2.39.2

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/isar-users/20240814173524.383600-4-nicusor.huhulea%40siemens.com.

Re: [PATCH 2/3] linux-custom: compile only the specified dtbs by using specific target build

[Uladzimir Bely]

On Wed, 2024-08-14 at 20:35 +0300, 'Nicusor Huhulea' via isar-users
wrote:
> The current implementation makes a full kernel build and that means
> it will
> compile all dtbs specified by the bsp makefile e.g
> arch/arm64/boot/dts/ti/Makefile
> Currently the required dtbs are given by the DTB_FILES.
> 
> These changes are checking if the DTB_FILES is specified, compile
> only those
> and avoids compiling all the dtbs from the bsp directory. Otherwise
> build all dtbs
> specified by the kernel source.
> Preventing the automatic dtb builds is made by using target specific
> build
> commands e.g KERNEL_IMAGETYPE
> The compilation of the dtbs accepts the following formats e.g:
> Kernel Post 6.x:
> a) full path from the architecture directory:
>    arch/${ARCH}/boot/dts/bsp_dir/test.dts
>    arch/${ARCH}/boot/dts/bsp_dir/test.dtb
> b) relative path from the dts directory: bsp_dir/test.dts
> c) device tree blob file: bsp_dir/test.dtb
> Kernel Pre 6.x:
> The same as on Kernel Post 6.x but without the bsp_dir
> 
> The differences when building a full build and when using a specific
> target
> build are:(tests were made on the machine phyboard-mira on 6.x and
> 4.x versions)
> specific target: Build needed 00:02:08, 2219688k disk space linux-phy
> 6.x
> full build:      Build needed 00:02:14, 2271528k disk space linux-phy
> 6.x
> The time difference: the full build took 6s longer than the specific
> target build
> The disk space difference: the full build used 50.6MB more disk space
> than
> the specific target build.
> In conclusion the specific target build is slightly faster and uses
> less
> disk space.
> 
> For now the DTB_FILES variable should accept only Device Tree
> Blobs(.dtb)
> because some areas still depends on it e.g: do_copy_boot_files,
> plugins
> 
> Signed-off-by: Nicusor Huhulea <nicusor.huhulea@siemens.com>
> ---
>  .../linux/files/debian/isar/build.tmpl        | 56
> ++++++++++++++++++-
>  .../linux/files/debian/isar/install.tmpl      | 12 +++-
>  meta/recipes-kernel/linux/linux-custom.inc    |  6 ++
>  3 files changed, 71 insertions(+), 3 deletions(-)
> 
> diff --git a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl
> b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl
> index 1fd6f948..6f186e02 100644
> --- a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl
> +++ b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl
> @@ -22,8 +22,13 @@ do_build() {
>      sed -i "s/@KR@/${KR}/g" ${S}/debian/control ${S}/debian/linux-
> image-${KERNEL_NAME_PROVIDED}.*
>  
>      MAKE_COMMON_ARGS="O=${KERNEL_BUILD_DIR} ${PARALLEL_MAKE}
> KCFLAGS=${KCFLAGS} KAFLAGS=${KAFLAGS}"
> -    if echo "${DEB_BUILD_PROFILES}" | grep -q "kernel"; then # Full
> build
> -        ${MAKE} ${MAKE_COMMON_ARGS} ${KERNEL_EXTRA_BUILDARGS}
> +    if echo "${DEB_BUILD_PROFILES}" | grep -q "kernel"; then
> +        if [ -n "${KERNEL_IMAGETYPE}" ]; then # Override the default
> compilation
> +            ${MAKE} ${MAKE_COMMON_ARGS} ${KERNEL_EXTRA_BUILDARGS}
> ${KERNEL_IMAGETYPE} modules
> +            compile_dtbs
> +        else $ Full build

A typo causing build error.

'/<<PKGBUILDDIR>>/debian/isar/build: line 29: $: command not found'.

> +            ${MAKE} ${MAKE_COMMON_ARGS} ${KERNEL_EXTRA_BUILDARGS}
> +        fi
>      elif echo "${DEB_BUILD_PROFILES}" | grep -q "kbuild"; then #
> Build kernel scripts and tools
>          ${MAKE} ${MAKE_COMMON_ARGS} ${KERNEL_EXTRA_BUILDARGS}
> scripts
>          if grep -q -E
> "CONFIG_STACK_VALIDATION=y|CONFIG_HAVE_OBJTOOL=y"
> ${KERNEL_BUILD_DIR}/.config && [ -d "tools/objtool" ]; then
> @@ -38,6 +43,53 @@ do_build() {
>      set +x
>  }
>  
> +compile_dtbs() {
> +    local prefix="arch/${ARCH}/boot/dts/"
> +
> +    for dtb in ${DTB_FILES}; do
> +        # Handle the case where a standalone(with no path .dtb/.dts)
> file may reside within
> +        # a specific BSP directory or not(kernel pre-6.x)
> +        if [[ ("${dtb}" == *.dtb || "${dtb}" == *.dts) && "${dtb}"
> != */* ]]; then
> +            bsp_dir=""
> +            dts_path=""
> +            if [[ "${dtb}" == *.dtb ]]; then
> +                dts="${dtb%.dtb}.dts"
> +            else
> +                dts="${dtb}"
> +            fi
> +
> +            # recursively search for the dts file in all possible
> BSP directories
> +            while IFS= read -r dts_path; do
> +                if [ -n "${dts_path}" ]; then
> +                    # Eliminate the prefix and dts to get only the
> bsp_dir
> +                    bsp_dir="${dts_path#${prefix}}"
> +                    bsp_dir="${bsp_dir%${dts}}"
> +
> +                    relative_dtb_path="${bsp_dir}${dtb}"
> +                else
> +                    relative_dtb_path="${prefix}${dtb}"
> +                fi
> +            done < <(find "${prefix}" -type f -name "${dts}")
> +
> +        # Check if the path is relative (starts with arch/${ARCH}/)
> +        # and if it does then extract the relative path from the
> architecture specific path
> +        elif [[ "${dtb}" == "${prefix}"* ]]; then
> +            relative_dtb_path="${dtb#${prefix}}"
> +        else
> +            # directly use the given dtb
> +            relative_dtb_path="${dtb}"
> +        fi
> +
> +        # Check if it's a .dts file
> +        if [[ "${relative_dtb_path}" == *.dts ]]; then
> +            #  Change .dts to .dtb
> +            relative_dtb_path="${relative_dtb_path%.dts}.dtb"
> +        fi
> +
> +        ${MAKE} ${MAKE_COMMON_ARGS} ${relative_dtb_path}
> +    done
> +}
> +
>  print_settings() {
>      cat <<EOF
>  # Build settings:
> diff --git a/meta/recipes-kernel/linux/files/debian/isar/install.tmpl
> b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl
> index 00011517..c0a60f4e 100644
> --- a/meta/recipes-kernel/linux/files/debian/isar/install.tmpl
> +++ b/meta/recipes-kernel/linux/files/debian/isar/install.tmpl
> @@ -124,7 +124,17 @@ install_hooks() {
>  
>  install_dtbs() {
>      [ -n "${CONFIG_OF}" ] || return 0
> -    ${MAKE} O=${O} INSTALL_DTBS_PATH=${deb_img_dir}/usr/lib/linux-
> image-${krel} dtbs_install
> +    prefix="arch/${ARCH}/boot/dts/"
> +    src_dir="${O}/${prefix}"
> +    dst_dir=""
> +
> +    dst_dir="${deb_img_dir}/usr/lib/linux-image-${krel}/"
> +    find "${src_dir}" -type f -name '*.dtb' | while IFS= read -r
> dtb_file; do
> +        local relative_path="${dtb_file#"${src_dir}"}"
> +        mkdir -p "${dst_dir}$(dirname "${relative_path}")"
> +        cp "${dtb_file}" "${dst_dir}/${relative_path}"
> +        echo "Copying ${dtb_file} to ${dest_dir}/${relative_path}"
> +    done
>  }
>  
>  install_kmods() {
> diff --git a/meta/recipes-kernel/linux/linux-custom.inc
> b/meta/recipes-kernel/linux/linux-custom.inc
> index 6aa70fd3..31cc480b 100644
> --- a/meta/recipes-kernel/linux/linux-custom.inc
> +++ b/meta/recipes-kernel/linux/linux-custom.inc
> @@ -76,6 +76,7 @@ TEMPLATE_VARS += "                \
>      KERNEL_DEBIAN_DEPENDS         \
>      KERNEL_BUILD_DIR              \
>      KERNEL_FILE                   \
> +    KERNEL_IMAGETYPE              \
>      KERNEL_HEADERS_DEBIAN_DEPENDS \
>      LINUX_VERSION_EXTENSION       \
>      KERNEL_NAME_PROVIDED          \
> @@ -84,12 +85,17 @@ TEMPLATE_VARS += "                \
>      KAFLAGS                       \
>      DISTRIBUTOR                   \
>      KERNEL_EXTRA_BUILDARGS        \
> +    DTB_FILES                     \
>  "
>  
>  inherit dpkg
>  inherit template
>  inherit kbuildtarget
>  
> +DTB_FILES ?= ""
> +
> +KERNEL_IMAGETYPE ?= ""
> +
>  # Add custom cflags to the kernel build
>  KCFLAGS ?= "-fdebug-prefix-map=${CURDIR}=."
>  KAFLAGS ?= "-fdebug-prefix-map=${CURDIR}=."
> -- 
> 2.39.2
> 

-- 
Best regards,
Uladzimir.

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/isar-users/34d9cdff55bf0eba83a40d49cef32792020f7bbc.camel%40ilbers.de.

Re: [PATCH 2/3] linux-custom: compile only the specified dtbs by using specific target build

['Jan Kiszka' via isar-users]

On 23.08.24 10:44, Uladzimir Bely wrote:
> On Wed, 2024-08-14 at 20:35 +0300, 'Nicusor Huhulea' via isar-users
> wrote:
>> The current implementation makes a full kernel build and that means
>> it will
>> compile all dtbs specified by the bsp makefile e.g
>> arch/arm64/boot/dts/ti/Makefile
>> Currently the required dtbs are given by the DTB_FILES.
>>
>> These changes are checking if the DTB_FILES is specified, compile
>> only those
>> and avoids compiling all the dtbs from the bsp directory. Otherwise
>> build all dtbs
>> specified by the kernel source.
>> Preventing the automatic dtb builds is made by using target specific
>> build
>> commands e.g KERNEL_IMAGETYPE
>> The compilation of the dtbs accepts the following formats e.g:
>> Kernel Post 6.x:
>> a) full path from the architecture directory:
>>    arch/${ARCH}/boot/dts/bsp_dir/test.dts
>>    arch/${ARCH}/boot/dts/bsp_dir/test.dtb
>> b) relative path from the dts directory: bsp_dir/test.dts
>> c) device tree blob file: bsp_dir/test.dtb
>> Kernel Pre 6.x:
>> The same as on Kernel Post 6.x but without the bsp_dir
>>
>> The differences when building a full build and when using a specific
>> target
>> build are:(tests were made on the machine phyboard-mira on 6.x and
>> 4.x versions)
>> specific target: Build needed 00:02:08, 2219688k disk space linux-phy
>> 6.x
>> full build:      Build needed 00:02:14, 2271528k disk space linux-phy
>> 6.x
>> The time difference: the full build took 6s longer than the specific
>> target build
>> The disk space difference: the full build used 50.6MB more disk space
>> than
>> the specific target build.
>> In conclusion the specific target build is slightly faster and uses
>> less
>> disk space.
>>
>> For now the DTB_FILES variable should accept only Device Tree
>> Blobs(.dtb)
>> because some areas still depends on it e.g: do_copy_boot_files,
>> plugins
>>
>> Signed-off-by: Nicusor Huhulea <nicusor.huhulea@siemens.com>
>> ---
>>  .../linux/files/debian/isar/build.tmpl        | 56
>> ++++++++++++++++++-
>>  .../linux/files/debian/isar/install.tmpl      | 12 +++-
>>  meta/recipes-kernel/linux/linux-custom.inc    |  6 ++
>>  3 files changed, 71 insertions(+), 3 deletions(-)
>>
>> diff --git a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl
>> b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl
>> index 1fd6f948..6f186e02 100644
>> --- a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl
>> +++ b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl
>> @@ -22,8 +22,13 @@ do_build() {
>>      sed -i "s/@KR@/${KR}/g" ${S}/debian/control ${S}/debian/linux-
>> image-${KERNEL_NAME_PROVIDED}.*
>>  
>>      MAKE_COMMON_ARGS="O=${KERNEL_BUILD_DIR} ${PARALLEL_MAKE}
>> KCFLAGS=${KCFLAGS} KAFLAGS=${KAFLAGS}"
>> -    if echo "${DEB_BUILD_PROFILES}" | grep -q "kernel"; then # Full
>> build
>> -        ${MAKE} ${MAKE_COMMON_ARGS} ${KERNEL_EXTRA_BUILDARGS}
>> +    if echo "${DEB_BUILD_PROFILES}" | grep -q "kernel"; then
>> +        if [ -n "${KERNEL_IMAGETYPE}" ]; then # Override the default
>> compilation
>> +            ${MAKE} ${MAKE_COMMON_ARGS} ${KERNEL_EXTRA_BUILDARGS}
>> ${KERNEL_IMAGETYPE} modules
>> +            compile_dtbs
>> +        else $ Full build
> 
> A typo causing build error.
> 
> '/<<PKGBUILDDIR>>/debian/isar/build: line 29: $: command not found'.
> 

Just to be clear: even once issues in such details are resolved, I'm
still not convinced about the value of the approach, compared to the
maintenance risks it would introduce.

Jan

-- 
Siemens AG, Technology
Linux Expert Center

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/isar-users/f7ce8b07-2ead-4b00-890d-b4de9313caf5%40siemens.com.

Re: [PATCH 3/3] phyboard-mira.conf: use a specific target build

['Jan Kiszka' via isar-users]

On 13.08.24 13:55, 'Nicusor Huhulea' via isar-users wrote:
> The reason for this change is that phyboard-mira full build produces
> several tens of dtbs from the nxp bsp dir, therefore use a target build
> to get only the desired dtb
> 

It's also needed so that we have at least one test in-tree.

> Signed-off-by: Nicusor Huhulea <nicusor.huhulea@siemens.com>
> ---
>  meta-isar/conf/machine/phyboard-mira.conf | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/meta-isar/conf/machine/phyboard-mira.conf b/meta-isar/conf/machine/phyboard-mira.conf
> index feec4e54..1d2b3e1c 100644
> --- a/meta-isar/conf/machine/phyboard-mira.conf
> +++ b/meta-isar/conf/machine/phyboard-mira.conf
> @@ -20,6 +20,8 @@ DTB_FILES = "imx6q-phytec-mira-rdk-nand.dtb"
>  
>  IMAGE_INSTALL += "barebox"
>  
> +KERNEL_IMAGETYPE = "zImage"
> +
>  BAREBOX_CONFIG = "imx_v7_defconfig"
>  BAREBOX_ENV = "phytec-mira-env"
>  BAREBOX_BASE_BIN = "barebox-phytec-phycore-imx6q-som-nand-1gib"

BTW, you are missing patch 4 in your series. I reminded of it in V1 of
your submission.

Jan

-- 
Siemens AG, Technology
Linux Expert Center

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/isar-users/d554fecd-e54b-49ae-ba0c-6997ddfd3257%40siemens.com.

RE: [PATCH 3/3] phyboard-mira.conf: use a specific target build

['nicusor.huhulea@siemens.com' via isar-users]

I can add another con to the cons already mentioned in this thread, particularly for ARM architectures.
So, it's better to use a targeted build needed for the target hardware and a full build should be only reserved for the entire range of hardware supported by the bsp dir.
e.g: for me it make sense to have a full build if I'm going to do a test on all the bsps from arch/arm/nxp/freescale/*.dtbs but it does not make sense to have a full build when I'm using one bsp.

>>-----Original Message-----
>>From: Huhulea, Nicusor Liviu (DI CTO FDS CES LX SVCS)
>><nicusor.huhulea@siemens.com>
>>Sent: Tuesday, August 13, 2024 2:55 PM
>>To: isar-users@googlegroups.com
>>Cc: Huhulea, Nicusor Liviu (DI CTO FDS CES LX SVCS)
>><nicusor.huhulea@siemens.com>
>>Subject: [PATCH 3/3] phyboard-mira.conf: use a specific target build
>>
>>The reason for this change is that phyboard-mira full build produces several tens
>>of dtbs from the nxp bsp dir, therefore use a target build to get only the desired
>>dtb
>>
>>Signed-off-by: Nicusor Huhulea <nicusor.huhulea@siemens.com>
>>---
>> meta-isar/conf/machine/phyboard-mira.conf | 2 ++
>> 1 file changed, 2 insertions(+)
>>
>>diff --git a/meta-isar/conf/machine/phyboard-mira.conf b/meta-
>>isar/conf/machine/phyboard-mira.conf
>>index feec4e54..1d2b3e1c 100644
>>--- a/meta-isar/conf/machine/phyboard-mira.conf
>>+++ b/meta-isar/conf/machine/phyboard-mira.conf
>>@@ -20,6 +20,8 @@ DTB_FILES = "imx6q-phytec-mira-rdk-nand.dtb"
>>
>> IMAGE_INSTALL += "barebox"
>>
>>+KERNEL_IMAGETYPE = "zImage"
>>+
>> BAREBOX_CONFIG = "imx_v7_defconfig"
>> BAREBOX_ENV = "phytec-mira-env"
>> BAREBOX_BASE_BIN = "barebox-phytec-phycore-imx6q-som-nand-1gib"
>>--
>>2.39.2

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/isar-users/DB3PR10MB690863F9825DE317E5F92AC9E6862%40DB3PR10MB6908.EURPRD10.PROD.OUTLOOK.COM.

[PATCH 3/3] phyboard-mira.conf: use a specific target build

['Nicusor Huhulea' via isar-users]

The reason for this change is that phyboard-mira full build produces
several tens of dtbs from the nxp bsp dir, therefore use a target build
to get only the desired dtb

Signed-off-by: Nicusor Huhulea <nicusor.huhulea@siemens.com>
---
 meta-isar/conf/machine/phyboard-mira.conf | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/meta-isar/conf/machine/phyboard-mira.conf b/meta-isar/conf/machine/phyboard-mira.conf
index feec4e54..1d2b3e1c 100644
--- a/meta-isar/conf/machine/phyboard-mira.conf
+++ b/meta-isar/conf/machine/phyboard-mira.conf
@@ -20,6 +20,8 @@ DTB_FILES = "imx6q-phytec-mira-rdk-nand.dtb"
 
 IMAGE_INSTALL += "barebox"
 
+KERNEL_IMAGETYPE = "zImage"
+
 BAREBOX_CONFIG = "imx_v7_defconfig"
 BAREBOX_ENV = "phytec-mira-env"
 BAREBOX_BASE_BIN = "barebox-phytec-phycore-imx6q-som-nand-1gib"
-- 
2.39.2

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/isar-users/20240813115521.381481-3-nicusor.huhulea%40siemens.com.