[RFC PATCH 0/2] optimize caching of source packages

[RFC PATCH 0/2] optimize caching of source packages

['Cedric Hombourger' via isar-users]

When building root file-systems for a foreign architecture and needing
to cache source packages, apt will be called from within the rootfs and
consequently be executed under QEMU: this is terribly slow especially
considering that source packages are downloaded one by one. This RFC
patch series introduces a wrapper function to run native commands
against a rootfs with our special mounts such as /isar-apt. Some basic
tests were performed to smoke test this approach and evaluate its
performance. It should be noted that the changes introduce a new
dependency to bubblewrap. That would need to be documented if moving
forward and some alignment with projects such as kas would be required.
It is believed that this approach could be implemented in other places
and further remove places where we would need elevated privileges.

Cedric Hombourger (2):
  rootfs: introduce wrapper to run native commands against a rootfs
  deb-dl-dir: optimize caching of source packages using apt natively

 meta/classes/deb-dl-dir.bbclass | 36 +++++--------------
 meta/classes/rootfs.bbclass     | 64 +++++++++++++++++++++++++++++++++
 2 files changed, 72 insertions(+), 28 deletions(-)

-- 
2.39.5

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20250515150727.1764989-1-cedric.hombourger%40siemens.com.

[RFC PATCH 1/2] rootfs: introduce wrapper to run native commands against a rootfs

['Cedric Hombourger' via isar-users]

"sudo chroot" is used in several places to run commands inside rootfs
directories constructed by Isar. There are cases where a native command
could be used without elevated privileges as long as special folders
such as /isar-apt are mounted (they are often referenced as /isar-apt
in configuration files found in the target rootfs). For such cases,
bubblewrap may be used to create a non-privileged namespace (either
in a bare/native environment or within a docker/podman container)
to achieve better performance when execution through QEMU may be
avoided.

Signed-off-by: Cedric Hombourger <cedric.hombourger@siemens.com>
---
 meta/classes/rootfs.bbclass | 64 +++++++++++++++++++++++++++++++++++++
 1 file changed, 64 insertions(+)

diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass
index f16ecc00..2975eb6b 100644
--- a/meta/classes/rootfs.bbclass
+++ b/meta/classes/rootfs.bbclass
@@ -30,6 +30,70 @@ export LANG = "C"
 export LANGUAGE = "C"
 export LC_ALL = "C"
 
+# Execute a native command against a rootfs and with isar-apt bind-mounted.
+# Additional mounts may be specified using --bind <source> <target> and a
+# custom directory for the command to be executed with --chdir <dir>. The
+# command is assumed to follow the special "--" argument. This would replace
+# "sudo chroot" calls especially when a native command may be used instead of
+# chroot'ed command and without elevated privileges (the command will likely
+# take the rootfs as argument; e.g. apt-get -o Dir=${ROOTFSDIR}).
+#
+# Usage: rootfs_native_cmd [options] -- command
+#
+rootfs_native_cmd() {
+    set -- "$@"
+    bwrap_args="--bind ${REPO_ISAR_DIR}/${DISTRO} /isar-apt"
+    rootfs=""
+
+    while [ "${#}" -gt "0" ] && [ "${1}" != "--" ]; do
+        case "${1}" in
+            --bind)
+                if [ "${#}" -lt "3" ]; then
+                    bbfatal "--bind requires two arguments"
+                fi
+                bwrap_args="${bwrap_args} --bind ${2} ${3}"
+                shift 3
+                ;;
+            --chdir)
+                if [ "${#}" -lt "2" ]; then
+                    bbfatal "${1} requires an argument"
+                fi
+                bwrap_args="${bwrap_args} ${1} ${2}"
+                shift 2
+                ;;
+            -*)
+                bbfatal "${1} is not a supported option!"
+                ;;
+            *)
+                if [ -z "${rootfs}" ]; then
+                    rootfs="${1}"
+                    shift
+                else
+                    bbfatal "unexpected argument '${1}'"
+                fi
+                ;;
+        esac
+    done
+
+    [ -n "${rootfs}" ] || bbfatal "no rootfs path provided"
+
+    if [ "${#}" -le "1" ] || [ "${1}" != "--" ]; then
+        bbfatal "no command specified (missing --)"
+    fi
+
+    shift  # remove the "--"
+    exec bwrap \
+        ${bwrap_args} \
+        --bind "${rootfs}" "${rootfs}" \
+        --unshare-user \
+        --unshare-pid \
+        --dev-bind /dev /dev --proc /proc --ro-bind /sys /sys \
+        --ro-bind /etc /etc --ro-bind /bin /bin \
+        --ro-bind /lib /lib --ro-bind /lib64 /lib64 \
+        --ro-bind /usr /usr --tmpfs /tmp \
+        -- "${@}"
+}
+
 rootfs_do_mounts[weight] = "3"
 rootfs_do_mounts() {
     sudo -s <<'EOSUDO'
-- 
2.39.5

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20250515150727.1764989-2-cedric.hombourger%40siemens.com.

[RFC PATCH 2/2] deb-dl-dir: optimize caching of source packages using apt natively

['Cedric Hombourger' via isar-users]

source package are downloaded by entering the target rootfs and run
apt there. For foreign architectures, this results in apt being
executed under QEMU and leads to poor performance. By using the
recently introduced rootfs_native_cmd command wrapper, apt will be
executed natively against the target rootfs and without elevated
privileges. For our test work-load, caching was reduced from more
than 10 hours to an hour. Performance is also more consistent as
it will no longer depend as to when bitbake kicks caching of
source packages for foreign architecture rootfs vs rootfs for the
host (in multiconfig builds).

Signed-off-by: Cedric Hombourger <cedric.hombourger@siemens.com>
---
 meta/classes/deb-dl-dir.bbclass | 36 ++++++++-------------------------
 1 file changed, 8 insertions(+), 28 deletions(-)

diff --git a/meta/classes/deb-dl-dir.bbclass b/meta/classes/deb-dl-dir.bbclass
index e75e6be5..a8e8261b 100644
--- a/meta/classes/deb-dl-dir.bbclass
+++ b/meta/classes/deb-dl-dir.bbclass
@@ -5,25 +5,6 @@
 
 inherit repository
 
-debsrc_do_mounts() {
-    sudo -s <<EOSUDO
-    set -e
-    mkdir -p "${1}/deb-src"
-    mountpoint -q "${1}/deb-src" || \
-    mount -o bind,private "${DEBSRCDIR}" "${1}/deb-src"
-EOSUDO
-}
-
-debsrc_undo_mounts() {
-    sudo -s <<EOSUDO
-    set -e
-    mkdir -p "${1}/deb-src"
-    mountpoint -q "${1}/deb-src" && \
-    umount "${1}/deb-src"
-    rm -rf "${1}/deb-src"
-EOSUDO
-}
-
 debsrc_source_version_filter() {
     # Filter the input to only consider Package, Version and Source lines
     #
@@ -51,11 +32,6 @@ debsrc_download() {
     export rootfs_distro="$2"
     mkdir -p "${DEBSRCDIR}"/"${rootfs_distro}"
 
-    debsrc_do_mounts "${rootfs}"
-
-    trap 'exit 1' INT HUP QUIT TERM ALRM USR1
-    trap 'debsrc_undo_mounts "${rootfs}"' EXIT
-
     ( flock 9
     set -e
     printenv | grep -q BB_VERBOSE_LOGS && set -x
@@ -90,13 +66,17 @@ debsrc_download() {
         dscname="${src}_${version#*:}.dsc"
         [ -f "${DEBSRCDIR}"/"${rootfs_distro}"/"${src}"/"${dscname}" ] || {
             # use apt-get source to download sources in DEBSRCDIR
-            sudo -E chroot --userspec=$( id -u ):$( id -g ) ${rootfs} \
-                sh -c ' mkdir -p "/deb-src/${1}/${2}" && cd "/deb-src/${1}/${2}" && apt-get -y --download-only --only-source source "$2"="$3" ' download-src "${rootfs_distro}" "${src}" "${version}"
+            mkdir -p "${DEBSRCDIR}/${rootfs_distro}"/"${src}"
+            rootfs_native_cmd \
+                --bind "${DEBSRCDIR}" "/deb-src" \
+                --chdir "/deb-src/${rootfs_distro}/${src}" \
+                "${rootfs}" -- \
+                apt-get -o APT::Architecture=${DISTRO_ARCH} \
+                        -o Dir="${rootfs}" -y --download-only \
+                        --only-source source "${src}=${version}"
         }
     done
     ) 9>"${DEBSRCDIR}/${rootfs_distro}.lock"
-
-    debsrc_undo_mounts "${rootfs}"
 }
 
 dbg_pkgs_download() {
-- 
2.39.5

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20250515150727.1764989-3-cedric.hombourger%40siemens.com.

[PATCH 0/4] non-privileged commands in chroot

['Cedric Hombourger' via isar-users]

When building root file-systems for a foreign architecture and needing
to cache source packages, apt will be called from within the rootfs and
consequently be executed under QEMU: this is terribly slow especially
considering that source packages are downloaded one by one. This patch
series introduces a wrapper function to run native commands against a
rootfs with our special mounts such as /isar-apt. Some basic tests were
performed to smoke test this approach and evaluate its performance. It
should be noted that the changes introduce a new host tool dependency:
bubblewrap. Alignment with projects such as kas would be required (had
a positive discussion with Felix but no actions will be taken prior to
receiving some form of agreement for this patch series).

Changes since RFC patch:
  - Let caller decide where to bind-mount the rootfs to
  - Make the rootfs argument optional
  - Support 32-bit rootfs (no lib64 there)

Validated with "citest.py -t dev" (in a kas-container):

 (1/6) citest.py:DevTest.test_dev: STARTED
 (1/6) citest.py:DevTest.test_dev: PASS (752.07 s)
 (2/6) citest.py:DevTest.test_dev_apps: STARTED
 (2/6) citest.py:DevTest.test_dev_apps: PASS (770.95 s)
 (3/6) citest.py:DevTest.test_dev_rebuild: STARTED
 (3/6) citest.py:DevTest.test_dev_rebuild: PASS (275.02 s)
 (4/6) citest.py:DevTest.test_dev_run_amd64_bookworm: STARTED
 (4/6) citest.py:DevTest.test_dev_run_amd64_bookworm: PASS (47.87 s)
 (5/6) citest.py:DevTest.test_dev_run_arm64_bookworm: STARTED
 (5/6) citest.py:DevTest.test_dev_run_arm64_bookworm: PASS (31.20 s)
 (6/6) citest.py:DevTest.test_dev_run_arm_bookworm: STARTED
 (6/6) citest.py:DevTest.test_dev_run_arm_bookworm: PASS (32.34 s)

Cedric Hombourger (4):
  rootfs: introduce wrapper to run commands against a rootfs
  deb-dl-dir: optimize caching of source packages using apt natively
  image-postproc-extension: refactor systemd version checks
  image-postproc-extension: extract systemd's version using rootfs_cmd

 RECIPE-API-CHANGELOG.md                       |  6 ++
 doc/user_manual.md                            |  1 +
 meta/classes/deb-dl-dir.bbclass               | 37 +++--------
 meta/classes/image-postproc-extension.bbclass | 12 ++--
 meta/classes/rootfs.bbclass                   | 66 +++++++++++++++++++
 5 files changed, 89 insertions(+), 33 deletions(-)

-- 
2.39.5

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20250519115750.3195300-1-cedric.hombourger%40siemens.com.

[PATCH 1/4] rootfs: introduce wrapper to run commands against a rootfs

['Cedric Hombourger' via isar-users]

"sudo chroot" is used in several places to run commands inside rootfs
directories constructed by Isar. There are cases where a command could
be used without elevated privileges as long as special folders such as
/isar-apt are mounted (they are often referenced as /isar-apt in
configuration files found in the target rootfs). For such cases,
bubblewrap may be used to create a non-privileged namespace (either
in a bare/native environment or within a docker/podman container)
where the command will be executed as if chroot had been used. The
rootfs may also be the host root file-system: this should however
be used with care to avoid host contamination problems (note: Isar
already relies on a number of host tools).

Signed-off-by: Cedric Hombourger <cedric.hombourger@siemens.com>
---
 RECIPE-API-CHANGELOG.md     |  6 ++++
 doc/user_manual.md          |  1 +
 meta/classes/rootfs.bbclass | 66 +++++++++++++++++++++++++++++++++++++
 3 files changed, 73 insertions(+)

diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md
index a4cf1338..725737b2 100644
--- a/RECIPE-API-CHANGELOG.md
+++ b/RECIPE-API-CHANGELOG.md
@@ -722,3 +722,9 @@ Optional fields of the isar-apt repo can be controlled by adding to the
 
 Changes in next
 ---------------
+
+### Require bubblewrap to run non-privileged commands with bind-mounts
+
+Isar occasionally needs to run commands within root file-systems that it
+builds and with several bind-mounts (e.g. /isar-apt). bubblewrap may be
+used in Isar classes instead of `sudo chroot`.
diff --git a/doc/user_manual.md b/doc/user_manual.md
index 0dc317c3..3cf1a9aa 100644
--- a/doc/user_manual.md
+++ b/doc/user_manual.md
@@ -75,6 +75,7 @@ Install the following packages:
 ```
 apt install \
   binfmt-support \
+  bubblewrap \
   bzip2 \
   mmdebstrap \
   arch-test \
diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass
index 5f877962..5b96b414 100644
--- a/meta/classes/rootfs.bbclass
+++ b/meta/classes/rootfs.bbclass
@@ -34,6 +34,72 @@ export LANG = "C"
 export LANGUAGE = "C"
 export LC_ALL = "C"
 
+# Execute a command against a rootfs and with isar-apt bind-mounted.
+# Additional mounts may be specified using --bind <source> <target> and a
+# custom directory for the command to be executed with --chdir <dir>. The
+# command is assumed to follow the special "--" argument. This would replace
+# "sudo chroot" calls especially when a native command may be used instead of
+# chroot'ed command and without elevated privileges (the command will likely
+# take the rootfs as argument; e.g. apt-get -o Dir=${ROOTFSDIR}). If the
+# optional rootfs argument is omitted, the host rootfs will be used (e.g. to
+# run native commands): this should be used with care.
+#
+# Usage: rootfs_cmd [options] [rootfs] -- command
+#
+rootfs_cmd() {
+    set -- "$@"
+    bwrap_args="--bind ${REPO_ISAR_DIR}/${DISTRO} /isar-apt"
+    rootfs=""
+
+    while [ "${#}" -gt "0" ] && [ "${1}" != "--" ]; do
+        case "${1}" in
+            --bind)
+                if [ "${#}" -lt "3" ]; then
+                    bbfatal "--bind requires two arguments"
+                fi
+                bwrap_args="${bwrap_args} --bind ${2} ${3}"
+                shift 3
+                ;;
+            --chdir)
+                if [ "${#}" -lt "2" ]; then
+                    bbfatal "${1} requires an argument"
+                fi
+                bwrap_args="${bwrap_args} ${1} ${2}"
+                shift 2
+                ;;
+            -*)
+                bbfatal "${1} is not a supported option!"
+                ;;
+            *)
+                if [ -z "${rootfs}" ]; then
+                    rootfs="${1}"
+                    shift
+                else
+                    bbfatal "unexpected argument '${1}'"
+                fi
+                ;;
+        esac
+    done
+
+    if [ -n "${rootfs}" ]; then
+        bwrap_args="${bwrap_args} --bind ${rootfs} ${rootfs}"
+    fi
+
+    if [ "${#}" -le "1" ] || [ "${1}" != "--" ]; then
+        bbfatal "no command specified (missing --)"
+    fi
+    shift  # remove "--", command and its arguments follows
+
+    for ro_d in bin etc lib lib64 sys usr var; do
+        [ -d ${rootfs}/${ro_d} ] || continue
+        bwrap_args="${bwrap_args} --ro-bind ${rootfs}/${ro_d} /${ro_d}"
+    done
+
+    bwrap --unshare-user --unshare-pid ${bwrap_args} \
+        --dev-bind /dev /dev --proc /proc --tmpfs /tmp \
+        -- "${@}"
+}
+
 rootfs_do_mounts[weight] = "3"
 rootfs_do_mounts() {
     sudo -s <<'EOSUDO'
-- 
2.39.5

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20250519115750.3195300-2-cedric.hombourger%40siemens.com.

[PATCH 2/4] deb-dl-dir: optimize caching of source packages using apt natively

['Cedric Hombourger' via isar-users]

source package are downloaded by entering the target rootfs and run
apt there. For foreign architectures, this results in apt being
executed under QEMU and leads to poor performance. By using the
recently introduced rootfs_native_cmd command wrapper, apt will be
executed natively against the target rootfs and without elevated
privileges. For our test work-load, caching was reduced from more
than 10 hours to an hour. Performance is also more consistent as
it will no longer depend as to when bitbake kicks caching of
source packages for foreign architecture rootfs vs rootfs for the
host (in multiconfig builds).

Signed-off-by: Cedric Hombourger <cedric.hombourger@siemens.com>
---
 meta/classes/deb-dl-dir.bbclass | 37 ++++++++-------------------------
 1 file changed, 9 insertions(+), 28 deletions(-)

diff --git a/meta/classes/deb-dl-dir.bbclass b/meta/classes/deb-dl-dir.bbclass
index 3f560da4..7026f4f4 100644
--- a/meta/classes/deb-dl-dir.bbclass
+++ b/meta/classes/deb-dl-dir.bbclass
@@ -5,25 +5,6 @@
 
 inherit repository
 
-debsrc_do_mounts() {
-    sudo -s <<EOSUDO
-    set -e
-    mkdir -p "${1}/deb-src"
-    mountpoint -q "${1}/deb-src" || \
-    mount -o bind,private "${DEBSRCDIR}" "${1}/deb-src"
-EOSUDO
-}
-
-debsrc_undo_mounts() {
-    sudo -s <<EOSUDO
-    set -e
-    mkdir -p "${1}/deb-src"
-    mountpoint -q "${1}/deb-src" && \
-    umount "${1}/deb-src"
-    rm -rf "${1}/deb-src"
-EOSUDO
-}
-
 debsrc_source_version_filter() {
     # Filter the input to only consider Package, Version and Source lines
     #
@@ -51,11 +32,6 @@ debsrc_download() {
     export rootfs_distro="$2"
     mkdir -p "${DEBSRCDIR}"/"${rootfs_distro}"
 
-    debsrc_do_mounts "${rootfs}"
-
-    trap 'exit 1' INT HUP QUIT TERM ALRM USR1
-    trap 'debsrc_undo_mounts "${rootfs}"' EXIT
-
     ( flock 9
     set -e
     printenv | grep -q BB_VERBOSE_LOGS && set -x
@@ -89,13 +65,18 @@ debsrc_download() {
         dscname="${src}_${version#*:}.dsc"
         [ -f "${DEBSRCDIR}"/"${rootfs_distro}"/"${src}"/"${dscname}" ] || {
             # use apt-get source to download sources in DEBSRCDIR
-            sudo -E chroot --userspec=$( id -u ):$( id -g ) ${rootfs} \
-                sh -c ' mkdir -p "/deb-src/${1}/${2}" && cd "/deb-src/${1}/${2}" && apt-get -y --download-only --only-source source "$2"="$3" ' download-src "${rootfs_distro}" "${src}" "${version}"
+            mkdir -p "${DEBSRCDIR}/${rootfs_distro}"/"${src}"
+            rootfs_cmd \
+                --bind "${DEBSRCDIR}" "/deb-src" \
+                --bind "${rootfs}" "${rootfs}" \
+                --chdir "/deb-src/${rootfs_distro}/${src}" \
+                -- \
+                apt-get -o APT::Architecture=${DISTRO_ARCH} \
+                        -o Dir="${rootfs}" -y --download-only \
+                        --only-source source "${src}=${version}"
         }
     done
     ) 9>"${DEBSRCDIR}/${rootfs_distro}.lock"
-
-    debsrc_undo_mounts "${rootfs}"
 }
 
 dbg_pkgs_download() {
-- 
2.39.5

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20250519115750.3195300-3-cedric.hombourger%40siemens.com.

[PATCH 3/4] image-postproc-extension: refactor systemd version checks

['Cedric Hombourger' via isar-users]

Signed-off-by: Cedric Hombourger <cedric.hombourger@siemens.com>
---
 meta/classes/image-postproc-extension.bbclass | 12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)

diff --git a/meta/classes/image-postproc-extension.bbclass b/meta/classes/image-postproc-extension.bbclass
index 991bac4c..0af588d8 100644
--- a/meta/classes/image-postproc-extension.bbclass
+++ b/meta/classes/image-postproc-extension.bbclass
@@ -53,12 +53,17 @@ image_postprocess_mark() {
         --build-id "${BUILD_ID}" --variant "${DESCRIPTION}" --version "${PV}"
 }
 
+# Use dpkg to find out which version of systemd is installed into the image or reports "0"
+image_systemd_version() {
+    sudo chroot ${IMAGE_ROOTFS} dpkg-query --showformat='${source:Upstream-Version}' --show systemd || echo "0"
+}
+
 ROOTFS_POSTPROCESS_COMMAND =+ "image_postprocess_machine_id"
 image_postprocess_machine_id() {
     # systemd(1) takes care of recreating the machine-id on first boot
     # for systemd < v247, set to empty string, else set to uninitialized
     # (required if initramfs with ro root is used)
-    SYSTEMD_VERSION=$( sudo chroot ${IMAGE_ROOTFS} dpkg-query --showformat='${source:Upstream-Version}' --show systemd || echo "0" )
+    SYSTEMD_VERSION=$( image_systemd_version )
     MACHINE_ID="uninitialized"
     if dpkg --compare-versions "$SYSTEMD_VERSION" "lt" "247"; then
         MACHINE_ID=""
@@ -82,10 +87,7 @@ image_postprocess_sshd_key_regen() {
 
 ROOTFS_POSTPROCESS_COMMAND =+ "image_posprocess_disable_systemd_firstboot"
 image_posprocess_disable_systemd_firstboot() {
-    SYSTEMD_VERSION=$(sudo chroot '${ROOTFSDIR}' dpkg-query \
-        --showformat='${source:Upstream-Version}' \
-        --show systemd || echo "0" )
-
+    SYSTEMD_VERSION=$( image_systemd_version )
     if dpkg --compare-versions "$SYSTEMD_VERSION" "ge" "251"; then
         sudo chroot '${ROOTFSDIR}' systemctl mask systemd-firstboot
         if ! cmd_output=$(sudo chroot '${ROOTFSDIR}' systemd-firstboot \
-- 
2.39.5

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20250519115750.3195300-4-cedric.hombourger%40siemens.com.

[PATCH 4/4] image-postproc-extension: extract systemd's version using rootfs_cmd

['Cedric Hombourger' via isar-users]

Elevated privileges are not required to query the rootfs for the version
of systemd: replace "sudo chroot" with "rootfs_cmd"

Signed-off-by: Cedric Hombourger <cedric.hombourger@siemens.com>
---
 meta/classes/image-postproc-extension.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/image-postproc-extension.bbclass b/meta/classes/image-postproc-extension.bbclass
index 0af588d8..21dcfccc 100644
--- a/meta/classes/image-postproc-extension.bbclass
+++ b/meta/classes/image-postproc-extension.bbclass
@@ -55,7 +55,7 @@ image_postprocess_mark() {
 
 # Use dpkg to find out which version of systemd is installed into the image or reports "0"
 image_systemd_version() {
-    sudo chroot ${IMAGE_ROOTFS} dpkg-query --showformat='${source:Upstream-Version}' --show systemd || echo "0"
+    rootfs_cmd ${IMAGE_ROOTFS} -- dpkg-query --showformat='${source:Upstream-Version}' --show systemd || echo "0"
 }
 
 ROOTFS_POSTPROCESS_COMMAND =+ "image_postprocess_machine_id"
-- 
2.39.5

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20250519115750.3195300-5-cedric.hombourger%40siemens.com.

Re: [PATCH 0/4] non-privileged commands in chroot

[Srinuvasan Arjunan]

[-- Attachment #1: Type: text/plain, Size: 3416 bytes --]

On Mon, May 19, 2025 at 5:28 PM 'Cedric Hombourger' via isar-users <
isar-users@googlegroups.com> wrote:

> When building root file-systems for a foreign architecture and needing
> to cache source packages, apt will be called from within the rootfs and
> consequently be executed under QEMU: this is terribly slow especially
> considering that source packages are downloaded one by one. This patch
> series introduces a wrapper function to run native commands against a
> rootfs with our special mounts such as /isar-apt. Some basic tests were
> performed to smoke test this approach and evaluate its performance. It
> should be noted that the changes introduce a new host tool dependency:
> bubblewrap. Alignment with projects such as kas would be required (had
> a positive discussion with Felix but no actions will be taken prior to
> receiving some form of agreement for this patch series).
>
> Changes since RFC patch:
>   - Let caller decide where to bind-mount the rootfs to
>   - Make the rootfs argument optional
>   - Support 32-bit rootfs (no lib64 there)
>
> Validated with "citest.py -t dev" (in a kas-container):
>
>  (1/6) citest.py:DevTest.test_dev: STARTED
>  (1/6) citest.py:DevTest.test_dev: PASS (752.07 s)
>  (2/6) citest.py:DevTest.test_dev_apps: STARTED
>  (2/6) citest.py:DevTest.test_dev_apps: PASS (770.95 s)
>  (3/6) citest.py:DevTest.test_dev_rebuild: STARTED
>  (3/6) citest.py:DevTest.test_dev_rebuild: PASS (275.02 s)
>  (4/6) citest.py:DevTest.test_dev_run_amd64_bookworm: STARTED
>  (4/6) citest.py:DevTest.test_dev_run_amd64_bookworm: PASS (47.87 s)
>  (5/6) citest.py:DevTest.test_dev_run_arm64_bookworm: STARTED
>  (5/6) citest.py:DevTest.test_dev_run_arm64_bookworm: PASS (31.20 s)
>  (6/6) citest.py:DevTest.test_dev_run_arm_bookworm: STARTED
>  (6/6) citest.py:DevTest.test_dev_run_arm_bookworm: PASS (32.34 s)
>
> Cedric Hombourger (4):
>   rootfs: introduce wrapper to run commands against a rootfs
>   deb-dl-dir: optimize caching of source packages using apt natively
>   image-postproc-extension: refactor systemd version checks
>   image-postproc-extension: extract systemd's version using rootfs_cmd
>
>  RECIPE-API-CHANGELOG.md                       |  6 ++
>  doc/user_manual.md                            |  1 +
>  meta/classes/deb-dl-dir.bbclass               | 37 +++--------
>  meta/classes/image-postproc-extension.bbclass | 12 ++--
>  meta/classes/rootfs.bbclass                   | 66 +++++++++++++++++++
>  5 files changed, 89 insertions(+), 33 deletions(-)
>


 Tested-by: Srinuvasan Arjunan <srinuvasan.a@siemens.com>

>
> --
> 2.39.5
>
> --
> You received this message because you are subscribed to the Google Groups
> "isar-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to isar-users+unsubscribe@googlegroups.com.
> To view this discussion visit
> https://groups.google.com/d/msgid/isar-users/20250519115750.3195300-1-cedric.hombourger%40siemens.com
> .
>

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/CAB2Z2nPp5uOcv1m%3DOw1%3DLyJ2XwiGcwZkujeWDkP%2BZO3EQk%3DR-g%40mail.gmail.com.

[-- Attachment #2: Type: text/html, Size: 4789 bytes --]

Re: [PATCH 1/4] rootfs: introduce wrapper to run commands against a rootfs

['MOESSBAUER, Felix' via isar-users]

On Mon, 2025-05-19 at 13:57 +0200, Cedric Hombourger wrote:
> "sudo chroot" is used in several places to run commands inside rootfs
> directories constructed by Isar. There are cases where a command
> could
> be used without elevated privileges as long as special folders such
> as
> /isar-apt are mounted (they are often referenced as /isar-apt in
> configuration files found in the target rootfs). For such cases,
> bubblewrap may be used to create a non-privileged namespace (either
> in a bare/native environment or within a docker/podman container)
> where the command will be executed as if chroot had been used. The
> rootfs may also be the host root file-system: this should however
> be used with care to avoid host contamination problems (note: Isar
> already relies on a number of host tools).

Hi, this looks promising. I gave it a try on some of our internal
layers (arm64) in a custom kas container under podman.

I'm wondering if this could also be used to run the apt in
do_rootfs_install natively (maybe in combination with dpkg --root).

Tested-by: Felix Moessbauer <felix.moessbauer@siemens.com>

Felix

> 
> Signed-off-by: Cedric Hombourger <cedric.hombourger@siemens.com>
> ---
>  RECIPE-API-CHANGELOG.md     |  6 ++++
>  doc/user_manual.md          |  1 +
>  meta/classes/rootfs.bbclass | 66
> +++++++++++++++++++++++++++++++++++++
>  3 files changed, 73 insertions(+)
> 
> diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md
> index a4cf1338..725737b2 100644
> --- a/RECIPE-API-CHANGELOG.md
> +++ b/RECIPE-API-CHANGELOG.md
> @@ -722,3 +722,9 @@ Optional fields of the isar-apt repo can be
> controlled by adding to the
>  
>  Changes in next
>  ---------------
> +
> +### Require bubblewrap to run non-privileged commands with bind-
> mounts
> +
> +Isar occasionally needs to run commands within root file-systems
> that it
> +builds and with several bind-mounts (e.g. /isar-apt). bubblewrap may
> be
> +used in Isar classes instead of `sudo chroot`.
> diff --git a/doc/user_manual.md b/doc/user_manual.md
> index 0dc317c3..3cf1a9aa 100644
> --- a/doc/user_manual.md
> +++ b/doc/user_manual.md
> @@ -75,6 +75,7 @@ Install the following packages:
>  ```
>  apt install \
>    binfmt-support \
> +  bubblewrap \
>    bzip2 \
>    mmdebstrap \
>    arch-test \
> diff --git a/meta/classes/rootfs.bbclass
> b/meta/classes/rootfs.bbclass
> index 5f877962..5b96b414 100644
> --- a/meta/classes/rootfs.bbclass
> +++ b/meta/classes/rootfs.bbclass
> @@ -34,6 +34,72 @@ export LANG = "C"
>  export LANGUAGE = "C"
>  export LC_ALL = "C"
>  
> +# Execute a command against a rootfs and with isar-apt bind-mounted.
> +# Additional mounts may be specified using --bind <source> <target>
> and a
> +# custom directory for the command to be executed with --chdir
> <dir>. The
> +# command is assumed to follow the special "--" argument. This would
> replace
> +# "sudo chroot" calls especially when a native command may be used
> instead of
> +# chroot'ed command and without elevated privileges (the command
> will likely
> +# take the rootfs as argument; e.g. apt-get -o Dir=${ROOTFSDIR}). If
> the
> +# optional rootfs argument is omitted, the host rootfs will be used
> (e.g. to
> +# run native commands): this should be used with care.
> +#
> +# Usage: rootfs_cmd [options] [rootfs] -- command
> +#
> +rootfs_cmd() {
> +    set -- "$@"
> +    bwrap_args="--bind ${REPO_ISAR_DIR}/${DISTRO} /isar-apt"
> +    rootfs=""
> +
> +    while [ "${#}" -gt "0" ] && [ "${1}" != "--" ]; do
> +        case "${1}" in
> +            --bind)
> +                if [ "${#}" -lt "3" ]; then
> +                    bbfatal "--bind requires two arguments"
> +                fi
> +                bwrap_args="${bwrap_args} --bind ${2} ${3}"
> +                shift 3
> +                ;;
> +            --chdir)
> +                if [ "${#}" -lt "2" ]; then
> +                    bbfatal "${1} requires an argument"
> +                fi
> +                bwrap_args="${bwrap_args} ${1} ${2}"
> +                shift 2
> +                ;;
> +            -*)
> +                bbfatal "${1} is not a supported option!"
> +                ;;
> +            *)
> +                if [ -z "${rootfs}" ]; then
> +                    rootfs="${1}"
> +                    shift
> +                else
> +                    bbfatal "unexpected argument '${1}'"
> +                fi
> +                ;;
> +        esac
> +    done
> +
> +    if [ -n "${rootfs}" ]; then
> +        bwrap_args="${bwrap_args} --bind ${rootfs} ${rootfs}"
> +    fi
> +
> +    if [ "${#}" -le "1" ] || [ "${1}" != "--" ]; then
> +        bbfatal "no command specified (missing --)"
> +    fi
> +    shift  # remove "--", command and its arguments follows
> +
> +    for ro_d in bin etc lib lib64 sys usr var; do
> +        [ -d ${rootfs}/${ro_d} ] || continue
> +        bwrap_args="${bwrap_args} --ro-bind ${rootfs}/${ro_d}
> /${ro_d}"
> +    done
> +
> +    bwrap --unshare-user --unshare-pid ${bwrap_args} \
> +        --dev-bind /dev /dev --proc /proc --tmpfs /tmp \
> +        -- "${@}"
> +}
> +
>  rootfs_do_mounts[weight] = "3"
>  rootfs_do_mounts() {
>      sudo -s <<'EOSUDO'

-- 
Siemens AG
Linux Expert Center
Friedrich-Ludwig-Bauer-Str. 3
85748 Garching, Germany

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/27cd63ffc8d2ae1c7ad97367df6e1327993f4d1b.camel%40siemens.com.

Re: [PATCH 1/4] rootfs: introduce wrapper to run commands against a rootfs

['cedric.hombourger@siemens.com' via isar-users]

On Thu, 2025-05-22 at 14:32 +0000, Moessbauer, Felix (FT RPD CED OES-
DE) wrote:
> On Mon, 2025-05-19 at 13:57 +0200, Cedric Hombourger wrote:
> > "sudo chroot" is used in several places to run commands inside
> > rootfs
> > directories constructed by Isar. There are cases where a command
> > could
> > be used without elevated privileges as long as special folders such
> > as
> > /isar-apt are mounted (they are often referenced as /isar-apt in
> > configuration files found in the target rootfs). For such cases,
> > bubblewrap may be used to create a non-privileged namespace (either
> > in a bare/native environment or within a docker/podman container)
> > where the command will be executed as if chroot had been used. The
> > rootfs may also be the host root file-system: this should however
> > be used with care to avoid host contamination problems (note: Isar
> > already relies on a number of host tools).
> 
> Hi, this looks promising. I gave it a try on some of our internal
> layers (arm64) in a custom kas container under podman.
> 
> I'm wondering if this could also be used to run the apt in
> do_rootfs_install natively (maybe in combination with dpkg --root).
> 
> Tested-by: Felix Moessbauer <felix.moessbauer@siemens.com>

Dear maintainers, can we move forward with these changes or are there
any concerns that need to be addressed?

Thank you!

> 
> Felix
> 
> > 
> > Signed-off-by: Cedric Hombourger <cedric.hombourger@siemens.com>
> > ---
> >  RECIPE-API-CHANGELOG.md     |  6 ++++
> >  doc/user_manual.md          |  1 +
> >  meta/classes/rootfs.bbclass | 66
> > +++++++++++++++++++++++++++++++++++++
> >  3 files changed, 73 insertions(+)
> > 
> > diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md
> > index a4cf1338..725737b2 100644
> > --- a/RECIPE-API-CHANGELOG.md
> > +++ b/RECIPE-API-CHANGELOG.md
> > @@ -722,3 +722,9 @@ Optional fields of the isar-apt repo can be
> > controlled by adding to the
> >  
> >  Changes in next
> >  ---------------
> > +
> > +### Require bubblewrap to run non-privileged commands with bind-
> > mounts
> > +
> > +Isar occasionally needs to run commands within root file-systems
> > that it
> > +builds and with several bind-mounts (e.g. /isar-apt). bubblewrap
> > may
> > be
> > +used in Isar classes instead of `sudo chroot`.
> > diff --git a/doc/user_manual.md b/doc/user_manual.md
> > index 0dc317c3..3cf1a9aa 100644
> > --- a/doc/user_manual.md
> > +++ b/doc/user_manual.md
> > @@ -75,6 +75,7 @@ Install the following packages:
> >  ```
> >  apt install \
> >    binfmt-support \
> > +  bubblewrap \
> >    bzip2 \
> >    mmdebstrap \
> >    arch-test \
> > diff --git a/meta/classes/rootfs.bbclass
> > b/meta/classes/rootfs.bbclass
> > index 5f877962..5b96b414 100644
> > --- a/meta/classes/rootfs.bbclass
> > +++ b/meta/classes/rootfs.bbclass
> > @@ -34,6 +34,72 @@ export LANG = "C"
> >  export LANGUAGE = "C"
> >  export LC_ALL = "C"
> >  
> > +# Execute a command against a rootfs and with isar-apt bind-
> > mounted.
> > +# Additional mounts may be specified using --bind <source>
> > <target>
> > and a
> > +# custom directory for the command to be executed with --chdir
> > <dir>. The
> > +# command is assumed to follow the special "--" argument. This
> > would
> > replace
> > +# "sudo chroot" calls especially when a native command may be used
> > instead of
> > +# chroot'ed command and without elevated privileges (the command
> > will likely
> > +# take the rootfs as argument; e.g. apt-get -o Dir=${ROOTFSDIR}).
> > If
> > the
> > +# optional rootfs argument is omitted, the host rootfs will be
> > used
> > (e.g. to
> > +# run native commands): this should be used with care.
> > +#
> > +# Usage: rootfs_cmd [options] [rootfs] -- command
> > +#
> > +rootfs_cmd() {
> > +    set -- "$@"
> > +    bwrap_args="--bind ${REPO_ISAR_DIR}/${DISTRO} /isar-apt"
> > +    rootfs=""
> > +
> > +    while [ "${#}" -gt "0" ] && [ "${1}" != "--" ]; do
> > +        case "${1}" in
> > +            --bind)
> > +                if [ "${#}" -lt "3" ]; then
> > +                    bbfatal "--bind requires two arguments"
> > +                fi
> > +                bwrap_args="${bwrap_args} --bind ${2} ${3}"
> > +                shift 3
> > +                ;;
> > +            --chdir)
> > +                if [ "${#}" -lt "2" ]; then
> > +                    bbfatal "${1} requires an argument"
> > +                fi
> > +                bwrap_args="${bwrap_args} ${1} ${2}"
> > +                shift 2
> > +                ;;
> > +            -*)
> > +                bbfatal "${1} is not a supported option!"
> > +                ;;
> > +            *)
> > +                if [ -z "${rootfs}" ]; then
> > +                    rootfs="${1}"
> > +                    shift
> > +                else
> > +                    bbfatal "unexpected argument '${1}'"
> > +                fi
> > +                ;;
> > +        esac
> > +    done
> > +
> > +    if [ -n "${rootfs}" ]; then
> > +        bwrap_args="${bwrap_args} --bind ${rootfs} ${rootfs}"
> > +    fi
> > +
> > +    if [ "${#}" -le "1" ] || [ "${1}" != "--" ]; then
> > +        bbfatal "no command specified (missing --)"
> > +    fi
> > +    shift  # remove "--", command and its arguments follows
> > +
> > +    for ro_d in bin etc lib lib64 sys usr var; do
> > +        [ -d ${rootfs}/${ro_d} ] || continue
> > +        bwrap_args="${bwrap_args} --ro-bind ${rootfs}/${ro_d}
> > /${ro_d}"
> > +    done
> > +
> > +    bwrap --unshare-user --unshare-pid ${bwrap_args} \
> > +        --dev-bind /dev /dev --proc /proc --tmpfs /tmp \
> > +        -- "${@}"
> > +}
> > +
> >  rootfs_do_mounts[weight] = "3"
> >  rootfs_do_mounts() {
> >      sudo -s <<'EOSUDO'
> 

-- 
Cedric Hombourger
Siemens AG
www.siemens.com

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/b38288e179d92bca6d70957fba9b441145dcb76b.camel%40siemens.com.

Re: [PATCH 1/4] rootfs: introduce wrapper to run commands against a rootfs

['MOESSBAUER, Felix' via isar-users]

On Thu, 2025-06-05 at 06:42 +0000, Hombourger, Cedric (FT FDS CES LX)
wrote:
> On Thu, 2025-05-22 at 14:32 +0000, Moessbauer, Felix (FT RPD CED OES-
> DE) wrote:
> > On Mon, 2025-05-19 at 13:57 +0200, Cedric Hombourger wrote:
> > > "sudo chroot" is used in several places to run commands inside
> > > rootfs
> > > directories constructed by Isar. There are cases where a command
> > > could
> > > be used without elevated privileges as long as special folders
> > > such
> > > as
> > > /isar-apt are mounted (they are often referenced as /isar-apt in
> > > configuration files found in the target rootfs). For such cases,
> > > bubblewrap may be used to create a non-privileged namespace
> > > (either
> > > in a bare/native environment or within a docker/podman container)
> > > where the command will be executed as if chroot had been used.
> > > The
> > > rootfs may also be the host root file-system: this should however
> > > be used with care to avoid host contamination problems (note:
> > > Isar
> > > already relies on a number of host tools).
> > 
> > Hi, this looks promising. I gave it a try on some of our internal
> > layers (arm64) in a custom kas container under podman.
> > 
> > I'm wondering if this could also be used to run the apt in
> > do_rootfs_install natively (maybe in combination with dpkg --root).
> > 
> > Tested-by: Felix Moessbauer <felix.moessbauer@siemens.com>
> 
> Dear maintainers, can we move forward with these changes or are there
> any concerns that need to be addressed?

If we decide to integrate this (which I vote for!), we should also add
the bubblewrap package to the upcoming kas 4.8 release (putting Jan in
CC).

Felix

> 
> Thank you!
> 
> > 
> > Felix
> > 
> > > 
> > > Signed-off-by: Cedric Hombourger <cedric.hombourger@siemens.com>
> > > ---
> > >  RECIPE-API-CHANGELOG.md     |  6 ++++
> > >  doc/user_manual.md          |  1 +
> > >  meta/classes/rootfs.bbclass | 66
> > > +++++++++++++++++++++++++++++++++++++
> > >  3 files changed, 73 insertions(+)
> > > 
> > > diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md
> > > index a4cf1338..725737b2 100644
> > > --- a/RECIPE-API-CHANGELOG.md
> > > +++ b/RECIPE-API-CHANGELOG.md
> > > @@ -722,3 +722,9 @@ Optional fields of the isar-apt repo can be
> > > controlled by adding to the
> > >  
> > >  Changes in next
> > >  ---------------
> > > +
> > > +### Require bubblewrap to run non-privileged commands with bind-
> > > mounts
> > > +
> > > +Isar occasionally needs to run commands within root file-systems
> > > that it
> > > +builds and with several bind-mounts (e.g. /isar-apt). bubblewrap
> > > may
> > > be
> > > +used in Isar classes instead of `sudo chroot`.
> > > diff --git a/doc/user_manual.md b/doc/user_manual.md
> > > index 0dc317c3..3cf1a9aa 100644
> > > --- a/doc/user_manual.md
> > > +++ b/doc/user_manual.md
> > > @@ -75,6 +75,7 @@ Install the following packages:
> > >  ```
> > >  apt install \
> > >    binfmt-support \
> > > +  bubblewrap \
> > >    bzip2 \
> > >    mmdebstrap \
> > >    arch-test \
> > > diff --git a/meta/classes/rootfs.bbclass
> > > b/meta/classes/rootfs.bbclass
> > > index 5f877962..5b96b414 100644
> > > --- a/meta/classes/rootfs.bbclass
> > > +++ b/meta/classes/rootfs.bbclass
> > > @@ -34,6 +34,72 @@ export LANG = "C"
> > >  export LANGUAGE = "C"
> > >  export LC_ALL = "C"
> > >  
> > > +# Execute a command against a rootfs and with isar-apt bind-
> > > mounted.
> > > +# Additional mounts may be specified using --bind <source>
> > > <target>
> > > and a
> > > +# custom directory for the command to be executed with --chdir
> > > <dir>. The
> > > +# command is assumed to follow the special "--" argument. This
> > > would
> > > replace
> > > +# "sudo chroot" calls especially when a native command may be
> > > used
> > > instead of
> > > +# chroot'ed command and without elevated privileges (the command
> > > will likely
> > > +# take the rootfs as argument; e.g. apt-get -o
> > > Dir=${ROOTFSDIR}).
> > > If
> > > the
> > > +# optional rootfs argument is omitted, the host rootfs will be
> > > used
> > > (e.g. to
> > > +# run native commands): this should be used with care.
> > > +#
> > > +# Usage: rootfs_cmd [options] [rootfs] -- command
> > > +#
> > > +rootfs_cmd() {
> > > +    set -- "$@"
> > > +    bwrap_args="--bind ${REPO_ISAR_DIR}/${DISTRO} /isar-apt"
> > > +    rootfs=""
> > > +
> > > +    while [ "${#}" -gt "0" ] && [ "${1}" != "--" ]; do
> > > +        case "${1}" in
> > > +            --bind)
> > > +                if [ "${#}" -lt "3" ]; then
> > > +                    bbfatal "--bind requires two arguments"
> > > +                fi
> > > +                bwrap_args="${bwrap_args} --bind ${2} ${3}"
> > > +                shift 3
> > > +                ;;
> > > +            --chdir)
> > > +                if [ "${#}" -lt "2" ]; then
> > > +                    bbfatal "${1} requires an argument"
> > > +                fi
> > > +                bwrap_args="${bwrap_args} ${1} ${2}"
> > > +                shift 2
> > > +                ;;
> > > +            -*)
> > > +                bbfatal "${1} is not a supported option!"
> > > +                ;;
> > > +            *)
> > > +                if [ -z "${rootfs}" ]; then
> > > +                    rootfs="${1}"
> > > +                    shift
> > > +                else
> > > +                    bbfatal "unexpected argument '${1}'"
> > > +                fi
> > > +                ;;
> > > +        esac
> > > +    done
> > > +
> > > +    if [ -n "${rootfs}" ]; then
> > > +        bwrap_args="${bwrap_args} --bind ${rootfs} ${rootfs}"
> > > +    fi
> > > +
> > > +    if [ "${#}" -le "1" ] || [ "${1}" != "--" ]; then
> > > +        bbfatal "no command specified (missing --)"
> > > +    fi
> > > +    shift  # remove "--", command and its arguments follows
> > > +
> > > +    for ro_d in bin etc lib lib64 sys usr var; do
> > > +        [ -d ${rootfs}/${ro_d} ] || continue
> > > +        bwrap_args="${bwrap_args} --ro-bind ${rootfs}/${ro_d}
> > > /${ro_d}"
> > > +    done
> > > +
> > > +    bwrap --unshare-user --unshare-pid ${bwrap_args} \
> > > +        --dev-bind /dev /dev --proc /proc --tmpfs /tmp \
> > > +        -- "${@}"
> > > +}
> > > +
> > >  rootfs_do_mounts[weight] = "3"
> > >  rootfs_do_mounts() {
> > >      sudo -s <<'EOSUDO'
> > 
> 
> -- 
> Cedric Hombourger
> Siemens AG
> www.siemens.com

-- 
Siemens AG
Linux Expert Center
Friedrich-Ludwig-Bauer-Str. 3
85748 Garching, Germany

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/c23de399099b651ad8d6fbb264cbf69564c640f4.camel%40siemens.com.

Re: [PATCH 1/4] rootfs: introduce wrapper to run commands against a rootfs

[Baurzhan Ismagulov]

On 2025-06-05 12:20, 'MOESSBAUER, Felix' via isar-users wrote:
> If we decide to integrate this (which I vote for!), we should also add
> the bubblewrap package to the upcoming kas 4.8 release (putting Jan in
> CC).

Yes, that was also my question. We'll check the patches once more and provide
feedback. @Felix, would you then like to update kas first?

This would need to be touched when we'll continue working on sudo removal.
@Cedric, could this be meaningfully tested in a testcase?

With kind regards,
Baurzhan

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/aEGRBG4qoT1-u-uz%40abai.de.

Re: [PATCH 1/4] rootfs: introduce wrapper to run commands against a rootfs

['Jan Kiszka' via isar-users]

On 19.05.25 13:57, 'Cedric Hombourger' via isar-users wrote:
> "sudo chroot" is used in several places to run commands inside rootfs
> directories constructed by Isar. There are cases where a command could
> be used without elevated privileges as long as special folders such as
> /isar-apt are mounted (they are often referenced as /isar-apt in
> configuration files found in the target rootfs). For such cases,
> bubblewrap may be used to create a non-privileged namespace (either
> in a bare/native environment or within a docker/podman container)
> where the command will be executed as if chroot had been used. The
> rootfs may also be the host root file-system: this should however
> be used with care to avoid host contamination problems (note: Isar
> already relies on a number of host tools).
> 
> Signed-off-by: Cedric Hombourger <cedric.hombourger@siemens.com>
> ---
>  RECIPE-API-CHANGELOG.md     |  6 ++++
>  doc/user_manual.md          |  1 +
>  meta/classes/rootfs.bbclass | 66 +++++++++++++++++++++++++++++++++++++
>  3 files changed, 73 insertions(+)
> 
> diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md
> index a4cf1338..725737b2 100644
> --- a/RECIPE-API-CHANGELOG.md
> +++ b/RECIPE-API-CHANGELOG.md
> @@ -722,3 +722,9 @@ Optional fields of the isar-apt repo can be controlled by adding to the
>  
>  Changes in next
>  ---------------
> +
> +### Require bubblewrap to run non-privileged commands with bind-mounts
> +
> +Isar occasionally needs to run commands within root file-systems that it
> +builds and with several bind-mounts (e.g. /isar-apt). bubblewrap may be
> +used in Isar classes instead of `sudo chroot`.
> diff --git a/doc/user_manual.md b/doc/user_manual.md
> index 0dc317c3..3cf1a9aa 100644
> --- a/doc/user_manual.md
> +++ b/doc/user_manual.md
> @@ -75,6 +75,7 @@ Install the following packages:
>  ```
>  apt install \
>    binfmt-support \
> +  bubblewrap \

Does the bubblewrap (and kernel features) of bullseye suffice here, or
is that a bookworm+ thing? How about buster (still listed as host)?

Jan

-- 
Siemens AG, Foundational Technologies
Linux Expert Center

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/8ddc4d94-cf29-4f7a-8f90-12901ec4f25d%40siemens.com.

Re: [PATCH 1/4] rootfs: introduce wrapper to run commands against a rootfs

['cedric.hombourger@siemens.com' via isar-users]

On Thu, 2025-06-05 at 15:57 +0200, Jan Kiszka wrote:
> On 19.05.25 13:57, 'Cedric Hombourger' via isar-users wrote:
> > "sudo chroot" is used in several places to run commands inside
> > rootfs
> > directories constructed by Isar. There are cases where a command
> > could
> > be used without elevated privileges as long as special folders such
> > as
> > /isar-apt are mounted (they are often referenced as /isar-apt in
> > configuration files found in the target rootfs). For such cases,
> > bubblewrap may be used to create a non-privileged namespace (either
> > in a bare/native environment or within a docker/podman container)
> > where the command will be executed as if chroot had been used. The
> > rootfs may also be the host root file-system: this should however
> > be used with care to avoid host contamination problems (note: Isar
> > already relies on a number of host tools).
> > 
> > Signed-off-by: Cedric Hombourger <cedric.hombourger@siemens.com>
> > ---
> >  RECIPE-API-CHANGELOG.md     |  6 ++++
> >  doc/user_manual.md          |  1 +
> >  meta/classes/rootfs.bbclass | 66
> > +++++++++++++++++++++++++++++++++++++
> >  3 files changed, 73 insertions(+)
> > 
> > diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md
> > index a4cf1338..725737b2 100644
> > --- a/RECIPE-API-CHANGELOG.md
> > +++ b/RECIPE-API-CHANGELOG.md
> > @@ -722,3 +722,9 @@ Optional fields of the isar-apt repo can be
> > controlled by adding to the
> >  
> >  Changes in next
> >  ---------------
> > +
> > +### Require bubblewrap to run non-privileged commands with bind-
> > mounts
> > +
> > +Isar occasionally needs to run commands within root file-systems
> > that it
> > +builds and with several bind-mounts (e.g. /isar-apt). bubblewrap
> > may be
> > +used in Isar classes instead of `sudo chroot`.
> > diff --git a/doc/user_manual.md b/doc/user_manual.md
> > index 0dc317c3..3cf1a9aa 100644
> > --- a/doc/user_manual.md
> > +++ b/doc/user_manual.md
> > @@ -75,6 +75,7 @@ Install the following packages:
> >  ```
> >  apt install \
> >    binfmt-support \
> > +  bubblewrap \
> 
> Does the bubblewrap (and kernel features) of bullseye suffice here,
> or
> is that a bookworm+ thing? How about buster (still listed as host)?

bubblewrap has been around for ages: these older distros did support
flatpak. buster included.

https://packages.debian.org/buster/bubblewrap
 
> 
> Jan
> 

-- 
Cedric Hombourger
Siemens AG
www.siemens.com

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/9d83b61d0b9e3f7434487becd4dbe12aee88e12a.camel%40siemens.com.

Re: [PATCH 1/4] rootfs: introduce wrapper to run commands against a rootfs

['cedric.hombourger@siemens.com' via isar-users]

On Thu, 2025-06-05 at 14:43 +0200, Baurzhan Ismagulov wrote:
> On 2025-06-05 12:20, 'MOESSBAUER, Felix' via isar-users wrote:
> > If we decide to integrate this (which I vote for!), we should also
> > add
> > the bubblewrap package to the upcoming kas 4.8 release (putting Jan
> > in
> > CC).
> 
> Yes, that was also my question. We'll check the patches once more and
> provide
> feedback. @Felix, would you then like to update kas first?
> 
> This would need to be touched when we'll continue working on sudo
> removal.
> @Cedric, could this be meaningfully tested in a testcase?

this is tested via caching of Debian source packages but also via the
systemd version check. Are you seeking explicit tests for this new
internal API? I am asking as I was under the impression that our tests
focus on blackbox tests and not so much whitebox tests

> 
> With kind regards,
> Baurzhan

-- 
Cedric Hombourger
Siemens AG
www.siemens.com

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/7326ece3d9ecb1ed1443d5030adca16b2a47b2ed.camel%40siemens.com.

Re: [PATCH 1/4] rootfs: introduce wrapper to run commands against a rootfs

['Jan Kiszka' via isar-users]

On 06.06.25 08:02, Hombourger, Cedric (FT FDS CES LX) wrote:
> On Thu, 2025-06-05 at 15:57 +0200, Jan Kiszka wrote:
>> On 19.05.25 13:57, 'Cedric Hombourger' via isar-users wrote:
>>> "sudo chroot" is used in several places to run commands inside
>>> rootfs
>>> directories constructed by Isar. There are cases where a command
>>> could
>>> be used without elevated privileges as long as special folders such
>>> as
>>> /isar-apt are mounted (they are often referenced as /isar-apt in
>>> configuration files found in the target rootfs). For such cases,
>>> bubblewrap may be used to create a non-privileged namespace (either
>>> in a bare/native environment or within a docker/podman container)
>>> where the command will be executed as if chroot had been used. The
>>> rootfs may also be the host root file-system: this should however
>>> be used with care to avoid host contamination problems (note: Isar
>>> already relies on a number of host tools).
>>>
>>> Signed-off-by: Cedric Hombourger <cedric.hombourger@siemens.com>
>>> ---
>>>  RECIPE-API-CHANGELOG.md     |  6 ++++
>>>  doc/user_manual.md          |  1 +
>>>  meta/classes/rootfs.bbclass | 66
>>> +++++++++++++++++++++++++++++++++++++
>>>  3 files changed, 73 insertions(+)
>>>
>>> diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md
>>> index a4cf1338..725737b2 100644
>>> --- a/RECIPE-API-CHANGELOG.md
>>> +++ b/RECIPE-API-CHANGELOG.md
>>> @@ -722,3 +722,9 @@ Optional fields of the isar-apt repo can be
>>> controlled by adding to the
>>>  
>>>  Changes in next
>>>  ---------------
>>> +
>>> +### Require bubblewrap to run non-privileged commands with bind-
>>> mounts
>>> +
>>> +Isar occasionally needs to run commands within root file-systems
>>> that it
>>> +builds and with several bind-mounts (e.g. /isar-apt). bubblewrap
>>> may be
>>> +used in Isar classes instead of `sudo chroot`.
>>> diff --git a/doc/user_manual.md b/doc/user_manual.md
>>> index 0dc317c3..3cf1a9aa 100644
>>> --- a/doc/user_manual.md
>>> +++ b/doc/user_manual.md
>>> @@ -75,6 +75,7 @@ Install the following packages:
>>>  ```
>>>  apt install \
>>>    binfmt-support \
>>> +  bubblewrap \
>>
>> Does the bubblewrap (and kernel features) of bullseye suffice here,
>> or
>> is that a bookworm+ thing? How about buster (still listed as host)?
> 
> bubblewrap has been around for ages: these older distros did support
> flatpak. buster included.
> 
> https://packages.debian.org/buster/bubblewrap
>  

Then I suppose our CI would catch any nasty difference in our usage
compared to those standard use cases, right?

Jan

-- 
Siemens AG, Foundational Technologies
Linux Expert Center

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/a324f93b-ae8a-46f3-a8f7-10088b8d4ef4%40siemens.com.

[PATCH v2 0/4] non-privileged commands in chroot

['Cedric Hombourger' via isar-users]

When building root filesystems for foreign architectures with package source
caching enabled, apt operations are executed within the rootfs through QEMU
emulation. This results in significantly degraded performance, particularly
when downloading source packages sequentially.

This patch series introduces a new wrapper function that enables native
command execution against a rootfs while preserving special mount points
(such as /isar-apt). The approach:

- Improves build performance for foreign architecture builds
- Maintains filesystem isolation using bubblewrap
- Preserves access to special mount points required by isar

Testing:
- Basic smoke tests performed successfully (citest.py -t dev)
- Performance improvements observed in source package acquisition
- Tested with various foreign architecture configurations

Dependencies:
- Adds bubblewrap as a new host tool requirement
- Uses kas-container 4.8.0 or later (see [1])

Changes since v1 patch:
  - Rebase (resolve RECIPE-API-CHANGELOG.md merge conflicts)
  - Prefix rootfs variable in rootfs_cmd with bwrap to avoid clashes

Changes since RFC patch:
  - Let caller decide where to bind-mount the rootfs to
  - Make the rootfs argument optional
  - Support 32-bit rootfs (no lib64 there)

(Re-)validated with "citest.py -t dev" (using kas-container 4.8.1):

 JOB ID  : be45cf0e3937b95d283e7acd687787df259c4341
 JOB LOG : job-results/job-2025-06-18T12.43-be45cf0/job.log
  (1/6) citest.py:DevTest.test_dev: STARTED
  (1/6) citest.py:DevTest.test_dev: PASS (1177.32 s)
  (2/6) citest.py:DevTest.test_dev_apps: STARTED
  (2/6) citest.py:DevTest.test_dev_apps: PASS (1128.83 s)
  (3/6) citest.py:DevTest.test_dev_rebuild: STARTED
  (3/6) citest.py:DevTest.test_dev_rebuild: PASS (412.72 s)
  (4/6) citest.py:DevTest.test_dev_run_amd64_bookworm: STARTED
  (4/6) citest.py:DevTest.test_dev_run_amd64_bookworm: PASS (77.60 s)
  (5/6) citest.py:DevTest.test_dev_run_arm64_bookworm: STARTED
  (5/6) citest.py:DevTest.test_dev_run_arm64_bookworm: PASS (50.17 s)
  (6/6) citest.py:DevTest.test_dev_run_arm_bookworm: STARTED
  (6/6) citest.py:DevTest.test_dev_run_arm_bookworm: PASS (52.95 s)
 RESULTS    : PASS 6 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
 JOB TIME   : 2905.62 s

Cedric Hombourger' via isar-users (4):
  rootfs: introduce wrapper to run commands against a rootfs
  deb-dl-dir: optimize caching of source packages using apt natively
  image-postproc-extension: refactor systemd version checks
  image-postproc-extension: extract systemd's version using rootfs_cmd

 RECIPE-API-CHANGELOG.md                       |  7 ++
 doc/user_manual.md                            |  1 +
 meta/classes/deb-dl-dir.bbclass               | 37 +++--------
 meta/classes/image-postproc-extension.bbclass | 12 ++--
 meta/classes/rootfs.bbclass                   | 66 +++++++++++++++++++
 5 files changed, 90 insertions(+), 33 deletions(-)

[1] https://lists.isar-build.org/isar-users/20250616155748.561641-1-cedric.hombourger@siemens.com/T/#u

-- 
2.39.5

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20250618135040.8252-1-cedric.hombourger%40siemens.com.

[PATCH v2 1/4] rootfs: introduce wrapper to run commands against a rootfs

['Cedric Hombourger' via isar-users]

"sudo chroot" is used in several places to run commands inside rootfs
directories constructed by Isar. There are cases where a command could
be used without elevated privileges as long as special folders such as
/isar-apt are mounted (they are often referenced as /isar-apt in
configuration files found in the target rootfs). For such cases,
bubblewrap may be used to create a non-privileged namespace (either
in a bare/native environment or within a docker/podman container)
where the command will be executed as if chroot had been used. The
rootfs may also be the host root file-system: this should however
be used with care to avoid host contamination problems (note: Isar
already relies on a number of host tools).

Signed-off-by: Cedric Hombourger <cedric.hombourger@siemens.com>
---
 RECIPE-API-CHANGELOG.md     |  7 ++++
 doc/user_manual.md          |  1 +
 meta/classes/rootfs.bbclass | 66 +++++++++++++++++++++++++++++++++++++
 3 files changed, 74 insertions(+)

diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md
index 8468717d..18b90555 100644
--- a/RECIPE-API-CHANGELOG.md
+++ b/RECIPE-API-CHANGELOG.md
@@ -727,3 +727,10 @@ Changes in next
 
 This was never documented and never had practical relevance. `oci-archive` is
 the useful OCI image format that can be imported, e.g., by podman.
+
+### Require bubblewrap to run non-privileged commands with bind-mounts
+
+Isar occasionally needs to run commands within root file-systems that it
+builds and with several bind-mounts (e.g. /isar-apt). bubblewrap may be
+used in Isar classes instead of `sudo chroot`. It is pre-installed in
+kas-container version 4.8 (or later).
diff --git a/doc/user_manual.md b/doc/user_manual.md
index ca551a0d..a4fff34a 100644
--- a/doc/user_manual.md
+++ b/doc/user_manual.md
@@ -75,6 +75,7 @@ Install the following packages:
 ```
 apt install \
   binfmt-support \
+  bubblewrap \
   bzip2 \
   mmdebstrap \
   arch-test \
diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass
index 5f877962..f0c172b8 100644
--- a/meta/classes/rootfs.bbclass
+++ b/meta/classes/rootfs.bbclass
@@ -34,6 +34,72 @@ export LANG = "C"
 export LANGUAGE = "C"
 export LC_ALL = "C"
 
+# Execute a command against a rootfs and with isar-apt bind-mounted.
+# Additional mounts may be specified using --bind <source> <target> and a
+# custom directory for the command to be executed with --chdir <dir>. The
+# command is assumed to follow the special "--" argument. This would replace
+# "sudo chroot" calls especially when a native command may be used instead of
+# chroot'ed command and without elevated privileges (the command will likely
+# take the rootfs as argument; e.g. apt-get -o Dir=${ROOTFSDIR}). If the
+# optional rootfs argument is omitted, the host rootfs will be used (e.g. to
+# run native commands): this should be used with care.
+#
+# Usage: rootfs_cmd [options] [rootfs] -- command
+#
+rootfs_cmd() {
+    set -- "$@"
+    bwrap_args="--bind ${REPO_ISAR_DIR}/${DISTRO} /isar-apt"
+    bwrap_rootfs=""
+
+    while [ "${#}" -gt "0" ] && [ "${1}" != "--" ]; do
+        case "${1}" in
+            --bind)
+                if [ "${#}" -lt "3" ]; then
+                    bbfatal "--bind requires two arguments"
+                fi
+                bwrap_args="${bwrap_args} --bind ${2} ${3}"
+                shift 3
+                ;;
+            --chdir)
+                if [ "${#}" -lt "2" ]; then
+                    bbfatal "${1} requires an argument"
+                fi
+                bwrap_args="${bwrap_args} ${1} ${2}"
+                shift 2
+                ;;
+            -*)
+                bbfatal "${1} is not a supported option!"
+                ;;
+            *)
+                if [ -z "${bwrap_rootfs}" ]; then
+                    bwrap_rootfs="${1}"
+                    shift
+                else
+                    bbfatal "unexpected argument '${1}'"
+                fi
+                ;;
+        esac
+    done
+
+    if [ -n "${bwrap_rootfs}" ]; then
+        bwrap_args="${bwrap_args} --bind ${bwrap_rootfs} /"
+    fi
+
+    if [ "${#}" -le "1" ] || [ "${1}" != "--" ]; then
+        bbfatal "no command specified (missing --)"
+    fi
+    shift  # remove "--", command and its arguments follows
+
+    for ro_d in bin etc lib lib64 sys usr var; do
+        [ -d ${bwrap_rootfs}/${ro_d} ] || continue
+        bwrap_args="${bwrap_args} --ro-bind ${bwrap_rootfs}/${ro_d} /${ro_d}"
+    done
+
+    bwrap --unshare-user --unshare-pid ${bwrap_args} \
+        --dev-bind /dev /dev --proc /proc --tmpfs /tmp \
+        -- "${@}"
+}
+
 rootfs_do_mounts[weight] = "3"
 rootfs_do_mounts() {
     sudo -s <<'EOSUDO'
-- 
2.39.5

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20250618135040.8252-2-cedric.hombourger%40siemens.com.

[PATCH v2 2/4] deb-dl-dir: optimize caching of source packages using apt natively

['Cedric Hombourger' via isar-users]

source package are downloaded by entering the target rootfs and run
apt there. For foreign architectures, this results in apt being
executed under QEMU and leads to poor performance. By using the
recently introduced rootfs_native_cmd command wrapper, apt will be
executed natively against the target rootfs and without elevated
privileges. For our test work-load, caching was reduced from more
than 10 hours to an hour. Performance is also more consistent as
it will no longer depend as to when bitbake kicks caching of
source packages for foreign architecture rootfs vs rootfs for the
host (in multiconfig builds).

Signed-off-by: Cedric Hombourger <cedric.hombourger@siemens.com>
---
 meta/classes/deb-dl-dir.bbclass | 37 ++++++++-------------------------
 1 file changed, 9 insertions(+), 28 deletions(-)

diff --git a/meta/classes/deb-dl-dir.bbclass b/meta/classes/deb-dl-dir.bbclass
index 3f560da4..7026f4f4 100644
--- a/meta/classes/deb-dl-dir.bbclass
+++ b/meta/classes/deb-dl-dir.bbclass
@@ -5,25 +5,6 @@
 
 inherit repository
 
-debsrc_do_mounts() {
-    sudo -s <<EOSUDO
-    set -e
-    mkdir -p "${1}/deb-src"
-    mountpoint -q "${1}/deb-src" || \
-    mount -o bind,private "${DEBSRCDIR}" "${1}/deb-src"
-EOSUDO
-}
-
-debsrc_undo_mounts() {
-    sudo -s <<EOSUDO
-    set -e
-    mkdir -p "${1}/deb-src"
-    mountpoint -q "${1}/deb-src" && \
-    umount "${1}/deb-src"
-    rm -rf "${1}/deb-src"
-EOSUDO
-}
-
 debsrc_source_version_filter() {
     # Filter the input to only consider Package, Version and Source lines
     #
@@ -51,11 +32,6 @@ debsrc_download() {
     export rootfs_distro="$2"
     mkdir -p "${DEBSRCDIR}"/"${rootfs_distro}"
 
-    debsrc_do_mounts "${rootfs}"
-
-    trap 'exit 1' INT HUP QUIT TERM ALRM USR1
-    trap 'debsrc_undo_mounts "${rootfs}"' EXIT
-
     ( flock 9
     set -e
     printenv | grep -q BB_VERBOSE_LOGS && set -x
@@ -89,13 +65,18 @@ debsrc_download() {
         dscname="${src}_${version#*:}.dsc"
         [ -f "${DEBSRCDIR}"/"${rootfs_distro}"/"${src}"/"${dscname}" ] || {
             # use apt-get source to download sources in DEBSRCDIR
-            sudo -E chroot --userspec=$( id -u ):$( id -g ) ${rootfs} \
-                sh -c ' mkdir -p "/deb-src/${1}/${2}" && cd "/deb-src/${1}/${2}" && apt-get -y --download-only --only-source source "$2"="$3" ' download-src "${rootfs_distro}" "${src}" "${version}"
+            mkdir -p "${DEBSRCDIR}/${rootfs_distro}"/"${src}"
+            rootfs_cmd \
+                --bind "${DEBSRCDIR}" "/deb-src" \
+                --bind "${rootfs}" "${rootfs}" \
+                --chdir "/deb-src/${rootfs_distro}/${src}" \
+                -- \
+                apt-get -o APT::Architecture=${DISTRO_ARCH} \
+                        -o Dir="${rootfs}" -y --download-only \
+                        --only-source source "${src}=${version}"
         }
     done
     ) 9>"${DEBSRCDIR}/${rootfs_distro}.lock"
-
-    debsrc_undo_mounts "${rootfs}"
 }
 
 dbg_pkgs_download() {
-- 
2.39.5

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20250618135040.8252-3-cedric.hombourger%40siemens.com.

[PATCH v2 3/4] image-postproc-extension: refactor systemd version checks

['Cedric Hombourger' via isar-users]

Signed-off-by: Cedric Hombourger <cedric.hombourger@siemens.com>
---
 meta/classes/image-postproc-extension.bbclass | 12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)

diff --git a/meta/classes/image-postproc-extension.bbclass b/meta/classes/image-postproc-extension.bbclass
index 991bac4c..0af588d8 100644
--- a/meta/classes/image-postproc-extension.bbclass
+++ b/meta/classes/image-postproc-extension.bbclass
@@ -53,12 +53,17 @@ image_postprocess_mark() {
         --build-id "${BUILD_ID}" --variant "${DESCRIPTION}" --version "${PV}"
 }
 
+# Use dpkg to find out which version of systemd is installed into the image or reports "0"
+image_systemd_version() {
+    sudo chroot ${IMAGE_ROOTFS} dpkg-query --showformat='${source:Upstream-Version}' --show systemd || echo "0"
+}
+
 ROOTFS_POSTPROCESS_COMMAND =+ "image_postprocess_machine_id"
 image_postprocess_machine_id() {
     # systemd(1) takes care of recreating the machine-id on first boot
     # for systemd < v247, set to empty string, else set to uninitialized
     # (required if initramfs with ro root is used)
-    SYSTEMD_VERSION=$( sudo chroot ${IMAGE_ROOTFS} dpkg-query --showformat='${source:Upstream-Version}' --show systemd || echo "0" )
+    SYSTEMD_VERSION=$( image_systemd_version )
     MACHINE_ID="uninitialized"
     if dpkg --compare-versions "$SYSTEMD_VERSION" "lt" "247"; then
         MACHINE_ID=""
@@ -82,10 +87,7 @@ image_postprocess_sshd_key_regen() {
 
 ROOTFS_POSTPROCESS_COMMAND =+ "image_posprocess_disable_systemd_firstboot"
 image_posprocess_disable_systemd_firstboot() {
-    SYSTEMD_VERSION=$(sudo chroot '${ROOTFSDIR}' dpkg-query \
-        --showformat='${source:Upstream-Version}' \
-        --show systemd || echo "0" )
-
+    SYSTEMD_VERSION=$( image_systemd_version )
     if dpkg --compare-versions "$SYSTEMD_VERSION" "ge" "251"; then
         sudo chroot '${ROOTFSDIR}' systemctl mask systemd-firstboot
         if ! cmd_output=$(sudo chroot '${ROOTFSDIR}' systemd-firstboot \
-- 
2.39.5

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20250618135040.8252-4-cedric.hombourger%40siemens.com.

[PATCH v2 4/4] image-postproc-extension: extract systemd's version using rootfs_cmd

['Cedric Hombourger' via isar-users]

Elevated privileges are not required to query the rootfs for the version
of systemd: replace "sudo chroot" with "rootfs_cmd"

Signed-off-by: Cedric Hombourger <cedric.hombourger@siemens.com>
---
 meta/classes/image-postproc-extension.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/image-postproc-extension.bbclass b/meta/classes/image-postproc-extension.bbclass
index 0af588d8..21dcfccc 100644
--- a/meta/classes/image-postproc-extension.bbclass
+++ b/meta/classes/image-postproc-extension.bbclass
@@ -55,7 +55,7 @@ image_postprocess_mark() {
 
 # Use dpkg to find out which version of systemd is installed into the image or reports "0"
 image_systemd_version() {
-    sudo chroot ${IMAGE_ROOTFS} dpkg-query --showformat='${source:Upstream-Version}' --show systemd || echo "0"
+    rootfs_cmd ${IMAGE_ROOTFS} -- dpkg-query --showformat='${source:Upstream-Version}' --show systemd || echo "0"
 }
 
 ROOTFS_POSTPROCESS_COMMAND =+ "image_postprocess_machine_id"
-- 
2.39.5

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20250618135040.8252-5-cedric.hombourger%40siemens.com.

Re: [PATCH v2 4/4] image-postproc-extension: extract systemd's version using rootfs_cmd

['Quirin Gylstorff' via isar-users]

On 6/18/25 15:50, 'Cedric Hombourger' via isar-users wrote:
> Elevated privileges are not required to query the rootfs for the version
> of systemd: replace "sudo chroot" with "rootfs_cmd"
> 
> Signed-off-by: Cedric Hombourger <cedric.hombourger@siemens.com>
> ---
>   meta/classes/image-postproc-extension.bbclass | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/meta/classes/image-postproc-extension.bbclass b/meta/classes/image-postproc-extension.bbclass
> index 0af588d8..21dcfccc 100644
> --- a/meta/classes/image-postproc-extension.bbclass
> +++ b/meta/classes/image-postproc-extension.bbclass
> @@ -55,7 +55,7 @@ image_postprocess_mark() {
>   
>   # Use dpkg to find out which version of systemd is installed into the image or reports "0"
>   image_systemd_version() {
> -    sudo chroot ${IMAGE_ROOTFS} dpkg-query --showformat='${source:Upstream-Version}' --show systemd || echo "0"
> +    rootfs_cmd ${IMAGE_ROOTFS} -- dpkg-query --showformat='${source:Upstream-Version}' --show systemd || echo "0"
>   }
Why are we not using the root parameter of dpkg-query for this?

Quirin
>   
>   ROOTFS_POSTPROCESS_COMMAND =+ "image_postprocess_machine_id"

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/376af427-8a16-41ed-b361-8bdbe96a0c29%40siemens.com.