* [PATCH v2 1/1] fix(dpkg-build): deploy debs via shared sstate dir
@ 2026-07-31 10:03 'Felix Moessbauer' via isar-users
2026-08-05 8:40 ` Zhihang Wei
0 siblings, 1 reply; 2+ messages in thread
From: 'Felix Moessbauer' via isar-users @ 2026-07-31 10:03 UTC (permalink / raw)
To: isar-users; +Cc: wzh, jan.kiszka, Felix Moessbauer
The built debs were stored in a WORKDIR-internal plaindir and consumed
from there by do_deploy_deb. do_deploy_deb is a live task that mutates
the external isar-apt repository and can therefore never be sstate
cached, so it always runs and relies purely on the plaindir being
present on disk.
When do_dpkg_build is served from the sstate cache, that WORKDIR-internal
plaindir is not guaranteed to be materialized (e.g. when the task is
covered via hash equivalence without a setscene restore). do_deploy_deb
then runs against an empty deploy dir, passes an unexpanded glob to
dpkg-deb and aborts the build.
Fix it by deploying the debs into a shared, sstate-tracked DEPLOY_DIR_DEB
via sstate-inputdirs/sstate-outputdirs, the same pattern already used by
do_bootstrap, do_rootfs_install and friends. This makes the deploy dir a
first-class sstate output that bitbake reliably restores whenever a
consumer needs it, decoupled from the WORKDIR lifecycle. do_deploy_deb
additionally guards against an empty deploy dir so it can no longer feed
a literal glob to dpkg-deb.
Fixes: 3fc96652 ("dpkg: directly deploy via sstate cache")
Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
---
Changes since v1:
- rebased on top of the latest next, no functional changes
RECIPE-API-CHANGELOG.md | 8 ++++++++
doc/technical_overview.md | 2 +-
meta/classes-recipe/barebox.bbclass | 4 ++--
meta/classes-recipe/dpkg-base.bbclass | 19 ++++++++++++++-----
4 files changed, 25 insertions(+), 8 deletions(-)
diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md
index 7b13cc7e..7e923e55 100644
--- a/RECIPE-API-CHANGELOG.md
+++ b/RECIPE-API-CHANGELOG.md
@@ -1138,6 +1138,14 @@ network access. Recipes that still require network access during the build can s
`DPKG_BUILD_ENABLE_NETWORK = "1"` to explicitly enable it when using an sbuild
version newer than 0.89.
+### Built debs are deployed to `DEPLOY_DIR_DEB`
+
+The debs produced by `do_dpkg_build` are now exported into a shared, sstate-tracked
+deploy directory `DEPLOY_DIR_DEB` instead of being deployed to the `WORKDIR`.
+
+Recipes that accessed the built debs through `${WORKDIR}/*.deb` (e.g. to unpack an
+artifact in a `do_deploy` task) must now reference `${DEPLOY_DIR_DEB}/*.deb` instead.
+
### Add Hyper-V machine support
A new machine `hyper-v` has been introduced for building images
diff --git a/doc/technical_overview.md b/doc/technical_overview.md
index fe427cae..5d2e1b8a 100644
--- a/doc/technical_overview.md
+++ b/doc/technical_overview.md
@@ -217,7 +217,7 @@ Both consist of the following steps:
finally unmount again (`dpkg_undo_mounts`).
6. Task `do_deploy_deb`: add successfully built packages
- `${DEPLOYDIR}/*.deb` to the isar-apt repository
+ `${DEPLOY_DIR_DEB}/*.deb` to the isar-apt repository
`${REPO_ISAR_DIR}/${DISTRO}`
## 3.6 Populate Target Filesystem
diff --git a/meta/classes-recipe/barebox.bbclass b/meta/classes-recipe/barebox.bbclass
index c9a767d5..094f8018 100644
--- a/meta/classes-recipe/barebox.bbclass
+++ b/meta/classes-recipe/barebox.bbclass
@@ -47,12 +47,12 @@ BAREBOX_BASE_BIN ?= "barebox"
do_deploy[dirs] = "${DEPLOY_DIR_IMAGE}"
do_deploy() {
- dpkg --fsys-tarfile ${DEPLOYDIR}/${PN}_${CHANGELOG_V}_${DISTRO_ARCH}.deb | \
+ dpkg --fsys-tarfile ${DEPLOY_DIR_DEB}/${PN}_${CHANGELOG_V}_${DISTRO_ARCH}.deb | \
tar xOf - ./usr/lib/barebox/barebox.bin \
> "${DEPLOY_DIR_IMAGE}/${BAREBOX_BASE_BIN}.img"
ln -sf ${BAREBOX_BASE_BIN}.img ${DEPLOY_DIR_IMAGE}/barebox.bin
- dpkg --fsys-tarfile ${DEPLOYDIR}/${PN}_${CHANGELOG_V}_${DISTRO_ARCH}.deb | \
+ dpkg --fsys-tarfile ${DEPLOY_DIR_DEB}/${PN}_${CHANGELOG_V}_${DISTRO_ARCH}.deb | \
tar xOf - ./usr/lib/barebox/barebox.config \
> "${DEPLOY_DIR_IMAGE}/${BAREBOX_BASE_BIN}.config"
ln -sf ${BAREBOX_BASE_BIN}.config ${DEPLOY_DIR_IMAGE}/barebox.config
diff --git a/meta/classes-recipe/dpkg-base.bbclass b/meta/classes-recipe/dpkg-base.bbclass
index e2ed4dfb..610213b8 100644
--- a/meta/classes-recipe/dpkg-base.bbclass
+++ b/meta/classes-recipe/dpkg-base.bbclass
@@ -12,7 +12,12 @@ inherit repository
inherit deb-dl-dir
inherit essential
+# Local (WORKDIR-internal) dir where do_dpkg_build collects the built debs.
+# Its content is exported into the shared, sstate-tracked DEPLOY_DIR_DEB so
+# that consumers (do_deploy_deb) always find the artifacts, independent of the
+# WORKDIR lifecycle.
DEPLOYDIR = "${WORKDIR}/deploy"
+DEPLOY_DIR_DEB = "${DEPLOY_DIR}/isar-deb/${DISTRO}-${DISTRO_ARCH}/${PN}"
DEPENDS ?= ""
RPROVIDES ?= "${PROVIDES}"
@@ -200,7 +205,8 @@ python do_dpkg_build() {
bb.build.exec_func('dpkg_chroot_finalize', d)
}
do_dpkg_build[cleandirs] = "${DEPLOYDIR}"
-do_dpkg_build[sstate-plaindirs] = "${DEPLOYDIR}"
+do_dpkg_build[sstate-inputdirs] = "${DEPLOYDIR}"
+do_dpkg_build[sstate-outputdirs] = "${DEPLOY_DIR_DEB}"
do_dpkg_build[network] = "${TASK_USE_NETWORK_AND_SUDO}"
do_dpkg_build[depends] = "${SCHROOT_DEP}"
do_dpkg_build[postfuncs] += "dpkg_collect_debs"
@@ -218,7 +224,7 @@ addtask dpkg_build_setscene
CLEANFUNCS += "deb_clean"
deb_clean() {
- DEBS=$( find ${DEPLOYDIR} -maxdepth 1 -name "*.deb" || [ ! -d ${S} ] )
+ DEBS=$( find ${DEPLOY_DIR_DEB} -maxdepth 1 -name "*.deb" || [ ! -d ${S} ] )
if [ -n "${DEBS}" ]; then
for d in ${DEBS}; do
repo_del_package "${REPO_ISAR_DIR}"/"${DISTRO}" \
@@ -232,8 +238,11 @@ do_clean[network] = "${TASK_USE_SUDO}"
do_deploy_deb() {
deb_clean
- repo_add_packages "${REPO_ISAR_DIR}"/"${DISTRO}" \
- "${REPO_ISAR_DB_DIR}"/"${DISTRO}" "${DEBDISTRONAME}" ${DEPLOYDIR}/*.deb
+ debs=$(find ${DEPLOY_DIR_DEB} -maxdepth 1 -name '*.deb')
+ if [ -n "${debs}" ]; then
+ repo_add_packages "${REPO_ISAR_DIR}"/"${DISTRO}" \
+ "${REPO_ISAR_DB_DIR}"/"${DISTRO}" "${DEBDISTRONAME}" ${debs}
+ fi
}
addtask deploy_deb after do_dpkg_build before do_build
@@ -241,7 +250,7 @@ do_deploy_deb[deptask] = "do_deploy_deb"
do_deploy_deb[rdeptask] = "do_deploy_deb"
do_deploy_deb[depends] += "isar-apt:do_cache_config"
do_deploy_deb[lockfiles] = "${REPO_ISAR_DIR}/isar.lock"
-do_deploy_deb[dirs] = "${S}"
+do_deploy_deb[dirs] = "${S} ${DEPLOY_DIR_DEB}"
python do_devshell() {
isar_export_proxies(d)
--
2.53.0
--
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20260731100311.449086-1-felix.moessbauer%40siemens.com.
^ permalink raw reply [flat|nested] 2+ messages in thread
* Re: [PATCH v2 1/1] fix(dpkg-build): deploy debs via shared sstate dir
2026-07-31 10:03 [PATCH v2 1/1] fix(dpkg-build): deploy debs via shared sstate dir 'Felix Moessbauer' via isar-users
@ 2026-08-05 8:40 ` Zhihang Wei
0 siblings, 0 replies; 2+ messages in thread
From: Zhihang Wei @ 2026-08-05 8:40 UTC (permalink / raw)
To: Felix Moessbauer, isar-users; +Cc: jan.kiszka
Applied to next, thanks.
Zhihang
On 7/31/26 12:03, Felix Moessbauer wrote:
> The built debs were stored in a WORKDIR-internal plaindir and consumed
> from there by do_deploy_deb. do_deploy_deb is a live task that mutates
> the external isar-apt repository and can therefore never be sstate
> cached, so it always runs and relies purely on the plaindir being
> present on disk.
>
> When do_dpkg_build is served from the sstate cache, that WORKDIR-internal
> plaindir is not guaranteed to be materialized (e.g. when the task is
> covered via hash equivalence without a setscene restore). do_deploy_deb
> then runs against an empty deploy dir, passes an unexpanded glob to
> dpkg-deb and aborts the build.
>
> Fix it by deploying the debs into a shared, sstate-tracked DEPLOY_DIR_DEB
> via sstate-inputdirs/sstate-outputdirs, the same pattern already used by
> do_bootstrap, do_rootfs_install and friends. This makes the deploy dir a
> first-class sstate output that bitbake reliably restores whenever a
> consumer needs it, decoupled from the WORKDIR lifecycle. do_deploy_deb
> additionally guards against an empty deploy dir so it can no longer feed
> a literal glob to dpkg-deb.
>
> Fixes: 3fc96652 ("dpkg: directly deploy via sstate cache")
> Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
> ---
> Changes since v1:
> - rebased on top of the latest next, no functional changes
>
> RECIPE-API-CHANGELOG.md | 8 ++++++++
> doc/technical_overview.md | 2 +-
> meta/classes-recipe/barebox.bbclass | 4 ++--
> meta/classes-recipe/dpkg-base.bbclass | 19 ++++++++++++++-----
> 4 files changed, 25 insertions(+), 8 deletions(-)
>
> diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md
> index 7b13cc7e..7e923e55 100644
> --- a/RECIPE-API-CHANGELOG.md
> +++ b/RECIPE-API-CHANGELOG.md
> @@ -1138,6 +1138,14 @@ network access. Recipes that still require network access during the build can s
> `DPKG_BUILD_ENABLE_NETWORK = "1"` to explicitly enable it when using an sbuild
> version newer than 0.89.
>
> +### Built debs are deployed to `DEPLOY_DIR_DEB`
> +
> +The debs produced by `do_dpkg_build` are now exported into a shared, sstate-tracked
> +deploy directory `DEPLOY_DIR_DEB` instead of being deployed to the `WORKDIR`.
> +
> +Recipes that accessed the built debs through `${WORKDIR}/*.deb` (e.g. to unpack an
> +artifact in a `do_deploy` task) must now reference `${DEPLOY_DIR_DEB}/*.deb` instead.
> +
> ### Add Hyper-V machine support
>
> A new machine `hyper-v` has been introduced for building images
> diff --git a/doc/technical_overview.md b/doc/technical_overview.md
> index fe427cae..5d2e1b8a 100644
> --- a/doc/technical_overview.md
> +++ b/doc/technical_overview.md
> @@ -217,7 +217,7 @@ Both consist of the following steps:
> finally unmount again (`dpkg_undo_mounts`).
>
> 6. Task `do_deploy_deb`: add successfully built packages
> - `${DEPLOYDIR}/*.deb` to the isar-apt repository
> + `${DEPLOY_DIR_DEB}/*.deb` to the isar-apt repository
> `${REPO_ISAR_DIR}/${DISTRO}`
>
> ## 3.6 Populate Target Filesystem
> diff --git a/meta/classes-recipe/barebox.bbclass b/meta/classes-recipe/barebox.bbclass
> index c9a767d5..094f8018 100644
> --- a/meta/classes-recipe/barebox.bbclass
> +++ b/meta/classes-recipe/barebox.bbclass
> @@ -47,12 +47,12 @@ BAREBOX_BASE_BIN ?= "barebox"
>
> do_deploy[dirs] = "${DEPLOY_DIR_IMAGE}"
> do_deploy() {
> - dpkg --fsys-tarfile ${DEPLOYDIR}/${PN}_${CHANGELOG_V}_${DISTRO_ARCH}.deb | \
> + dpkg --fsys-tarfile ${DEPLOY_DIR_DEB}/${PN}_${CHANGELOG_V}_${DISTRO_ARCH}.deb | \
> tar xOf - ./usr/lib/barebox/barebox.bin \
> > "${DEPLOY_DIR_IMAGE}/${BAREBOX_BASE_BIN}.img"
> ln -sf ${BAREBOX_BASE_BIN}.img ${DEPLOY_DIR_IMAGE}/barebox.bin
>
> - dpkg --fsys-tarfile ${DEPLOYDIR}/${PN}_${CHANGELOG_V}_${DISTRO_ARCH}.deb | \
> + dpkg --fsys-tarfile ${DEPLOY_DIR_DEB}/${PN}_${CHANGELOG_V}_${DISTRO_ARCH}.deb | \
> tar xOf - ./usr/lib/barebox/barebox.config \
> > "${DEPLOY_DIR_IMAGE}/${BAREBOX_BASE_BIN}.config"
> ln -sf ${BAREBOX_BASE_BIN}.config ${DEPLOY_DIR_IMAGE}/barebox.config
> diff --git a/meta/classes-recipe/dpkg-base.bbclass b/meta/classes-recipe/dpkg-base.bbclass
> index e2ed4dfb..610213b8 100644
> --- a/meta/classes-recipe/dpkg-base.bbclass
> +++ b/meta/classes-recipe/dpkg-base.bbclass
> @@ -12,7 +12,12 @@ inherit repository
> inherit deb-dl-dir
> inherit essential
>
> +# Local (WORKDIR-internal) dir where do_dpkg_build collects the built debs.
> +# Its content is exported into the shared, sstate-tracked DEPLOY_DIR_DEB so
> +# that consumers (do_deploy_deb) always find the artifacts, independent of the
> +# WORKDIR lifecycle.
> DEPLOYDIR = "${WORKDIR}/deploy"
> +DEPLOY_DIR_DEB = "${DEPLOY_DIR}/isar-deb/${DISTRO}-${DISTRO_ARCH}/${PN}"
>
> DEPENDS ?= ""
> RPROVIDES ?= "${PROVIDES}"
> @@ -200,7 +205,8 @@ python do_dpkg_build() {
> bb.build.exec_func('dpkg_chroot_finalize', d)
> }
> do_dpkg_build[cleandirs] = "${DEPLOYDIR}"
> -do_dpkg_build[sstate-plaindirs] = "${DEPLOYDIR}"
> +do_dpkg_build[sstate-inputdirs] = "${DEPLOYDIR}"
> +do_dpkg_build[sstate-outputdirs] = "${DEPLOY_DIR_DEB}"
> do_dpkg_build[network] = "${TASK_USE_NETWORK_AND_SUDO}"
> do_dpkg_build[depends] = "${SCHROOT_DEP}"
> do_dpkg_build[postfuncs] += "dpkg_collect_debs"
> @@ -218,7 +224,7 @@ addtask dpkg_build_setscene
> CLEANFUNCS += "deb_clean"
>
> deb_clean() {
> - DEBS=$( find ${DEPLOYDIR} -maxdepth 1 -name "*.deb" || [ ! -d ${S} ] )
> + DEBS=$( find ${DEPLOY_DIR_DEB} -maxdepth 1 -name "*.deb" || [ ! -d ${S} ] )
> if [ -n "${DEBS}" ]; then
> for d in ${DEBS}; do
> repo_del_package "${REPO_ISAR_DIR}"/"${DISTRO}" \
> @@ -232,8 +238,11 @@ do_clean[network] = "${TASK_USE_SUDO}"
>
> do_deploy_deb() {
> deb_clean
> - repo_add_packages "${REPO_ISAR_DIR}"/"${DISTRO}" \
> - "${REPO_ISAR_DB_DIR}"/"${DISTRO}" "${DEBDISTRONAME}" ${DEPLOYDIR}/*.deb
> + debs=$(find ${DEPLOY_DIR_DEB} -maxdepth 1 -name '*.deb')
> + if [ -n "${debs}" ]; then
> + repo_add_packages "${REPO_ISAR_DIR}"/"${DISTRO}" \
> + "${REPO_ISAR_DB_DIR}"/"${DISTRO}" "${DEBDISTRONAME}" ${debs}
> + fi
> }
>
> addtask deploy_deb after do_dpkg_build before do_build
> @@ -241,7 +250,7 @@ do_deploy_deb[deptask] = "do_deploy_deb"
> do_deploy_deb[rdeptask] = "do_deploy_deb"
> do_deploy_deb[depends] += "isar-apt:do_cache_config"
> do_deploy_deb[lockfiles] = "${REPO_ISAR_DIR}/isar.lock"
> -do_deploy_deb[dirs] = "${S}"
> +do_deploy_deb[dirs] = "${S} ${DEPLOY_DIR_DEB}"
>
> python do_devshell() {
> isar_export_proxies(d)
--
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/754fd903-8735-4a08-a730-8066c982e13c%40ilbers.de.
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2026-08-05 8:40 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-07-31 10:03 [PATCH v2 1/1] fix(dpkg-build): deploy debs via shared sstate dir 'Felix Moessbauer' via isar-users
2026-08-05 8:40 ` Zhihang Wei
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox