[PATCH v2 0/2] ensure source packages are deployed on rebuild with sstate

[PATCH v2 0/2] ensure source packages are deployed on rebuild with sstate

['Felix Moessbauer' via isar-users]

Changes since v1:

- move changelog entry to correct patch
- rework caching logic to not break gbp or apt fetcher interfaces

The series has now been tested in fast CI.

Best regards,
Felix Moessbauer

Felix Moessbauer (2):
  dpkg: let do_deploy_deb runtime depend on do_deploy_source
  dpkg: cache do_dpkg_source results in sstate

 RECIPE-API-CHANGELOG.md                 |  8 ++++++++
 meta/classes-recipe/dpkg-source.bbclass | 21 +++++++++++++++++++--
 meta/classes-recipe/dpkg.bbclass        |  2 +-
 3 files changed, 28 insertions(+), 3 deletions(-)

-- 
2.55.0

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20260806072428.2409275-1-felix.moessbauer%40siemens.com.

[PATCH v2 1/2] dpkg: let do_deploy_deb runtime depend on do_deploy_source

['Felix Moessbauer' via isar-users]

The isar-apt is an artifact that is deployed by isar and can be consumed
as-is as an apt repository by downstream users. As the isar-apt also
provides the debian source packages, we have to ensure that these are
also deployed when the do_dpkg_build task is provided by the SState
cache. Otherwise the isar-apt is not reproducible across runs with and
without the SState cache.

To fix this, we add a runtime dependency between do_deploy_deb and
do_deploy_source, to ensure all sources of our debs are deployed as
well.

Reported-by: Andreas Ziegler <ziegler.andreas@siemens.com>
Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
---
 meta/classes-recipe/dpkg-source.bbclass | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/meta/classes-recipe/dpkg-source.bbclass b/meta/classes-recipe/dpkg-source.bbclass
index b2b45ed3..92d06e1b 100644
--- a/meta/classes-recipe/dpkg-source.bbclass
+++ b/meta/classes-recipe/dpkg-source.bbclass
@@ -49,6 +49,8 @@ do_deploy_source() {
 addtask deploy_source after do_dpkg_source
 
 do_dpkg_build[depends] += "${BPN}:do_deploy_source"
+# ensure that the source package is deployed into isar-apt
+do_deploy_deb[rdepends] += "${BPN}:do_deploy_source"
 
 SCHROOT_MOUNTS = "${WORKDIR}:/work ${REPO_ISAR_DIR}/${DISTRO}:/isar-apt"
 
-- 
2.55.0

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20260806072428.2409275-2-felix.moessbauer%40siemens.com.

[PATCH v2 2/2] dpkg: cache do_dpkg_source results in sstate

['Felix Moessbauer' via isar-users]

As the do_dpkg_source task now runs for each corresponding
do_deploy_deb task, we also want to cache the generated results (to
avoid pulling in a much bigger dependency tree on otherwise cached
rebuilds).

For that, we cache the .dsc and the .tar.* artifacts in the deploy dir.
This is implemented similar to how do_dpkg_build artifacts (.debs) are
cached, which is also compatible with a hash equivalence server.

Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
---
 RECIPE-API-CHANGELOG.md                 |  8 ++++++++
 meta/classes-recipe/dpkg-source.bbclass | 19 +++++++++++++++++--
 meta/classes-recipe/dpkg.bbclass        |  2 +-
 3 files changed, 26 insertions(+), 3 deletions(-)

diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md
index 7e923e55..7386f5d1 100644
--- a/RECIPE-API-CHANGELOG.md
+++ b/RECIPE-API-CHANGELOG.md
@@ -1146,6 +1146,14 @@ deploy directory `DEPLOY_DIR_DEB` instead of being deployed to the `WORKDIR`.
 Recipes that accessed the built debs through `${WORKDIR}/*.deb` (e.g. to unpack an
 artifact in a `do_deploy` task) must now reference `${DEPLOY_DIR_DEB}/*.deb` instead.
 
+### Debian source packages are deployed to `DEPLOY_DIR_SRC`
+
+The source packages produced by `do_dpkg_source` are now exported into a shared,
+sstate-tracked deploy dir `DEPLOY_DIR_SRC` instead of being deployed to the `WORKDIR`.
+
+Recipes that access the source packages through `${WORKDIR}/*.tar.{gz,xz}` must now
+reference the local copy in `${DEPLOY_DIR_SRC}` instead.
+
 ### Add Hyper-V machine support
 
 A new machine `hyper-v` has been introduced for building images
diff --git a/meta/classes-recipe/dpkg-source.bbclass b/meta/classes-recipe/dpkg-source.bbclass
index 92d06e1b..97cf9714 100644
--- a/meta/classes-recipe/dpkg-source.bbclass
+++ b/meta/classes-recipe/dpkg-source.bbclass
@@ -13,7 +13,12 @@ TAR_REPRO_OPTS ?= "--exclude=.git --exclude=debian \
 DPKG_SOURCE_EXTRA_ARGS ?= "-I"
 
 DEBIAN_SOURCE ?= "${BPN}"
+SRCPKG_DIR = "${WORKDIR}/deploy-srcpkg"
+DEPLOY_DIR_SRC = "${DEPLOY_DIR}/isar-source/${DISTRO}/${BPN}"
 
+do_dpkg_source[cleandirs] = "${SRCPKG_DIR}"
+do_dpkg_source[sstate-inputdirs] = "${SRCPKG_DIR}"
+do_dpkg_source[sstate-outputdirs] = "${DEPLOY_DIR_SRC}"
 do_dpkg_source() {
     # Create a .dsc file from source directory to use it with sbuild
     DEB_SOURCE_NAME=$(dpkg-parsechangelog --show-field Source --file ${WORKDIR}/${PPS}/debian/changelog)
@@ -22,9 +27,19 @@ do_dpkg_source() {
     fi
     find ${WORKDIR} -maxdepth 1 -name "${DEBIAN_SOURCE}_*.dsc" -delete
     sh -c "cd ${WORKDIR}; dpkg-source ${DPKG_SOURCE_EXTRA_ARGS} -b ${PPS}"
+    # move packages to deploy directory
+    find ${WORKDIR} -maxdepth 1 \( -name "${DEBIAN_SOURCE}_*.tar.*" -o -name "${DEBIAN_SOURCE}_*.dsc" \) -exec mv {} ${SRCPKG_DIR}/ \;
 }
 addtask dpkg_source after do_prepare_build
 
+SSTATETASKS += "do_dpkg_source"
+
+python do_dpkg_source_setscene() {
+    sstate_setscene(d)
+}
+
+addtask dpkg_source_setscene
+
 CLEANFUNCS += "deb_clean_source"
 
 deb_clean_source() {
@@ -34,11 +49,11 @@ deb_clean_source() {
 
 do_deploy_source[depends] += "isar-apt:do_cache_config"
 do_deploy_source[lockfiles] = "${REPO_ISAR_DIR}/isar.lock"
-do_deploy_source[dirs] = "${S}"
+do_deploy_source[dirs] = "${S} ${DEPLOY_DIR_SRC}"
 do_deploy_source() {
     repo_del_srcpackage "${REPO_ISAR_DIR}"/"${DISTRO}" \
         "${REPO_ISAR_DB_DIR}"/"${DISTRO}" "${DEBDISTRONAME}" "${DEBIAN_SOURCE}"
-    DSC_FILE=$(find ${WORKDIR} -maxdepth 1 -name "${DEBIAN_SOURCE}_*.dsc")
+    DSC_FILE=$(find ${DEPLOY_DIR_SRC} -maxdepth 1 -name "${DEBIAN_SOURCE}_*.dsc")
     if [ -n "${DSC_FILE}" ]; then
         repo_add_srcpackage "${REPO_ISAR_DIR}"/"${DISTRO}" \
             "${REPO_ISAR_DB_DIR}"/"${DISTRO}" \
diff --git a/meta/classes-recipe/dpkg.bbclass b/meta/classes-recipe/dpkg.bbclass
index d8e129bb..a42703d5 100644
--- a/meta/classes-recipe/dpkg.bbclass
+++ b/meta/classes-recipe/dpkg.bbclass
@@ -116,7 +116,7 @@ dpkg_runbuild() {
     echo '$apt_keep_downloaded_packages = 1;' >> ${SBUILD_CONFIG}
     echo '$stalled_pkg_timeout = ${DPKG_BUILD_TIMEOUT};' >> ${SBUILD_CONFIG}
 
-    DSC_FILE=$(find ${WORKDIR} -maxdepth 1 -name "${DEBIAN_SOURCE}_*.dsc" -print)
+    DSC_FILE=$(find ${DEPLOY_DIR_SRC} -maxdepth 1 -name "${DEBIAN_SOURCE}_*.dsc" -print)
 
     # networking is automatically enabled on older versions of sbuild
     sbuild_network_option=""
-- 
2.55.0

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20260806072428.2409275-3-felix.moessbauer%40siemens.com.

Re: [PATCH v2 1/2] dpkg: let do_deploy_deb runtime depend on do_deploy_source

['Andreas Ziegler' via isar-users]

Am 06.08.2026 um 09:24 schrieb Felix Moessbauer:
> The isar-apt is an artifact that is deployed by isar and can be consumed
> as-is as an apt repository by downstream users. As the isar-apt also
> provides the debian source packages, we have to ensure that these are
> also deployed when the do_dpkg_build task is provided by the SState
> cache. Otherwise the isar-apt is not reproducible across runs with and
> without the SState cache.
> 
> To fix this, we add a runtime dependency between do_deploy_deb and
> do_deploy_source, to ensure all sources of our debs are deployed as
> well.
> 
> Reported-by: Andreas Ziegler <ziegler.andreas@siemens.com>
> Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
> ---
>   meta/classes-recipe/dpkg-source.bbclass | 2 ++
>   1 file changed, 2 insertions(+)
> 
> diff --git a/meta/classes-recipe/dpkg-source.bbclass b/meta/classes-recipe/dpkg-source.bbclass
> index b2b45ed3..92d06e1b 100644
> --- a/meta/classes-recipe/dpkg-source.bbclass
> +++ b/meta/classes-recipe/dpkg-source.bbclass
> @@ -49,6 +49,8 @@ do_deploy_source() {
>   addtask deploy_source after do_dpkg_source
>   
>   do_dpkg_build[depends] += "${BPN}:do_deploy_source"
> +# ensure that the source package is deployed into isar-apt
> +do_deploy_deb[rdepends] += "${BPN}:do_deploy_source"
>   
>   SCHROOT_MOUNTS = "${WORKDIR}:/work ${REPO_ISAR_DIR}/${DISTRO}:/isar-apt"
>   

Works like a charm in our pipeline, so:

Tested-by: Andreas Ziegler <ziegler.andreas@siemens.com>

Thanks!

Andreas

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/fac5f582-a383-4b85-b149-afbc34f3a450%40siemens.com.

Re: [PATCH v2 2/2] dpkg: cache do_dpkg_source results in sstate

['Andreas Ziegler' via isar-users]

Am 06.08.2026 um 09:24 schrieb Felix Moessbauer:
> As the do_dpkg_source task now runs for each corresponding
> do_deploy_deb task, we also want to cache the generated results (to
> avoid pulling in a much bigger dependency tree on otherwise cached
> rebuilds).
> 
> For that, we cache the .dsc and the .tar.* artifacts in the deploy dir.
> This is implemented similar to how do_dpkg_build artifacts (.debs) are
> cached, which is also compatible with a hash equivalence server.
> 
> Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
> ---
>   RECIPE-API-CHANGELOG.md                 |  8 ++++++++
>   meta/classes-recipe/dpkg-source.bbclass | 19 +++++++++++++++++--
>   meta/classes-recipe/dpkg.bbclass        |  2 +-
>   3 files changed, 26 insertions(+), 3 deletions(-)
> 
> diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md
> index 7e923e55..7386f5d1 100644
> --- a/RECIPE-API-CHANGELOG.md
> +++ b/RECIPE-API-CHANGELOG.md
> @@ -1146,6 +1146,14 @@ deploy directory `DEPLOY_DIR_DEB` instead of being deployed to the `WORKDIR`.
>   Recipes that accessed the built debs through `${WORKDIR}/*.deb` (e.g. to unpack an
>   artifact in a `do_deploy` task) must now reference `${DEPLOY_DIR_DEB}/*.deb` instead.
>   
> +### Debian source packages are deployed to `DEPLOY_DIR_SRC`
> +
> +The source packages produced by `do_dpkg_source` are now exported into a shared,
> +sstate-tracked deploy dir `DEPLOY_DIR_SRC` instead of being deployed to the `WORKDIR`.
> +
> +Recipes that access the source packages through `${WORKDIR}/*.tar.{gz,xz}` must now
> +reference the local copy in `${DEPLOY_DIR_SRC}` instead.
> +
>   ### Add Hyper-V machine support
>   
>   A new machine `hyper-v` has been introduced for building images
> diff --git a/meta/classes-recipe/dpkg-source.bbclass b/meta/classes-recipe/dpkg-source.bbclass
> index 92d06e1b..97cf9714 100644
> --- a/meta/classes-recipe/dpkg-source.bbclass
> +++ b/meta/classes-recipe/dpkg-source.bbclass
> @@ -13,7 +13,12 @@ TAR_REPRO_OPTS ?= "--exclude=.git --exclude=debian \
>   DPKG_SOURCE_EXTRA_ARGS ?= "-I"
>   
>   DEBIAN_SOURCE ?= "${BPN}"
> +SRCPKG_DIR = "${WORKDIR}/deploy-srcpkg"
> +DEPLOY_DIR_SRC = "${DEPLOY_DIR}/isar-source/${DISTRO}/${BPN}"
>   
> +do_dpkg_source[cleandirs] = "${SRCPKG_DIR}"
> +do_dpkg_source[sstate-inputdirs] = "${SRCPKG_DIR}"
> +do_dpkg_source[sstate-outputdirs] = "${DEPLOY_DIR_SRC}"
>   do_dpkg_source() {
>       # Create a .dsc file from source directory to use it with sbuild
>       DEB_SOURCE_NAME=$(dpkg-parsechangelog --show-field Source --file ${WORKDIR}/${PPS}/debian/changelog)
> @@ -22,9 +27,19 @@ do_dpkg_source() {
>       fi
>       find ${WORKDIR} -maxdepth 1 -name "${DEBIAN_SOURCE}_*.dsc" -delete
>       sh -c "cd ${WORKDIR}; dpkg-source ${DPKG_SOURCE_EXTRA_ARGS} -b ${PPS}"
> +    # move packages to deploy directory
> +    find ${WORKDIR} -maxdepth 1 \( -name "${DEBIAN_SOURCE}_*.tar.*" -o -name "${DEBIAN_SOURCE}_*.dsc" \) -exec mv {} ${SRCPKG_DIR}/ \;
>   }
>   addtask dpkg_source after do_prepare_build
>   
> +SSTATETASKS += "do_dpkg_source"
> +
> +python do_dpkg_source_setscene() {
> +    sstate_setscene(d)
> +}
> +
> +addtask dpkg_source_setscene
> +
>   CLEANFUNCS += "deb_clean_source"
>   
>   deb_clean_source() {
> @@ -34,11 +49,11 @@ deb_clean_source() {
>   
>   do_deploy_source[depends] += "isar-apt:do_cache_config"
>   do_deploy_source[lockfiles] = "${REPO_ISAR_DIR}/isar.lock"
> -do_deploy_source[dirs] = "${S}"
> +do_deploy_source[dirs] = "${S} ${DEPLOY_DIR_SRC}"
>   do_deploy_source() {
>       repo_del_srcpackage "${REPO_ISAR_DIR}"/"${DISTRO}" \
>           "${REPO_ISAR_DB_DIR}"/"${DISTRO}" "${DEBDISTRONAME}" "${DEBIAN_SOURCE}"
> -    DSC_FILE=$(find ${WORKDIR} -maxdepth 1 -name "${DEBIAN_SOURCE}_*.dsc")
> +    DSC_FILE=$(find ${DEPLOY_DIR_SRC} -maxdepth 1 -name "${DEBIAN_SOURCE}_*.dsc")
>       if [ -n "${DSC_FILE}" ]; then
>           repo_add_srcpackage "${REPO_ISAR_DIR}"/"${DISTRO}" \
>               "${REPO_ISAR_DB_DIR}"/"${DISTRO}" \
> diff --git a/meta/classes-recipe/dpkg.bbclass b/meta/classes-recipe/dpkg.bbclass
> index d8e129bb..a42703d5 100644
> --- a/meta/classes-recipe/dpkg.bbclass
> +++ b/meta/classes-recipe/dpkg.bbclass
> @@ -116,7 +116,7 @@ dpkg_runbuild() {
>       echo '$apt_keep_downloaded_packages = 1;' >> ${SBUILD_CONFIG}
>       echo '$stalled_pkg_timeout = ${DPKG_BUILD_TIMEOUT};' >> ${SBUILD_CONFIG}
>   
> -    DSC_FILE=$(find ${WORKDIR} -maxdepth 1 -name "${DEBIAN_SOURCE}_*.dsc" -print)
> +    DSC_FILE=$(find ${DEPLOY_DIR_SRC} -maxdepth 1 -name "${DEBIAN_SOURCE}_*.dsc" -print)
>   
>       # networking is automatically enabled on older versions of sbuild
>       sbuild_network_option=""

Tested-by: Andreas Ziegler <ziegler.andreas@siemens.com>

Thanks!

Andreas

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/2f561e79-2776-4c70-b36b-7e4829884fd6%40siemens.com.

Re: [PATCH v2 2/2] dpkg: cache do_dpkg_source results in sstate

[Zhihang Wei]

On 8/6/26 09:24, 'Felix Moessbauer' via isar-users wrote:
> As the do_dpkg_source task now runs for each corresponding
> do_deploy_deb task, we also want to cache the generated results (to
> avoid pulling in a much bigger dependency tree on otherwise cached
> rebuilds).
>
> For that, we cache the .dsc and the .tar.* artifacts in the deploy dir.
> This is implemented similar to how do_dpkg_build artifacts (.debs) are
> cached, which is also compatible with a hash equivalence server.
>
> Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
> ---
>   RECIPE-API-CHANGELOG.md                 |  8 ++++++++
>   meta/classes-recipe/dpkg-source.bbclass | 19 +++++++++++++++++--
>   meta/classes-recipe/dpkg.bbclass        |  2 +-
>   3 files changed, 26 insertions(+), 3 deletions(-)
>
> diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md
> index 7e923e55..7386f5d1 100644
> --- a/RECIPE-API-CHANGELOG.md
> +++ b/RECIPE-API-CHANGELOG.md
> @@ -1146,6 +1146,14 @@ deploy directory `DEPLOY_DIR_DEB` instead of being deployed to the `WORKDIR`.
>   Recipes that accessed the built debs through `${WORKDIR}/*.deb` (e.g. to unpack an
>   artifact in a `do_deploy` task) must now reference `${DEPLOY_DIR_DEB}/*.deb` instead.
>   
> +### Debian source packages are deployed to `DEPLOY_DIR_SRC`
> +
> +The source packages produced by `do_dpkg_source` are now exported into a shared,
> +sstate-tracked deploy dir `DEPLOY_DIR_SRC` instead of being deployed to the `WORKDIR`.
> +
> +Recipes that access the source packages through `${WORKDIR}/*.tar.{gz,xz}` must now
> +reference the local copy in `${DEPLOY_DIR_SRC}` instead.
> +
>   ### Add Hyper-V machine support
>   
>   A new machine `hyper-v` has been introduced for building images
> diff --git a/meta/classes-recipe/dpkg-source.bbclass b/meta/classes-recipe/dpkg-source.bbclass
> index 92d06e1b..97cf9714 100644
> --- a/meta/classes-recipe/dpkg-source.bbclass
> +++ b/meta/classes-recipe/dpkg-source.bbclass
> @@ -13,7 +13,12 @@ TAR_REPRO_OPTS ?= "--exclude=.git --exclude=debian \
>   DPKG_SOURCE_EXTRA_ARGS ?= "-I"
>   
>   DEBIAN_SOURCE ?= "${BPN}"
> +SRCPKG_DIR = "${WORKDIR}/deploy-srcpkg"
> +DEPLOY_DIR_SRC = "${DEPLOY_DIR}/isar-source/${DISTRO}/${BPN}"
>   
> +do_dpkg_source[cleandirs] = "${SRCPKG_DIR}"
> +do_dpkg_source[sstate-inputdirs] = "${SRCPKG_DIR}"
> +do_dpkg_source[sstate-outputdirs] = "${DEPLOY_DIR_SRC}"
>   do_dpkg_source() {
>       # Create a .dsc file from source directory to use it with sbuild
>       DEB_SOURCE_NAME=$(dpkg-parsechangelog --show-field Source --file ${WORKDIR}/${PPS}/debian/changelog)
> @@ -22,9 +27,19 @@ do_dpkg_source() {
>       fi
>       find ${WORKDIR} -maxdepth 1 -name "${DEBIAN_SOURCE}_*.dsc" -delete
>       sh -c "cd ${WORKDIR}; dpkg-source ${DPKG_SOURCE_EXTRA_ARGS} -b ${PPS}"
> +    # move packages to deploy directory
> +    find ${WORKDIR} -maxdepth 1 \( -name "${DEBIAN_SOURCE}_*.tar.*" -o -name "${DEBIAN_SOURCE}_*.dsc" \) -exec mv {} ${SRCPKG_DIR}/ \;
>   }
>   addtask dpkg_source after do_prepare_build
>   
> +SSTATETASKS += "do_dpkg_source"
> +
> +python do_dpkg_source_setscene() {
> +    sstate_setscene(d)
> +}
> +
> +addtask dpkg_source_setscene
> +
>   CLEANFUNCS += "deb_clean_source"
>   
>   deb_clean_source() {
> @@ -34,11 +49,11 @@ deb_clean_source() {
>   
>   do_deploy_source[depends] += "isar-apt:do_cache_config"
>   do_deploy_source[lockfiles] = "${REPO_ISAR_DIR}/isar.lock"
> -do_deploy_source[dirs] = "${S}"
> +do_deploy_source[dirs] = "${S} ${DEPLOY_DIR_SRC}"
>   do_deploy_source() {
>       repo_del_srcpackage "${REPO_ISAR_DIR}"/"${DISTRO}" \
>           "${REPO_ISAR_DB_DIR}"/"${DISTRO}" "${DEBDISTRONAME}" "${DEBIAN_SOURCE}"
> -    DSC_FILE=$(find ${WORKDIR} -maxdepth 1 -name "${DEBIAN_SOURCE}_*.dsc")
> +    DSC_FILE=$(find ${DEPLOY_DIR_SRC} -maxdepth 1 -name "${DEBIAN_SOURCE}_*.dsc")
>       if [ -n "${DSC_FILE}" ]; then
>           repo_add_srcpackage "${REPO_ISAR_DIR}"/"${DISTRO}" \
>               "${REPO_ISAR_DB_DIR}"/"${DISTRO}" \
> diff --git a/meta/classes-recipe/dpkg.bbclass b/meta/classes-recipe/dpkg.bbclass
> index d8e129bb..a42703d5 100644
> --- a/meta/classes-recipe/dpkg.bbclass
> +++ b/meta/classes-recipe/dpkg.bbclass
> @@ -116,7 +116,7 @@ dpkg_runbuild() {
>       echo '$apt_keep_downloaded_packages = 1;' >> ${SBUILD_CONFIG}
>       echo '$stalled_pkg_timeout = ${DPKG_BUILD_TIMEOUT};' >> ${SBUILD_CONFIG}
>   
> -    DSC_FILE=$(find ${WORKDIR} -maxdepth 1 -name "${DEBIAN_SOURCE}_*.dsc" -print)
> +    DSC_FILE=$(find ${DEPLOY_DIR_SRC} -maxdepth 1 -name "${DEBIAN_SOURCE}_*.dsc" -print)
>   
>       # networking is automatically enabled on older versions of sbuild
>       sbuild_network_option=""

Hi Felix,

This test case builds containers for qemuamd64-bookworm and 
qemuarm64-bookworm
together using multiconfig. They can be built separately. But when they are
built together in this test case, the following error occurs:

[stdlog] 2026-08-07 09:00:54,845 avocado.app cibuilder L0371 ERROR| 
ERROR: mc:qemuamd64-bookworm:prebuilt-docker-img-0.1-r0 do_dpkg_build: 
ExecutionError('/build/isar_wzh_devel_1z_others/42/build/tmp/work/debian-bookworm-amd64/prebuilt-docker-img/0.1-r0/temp/run.dpkg_runbuild.399927', 
1, None, None)
[stdlog] 2026-08-07 09:00:54,845 avocado.app cibuilder L0371 ERROR| 
ERROR: Logfile of failure stored in: 
/build/isar_wzh_devel_1z_others/42/build/tmp/work/debian-bookworm-amd64/prebuilt-docker-img/0.1-r0/temp/log.do_dpkg_build.399927
[stdlog] 2026-08-07 09:00:54,845 avocado.test cibuilder L0369 INFO | Log 
data follows:

......
+------------------------------------------------------------------------------+
| Check architectures             |
+------------------------------------------------------------------------------+

Initial Foreign Architectures: i386
E: dsc: amd64 not in arch list or does not match any arch wildcards: 
arm64 -- skipping

+------------------------------------------------------------------------------+
| Cleanup             |
+------------------------------------------------------------------------------+

Purging /<<BUILDDIR>>
Not cleaning session: cloned chroot in use
E: dsc: amd64 not in arch list or does not match any arch wildcards: 
arm64 -- skipping

+------------------------------------------------------------------------------+
| Summary             |
+------------------------------------------------------------------------------+

Build Architecture: amd64
Build Type: binary
Build-Space: n/a
Build-Time: 0
Distribution: isar
Fail-Stage: arch-check
Foreign Architectures: i386
Host Architecture: amd64
Install-Time: 2
Job: 
/build/isar_wzh_devel_1z_others/42/build/tmp/deploy/isar-source/debian-bookworm/prebuilt-docker-img/prebuilt-docker-img_0.1.dsc
Machine Architecture: amd64
Package: prebuilt-docker-img
Package-Time: 0
Source-Version: 0.1
Space: n/a
Status: skipped
Version: 0.1
--------------------------------------------------------------------------------
Finished at 2026-08-07T07:00:50Z
Build needed 00:00:00, no disk space
E: dsc: amd64 not in arch list or does not match any arch wildcards: 
arm64 -- skipping
cat: 
/build/isar_wzh_devel_1z_others/42/build/tmp/work/debian-bookworm-amd64/prebuilt-docker-img/0.1-r0/rootfs/dpkg_partial.logWARNING: 
exit code 1 from a shell command.
: No such file or directory
DEBUG: Executing shell function dpkg_chroot_finalize

Zhihang

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/19e5d32e-e1b8-4860-884b-c05a87952006%40ilbers.de.

Re: [PATCH v2 2/2] dpkg: cache do_dpkg_source results in sstate

['MOESSBAUER, Felix' via isar-users]

On Fri, 2026-08-07 at 11:24 +0200, Zhihang Wei wrote:
> On 8/6/26 09:24, 'Felix Moessbauer' via isar-users wrote:
> > As the do_dpkg_source task now runs for each corresponding
> > do_deploy_deb task, we also want to cache the generated results (to
> > avoid pulling in a much bigger dependency tree on otherwise cached
> > rebuilds).
> > 
> > For that, we cache the .dsc and the .tar.* artifacts in the deploy dir.
> > This is implemented similar to how do_dpkg_build artifacts (.debs) are
> > cached, which is also compatible with a hash equivalence server.
> > 
> > Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
> > ---
> >   RECIPE-API-CHANGELOG.md                 |  8 ++++++++
> >   meta/classes-recipe/dpkg-source.bbclass | 19 +++++++++++++++++--
> >   meta/classes-recipe/dpkg.bbclass        |  2 +-
> >   3 files changed, 26 insertions(+), 3 deletions(-)
> > 
> > diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md
> > index 7e923e55..7386f5d1 100644
> > --- a/RECIPE-API-CHANGELOG.md
> > +++ b/RECIPE-API-CHANGELOG.md
> > @@ -1146,6 +1146,14 @@ deploy directory `DEPLOY_DIR_DEB` instead of being deployed to the `WORKDIR`.
> >   Recipes that accessed the built debs through `${WORKDIR}/*.deb` (e.g. to unpack an
> >   artifact in a `do_deploy` task) must now reference `${DEPLOY_DIR_DEB}/*.deb` instead.
> >   
> > +### Debian source packages are deployed to `DEPLOY_DIR_SRC`
> > +
> > +The source packages produced by `do_dpkg_source` are now exported into a shared,
> > +sstate-tracked deploy dir `DEPLOY_DIR_SRC` instead of being deployed to the `WORKDIR`.
> > +
> > +Recipes that access the source packages through `${WORKDIR}/*.tar.{gz,xz}` must now
> > +reference the local copy in `${DEPLOY_DIR_SRC}` instead.
> > +
> >   ### Add Hyper-V machine support
> >   
> >   A new machine `hyper-v` has been introduced for building images
> > diff --git a/meta/classes-recipe/dpkg-source.bbclass b/meta/classes-recipe/dpkg-source.bbclass
> > index 92d06e1b..97cf9714 100644
> > --- a/meta/classes-recipe/dpkg-source.bbclass
> > +++ b/meta/classes-recipe/dpkg-source.bbclass
> > @@ -13,7 +13,12 @@ TAR_REPRO_OPTS ?= "--exclude=.git --exclude=debian \
> >   DPKG_SOURCE_EXTRA_ARGS ?= "-I"
> >   
> >   DEBIAN_SOURCE ?= "${BPN}"
> > +SRCPKG_DIR = "${WORKDIR}/deploy-srcpkg"
> > +DEPLOY_DIR_SRC = "${DEPLOY_DIR}/isar-source/${DISTRO}/${BPN}"
> >   
> > +do_dpkg_source[cleandirs] = "${SRCPKG_DIR}"
> > +do_dpkg_source[sstate-inputdirs] = "${SRCPKG_DIR}"
> > +do_dpkg_source[sstate-outputdirs] = "${DEPLOY_DIR_SRC}"
> >   do_dpkg_source() {
> >       # Create a .dsc file from source directory to use it with sbuild
> >       DEB_SOURCE_NAME=$(dpkg-parsechangelog --show-field Source --file ${WORKDIR}/${PPS}/debian/changelog)
> > @@ -22,9 +27,19 @@ do_dpkg_source() {
> >       fi
> >       find ${WORKDIR} -maxdepth 1 -name "${DEBIAN_SOURCE}_*.dsc" -delete
> >       sh -c "cd ${WORKDIR}; dpkg-source ${DPKG_SOURCE_EXTRA_ARGS} -b ${PPS}"
> > +    # move packages to deploy directory
> > +    find ${WORKDIR} -maxdepth 1 \( -name "${DEBIAN_SOURCE}_*.tar.*" -o -name "${DEBIAN_SOURCE}_*.dsc" \) -exec mv {} ${SRCPKG_DIR}/ \;
> >   }
> >   addtask dpkg_source after do_prepare_build
> >   
> > +SSTATETASKS += "do_dpkg_source"
> > +
> > +python do_dpkg_source_setscene() {
> > +    sstate_setscene(d)
> > +}
> > +
> > +addtask dpkg_source_setscene
> > +
> >   CLEANFUNCS += "deb_clean_source"
> >   
> >   deb_clean_source() {
> > @@ -34,11 +49,11 @@ deb_clean_source() {
> >   
> >   do_deploy_source[depends] += "isar-apt:do_cache_config"
> >   do_deploy_source[lockfiles] = "${REPO_ISAR_DIR}/isar.lock"
> > -do_deploy_source[dirs] = "${S}"
> > +do_deploy_source[dirs] = "${S} ${DEPLOY_DIR_SRC}"
> >   do_deploy_source() {
> >       repo_del_srcpackage "${REPO_ISAR_DIR}"/"${DISTRO}" \
> >           "${REPO_ISAR_DB_DIR}"/"${DISTRO}" "${DEBDISTRONAME}" "${DEBIAN_SOURCE}"
> > -    DSC_FILE=$(find ${WORKDIR} -maxdepth 1 -name "${DEBIAN_SOURCE}_*.dsc")
> > +    DSC_FILE=$(find ${DEPLOY_DIR_SRC} -maxdepth 1 -name "${DEBIAN_SOURCE}_*.dsc")
> >       if [ -n "${DSC_FILE}" ]; then
> >           repo_add_srcpackage "${REPO_ISAR_DIR}"/"${DISTRO}" \
> >               "${REPO_ISAR_DB_DIR}"/"${DISTRO}" \
> > diff --git a/meta/classes-recipe/dpkg.bbclass b/meta/classes-recipe/dpkg.bbclass
> > index d8e129bb..a42703d5 100644
> > --- a/meta/classes-recipe/dpkg.bbclass
> > +++ b/meta/classes-recipe/dpkg.bbclass
> > @@ -116,7 +116,7 @@ dpkg_runbuild() {
> >       echo '$apt_keep_downloaded_packages = 1;' >> ${SBUILD_CONFIG}
> >       echo '$stalled_pkg_timeout = ${DPKG_BUILD_TIMEOUT};' >> ${SBUILD_CONFIG}
> >   
> > -    DSC_FILE=$(find ${WORKDIR} -maxdepth 1 -name "${DEBIAN_SOURCE}_*.dsc" -print)
> > +    DSC_FILE=$(find ${DEPLOY_DIR_SRC} -maxdepth 1 -name "${DEBIAN_SOURCE}_*.dsc" -print)
> >   
> >       # networking is automatically enabled on older versions of sbuild
> >       sbuild_network_option=""
> 
> Hi Felix,
> 
> This test case builds containers for qemuamd64-bookworm and 
> qemuarm64-bookworm
> together using multiconfig. They can be built separately. But when they are
> built together in this test case, the following error occurs:
> 
> [stdlog] 2026-08-07 09:00:54,845 avocado.app cibuilder L0371 ERROR| 
> ERROR: mc:qemuamd64-bookworm:prebuilt-docker-img-0.1-r0 do_dpkg_build: 
> ExecutionError('/build/isar_wzh_devel_1z_others/42/build/tmp/work/debian-bookworm-amd64/prebuilt-docker-img/0.1-r0/temp/run.dpkg_runbuild.399927', 
> 1, None, None)
> [stdlog] 2026-08-07 09:00:54,845 avocado.app cibuilder L0371 ERROR| 
> ERROR: Logfile of failure stored in: 
> /build/isar_wzh_devel_1z_others/42/build/tmp/work/debian-bookworm-amd64/prebuilt-docker-img/0.1-r0/temp/log.do_dpkg_build.399927
> [stdlog] 2026-08-07 09:00:54,845 avocado.test cibuilder L0369 INFO | Log 
> data follows:

Hi, thanks for the logs. This patch series seems to expose the
underlying bug: We build the same source package with different content
for multiple architectures. Even before this patch, this silently raced
in isar-apt, as only last-built .dsc file was added there. Now it races
on the deb-source deploy dir, which exposes the bug reliably.

When building a source package that varies in content across
architectures, we must add the architecture to its name. I will send a
patch for that.

Note, that in general it is problematic to add DPKG_ARCH != {all, any},
but for cases like firmware, where anyways just one architecture is
built, it does not matter.

Felix

> ......
> +------------------------------------------------------------------------------+
> | Check architectures             |
> +------------------------------------------------------------------------------+
> 
> Initial Foreign Architectures: i386
> E: dsc: amd64 not in arch list or does not match any arch wildcards: 
> arm64 -- skipping
> 
> +------------------------------------------------------------------------------+
> | Cleanup             |
> +------------------------------------------------------------------------------+
> 
> Purging /<<BUILDDIR>>
> Not cleaning session: cloned chroot in use
> E: dsc: amd64 not in arch list or does not match any arch wildcards: 
> arm64 -- skipping
> 
> +------------------------------------------------------------------------------+
> | Summary             |
> +------------------------------------------------------------------------------+
> 
> Build Architecture: amd64
> Build Type: binary
> Build-Space: n/a
> Build-Time: 0
> Distribution: isar
> Fail-Stage: arch-check
> Foreign Architectures: i386
> Host Architecture: amd64
> Install-Time: 2
> Job: 
> /build/isar_wzh_devel_1z_others/42/build/tmp/deploy/isar-source/debian-bookworm/prebuilt-docker-img/prebuilt-docker-img_0.1.dsc
> Machine Architecture: amd64
> Package: prebuilt-docker-img
> Package-Time: 0
> Source-Version: 0.1
> Space: n/a
> Status: skipped
> Version: 0.1
> --------------------------------------------------------------------------------
> Finished at 2026-08-07T07:00:50Z
> Build needed 00:00:00, no disk space
> E: dsc: amd64 not in arch list or does not match any arch wildcards: 
> arm64 -- skipping
> cat: 
> /build/isar_wzh_devel_1z_others/42/build/tmp/work/debian-bookworm-amd64/prebuilt-docker-img/0.1-r0/rootfs/dpkg_partial.logWARNING: 
> exit code 1 from a shell command.
> : No such file or directory
> DEBUG: Executing shell function dpkg_chroot_finalize
> 
> Zhihang

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/7c2e0d895fad8086b239c709820269c1a4ea265e.camel%40siemens.com.