Re: [PATCH] fix: add currenttask-guards to SSTATE(CREATE|POSTINST)FUNCS

[Baurzhan Ismagulov <ibr@radix50.net>]

On 2025-04-11 16:33, 'Adriaan Schmidt' via isar-users wrote:
> We use SSTATECREATEFUNCS and SSTATEPOSTINSTFUNCS to insert
> code for preparation/postprocessing of sstate artifacts.
> As it turns out, those are not per-job, but global, so if
> a recipe has multiple sstate-cacheable tasks, all of those
> functions are executed, and we need to make sure to bail out
> of all unintended calls.

Thanks Adriaan for the patch,

we've had a look and have a few comments (numbering for easier referencing):

1. For this use case, I'd like to have a testcase. Would you like to submit one
   or provide quick hints what to add to trigger the behavior? We've started
   checking some simple cases but haven't reproduced it so far.


> --- a/meta/classes/base.bbclass
> +++ b/meta/classes/base.bbclass
> @@ -204,6 +204,15 @@ def isar_export_ccache(d):
>      else:
>          os.environ['CCACHE_DISABLE'] = '1'
>  
> +isar_check_current_task() {
> +    if [ "do_${BB_CURRENTTASK}" = "$1" ]; then
> +        return 0
> +    else
> +        echo "skipping because current task is not $1"
> +        return 1
> +    fi
> +}

2. Could we name "isar_check_current_task" to reflect when it returns what?
   E.g., "fail_if_task_not" or something. (Minor but easy and helpful)

3. Technically, this function itself doesn't skip, so the message might be
   misleading if the caller makes a mistake. (Just a comment; I don't have a
   better wording on the spot)


> --- a/meta/classes/dpkg-base.bbclass
> +++ b/meta/classes/dpkg-base.bbclass
> @@ -184,6 +184,7 @@ SSTATECREATEFUNCS += "dpkg_build_sstate_prepare"
>  SSTATEPOSTINSTFUNCS += "dpkg_build_sstate_finalize"
>  
>  dpkg_build_sstate_prepare() {
> +    (isar_check_current_task "do_dpkg_build" || return) || true

4. If isar_check_current_task succeeds, the execution should continue. If it
   fails, we return. Why is || true needed? (Just double-checking)

   Coincidentally, many || true occurrences needed fixing in the past, so I'm
   not impartial towards those :) , although it's just a handier form of "if".

5. I wonder if there are ways to remove hard-coding of "do_dpkg_build" into the
   function (something similar to overrides, flags, etc.). (Just brainstorming)

   (I've always wanted to separate hard-coded cases like "i386", "stretch",
   etc. from the code = separate generic mechanism from the specific machine,
   distro, etc. -- haven't come to that so far)


6. Just for my understanding, what are some examples where *CREATE* and
   *POSTINST* are useful?

7. Given that this touches central classes and sstate bugs could silently
   produce incorrect deliverables, what are the opinions regarding applying
   this after the 0.11 release? The use case doesn't sound (to me, in any case)
   like it would affect the most downstreams, and they are using next anyway.


With kind regards,
Baurzhan


> 
> Signed-off-by: Adriaan Schmidt <adriaan.schmidt@siemens.com>
> ---
>  meta/classes/base.bbclass                             | 9 +++++++++
>  meta/classes/dpkg-base.bbclass                        | 2 ++
>  meta/classes/rootfs.bbclass                           | 2 ++
>  meta/recipes-core/isar-bootstrap/isar-bootstrap.inc   | 2 ++
>  meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc | 2 ++
>  5 files changed, 17 insertions(+)
> 
> diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
> index c730eec9..d5a3142b 100644
> --- a/meta/classes/base.bbclass
> +++ b/meta/classes/base.bbclass
> @@ -204,6 +204,15 @@ def isar_export_ccache(d):
>      else:
>          os.environ['CCACHE_DISABLE'] = '1'
>  
> +isar_check_current_task() {
> +    if [ "do_${BB_CURRENTTASK}" = "$1" ]; then
> +        return 0
> +    else
> +        echo "skipping because current task is not $1"
> +        return 1
> +    fi
> +}
> +
>  do_fetch[dirs] = "${DL_DIR}"
>  do_fetch[file-checksums] = "${@bb.fetch.get_checksum_file_list(d)}"
>  do_fetch[vardeps] += "SRCREV"
> diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass
> index 4468a49a..7f2dcd3f 100644
> --- a/meta/classes/dpkg-base.bbclass
> +++ b/meta/classes/dpkg-base.bbclass
> @@ -184,6 +184,7 @@ SSTATECREATEFUNCS += "dpkg_build_sstate_prepare"
>  SSTATEPOSTINSTFUNCS += "dpkg_build_sstate_finalize"
>  
>  dpkg_build_sstate_prepare() {
> +    (isar_check_current_task "do_dpkg_build" || return) || true
>      # this runs in SSTATE_BUILDDIR, which will be deleted automatically
>      if [ -n "$(find ${WORKDIR} -maxdepth 1 -name '*.deb' -print -quit)" ]; then
>          cp -f ${WORKDIR}/*.deb -t .
> @@ -191,6 +192,7 @@ dpkg_build_sstate_prepare() {
>  }
>  
>  dpkg_build_sstate_finalize() {
> +    (isar_check_current_task "do_dpkg_build_setscene" || return) || true
>      # this runs in SSTATE_INSTDIR
>      if [ -n "$(find . -maxdepth 1 -name '*.deb' -print -quit)" ]; then
>          mv -f ./*.deb -t ${WORKDIR}/
> diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass
> index cd8fd118..aed50a91 100644
> --- a/meta/classes/rootfs.bbclass
> +++ b/meta/classes/rootfs.bbclass
> @@ -437,6 +437,7 @@ SSTATE_TAR_ATTR_FLAGS ?= "--xattrs --xattrs-include='*'"
>  
>  # the rootfs is owned by root, so we need some sudoing to pack and unpack
>  rootfs_install_sstate_prepare() {
> +    (isar_check_current_task "do_rootfs_install" || return) || true
>      # this runs in SSTATE_BUILDDIR, which will be deleted automatically
>      # tar --one-file-system will cross bind-mounts to the same filesystem,
>      # so we use some mount magic to prevent that
> @@ -450,6 +451,7 @@ rootfs_install_sstate_prepare() {
>  do_rootfs_install_sstate_prepare[lockfiles] = "${REPO_ISAR_DIR}/isar.lock"
>  
>  rootfs_install_sstate_finalize() {
> +    (isar_check_current_task "do_rootfs_install_setscene" || return) || true
>      # this runs in SSTATE_INSTDIR
>      # - after building the rootfs, the tar won't be there, but we also don't need to unpack
>      # - after restoring from cache, there will be a tar which we unpack and then delete
> diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
> index 08d61a84..a3ab28ca 100644
> --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
> +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
> @@ -219,12 +219,14 @@ SSTATECREATEFUNCS += "bootstrap_sstate_prepare"
>  SSTATEPOSTINSTFUNCS += "bootstrap_sstate_finalize"
>  
>  bootstrap_sstate_prepare() {
> +    (isar_check_current_task "do_bootstrap" || return) || true
>      # this runs in SSTATE_BUILDDIR, which will be deleted automatically
>      sudo cp -a "${WORKDIR}/rootfs.tar.zst" ./bootstrap.tar.zst
>      sudo chown $(id -u):$(id -g) bootstrap.tar.zst
>  }
>  
>  bootstrap_sstate_finalize() {
> +    (isar_check_current_task "do_bootstrap_setscene" || return) || true
>      # this runs in SSTATE_INSTDIR
>      # we should restore symlinks after using tar
>      if [ -f bootstrap.tar.zst ]; then
> diff --git a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc
> index 931f6f13..44775bc5 100644
> --- a/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc
> +++ b/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap.inc
> @@ -222,12 +222,14 @@ SSTATECREATEFUNCS += "bootstrap_sstate_prepare"
>  SSTATEPOSTINSTFUNCS += "bootstrap_sstate_finalize"
>  
>  bootstrap_sstate_prepare() {
> +    (isar_check_current_task "do_bootstrap" || return) || true
>      # this runs in SSTATE_BUILDDIR, which will be deleted automatically
>      sudo cp -a "${WORKDIR}/rootfs.tar.zst" ./bootstrap.tar.zst
>      sudo chown $(id -u):$(id -g) bootstrap.tar.zst
>  }
>  
>  bootstrap_sstate_finalize() {
> +    (isar_check_current_task "do_bootstrap_setscene" || return) || true
>      # this runs in SSTATE_INSTDIR
>      # we should restore symlinks after using tar
>      if [ -f bootstrap.tar.zst ]; then
> -- 
> 2.39.5

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/Z_-663VgehHkoczP%40abai.de.