[PATCH 0/3] Rootfs install race fix for isar-apt packages

[PATCH 0/3] Rootfs install race fix for isar-apt packages

[Anton Mikanovich]

After merging isar apt for different architecures in 50d641e it unlocks
a race during isar-apt packages installation into rootfs. If some
package performs do_deploy_deb task during do_rootfs_install execution
it can remove its deb in the moment when it read by apt-get.

This happens because our existing logic for precaching debs under
isar-apt lock was not working for isar-apt packages, because apt-get
just do not cache packages from local repos in /var/cache/apt/archives.

To fix this, introduce a new rootfs install command which obtains all
the packages not been cached and download them manually.

Also add "--no-download" option to the final apt-get install call to
prevent masking such an issues in the future.

Anton Mikanovich (3):
  rootfs: Allow locking on single install command
  rootfs: Download isar-apt packages under isar-apt lock
  rootfs: Deny packages download during install

 meta/classes-recipe/rootfs.bbclass | 29 ++++++++++++++++++++++++++---
 1 file changed, 26 insertions(+), 3 deletions(-)

-- 
2.34.1

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20260610064812.4010511-1-amikan%40ilbers.de.

[PATCH 1/3] rootfs: Allow locking on single install command

[Anton Mikanovich]

In some cases only one command from all the ROOTFS_INSTALL_COMMAND list
can require isar-apt locking. To allow this logic improve flag checking
to accept the list of codenames. Existing API and codenames
("acquire-before" and "release-after") were not changed.

So the usage of single task declaration now looks like:

rootfs_install_cmd[isar-apt-lock] = "acquire-before release-after"

Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
 meta/classes-recipe/rootfs.bbclass | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/classes-recipe/rootfs.bbclass b/meta/classes-recipe/rootfs.bbclass
index 8b502a50..dc4e702c 100644
--- a/meta/classes-recipe/rootfs.bbclass
+++ b/meta/classes-recipe/rootfs.bbclass
@@ -427,13 +427,13 @@ python do_rootfs_install() {
         for cmd in cmds:
             progress_reporter.next_stage()
 
-            if (d.getVarFlag(cmd, 'isar-apt-lock') or "") == "acquire-before":
+            if "acquire-before" in (d.getVarFlag(cmd, 'isar-apt-lock') or ""):
                 lock = bb.utils.lockfile(d.getVar("REPO_ISAR_DIR") + "/isar.lock",
                                          shared=True)
 
             bb.build.exec_func(cmd, d)
 
-            if (d.getVarFlag(cmd, 'isar-apt-lock') or "") == "release-after":
+            if "release-after" in (d.getVarFlag(cmd, 'isar-apt-lock') or ""):
                 bb.utils.unlockfile(lock)
             progress_reporter.finish()
     finally:
-- 
2.34.1

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20260610064812.4010511-2-amikan%40ilbers.de.

[PATCH 2/3] rootfs: Download isar-apt packages under isar-apt lock

[Anton Mikanovich]

It turns out apt-get install do not cache debs from local repos
(started with file:/) in /var/cache/apt/archives even with
--download-only option enabled. It results in isar-apt packages
download execution during rootfs_install_pkgs_install command, which is
not covered under isar-apt lock.
To put isar-apt packages in local cache under isar-apt lock new command
was introduced. It obtain all the packages not been cached and download
them manually.

This change fixes "file not found" issues during rootfs installation.

Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
 meta/classes-recipe/rootfs.bbclass | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)

diff --git a/meta/classes-recipe/rootfs.bbclass b/meta/classes-recipe/rootfs.bbclass
index dc4e702c..81384340 100644
--- a/meta/classes-recipe/rootfs.bbclass
+++ b/meta/classes-recipe/rootfs.bbclass
@@ -363,6 +363,29 @@ rootfs_export_package_cache() {
     deb_dl_dir_export ${ROOTFSDIR} ${ROOTFS_BASE_DISTRO}-${BASE_DISTRO_CODENAME}
 }
 
+ROOTFS_INSTALL_COMMAND += "rootfs_install_pkgs_isar_download"
+rootfs_install_pkgs_isar_download[weight] = "50"
+rootfs_install_pkgs_isar_download[isar-apt-lock] = "acquire-before release-after"
+rootfs_install_pkgs_isar_download() {
+    mkdir -p "${WORKDIR}/dpkg"
+
+    # Use our own dpkg lock files rather than those in the rootfs since we are not root
+    # (this is safe as there are no concurrent apt/dpkg operations for that rootfs)
+    touch "${WORKDIR}/dpkg/lock" "${WORKDIR}/dpkg/lock-frontend"
+
+    # Command apt-get install do not cache packages from local repos
+    # We can obtain non cached package URIs by recalling install command here
+    # No need in export those files to dl_dir, so we can run it right after
+    rootfs_cmd --bind "${ROOTFSDIR}/var/cache/apt/archives" /var/cache/apt/archives \
+               --bind "${WORKDIR}/dpkg/lock" /var/lib/dpkg/lock \
+               --bind "${WORKDIR}/dpkg/lock-frontend" /var/lib/dpkg/lock-frontend \
+               --chdir "/var/cache/apt/archives" \
+               ${ROOTFSDIR} \
+               -- /usr/bin/sh -c "apt-get ${ROOTFS_APT_ARGS} --print-uris ${ROOTFS_PACKAGES} | \
+                                  sed -n \"s|^.*/\\(.*\\)_[^_]*_[^_]*\\.deb'.*|\\1|p\" | \
+                                  xargs -r apt-get download"
+}
+
 ROOTFS_INSTALL_COMMAND += "${@ 'rootfs_install_clean_files' if (d.getVar('ROOTFS_CLEAN_FILES') or '').strip() else ''}"
 rootfs_install_clean_files[weight] = "2"
 rootfs_install_clean_files() {
-- 
2.34.1

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20260610064812.4010511-3-amikan%40ilbers.de.

[PATCH 3/3] rootfs: Deny packages download during install

[Anton Mikanovich]

During the command rootfs_install_pkgs_install no actuall packages
downloading should be done. All the debs must be precached and already
placed in /var/cache/apt/archives, so apt-get can reuse them.

Currently any downloads in this command are really succeed because we
still have "network" flag on for using sudo, which masks all the
caching issues in previous rootfs install commands.

To prevent such a cases add "--no-download" option to apt-get.

Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
 meta/classes-recipe/rootfs.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes-recipe/rootfs.bbclass b/meta/classes-recipe/rootfs.bbclass
index 81384340..f4580421 100644
--- a/meta/classes-recipe/rootfs.bbclass
+++ b/meta/classes-recipe/rootfs.bbclass
@@ -402,7 +402,7 @@ rootfs_install_pkgs_install[progress] = "custom:rootfs_progress.PkgsInstallProgr
 rootfs_install_pkgs_install[network] = "${TASK_USE_SUDO}"
 rootfs_install_pkgs_install() {
     sudo -E chroot "${ROOTFSDIR}" \
-        /usr/bin/apt-get ${ROOTFS_APT_ARGS} ${ROOTFS_PACKAGES}
+        /usr/bin/apt-get ${ROOTFS_APT_ARGS} --no-download ${ROOTFS_PACKAGES}
 }
 
 ROOTFS_INSTALL_COMMAND += "rootfs_restore_initrd_tooling"
-- 
2.34.1

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20260610064812.4010511-4-amikan%40ilbers.de.

Re: [PATCH 1/3] rootfs: Allow locking on single install command

['MOESSBAUER, Felix' via isar-users]

On Wed, 2026-06-10 at 09:48 +0300, Anton Mikanovich wrote:
> In some cases only one command from all the ROOTFS_INSTALL_COMMAND list
> can require isar-apt locking. To allow this logic improve flag checking
> to accept the list of codenames. Existing API and codenames
> ("acquire-before" and "release-after") were not changed.
> 
> So the usage of single task declaration now looks like:
> 
> rootfs_install_cmd[isar-apt-lock] = "acquire-before release-after"

Hi, does it even make sense to do an acquire release in two different
commands? To me this looks like a source for deadlocks.

Felix

> 
> Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
> ---
>  meta/classes-recipe/rootfs.bbclass | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/meta/classes-recipe/rootfs.bbclass b/meta/classes-recipe/rootfs.bbclass
> index 8b502a50..dc4e702c 100644
> --- a/meta/classes-recipe/rootfs.bbclass
> +++ b/meta/classes-recipe/rootfs.bbclass
> @@ -427,13 +427,13 @@ python do_rootfs_install() {
>          for cmd in cmds:
>              progress_reporter.next_stage()
>  
> -            if (d.getVarFlag(cmd, 'isar-apt-lock') or "") == "acquire-before":
> +            if "acquire-before" in (d.getVarFlag(cmd, 'isar-apt-lock') or ""):
>                  lock = bb.utils.lockfile(d.getVar("REPO_ISAR_DIR") + "/isar.lock",
>                                           shared=True)
>  
>              bb.build.exec_func(cmd, d)
>  
> -            if (d.getVarFlag(cmd, 'isar-apt-lock') or "") == "release-after":
> +            if "release-after" in (d.getVarFlag(cmd, 'isar-apt-lock') or ""):
>                  bb.utils.unlockfile(lock)
>              progress_reporter.finish()
>      finally:
> -- 
> 2.34.1
> 
> -- 
> You received this message because you are subscribed to the Google Groups "isar-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
> To view this discussion visit https://groups.google.com/d/msgid/isar-users/20260610064812.4010511-2-amikan%40ilbers.de.

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/a6075cf2edd68a30f1927a2859271da27acc13e9.camel%40siemens.com.

Re: [PATCH 1/3] rootfs: Allow locking on single install command

[Anton Mikanovich]

10.06.2026 11:23, MOESSBAUER, Felix wrote:
> On Wed, 2026-06-10 at 09:48 +0300, Anton Mikanovich wrote:
>> In some cases only one command from all the ROOTFS_INSTALL_COMMAND list
>> can require isar-apt locking. To allow this logic improve flag checking
>> to accept the list of codenames. Existing API and codenames
>> ("acquire-before" and "release-after") were not changed.
>>
>> So the usage of single task declaration now looks like:
>>
>> rootfs_install_cmd[isar-apt-lock] = "acquire-before release-after"
> Hi, does it even make sense to do an acquire release in two different
> commands? To me this looks like a source for deadlocks.
>
> Felix
Current implementation have 4 tasks combined in one lock section.
This is done to be sure isar-apt will not changed between "apt-get update"
and "apt-get install" commands. This probably can be reordered and 
optimized,
but it will be out of scope for the current patchset.

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/0d561b30-d3d1-4a26-ae45-16e61775ca68%40ilbers.de.

Re: [PATCH 1/3] rootfs: Allow locking on single install command

['MOESSBAUER, Felix' via isar-users]

On Thu, 2026-06-11 at 10:01 +0300, Anton Mikanovich wrote:
> 10.06.2026 11:23, MOESSBAUER, Felix wrote:
> > On Wed, 2026-06-10 at 09:48 +0300, Anton Mikanovich wrote:
> > > In some cases only one command from all the ROOTFS_INSTALL_COMMAND list
> > > can require isar-apt locking. To allow this logic improve flag checking
> > > to accept the list of codenames. Existing API and codenames
> > > ("acquire-before" and "release-after") were not changed.
> > > 
> > > So the usage of single task declaration now looks like:
> > > 
> > > rootfs_install_cmd[isar-apt-lock] = "acquire-before release-after"
> > Hi, does it even make sense to do an acquire release in two different
> > commands? To me this looks like a source for deadlocks.
> > 
> > Felix
> Current implementation have 4 tasks combined in one lock section.
> This is done to be sure isar-apt will not changed between "apt-get update"
> and "apt-get install" commands. This probably can be reordered and 
> optimized,
> but it will be out of scope for the current patchset.

Makes sense. Then the patchset is fine.

Reviewed-by: Felix Moessbauer <felix.moessbauer@siemens.com>

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/e4057d6795a7c56c61b54ff605fb27e72a3f3f4e.camel%40siemens.com.

Re: [PATCH 0/3] Rootfs install race fix for isar-apt packages

[Anton Mikanovich]

On 10/06/2026 09:48, Anton Mikanovich wrote:
> After merging isar apt for different architecures in 50d641e it unlocks
> a race during isar-apt packages installation into rootfs. If some
> package performs do_deploy_deb task during do_rootfs_install execution
> it can remove its deb in the moment when it read by apt-get.
>
> This happens because our existing logic for precaching debs under
> isar-apt lock was not working for isar-apt packages, because apt-get
> just do not cache packages from local repos in /var/cache/apt/archives.
>
> To fix this, introduce a new rootfs install command which obtains all
> the packages not been cached and download them manually.
>
> Also add "--no-download" option to the final apt-get install call to
> prevent masking such an issues in the future.
>
> Anton Mikanovich (3):
>    rootfs: Allow locking on single install command
>    rootfs: Download isar-apt packages under isar-apt lock
>    rootfs: Deny packages download during install
>
>   meta/classes-recipe/rootfs.bbclass | 29 ++++++++++++++++++++++++++---
>   1 file changed, 26 insertions(+), 3 deletions(-)
>
Applied to next.

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/f5190047-da9e-458c-b1ac-3f784aa50f2e%40ilbers.de.