[PATCH] rootfs: Take isar-apt shared lock during repository

[PATCH] rootfs: Take isar-apt shared lock during repository

[Jan Kiszka]

From: Jan Kiszka <jan.kiszka@siemens.com>

We take this lock already for install_imager_deps, but we also need it
conceptually for the rootfs installation. Also here, pulling from
isar-apt can race with not-yet finished deployments of parallel builds,
even if the target package is not shared. See e9a5d8adf6b5 for further
details.

To avoid holding the lock across the whole lengthy package installation,
introduce command flags that define the beginning and the end of the
locked section so that do_rootfs_install can manage that.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 meta/classes/rootfs.bbclass | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass
index c7e0435..314efd7 100644
--- a/meta/classes/rootfs.bbclass
+++ b/meta/classes/rootfs.bbclass
@@ -93,6 +93,7 @@ EOSUDO
 
 ROOTFS_INSTALL_COMMAND += "rootfs_install_pkgs_update"
 rootfs_install_pkgs_update[weight] = "5"
+rootfs_install_pkgs_update[isar-lock] = "acquire-before"
 rootfs_install_pkgs_update() {
     sudo -E chroot '${ROOTFSDIR}' /usr/bin/apt-get update \
         -o Dir::Etc::sourcelist="sources.list.d/isar-apt.list" \
@@ -108,6 +109,7 @@ rootfs_install_resolvconf() {
 
 ROOTFS_INSTALL_COMMAND += "rootfs_install_pkgs_download"
 rootfs_install_pkgs_download[weight] = "600"
+rootfs_install_pkgs_download[isar-lock] = "release-after"
 rootfs_install_pkgs_download() {
     sudo -E chroot '${ROOTFSDIR}' \
         /usr/bin/apt-get ${ROOTFS_APT_ARGS} --download-only ${ROOTFS_PACKAGES}
@@ -153,7 +155,15 @@ python do_rootfs_install() {
 
     for cmd in cmds:
         progress_reporter.next_stage()
+
+        if (d.getVarFlag(cmd, 'isar-lock') or "") == "acquire-before":
+            lock = bb.utils.lockfile(d.getVar("REPO_ISAR_DIR") + "/isar.lock",
+                                     shared=True)
+
         bb.build.exec_func(cmd, d)
+
+        if (d.getVarFlag(cmd, 'isar-lock') or "") == "release-after":
+            bb.utils.unlockfile(lock)
     progress_reporter.finish()
 }
 addtask rootfs_install before do_rootfs_postprocess after do_unpack
-- 

2.16.4

Re: [PATCH] rootfs: Take isar-apt shared lock during repository

[Baurzhan Ismagulov]

On Tue, Sep 03, 2019 at 07:41:40AM +0200, Jan Kiszka wrote:
> We take this lock already for install_imager_deps, but we also need it
> conceptually for the rootfs installation. Also here, pulling from
> isar-apt can race with not-yet finished deployments of parallel builds,
> even if the target package is not shared. See e9a5d8adf6b5 for further
> details.
> 
> To avoid holding the lock across the whole lengthy package installation,
> introduce command flags that define the beginning and the end of the
> locked section so that do_rootfs_install can manage that.

Looks good to me.

The title "rootfs: Take isar-apt shared lock during repository" sounds
incomplete. Do you mean something like "rootfs: Take shared lock reading
isar-apt"?

With kind regards,
Baurzhan.