[RFC PATCH 1/1] cleanup lockfile in downloads dir after build

[RFC PATCH 1/1] cleanup lockfile in downloads dir after build

['Felix Moessbauer' via isar-users]

We currently leave a lockfile in the downloads dir behind which is even
owned by root. This is unfortunate, as the downloads dir cannot easily
be cached (uploaded / downloaded) with that lockfile in place.

As the lockfile is created by independent tasks, we add a handler to
remove it after the build.

Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
---
Note, that the removal is only safe if the downloads dir is not shared
across multiple parallel bitbake executions. I don't know if that is
allowed at all, so I'm sending this patch as a RFC.

Best regards,
Felix Moessbauer
Siemens AG

 meta/classes/isar-events.bbclass | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/meta/classes/isar-events.bbclass b/meta/classes/isar-events.bbclass
index 76b713c7..4101e756 100644
--- a/meta/classes/isar-events.bbclass
+++ b/meta/classes/isar-events.bbclass
@@ -47,6 +47,7 @@ addhandler build_completed
 
 python build_completed() {
     import subprocess
+    import os
 
     tmpdir = d.getVar('TMPDIR')
     if not tmpdir:
@@ -84,6 +85,12 @@ python build_completed() {
             except subprocess.CalledProcessError as e:
                 bb.error(str(e))
 
+    # cleanup downloads lock
+    dl_lock = "{}/deb/{}-{}.lock".format(d.getVar('DL_DIR'), d.getVar('BASE_DISTRO'), d.getVar('BASE_DISTRO_CODENAME'))
+    if os.path.exists(dl_lock):
+        bb.debug('cleaning up downloads lock: ', dl_lock)
+        subprocess.run(f"sudo rm {dl_lock}", shell=True, check=True)
+
     # Cleanup build UUID, the next bitbake run will generate new one
     bb.persist_data.persist('BB_ISAR_UUID_DATA', d).clear()
 }
-- 
2.51.0

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/20250925095155.1095671-1-felix.moessbauer%40siemens.com.

Re: [RFC PATCH 1/1] cleanup lockfile in downloads dir after build

['Jan Kiszka' via isar-users]

On 25.09.25 11:51, 'Felix Moessbauer' via isar-users wrote:
> We currently leave a lockfile in the downloads dir behind which is even
> owned by root. This is unfortunate, as the downloads dir cannot easily
> be cached (uploaded / downloaded) with that lockfile in place.
> 
> As the lockfile is created by independent tasks, we add a handler to
> remove it after the build.
> 
> Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
> ---
> Note, that the removal is only safe if the downloads dir is not shared
> across multiple parallel bitbake executions. I don't know if that is
> allowed at all, so I'm sending this patch as a RFC.

I think this is possible, so purging the lock file by one build while
another one is still using it would not be a good idea.

But what we now have a is one-time glitch when coming from a
pre-bubblewrap time where the lock file was root-owned. That is failing
when switching gears, and you either have to change ownership manually
once or purge the cache when that is not directly accessible (CI...).

Jan

> 
> Best regards,
> Felix Moessbauer
> Siemens AG
> 
>  meta/classes/isar-events.bbclass | 7 +++++++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/meta/classes/isar-events.bbclass b/meta/classes/isar-events.bbclass
> index 76b713c7..4101e756 100644
> --- a/meta/classes/isar-events.bbclass
> +++ b/meta/classes/isar-events.bbclass
> @@ -47,6 +47,7 @@ addhandler build_completed
>  
>  python build_completed() {
>      import subprocess
> +    import os
>  
>      tmpdir = d.getVar('TMPDIR')
>      if not tmpdir:
> @@ -84,6 +85,12 @@ python build_completed() {
>              except subprocess.CalledProcessError as e:
>                  bb.error(str(e))
>  
> +    # cleanup downloads lock
> +    dl_lock = "{}/deb/{}-{}.lock".format(d.getVar('DL_DIR'), d.getVar('BASE_DISTRO'), d.getVar('BASE_DISTRO_CODENAME'))
> +    if os.path.exists(dl_lock):
> +        bb.debug('cleaning up downloads lock: ', dl_lock)
> +        subprocess.run(f"sudo rm {dl_lock}", shell=True, check=True)
> +
>      # Cleanup build UUID, the next bitbake run will generate new one
>      bb.persist_data.persist('BB_ISAR_UUID_DATA', d).clear()
>  }

-- 
Siemens AG, Foundational Technologies
Linux Expert Center

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/e5d2b289-a642-4a5f-89c0-fa07b89af2c1%40siemens.com.