Re: schroot umount target is busy

["'Jan Kiszka' via isar-users" <isar-users@googlegroups.com>]

On 08.07.24 16:17, Gökhan Çetin wrote:
> Hi everyone,
> 
> Similarly, we have been observing do_image tasks fail sporadically in
> CI/CD pipelines for a while.
> You can see a few examples below:
> 
> INFO - | E: 10mount: umount:
> /run/schroot/mount/isar-builder-31888f6e-6228-4bc8-9570-dd4d69ad25ba-367048-331187ba-b359-48ad-91f4-2962f4bdf32b: target is busy.
> INFO - | E: 10mount: rmdir: failed to remove
> '/var/run/schroot/mount/isar-builder-31888f6e-6228-4bc8-9570-dd4d69ad25ba-367048-331187ba-b359-48ad-91f4-2962f4bdf32b': Device or resource busy
> INFO - | E:
> isar-builder-31888f6e-6228-4bc8-9570-dd4d69ad25ba-367048-331187ba-b359-48ad-91f4-2962f4bdf32b: Chroot setup failed: stage=setup-stop
> INFO - NOTE: recipe .......: task do_image_swu: Failed
> ERROR - ERROR: Task (/builds/.......bb:do_image_swu) failed with exit
> code '1'
> 
> INFO - | E: 10mount: umount:
> /run/schroot/mount/isar-builder-5f43627d-fad6-488f-bade-57d0635ee0c6-408837-d15a1d4b-8c19-4fc8-9b5e-d2ccb3d67d82: target is busy.
> INFO - | E: 10mount: rmdir: failed to remove
> '/var/run/schroot/mount/isar-builder-5f43627d-fad6-488f-bade-57d0635ee0c6-408837-d15a1d4b-8c19-4fc8-9b5e-d2ccb3d67d82': Device or resource busy
> INFO - | E:
> isar-builder-5f43627d-fad6-488f-bade-57d0635ee0c6-408837-d15a1d4b-8c19-4fc8-9b5e-d2ccb3d67d82: Chroot setup failed: stage=setup-stop
> INFO - NOTE: recipe ......: task do_image_ova: Failed
> ERROR - ERROR: Task (/builds/.....bb:do_image_ova) failed with exit code '1'
> 
> Do you have any information regarding the root cause?

Trying once more to pull Cedric in (I think I used the wrong address
last time): Weren't you trying to resolve that with a lazy umount? Or
was it for a different corner of isar?

But even if we lazily umount, this won't help when the next command is
trying to rmdir the mount point...

Does anyone have a feeling when this started? Does it correlate with any
major changes in isar?

Jan

> 
> Thanks,
> Gokhan
> 
> On Wednesday, June 5, 2024 at 11:55:34 AM UTC+3 Jan Kiszka wrote:
> 
>     On 05.06.24 10:47, 'Schultschik, Sven' via isar-users wrote:
>     > Hi all,
>     >
>     > Our ISAR setup is running smooth on many machines different sizes.
>     But
>     > on our CI/CD pipeline it throws from time to time an error.
>     Starting it
>     > over again it works.
>     >
>     > Does someone knows this kind of error and where I have to look or how
>     > to fix?
>     >
>     > 2024-06-05 04:00:41 - INFO - | Processing triggers for man-db
>     > (2.11.2-2) ...
>     > 2024-06-05 04:00:41 - INFO - | e2fsck 1.47.0 (5-Feb-2023)
>     > 2024-06-05 04:00:41 - INFO - | Pass 1: Checking inodes, blocks, and
>     > sizes
>     > 2024-06-05 04:00:41 - INFO - | Pass 2: Checking directory structure
>     > 2024-06-05 04:00:41 - INFO - | Pass 3: Checking directory
>     > connectivity
>     > 2024-06-05 04:00:41 - INFO - | Pass 3A: Optimizing directories
>     > 2024-06-05 04:00:41 - INFO - | Pass 4: Checking reference counts
>     > 2024-06-05 04:00:41 - INFO - | Pass 5: Checking group summary
>     > information
>     > 2024-06-05 04:00:41 - INFO - |
>     > 2024-06-05 04:00:41 - INFO - | /home/builder/edge-debug-machine-
>     > std/deploy/debug-machine-std.ext4: ***** FILE SYSTEM WAS MODIFIED
>     *****
>     > 2024-06-05 04:00:41 - INFO - | /home/builder/debug-machine-
>     > std/deploy/debug-machine-std.ext4: 25039/77600 files (0.1% non-
>     > contiguous), 292861/310076 blocks
>     > 2024-06-05 04:00:41 - INFO - | E: 10mount: umount:
>     > /run/schroot/mount/isar-builder-e7f8d532-aa0e-41cf-85dd-11546112ec7c-
>     > 982511-2d2ada00-82dd-45bc-9bd4-f9629d1a55f3: target is busy.
>     > 2024-06-05 04:00:41 - INFO - | E: 10mount: rmdir: failed to remove
>     > '/var/run/schroot/mount/isar-builder-e7f8d532-aa0e-41cf-85dd-
>     > 11546112ec7c-982511-2d2ada00-82dd-45bc-9bd4-f9629d1a55f3': Device or
>     > resource busy
>     > 2024-06-05 04:00:41 - INFO - | E: isar-builder-e7f8d532-aa0e-41cf-
>     > 85dd-11546112ec7c-982511-2d2ada00-82dd-45bc-9bd4-f9629d1a55f3: Chroot
>     > setup failed: stage=setup-stop
>     > 2024-06-05 04:00:41 - INFO - NOTE: recipe debug-1.0-r0: task
>     > do_image_ext4: Failed
>     > 2024-06-05 04:00:41 - ERROR - ERROR: Task (/build/../repo/recipes-
>     > core/images/debug.bb:do_image_ext4) failed with exit code '1'
>     > 2024-06-05 04:00:42 - INFO - NOTE: Tasks Summary: Attempted 896
>     > tasks of which 0 didn't need to be rerun and 1 failed.
>     > 2024-06-05 04:00:43 - INFO -
>     > 2024-06-05 04:00:43 - INFO - Summary: 1 task failed:
>     > 2024-06-05 04:00:43 - INFO - /build/../repo/recipes-
>     > core/images/debug.bb:do_image_ext4
>     > 2024-06-05 04:00:43 - INFO - Summary: There was 1 WARNING message.
>     > 2024-06-05 04:00:43 - INFO - Summary: There was 1 ERROR message,
>     > returning a non-zero exit code.
>     > 2024-06-05 04:00:43 - ERROR - Command "/build$
>     > /work/isar/bitbake/bin/bitbake -c build debug" failed
>     > --- Error summary ---
>     > ERROR: debug-1.0-r0 do_image_ext4:
>     > ExecutionError('/build/tmp/work/arm64/debug-machine-std/1.0-
>     > r0/temp/run.do_image_ext4.982511', 1, None, None)
>     > ERROR: Logfile of failure stored in: /build/tmp/work/arm64/debug-
>     > machine-std/1.0-r0/temp/log.do_image_ext4.982511
>     > ERROR: Task
>     (/build/../repo/recipes-core/images/debug.bb:do_image_ext4)
>     > failed with exit code '1'
>     >
> 
>     Cedric, weren't you seeing something very similar?
> 
>     Jan
> 
>     -- 
>     Siemens AG, Technology
>     Linux Expert Center
> 
> -- 
> You received this message because you are subscribed to the Google
> Groups "isar-users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to isar-users+unsubscribe@googlegroups.com
> <mailto:isar-users+unsubscribe@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/isar-users/d2508b62-d2fb-44a7-936f-aa2367347662n%40googlegroups.com <https://groups.google.com/d/msgid/isar-users/d2508b62-d2fb-44a7-936f-aa2367347662n%40googlegroups.com?utm_medium=email&utm_source=footer>.

-- 
Siemens AG, Technology
Linux Expert Center

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/isar-users/940834c5-ccfd-4a3a-b1fa-e8c49fba7a11%40siemens.com.