Hi Jan, Thanks for pointing that out, I was not aware the fix on lazy umounts commited 2 weeks ago: https://github.com/ilbers/isar/commit/ed2f8fda7cc85309c4f9a1f49a3f4b96df9886bb Don't know if the issue on Sven's side has been resolved by this. Just trying out, will let you know if it will help us. Best, Gokhan On Monday, July 8, 2024 at 6:13:54 PM UTC+3 Jan Kiszka wrote: > On 08.07.24 16:17, Gökhan Çetin wrote: > > Hi everyone, > > > > Similarly, we have been observing do_image tasks fail sporadically in > > CI/CD pipelines for a while. > > You can see a few examples below: > > > > INFO - | E: 10mount: umount: > > > /run/schroot/mount/isar-builder-31888f6e-6228-4bc8-9570-dd4d69ad25ba-367048-331187ba-b359-48ad-91f4-2962f4bdf32b: > target is busy. > > INFO - | E: 10mount: rmdir: failed to remove > > > '/var/run/schroot/mount/isar-builder-31888f6e-6228-4bc8-9570-dd4d69ad25ba-367048-331187ba-b359-48ad-91f4-2962f4bdf32b': > Device or resource busy > > INFO - | E: > > > isar-builder-31888f6e-6228-4bc8-9570-dd4d69ad25ba-367048-331187ba-b359-48ad-91f4-2962f4bdf32b: > Chroot setup failed: stage=setup-stop > > INFO - NOTE: recipe .......: task do_image_swu: Failed > > ERROR - ERROR: Task (/builds/.......bb:do_image_swu) failed with exit > > code '1' > > > > INFO - | E: 10mount: umount: > > > /run/schroot/mount/isar-builder-5f43627d-fad6-488f-bade-57d0635ee0c6-408837-d15a1d4b-8c19-4fc8-9b5e-d2ccb3d67d82: > target is busy. > > INFO - | E: 10mount: rmdir: failed to remove > > > '/var/run/schroot/mount/isar-builder-5f43627d-fad6-488f-bade-57d0635ee0c6-408837-d15a1d4b-8c19-4fc8-9b5e-d2ccb3d67d82': > Device or resource busy > > INFO - | E: > > > isar-builder-5f43627d-fad6-488f-bade-57d0635ee0c6-408837-d15a1d4b-8c19-4fc8-9b5e-d2ccb3d67d82: > Chroot setup failed: stage=setup-stop > > INFO - NOTE: recipe ......: task do_image_ova: Failed > > ERROR - ERROR: Task (/builds/.....bb:do_image_ova) failed with exit code > '1' > > > > Do you have any information regarding the root cause? > > Trying once more to pull Cedric in (I think I used the wrong address > last time): Weren't you trying to resolve that with a lazy umount? Or > was it for a different corner of isar? > > But even if we lazily umount, this won't help when the next command is > trying to rmdir the mount point... > > Does anyone have a feeling when this started? Does it correlate with any > major changes in isar? > > Jan > > > > > Thanks, > > Gokhan > > > > On Wednesday, June 5, 2024 at 11:55:34 AM UTC+3 Jan Kiszka wrote: > > > > On 05.06.24 10:47, 'Schultschik, Sven' via isar-users wrote: > > > Hi all, > > > > > > Our ISAR setup is running smooth on many machines different sizes. > > But > > > on our CI/CD pipeline it throws from time to time an error. > > Starting it > > > over again it works. > > > > > > Does someone knows this kind of error and where I have to look or how > > > to fix? > > > > > > 2024-06-05 04:00:41 - INFO - | Processing triggers for man-db > > > (2.11.2-2) ... > > > 2024-06-05 04:00:41 - INFO - | e2fsck 1.47.0 (5-Feb-2023) > > > 2024-06-05 04:00:41 - INFO - | Pass 1: Checking inodes, blocks, and > > > sizes > > > 2024-06-05 04:00:41 - INFO - | Pass 2: Checking directory structure > > > 2024-06-05 04:00:41 - INFO - | Pass 3: Checking directory > > > connectivity > > > 2024-06-05 04:00:41 - INFO - | Pass 3A: Optimizing directories > > > 2024-06-05 04:00:41 - INFO - | Pass 4: Checking reference counts > > > 2024-06-05 04:00:41 - INFO - | Pass 5: Checking group summary > > > information > > > 2024-06-05 04:00:41 - INFO - | > > > 2024-06-05 04:00:41 - INFO - | /home/builder/edge-debug-machine- > > > std/deploy/debug-machine-std.ext4: ***** FILE SYSTEM WAS MODIFIED > > ***** > > > 2024-06-05 04:00:41 - INFO - | /home/builder/debug-machine- > > > std/deploy/debug-machine-std.ext4: 25039/77600 files (0.1% non- > > > contiguous), 292861/310076 blocks > > > 2024-06-05 04:00:41 - INFO - | E: 10mount: umount: > > > /run/schroot/mount/isar-builder-e7f8d532-aa0e-41cf-85dd-11546112ec7c- > > > 982511-2d2ada00-82dd-45bc-9bd4-f9629d1a55f3: target is busy. > > > 2024-06-05 04:00:41 - INFO - | E: 10mount: rmdir: failed to remove > > > '/var/run/schroot/mount/isar-builder-e7f8d532-aa0e-41cf-85dd- > > > 11546112ec7c-982511-2d2ada00-82dd-45bc-9bd4-f9629d1a55f3': Device or > > > resource busy > > > 2024-06-05 04:00:41 - INFO - | E: isar-builder-e7f8d532-aa0e-41cf- > > > 85dd-11546112ec7c-982511-2d2ada00-82dd-45bc-9bd4-f9629d1a55f3: Chroot > > > setup failed: stage=setup-stop > > > 2024-06-05 04:00:41 - INFO - NOTE: recipe debug-1.0-r0: task > > > do_image_ext4: Failed > > > 2024-06-05 04:00:41 - ERROR - ERROR: Task (/build/../repo/recipes- > > > core/images/debug.bb:do_image_ext4) failed with exit code '1' > > > 2024-06-05 04:00:42 - INFO - NOTE: Tasks Summary: Attempted 896 > > > tasks of which 0 didn't need to be rerun and 1 failed. > > > 2024-06-05 04:00:43 - INFO - > > > 2024-06-05 04:00:43 - INFO - Summary: 1 task failed: > > > 2024-06-05 04:00:43 - INFO - /build/../repo/recipes- > > > core/images/debug.bb:do_image_ext4 > > > 2024-06-05 04:00:43 - INFO - Summary: There was 1 WARNING message. > > > 2024-06-05 04:00:43 - INFO - Summary: There was 1 ERROR message, > > > returning a non-zero exit code. > > > 2024-06-05 04:00:43 - ERROR - Command "/build$ > > > /work/isar/bitbake/bin/bitbake -c build debug" failed > > > --- Error summary --- > > > ERROR: debug-1.0-r0 do_image_ext4: > > > ExecutionError('/build/tmp/work/arm64/debug-machine-std/1.0- > > > r0/temp/run.do_image_ext4.982511', 1, None, None) > > > ERROR: Logfile of failure stored in: /build/tmp/work/arm64/debug- > > > machine-std/1.0-r0/temp/log.do_image_ext4.982511 > > > ERROR: Task > > (/build/../repo/recipes-core/images/debug.bb:do_image_ext4) > > > failed with exit code '1' > > > > > > > Cedric, weren't you seeing something very similar? > > > > Jan > > > > -- > > Siemens AG, Technology > > Linux Expert Center > > > > -- > > You received this message because you are subscribed to the Google > > Groups "isar-users" group. > > To unsubscribe from this group and stop receiving emails from it, send > > an email to isar-users+...@googlegroups.com > > . > > To view this discussion on the web visit > > > https://groups.google.com/d/msgid/isar-users/d2508b62-d2fb-44a7-936f-aa2367347662n%40googlegroups.com > < > https://groups.google.com/d/msgid/isar-users/d2508b62-d2fb-44a7-936f-aa2367347662n%40googlegroups.com?utm_medium=email&utm_source=footer > >. > > -- > Siemens AG, Technology > Linux Expert Center > > -- You received this message because you are subscribed to the Google Groups "isar-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/isar-users/f4495e54-a4f3-417d-8b4d-bd38b723a232n%40googlegroups.com.