* schroot umount target is busy
@ 2024-06-05 8:47 Schultschik, Sven
2024-06-05 8:55 ` Jan Kiszka
0 siblings, 1 reply; 11+ messages in thread
From: Schultschik, Sven @ 2024-06-05 8:47 UTC (permalink / raw)
To: isar-users
Hi all,
Our ISAR setup is running smooth on many machines different sizes. But
on our CI/CD pipeline it throws from time to time an error. Starting it
over again it works.
Does someone knows this kind of error and where I have to look or how
to fix?
2024-06-05 04:00:41 - INFO - | Processing triggers for man-db
(2.11.2-2) ...
2024-06-05 04:00:41 - INFO - | e2fsck 1.47.0 (5-Feb-2023)
2024-06-05 04:00:41 - INFO - | Pass 1: Checking inodes, blocks, and
sizes
2024-06-05 04:00:41 - INFO - | Pass 2: Checking directory structure
2024-06-05 04:00:41 - INFO - | Pass 3: Checking directory
connectivity
2024-06-05 04:00:41 - INFO - | Pass 3A: Optimizing directories
2024-06-05 04:00:41 - INFO - | Pass 4: Checking reference counts
2024-06-05 04:00:41 - INFO - | Pass 5: Checking group summary
information
2024-06-05 04:00:41 - INFO - |
2024-06-05 04:00:41 - INFO - | /home/builder/edge-debug-machine-
std/deploy/debug-machine-std.ext4: ***** FILE SYSTEM WAS MODIFIED *****
2024-06-05 04:00:41 - INFO - | /home/builder/debug-machine-
std/deploy/debug-machine-std.ext4: 25039/77600 files (0.1% non-
contiguous), 292861/310076 blocks
2024-06-05 04:00:41 - INFO - | E: 10mount: umount:
/run/schroot/mount/isar-builder-e7f8d532-aa0e-41cf-85dd-11546112ec7c-
982511-2d2ada00-82dd-45bc-9bd4-f9629d1a55f3: target is busy.
2024-06-05 04:00:41 - INFO - | E: 10mount: rmdir: failed to remove
'/var/run/schroot/mount/isar-builder-e7f8d532-aa0e-41cf-85dd-
11546112ec7c-982511-2d2ada00-82dd-45bc-9bd4-f9629d1a55f3': Device or
resource busy
2024-06-05 04:00:41 - INFO - | E: isar-builder-e7f8d532-aa0e-41cf-
85dd-11546112ec7c-982511-2d2ada00-82dd-45bc-9bd4-f9629d1a55f3: Chroot
setup failed: stage=setup-stop
2024-06-05 04:00:41 - INFO - NOTE: recipe debug-1.0-r0: task
do_image_ext4: Failed
2024-06-05 04:00:41 - ERROR - ERROR: Task (/build/../repo/recipes-
core/images/debug.bb:do_image_ext4) failed with exit code '1'
2024-06-05 04:00:42 - INFO - NOTE: Tasks Summary: Attempted 896
tasks of which 0 didn't need to be rerun and 1 failed.
2024-06-05 04:00:43 - INFO -
2024-06-05 04:00:43 - INFO - Summary: 1 task failed:
2024-06-05 04:00:43 - INFO - /build/../repo/recipes-
core/images/debug.bb:do_image_ext4
2024-06-05 04:00:43 - INFO - Summary: There was 1 WARNING message.
2024-06-05 04:00:43 - INFO - Summary: There was 1 ERROR message,
returning a non-zero exit code.
2024-06-05 04:00:43 - ERROR - Command "/build$
/work/isar/bitbake/bin/bitbake -c build debug" failed
--- Error summary ---
ERROR: debug-1.0-r0 do_image_ext4:
ExecutionError('/build/tmp/work/arm64/debug-machine-std/1.0-
r0/temp/run.do_image_ext4.982511', 1, None, None)
ERROR: Logfile of failure stored in: /build/tmp/work/arm64/debug-
machine-std/1.0-r0/temp/log.do_image_ext4.982511
ERROR: Task (/build/../repo/recipes-core/images/debug.bb:do_image_ext4)
failed with exit code '1'
Thank you
br Sven
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: schroot umount target is busy
2024-06-05 8:47 schroot umount target is busy Schultschik, Sven
@ 2024-06-05 8:55 ` Jan Kiszka
2024-07-08 14:17 ` Gökhan Çetin
0 siblings, 1 reply; 11+ messages in thread
From: Jan Kiszka @ 2024-06-05 8:55 UTC (permalink / raw)
To: Schultschik, Sven, isar-users, Cedric Hombourger
On 05.06.24 10:47, 'Schultschik, Sven' via isar-users wrote:
> Hi all,
>
> Our ISAR setup is running smooth on many machines different sizes. But
> on our CI/CD pipeline it throws from time to time an error. Starting it
> over again it works.
>
> Does someone knows this kind of error and where I have to look or how
> to fix?
>
> 2024-06-05 04:00:41 - INFO - | Processing triggers for man-db
> (2.11.2-2) ...
> 2024-06-05 04:00:41 - INFO - | e2fsck 1.47.0 (5-Feb-2023)
> 2024-06-05 04:00:41 - INFO - | Pass 1: Checking inodes, blocks, and
> sizes
> 2024-06-05 04:00:41 - INFO - | Pass 2: Checking directory structure
> 2024-06-05 04:00:41 - INFO - | Pass 3: Checking directory
> connectivity
> 2024-06-05 04:00:41 - INFO - | Pass 3A: Optimizing directories
> 2024-06-05 04:00:41 - INFO - | Pass 4: Checking reference counts
> 2024-06-05 04:00:41 - INFO - | Pass 5: Checking group summary
> information
> 2024-06-05 04:00:41 - INFO - |
> 2024-06-05 04:00:41 - INFO - | /home/builder/edge-debug-machine-
> std/deploy/debug-machine-std.ext4: ***** FILE SYSTEM WAS MODIFIED *****
> 2024-06-05 04:00:41 - INFO - | /home/builder/debug-machine-
> std/deploy/debug-machine-std.ext4: 25039/77600 files (0.1% non-
> contiguous), 292861/310076 blocks
> 2024-06-05 04:00:41 - INFO - | E: 10mount: umount:
> /run/schroot/mount/isar-builder-e7f8d532-aa0e-41cf-85dd-11546112ec7c-
> 982511-2d2ada00-82dd-45bc-9bd4-f9629d1a55f3: target is busy.
> 2024-06-05 04:00:41 - INFO - | E: 10mount: rmdir: failed to remove
> '/var/run/schroot/mount/isar-builder-e7f8d532-aa0e-41cf-85dd-
> 11546112ec7c-982511-2d2ada00-82dd-45bc-9bd4-f9629d1a55f3': Device or
> resource busy
> 2024-06-05 04:00:41 - INFO - | E: isar-builder-e7f8d532-aa0e-41cf-
> 85dd-11546112ec7c-982511-2d2ada00-82dd-45bc-9bd4-f9629d1a55f3: Chroot
> setup failed: stage=setup-stop
> 2024-06-05 04:00:41 - INFO - NOTE: recipe debug-1.0-r0: task
> do_image_ext4: Failed
> 2024-06-05 04:00:41 - ERROR - ERROR: Task (/build/../repo/recipes-
> core/images/debug.bb:do_image_ext4) failed with exit code '1'
> 2024-06-05 04:00:42 - INFO - NOTE: Tasks Summary: Attempted 896
> tasks of which 0 didn't need to be rerun and 1 failed.
> 2024-06-05 04:00:43 - INFO -
> 2024-06-05 04:00:43 - INFO - Summary: 1 task failed:
> 2024-06-05 04:00:43 - INFO - /build/../repo/recipes-
> core/images/debug.bb:do_image_ext4
> 2024-06-05 04:00:43 - INFO - Summary: There was 1 WARNING message.
> 2024-06-05 04:00:43 - INFO - Summary: There was 1 ERROR message,
> returning a non-zero exit code.
> 2024-06-05 04:00:43 - ERROR - Command "/build$
> /work/isar/bitbake/bin/bitbake -c build debug" failed
> --- Error summary ---
> ERROR: debug-1.0-r0 do_image_ext4:
> ExecutionError('/build/tmp/work/arm64/debug-machine-std/1.0-
> r0/temp/run.do_image_ext4.982511', 1, None, None)
> ERROR: Logfile of failure stored in: /build/tmp/work/arm64/debug-
> machine-std/1.0-r0/temp/log.do_image_ext4.982511
> ERROR: Task (/build/../repo/recipes-core/images/debug.bb:do_image_ext4)
> failed with exit code '1'
>
Cedric, weren't you seeing something very similar?
Jan
--
Siemens AG, Technology
Linux Expert Center
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: schroot umount target is busy
2024-06-05 8:55 ` Jan Kiszka
@ 2024-07-08 14:17 ` Gökhan Çetin
2024-07-08 15:13 ` 'Jan Kiszka' via isar-users
0 siblings, 1 reply; 11+ messages in thread
From: Gökhan Çetin @ 2024-07-08 14:17 UTC (permalink / raw)
To: isar-users
[-- Attachment #1.1: Type: text/plain, Size: 5306 bytes --]
Hi everyone,
Similarly, we have been observing do_image tasks fail sporadically in CI/CD
pipelines for a while.
You can see a few examples below:
INFO - | E: 10mount: umount:
/run/schroot/mount/isar-builder-31888f6e-6228-4bc8-9570-dd4d69ad25ba-367048-331187ba-b359-48ad-91f4-2962f4bdf32b:
target is busy.
INFO - | E: 10mount: rmdir: failed to remove
'/var/run/schroot/mount/isar-builder-31888f6e-6228-4bc8-9570-dd4d69ad25ba-367048-331187ba-b359-48ad-91f4-2962f4bdf32b':
Device or resource busy
INFO - | E:
isar-builder-31888f6e-6228-4bc8-9570-dd4d69ad25ba-367048-331187ba-b359-48ad-91f4-2962f4bdf32b:
Chroot setup failed: stage=setup-stop
INFO - NOTE: recipe .......: task do_image_swu: Failed
ERROR - ERROR: Task (/builds/.......bb:do_image_swu) failed with exit code
'1'
INFO - | E: 10mount: umount:
/run/schroot/mount/isar-builder-5f43627d-fad6-488f-bade-57d0635ee0c6-408837-d15a1d4b-8c19-4fc8-9b5e-d2ccb3d67d82:
target is busy.
INFO - | E: 10mount: rmdir: failed to remove
'/var/run/schroot/mount/isar-builder-5f43627d-fad6-488f-bade-57d0635ee0c6-408837-d15a1d4b-8c19-4fc8-9b5e-d2ccb3d67d82':
Device or resource busy
INFO - | E:
isar-builder-5f43627d-fad6-488f-bade-57d0635ee0c6-408837-d15a1d4b-8c19-4fc8-9b5e-d2ccb3d67d82:
Chroot setup failed: stage=setup-stop
INFO - NOTE: recipe ......: task do_image_ova: Failed
ERROR - ERROR: Task (/builds/.....bb:do_image_ova) failed with exit code '1'
Do you have any information regarding the root cause?
Thanks,
Gokhan
On Wednesday, June 5, 2024 at 11:55:34 AM UTC+3 Jan Kiszka wrote:
> On 05.06.24 10:47, 'Schultschik, Sven' via isar-users wrote:
> > Hi all,
> >
> > Our ISAR setup is running smooth on many machines different sizes. But
> > on our CI/CD pipeline it throws from time to time an error. Starting it
> > over again it works.
> >
> > Does someone knows this kind of error and where I have to look or how
> > to fix?
> >
> > 2024-06-05 04:00:41 - INFO - | Processing triggers for man-db
> > (2.11.2-2) ...
> > 2024-06-05 04:00:41 - INFO - | e2fsck 1.47.0 (5-Feb-2023)
> > 2024-06-05 04:00:41 - INFO - | Pass 1: Checking inodes, blocks, and
> > sizes
> > 2024-06-05 04:00:41 - INFO - | Pass 2: Checking directory structure
> > 2024-06-05 04:00:41 - INFO - | Pass 3: Checking directory
> > connectivity
> > 2024-06-05 04:00:41 - INFO - | Pass 3A: Optimizing directories
> > 2024-06-05 04:00:41 - INFO - | Pass 4: Checking reference counts
> > 2024-06-05 04:00:41 - INFO - | Pass 5: Checking group summary
> > information
> > 2024-06-05 04:00:41 - INFO - |
> > 2024-06-05 04:00:41 - INFO - | /home/builder/edge-debug-machine-
> > std/deploy/debug-machine-std.ext4: ***** FILE SYSTEM WAS MODIFIED *****
> > 2024-06-05 04:00:41 - INFO - | /home/builder/debug-machine-
> > std/deploy/debug-machine-std.ext4: 25039/77600 files (0.1% non-
> > contiguous), 292861/310076 blocks
> > 2024-06-05 04:00:41 - INFO - | E: 10mount: umount:
> > /run/schroot/mount/isar-builder-e7f8d532-aa0e-41cf-85dd-11546112ec7c-
> > 982511-2d2ada00-82dd-45bc-9bd4-f9629d1a55f3: target is busy.
> > 2024-06-05 04:00:41 - INFO - | E: 10mount: rmdir: failed to remove
> > '/var/run/schroot/mount/isar-builder-e7f8d532-aa0e-41cf-85dd-
> > 11546112ec7c-982511-2d2ada00-82dd-45bc-9bd4-f9629d1a55f3': Device or
> > resource busy
> > 2024-06-05 04:00:41 - INFO - | E: isar-builder-e7f8d532-aa0e-41cf-
> > 85dd-11546112ec7c-982511-2d2ada00-82dd-45bc-9bd4-f9629d1a55f3: Chroot
> > setup failed: stage=setup-stop
> > 2024-06-05 04:00:41 - INFO - NOTE: recipe debug-1.0-r0: task
> > do_image_ext4: Failed
> > 2024-06-05 04:00:41 - ERROR - ERROR: Task (/build/../repo/recipes-
> > core/images/debug.bb:do_image_ext4) failed with exit code '1'
> > 2024-06-05 04:00:42 - INFO - NOTE: Tasks Summary: Attempted 896
> > tasks of which 0 didn't need to be rerun and 1 failed.
> > 2024-06-05 04:00:43 - INFO -
> > 2024-06-05 04:00:43 - INFO - Summary: 1 task failed:
> > 2024-06-05 04:00:43 - INFO - /build/../repo/recipes-
> > core/images/debug.bb:do_image_ext4
> > 2024-06-05 04:00:43 - INFO - Summary: There was 1 WARNING message.
> > 2024-06-05 04:00:43 - INFO - Summary: There was 1 ERROR message,
> > returning a non-zero exit code.
> > 2024-06-05 04:00:43 - ERROR - Command "/build$
> > /work/isar/bitbake/bin/bitbake -c build debug" failed
> > --- Error summary ---
> > ERROR: debug-1.0-r0 do_image_ext4:
> > ExecutionError('/build/tmp/work/arm64/debug-machine-std/1.0-
> > r0/temp/run.do_image_ext4.982511', 1, None, None)
> > ERROR: Logfile of failure stored in: /build/tmp/work/arm64/debug-
> > machine-std/1.0-r0/temp/log.do_image_ext4.982511
> > ERROR: Task (/build/../repo/recipes-core/images/debug.bb:do_image_ext4)
> > failed with exit code '1'
> >
>
> Cedric, weren't you seeing something very similar?
>
> Jan
>
> --
> Siemens AG, Technology
> Linux Expert Center
>
>
--
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/isar-users/d2508b62-d2fb-44a7-936f-aa2367347662n%40googlegroups.com.
[-- Attachment #1.2: Type: text/html, Size: 6409 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: schroot umount target is busy
2024-07-08 14:17 ` Gökhan Çetin
@ 2024-07-08 15:13 ` 'Jan Kiszka' via isar-users
2024-07-09 6:28 ` Gökhan Çetin
2024-07-09 6:52 ` 'cedric.hombourger@siemens.com' via isar-users
0 siblings, 2 replies; 11+ messages in thread
From: 'Jan Kiszka' via isar-users @ 2024-07-08 15:13 UTC (permalink / raw)
To: Gökhan Çetin, isar-users, Cedric Hombourger,
Schultschik, Sven Angelo (PD PA CI R&D 4)
On 08.07.24 16:17, Gökhan Çetin wrote:
> Hi everyone,
>
> Similarly, we have been observing do_image tasks fail sporadically in
> CI/CD pipelines for a while.
> You can see a few examples below:
>
> INFO - | E: 10mount: umount:
> /run/schroot/mount/isar-builder-31888f6e-6228-4bc8-9570-dd4d69ad25ba-367048-331187ba-b359-48ad-91f4-2962f4bdf32b: target is busy.
> INFO - | E: 10mount: rmdir: failed to remove
> '/var/run/schroot/mount/isar-builder-31888f6e-6228-4bc8-9570-dd4d69ad25ba-367048-331187ba-b359-48ad-91f4-2962f4bdf32b': Device or resource busy
> INFO - | E:
> isar-builder-31888f6e-6228-4bc8-9570-dd4d69ad25ba-367048-331187ba-b359-48ad-91f4-2962f4bdf32b: Chroot setup failed: stage=setup-stop
> INFO - NOTE: recipe .......: task do_image_swu: Failed
> ERROR - ERROR: Task (/builds/.......bb:do_image_swu) failed with exit
> code '1'
>
> INFO - | E: 10mount: umount:
> /run/schroot/mount/isar-builder-5f43627d-fad6-488f-bade-57d0635ee0c6-408837-d15a1d4b-8c19-4fc8-9b5e-d2ccb3d67d82: target is busy.
> INFO - | E: 10mount: rmdir: failed to remove
> '/var/run/schroot/mount/isar-builder-5f43627d-fad6-488f-bade-57d0635ee0c6-408837-d15a1d4b-8c19-4fc8-9b5e-d2ccb3d67d82': Device or resource busy
> INFO - | E:
> isar-builder-5f43627d-fad6-488f-bade-57d0635ee0c6-408837-d15a1d4b-8c19-4fc8-9b5e-d2ccb3d67d82: Chroot setup failed: stage=setup-stop
> INFO - NOTE: recipe ......: task do_image_ova: Failed
> ERROR - ERROR: Task (/builds/.....bb:do_image_ova) failed with exit code '1'
>
> Do you have any information regarding the root cause?
Trying once more to pull Cedric in (I think I used the wrong address
last time): Weren't you trying to resolve that with a lazy umount? Or
was it for a different corner of isar?
But even if we lazily umount, this won't help when the next command is
trying to rmdir the mount point...
Does anyone have a feeling when this started? Does it correlate with any
major changes in isar?
Jan
>
> Thanks,
> Gokhan
>
> On Wednesday, June 5, 2024 at 11:55:34 AM UTC+3 Jan Kiszka wrote:
>
> On 05.06.24 10:47, 'Schultschik, Sven' via isar-users wrote:
> > Hi all,
> >
> > Our ISAR setup is running smooth on many machines different sizes.
> But
> > on our CI/CD pipeline it throws from time to time an error.
> Starting it
> > over again it works.
> >
> > Does someone knows this kind of error and where I have to look or how
> > to fix?
> >
> > 2024-06-05 04:00:41 - INFO - | Processing triggers for man-db
> > (2.11.2-2) ...
> > 2024-06-05 04:00:41 - INFO - | e2fsck 1.47.0 (5-Feb-2023)
> > 2024-06-05 04:00:41 - INFO - | Pass 1: Checking inodes, blocks, and
> > sizes
> > 2024-06-05 04:00:41 - INFO - | Pass 2: Checking directory structure
> > 2024-06-05 04:00:41 - INFO - | Pass 3: Checking directory
> > connectivity
> > 2024-06-05 04:00:41 - INFO - | Pass 3A: Optimizing directories
> > 2024-06-05 04:00:41 - INFO - | Pass 4: Checking reference counts
> > 2024-06-05 04:00:41 - INFO - | Pass 5: Checking group summary
> > information
> > 2024-06-05 04:00:41 - INFO - |
> > 2024-06-05 04:00:41 - INFO - | /home/builder/edge-debug-machine-
> > std/deploy/debug-machine-std.ext4: ***** FILE SYSTEM WAS MODIFIED
> *****
> > 2024-06-05 04:00:41 - INFO - | /home/builder/debug-machine-
> > std/deploy/debug-machine-std.ext4: 25039/77600 files (0.1% non-
> > contiguous), 292861/310076 blocks
> > 2024-06-05 04:00:41 - INFO - | E: 10mount: umount:
> > /run/schroot/mount/isar-builder-e7f8d532-aa0e-41cf-85dd-11546112ec7c-
> > 982511-2d2ada00-82dd-45bc-9bd4-f9629d1a55f3: target is busy.
> > 2024-06-05 04:00:41 - INFO - | E: 10mount: rmdir: failed to remove
> > '/var/run/schroot/mount/isar-builder-e7f8d532-aa0e-41cf-85dd-
> > 11546112ec7c-982511-2d2ada00-82dd-45bc-9bd4-f9629d1a55f3': Device or
> > resource busy
> > 2024-06-05 04:00:41 - INFO - | E: isar-builder-e7f8d532-aa0e-41cf-
> > 85dd-11546112ec7c-982511-2d2ada00-82dd-45bc-9bd4-f9629d1a55f3: Chroot
> > setup failed: stage=setup-stop
> > 2024-06-05 04:00:41 - INFO - NOTE: recipe debug-1.0-r0: task
> > do_image_ext4: Failed
> > 2024-06-05 04:00:41 - ERROR - ERROR: Task (/build/../repo/recipes-
> > core/images/debug.bb:do_image_ext4) failed with exit code '1'
> > 2024-06-05 04:00:42 - INFO - NOTE: Tasks Summary: Attempted 896
> > tasks of which 0 didn't need to be rerun and 1 failed.
> > 2024-06-05 04:00:43 - INFO -
> > 2024-06-05 04:00:43 - INFO - Summary: 1 task failed:
> > 2024-06-05 04:00:43 - INFO - /build/../repo/recipes-
> > core/images/debug.bb:do_image_ext4
> > 2024-06-05 04:00:43 - INFO - Summary: There was 1 WARNING message.
> > 2024-06-05 04:00:43 - INFO - Summary: There was 1 ERROR message,
> > returning a non-zero exit code.
> > 2024-06-05 04:00:43 - ERROR - Command "/build$
> > /work/isar/bitbake/bin/bitbake -c build debug" failed
> > --- Error summary ---
> > ERROR: debug-1.0-r0 do_image_ext4:
> > ExecutionError('/build/tmp/work/arm64/debug-machine-std/1.0-
> > r0/temp/run.do_image_ext4.982511', 1, None, None)
> > ERROR: Logfile of failure stored in: /build/tmp/work/arm64/debug-
> > machine-std/1.0-r0/temp/log.do_image_ext4.982511
> > ERROR: Task
> (/build/../repo/recipes-core/images/debug.bb:do_image_ext4)
> > failed with exit code '1'
> >
>
> Cedric, weren't you seeing something very similar?
>
> Jan
>
> --
> Siemens AG, Technology
> Linux Expert Center
>
> --
> You received this message because you are subscribed to the Google
> Groups "isar-users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to isar-users+unsubscribe@googlegroups.com
> <mailto:isar-users+unsubscribe@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/isar-users/d2508b62-d2fb-44a7-936f-aa2367347662n%40googlegroups.com <https://groups.google.com/d/msgid/isar-users/d2508b62-d2fb-44a7-936f-aa2367347662n%40googlegroups.com?utm_medium=email&utm_source=footer>.
--
Siemens AG, Technology
Linux Expert Center
--
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/isar-users/940834c5-ccfd-4a3a-b1fa-e8c49fba7a11%40siemens.com.
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: schroot umount target is busy
2024-07-08 15:13 ` 'Jan Kiszka' via isar-users
@ 2024-07-09 6:28 ` Gökhan Çetin
2024-07-09 6:48 ` Anton Mikanovich
2024-07-12 7:04 ` Anton Mikanovich
2024-07-09 6:52 ` 'cedric.hombourger@siemens.com' via isar-users
1 sibling, 2 replies; 11+ messages in thread
From: Gökhan Çetin @ 2024-07-09 6:28 UTC (permalink / raw)
To: isar-users
[-- Attachment #1.1: Type: text/plain, Size: 7147 bytes --]
Hi Jan,
Thanks for pointing that out, I was not aware the fix on lazy umounts
commited 2 weeks
ago: https://github.com/ilbers/isar/commit/ed2f8fda7cc85309c4f9a1f49a3f4b96df9886bb
Don't know if the issue on Sven's side has been resolved by this. Just
trying out, will let you know if it will help us.
Best,
Gokhan
On Monday, July 8, 2024 at 6:13:54 PM UTC+3 Jan Kiszka wrote:
> On 08.07.24 16:17, Gökhan Çetin wrote:
> > Hi everyone,
> >
> > Similarly, we have been observing do_image tasks fail sporadically in
> > CI/CD pipelines for a while.
> > You can see a few examples below:
> >
> > INFO - | E: 10mount: umount:
> >
> /run/schroot/mount/isar-builder-31888f6e-6228-4bc8-9570-dd4d69ad25ba-367048-331187ba-b359-48ad-91f4-2962f4bdf32b:
> target is busy.
> > INFO - | E: 10mount: rmdir: failed to remove
> >
> '/var/run/schroot/mount/isar-builder-31888f6e-6228-4bc8-9570-dd4d69ad25ba-367048-331187ba-b359-48ad-91f4-2962f4bdf32b':
> Device or resource busy
> > INFO - | E:
> >
> isar-builder-31888f6e-6228-4bc8-9570-dd4d69ad25ba-367048-331187ba-b359-48ad-91f4-2962f4bdf32b:
> Chroot setup failed: stage=setup-stop
> > INFO - NOTE: recipe .......: task do_image_swu: Failed
> > ERROR - ERROR: Task (/builds/.......bb:do_image_swu) failed with exit
> > code '1'
> >
> > INFO - | E: 10mount: umount:
> >
> /run/schroot/mount/isar-builder-5f43627d-fad6-488f-bade-57d0635ee0c6-408837-d15a1d4b-8c19-4fc8-9b5e-d2ccb3d67d82:
> target is busy.
> > INFO - | E: 10mount: rmdir: failed to remove
> >
> '/var/run/schroot/mount/isar-builder-5f43627d-fad6-488f-bade-57d0635ee0c6-408837-d15a1d4b-8c19-4fc8-9b5e-d2ccb3d67d82':
> Device or resource busy
> > INFO - | E:
> >
> isar-builder-5f43627d-fad6-488f-bade-57d0635ee0c6-408837-d15a1d4b-8c19-4fc8-9b5e-d2ccb3d67d82:
> Chroot setup failed: stage=setup-stop
> > INFO - NOTE: recipe ......: task do_image_ova: Failed
> > ERROR - ERROR: Task (/builds/.....bb:do_image_ova) failed with exit code
> '1'
> >
> > Do you have any information regarding the root cause?
>
> Trying once more to pull Cedric in (I think I used the wrong address
> last time): Weren't you trying to resolve that with a lazy umount? Or
> was it for a different corner of isar?
>
> But even if we lazily umount, this won't help when the next command is
> trying to rmdir the mount point...
>
> Does anyone have a feeling when this started? Does it correlate with any
> major changes in isar?
>
> Jan
>
> >
> > Thanks,
> > Gokhan
> >
> > On Wednesday, June 5, 2024 at 11:55:34 AM UTC+3 Jan Kiszka wrote:
> >
> > On 05.06.24 10:47, 'Schultschik, Sven' via isar-users wrote:
> > > Hi all,
> > >
> > > Our ISAR setup is running smooth on many machines different sizes.
> > But
> > > on our CI/CD pipeline it throws from time to time an error.
> > Starting it
> > > over again it works.
> > >
> > > Does someone knows this kind of error and where I have to look or how
> > > to fix?
> > >
> > > 2024-06-05 04:00:41 - INFO - | Processing triggers for man-db
> > > (2.11.2-2) ...
> > > 2024-06-05 04:00:41 - INFO - | e2fsck 1.47.0 (5-Feb-2023)
> > > 2024-06-05 04:00:41 - INFO - | Pass 1: Checking inodes, blocks, and
> > > sizes
> > > 2024-06-05 04:00:41 - INFO - | Pass 2: Checking directory structure
> > > 2024-06-05 04:00:41 - INFO - | Pass 3: Checking directory
> > > connectivity
> > > 2024-06-05 04:00:41 - INFO - | Pass 3A: Optimizing directories
> > > 2024-06-05 04:00:41 - INFO - | Pass 4: Checking reference counts
> > > 2024-06-05 04:00:41 - INFO - | Pass 5: Checking group summary
> > > information
> > > 2024-06-05 04:00:41 - INFO - |
> > > 2024-06-05 04:00:41 - INFO - | /home/builder/edge-debug-machine-
> > > std/deploy/debug-machine-std.ext4: ***** FILE SYSTEM WAS MODIFIED
> > *****
> > > 2024-06-05 04:00:41 - INFO - | /home/builder/debug-machine-
> > > std/deploy/debug-machine-std.ext4: 25039/77600 files (0.1% non-
> > > contiguous), 292861/310076 blocks
> > > 2024-06-05 04:00:41 - INFO - | E: 10mount: umount:
> > > /run/schroot/mount/isar-builder-e7f8d532-aa0e-41cf-85dd-11546112ec7c-
> > > 982511-2d2ada00-82dd-45bc-9bd4-f9629d1a55f3: target is busy.
> > > 2024-06-05 04:00:41 - INFO - | E: 10mount: rmdir: failed to remove
> > > '/var/run/schroot/mount/isar-builder-e7f8d532-aa0e-41cf-85dd-
> > > 11546112ec7c-982511-2d2ada00-82dd-45bc-9bd4-f9629d1a55f3': Device or
> > > resource busy
> > > 2024-06-05 04:00:41 - INFO - | E: isar-builder-e7f8d532-aa0e-41cf-
> > > 85dd-11546112ec7c-982511-2d2ada00-82dd-45bc-9bd4-f9629d1a55f3: Chroot
> > > setup failed: stage=setup-stop
> > > 2024-06-05 04:00:41 - INFO - NOTE: recipe debug-1.0-r0: task
> > > do_image_ext4: Failed
> > > 2024-06-05 04:00:41 - ERROR - ERROR: Task (/build/../repo/recipes-
> > > core/images/debug.bb:do_image_ext4) failed with exit code '1'
> > > 2024-06-05 04:00:42 - INFO - NOTE: Tasks Summary: Attempted 896
> > > tasks of which 0 didn't need to be rerun and 1 failed.
> > > 2024-06-05 04:00:43 - INFO -
> > > 2024-06-05 04:00:43 - INFO - Summary: 1 task failed:
> > > 2024-06-05 04:00:43 - INFO - /build/../repo/recipes-
> > > core/images/debug.bb:do_image_ext4
> > > 2024-06-05 04:00:43 - INFO - Summary: There was 1 WARNING message.
> > > 2024-06-05 04:00:43 - INFO - Summary: There was 1 ERROR message,
> > > returning a non-zero exit code.
> > > 2024-06-05 04:00:43 - ERROR - Command "/build$
> > > /work/isar/bitbake/bin/bitbake -c build debug" failed
> > > --- Error summary ---
> > > ERROR: debug-1.0-r0 do_image_ext4:
> > > ExecutionError('/build/tmp/work/arm64/debug-machine-std/1.0-
> > > r0/temp/run.do_image_ext4.982511', 1, None, None)
> > > ERROR: Logfile of failure stored in: /build/tmp/work/arm64/debug-
> > > machine-std/1.0-r0/temp/log.do_image_ext4.982511
> > > ERROR: Task
> > (/build/../repo/recipes-core/images/debug.bb:do_image_ext4)
> > > failed with exit code '1'
> > >
> >
> > Cedric, weren't you seeing something very similar?
> >
> > Jan
> >
> > --
> > Siemens AG, Technology
> > Linux Expert Center
> >
> > --
> > You received this message because you are subscribed to the Google
> > Groups "isar-users" group.
> > To unsubscribe from this group and stop receiving emails from it, send
> > an email to isar-users+...@googlegroups.com
> > <mailto:isar-users+...@googlegroups.com>.
> > To view this discussion on the web visit
> >
> https://groups.google.com/d/msgid/isar-users/d2508b62-d2fb-44a7-936f-aa2367347662n%40googlegroups.com
> <
> https://groups.google.com/d/msgid/isar-users/d2508b62-d2fb-44a7-936f-aa2367347662n%40googlegroups.com?utm_medium=email&utm_source=footer
> >.
>
> --
> Siemens AG, Technology
> Linux Expert Center
>
>
--
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/isar-users/f4495e54-a4f3-417d-8b4d-bd38b723a232n%40googlegroups.com.
[-- Attachment #1.2: Type: text/html, Size: 9778 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: schroot umount target is busy
2024-07-09 6:28 ` Gökhan Çetin
@ 2024-07-09 6:48 ` Anton Mikanovich
2024-07-12 7:04 ` Anton Mikanovich
1 sibling, 0 replies; 11+ messages in thread
From: Anton Mikanovich @ 2024-07-09 6:48 UTC (permalink / raw)
To: Gökhan Çetin, isar-users, Kiszka, Jan,
Cedric Hombourger, Schultschik, Sven Angelo (PD PA CI R&D 4),
Baurzhan Ismagulov
09/07/2024 09:28, Gökhan Çetin wrote:
> Hi Jan,
>
> Thanks for pointing that out, I was not aware the fix on lazy umounts
> commited 2 weeks
> ago: https://github.com/ilbers/isar/commit/ed2f8fda7cc85309c4f9a1f49a3f4b96df9886bb
>
> Don't know if the issue on Sven's side has been resolved by this. Just
> trying out, will let you know if it will help us.
>
> Best,
> Gokhan
Hello Gökhan,
The issue you've observed is probably caused by some mounts still left
mounted
during schroot session stop phase (imager_run API).
Lazy umount fix will not resolve those mounts, but makes them visible
for debug.
Schroot mounts are controlled by fstab, which is filled from SCHROOT_MOUNTS
value. If there are some mounts performing manually during imager_run
call, it
should be also unmounted in the same call.
--
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/isar-users/32cb6d50-4afd-44f3-8668-0c1ef7836d94%40ilbers.de.
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: schroot umount target is busy
2024-07-08 15:13 ` 'Jan Kiszka' via isar-users
2024-07-09 6:28 ` Gökhan Çetin
@ 2024-07-09 6:52 ` 'cedric.hombourger@siemens.com' via isar-users
2024-07-09 10:27 ` 'Jan Kiszka' via isar-users
1 sibling, 1 reply; 11+ messages in thread
From: 'cedric.hombourger@siemens.com' via isar-users @ 2024-07-09 6:52 UTC (permalink / raw)
To: Schultschik, Sven, isar-users, Kiszka, Jan, ctngkhn
On Mon, 2024-07-08 at 17:13 +0200, Jan Kiszka wrote:
> On 08.07.24 16:17, Gökhan Çetin wrote:
> > Hi everyone,
> >
> > Similarly, we have been observing do_image tasks fail sporadically
> > in
> > CI/CD pipelines for a while.
> > You can see a few examples below:
> >
> > INFO - | E: 10mount: umount:
> > /run/schroot/mount/isar-builder-31888f6e-6228-4bc8-9570-
> > dd4d69ad25ba-367048-331187ba-b359-48ad-91f4-2962f4bdf32b: target is
> > busy.
> > INFO - | E: 10mount: rmdir: failed to remove
> > '/var/run/schroot/mount/isar-builder-31888f6e-6228-4bc8-9570-
> > dd4d69ad25ba-367048-331187ba-b359-48ad-91f4-2962f4bdf32b': Device
> > or resource busy
> > INFO - | E:
> > isar-builder-31888f6e-6228-4bc8-9570-dd4d69ad25ba-367048-331187ba-
> > b359-48ad-91f4-2962f4bdf32b: Chroot setup failed: stage=setup-stop
> > INFO - NOTE: recipe .......: task do_image_swu: Failed
> > ERROR - ERROR: Task (/builds/.......bb:do_image_swu) failed with
> > exit
> > code '1'
> >
> > INFO - | E: 10mount: umount:
> > /run/schroot/mount/isar-builder-5f43627d-fad6-488f-bade-
> > 57d0635ee0c6-408837-d15a1d4b-8c19-4fc8-9b5e-d2ccb3d67d82: target is
> > busy.
> > INFO - | E: 10mount: rmdir: failed to remove
> > '/var/run/schroot/mount/isar-builder-5f43627d-fad6-488f-bade-
> > 57d0635ee0c6-408837-d15a1d4b-8c19-4fc8-9b5e-d2ccb3d67d82': Device
> > or resource busy
> > INFO - | E:
> > isar-builder-5f43627d-fad6-488f-bade-57d0635ee0c6-408837-d15a1d4b-
> > 8c19-4fc8-9b5e-d2ccb3d67d82: Chroot setup failed: stage=setup-stop
> > INFO - NOTE: recipe ......: task do_image_ova: Failed
> > ERROR - ERROR: Task (/builds/.....bb:do_image_ova) failed with exit
> > code '1'
> >
> > Do you have any information regarding the root cause?
>
> Trying once more to pull Cedric in (I think I used the wrong address
> last time): Weren't you trying to resolve that with a lazy umount? Or
> was it for a different corner of isar?
Correct. On our side, this issue is seen sporadically on servers
managed by our IT where they have some proprietary / black-box services
scanning the file-system when new mounts are discovered...
Have not found an elegant way to fix or work-around this (and obviously
no luck in getting IT to remove their software or configure it
differently)
>
> But even if we lazily umount, this won't help when the next command
> is
> trying to rmdir the mount point...
>
> Does anyone have a feeling when this started? Does it correlate with
> any
> major changes in isar?
>
> Jan
>
> >
> > Thanks,
> > Gokhan
> >
> > On Wednesday, June 5, 2024 at 11:55:34 AM UTC+3 Jan Kiszka wrote:
> >
> > On 05.06.24 10:47, 'Schultschik, Sven' via isar-users wrote:
> > > Hi all,
> > >
> > > Our ISAR setup is running smooth on many machines different
> > sizes.
> > But
> > > on our CI/CD pipeline it throws from time to time an error.
> > Starting it
> > > over again it works.
> > >
> > > Does someone knows this kind of error and where I have to
> > look or how
> > > to fix?
> > >
> > > 2024-06-05 04:00:41 - INFO - | Processing triggers for man-db
> > > (2.11.2-2) ...
> > > 2024-06-05 04:00:41 - INFO - | e2fsck 1.47.0 (5-Feb-2023)
> > > 2024-06-05 04:00:41 - INFO - | Pass 1: Checking inodes,
> > blocks, and
> > > sizes
> > > 2024-06-05 04:00:41 - INFO - | Pass 2: Checking directory
> > structure
> > > 2024-06-05 04:00:41 - INFO - | Pass 3: Checking directory
> > > connectivity
> > > 2024-06-05 04:00:41 - INFO - | Pass 3A: Optimizing
> > directories
> > > 2024-06-05 04:00:41 - INFO - | Pass 4: Checking reference
> > counts
> > > 2024-06-05 04:00:41 - INFO - | Pass 5: Checking group summary
> > > information
> > > 2024-06-05 04:00:41 - INFO - |
> > > 2024-06-05 04:00:41 - INFO - | /home/builder/edge-debug-
> > machine-
> > > std/deploy/debug-machine-std.ext4: ***** FILE SYSTEM WAS
> > MODIFIED
> > *****
> > > 2024-06-05 04:00:41 - INFO - | /home/builder/debug-machine-
> > > std/deploy/debug-machine-std.ext4: 25039/77600 files (0.1%
> > non-
> > > contiguous), 292861/310076 blocks
> > > 2024-06-05 04:00:41 - INFO - | E: 10mount: umount:
> > > /run/schroot/mount/isar-builder-e7f8d532-aa0e-41cf-85dd-
> > 11546112ec7c-
> > > 982511-2d2ada00-82dd-45bc-9bd4-f9629d1a55f3: target is busy.
> > > 2024-06-05 04:00:41 - INFO - | E: 10mount: rmdir: failed to
> > remove
> > > '/var/run/schroot/mount/isar-builder-e7f8d532-aa0e-41cf-85dd-
> > > 11546112ec7c-982511-2d2ada00-82dd-45bc-9bd4-f9629d1a55f3':
> > Device or
> > > resource busy
> > > 2024-06-05 04:00:41 - INFO - | E: isar-builder-e7f8d532-aa0e-
> > 41cf-
> > > 85dd-11546112ec7c-982511-2d2ada00-82dd-45bc-9bd4-
> > f9629d1a55f3: Chroot
> > > setup failed: stage=setup-stop
> > > 2024-06-05 04:00:41 - INFO - NOTE: recipe debug-1.0-r0: task
> > > do_image_ext4: Failed
> > > 2024-06-05 04:00:41 - ERROR - ERROR: Task
> > (/build/../repo/recipes-
> > > core/images/debug.bb:do_image_ext4) failed with exit code '1'
> > > 2024-06-05 04:00:42 - INFO - NOTE: Tasks Summary: Attempted
> > 896
> > > tasks of which 0 didn't need to be rerun and 1 failed.
> > > 2024-06-05 04:00:43 - INFO -
> > > 2024-06-05 04:00:43 - INFO - Summary: 1 task failed:
> > > 2024-06-05 04:00:43 - INFO - /build/../repo/recipes-
> > > core/images/debug.bb:do_image_ext4
> > > 2024-06-05 04:00:43 - INFO - Summary: There was 1 WARNING
> > message.
> > > 2024-06-05 04:00:43 - INFO - Summary: There was 1 ERROR
> > message,
> > > returning a non-zero exit code.
> > > 2024-06-05 04:00:43 - ERROR - Command "/build$
> > > /work/isar/bitbake/bin/bitbake -c build debug" failed
> > > --- Error summary ---
> > > ERROR: debug-1.0-r0 do_image_ext4:
> > > ExecutionError('/build/tmp/work/arm64/debug-machine-std/1.0-
> > > r0/temp/run.do_image_ext4.982511', 1, None, None)
> > > ERROR: Logfile of failure stored in:
> > /build/tmp/work/arm64/debug-
> > > machine-std/1.0-r0/temp/log.do_image_ext4.982511
> > > ERROR: Task
> > (/build/../repo/recipes-core/images/debug.bb:do_image_ext4)
> > > failed with exit code '1'
> > >
> >
> > Cedric, weren't you seeing something very similar?
> >
> > Jan
> >
> > --
> > Siemens AG, Technology
> > Linux Expert Center
> >
> > --
> > You received this message because you are subscribed to the Google
> > Groups "isar-users" group.
> > To unsubscribe from this group and stop receiving emails from it,
> > send
> > an email to isar-users+unsubscribe@googlegroups.com
> > <mailto:isar-users+unsubscribe@googlegroups.com>.
> > To view this discussion on the web visit
> > https://groups.google.com/d/msgid/isar-users/d2508b62-d2fb-44a7-936f-aa2367347662n%40googlegroups.com
> > <
> > https://groups.google.com/d/msgid/isar-users/d2508b62-d2fb-44a7-936f-aa2367347662n%40googlegroups.com?utm_medium=email&utm_source=footer
> > >.
>
--
Cedric Hombourger
Siemens AG
http://www.siemens.com/
--
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/isar-users/10335bb9aa9443e71402b883db9ab30c92af6355.camel%40siemens.com.
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: schroot umount target is busy
2024-07-09 6:52 ` 'cedric.hombourger@siemens.com' via isar-users
@ 2024-07-09 10:27 ` 'Jan Kiszka' via isar-users
0 siblings, 0 replies; 11+ messages in thread
From: 'Jan Kiszka' via isar-users @ 2024-07-09 10:27 UTC (permalink / raw)
To: Hombourger, Cedric (DI CTO FDS CES LX),
Schultschik, Sven (DI PA DCP R&D 2),
isar-users, ctngkhn
On 09.07.24 08:52, Hombourger, Cedric (DI CTO FDS CES LX) wrote:
> On Mon, 2024-07-08 at 17:13 +0200, Jan Kiszka wrote:
>> On 08.07.24 16:17, Gökhan Çetin wrote:
>>> Hi everyone,
>>>
>>> Similarly, we have been observing do_image tasks fail sporadically
>>> in
>>> CI/CD pipelines for a while.
>>> You can see a few examples below:
>>>
>>> INFO - | E: 10mount: umount:
>>> /run/schroot/mount/isar-builder-31888f6e-6228-4bc8-9570-
>>> dd4d69ad25ba-367048-331187ba-b359-48ad-91f4-2962f4bdf32b: target is
>>> busy.
>>> INFO - | E: 10mount: rmdir: failed to remove
>>> '/var/run/schroot/mount/isar-builder-31888f6e-6228-4bc8-9570-
>>> dd4d69ad25ba-367048-331187ba-b359-48ad-91f4-2962f4bdf32b': Device
>>> or resource busy
>>> INFO - | E:
>>> isar-builder-31888f6e-6228-4bc8-9570-dd4d69ad25ba-367048-331187ba-
>>> b359-48ad-91f4-2962f4bdf32b: Chroot setup failed: stage=setup-stop
>>> INFO - NOTE: recipe .......: task do_image_swu: Failed
>>> ERROR - ERROR: Task (/builds/.......bb:do_image_swu) failed with
>>> exit
>>> code '1'
>>>
>>> INFO - | E: 10mount: umount:
>>> /run/schroot/mount/isar-builder-5f43627d-fad6-488f-bade-
>>> 57d0635ee0c6-408837-d15a1d4b-8c19-4fc8-9b5e-d2ccb3d67d82: target is
>>> busy.
>>> INFO - | E: 10mount: rmdir: failed to remove
>>> '/var/run/schroot/mount/isar-builder-5f43627d-fad6-488f-bade-
>>> 57d0635ee0c6-408837-d15a1d4b-8c19-4fc8-9b5e-d2ccb3d67d82': Device
>>> or resource busy
>>> INFO - | E:
>>> isar-builder-5f43627d-fad6-488f-bade-57d0635ee0c6-408837-d15a1d4b-
>>> 8c19-4fc8-9b5e-d2ccb3d67d82: Chroot setup failed: stage=setup-stop
>>> INFO - NOTE: recipe ......: task do_image_ova: Failed
>>> ERROR - ERROR: Task (/builds/.....bb:do_image_ova) failed with exit
>>> code '1'
>>>
>>> Do you have any information regarding the root cause?
>>
>> Trying once more to pull Cedric in (I think I used the wrong address
>> last time): Weren't you trying to resolve that with a lazy umount? Or
>> was it for a different corner of isar?
>
> Correct. On our side, this issue is seen sporadically on servers
> managed by our IT where they have some proprietary / black-box services
> scanning the file-system when new mounts are discovered...
>
> Have not found an elegant way to fix or work-around this (and obviously
> no luck in getting IT to remove their software or configure it
> differently)
Hehe, this is the point where moving to a VM will both be more secure
and faster.
Seriously, all we could do for such a scenario is adding some finite
retry loop. Messy.
Jan
--
Siemens AG, Technology
Linux Expert Center
--
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/isar-users/4e811a3d-fbfb-4030-ad02-f1cc0fa66799%40siemens.com.
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: schroot umount target is busy
2024-07-09 6:28 ` Gökhan Çetin
2024-07-09 6:48 ` Anton Mikanovich
@ 2024-07-12 7:04 ` Anton Mikanovich
2024-08-06 8:02 ` Anton Mikanovich
1 sibling, 1 reply; 11+ messages in thread
From: Anton Mikanovich @ 2024-07-12 7:04 UTC (permalink / raw)
To: Gökhan Çetin, isar-users, Kiszka, Jan,
cedric.hombourger, Baurzhan Ismagulov
Hello Gökhan,
I've found an issue with isar-apt usage without lock protection in
deb_dl_dir_export and imager_run implementations. This can influence on
umount
issues you've observed, so please try the fix from patchset:
[PATCH 0/2] Protect isar-apt usages in shell
--
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/isar-users/8a611b46-370e-46f4-8ebc-725f6dbfbef5%40ilbers.de.
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: schroot umount target is busy
2024-07-12 7:04 ` Anton Mikanovich
@ 2024-08-06 8:02 ` Anton Mikanovich
2024-08-06 8:19 ` Gökhan Çetin
0 siblings, 1 reply; 11+ messages in thread
From: Anton Mikanovich @ 2024-08-06 8:02 UTC (permalink / raw)
To: Gökhan Çetin, isar-users, Kiszka, Jan,
cedric.hombourger, Baurzhan Ismagulov
12/07/2024 10:04, Anton Mikanovich wrote:
> Hello Gökhan,
>
> I've found an issue with isar-apt usage without lock protection in
> deb_dl_dir_export and imager_run implementations. This can influence
> on umount
> issues you've observed, so please try the fix from patchset:
>
> [PATCH 0/2] Protect isar-apt usages in shell
Unfortunately this fix didn't help.
Even with it we've got the same issue (it looks like there is the first time
we've catched it in our CI):
ERROR: Task
(mc:qemuarm-bullseye:/workspace/build-auto/isar_next/1324/meta-test/recipes-core/images/isar-image-ci.bb:do_image_ext4)
failed with exit code '1'
E: 10mount: umount:
/run/schroot/mount/isar-jenkins-0c03d5d2-da53-444c-93f6-fc9bdfafc290-2097734-1868b26c-f678-4e99-9e9a-360c7c05fcd3:
target is busy.
E: 10mount: rmdir: failed to remove
'/var/run/schroot/mount/isar-jenkins-0c03d5d2-da53-444c-93f6-fc9bdfafc290-2097734-1868b26c-f678-4e99-9e9a-360c7c05fcd3':
Device or resource busy
E:
isar-jenkins-0c03d5d2-da53-444c-93f6-fc9bdfafc290-2097734-1868b26c-f678-4e99-9e9a-360c7c05fcd3:
Chroot setup failed: stage=setup-stop
--
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/isar-users/a53a8580-854b-481c-a788-16fdab357e88%40ilbers.de.
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: schroot umount target is busy
2024-08-06 8:02 ` Anton Mikanovich
@ 2024-08-06 8:19 ` Gökhan Çetin
0 siblings, 0 replies; 11+ messages in thread
From: Gökhan Çetin @ 2024-08-06 8:19 UTC (permalink / raw)
To: isar-users
[-- Attachment #1.1: Type: text/plain, Size: 2082 bytes --]
Hi Anton,
I haven't had a chance to try the patch you mentioned yet.
But this fix
<https://github.com/ilbers/isar/commit/ed2f8fda7cc85309c4f9a1f49a3f4b96df9886bb> has
greatly reduced the error frequency on our side.
As Cedric mentioned, we also have managed servers and several IT services
that might cause such issues.
That's why, I can say that we have achieved acceptable stability for now.
Thanks,
Gokhan
On Tuesday, August 6, 2024 at 11:02:55 AM UTC+3 Anton Mikanovich wrote:
> 12/07/2024 10:04, Anton Mikanovich wrote:
> > Hello Gökhan,
> >
> > I've found an issue with isar-apt usage without lock protection in
> > deb_dl_dir_export and imager_run implementations. This can influence
> > on umount
> > issues you've observed, so please try the fix from patchset:
> >
> > [PATCH 0/2] Protect isar-apt usages in shell
>
> Unfortunately this fix didn't help.
> Even with it we've got the same issue (it looks like there is the first
> time
> we've catched it in our CI):
>
> ERROR: Task
> (mc:qemuarm-bullseye:/workspace/build-auto/isar_next/1324/meta-test/recipes-core/images/isar-image-ci.bb:do_image_ext4)
>
> failed with exit code '1'
> E: 10mount: umount:
> /run/schroot/mount/isar-jenkins-0c03d5d2-da53-444c-93f6-fc9bdfafc290-2097734-1868b26c-f678-4e99-9e9a-360c7c05fcd3:
>
> target is busy.
> E: 10mount: rmdir: failed to remove
> '/var/run/schroot/mount/isar-jenkins-0c03d5d2-da53-444c-93f6-fc9bdfafc290-2097734-1868b26c-f678-4e99-9e9a-360c7c05fcd3':
>
> Device or resource busy
> E:
> isar-jenkins-0c03d5d2-da53-444c-93f6-fc9bdfafc290-2097734-1868b26c-f678-4e99-9e9a-360c7c05fcd3:
>
> Chroot setup failed: stage=setup-stop
>
>
--
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/isar-users/f9dac9c3-e29e-4093-a3c0-18f01e5317c2n%40googlegroups.com.
[-- Attachment #1.2: Type: text/html, Size: 2762 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2024-08-06 8:19 UTC | newest]
Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-06-05 8:47 schroot umount target is busy Schultschik, Sven
2024-06-05 8:55 ` Jan Kiszka
2024-07-08 14:17 ` Gökhan Çetin
2024-07-08 15:13 ` 'Jan Kiszka' via isar-users
2024-07-09 6:28 ` Gökhan Çetin
2024-07-09 6:48 ` Anton Mikanovich
2024-07-12 7:04 ` Anton Mikanovich
2024-08-06 8:02 ` Anton Mikanovich
2024-08-06 8:19 ` Gökhan Çetin
2024-07-09 6:52 ` 'cedric.hombourger@siemens.com' via isar-users
2024-07-09 10:27 ` 'Jan Kiszka' via isar-users
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox