Re: [PATCH] mount: Cleanup reference counters before build

[Anton Mikanovich <amikan@ilbers.de>]

29.06.2021 16:49, Jan Kiszka wrote:
> We need a clearer picture what it fixes, what it possibly can't fix (I
> doubt it can fix all error cases), and if we still want it then. Maybe
> start with documenting the API changes and their reasoning.
>
> Also describe more clearly how to reproduce the issues you were seeing
> and trying to fix. That is key to understand if the new solution is
> actually a solution or just shifting the problem from left to right.

To reproduce the original issue (fixed by `Rebuild mount logic` 
patchset), a task running twice for two targets (linux-mainline in our 
case) should fail, the second task run should start and succeed:

To have the original issue fixed by `Rebuild mount logic` patchset be 
reproduced you need one recipe to be included into two targets, and this 
recipe should also failed on the first build attempt, but succeed on 
second one. Moreover, this issue will be reproduced only in case the 
second run will start directly after the fail:

 >bitbake mc:de0-nano-soc-buster:isar-image-base 
mc:stm32mp15x-buster:isar-image-base
 >...
 >ERROR: Logfile of failure stored in: 
/opt/isar/build/tmp/work/debian-buster-armhf/linux-mainline/5.4.70-r0/temp/log.do_dpkg_build.12459
 >NOTE: recipe linux-mainline-5.4.70-r0: task do_dpkg_build: Failed
 >ERROR: Task 
(mc:stm32mp15x-buster:/opt/isar/meta-isar/recipes-kernel/linux/linux-mainline_5.4.70.bb:do_dpkg_build) 
failed with exit code '1'
 >NOTE: Running task 304 of 350 
(mc:de0-nano-soc-buster:/opt/isar/meta-isar/recipes-kernel/linux/linux-mainline_5.4.70.bb:do_dpkg_build)
 >NOTE: recipe linux-mainline-5.4.70-r0: task do_dpkg_build: Started
 >WARNING: mc:de0-nano-soc-buster:linux-mainline-5.4.70-r0 
do_dpkg_build: 
/opt/isar/build/tmp/deploy/buildchroot-target/debian-buster-armhf//home/builder/linux-mainline: 
Couldn't unmount, retrying...

In most cases second one does not start and bitbake exit due to the 
error, but when it start the build is went to the infinite loop.

In that scenario:
1) mc:stm32mp15x-buster:linux-mainline:do_dpkg_build perform mounting of 
WORKDIR to buildchroot-target/WORKDIR, then failed and leave WORKDIR to 
be mounted.
2) mc:de0-nano-soc-buster:linux-mainline:do_dpkg_build perform the same 
mount again which lead to circullar mount 
WORKDIR->WORKDIR->buildchroot-target/WORKDIR, then package build is 
performed without errors.
3) mc:de0-nano-soc-buster:linux-mainline:do_dpkg_build stuck inside 
dpkg_undo_mounts. When package build finish it will put 
run.dpkg_undo_mounts.PID shell function into WORKDIR/temp and execute. 
But when run.dpkg_undo_mounts.PID try to unmount 
buildchroot-target/WORKDIR the kernel will also unmount circullar mount 
to WORKDIR which is locked by run.dpkg_undo_mounts.PID itself.

This scenario can be demonstrated with the following steps:

$ mkdir dirA
$ mkdir dirB
$ echo "sudo umount /home/user/dirB" > dirA/um
$ sudo mount --bind dirA dirB
$ mount | grep dir
/dev/sdb1 on /home/user/dirB type ext4 (rw,relatime)
$ sudo mount --bind dirA dirB
$ mount | grep dir
/dev/sdb1 on /home/user/dirB type ext4 (rw,relatime)
/dev/sdb1 on /home/user/dirB type ext4 (rw,relatime)
/dev/sdb1 on /home/user/dirA type ext4 (rw,relatime)
$ bash ~/dirA/um
umount: /home/user/dirB: target is busy.

So mount rebuild patchset actually make dpkg_undo_mounts to be always 
run even after build fail and not allow double mounting.
It was done only for dpkg_do_mounts/dpkg_undo_mounts because only for 
this scenario double mount is critical and can lead to dead lock. 
Implementation can be completed for all the mounts if needed.

We understand that the API change is quite significant. I'm checking 
whether the final unmounting could be solved without change the API.


P.S. in case anyone will try to reproduce the original issue this patch 
can help:

diff --git a/meta-isar/recipes-kernel/linux/linux-mainline_5.4.70.bb 
b/meta-isar/recipes-kernel/linux/linux-mainline_5.4.70.bb
index 209ad9c..49b9739 100644
--- a/meta-isar/recipes-kernel/linux/linux-mainline_5.4.70.bb
+++ b/meta-isar/recipes-kernel/linux/linux-mainline_5.4.70.bb
@@ -32,3 +32,11 @@ dpkg_configure_kernel_append() {
      grep "CONFIG_ROOT_NFS=y" ${S}/${KERNEL_BUILD_DIR}/.config || \
          bbfatal "Self-check failed: CONFIG_ROOT_NFS not enabled"
  }
+
+dpkg_runbuild_prepend() {
+    if [ ! -e "${WORKDIR}/stampfile" ]; then
+        touch "${WORKDIR}/stampfile"
+        sleep 5
+        exit 1
+    fi
+}

-- 
Anton Mikanovich
Promwad Ltd.
External service provider of ilbers GmbH
Maria-Merian-Str. 8
85521 Ottobrunn, Germany
+49 (89) 122 67 24-0
Commercial register Munich, HRB 214197
General Manager: Baurzhan Ismagulov