On 12/3/18 3:59 PM, Jan Kiszka wrote: > On 30.11.18 10:20, Maxim Yu. Osipov wrote: >> Hi Jan, >> >> I've just tried this patch (on the 'next' with reverted patch >> d40a9ac0) and ran "fast" CI >> >> isar$mount | wc -l >> 34 >> >> isar$./scripts/ci_build.sh -q -f >> >> CI script hung on CI stage when dpkg-base is modified >> causing rebuilding recipes based on dpkg-base. >> >> The mount reports less (!) mount points than before launching the script. >> >> mount | wc -l >> 31 > > Any news on what's different on your side? Where exactly does your build > hang? Was your CI environment in a clean state when running this test? > Before the comment lots of things leaked. On my stretch laptop (i7-6820HQ CPU @ 2.70GHz (8 cores) with SSD) the reported problem is reproducible (I rerun 'ci_build.sh -q -f' several times in clean state) it hung and with the less mount points (the mount points before and after running are attached). The strange thing that I observe two bitbake processes: myo 26373 0.0 0.3 153116 29732 pts/0 Sl+ 12:31 0:01 python3 /home/myo/work/isar/src/trunk/isar/bitbake/bin/bitbake multiconfig:qemuarm-stretch:isar-image-base multiconfig:qemuarm64-stretch:isar-image-base multiconfig:qemuamd64-stretch:isar-image-base myo 26379 2.5 0.6 328476 50028 ? Sl 12:31 0:40 python3 /home/myo/work/isar/src/trunk/isar/bitbake/bin/bitbake multiconfig:qemuarm-stretch:isar-image-base multiconfig:qemuarm64-stretch:isar-image-base multiconfig:qemuamd64-stretch:isar-image-base When I turned on verbosity (called ci_build.sh without '-q' switch) 'scripts/ci_build -f' succeeded. Build also fails on isar-build.org http://isar-build.org:8080/job/isar_mosipov_next/98/console: it complains on 'rm: cannot remove '/workspace/build/isar_mosipov_next/98/build/tmp/work/debian-stretch-armhf/isar-bootstrap-target/rootfs/dev': Device or resource busy' Note that ci-build.sh on isar-build.org is also called with '-q' switch. Questions: 1) Did you launch ci_build with '-q' option? 2) Did you inspect mount points before and after launching ci_build? Are they identical? 3) Do you run this in docker? Maxim. > We still do not see problems with this bug fix, neither locally nor in > CI (which is ci_build-based now, too). And it's actually a precondition > to make "isar-image: umount base-apt when doing offline build" work as > expected. > > Thanks, > Jan > >> >> Maxim. >> >> On 11/29/18 9:39 PM, Henning Schild wrote: >>> Maybe that is where the leftover mounts came from, and the reverts are >>> not required. >>> >>> Henning >>> >>> Am Thu, 29 Nov 2018 17:29:17 +0100 >>> schrieb "[ext] Jan Kiszka" : >>> >>>> First, wrong variables were used (there is no BUILDCHROOT_DIR for >>>> isar_bootstrap). And then we simply left the mount points active after >>>> completing the bootstrap. The led to setup_root_file_system copying >>>> all the mount point content over into the buildchroot or rootfs - >>>> including base-apt when it was used. >>>> >>>> Signed-off-by: Jan Kiszka >>>> --- >>>> >>>> This is a precondition for "isar-image: umount base-apt when doing >>>> offline build" to work properly. IOW, that patch reveal these bugs. >>>> >>>> Possibly, this also explains the increasing mount count that was >>>> visible in CI. >>>> >>>>   meta/recipes-core/isar-bootstrap/isar-bootstrap.inc | 21 >>>> +++++++++++---------- 1 file changed, 11 insertions(+), 10 >>>> deletions(-) >>>> >>>> diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc >>>> b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc index >>>> 8ea3b08..da077d0 100644 --- >>>> a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc +++ >>>> b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc @@ -185,12 >>>> +185,8 @@ isar_bootstrap() { sudo -E flock "${ISAR_BOOTSTRAP_LOCK}" >>>> -c "\ set -e >>>>           if [ ! -e "${DEPLOY_ISAR_BOOTSTRAP}" ]; then >>>> -            if [ -e "${ROOTFSDIR}" ]; then >>>> -               umount -R -l "${ROOTFSDIR}/dev" || true >>>> -               umount -l "${ROOTFSDIR}/proc" || true >>>> -               umount -R -l "${ROOTFSDIR}/sys" || true >>>> -               rm -rf "${ROOTFSDIR}" >>>> -            fi >>>> +            rm -rf "${ROOTFSDIR}" >>>> + >>>>               if [ ${IS_HOST} ]; then >>>>                   ${DEBOOTSTRAP} $debootstrap_args \ >>>>                                  ${@get_distro_components_argument(d, >>>> True)} \ @@ -247,17 +243,22 @@ isar_bootstrap() { >>>>               "${ROOTFSDIR}/chroot-setup.sh" "setup" "${ROOTFSDIR}" >>>>               # update APT >>>> -            mount --rbind /dev ${BUILDCHROOT_DIR}/dev >>>> -            mount --make-rslave ${BUILDCHROOT_DIR}/dev >>>> +            mount --rbind /dev ${ROOTFSDIR}/dev >>>> +            mount --make-rslave ${ROOTFSDIR}/dev >>>>               mount -t proc none ${ROOTFSDIR}/proc >>>> -            mount --rbind /sys ${BUILDCHROOT_DIR}/sys >>>> -            mount --make-rslave ${BUILDCHROOT_DIR}/sys >>>> +            mount --rbind /sys ${ROOTFSDIR}/sys >>>> +            mount --make-rslave ${ROOTFSDIR}/sys >>>>               export DEBIAN_FRONTEND=noninteractive >>>>               chroot "${ROOTFSDIR}" /usr/bin/apt-get update -y >>>>               chroot "${ROOTFSDIR}" /usr/bin/apt-get dist-upgrade -y \ >>>>                                     -o Debug::pkgProblemResolver=yes >>>> +            umount -R -l "${ROOTFSDIR}/dev" >>>> +            umount -l "${ROOTFSDIR}/proc" >>>> +            umount -R -l "${ROOTFSDIR}/sys" >>>> +            umount -l "${ROOTFSDIR}/base-apt" || true >>>> + >>>>               # Finalize debootstrap by setting the link in deploy >>>>               ln -Tfsr "${ROOTFSDIR}" "${DEPLOY_ISAR_BOOTSTRAP}" >>>>           fi" >>> >> >> > -- Maxim Osipov ilbers GmbH Maria-Merian-Str. 8 85521 Ottobrunn Germany +49 (151) 6517 6917 mosipov@ilbers.de http://ilbers.de/ Commercial register Munich, HRB 214197 General Manager: Baurzhan Ismagulov