From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7339855054696873984 X-Forwarded-Encrypted: i=2; AJvYcCUjPbEo+CO4QzXEymcdUZO/3Cr0q6PDHSP8Qk2XushN1gSS5vy8ubKXD2q9G0Ame1CCe98K63tAWEbzYIxUOModUG2l6JwohGLVMMU= X-Received: by 2002:aa7:df93:0:b0:565:e294:5119 with SMTP id b19-20020aa7df93000000b00565e2945119mr1841095edy.18.1708946828682; Mon, 26 Feb 2024 03:27:08 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:6402:4022:b0:563:7e67:4a84 with SMTP id d34-20020a056402402200b005637e674a84ls32538eda.1.-pod-prod-08-eu; Mon, 26 Feb 2024 03:27:06 -0800 (PST) X-Forwarded-Encrypted: i=2; AJvYcCUaNKbK8fneOnqdJWAzaJgMnttd7EMItEysVH4CV0RjPt6Fhvez1Yfctgktk7gzxNs9Hixtc5uy0aMXcPPfY7vzj1UkfBlZovy/NYs= X-Google-Smtp-Source: AGHT+IGmGFvGEZGb957RxXX4q06FyPkRIpzGHGu4IBDg8bWktXros95WyOoU+jqikViev8cFuj2S X-Received: by 2002:aa7:c250:0:b0:565:221a:e9f8 with SMTP id y16-20020aa7c250000000b00565221ae9f8mr4506563edo.26.1708946826409; Mon, 26 Feb 2024 03:27:06 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1708946826; cv=none; d=google.com; s=arc-20160816; b=dHNVYEllokD8SiddJjw/vuEGdyxjsBZaQadYtnxipvPobSqhl1m/cDIG6dIg2Ed0Dp RMmjw9oHTuB0MkXGVwKluAlxDO2mmqY9+guPqQN0OgkCQK7ve8t3d2EK7q7fC9a9cZa6 2RavnJxLqHMePNzLrB/F3eDIV1pa6MpAHR2K4U0egUmgvqPpo+QAJBTa8aqw8a/SFf5T pZ/XYh+W61PwrcqOcgJ62au6Hx6FjlLj98ozcs+yORf4ezqHxMsWbGTqxHZDTUNlJjGu xC8i7GYWz12xME8LQPullmjbGGaK+TP4decMYds7gYQQrk8JNqvd0T+9kqFN8WxIoSGz Y6Jw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:in-reply-to:from:references:to :content-language:subject:user-agent:mime-version:date:message-id; bh=wTxZfly1hAwQlLQ6LRuw9NFTJfudh7F4EwrM5nqc9pE=; fh=ZjPjk0Tv6NIOienZPuWkm6ez+WL6tmkYxcG6aGEkSMA=; b=eod4sLvDE+v2gUWzkhHYzoIPvetVpm2mFtahlkX2srLortIYpSHeR//OCgBKh1BIJ6 seRG+Qe1IG57/twwWAkspcuz24QSZTicj2SkW2fPeLuBlg8R2FoEBaS5ZhPgc5YpDb/F Rqa2cBeE4j7jAOmTaj7hJ2UvqVhNB3T8+y9Jr7XLp5hNyuC9ABSeM+OrYpFlr5fD5kki vaO93b9ZHlzEded14rzyUJOOWj4/keBk6esanho+eYzBX34Sl+Kl+9I5vStgbj9VZLe5 phh/1z2ISvIWqKKQkcPxZ6TMC0oGICS74N8ByJ8UTzy3YRtqsmFQF/w9BHjuC8NmCNOo Ixyg==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Return-Path: Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id v24-20020a50d598000000b00564af3e693fsi437740edi.5.2024.02.26.03.27.06 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Mon, 26 Feb 2024 03:27:06 -0800 (PST) Received-SPF: pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Received: from [127.0.0.1] (host-80-81-17-52.static.customer.m-online.net [80.81.17.52]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 41QBR3Tr025586 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 26 Feb 2024 12:27:04 +0100 Message-ID: <0173efd6-b53b-4f8c-b6cb-ef52ff22db8e@ilbers.de> Date: Mon, 26 Feb 2024 13:27:03 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [HELP] tmp/work/debian-bullseye-arm64/isar-image-base-qemuarm64/1.0-r0/mnt/rootfs: target is busy Content-Language: en-US To: "cedric.hombourger@siemens.com" , "isar-users@googlegroups.com" References: <87ba787bf61b06a69fe8bcfeb08a728c32f4878e.camel@siemens.com> From: Anton Mikanovich In-Reply-To: <87ba787bf61b06a69fe8bcfeb08a728c32f4878e.camel@siemens.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-TUID: aLBwxtNaGYl4 26/02/2024 12:26, 'cedric.hombourger@siemens.com' via isar-users wrote: > Hello, > > Seeing some sporadic failures with the test suite (but also with builds > of our Isar-based product) when rootfs_install_sstate_prepare gets > executed: > > DEBUG: Executing shell function rootfs_install_sstate_prepare > umount: /home/sutlej/isar/build/tmp/work/debian-bullseye-arm64/isar- > image-base-qemuarm64/1.0-r0/mnt/rootfs: target is busy. > WARNING: exit code 32 from a shell command. > > What's special about the machine I am running the builds on is that > /proc/cpuinfo reports 64 processors hence builds get massively > parallelized > > I wasn't able to get to the bottom of this issue and understand why the > bind mount is busy since rootfs_install_sstate_prepare creates, uses > and removes that bind mount in the same function. > > As a work-around, a lazy umount could be used but that annoys me as I'd > like to understand what could cause this. Any ideas? > > Cedric > Hello Cedric, Does this issue happens with testsuite only or on manual build also? How much targets and how much images for the same target are been executed in this bitbake call? Which Isar commit are you using? We can try to reproduce the issue but need some details of your setup. The issue like that can be caused by double-mount scenario we already faced with: if some of the folders in Isar sources/builddir path is mounted inside itself or its parent, it can cause doubling the mount and umount will fail. Here is an example of this scenario: $ mkdir dirA $ mkdir dirB $ echo "sudo umount ~/dirB" > dirA/um $ sudo mount --bind dirA dirB $ mount | grep dir /dev/sdb1 on /path/dirB type ext4 (rw,relatime) $ sudo mount --bind dirA dirB $ mount | grep dir /dev/sdb1 on /path/dirB type ext4 (rw,relatime) /dev/sdb1 on /path/dirB type ext4 (rw,relatime) /dev/sdb1 on /path/dirA type ext4 (rw,relatime) $ bash ~/dirA/um umount: /path/dirB: target is busy. As a debug you can try to add some lsof/mount/findmnt inside the rootfs_install_sstate_prepare task before umount/ P.S. lazy umount is not an option because it will only cover the issue which can cause other issues to appear later on.