From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6521574339082452992 X-Received: by 10.25.219.137 with SMTP id t9mr310094lfi.11.1518599389438; Wed, 14 Feb 2018 01:09:49 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 10.25.193.87 with SMTP id r84ls697467lff.9.gmail; Wed, 14 Feb 2018 01:09:48 -0800 (PST) X-Google-Smtp-Source: AH8x227gV/hBgRU9o9vb8D+F780bIHsLz+EiWLWvqoIgnycmWl6lQyxQAjoW6VQnD48znuiajWyn X-Received: by 10.25.160.195 with SMTP id j186mr309541lfe.44.1518599388599; Wed, 14 Feb 2018 01:09:48 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1518599388; cv=none; d=google.com; s=arc-20160816; b=DUP75WeZXVC5LsiTDKPTdTZAYwxdOGO85TALwgjGRakp6synFr96EiRdzfmbjTzwEb y4r7c44wZu1UT6Y+OETgtr+58EADQq+fOgiGdhlrxMrwtRdS6VodOe4BFbYw1zccgbDb O49LbOQRPwBM+bzDvY+zu+nyTG9MS9TiQhTmG6x1AGpNyzPkx7vB50xKzypljOx6J0dn nJ9Qd216o+m53LbjwLt8op7qh+C8ptFCScGFjoniHcvaMvd/myyE+AAy0yzUam46AYOb 45TSA69gTaVOF/SBQUoqLD2TQMUj/sVKKrzuK6MAiJJIQ0q9jnP4hoIU1V46P5zruGDt /rEQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:to:subject :arc-authentication-results; bh=ZulZ5jkf2NivcCmFYZo8SHBd06W7SU1omBcEjOdLk8E=; b=ZIDn+sNEHqghrJncMp/H8sRWTEs7Pw0BQrjz4E2dn0k5srfeRFH9m9y+etR8MFD2m4 lJxHONKm511WIhUXyplFjV55VwI1+fbxAKuTpY4NLnkxar/N4J/enOsfNnGmUwPQ9AgI UVtzQROT0vQ+4OxKQLJU+d2K3158nGzqI2kNPzuItBApUKf3yV+dLZSeZrP2lQ6dxmpN R1OAHPsRm0BqVNcT2rWm7XmgngBweLqv+ujOxUmixkcnpRKilF3CqD2dB+bPWcCwglQP iT063be2KTnTev0W3kJRo2jwF+H1LVx1AfA+tOPALiMBbr0u8qk0s8ocU+ZKlQZiP8ZR CRDw== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) smtp.mailfrom=asmirnov@ilbers.de Return-Path: Received: from aqmola.ilbers.de (aqmola.ilbers.de. [85.214.62.211]) by gmr-mx.google.com with ESMTPS id u10si704741lju.0.2018.02.14.01.09.48 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 14 Feb 2018 01:09:48 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) client-ip=85.214.62.211; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) smtp.mailfrom=asmirnov@ilbers.de Received: from [10.0.2.15] ([188.227.110.165]) (authenticated bits=0) by aqmola.ilbers.de (8.14.4/8.14.4/Debian-4+deb7u1) with ESMTP id w1E99iL7009531 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Wed, 14 Feb 2018 10:09:46 +0100 Subject: Re: [PATCH v3 5/9] Move buildchroot mounts into dpkg-base class To: Jan Kiszka , isar-users References: <697b09bb60a515430f655989e859263922148264.1518552327.git.jan.kiszka@siemens.com> <0ce19aaa-263d-7d13-18ea-5a665f59852d@siemens.com> From: Alexander Smirnov Message-ID: Date: Wed, 14 Feb 2018 12:09:39 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: <0ce19aaa-263d-7d13-18ea-5a665f59852d@siemens.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-TUID: x1WJtIp3Bc87 On 02/14/2018 11:52 AM, Jan Kiszka wrote: > From: Jan Kiszka > > This avoids the unfortunate setup_mounts task which is either not > executed on rebuilds or causes excessive package rebuilds. We just need > ensure prior to switching into the buildchroot that all mount points are > populated. The mounts may race with parallel build, so lock-protect the > check if they are needed and their execution. > > Note that the more beautiful syntax using ( flock -n 9; ... ) 9>lockfile > as described in the flock man page does not work in bitbake shell > functions. > > Signed-off-by: Jan Kiszka > --- > > Changes in v3: > - use flock instead of unreliable check pattern > > meta/classes/dpkg-base.bbclass | 11 ++++++++- > meta/recipes-devtools/buildchroot/buildchroot.bb | 30 +++--------------------- > 2 files changed, 13 insertions(+), 28 deletions(-) > > diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass > index a45fbce..895ff26 100644 > --- a/meta/classes/dpkg-base.bbclass > +++ b/meta/classes/dpkg-base.bbclass > @@ -2,7 +2,7 @@ > # Copyright (C) 2017 Siemens AG > > # Add dependency from buildchroot creation > -do_build[depends] = "buildchroot:do_setup_mounts" > +do_build[depends] = "buildchroot:do_build" > > # Add dependency between Isar recipes > DEPENDS ?= "" > @@ -20,11 +20,20 @@ dpkg_runbuild() { > die "This should never be called, overwrite it in your derived class" > } > > +MOUNT_LOCKFILE = "${TMPDIR}/buildchroot-mount.lock" With this name lockfile will be shared for all multiconfigs. I realize that it's small, but unnecessary serialization of parallel parts. Would it be ok? MOUNT_LOCKFILE = "${TMPDIR}/buildchroot-mount-${DISTRO}-${DISTRO_ARCH}.lock" or MOUNT_LOCKFILE = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/buildchroot/mount.lock" or MOUNT_LOCKFILE = "${BUILDCHROOT_DIR}/mount.lock" Alex > + > # Wrap the function dpkg_runbuild with the bind mount for buildroot > do_build() { > mkdir -p ${BUILDROOT} > sudo mount --bind ${WORKDIR} ${BUILDROOT} > > + sudo flock ${MOUNT_LOCKFILE} -c ' \ > + if ! grep -q ${BUILDCHROOT_DIR}/isar-apt /proc/mounts; then \ > + mount --bind ${DEPLOY_DIR_APT}/${DISTRO} ${BUILDCHROOT_DIR}/isar-apt; \ > + mount -t devtmpfs -o mode=0755,nosuid devtmpfs ${BUILDCHROOT_DIR}/dev; \ > + mount -t proc none ${BUILDCHROOT_DIR}/proc; \ > + fi' > + > dpkg_runbuild > > sudo umount ${BUILDROOT} 2>/dev/null || true > diff --git a/meta/recipes-devtools/buildchroot/buildchroot.bb b/meta/recipes-devtools/buildchroot/buildchroot.bb > index 570c0ad..cd91228 100644 > --- a/meta/recipes-devtools/buildchroot/buildchroot.bb > +++ b/meta/recipes-devtools/buildchroot/buildchroot.bb > @@ -60,7 +60,9 @@ do_build() { > -e 's|##DIR_HOOKS##|./'"$WORKDIR_REL"'/hooks_multistrap|g' \ > "${WORKDIR}/multistrap.conf.in" > "${WORKDIR}/multistrap.conf" > > - do_setup_mounts > + sudo mount --bind ${DEPLOY_DIR_APT}/${DISTRO} ${BUILDCHROOT_DIR}/isar-apt > + sudo mount -t devtmpfs -o mode=0755,nosuid devtmpfs ${BUILDCHROOT_DIR}/dev > + sudo mount -t proc none ${BUILDCHROOT_DIR}/proc > > # Create root filesystem > sudo -E multistrap -a ${DISTRO_ARCH} -d "${BUILDCHROOT_DIR}" -f "${WORKDIR}/multistrap.conf" > @@ -70,30 +72,4 @@ do_build() { > > # Configure root filesystem > sudo chroot ${BUILDCHROOT_DIR} /configscript.sh > - > - do_cleanup_mounts > -} > - > -# Invalidate stamp for do_setup_mounts before each build start. > -# This will guarantee that this function will be executed once > -# per build. > -python __anonymous() { > - stamp = d.getVar("STAMP") + ".do_setup_mounts." + d.getVarFlag("do_setup_mounts", 'stamp-extra-info') > - os.remove(stamp) if os.path.exists(stamp) else None > -} > - > -do_setup_mounts[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" > - > -do_setup_mounts() { > - sudo mount --bind ${DEPLOY_DIR_APT}/${DISTRO} ${BUILDCHROOT_DIR}/isar-apt > - sudo mount -t devtmpfs -o mode=0755,nosuid devtmpfs ${BUILDCHROOT_DIR}/dev > - sudo mount -t proc none ${BUILDCHROOT_DIR}/proc > -} > - > -addtask setup_mounts after do_build > - > -do_cleanup_mounts() { > - sudo umount ${BUILDCHROOT_DIR}/isar-apt 2>/dev/null || true > - sudo umount ${BUILDCHROOT_DIR}/dev 2>/dev/null || true > - sudo umount ${BUILDCHROOT_DIR}/proc 2>/dev/null || true > } >