From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6521574336665485312 X-Received: by 10.25.39.65 with SMTP id n62mr182070lfn.0.1518547495899; Tue, 13 Feb 2018 10:44:55 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 10.46.33.225 with SMTP id h94ls1575374lji.0.gmail; Tue, 13 Feb 2018 10:44:55 -0800 (PST) X-Google-Smtp-Source: AH8x2274nKOUFVYCCF/o5bqq8jfNhQQE+kDfRJbbjCW01afl9oBkenbgxSeZWRkandhxMN3Xj2Qm X-Received: by 10.46.87.8 with SMTP id l8mr80199ljb.12.1518547495192; Tue, 13 Feb 2018 10:44:55 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1518547495; cv=none; d=google.com; s=arc-20160816; b=qRUpmq1deyeHzKRZldQbY9H0UFOFErgL0zkAqbuvJHs7v7C/t8488oUVgY9vImMD2t qjahIySapIllMjx33Bi5rnLZgtUtM58GVYjDeL3NthKzGF6GWa4b/sDmSmVitIDkqjsB H6sc1yXs3jfJ5XpQUnV3qlHWJhUvL8bQJDq52tsHVSJecFdj/K9Smc0JhPzO5X+hbwso Oh1Lk3cNp8n4F43mamqoNDBvaC1IH40V7I6UChQ73wWqxbUyxTgTD0FgGE1RQ706BJw6 WGcKQnTrqeio2I74+aYwozPI5REFrTMk1gCvJRebX3Fx4JLqZ9G3Tt9WRWHelUMwlicK KU2g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:to:subject :arc-authentication-results; bh=Frd0GSJL4osINO/tAv4mwV/O7qc5THHLwzRKTjGDSzE=; b=IVMgxb34cOFFFlUlIXXaErlXIyekOBpxjEB/nRNN+4EiT4EV5eKUGJoKD+QWtS6U5L IGo6z1FsTtKSfMWC49edfcM5Cg0wmPv3Li9qSmHNBCkjwkQvQ6maloitWMULhYZyblr9 857JpO5xLjGgjb+N5PfTkkDUEV5tVC/n7NqUyRdE2kGS1iVflQCfa44TjDmBdQUogm62 IHzg5xqB4p1PI2CcKU5HT1WygTOmt12qrU18ERTFzAihM4H2bfRp1BUI5+kFewr1BEdH 0crLnAtphGeoQbMMkoeT2b9Vj4+/QZgVgLcvZ3E511GvlxSoSQWyw6CiT55LSOVELlcM GgpA== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.14 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com Return-Path: Received: from david.siemens.de (david.siemens.de. [192.35.17.14]) by gmr-mx.google.com with ESMTPS id l30si607616ljb.3.2018.02.13.10.44.55 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 13 Feb 2018 10:44:55 -0800 (PST) Received-SPF: pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.14 as permitted sender) client-ip=192.35.17.14; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.14 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com Received: from mail3.siemens.de (mail3.siemens.de [139.25.208.14]) by david.siemens.de (8.15.2/8.15.2) with ESMTPS id w1DIisRm032614 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Feb 2018 19:44:54 +0100 Received: from [139.25.68.37] (md1q0hnc.ad001.siemens.net [139.25.68.37] (may be forged)) by mail3.siemens.de (8.15.2/8.15.2) with ESMTP id w1DIismS021614; Tue, 13 Feb 2018 19:44:54 +0100 Subject: Re: [PATCH 4/6] Enable proper rebuilds on dependency changes To: Alexander Smirnov , isar-users References: <7d0504da7640612abded5ffe1e1b3c6d10168b49.1518422347.git.jan.kiszka@siemens.com> <73bd323b-0dc2-a84a-ae5a-59177cfe6921@ilbers.de> <4b84a0e3-df9e-8343-ff09-7b74474a0e95@siemens.com> From: Jan Kiszka Message-ID: <20278b99-b41b-ee89-b5cd-eafc2019875d@siemens.com> Date: Tue, 13 Feb 2018 19:44:53 +0100 User-Agent: Mozilla/5.0 (X11; U; Linux i686 (x86_64); de; rv:1.8.1.12) Gecko/20080226 SUSE/2.0.0.12-1.1 Thunderbird/2.0.0.12 Mnenhy/0.7.5.666 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-TUID: PIH7R166zUbB On 2018-02-13 19:08, Alexander Smirnov wrote: > On 02/13/2018 08:57 PM, Jan Kiszka wrote: >> On 2018-02-13 17:21, Jan Kiszka wrote: >>> On 2018-02-13 14:03, Alexander Smirnov wrote: >>>> On 02/12/2018 10:59 AM, Jan Kiszka wrote: >>>>> From: Jan Kiszka >>>>> >>>>> Install a basichash as signature handler and set the stamp policy to >>>>> full - and suddenly we get proper rebuilds of the image and all >>>>> affected >>>>> packages when some recipe down the dependency chain changed! >>>>> >>>>> We are using the legacy bitbake mechanism here as we do not have >>>>> setscene machinery like OE. Still good enough for us. >>>>> >>>> >>>> BTW: have you tried this? Two years ago I tried it in Yocto and it >>>> didn't work properly, not all the dependents were rebuilt. If it works, >>>> it's a very good feature! >>> >>> Yes, I'm using it since this weekend for both Isar as well as >>> jailhouse-images development, and it helped a lot already. The only >>> limitation I found, but I do not remember right now if that isn't >>> inherent to bitbake, is that it does not detect changes in files that >>> the recipes carries in its SRC_URI (file://...). Then you need "-c >>> clean" for the affected target - which now works as well. >>> >> >> Found another issue, which worked surprisingly well so far despite being >> fairly broken: do_setup_mounts is not re-run on rebuilds. > > Investigating exactly this issue. It occures for clean builds also. My > guess is patch #3, currently I'm building with only 2 first applied. > With 3rd one applied build failed. > That's not the right approach, the concept of the setup_mounts task is broken. I'm still testing, but it looks like as if this massive simplification can also fix the rebuild issue: diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass index a45fbce..aa46f45 100644 --- a/meta/classes/dpkg-base.bbclass +++ b/meta/classes/dpkg-base.bbclass @@ -2,7 +2,7 @@ # Copyright (C) 2017 Siemens AG # Add dependency from buildchroot creation -do_build[depends] = "buildchroot:do_setup_mounts" +do_build[depends] = "buildchroot:do_build" # Add dependency between Isar recipes DEPENDS ?= "" @@ -25,6 +25,14 @@ do_build() { mkdir -p ${BUILDROOT} sudo mount --bind ${WORKDIR} ${BUILDROOT} + # These tests may race with parallel builds, so ignore errors. + grep -q ${BUILDCHROOT_DIR}/isar-apt /proc/mounts && + sudo mount --bind ${DEPLOY_DIR_APT}/${DISTRO} ${BUILDCHROOT_DIR}/isar-apt || true + grep -q ${BUILDCHROOT_DIR}/dev /proc/mounts && + sudo mount -t devtmpfs -o mode=0755,nosuid devtmpfs ${BUILDCHROOT_DIR}/dev || true + grep -q ${BUILDCHROOT_DIR}/proc /proc/mounts && + sudo mount -t proc none ${BUILDCHROOT_DIR}/proc || true + dpkg_runbuild sudo umount ${BUILDROOT} 2>/dev/null || true diff --git a/meta/recipes-devtools/buildchroot/buildchroot.bb b/meta/recipes-devtools/buildchroot/buildchroot.bb index 570c0ad..cd91228 100644 --- a/meta/recipes-devtools/buildchroot/buildchroot.bb +++ b/meta/recipes-devtools/buildchroot/buildchroot.bb @@ -60,7 +60,9 @@ do_build() { -e 's|##DIR_HOOKS##|./'"$WORKDIR_REL"'/hooks_multistrap|g' \ "${WORKDIR}/multistrap.conf.in" > "${WORKDIR}/multistrap.conf" - do_setup_mounts + sudo mount --bind ${DEPLOY_DIR_APT}/${DISTRO} ${BUILDCHROOT_DIR}/isar-apt + sudo mount -t devtmpfs -o mode=0755,nosuid devtmpfs ${BUILDCHROOT_DIR}/dev + sudo mount -t proc none ${BUILDCHROOT_DIR}/proc # Create root filesystem sudo -E multistrap -a ${DISTRO_ARCH} -d "${BUILDCHROOT_DIR}" -f "${WORKDIR}/multistrap.conf" @@ -70,30 +72,4 @@ do_build() { # Configure root filesystem sudo chroot ${BUILDCHROOT_DIR} /configscript.sh - - do_cleanup_mounts -} - -# Invalidate stamp for do_setup_mounts before each build start. -# This will guarantee that this function will be executed once -# per build. -python __anonymous() { - stamp = d.getVar("STAMP") + ".do_setup_mounts." + d.getVarFlag("do_setup_mounts", 'stamp-extra-info') - os.remove(stamp) if os.path.exists(stamp) else None -} - -do_setup_mounts[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" - -do_setup_mounts() { - sudo mount --bind ${DEPLOY_DIR_APT}/${DISTRO} ${BUILDCHROOT_DIR}/isar-apt - sudo mount -t devtmpfs -o mode=0755,nosuid devtmpfs ${BUILDCHROOT_DIR}/dev - sudo mount -t proc none ${BUILDCHROOT_DIR}/proc -} - -addtask setup_mounts after do_build - -do_cleanup_mounts() { - sudo umount ${BUILDCHROOT_DIR}/isar-apt 2>/dev/null || true - sudo umount ${BUILDCHROOT_DIR}/dev 2>/dev/null || true - sudo umount ${BUILDCHROOT_DIR}/proc 2>/dev/null || true } Jan