From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6724331065971834880 X-Received: by 2002:a5d:4b05:: with SMTP id v5mr19851596wrq.208.1565684069287; Tue, 13 Aug 2019 01:14:29 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a5d:514b:: with SMTP id u11ls3945420wrt.8.gmail; Tue, 13 Aug 2019 01:14:28 -0700 (PDT) X-Google-Smtp-Source: APXvYqxuwtuUbw5Ca3E/ToYjJtrdONxRxFf0meI75TUYQD4xl/U9r3F4m2iClwMuQlbaoHDriyGq X-Received: by 2002:a05:6000:10c9:: with SMTP id b9mr32103237wrx.11.1565684068735; Tue, 13 Aug 2019 01:14:28 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1565684068; cv=none; d=google.com; s=arc-20160816; b=r9vetEw4Q+PeAKM1tgSVp/zpwkXBUXdAmeTZyfdi2qHOHEq9AyrD3yACM0snCgzUO6 G+pThxmx7zTCpovHnegapjSuLy8XJWm9VmuGy97PVUbzKCXmnCffcd1RC5zksbMrciRW sBI7tb6w2O8BWi764+y1Ur20Bacsg6u97GZSST8YLaBeicK4DWl7MEmWQyJ8v1NR5Z7I sFnExLyqBu0iOm+N6h4yO9QiFrNpEm+keFuSSlFLmUXpcAS6qsSSvU301ePgaXDjIGMy dgMI5j30b3GvMSobZggxJNf3ZVErqCXilMhtoip6BiIRvWVWkpPdbA2XPos7OhZhqMze SDTw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:to:subject; bh=eX4AtJB2nIi8dM8Di32lt+NYYrxw3Fu/hX3M85f/nLs=; b=uD69h+60N7iWHd0g90rDVIp+ZEaxSyOOuFXFbswI+IVdedg4VRqt8GB9TMl00GGQS1 uFSYKCMsugw8P5agkIGbxIF2++zZ5KdKMJ/CN08YU2oPUMipnvEOSwAoPFk/gyNT/eii 2WcPeGpNj+fO5/oN4JNRUeWK9CmUiXSKF0DpASsxHcu3QfpfKNGrImljiKxmkjuck+oS 8C1sIKcoeeeP7sccL3VFcb0JyZixzmApnBbf/+rLL2HMKsusZvBZ1VZURqkFy/P8nQoB N2HFNaAqUZVomzAIOmdQRza0T7mrSkifIleNJQv4huD1ddbR497aBuHW8Joj/yuXByVX NMNQ== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.2 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from thoth.sbs.de (thoth.sbs.de. [192.35.17.2]) by gmr-mx.google.com with ESMTPS id o4si2120904wrp.4.2019.08.13.01.14.28 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 13 Aug 2019 01:14:28 -0700 (PDT) Received-SPF: pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.2 as permitted sender) client-ip=192.35.17.2; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.2 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: from mail1.sbs.de (mail1.sbs.de [192.129.41.35]) by thoth.sbs.de (8.15.2/8.15.2) with ESMTPS id x7D8ESHc024152 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Aug 2019 10:14:28 +0200 Received: from [139.25.68.37] (md1q0hnc.ad001.siemens.net [139.25.68.37] (may be forged)) by mail1.sbs.de (8.15.2/8.15.2) with ESMTP id x7D8ERuq008500; Tue, 13 Aug 2019 10:14:27 +0200 Subject: Re: isar-bootstrap fails to rebuild after config changes To: Claudius Heine , isar-users , Claudius Heine , Maksim Osipov References: From: Jan Kiszka Message-ID: <52a51b4f-483a-3b42-8edf-3e0db8ffc722@siemens.com> Date: Tue, 13 Aug 2019 10:14:26 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.8.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-TUID: ilJ7U96BqQfv On 13.08.19 10:01, Claudius Heine wrote: > Hi Jan, > > On 12/08/2019 19.18, [ext] Jan Kiszka wrote: >> Hi, >> >> I noticed today that I'm not getting a new bootstrap rootfs after adding a >> repo to DISTRO_APT_SOURCES. Digging deeper, this revealed missing STAMPCLEAN >> setting, but also more: >> >> Why are we probing for the existence of the DEPLOY_ISAR_BOOTSTRAP link in >> do_apt_config_prepare(), Claudius? This obviously exists after a first run, >> and that blocks refreshing all the apt settings if they have changed. > > I do not know. That was added by a patch of Maxim > (ab0a1c8c357a18ddd67fe4c0efc66a296d93dae2), maybe he knows. > >     isar-bootstrap: Eliminate multiple debootstraps for the same distro/host > >     After applying this patch only single debootstrap for >     particular platform is created and pointed by corresponding >     link under DEPLOY_DIR_IMAGE. > >     For proper locking of parallel builds shell tasks >     apt_config_install, set_locale, setup_chroot, apt_update >     and deploy were collapsed to common debootstrap helper. >     Due to problems in bitbake with shell functions expansion >     under quotes these functions were substituted  by their >     bodies. > >     Signed-off-by: Maxim Yu. Osipov > > That is by the way a pretty big patch. Also parts of it could probably be > reverted, since the 'problems in bitbake with shell functions expansion under > quotes' are resolved in the current version by removing these quotes IIUC. > >> There is CLEANFUNCS set with clean_deploy(), but that is only run on true >> do_clean(), not on rebuilds. > > Right. > >> >> And finally, when I look at isar_bootstrap(), run by do_bootstrap(), it >> protects checking and creating DEPLOY_ISAR_BOOTSTRAP with a lock. But >> do_apt_config_prepare() runs lockless. This cannot work reliably. > > When I initially wrote isar-bootstrap, it didn't contain any looks. IIRC the > locks where added when I wasn't active with isar and I haven't yet looked into > why exactly they are needed and what they protect against etc. So you know > probably more about them than me. Ah, sorry, wrongly accounted you for this. Maxim, can you comment on the design idea and the issues I found? Jan -- Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux