From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6729358150130991104 X-Received: by 2002:a2e:87c8:: with SMTP id v8mr869304ljj.24.1569054541559; Sat, 21 Sep 2019 01:29:01 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a19:5d57:: with SMTP id p23ls921673lfj.4.gmail; Sat, 21 Sep 2019 01:29:00 -0700 (PDT) X-Google-Smtp-Source: APXvYqwj1/QrWGzhmeVYnWHD17qqWRuASuTlC+Rv3QvfX9VhuQRF64iSZCJe0XrtNM5GkOW+Zzdg X-Received: by 2002:a19:6556:: with SMTP id c22mr9311390lfj.90.1569054540857; Sat, 21 Sep 2019 01:29:00 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1569054540; cv=none; d=google.com; s=arc-20160816; b=MnciaTwBvGdeH2HILaY27vG2yGh/TSX6WCiliCJGO3qHGdtdGnc7LwP/lIPM6cvTOK tbnqdLJ1D1s92Vs3+6UL4ImrCKu1Jl73etl90a5ciMrchks7/GLqqGbz7vtieD0rct5l AkPO2PZqxsFoxAnUbioeb9WrGts+5zcz+3pMX50EoccCUWYcVawn7T8m3fnxrDfDDKZD zfBX3Zi+nT4ivtBVE3CVxuVTX61yQOmnA3vQMPBgisF2gQ6nTBwHSMZv7GFrRxBxnMNr n9xb44cbBhnqC35n9TyXqdKsNvnr2skOqwEVGg/VOVVg47B1ZO3lohYyqldhDngNPrXb 9zoQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:references:to:from:subject; bh=XqzXlQwpZa7xPeHDgocTWKficSOqYrbvBvfPI+7rUkU=; b=IHojNbh9ihUXkymm9Z69Jsqyi08cdiRekPpIfF5SOlEJXQA7zDQWK4wWXqARGY3qna NpPrxwBFlsYcCOHcU+2hwFel6vzo2PmzmEgz5zA0iqtwEJhlQcMmUFonXKQsS9dbl+Bt /kJznjd+J5SbGkegZ+Gs3fEK3mMpECaK4D2r6DgcX//u8YjiXabTY9mdprvEfIppA4gj E9lsyiDxKg2nuYPzviITu+qNNQWzotO5Mz0ZKVDyzabez8ebiJtq7QL1wONb++p2D4u+ r/dxLmNpU1cAnF6+EgxgtOIkOBlJdJ43JIJ2STyLtH81n1RoB7+gjouX/Vc8FmIUA/lZ sk0w== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 194.138.37.40 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from gecko.sbs.de (gecko.sbs.de. [194.138.37.40]) by gmr-mx.google.com with ESMTPS id a9si252940lfk.5.2019.09.21.01.29.00 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 21 Sep 2019 01:29:00 -0700 (PDT) Received-SPF: pass (google.com: domain of jan.kiszka@siemens.com designates 194.138.37.40 as permitted sender) client-ip=194.138.37.40; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 194.138.37.40 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: from mail1.sbs.de (mail1.sbs.de [192.129.41.35]) by gecko.sbs.de (8.15.2/8.15.2) with ESMTPS id x8L8SxnV010166 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Sat, 21 Sep 2019 10:28:59 +0200 Received: from [139.22.38.101] ([139.22.38.101]) by mail1.sbs.de (8.15.2/8.15.2) with ESMTP id x8L8SxID001149 for ; Sat, 21 Sep 2019 10:28:59 +0200 Subject: Re: [PATCH 22/26] isar-bootstrap: Clean up and fix rebuild From: Jan Kiszka To: isar-users References: <2be0f134cdc2a6167b4c6722a7cc9e9977f35c70.1566800787.git.jan.kiszka@siemens.com> Message-ID: <5f42938f-ee6e-4b2e-cd35-d03cfaac9eeb@siemens.com> Date: Sat, 21 Sep 2019 10:28:59 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.8.0 MIME-Version: 1.0 In-Reply-To: <2be0f134cdc2a6167b4c6722a7cc9e9977f35c70.1566800787.git.jan.kiszka@siemens.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-TUID: LE97+r0n6cBP On 26.08.19 08:26, [ext] Jan Kiszka wrote: > From: Jan Kiszka > > There is no reason for using a lock here. All bootstraps for the same > distro/arch are detected by bitbake as identical targets in multiconfig > scenarios and not run multiple times. If they are run multiple times, > there is another bug, such as with the recently fixed bananapi machine. > > Also remove the equally bogus re-execution test in > do_apt_config_prepare. This also fixes rebuilds of the isar-bootstrap > target in case some parameter has changed. > > Signed-off-by: Jan Kiszka > --- > .../isar-bootstrap/isar-bootstrap-host.bb | 30 ++++++++++------------ > .../isar-bootstrap/isar-bootstrap-target.bb | 27 +++++++++---------- > .../recipes-core/isar-bootstrap/isar-bootstrap.inc | 5 ---- > 3 files changed, 26 insertions(+), 36 deletions(-) > > diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb b/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb > index b2dded5..8c211a9 100644 > --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb > +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb > @@ -8,7 +8,6 @@ > Description = "Minimal host Debian root file system" > > DEPLOY_ISAR_BOOTSTRAP = "${DEPLOY_DIR_BOOTSTRAP}/${HOST_DISTRO}-${HOST_ARCH}-${DISTRO_ARCH}" > -ISAR_BOOTSTRAP_LOCK = "${DEPLOY_DIR_BOOTSTRAP}/${HOST_DISTRO}-${HOST_ARCH}-${DISTRO_ARCH}.lock" > > require isar-bootstrap.inc > > @@ -25,21 +24,20 @@ do_apt_config_prepare[vardeps] += "\ > DEPLOY_ISAR_BOOTSTRAP \ > " > python do_apt_config_prepare() { > - if not os.path.islink(d.getVar("DEPLOY_ISAR_BOOTSTRAP", True)): > - apt_preferences_out = d.getVar("APTPREFS", True) > - apt_preferences_list = ( > - d.getVar("HOST_DISTRO_APT_PREFERENCES", True) or "" > - ).split() > - aggregate_files(d, apt_preferences_list, apt_preferences_out) > - > - apt_sources_out = d.getVar("APTSRCS", True) > - apt_sources_init_out = d.getVar("APTSRCS_INIT", True) > - apt_sources_list = ( > - d.getVar("HOST_DISTRO_APT_SOURCES", True) or "" > - ).split() > - > - aggregate_files(d, apt_sources_list, apt_sources_init_out) > - aggregate_aptsources_list(d, apt_sources_list, apt_sources_out) > + apt_preferences_out = d.getVar("APTPREFS", True) > + apt_preferences_list = ( > + d.getVar("HOST_DISTRO_APT_PREFERENCES", True) or "" > + ).split() > + aggregate_files(d, apt_preferences_list, apt_preferences_out) > + > + apt_sources_out = d.getVar("APTSRCS", True) > + apt_sources_init_out = d.getVar("APTSRCS_INIT", True) > + apt_sources_list = ( > + d.getVar("HOST_DISTRO_APT_SOURCES", True) or "" > + ).split() > + > + aggregate_files(d, apt_sources_list, apt_sources_init_out) > + aggregate_aptsources_list(d, apt_sources_list, apt_sources_out) > } > addtask apt_config_prepare before do_bootstrap after do_unpack > > diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb b/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb > index d7fcdbf..80e7f40 100644 > --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb > +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb > @@ -8,7 +8,6 @@ > Description = "Minimal target Debian root file system" > > DEPLOY_ISAR_BOOTSTRAP = "${DEPLOY_DIR_BOOTSTRAP}/${DISTRO}-${DISTRO_ARCH}" > -ISAR_BOOTSTRAP_LOCK = "${DEPLOY_DIR_BOOTSTRAP}/${DISTRO}-${DISTRO_ARCH}.lock" > > require isar-bootstrap.inc > > @@ -22,19 +21,18 @@ do_apt_config_prepare[vardeps] += "\ > DEPLOY_ISAR_BOOTSTRAP \ > " > python do_apt_config_prepare() { > - if not os.path.islink(d.getVar("DEPLOY_ISAR_BOOTSTRAP", True)): > - apt_preferences_out = d.getVar("APTPREFS", True) > - apt_preferences_list = ( > - d.getVar("DISTRO_APT_PREFERENCES", True) or "" > - ).split() > - aggregate_files(d, apt_preferences_list, apt_preferences_out) > - > - apt_sources_out = d.getVar("APTSRCS", True) > - apt_sources_init_out = d.getVar("APTSRCS_INIT", True) > - apt_sources_list = (d.getVar("DISTRO_APT_SOURCES", True) or "").split() > - > - aggregate_files(d, apt_sources_list, apt_sources_init_out) > - aggregate_aptsources_list(d, apt_sources_list, apt_sources_out) > + apt_preferences_out = d.getVar("APTPREFS", True) > + apt_preferences_list = ( > + d.getVar("DISTRO_APT_PREFERENCES", True) or "" > + ).split() > + aggregate_files(d, apt_preferences_list, apt_preferences_out) > + > + apt_sources_out = d.getVar("APTSRCS", True) > + apt_sources_init_out = d.getVar("APTSRCS_INIT", True) > + apt_sources_list = (d.getVar("DISTRO_APT_SOURCES", True) or "").split() > + > + aggregate_files(d, apt_sources_list, apt_sources_init_out) > + aggregate_aptsources_list(d, apt_sources_list, apt_sources_out) > } > addtask apt_config_prepare before do_bootstrap after do_unpack > > @@ -45,4 +43,3 @@ do_bootstrap() { > isar_bootstrap > } > addtask bootstrap before do_build after do_generate_keyrings > - > diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc > index aea89a9..36cefd2 100644 > --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc > +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc > @@ -212,14 +212,11 @@ do_generate_keyrings() { > } > addtask generate_keyrings before do_build after do_unpack > > - > - > def get_host_release(): > import platform > rel = platform.release() > return rel > > - > do_bootstrap[vardeps] += "DISTRO_APT_PREMIRRORS" > do_bootstrap[dirs] = "${DEPLOY_DIR_BOOTSTRAP}" > > @@ -246,7 +243,6 @@ isar_bootstrap() { > E="${@bb.utils.export_proxies(d)}" > export IS_HOST debootstrap_args E > sudo -E -s <<'EOSUDO' > - ( flock 9 > set -e > if [ ! -e "${DEPLOY_ISAR_BOOTSTRAP}" ]; then > rm -rf "${ROOTFSDIR}" > @@ -340,7 +336,6 @@ isar_bootstrap() { > # Finalize debootstrap by setting the link in deploy > ln -Tfsr "${ROOTFSDIR}" "${DEPLOY_ISAR_BOOTSTRAP}" > fi > - ) 9>'${ISAR_BOOTSTRAP_LOCK}' > EOSUDO > } > > I just understood (thanks to a rare CI failure) what this lock was papering over: Raspbian and Debian builds for the same target distro/arch share their host boostrap deployment. That needs to be fixed. Of course, that lock didn't solve anything because it forced the second user to live with the bootstrap settings of the first one. Again the time to remind everyone to try harder understanding issues as they occur, rather than throwing random stuff at them or even ignore them as "not my use case". Those things always tend to bite us again at some point. I think we are getting better here, though. Jan -- Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux