From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6729358150130991104 X-Received: by 2002:a5d:4d81:: with SMTP id b1mr19063569wru.27.1566808710349; Mon, 26 Aug 2019 01:38:30 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a1c:5411:: with SMTP id i17ls5986947wmb.2.canary-gmail; Mon, 26 Aug 2019 01:38:29 -0700 (PDT) X-Google-Smtp-Source: APXvYqwYiE0BYs5PC5/mIwch+bg6px32xWVolUoIep2gl4FcUJ29TUMmsjls9oQIRXX4q2aArwWy X-Received: by 2002:a7b:c216:: with SMTP id x22mr20490377wmi.19.1566808709773; Mon, 26 Aug 2019 01:38:29 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1566808709; cv=none; d=google.com; s=arc-20160816; b=p/lEazM0VgTqiqWj+n2D26ZLGtr+2ldcXIQwKCsPPNkxf0MFZHSjSREdgx81BCbiqN ZPBYOXSdSav6GtrLuFw26uuR/gXpYlC3L0Bd77fGdSsnDnxRZE8l+w7J4niEeFF2UOhP ufIsgj4j5ysht8mFWh/fYYiEhvqvP/Akx2vCs406OFmaduVihT71CsoeyPPlRGT9M8Bx JYjevICMffT1MaDnN7snLyAws3ZIUGcdV+z+Bm3u7A9TmtjZo82XXaLvDVeP4ySW9j2m tfLaAWeqbnVsUm1lsV3bbV01HxroExfkgxKgfb4kWkTM01+t6c2iFTOQxrzH+ZGgV9to S11g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:to:subject; bh=PMTsQe5vjNi2xCamNtpa4CZd8pFlfngGKJvFnVF5/4g=; b=F5Xt/urDMwdAy4e4f3FHTzBKcEvcV6x2p5GWKH10I+h2MN5oqa0qczhMyxiaTKIVsG 8mMlqnkj9KG3bwEBg8hBsVN6ivQK9ZQI2YlPqGxvn9vBCwJj8kp2lELFHo4FZEo8eQpX iCwB+BEFSEv/ibThxhKRR+6UVUpp5DsT4LdR8OTZ8Eq2ILj1yZBaYK882ey7BsG0ELx5 70ID724jJPYcgQhrDQcKCeIaS5BnpEIwIbH4goD0jyJzV0Im1/mIzgNOdfgr8mh1q6A9 YjmeD15uATp3j/5WacXp9O2mm1aactE6PsLc42wVM8KCEUStQ40j4tc6k9viN0XnVIaR Otqg== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.14 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from david.siemens.de (david.siemens.de. [192.35.17.14]) by gmr-mx.google.com with ESMTPS id o4si613074wrp.4.2019.08.26.01.38.29 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 26 Aug 2019 01:38:29 -0700 (PDT) Received-SPF: pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.14 as permitted sender) client-ip=192.35.17.14; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.14 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: from mail1.sbs.de (mail1.sbs.de [192.129.41.35]) by david.siemens.de (8.15.2/8.15.2) with ESMTPS id x7Q8cT5H000369 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Mon, 26 Aug 2019 10:38:29 +0200 Received: from [139.22.35.99] ([139.22.35.99]) by mail1.sbs.de (8.15.2/8.15.2) with ESMTP id x7Q8cSpg022380; Mon, 26 Aug 2019 10:38:28 +0200 Subject: Re: [PATCH 04/26] Rework STAMP layout and cleaning To: Claudius Heine , isar-users References: <0b8426ae-b9e5-9c2c-62ad-302d0f85874e@siemens.com> From: Jan Kiszka Message-ID: Date: Mon, 26 Aug 2019 10:38:28 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.8.0 MIME-Version: 1.0 In-Reply-To: <0b8426ae-b9e5-9c2c-62ad-302d0f85874e@siemens.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-TUID: BEIcUCwAGcNH On 26.08.19 10:30, Claudius Heine wrote: > Hi Jan, > > On 26/08/2019 08.26, [ext] Jan Kiszka wrote: >> From: Jan Kiszka >> >> Follow OE more closely by aligning the stamps folder structure to >> workdir. This may resolve false sharing of stamps between recipes of >> common PF but different DISTRO/DISTRO_ARCH without the otherwise >> required stamp-extra-info. That extra-info can now be removed. >> >> As long as recipes mess with PF, we can't use OE's pattern as-is which >> is PN-based. >> >> Furthermore add proper STAMPCLEAN. This ensures that stamp files are >> cleaned when a task is rebuilt. Not providing this pattern likely caused >> a lot of failures when doing partial rebuilds. >> >> As we are at it, switch do_clean to STAMPCLEAN as well. >> >> Signed-off-by: Jan Kiszka >> --- >>   meta/classes/base.bbclass                  | 5 ++--- >>   meta/classes/dpkg-base.bbclass             | 6 ------ >>   meta/classes/dpkg-raw.bbclass              | 1 - >>   meta/classes/dpkg.bbclass                  | 1 - >>   meta/classes/patch.bbclass                 | 2 -- >>   meta/classes/template.bbclass              | 1 - >>   meta/conf/bitbake.conf                     | 4 +++- >>   meta/recipes-kernel/linux/linux-custom.inc | 1 - >>   8 files changed, 5 insertions(+), 16 deletions(-) >> >> diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass >> index ca80bf8..1d2852a 100644 >> --- a/meta/classes/base.bbclass >> +++ b/meta/classes/base.bbclass >> @@ -163,7 +163,6 @@ python do_fetch() { >>   addtask fetch before do_build >>   do_unpack[dirs] = "${WORKDIR}" >> -do_unpack[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" >>   # Unpack package and put it into working directory >>   python do_unpack() { >> @@ -206,8 +205,8 @@ python do_clean() { >>       workdir = d.expand("${WORKDIR}") >>       subprocess.check_call(["sudo", "rm", "-rf", workdir]) >> -    stamppath = bb.data.expand(d.getVar('STAMP', False), d) >> -    stampdirs = glob.glob(stamppath + ".*") >> +    stampclean = bb.data.expand(d.getVar('STAMPCLEAN', False), d) >> +    stampdirs = glob.glob(stampclean) >>       subprocess.check_call(["sudo", "rm", "-rf"] + stampdirs) >>   } >> diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass >> index 39c8acb..2d26418 100644 >> --- a/meta/classes/dpkg-base.bbclass >> +++ b/meta/classes/dpkg-base.bbclass >> @@ -16,7 +16,6 @@ do_adjust_git() { >>   } >>   addtask adjust_git after do_unpack before do_patch >> -do_adjust_git[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" >>   inherit patch >>   addtask patch after do_adjust_git before do_build >> @@ -42,7 +41,6 @@ do_apt_fetch() { >>   addtask apt_fetch after do_unpack before do_patch >>   do_apt_fetch[lockfiles] += "${REPO_ISAR_DIR}/isar.lock" >> -do_apt_fetch[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" >>   def get_package_srcdir(d): >>       s = os.path.abspath(d.getVar("S", True)) >> @@ -60,15 +58,12 @@ def get_package_srcdir(d): >>   PP = "/home/builder/${PN}" >>   PPS ?= "${@get_package_srcdir(d)}" >> -do_build[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" >> - >>   # Empty do_prepare_build() implementation, to be overwritten if needed >>   do_prepare_build() { >>       true >>   } >>   addtask prepare_build after do_patch do_transform_template before do_build >> -do_prepare_build[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" >>   # If Isar recipes depend on each other, they typically need the package >>   # deployed to isar-apt >>   do_prepare_build[deptask] = "do_deploy_deb" >> @@ -132,7 +127,6 @@ do_deploy_deb() { >>   } >>   addtask deploy_deb after do_build >> -do_deploy_deb[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" >>   do_deploy_deb[lockfiles] = "${REPO_ISAR_DIR}/isar.lock" >>   do_deploy_deb[depends] = "isar-apt:do_cache_config" >>   do_deploy_deb[dirs] = "${S}" >> diff --git a/meta/classes/dpkg-raw.bbclass b/meta/classes/dpkg-raw.bbclass >> index ea03ea4..f8dfb2d 100644 >> --- a/meta/classes/dpkg-raw.bbclass >> +++ b/meta/classes/dpkg-raw.bbclass >> @@ -16,7 +16,6 @@ do_install() { >>   } >>   do_install[cleandirs] = "${D}" >> -do_install[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" >>   addtask install after do_unpack before do_prepare_build >>   do_prepare_build[cleandirs] += "${D}/debian" >> diff --git a/meta/classes/dpkg.bbclass b/meta/classes/dpkg.bbclass >> index 997f2f3..dd123c3 100644 >> --- a/meta/classes/dpkg.bbclass >> +++ b/meta/classes/dpkg.bbclass >> @@ -14,7 +14,6 @@ do_install_builddeps() { >>   addtask install_builddeps after do_prepare_build before do_build >>   # apt and reprepro may not run in parallel, acquire the Isar lock >>   do_install_builddeps[lockfiles] += "${REPO_ISAR_DIR}/isar.lock" >> -do_install_builddeps[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" >>   # Build package from sources using build script >>   dpkg_runbuild() { >> diff --git a/meta/classes/patch.bbclass b/meta/classes/patch.bbclass >> index 2662702..c5ba463 100644 >> --- a/meta/classes/patch.bbclass >> +++ b/meta/classes/patch.bbclass >> @@ -78,5 +78,3 @@ python do_patch() { >>           except bb.fetch2.BBFetchException as e: >>               raise bb.build.FuncFailed(e) >>   } >> - >> -do_patch[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" >> diff --git a/meta/classes/template.bbclass b/meta/classes/template.bbclass >> index f3b5f80..fb9d118 100644 >> --- a/meta/classes/template.bbclass >> +++ b/meta/classes/template.bbclass >> @@ -7,7 +7,6 @@ TEMPLATE_FILES ?= "" >>   TEMPLATE_VARS ?= "PN PV DESCRIPTION HOMEPAGE MAINTAINER DISTRO_ARCH" >>   do_transform_template[vardeps] = "TEMPLATE_FILES ${TEMPLATE_VARS}" >> -do_transform_template[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" >>   python do_transform_template() { >>       import subprocess, contextlib >> diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf >> index 3d18223..c41f294 100644 >> --- a/meta/conf/bitbake.conf >> +++ b/meta/conf/bitbake.conf >> @@ -35,7 +35,9 @@ PV = "${@bb.parse.BBHandler.vars_from_file(d.getVar('FILE', >> False),d)[1] or '1.0 >>   S = "${WORKDIR}/${P}" >>   AUTOREV = "${@bb.fetch2.get_autorev(d)}" >>   SRC_URI = "file://${FILE}" >> -STAMP = "${TMPDIR}/stamps/${PF}" >> +STAMPS_DIR ?= "${TMPDIR}/stamps" >> +STAMP = "${STAMPS_DIR}/${DISTRO}-${DISTRO_ARCH}/${PF}/${PV}-${PR}" > > Can you explain the issue of the PN/PF usage a bit more. I did not understand: > >> As long as recipes mess with PF, we can't use OE's pattern as-is which >> is PN-based. > > The recipes that mess with PF are: buildchroot-*.bb sdkchroot.bb > isar-bootstrap-*.bb and image.bbclass. All of which do not really have a version. > > So I would think that: > > STAMP = "${STAMPS_DIR}/${DISTRO}-${DISTRO_ARCH}/${PN}/${PF}" > STAMPCLEAN = "${STAMPS_DIR}/${DISTRO}-${DISTRO_ARCH}/${PN}/*-*" > > should work. Or am I missing something? PF is a effectively read-only. This is fixed later on this series. Then there is no need to deviate from OE any more. Jan -- Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux