From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6519007876157014016 X-Received: by 10.28.63.6 with SMTP id m6mr2267814wma.3.1517826795495; Mon, 05 Feb 2018 02:33:15 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 10.28.170.1 with SMTP id t1ls1478042wme.7.gmail; Mon, 05 Feb 2018 02:33:15 -0800 (PST) X-Google-Smtp-Source: AH8x227yO+PaIbPLMxykO97GFo3+twN0eGPTP9WnBjuz0Vaev948W2k58XVvoePOxOt/A1hDiNDk X-Received: by 10.28.173.203 with SMTP id w194mr5131878wme.24.1517826794979; Mon, 05 Feb 2018 02:33:14 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1517826794; cv=none; d=google.com; s=arc-20160816; b=k2gZgL60rVCdxi3cqDvZ/qaNix3yJRtKDd0Rn5qogR3I4vtjkXnCzrE2xFeAcGq1dI NwmyVnr4oRFcsaW4M4fRZM3hBtw8ktvH9KPsCj9Fz7n9Hfd0qdmcGE1J9yosUQDgjviI RCmysf7/V9J6t6/ul1Kz9SpterqyWqk+K9+Vt2s7qn7LQRFd+QkZhPUpKDCLOfhQ/Xnf hm75tMcJPZ29DN0vR/iA8tbaPseLWWA1IrMFatg+Lfd/KS+NFee97yqCzdELThbKG9Dh 18Dq5sPMrjpMDQ4IFzVBAdNp1rVxd9L7v1oQHNXUsnqrxjLvXEYH2F5umh7ltGCHkpyj auww== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:to:subject :arc-authentication-results; bh=xKrrcmDkQ2m4Vy61LzkOZTRDHAwNeobAkU+6LPwFA/0=; b=i7Hbib95B3LXlyPfKftUE8zn3xFNksnOYUqexEeZ+43AIe/OJnYDl3S/YYHXgD3CDw 71tRBgyC0xmoZPdz+8W9419SomWYvKXuL9dfwGeRsGg9aCy7LGx5bxaMbELWq3FYFBC0 l/36OXigYOlSIiiSfk25vjZZF+iF2dK+6BibOJXEk/rmwR6u2wT3AyQ/wwFrQNxMO/F5 IedTapfwe3weHADWDfeQkfnAcQ49COXoFbOyg3HcEupb8EVrBf9hlSBHZg78cg8LrEpq phbTUesmSPcEgEaOIuJamvmg/SC112JO1eueyblk64PBQ+X4FgLwToIZAj+F9Wo6MHsq 67BQ== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) smtp.mailfrom=asmirnov@ilbers.de Return-Path: Received: from aqmola.ilbers.de (aqmola.ilbers.de. [85.214.62.211]) by gmr-mx.google.com with ESMTPS id y42si616377wry.5.2018.02.05.02.33.14 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 05 Feb 2018 02:33:14 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) client-ip=85.214.62.211; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) smtp.mailfrom=asmirnov@ilbers.de Received: from [10.0.2.15] ([188.227.110.165]) (authenticated bits=0) by aqmola.ilbers.de (8.14.4/8.14.4/Debian-4+deb7u1) with ESMTP id w15AXBOk024553 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Mon, 5 Feb 2018 11:33:13 +0100 Subject: Re: [PATCH 3/5] buildchroot: Enable isar-apt To: Jan Kiszka , isar-users@googlegroups.com References: <20180205095931.23903-1-asmirnov@ilbers.de> <20180205095931.23903-4-asmirnov@ilbers.de> <0f008581-a852-79fa-f67e-6361c4ca35ed@siemens.com> From: Alexander Smirnov Message-ID: <45f4d21a-d9bb-ab9a-4864-b91668fa9ad0@ilbers.de> Date: Mon, 5 Feb 2018 13:33:06 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: <0f008581-a852-79fa-f67e-6361c4ca35ed@siemens.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-TUID: BtZpDCc8jtOF On 02/05/2018 01:18 PM, Jan Kiszka wrote: > On 2018-02-05 10:59, Alexander Smirnov wrote: >> This patch provides access to isar-apt from buildchroot. It does the >> following: >> - mount isar-apt during buildchroot building. >> - umount isar-apt using bitbake events. >> >> Also it needs to keep Isar build tree clean from any mounts despite on whether >> build succeed of failed. bitbake provides various events that could trigger >> custom python hooks. In this patch BuildCompleted event is used, which happened >> when bitbake finished its execution despite on the result. >> >> Signed-off-by: Alexander Smirnov >> --- >> meta-isar/conf/local.conf.sample | 3 +++ >> meta/classes/isar-events.bbclass | 20 ++++++++++++++++++++ >> meta/recipes-devtools/buildchroot/buildchroot.bb | 7 +++++++ >> .../buildchroot/files/multistrap.conf.in | 8 +++++++- >> 4 files changed, 37 insertions(+), 1 deletion(-) >> create mode 100644 meta/classes/isar-events.bbclass >> >> diff --git a/meta-isar/conf/local.conf.sample b/meta-isar/conf/local.conf.sample >> index 2ae43e7..fc760ed 100644 >> --- a/meta-isar/conf/local.conf.sample >> +++ b/meta-isar/conf/local.conf.sample >> @@ -158,3 +158,6 @@ IMAGE_INSTALL = "example-hello example-raw" >> # >> # Default parallel jobs for bitbake: >> BB_NUMBER_THREADS = "4" >> + >> +# Add event handlers for bitbake >> +INHERIT += "isar-events" >> diff --git a/meta/classes/isar-events.bbclass b/meta/classes/isar-events.bbclass >> new file mode 100644 >> index 0000000..55fc106 >> --- /dev/null >> +++ b/meta/classes/isar-events.bbclass >> @@ -0,0 +1,20 @@ >> +# Isar event handlers. >> +# >> +# This software is a part of ISAR. >> +# Copyright (C) 2015-2017 ilbers GmbH >> + >> +addhandler isar_handler >> + >> +python isar_handler () { >> + import subprocess >> + >> + devnull = open(os.devnull, 'w') >> + >> + if isinstance(e, bb.event.BuildCompleted): >> + bchroot = d.getVar('BUILDCHROOT_DIR', True) >> + >> + # Clean up buildchroot >> + subprocess.call('/usr/bin/sudo /bin/umount ' + bchroot + '/isar-apt || /bin/true', stdout=devnull, stderr=devnull, shell=True) >> + >> + devnull.close() >> +} >> diff --git a/meta/recipes-devtools/buildchroot/buildchroot.bb b/meta/recipes-devtools/buildchroot/buildchroot.bb >> index 51f9d5d..cb90148 100644 >> --- a/meta/recipes-devtools/buildchroot/buildchroot.bb >> +++ b/meta/recipes-devtools/buildchroot/buildchroot.bb >> @@ -33,6 +33,7 @@ WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}" >> >> do_build[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" >> do_build[dirs] = "${WORKDIR}/hooks_multistrap" >> +do_build[depends] = "isar-apt:do_cache_config" >> >> do_build() { >> E="${@ bb.utils.export_proxies(d)}" >> @@ -58,6 +59,9 @@ do_build() { >> -e 's|##DIR_HOOKS##|./'"$WORKDIR_REL"'/hooks_multistrap|g' \ >> "${WORKDIR}/multistrap.conf.in" > "${WORKDIR}/multistrap.conf" >> >> + [ ! -d ${BUILDCHROOT_DIR}/isar-apt ] && install -d -m 555 ${BUILDCHROOT_DIR}/isar-apt >> + sudo mount --bind ${DEPLOY_DIR_APT}/${DISTRO} ${BUILDCHROOT_DIR}/isar-apt >> + >> [ ! -d ${BUILDCHROOT_DIR}/proc ] && install -d -m 555 ${BUILDCHROOT_DIR}/proc >> sudo mount -t proc none ${BUILDCHROOT_DIR}/proc >> _do_build_cleanup() { >> @@ -73,6 +77,9 @@ do_build() { >> # Install package builder script >> sudo install -m 755 ${WORKDIR}/build.sh ${BUILDCHROOT_DIR} >> >> + # Create share point for isar-apt >> + sudo install -d ${BUILDCHROOT_DIR}/isar-apt >> + >> # Configure root filesystem >> sudo chroot ${BUILDCHROOT_DIR} /configscript.sh >> _do_build_cleanup >> diff --git a/meta/recipes-devtools/buildchroot/files/multistrap.conf.in b/meta/recipes-devtools/buildchroot/files/multistrap.conf.in >> index a0b28e3..480a4b8 100644 >> --- a/meta/recipes-devtools/buildchroot/files/multistrap.conf.in >> +++ b/meta/recipes-devtools/buildchroot/files/multistrap.conf.in >> @@ -6,7 +6,7 @@ noauth=true >> unpack=true >> ignorenativearch=true >> bootstrap=##DISTRO_MULTICONF_BOOTSTRAP## >> -aptsources=##DISTRO_MULTICONF_APTSOURCES## >> +aptsources=isar-apt ##DISTRO_MULTICONF_APTSOURCES## >> configscript=##CONFIG_SCRIPT## >> setupscript=##SETUP_SCRIPT## >> hookdir=##DIR_HOOKS## >> @@ -29,3 +29,9 @@ source=##DISTRO_APT_SOURCE_SEC## >> suite=##DISTRO_SUITE##/updates >> components=##DISTRO_COMPONENTS## >> omitdebsrc=true >> + >> +[isar-apt] >> +source=file:///isar-apt >> +suite=isar >> +components=main >> +omitdebsrc=true >> > > On first glance, this looks identical to v1 which I tried over the > current bitbake version we have in Isar. There I see two issues: > > - umount does not work if the build fails Started replying to your previous mail, but let's discuss it here. Clean up hooks is mostly a hack and IMHO should be replaced by events. I use here bitbake events, see file 'meta/classes/isar-events.bbclass'. There is event BuildCompleted which means that bitbake has finished its pipeline despite on the result. So I've added hook to this event which performs unmount, this also works good for double ^C. > - re-mount will not happen if the buildchroot was built in a previous > run and only a depending packages is built now Good point. Will fix it. Alex