From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6519007876157014016 X-Received: by 10.223.164.149 with SMTP id g21mr231600wrb.21.1517919022804; Tue, 06 Feb 2018 04:10:22 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 10.28.108.16 with SMTP id h16ls2059079wmc.1.gmail; Tue, 06 Feb 2018 04:10:22 -0800 (PST) X-Google-Smtp-Source: AH8x225HgG15WV1waz/Qp0aL30fxevCm6Sykz578PFoiwnHQkG1tWR1sa6GxRg0Cau749XU65dba X-Received: by 10.28.184.6 with SMTP id i6mr196910wmf.14.1517919022188; Tue, 06 Feb 2018 04:10:22 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1517919022; cv=none; d=google.com; s=arc-20160816; b=hHlywzUrPO42/o9IcIFOBYAk3ldn1syxEGktZTERpeJIdRJfuu/PFEiUkDytNqnYJg 8wdZbAFWKT3naYOC/gaWspzHnbmCBm+9amceU5AExmHYatw+2Zy0MgbBkRwIoFAeh3ye ng1HLWxrVNQMScK8cG69P0pI3PhXyGTfWbxWSXuNNYD/k3s7d8ke8MAX4K0zVxzQq/t3 haqRInob+HVUTt315eQS6lzwHZHF8S/1Jl8vaBJ/HFiFwFkNP0uLaAFtWmGhMAaHHb7l QHvGM9huLg5S47BO+L6ovROw/Q2Y2SW5QYTGDnjiglU9kOLRV264h6pLF/68Kx75AFDp S34Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:to:subject :arc-authentication-results; bh=qCFR6PN0QHqMZoRhCzQoi6HHuKE1+uLYa5IM+Ep/Y+w=; b=0Ypx1L4eM6d5BVl0cZtB9iJfVqEoDxmidvyWTaAi/J20ia0e1OFuLri/0/ZtQZ7JN2 4U2nbnN+XNUmMJacM6iczeTN6deekLcqca1ROcc4wOvmTulV2RccqpV6F4Eb1NU4ySpw oy6eZZBEgrKzmC9Lz99gLGdSoyjLAnnBbVSItxoo48u3gYbUnVuCFJwPhKfHRNBr71h3 Yr1O/ad94Zl2Frn1YL9M4+9bFkxkpvASVKUwKVURKNuBD2+ZFTPaYUCqPMt0loaU8v0Z SrxX2jJXRgNePk+nWrd590fgwqTZrQb9JIbDy2E6uAtssqL/5yP4CLXLaxUS0cYwo6rN eeoQ== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.2 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com Return-Path: Received: from thoth.sbs.de (thoth.sbs.de. [192.35.17.2]) by gmr-mx.google.com with ESMTPS id d8si609578wmf.1.2018.02.06.04.10.22 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 06 Feb 2018 04:10:22 -0800 (PST) Received-SPF: pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.2 as permitted sender) client-ip=192.35.17.2; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.2 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com Received: from mail2.siemens.de (mail2.siemens.de [139.25.208.11]) by thoth.sbs.de (8.15.2/8.15.2) with ESMTPS id w16CALTq020312 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 6 Feb 2018 13:10:21 +0100 Received: from [139.25.68.37] (md1q0hnc.ad001.siemens.net [139.25.68.37] (may be forged)) by mail2.siemens.de (8.15.2/8.15.2) with ESMTP id w16CALvw007126; Tue, 6 Feb 2018 13:10:21 +0100 Subject: Re: [PATCH 3/5 v3] buildchroot: Enable isar-apt To: Alexander Smirnov , isar-users@googlegroups.com References: <20180205134844.27221-1-asmirnov@ilbers.de> <20180206110119.10580-1-asmirnov@ilbers.de> From: Jan Kiszka Message-ID: <862a3edd-89c1-1278-d16e-833b12260dfc@siemens.com> Date: Tue, 6 Feb 2018 13:10:21 +0100 User-Agent: Mozilla/5.0 (X11; U; Linux i686 (x86_64); de; rv:1.8.1.12) Gecko/20080226 SUSE/2.0.0.12-1.1 Thunderbird/2.0.0.12 Mnenhy/0.7.5.666 MIME-Version: 1.0 In-Reply-To: <20180206110119.10580-1-asmirnov@ilbers.de> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-TUID: ej5qIA95l2iq On 2018-02-06 12:01, Alexander Smirnov wrote: > 8<-- > > Changes since v2: > - Move do_prepare after do_build. This will not trigger do_build for > reoccurent builds. Hmm, aren't you also moving it *before every* build? That's how I read the code. > > 8<-- > > This patch provides access to isar-apt from buildchroot. It does the > following: > - mount isar-apt during buildchroot building. > - umount isar-apt using bitbake events. > > Also it needs to keep Isar build tree clean from any mounts despite on whether > build succeed of failed. bitbake provides various events that could trigger > custom python hooks. In this patch BuildCompleted event is used, which happened > when bitbake finished its execution despite on the result. > > Signed-off-by: Alexander Smirnov > --- > meta-isar/conf/local.conf.sample | 3 +++ > meta/classes/dpkg-base.bbclass | 2 +- > meta/classes/isar-events.bbclass | 20 ++++++++++++++++++++ > meta/recipes-devtools/buildchroot/buildchroot.bb | 20 +++++++++++++++++++- > .../buildchroot/files/multistrap.conf.in | 8 +++++++- > 5 files changed, 50 insertions(+), 3 deletions(-) > create mode 100644 meta/classes/isar-events.bbclass > > diff --git a/meta-isar/conf/local.conf.sample b/meta-isar/conf/local.conf.sample > index 2ae43e7..fc760ed 100644 > --- a/meta-isar/conf/local.conf.sample > +++ b/meta-isar/conf/local.conf.sample > @@ -158,3 +158,6 @@ IMAGE_INSTALL = "example-hello example-raw" > # > # Default parallel jobs for bitbake: > BB_NUMBER_THREADS = "4" > + > +# Add event handlers for bitbake > +INHERIT += "isar-events" If you are already updating, please move this to a proper place (isar-bitbake.conf) like I suggested via my other patch. > diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass > index 026028f..74dedc5 100644 > --- a/meta/classes/dpkg-base.bbclass > +++ b/meta/classes/dpkg-base.bbclass > @@ -2,7 +2,7 @@ > # Copyright (C) 2017 Siemens AG > > # Add dependency from buildchroot creation > -do_build[depends] = "buildchroot:do_build" > +do_build[depends] = "buildchroot:do_prepare" > > # Each package should have its own unique build folder, so use > # recipe name as identifier > diff --git a/meta/classes/isar-events.bbclass b/meta/classes/isar-events.bbclass > new file mode 100644 > index 0000000..55fc106 > --- /dev/null > +++ b/meta/classes/isar-events.bbclass > @@ -0,0 +1,20 @@ > +# Isar event handlers. > +# > +# This software is a part of ISAR. > +# Copyright (C) 2015-2017 ilbers GmbH > + > +addhandler isar_handler > + > +python isar_handler () { > + import subprocess > + > + devnull = open(os.devnull, 'w') > + > + if isinstance(e, bb.event.BuildCompleted): > + bchroot = d.getVar('BUILDCHROOT_DIR', True) > + > + # Clean up buildchroot > + subprocess.call('/usr/bin/sudo /bin/umount ' + bchroot + '/isar-apt || /bin/true', stdout=devnull, stderr=devnull, shell=True) > + > + devnull.close() > +} > diff --git a/meta/recipes-devtools/buildchroot/buildchroot.bb b/meta/recipes-devtools/buildchroot/buildchroot.bb > index 51f9d5d..60195de 100644 > --- a/meta/recipes-devtools/buildchroot/buildchroot.bb > +++ b/meta/recipes-devtools/buildchroot/buildchroot.bb > @@ -32,7 +32,9 @@ BUILDCHROOT_PREINSTALL ?= "gcc \ > WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}" > > do_build[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" > -do_build[dirs] = "${WORKDIR}/hooks_multistrap" > +do_build[dirs] = "${WORKDIR}/hooks_multistrap \ > + ${BUILDCHROOT_DIR}/isar-apt" > +do_build[depends] = "isar-apt:do_cache_config" > > do_build() { > E="${@ bb.utils.export_proxies(d)}" > @@ -67,6 +69,8 @@ do_build() { > } > trap '_do_build_cleanup' EXIT > > + do_prepare > + > # Create root filesystem > sudo -E multistrap -a ${DISTRO_ARCH} -d "${BUILDCHROOT_DIR}" -f "${WORKDIR}/multistrap.conf" > > @@ -76,4 +80,18 @@ do_build() { > # Configure root filesystem > sudo chroot ${BUILDCHROOT_DIR} /configscript.sh > _do_build_cleanup > + > + do_cleanup > +} > + > +do_prepare[nostamp] = "1" > + > +do_prepare() { > + sudo mount --bind ${DEPLOY_DIR_APT}/${DISTRO} ${BUILDCHROOT_DIR}/isar-apt What prevents that this is run multiple times? Or will it simply silently fail if the mount already exists? > +} Maybe better call this "do_setup_mounts" or so > + > +addtask prepare after do_build That is weird. Don't we call this prior to every build anyway? > + > +do_cleanup() { > + sudo umount ${BUILDCHROOT_DIR}/isar-apt 2>/dev/null || true > } > diff --git a/meta/recipes-devtools/buildchroot/files/multistrap.conf.in b/meta/recipes-devtools/buildchroot/files/multistrap.conf.in > index a0b28e3..480a4b8 100644 > --- a/meta/recipes-devtools/buildchroot/files/multistrap.conf.in > +++ b/meta/recipes-devtools/buildchroot/files/multistrap.conf.in > @@ -6,7 +6,7 @@ noauth=true > unpack=true > ignorenativearch=true > bootstrap=##DISTRO_MULTICONF_BOOTSTRAP## > -aptsources=##DISTRO_MULTICONF_APTSOURCES## > +aptsources=isar-apt ##DISTRO_MULTICONF_APTSOURCES## > configscript=##CONFIG_SCRIPT## > setupscript=##SETUP_SCRIPT## > hookdir=##DIR_HOOKS## > @@ -29,3 +29,9 @@ source=##DISTRO_APT_SOURCE_SEC## > suite=##DISTRO_SUITE##/updates > components=##DISTRO_COMPONENTS## > omitdebsrc=true > + > +[isar-apt] > +source=file:///isar-apt > +suite=isar > +components=main > +omitdebsrc=true > Jan -- Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux