From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6519007876157014016 X-Received: by 10.28.192.26 with SMTP id q26mr243281wmf.29.1517923721566; Tue, 06 Feb 2018 05:28:41 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 10.223.129.228 with SMTP id 91ls3160939wra.6.gmail; Tue, 06 Feb 2018 05:28:41 -0800 (PST) X-Google-Smtp-Source: AH8x224IXR7j69FmvErJPSOxh5cuXbrsHN5DTeEqqc1pcxaujZIHYJ68/GU/hLc8CknKHLf1hKhn X-Received: by 10.223.157.199 with SMTP id q7mr268083wre.11.1517923721065; Tue, 06 Feb 2018 05:28:41 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1517923721; cv=none; d=google.com; s=arc-20160816; b=KEn+Bu/n+VOPbwRnkr+kmDQJ5hBv/aH8ZVvYN1VEp6Ath6BR1PGPKKr8fix6w4LYtH J8yCL+NcLhmTsWj9IcnDCvvBTmj/4OpOF+7ETcxoNYesVzql+EwsYcrseoZ4zXgbrJE9 hfPTC2QDxxkRtjkKmrdQYmdV+6iwlrn+01Vydxxq1WTd5QLZzNcjbSn0TH31Z70QQ6mz 2OOBX+mffqK1TKeY7RMKIPIv8aKHRI6L8UoubZnXOaEevFPAHpmXduzG5Ni+4wzYwRp9 ijfzDGNw7LhOd+aYDy6qH9NtvE4Ttzds7ej9ZvGr72CZ0PGHi9IWTCSSOJ404sB7HFLj oEvg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:to:subject :arc-authentication-results; bh=RkWiqYpijWtU8oxg+Xa8HAET1HbNlJPnpHLzThjWr8w=; b=yDs6rUkZemK8SemQdGNy9Af57QYKNvrJgHTEFDBFfnyvKXMIHmIYND3LsHilw327k8 5kRXPb4XVXXqvSEPLH1+AiF6dXqFGoEaQU9YZ47Q8vuadNl08Ua8ZBH/xFpK7/8mjWsi B4MU98JP0VYe59Yrxmotg9t6fM2epaDfPZPFi5bQjsl3jsgClr8YZ0bSefjuYWCs04vH ssbkSS1bJMgWr4XgOQcm4yYt9DHytkxsqJUKbJEiSmC2MuT2Ggoh8on7QSFjn8W2pRiJ ehixp+bDkxzeGo/KmN9FWUlhbXBM34xhS8OonGw5sEgaNccmy9RsGcl1V/fTMPBzXGiM 7jQw== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) smtp.mailfrom=asmirnov@ilbers.de Return-Path: Received: from aqmola.ilbers.de (aqmola.ilbers.de. [85.214.62.211]) by gmr-mx.google.com with ESMTPS id k21si651451wrd.1.2018.02.06.05.28.40 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 06 Feb 2018 05:28:40 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) client-ip=85.214.62.211; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) smtp.mailfrom=asmirnov@ilbers.de Received: from [10.0.2.15] ([188.227.110.165]) (authenticated bits=0) by aqmola.ilbers.de (8.14.4/8.14.4/Debian-4+deb7u1) with ESMTP id w16DSbIc029267 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Tue, 6 Feb 2018 14:28:39 +0100 Subject: Re: [PATCH 3/5 v3] buildchroot: Enable isar-apt To: Jan Kiszka , isar-users@googlegroups.com References: <20180205134844.27221-1-asmirnov@ilbers.de> <20180206110119.10580-1-asmirnov@ilbers.de> <862a3edd-89c1-1278-d16e-833b12260dfc@siemens.com> From: Alexander Smirnov Message-ID: Date: Tue, 6 Feb 2018 16:28:32 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: <862a3edd-89c1-1278-d16e-833b12260dfc@siemens.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-TUID: DAVsT/c165xK On 02/06/2018 03:10 PM, Jan Kiszka wrote: > On 2018-02-06 12:01, Alexander Smirnov wrote: >> 8<-- >> >> Changes since v2: >> - Move do_prepare after do_build. This will not trigger do_build for >> reoccurent builds. > > Hmm, aren't you also moving it *before every* build? That's how I read > the code. > Yes, there is dependency dpkg-base: do_build[depends] = "buildchroot:do_prepare" so it's a mandatory dependency for each recipe to be built. >> >> 8<-- >> >> This patch provides access to isar-apt from buildchroot. It does the >> following: >> - mount isar-apt during buildchroot building. >> - umount isar-apt using bitbake events. >> >> Also it needs to keep Isar build tree clean from any mounts despite on whether >> build succeed of failed. bitbake provides various events that could trigger >> custom python hooks. In this patch BuildCompleted event is used, which happened >> when bitbake finished its execution despite on the result. >> >> Signed-off-by: Alexander Smirnov >> --- >> meta-isar/conf/local.conf.sample | 3 +++ >> meta/classes/dpkg-base.bbclass | 2 +- >> meta/classes/isar-events.bbclass | 20 ++++++++++++++++++++ >> meta/recipes-devtools/buildchroot/buildchroot.bb | 20 +++++++++++++++++++- >> .../buildchroot/files/multistrap.conf.in | 8 +++++++- >> 5 files changed, 50 insertions(+), 3 deletions(-) >> create mode 100644 meta/classes/isar-events.bbclass >> >> diff --git a/meta-isar/conf/local.conf.sample b/meta-isar/conf/local.conf.sample >> index 2ae43e7..fc760ed 100644 >> --- a/meta-isar/conf/local.conf.sample >> +++ b/meta-isar/conf/local.conf.sample >> @@ -158,3 +158,6 @@ IMAGE_INSTALL = "example-hello example-raw" >> # >> # Default parallel jobs for bitbake: >> BB_NUMBER_THREADS = "4" >> + >> +# Add event handlers for bitbake >> +INHERIT += "isar-events" > > If you are already updating, please move this to a proper place > (isar-bitbake.conf) like I suggested via my other patch. Will be done in v4. > >> diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass >> index 026028f..74dedc5 100644 >> --- a/meta/classes/dpkg-base.bbclass >> +++ b/meta/classes/dpkg-base.bbclass >> @@ -2,7 +2,7 @@ >> # Copyright (C) 2017 Siemens AG >> >> # Add dependency from buildchroot creation >> -do_build[depends] = "buildchroot:do_build" >> +do_build[depends] = "buildchroot:do_prepare" >> >> # Each package should have its own unique build folder, so use >> # recipe name as identifier >> diff --git a/meta/classes/isar-events.bbclass b/meta/classes/isar-events.bbclass >> new file mode 100644 >> index 0000000..55fc106 >> --- /dev/null >> +++ b/meta/classes/isar-events.bbclass >> @@ -0,0 +1,20 @@ >> +# Isar event handlers. >> +# >> +# This software is a part of ISAR. >> +# Copyright (C) 2015-2017 ilbers GmbH >> + >> +addhandler isar_handler >> + >> +python isar_handler () { >> + import subprocess >> + >> + devnull = open(os.devnull, 'w') >> + >> + if isinstance(e, bb.event.BuildCompleted): >> + bchroot = d.getVar('BUILDCHROOT_DIR', True) >> + >> + # Clean up buildchroot >> + subprocess.call('/usr/bin/sudo /bin/umount ' + bchroot + '/isar-apt || /bin/true', stdout=devnull, stderr=devnull, shell=True) >> + >> + devnull.close() >> +} >> diff --git a/meta/recipes-devtools/buildchroot/buildchroot.bb b/meta/recipes-devtools/buildchroot/buildchroot.bb >> index 51f9d5d..60195de 100644 >> --- a/meta/recipes-devtools/buildchroot/buildchroot.bb >> +++ b/meta/recipes-devtools/buildchroot/buildchroot.bb >> @@ -32,7 +32,9 @@ BUILDCHROOT_PREINSTALL ?= "gcc \ >> WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}" >> >> do_build[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" >> -do_build[dirs] = "${WORKDIR}/hooks_multistrap" >> +do_build[dirs] = "${WORKDIR}/hooks_multistrap \ >> + ${BUILDCHROOT_DIR}/isar-apt" >> +do_build[depends] = "isar-apt:do_cache_config" >> >> do_build() { >> E="${@ bb.utils.export_proxies(d)}" >> @@ -67,6 +69,8 @@ do_build() { >> } >> trap '_do_build_cleanup' EXIT >> >> + do_prepare >> + >> # Create root filesystem >> sudo -E multistrap -a ${DISTRO_ARCH} -d "${BUILDCHROOT_DIR}" -f "${WORKDIR}/multistrap.conf" >> >> @@ -76,4 +80,18 @@ do_build() { >> # Configure root filesystem >> sudo chroot ${BUILDCHROOT_DIR} /configscript.sh >> _do_build_cleanup >> + >> + do_cleanup >> +} >> + >> +do_prepare[nostamp] = "1" >> + >> +do_prepare() { >> + sudo mount --bind ${DEPLOY_DIR_APT}/${DISTRO} ${BUILDCHROOT_DIR}/isar-apt > > What prevents that this is run multiple times? Or will it simply > silently fail if the mount already exists? > Here nothing, will fix in v4. >> +} > > Maybe better call this "do_setup_mounts" or so > Agreed. >> + >> +addtask prepare after do_build > > That is weird. Don't we call this prior to every build anyway? > We do. See the first comment above. Alex >> + >> +do_cleanup() { >> + sudo umount ${BUILDCHROOT_DIR}/isar-apt 2>/dev/null || true >> } >> diff --git a/meta/recipes-devtools/buildchroot/files/multistrap.conf.in b/meta/recipes-devtools/buildchroot/files/multistrap.conf.in >> index a0b28e3..480a4b8 100644 >> --- a/meta/recipes-devtools/buildchroot/files/multistrap.conf.in >> +++ b/meta/recipes-devtools/buildchroot/files/multistrap.conf.in >> @@ -6,7 +6,7 @@ noauth=true >> unpack=true >> ignorenativearch=true >> bootstrap=##DISTRO_MULTICONF_BOOTSTRAP## >> -aptsources=##DISTRO_MULTICONF_APTSOURCES## >> +aptsources=isar-apt ##DISTRO_MULTICONF_APTSOURCES## >> configscript=##CONFIG_SCRIPT## >> setupscript=##SETUP_SCRIPT## >> hookdir=##DIR_HOOKS## >> @@ -29,3 +29,9 @@ source=##DISTRO_APT_SOURCE_SEC## >> suite=##DISTRO_SUITE##/updates >> components=##DISTRO_COMPONENTS## >> omitdebsrc=true >> + >> +[isar-apt] >> +source=file:///isar-apt >> +suite=isar >> +components=main >> +omitdebsrc=true >> > > Jan >