Re: [PATCH 3/5 v4] buildchroot: Enable isar-apt

[Alexander Smirnov <asmirnov@ilbers.de>]

On 02/08/2018 05:37 PM, Henning Schild wrote:
> Am Tue, 6 Feb 2018 16:57:25 +0300
> schrieb Alexander Smirnov <asmirnov@ilbers.de>:
> 
>> 8<--
>>
>> Branch to test: asmirnov/devel
>>
>> 8<--
>>
>> This patch provides access to isar-apt from buildchroot. It does the
>> following:
>>   - mount isar-apt during buildchroot building.
>>   - umount isar-apt using bitbake events.
>>
>> Also it needs to keep Isar build tree clean from any mounts despite
>> on whether build succeed of failed. bitbake provides various events
>> that could trigger custom python hooks. In this patch BuildCompleted
>> event is used, which happened when bitbake finished its execution
>> despite on the result.
>>
>> Signed-off-by: Alexander Smirnov <asmirnov@ilbers.de>
>> ---
>>   meta/classes/dpkg-base.bbclass                     |  2 +-
>>   meta/classes/isar-events.bbclass                   | 20
>> ++++++++++++++++ meta/conf/isar-bitbake.conf
>> |  3 +++ meta/recipes-devtools/buildchroot/buildchroot.bb   | 28
>> +++++++++++++++++++++- .../buildchroot/files/multistrap.conf.in
>> |  8 ++++++- 5 files changed, 58 insertions(+), 3 deletions(-)
>>   create mode 100644 meta/classes/isar-events.bbclass
>>
>> diff --git a/meta/classes/dpkg-base.bbclass
>> b/meta/classes/dpkg-base.bbclass index 026028f..5d5a924 100644
>> --- a/meta/classes/dpkg-base.bbclass
>> +++ b/meta/classes/dpkg-base.bbclass
>> @@ -2,7 +2,7 @@
>>   # Copyright (C) 2017 Siemens AG
>>   
>>   # Add dependency from buildchroot creation
>> -do_build[depends] = "buildchroot:do_build"
>> +do_build[depends] = "buildchroot:do_setup_mounts"
>>   
>>   # Each package should have its own unique build folder, so use
>>   # recipe name as identifier
>> diff --git a/meta/classes/isar-events.bbclass
>> b/meta/classes/isar-events.bbclass new file mode 100644
>> index 0000000..55fc106
>> --- /dev/null
>> +++ b/meta/classes/isar-events.bbclass
>> @@ -0,0 +1,20 @@
>> +# Isar event handlers.
>> +#
>> +# This software is a part of ISAR.
>> +# Copyright (C) 2015-2017 ilbers GmbH
>> +
>> +addhandler isar_handler
>> +
>> +python isar_handler () {
>> +    import subprocess
>> +
>> +    devnull = open(os.devnull, 'w')
>> +
>> +    if isinstance(e, bb.event.BuildCompleted):
>> +        bchroot = d.getVar('BUILDCHROOT_DIR', True)
>> +
>> +        # Clean up buildchroot
>> +        subprocess.call('/usr/bin/sudo /bin/umount ' + bchroot +
>> '/isar-apt || /bin/true', stdout=devnull, stderr=devnull, shell=True)
> 
> I would suggest to drop the hardcoded paths where possible. Imagine i
> had my sudo sitting in /usr/local/bin/ or similar. And sudo internally
> will take care of its own PATH so the /bin/ for umount is also not
> required.
> Another use-case could be a wrapper around the actual sudo where people
> try to handle the sudo-problem of Isar by having a wrapper that uses a
> whitelist of allowed commands.
> 

This issue resolved in patch:
[PATCH] isar: Clean mount point on bitbake exit

Alex

> Henning
> 
>> +
>> +    devnull.close()
>> +}
>> diff --git a/meta/conf/isar-bitbake.conf b/meta/conf/isar-bitbake.conf
>> index b853c88..36cca29 100644
>> --- a/meta/conf/isar-bitbake.conf
>> +++ b/meta/conf/isar-bitbake.conf
>> @@ -27,6 +27,9 @@ BUILDCHROOT_DIR =
>> "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/buildchroot/rootfs" # Setup
>> our default hash policy BB_SIGNATURE_HANDLER ?= "noop"
>>   
>> +# Add event handlers for bitbake
>> +INHERIT += "isar-events"
>> +
>>   include conf/local.conf
>>   include conf/multiconfig/${BB_CURRENT_MC}.conf
>>   include conf/machine/${MACHINE}.conf
>> diff --git a/meta/recipes-devtools/buildchroot/buildchroot.bb
>> b/meta/recipes-devtools/buildchroot/buildchroot.bb index
>> 51f9d5d..304c67e 100644 ---
>> a/meta/recipes-devtools/buildchroot/buildchroot.bb +++
>> b/meta/recipes-devtools/buildchroot/buildchroot.bb @@ -32,7 +32,9 @@
>> BUILDCHROOT_PREINSTALL ?= "gcc \ WORKDIR =
>> "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}"
>>   do_build[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
>> -do_build[dirs] = "${WORKDIR}/hooks_multistrap"
>> +do_build[dirs] = "${WORKDIR}/hooks_multistrap \
>> +                  ${BUILDCHROOT_DIR}/isar-apt"
>> +do_build[depends] = "isar-apt:do_cache_config"
>>   
>>   do_build() {
>>       E="${@ bb.utils.export_proxies(d)}"
>> @@ -67,6 +69,8 @@ do_build() {
>>       }
>>       trap '_do_build_cleanup' EXIT
>>   
>> +    do_setup_mounts
>> +
>>       # Create root filesystem
>>       sudo -E multistrap -a ${DISTRO_ARCH} -d "${BUILDCHROOT_DIR}" -f
>> "${WORKDIR}/multistrap.conf"
>> @@ -76,4 +80,26 @@ do_build() {
>>       # Configure root filesystem
>>       sudo chroot ${BUILDCHROOT_DIR} /configscript.sh
>>       _do_build_cleanup
>> +
>> +    do_cleanup_mounts
>> +}
>> +
>> +# Invalidate stamp for do_setup_mounts before each build start.
>> +# This will guarantee that this function will be executed once
>> +# per build.
>> +python __anonymous() {
>> +    stamp = d.getVar("STAMP") + ".do_setup_mounts." +
>> d.getVarFlag("do_setup_mounts", 'stamp-extra-info')
>> +    os.remove(stamp) if os.path.exists(stamp) else None
>> +}
>> +
>> +do_setup_mounts[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
>> +
>> +do_setup_mounts() {
>> +    sudo mount --bind ${DEPLOY_DIR_APT}/${DISTRO}
>> ${BUILDCHROOT_DIR}/isar-apt +}
>> +
>> +addtask setup_mounts after do_build
>> +
>> +do_cleanup_mounts() {
>> +    sudo umount ${BUILDCHROOT_DIR}/isar-apt 2>/dev/null || true
>>   }
>> diff --git
>> a/meta/recipes-devtools/buildchroot/files/multistrap.conf.in
>> b/meta/recipes-devtools/buildchroot/files/multistrap.conf.in index
>> a0b28e3..480a4b8 100644 ---
>> a/meta/recipes-devtools/buildchroot/files/multistrap.conf.in +++
>> b/meta/recipes-devtools/buildchroot/files/multistrap.conf.in @@ -6,7
>> +6,7 @@ noauth=true unpack=true ignorenativearch=true
>>   bootstrap=##DISTRO_MULTICONF_BOOTSTRAP##
>> -aptsources=##DISTRO_MULTICONF_APTSOURCES##
>> +aptsources=isar-apt ##DISTRO_MULTICONF_APTSOURCES##
>>   configscript=##CONFIG_SCRIPT##
>>   setupscript=##SETUP_SCRIPT##
>>   hookdir=##DIR_HOOKS##
>> @@ -29,3 +29,9 @@ source=##DISTRO_APT_SOURCE_SEC##
>>   suite=##DISTRO_SUITE##/updates
>>   components=##DISTRO_COMPONENTS##
>>   omitdebsrc=true
>> +
>> +[isar-apt]
>> +source=file:///isar-apt
>> +suite=isar
>> +components=main
>> +omitdebsrc=true
> 

-- 
With best regards,
Alexander Smirnov

ilbers GmbH
Baierbrunner Str. 28c
D-81379 Munich
+49 (89) 122 67 24-0
http://ilbers.de/
Commercial register Munich, HRB 214197
General manager: Baurzhan Ismagulov