Re: [PATCH 3/5] buildchroot: Enable isar-apt

[Alexander Smirnov <asmirnov@ilbers.de>]

On 02/05/2018 01:18 PM, Jan Kiszka wrote:
> On 2018-02-05 10:59, Alexander Smirnov wrote:
>> This patch provides access to isar-apt from buildchroot. It does the
>> following:
>>   - mount isar-apt during buildchroot building.
>>   - umount isar-apt using bitbake events.
>>
>> Also it needs to keep Isar build tree clean from any mounts despite on whether
>> build succeed of failed. bitbake provides various events that could trigger
>> custom python hooks. In this patch BuildCompleted event is used, which happened
>> when bitbake finished its execution despite on the result.
>>
>> Signed-off-by: Alexander Smirnov <asmirnov@ilbers.de>
>> ---
>>   meta-isar/conf/local.conf.sample                     |  3 +++
>>   meta/classes/isar-events.bbclass                     | 20 ++++++++++++++++++++
>>   meta/recipes-devtools/buildchroot/buildchroot.bb     |  7 +++++++
>>   .../buildchroot/files/multistrap.conf.in             |  8 +++++++-
>>   4 files changed, 37 insertions(+), 1 deletion(-)
>>   create mode 100644 meta/classes/isar-events.bbclass
>>
>> diff --git a/meta-isar/conf/local.conf.sample b/meta-isar/conf/local.conf.sample
>> index 2ae43e7..fc760ed 100644
>> --- a/meta-isar/conf/local.conf.sample
>> +++ b/meta-isar/conf/local.conf.sample
>> @@ -158,3 +158,6 @@ IMAGE_INSTALL = "example-hello example-raw"
>>   #
>>   # Default parallel jobs for bitbake:
>>   BB_NUMBER_THREADS = "4"
>> +
>> +# Add event handlers for bitbake
>> +INHERIT += "isar-events"
>> diff --git a/meta/classes/isar-events.bbclass b/meta/classes/isar-events.bbclass
>> new file mode 100644
>> index 0000000..55fc106
>> --- /dev/null
>> +++ b/meta/classes/isar-events.bbclass
>> @@ -0,0 +1,20 @@
>> +# Isar event handlers.
>> +#
>> +# This software is a part of ISAR.
>> +# Copyright (C) 2015-2017 ilbers GmbH
>> +
>> +addhandler isar_handler
>> +
>> +python isar_handler () {
>> +    import subprocess
>> +
>> +    devnull = open(os.devnull, 'w')
>> +
>> +    if isinstance(e, bb.event.BuildCompleted):
>> +        bchroot = d.getVar('BUILDCHROOT_DIR', True)
>> +
>> +        # Clean up buildchroot
>> +        subprocess.call('/usr/bin/sudo /bin/umount ' + bchroot + '/isar-apt || /bin/true', stdout=devnull, stderr=devnull, shell=True)
>> +
>> +    devnull.close()
>> +}
>> diff --git a/meta/recipes-devtools/buildchroot/buildchroot.bb b/meta/recipes-devtools/buildchroot/buildchroot.bb
>> index 51f9d5d..cb90148 100644
>> --- a/meta/recipes-devtools/buildchroot/buildchroot.bb
>> +++ b/meta/recipes-devtools/buildchroot/buildchroot.bb
>> @@ -33,6 +33,7 @@ WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}"
>>   
>>   do_build[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
>>   do_build[dirs] = "${WORKDIR}/hooks_multistrap"
>> +do_build[depends] = "isar-apt:do_cache_config"
>>   
>>   do_build() {
>>       E="${@ bb.utils.export_proxies(d)}"
>> @@ -58,6 +59,9 @@ do_build() {
>>           -e 's|##DIR_HOOKS##|./'"$WORKDIR_REL"'/hooks_multistrap|g' \
>>              "${WORKDIR}/multistrap.conf.in" > "${WORKDIR}/multistrap.conf"
>>   
>> +    [ ! -d ${BUILDCHROOT_DIR}/isar-apt ] && install -d -m 555 ${BUILDCHROOT_DIR}/isar-apt
>> +    sudo mount --bind ${DEPLOY_DIR_APT}/${DISTRO} ${BUILDCHROOT_DIR}/isar-apt
>> +
>>       [ ! -d ${BUILDCHROOT_DIR}/proc ] && install -d -m 555 ${BUILDCHROOT_DIR}/proc
>>       sudo mount -t proc none ${BUILDCHROOT_DIR}/proc
>>       _do_build_cleanup() {
>> @@ -73,6 +77,9 @@ do_build() {
>>       # Install package builder script
>>       sudo install -m 755 ${WORKDIR}/build.sh ${BUILDCHROOT_DIR}
>>   
>> +    # Create share point for isar-apt
>> +    sudo install -d ${BUILDCHROOT_DIR}/isar-apt
>> +
>>       # Configure root filesystem
>>       sudo chroot ${BUILDCHROOT_DIR} /configscript.sh
>>       _do_build_cleanup
>> diff --git a/meta/recipes-devtools/buildchroot/files/multistrap.conf.in b/meta/recipes-devtools/buildchroot/files/multistrap.conf.in
>> index a0b28e3..480a4b8 100644
>> --- a/meta/recipes-devtools/buildchroot/files/multistrap.conf.in
>> +++ b/meta/recipes-devtools/buildchroot/files/multistrap.conf.in
>> @@ -6,7 +6,7 @@ noauth=true
>>   unpack=true
>>   ignorenativearch=true
>>   bootstrap=##DISTRO_MULTICONF_BOOTSTRAP##
>> -aptsources=##DISTRO_MULTICONF_APTSOURCES##
>> +aptsources=isar-apt ##DISTRO_MULTICONF_APTSOURCES##
>>   configscript=##CONFIG_SCRIPT##
>>   setupscript=##SETUP_SCRIPT##
>>   hookdir=##DIR_HOOKS##
>> @@ -29,3 +29,9 @@ source=##DISTRO_APT_SOURCE_SEC##
>>   suite=##DISTRO_SUITE##/updates
>>   components=##DISTRO_COMPONENTS##
>>   omitdebsrc=true
>> +
>> +[isar-apt]
>> +source=file:///isar-apt
>> +suite=isar
>> +components=main
>> +omitdebsrc=true
>>
> 
> On first glance, this looks identical to v1 which I tried over the
> current bitbake version we have in Isar. There I see two issues:
> 
> - umount does not work if the build fails

Started replying to your previous mail, but let's discuss it here.
Clean up hooks is mostly a hack and IMHO should be replaced by events.

I use here bitbake events, see file 'meta/classes/isar-events.bbclass'.
There is event BuildCompleted which means that bitbake has finished its 
pipeline despite on the result. So I've added hook to this event which 
performs unmount, this also works good for double ^C.

> - re-mount will not happen if the buildchroot was built in a previous
>    run and only a depending packages is built now

Good point. Will fix it.

Alex