Re: [PATCH 3/5 v4] buildchroot: Enable isar-apt

[Jan Kiszka <jan.kiszka@siemens.com>]

On 2018-02-06 14:57, Alexander Smirnov wrote:
> 8<--
> 
> Branch to test: asmirnov/devel
> 
> 8<--
> 
> This patch provides access to isar-apt from buildchroot. It does the
> following:
>  - mount isar-apt during buildchroot building.
>  - umount isar-apt using bitbake events.
> 
> Also it needs to keep Isar build tree clean from any mounts despite on whether
> build succeed of failed. bitbake provides various events that could trigger
> custom python hooks. In this patch BuildCompleted event is used, which happened
> when bitbake finished its execution despite on the result.
> 
> Signed-off-by: Alexander Smirnov <asmirnov@ilbers.de>
> ---
>  meta/classes/dpkg-base.bbclass                     |  2 +-
>  meta/classes/isar-events.bbclass                   | 20 ++++++++++++++++
>  meta/conf/isar-bitbake.conf                        |  3 +++
>  meta/recipes-devtools/buildchroot/buildchroot.bb   | 28 +++++++++++++++++++++-
>  .../buildchroot/files/multistrap.conf.in           |  8 ++++++-
>  5 files changed, 58 insertions(+), 3 deletions(-)
>  create mode 100644 meta/classes/isar-events.bbclass
> 
> diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass
> index 026028f..5d5a924 100644
> --- a/meta/classes/dpkg-base.bbclass
> +++ b/meta/classes/dpkg-base.bbclass
> @@ -2,7 +2,7 @@
>  # Copyright (C) 2017 Siemens AG
>  
>  # Add dependency from buildchroot creation
> -do_build[depends] = "buildchroot:do_build"
> +do_build[depends] = "buildchroot:do_setup_mounts"
>  
>  # Each package should have its own unique build folder, so use
>  # recipe name as identifier
> diff --git a/meta/classes/isar-events.bbclass b/meta/classes/isar-events.bbclass
> new file mode 100644
> index 0000000..55fc106
> --- /dev/null
> +++ b/meta/classes/isar-events.bbclass
> @@ -0,0 +1,20 @@
> +# Isar event handlers.
> +#
> +# This software is a part of ISAR.
> +# Copyright (C) 2015-2017 ilbers GmbH
> +
> +addhandler isar_handler
> +
> +python isar_handler () {
> +    import subprocess
> +
> +    devnull = open(os.devnull, 'w')
> +
> +    if isinstance(e, bb.event.BuildCompleted):
> +        bchroot = d.getVar('BUILDCHROOT_DIR', True)
> +
> +        # Clean up buildchroot
> +        subprocess.call('/usr/bin/sudo /bin/umount ' + bchroot + '/isar-apt || /bin/true', stdout=devnull, stderr=devnull, shell=True)
> +
> +    devnull.close()
> +}
> diff --git a/meta/conf/isar-bitbake.conf b/meta/conf/isar-bitbake.conf
> index b853c88..36cca29 100644
> --- a/meta/conf/isar-bitbake.conf
> +++ b/meta/conf/isar-bitbake.conf
> @@ -27,6 +27,9 @@ BUILDCHROOT_DIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/buildchroot/rootfs"
>  # Setup our default hash policy
>  BB_SIGNATURE_HANDLER ?= "noop"
>  
> +# Add event handlers for bitbake
> +INHERIT += "isar-events"
> +
>  include conf/local.conf
>  include conf/multiconfig/${BB_CURRENT_MC}.conf
>  include conf/machine/${MACHINE}.conf
> diff --git a/meta/recipes-devtools/buildchroot/buildchroot.bb b/meta/recipes-devtools/buildchroot/buildchroot.bb
> index 51f9d5d..304c67e 100644
> --- a/meta/recipes-devtools/buildchroot/buildchroot.bb
> +++ b/meta/recipes-devtools/buildchroot/buildchroot.bb
> @@ -32,7 +32,9 @@ BUILDCHROOT_PREINSTALL ?= "gcc \
>  WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}"
>  
>  do_build[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
> -do_build[dirs] = "${WORKDIR}/hooks_multistrap"
> +do_build[dirs] = "${WORKDIR}/hooks_multistrap \
> +                  ${BUILDCHROOT_DIR}/isar-apt"
> +do_build[depends] = "isar-apt:do_cache_config"
>  
>  do_build() {
>      E="${@ bb.utils.export_proxies(d)}"
> @@ -67,6 +69,8 @@ do_build() {
>      }
>      trap '_do_build_cleanup' EXIT
>  
> +    do_setup_mounts
> +
>      # Create root filesystem
>      sudo -E multistrap -a ${DISTRO_ARCH} -d "${BUILDCHROOT_DIR}" -f "${WORKDIR}/multistrap.conf"
>  
> @@ -76,4 +80,26 @@ do_build() {
>      # Configure root filesystem
>      sudo chroot ${BUILDCHROOT_DIR} /configscript.sh
>      _do_build_cleanup
> +
> +    do_cleanup_mounts
> +}
> +
> +# Invalidate stamp for do_setup_mounts before each build start.
> +# This will guarantee that this function will be executed once
> +# per build.
> +python __anonymous() {
> +    stamp = d.getVar("STAMP") + ".do_setup_mounts." + d.getVarFlag("do_setup_mounts", 'stamp-extra-info')
> +    os.remove(stamp) if os.path.exists(stamp) else None
> +}
> +
> +do_setup_mounts[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
> +
> +do_setup_mounts() {
> +    sudo mount --bind ${DEPLOY_DIR_APT}/${DISTRO} ${BUILDCHROOT_DIR}/isar-apt
> +}
> +
> +addtask setup_mounts after do_build
> +
> +do_cleanup_mounts() {
> +    sudo umount ${BUILDCHROOT_DIR}/isar-apt 2>/dev/null || true
>  }
> diff --git a/meta/recipes-devtools/buildchroot/files/multistrap.conf.in b/meta/recipes-devtools/buildchroot/files/multistrap.conf.in
> index a0b28e3..480a4b8 100644
> --- a/meta/recipes-devtools/buildchroot/files/multistrap.conf.in
> +++ b/meta/recipes-devtools/buildchroot/files/multistrap.conf.in
> @@ -6,7 +6,7 @@ noauth=true
>  unpack=true
>  ignorenativearch=true
>  bootstrap=##DISTRO_MULTICONF_BOOTSTRAP##
> -aptsources=##DISTRO_MULTICONF_APTSOURCES##
> +aptsources=isar-apt ##DISTRO_MULTICONF_APTSOURCES##
>  configscript=##CONFIG_SCRIPT##
>  setupscript=##SETUP_SCRIPT##
>  hookdir=##DIR_HOOKS##
> @@ -29,3 +29,9 @@ source=##DISTRO_APT_SOURCE_SEC##
>  suite=##DISTRO_SUITE##/updates
>  components=##DISTRO_COMPONENTS##
>  omitdebsrc=true
> +
> +[isar-apt]
> +source=file:///isar-apt
> +suite=isar
> +components=main
> +omitdebsrc=true
> 

Looks good! I'm testing it via your devel branch, and I also
successfully rebased "Permanently mount '/dev', '/sys' and '/proc'" on top.

Regarding the remaining cleanup of WORKDIR mounting: Can't each
dpkg-base class register their own cleanup handler in the same fashion?

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux