Re: [PATCH 3/5] meta: Implement two stage build

[Jan Kiszka <jan.kiszka@siemens.com>]

On 15.09.23 20:54, Anton Mikanovich wrote:
> Split package build into two stages: building source package and
> building binary package. This allows to obtain Debian package
> dependencies before any build steps will actually start.
> 
> Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
> ---
>  meta/classes/dpkg-gbp.bbclass              |  2 +-
>  meta/classes/dpkg-source.bbclass           | 33 ++++++++++++++++++++++
>  meta/classes/dpkg.bbclass                  |  4 +--
>  meta/classes/rootfs.bbclass                |  2 ++
>  meta/recipes-kernel/linux/linux-custom.inc |  2 +-
>  5 files changed, 38 insertions(+), 5 deletions(-)
>  create mode 100644 meta/classes/dpkg-source.bbclass
> 
> diff --git a/meta/classes/dpkg-gbp.bbclass b/meta/classes/dpkg-gbp.bbclass
> index 2bd09eaa..a151ade2 100644
> --- a/meta/classes/dpkg-gbp.bbclass
> +++ b/meta/classes/dpkg-gbp.bbclass
> @@ -11,7 +11,7 @@ PATCHTOOL ?= "git"
>  
>  GBP_EXTRA_OPTIONS ?= "--git-pristine-tar"
>  
> -dpkg_runbuild:prepend() {
> +do_dpkg_source:prepend() {
>      sh -c "
>          cd ${WORKDIR}/${PPS}
>          gbp buildpackage --git-ignore-new --git-builder=/bin/true ${GBP_EXTRA_OPTIONS}
> diff --git a/meta/classes/dpkg-source.bbclass b/meta/classes/dpkg-source.bbclass
> new file mode 100644
> index 00000000..434f1752
> --- /dev/null
> +++ b/meta/classes/dpkg-source.bbclass
> @@ -0,0 +1,33 @@
> +# This software is a part of ISAR.
> +# Copyright (C) 2023 ilbers GmbH
> +#
> +# SPDX-License-Identifier: MIT
> +
> +inherit dpkg-base
> +
> +do_dpkg_source() {
> +    # Create a .dsc file from source directory to use it with sbuild
> +    DEB_SOURCE_NAME=$(dpkg-parsechangelog --show-field Source --file ${WORKDIR}/${PPS}/debian/changelog)
> +    find ${WORKDIR} -name "${DEB_SOURCE_NAME}*.dsc" -maxdepth 1 -delete
> +    if [ ! -z "${SOURCE_DATE_EPOCH}" ]; then
> +        export SOURCE_DATE_EPOCH="${SOURCE_DATE_EPOCH}"
> +    fi
> +    sh -c "cd ${WORKDIR}; dpkg-source -I -b ${PPS}"
> +}
> +addtask dpkg_source after do_prepare_build before do_dpkg_build
> +
> +do_deploy_source[depends] += "isar-apt:do_cache_config"
> +do_deploy_source[lockfiles] = "${REPO_ISAR_DIR}/isar.lock"
> +do_deploy_source[dirs] = "${S}"
> +do_deploy_source() {
> +    repo_del_srcpackage "${REPO_ISAR_DIR}"/"${DISTRO}" \
> +        "${REPO_ISAR_DB_DIR}"/"${DISTRO}" "${DEBDISTRONAME}" "${BPN}"
> +    find "${S}/../" -name '*\.dsc' -maxdepth 1 | while read package; do
> +        repo_add_srcpackage "${REPO_ISAR_DIR}"/"${DISTRO}" \
> +            "${REPO_ISAR_DB_DIR}"/"${DISTRO}" \
> +            "${DEBDISTRONAME}" \
> +            "${package}"
> +    done
> +}
> +addtask deploy_source after do_dpkg_source before do_dpkg_build
> +
> diff --git a/meta/classes/dpkg.bbclass b/meta/classes/dpkg.bbclass
> index 90d717ca..2bb167e4 100644
> --- a/meta/classes/dpkg.bbclass
> +++ b/meta/classes/dpkg.bbclass
> @@ -2,6 +2,7 @@
>  # Copyright (C) 2015-2018 ilbers GmbH
>  
>  inherit dpkg-base
> +inherit dpkg-source
>  
>  PACKAGE_ARCH ?= "${DISTRO_ARCH}"
>  
> @@ -95,10 +96,7 @@ dpkg_runbuild() {
>  
>      echo '$apt_keep_downloaded_packages = 1;' >> ${SBUILD_CONFIG}
>  
> -    # Create a .dsc file from source directory to use it with sbuild
>      DEB_SOURCE_NAME=$(dpkg-parsechangelog --show-field Source --file ${WORKDIR}/${PPS}/debian/changelog)
> -    find ${WORKDIR} -name "${DEB_SOURCE_NAME}*.dsc" -maxdepth 1 -delete
> -    sh -c "cd ${WORKDIR}; dpkg-source -q -b ${PPS}"
>      DSC_FILE=$(find ${WORKDIR} -name "${DEB_SOURCE_NAME}*.dsc" -maxdepth 1 -print)
>  
>      sbuild -A -n -c ${SBUILD_CHROOT} --extra-repository="${ISAR_APT_REPO}" \
> diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass
> index 7dec7b36..65574a6c 100644
> --- a/meta/classes/rootfs.bbclass
> +++ b/meta/classes/rootfs.bbclass
> @@ -90,6 +90,8 @@ rootfs_configure_isar_apt() {
>      mkdir -p '${ROOTFSDIR}/etc/apt/sources.list.d'
>      echo 'deb [trusted=yes] file:///isar-apt ${DEBDISTRONAME} main' > \
>          '${ROOTFSDIR}/etc/apt/sources.list.d/isar-apt.list'
> +    echo 'deb-src [trusted=yes] file:///isar-apt ${DEBDISTRONAME} main' >> \
> +        '${ROOTFSDIR}/etc/apt/sources.list.d/isar-apt.list'
>  
>      mkdir -p '${ROOTFSDIR}/etc/apt/preferences.d'
>      cat << EOF > '${ROOTFSDIR}/etc/apt/preferences.d/isar-apt'
> diff --git a/meta/recipes-kernel/linux/linux-custom.inc b/meta/recipes-kernel/linux/linux-custom.inc
> index 8b367a2f..fa5d957d 100644
> --- a/meta/recipes-kernel/linux/linux-custom.inc
> +++ b/meta/recipes-kernel/linux/linux-custom.inc
> @@ -212,6 +212,6 @@ EOF
>  	fi
>  }
>  
> -dpkg_runbuild:prepend() {
> +do_dpkg_source:prepend() {
>  	dpkg_configure_kernel
>  }

This is not transparent to recipe as the very last hunk nicely
demonstrates. Where is the RECIPE-API-CHANGLOG entry?

I thought this change was already the reason, but I'm still searching
for a regression of this patch here on our downstream recipe
https://gitlab.com/cip-project/cip-core/isar-cip-core/-/tree/master/recipes-bsp/edk2
(can be watched under
https://github.com/siemens/meta-iot2050/actions/runs/6492616563/job/17631910317.
Any hints welcome.

Jan

-- 
Siemens AG, Technology
Linux Expert Center