Re: [PATCH 3/5] meta: Implement two stage build

[Jan Kiszka <jan.kiszka@siemens.com>]

On 12.10.23 10:48, Jan Kiszka wrote:
> On 15.09.23 20:54, Anton Mikanovich wrote:
>> Split package build into two stages: building source package and
>> building binary package. This allows to obtain Debian package
>> dependencies before any build steps will actually start.
>>
>> Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
>> ---
>>  meta/classes/dpkg-gbp.bbclass              |  2 +-
>>  meta/classes/dpkg-source.bbclass           | 33 ++++++++++++++++++++++
>>  meta/classes/dpkg.bbclass                  |  4 +--
>>  meta/classes/rootfs.bbclass                |  2 ++
>>  meta/recipes-kernel/linux/linux-custom.inc |  2 +-
>>  5 files changed, 38 insertions(+), 5 deletions(-)
>>  create mode 100644 meta/classes/dpkg-source.bbclass
>>
>> diff --git a/meta/classes/dpkg-gbp.bbclass b/meta/classes/dpkg-gbp.bbclass
>> index 2bd09eaa..a151ade2 100644
>> --- a/meta/classes/dpkg-gbp.bbclass
>> +++ b/meta/classes/dpkg-gbp.bbclass
>> @@ -11,7 +11,7 @@ PATCHTOOL ?= "git"
>>  
>>  GBP_EXTRA_OPTIONS ?= "--git-pristine-tar"
>>  
>> -dpkg_runbuild:prepend() {
>> +do_dpkg_source:prepend() {
>>      sh -c "
>>          cd ${WORKDIR}/${PPS}
>>          gbp buildpackage --git-ignore-new --git-builder=/bin/true ${GBP_EXTRA_OPTIONS}
>> diff --git a/meta/classes/dpkg-source.bbclass b/meta/classes/dpkg-source.bbclass
>> new file mode 100644
>> index 00000000..434f1752
>> --- /dev/null
>> +++ b/meta/classes/dpkg-source.bbclass
>> @@ -0,0 +1,33 @@
>> +# This software is a part of ISAR.
>> +# Copyright (C) 2023 ilbers GmbH
>> +#
>> +# SPDX-License-Identifier: MIT
>> +
>> +inherit dpkg-base
>> +
>> +do_dpkg_source() {
>> +    # Create a .dsc file from source directory to use it with sbuild
>> +    DEB_SOURCE_NAME=$(dpkg-parsechangelog --show-field Source --file ${WORKDIR}/${PPS}/debian/changelog)
>> +    find ${WORKDIR} -name "${DEB_SOURCE_NAME}*.dsc" -maxdepth 1 -delete
>> +    if [ ! -z "${SOURCE_DATE_EPOCH}" ]; then
>> +        export SOURCE_DATE_EPOCH="${SOURCE_DATE_EPOCH}"
>> +    fi
>> +    sh -c "cd ${WORKDIR}; dpkg-source -I -b ${PPS}"
>> +}
>> +addtask dpkg_source after do_prepare_build before do_dpkg_build
>> +
>> +do_deploy_source[depends] += "isar-apt:do_cache_config"
>> +do_deploy_source[lockfiles] = "${REPO_ISAR_DIR}/isar.lock"
>> +do_deploy_source[dirs] = "${S}"
>> +do_deploy_source() {
>> +    repo_del_srcpackage "${REPO_ISAR_DIR}"/"${DISTRO}" \
>> +        "${REPO_ISAR_DB_DIR}"/"${DISTRO}" "${DEBDISTRONAME}" "${BPN}"
>> +    find "${S}/../" -name '*\.dsc' -maxdepth 1 | while read package; do
>> +        repo_add_srcpackage "${REPO_ISAR_DIR}"/"${DISTRO}" \
>> +            "${REPO_ISAR_DB_DIR}"/"${DISTRO}" \
>> +            "${DEBDISTRONAME}" \
>> +            "${package}"
>> +    done
>> +}
>> +addtask deploy_source after do_dpkg_source before do_dpkg_build
>> +
>> diff --git a/meta/classes/dpkg.bbclass b/meta/classes/dpkg.bbclass
>> index 90d717ca..2bb167e4 100644
>> --- a/meta/classes/dpkg.bbclass
>> +++ b/meta/classes/dpkg.bbclass
>> @@ -2,6 +2,7 @@
>>  # Copyright (C) 2015-2018 ilbers GmbH
>>  
>>  inherit dpkg-base
>> +inherit dpkg-source
>>  
>>  PACKAGE_ARCH ?= "${DISTRO_ARCH}"
>>  
>> @@ -95,10 +96,7 @@ dpkg_runbuild() {
>>  
>>      echo '$apt_keep_downloaded_packages = 1;' >> ${SBUILD_CONFIG}
>>  
>> -    # Create a .dsc file from source directory to use it with sbuild
>>      DEB_SOURCE_NAME=$(dpkg-parsechangelog --show-field Source --file ${WORKDIR}/${PPS}/debian/changelog)
>> -    find ${WORKDIR} -name "${DEB_SOURCE_NAME}*.dsc" -maxdepth 1 -delete
>> -    sh -c "cd ${WORKDIR}; dpkg-source -q -b ${PPS}"
>>      DSC_FILE=$(find ${WORKDIR} -name "${DEB_SOURCE_NAME}*.dsc" -maxdepth 1 -print)
>>  
>>      sbuild -A -n -c ${SBUILD_CHROOT} --extra-repository="${ISAR_APT_REPO}" \
>> diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass
>> index 7dec7b36..65574a6c 100644
>> --- a/meta/classes/rootfs.bbclass
>> +++ b/meta/classes/rootfs.bbclass
>> @@ -90,6 +90,8 @@ rootfs_configure_isar_apt() {
>>      mkdir -p '${ROOTFSDIR}/etc/apt/sources.list.d'
>>      echo 'deb [trusted=yes] file:///isar-apt ${DEBDISTRONAME} main' > \
>>          '${ROOTFSDIR}/etc/apt/sources.list.d/isar-apt.list'
>> +    echo 'deb-src [trusted=yes] file:///isar-apt ${DEBDISTRONAME} main' >> \
>> +        '${ROOTFSDIR}/etc/apt/sources.list.d/isar-apt.list'
>>  
>>      mkdir -p '${ROOTFSDIR}/etc/apt/preferences.d'
>>      cat << EOF > '${ROOTFSDIR}/etc/apt/preferences.d/isar-apt'
>> diff --git a/meta/recipes-kernel/linux/linux-custom.inc b/meta/recipes-kernel/linux/linux-custom.inc
>> index 8b367a2f..fa5d957d 100644
>> --- a/meta/recipes-kernel/linux/linux-custom.inc
>> +++ b/meta/recipes-kernel/linux/linux-custom.inc
>> @@ -212,6 +212,6 @@ EOF
>>  	fi
>>  }
>>  
>> -dpkg_runbuild:prepend() {
>> +do_dpkg_source:prepend() {
>>  	dpkg_configure_kernel
>>  }
> 
> This is not transparent to recipe as the very last hunk nicely
> demonstrates. Where is the RECIPE-API-CHANGLOG entry?
> 

OK, found it later on. But "in some cases" is not a helpful explanation.
I would have expected that all extensions of dpkg_runbuild:prepend
touched ${S} need to be moved, or so.

Jan

> I thought this change was already the reason, but I'm still searching
> for a regression of this patch here on our downstream recipe
> https://gitlab.com/cip-project/cip-core/isar-cip-core/-/tree/master/recipes-bsp/edk2
> (can be watched under
> https://github.com/siemens/meta-iot2050/actions/runs/6492616563/job/17631910317.
> Any hints welcome.
> 
> Jan
> 

-- 
Siemens AG, Technology
Linux Expert Center