Re: [PATCH 3/5] meta: Implement two stage build

[Jan Kiszka <jan.kiszka@siemens.com>]

On 12.10.23 11:07, Anton Mikanovich wrote:
> 12/10/2023 11:48, Jan Kiszka wrote:
>> On 15.09.23 20:54, Anton Mikanovich wrote:
>>> Split package build into two stages: building source package and
>>> building binary package. This allows to obtain Debian package
>>> dependencies before any build steps will actually start.
>>>
>>> Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
>>> ---
>>>   meta/classes/dpkg-gbp.bbclass              |  2 +-
>>>   meta/classes/dpkg-source.bbclass           | 33 ++++++++++++++++++++++
>>>   meta/classes/dpkg.bbclass                  |  4 +--
>>>   meta/classes/rootfs.bbclass                |  2 ++
>>>   meta/recipes-kernel/linux/linux-custom.inc |  2 +-
>>>   5 files changed, 38 insertions(+), 5 deletions(-)
>>>   create mode 100644 meta/classes/dpkg-source.bbclass
>>>
>>> diff --git a/meta/classes/dpkg-gbp.bbclass
>>> b/meta/classes/dpkg-gbp.bbclass
>>> index 2bd09eaa..a151ade2 100644
>>> --- a/meta/classes/dpkg-gbp.bbclass
>>> +++ b/meta/classes/dpkg-gbp.bbclass
>>> @@ -11,7 +11,7 @@ PATCHTOOL ?= "git"
>>>     GBP_EXTRA_OPTIONS ?= "--git-pristine-tar"
>>>   -dpkg_runbuild:prepend() {
>>> +do_dpkg_source:prepend() {
>>>       sh -c "
>>>           cd ${WORKDIR}/${PPS}
>>>           gbp buildpackage --git-ignore-new --git-builder=/bin/true
>>> ${GBP_EXTRA_OPTIONS}
>>> diff --git a/meta/classes/dpkg-source.bbclass
>>> b/meta/classes/dpkg-source.bbclass
>>> new file mode 100644
>>> index 00000000..434f1752
>>> --- /dev/null
>>> +++ b/meta/classes/dpkg-source.bbclass
>>> @@ -0,0 +1,33 @@
>>> +# This software is a part of ISAR.
>>> +# Copyright (C) 2023 ilbers GmbH
>>> +#
>>> +# SPDX-License-Identifier: MIT
>>> +
>>> +inherit dpkg-base
>>> +
>>> +do_dpkg_source() {
>>> +    # Create a .dsc file from source directory to use it with sbuild
>>> +    DEB_SOURCE_NAME=$(dpkg-parsechangelog --show-field Source --file
>>> ${WORKDIR}/${PPS}/debian/changelog)
>>> +    find ${WORKDIR} -name "${DEB_SOURCE_NAME}*.dsc" -maxdepth 1 -delete
>>> +    if [ ! -z "${SOURCE_DATE_EPOCH}" ]; then
>>> +        export SOURCE_DATE_EPOCH="${SOURCE_DATE_EPOCH}"
>>> +    fi
>>> +    sh -c "cd ${WORKDIR}; dpkg-source -I -b ${PPS}"
>>> +}
>>> +addtask dpkg_source after do_prepare_build before do_dpkg_build
>>> +
>>> +do_deploy_source[depends] += "isar-apt:do_cache_config"
>>> +do_deploy_source[lockfiles] = "${REPO_ISAR_DIR}/isar.lock"
>>> +do_deploy_source[dirs] = "${S}"
>>> +do_deploy_source() {
>>> +    repo_del_srcpackage "${REPO_ISAR_DIR}"/"${DISTRO}" \
>>> +        "${REPO_ISAR_DB_DIR}"/"${DISTRO}" "${DEBDISTRONAME}" "${BPN}"
>>> +    find "${S}/../" -name '*\.dsc' -maxdepth 1 | while read package; do
>>> +        repo_add_srcpackage "${REPO_ISAR_DIR}"/"${DISTRO}" \
>>> +            "${REPO_ISAR_DB_DIR}"/"${DISTRO}" \
>>> +            "${DEBDISTRONAME}" \
>>> +            "${package}"
>>> +    done
>>> +}
>>> +addtask deploy_source after do_dpkg_source before do_dpkg_build
>>> +
>>> diff --git a/meta/classes/dpkg.bbclass b/meta/classes/dpkg.bbclass
>>> index 90d717ca..2bb167e4 100644
>>> --- a/meta/classes/dpkg.bbclass
>>> +++ b/meta/classes/dpkg.bbclass
>>> @@ -2,6 +2,7 @@
>>>   # Copyright (C) 2015-2018 ilbers GmbH
>>>     inherit dpkg-base
>>> +inherit dpkg-source
>>>     PACKAGE_ARCH ?= "${DISTRO_ARCH}"
>>>   @@ -95,10 +96,7 @@ dpkg_runbuild() {
>>>         echo '$apt_keep_downloaded_packages = 1;' >> ${SBUILD_CONFIG}
>>>   -    # Create a .dsc file from source directory to use it with sbuild
>>>       DEB_SOURCE_NAME=$(dpkg-parsechangelog --show-field Source
>>> --file ${WORKDIR}/${PPS}/debian/changelog)
>>> -    find ${WORKDIR} -name "${DEB_SOURCE_NAME}*.dsc" -maxdepth 1 -delete
>>> -    sh -c "cd ${WORKDIR}; dpkg-source -q -b ${PPS}"
>>>       DSC_FILE=$(find ${WORKDIR} -name "${DEB_SOURCE_NAME}*.dsc"
>>> -maxdepth 1 -print)
>>>         sbuild -A -n -c ${SBUILD_CHROOT}
>>> --extra-repository="${ISAR_APT_REPO}" \
>>> diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass
>>> index 7dec7b36..65574a6c 100644
>>> --- a/meta/classes/rootfs.bbclass
>>> +++ b/meta/classes/rootfs.bbclass
>>> @@ -90,6 +90,8 @@ rootfs_configure_isar_apt() {
>>>       mkdir -p '${ROOTFSDIR}/etc/apt/sources.list.d'
>>>       echo 'deb [trusted=yes] file:///isar-apt ${DEBDISTRONAME} main'
>>> > \
>>>           '${ROOTFSDIR}/etc/apt/sources.list.d/isar-apt.list'
>>> +    echo 'deb-src [trusted=yes] file:///isar-apt ${DEBDISTRONAME}
>>> main' >> \
>>> +        '${ROOTFSDIR}/etc/apt/sources.list.d/isar-apt.list'
>>>         mkdir -p '${ROOTFSDIR}/etc/apt/preferences.d'
>>>       cat << EOF > '${ROOTFSDIR}/etc/apt/preferences.d/isar-apt'
>>> diff --git a/meta/recipes-kernel/linux/linux-custom.inc
>>> b/meta/recipes-kernel/linux/linux-custom.inc
>>> index 8b367a2f..fa5d957d 100644
>>> --- a/meta/recipes-kernel/linux/linux-custom.inc
>>> +++ b/meta/recipes-kernel/linux/linux-custom.inc
>>> @@ -212,6 +212,6 @@ EOF
>>>       fi
>>>   }
>>>   -dpkg_runbuild:prepend() {
>>> +do_dpkg_source:prepend() {
>>>       dpkg_configure_kernel
>>>   }
>> This is not transparent to recipe as the very last hunk nicely
>> demonstrates. Where is the RECIPE-API-CHANGLOG entry?
> In p5.
>> I thought this change was already the reason, but I'm still searching
>> for a regression of this patch here on our downstream recipe
>> https://gitlab.com/cip-project/cip-core/isar-cip-core/-/tree/master/recipes-bsp/edk2
>> (can be watched under
>> https://github.com/siemens/meta-iot2050/actions/runs/6492616563/job/17631910317.
> This change shouldn't have any influence on the linker for such a
> trivial recipe.
> Are you sure this patchset is the only change between working and
> non-working hashes?

I found the reason: You added -I to dpkg-source, and that filters out
some questionable but needed prebuilt .a files in edk2. All fine again
when dropping it. I'll send a patch, we should not do that here.

Jan

-- 
Siemens AG, Technology
Linux Expert Center