* [PATCH] dpkg: Add support for building packages also for multi archs
@ 2023-01-11 20:05 Jan Kiszka
2023-01-16 11:50 ` Koch, Stefan
0 siblings, 1 reply; 3+ messages in thread
From: Jan Kiszka @ 2023-01-11 20:05 UTC (permalink / raw)
To: isar-users; +Cc: Koch, Stefan, Schmidt, Adriaan
From: Jan Kiszka <jan.kiszka@siemens.com>
This adds an interface to build packages not only for the target but
also for the build host and/or the compat arch. All it takes for a
recipe is to extend the new PACKAGE_ARCHS (note the plural) variable
with the desired architectures.
The convenience variables PACKAGE_HOST_ARCH and PACKAGE_COMPAT_ARCH are
introduced that fall back to empty strings if the cross-build or
compat-arch features are disabled.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
This was too easy to be done already. Even ssbuild seems to work fine
(Adriaan, please check as well).
I'm currently testing it very successfully to fix/improve our openssl
patching needs, but I think this can also help in the kbuild scenario.
The only thing missing for that is a nice idea how to define per-arch
DEB_BUILD_OPTIONS so that the kernel tool builds can be mapped on build
option, avoiding full builds for only getting those.
meta/classes/dpkg.bbclass | 34 +++++++++++++++++++++-------------
1 file changed, 21 insertions(+), 13 deletions(-)
diff --git a/meta/classes/dpkg.bbclass b/meta/classes/dpkg.bbclass
index 7822b14d..7a48aaa7 100644
--- a/meta/classes/dpkg.bbclass
+++ b/meta/classes/dpkg.bbclass
@@ -5,6 +5,12 @@ inherit dpkg-base
PACKAGE_ARCH ?= "${DISTRO_ARCH}"
+PACKAGE_ARCHS = "${PACKAGE_ARCH}"
+
+PACKAGE_HOST_ARCH = "${@d.getVar('HOST_ARCH') if d.getVar('ISAR_CROSS_COMPILE') == '1' else '' }"
+PACKAGE_COMPAT_ARCH = ""
+PACKAGE_COMPAT_ARCH_compat-arch = "${COMPAT_DISTRO_ARCH}"
+
DPKG_PREBUILD_ENV_FILE="${WORKDIR}/dpkg_prebuild.env"
# bitbake variables that should be passed into sbuild env
@@ -101,19 +107,21 @@ dpkg_runbuild() {
sh -c "cd ${WORKDIR}; dpkg-source -q -b ${PPS}"
DSC_FILE=$(find ${WORKDIR} -name "${DEB_SOURCE_NAME}*.dsc" -print)
- sbuild -A -n -c ${SBUILD_CHROOT} --extra-repository="${ISAR_APT_REPO}" \
- --host=${PACKAGE_ARCH} --build=${SBUILD_HOST_ARCH} ${profiles} \
- --no-run-lintian --no-run-piuparts --no-run-autopkgtest --resolve-alternatives \
- --no-apt-update \
- --chroot-setup-commands="echo \"Package: *\nPin: release n=${DEBDISTRONAME}\nPin-Priority: 1000\" > /etc/apt/preferences.d/isar-apt" \
- --chroot-setup-commands="echo \"APT::Get::allow-downgrades 1;\" > /etc/apt/apt.conf.d/50isar-apt" \
- --chroot-setup-commands="rm -f /var/log/dpkg.log" \
- --chroot-setup-commands="cp -n --no-preserve=owner ${ext_deb_dir}/*.deb -t ${deb_dir}/ || :" \
- --finished-build-commands="rm -f ${deb_dir}/sbuild-build-depends-main-dummy_*.deb" \
- --finished-build-commands="cp -n --no-preserve=owner ${deb_dir}/*.deb -t ${ext_deb_dir}/ || :" \
- --finished-build-commands="cp /var/log/dpkg.log ${ext_root}/dpkg_partial.log" \
- --debbuildopts="--source-option=-I" \
- --build-dir=${WORKDIR} --dist="isar" ${DSC_FILE}
+ for arch in ${PACKAGE_ARCHS}; do
+ sbuild -A -n -c ${SBUILD_CHROOT} --extra-repository="${ISAR_APT_REPO}" \
+ --host=$arch --build=${SBUILD_HOST_ARCH} ${profiles} \
+ --no-run-lintian --no-run-piuparts --no-run-autopkgtest --resolve-alternatives \
+ --no-apt-update \
+ --chroot-setup-commands="echo \"Package: *\nPin: release n=${DEBDISTRONAME}\nPin-Priority: 1000\" > /etc/apt/preferences.d/isar-apt" \
+ --chroot-setup-commands="echo \"APT::Get::allow-downgrades 1;\" > /etc/apt/apt.conf.d/50isar-apt" \
+ --chroot-setup-commands="rm -f /var/log/dpkg.log" \
+ --chroot-setup-commands="cp -n --no-preserve=owner ${ext_deb_dir}/*.deb -t ${deb_dir}/ || :" \
+ --finished-build-commands="rm -f ${deb_dir}/sbuild-build-depends-main-dummy_*.deb" \
+ --finished-build-commands="cp -n --no-preserve=owner ${deb_dir}/*.deb -t ${ext_deb_dir}/ || :" \
+ --finished-build-commands="cp /var/log/dpkg.log ${ext_root}/dpkg_partial.log" \
+ --debbuildopts="--source-option=-I" \
+ --build-dir=${WORKDIR} --dist="isar" ${DSC_FILE}
+ done
sbuild_dpkg_log_export "${WORKDIR}/rootfs/dpkg_partial.log"
deb_dl_dir_export "${WORKDIR}/rootfs" "${distro}"
--
2.35.3
^ permalink raw reply [flat|nested] 3+ messages in thread* Re: [PATCH] dpkg: Add support for building packages also for multi archs
2023-01-11 20:05 [PATCH] dpkg: Add support for building packages also for multi archs Jan Kiszka
@ 2023-01-16 11:50 ` Koch, Stefan
2023-01-16 13:09 ` Jan Kiszka
0 siblings, 1 reply; 3+ messages in thread
From: Koch, Stefan @ 2023-01-16 11:50 UTC (permalink / raw)
To: isar-users, Kiszka, Jan; +Cc: Schmidt, Adriaan
Sounds principally good. Thanks.
For the kbuild case I have submitted a similar patch:
[PATCH v2 3/5] dpkg: Add support for additional target and host builds
(Tue, 20 Dec 2022 18:09:19 +0100)
It modifies dpkg-base.bbclass instead of dpkg.bbclass
It uses the SBUILD_HOST variable for the same purpose as this patch
fulfills with the new PACKAGE_ARCH(S) variable. Migration from
SBUILD_HOST to PACKAGE_ARCHS should be possible. But I have not tested
this yet.
My patch will additionally set SCHROOT_DIR depending on
ISAR_CROSS_COMPILE to SCHROOT_TARGET_DIR or SCHROOT_HOST_DIR. That's
what I have not found inside this patch. Since my patch does it in the
dpkg-base.bbclass the schroot configuration is written before running
dpkg_runbuild with the correct SCHROOT_DIR. Doing this in dpkg.bbclass
would need a bit more code - it's not impossible I think.
Best regards
Stefan
On Wed, 2023-01-11 at 21:05 +0100, Jan Kiszka wrote:
> From: Jan Kiszka <jan.kiszka@siemens.com>
>
> This adds an interface to build packages not only for the target but
> also for the build host and/or the compat arch. All it takes for a
> recipe is to extend the new PACKAGE_ARCHS (note the plural) variable
> with the desired architectures.
>
> The convenience variables PACKAGE_HOST_ARCH and PACKAGE_COMPAT_ARCH
> are
> introduced that fall back to empty strings if the cross-build or
> compat-arch features are disabled.
>
> Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
> ---
>
> This was too easy to be done already. Even ssbuild seems to work fine
> (Adriaan, please check as well).
>
> I'm currently testing it very successfully to fix/improve our openssl
> patching needs, but I think this can also help in the kbuild
> scenario.
> The only thing missing for that is a nice idea how to define per-arch
> DEB_BUILD_OPTIONS so that the kernel tool builds can be mapped on
> build
> option, avoiding full builds for only getting those.
>
> meta/classes/dpkg.bbclass | 34 +++++++++++++++++++++-------------
> 1 file changed, 21 insertions(+), 13 deletions(-)
>
> diff --git a/meta/classes/dpkg.bbclass b/meta/classes/dpkg.bbclass
> index 7822b14d..7a48aaa7 100644
> --- a/meta/classes/dpkg.bbclass
> +++ b/meta/classes/dpkg.bbclass
> @@ -5,6 +5,12 @@ inherit dpkg-base
>
> PACKAGE_ARCH ?= "${DISTRO_ARCH}"
>
> +PACKAGE_ARCHS = "${PACKAGE_ARCH}"
> +
> +PACKAGE_HOST_ARCH = "${@d.getVar('HOST_ARCH') if
> d.getVar('ISAR_CROSS_COMPILE') == '1' else '' }"
> +PACKAGE_COMPAT_ARCH = ""
> +PACKAGE_COMPAT_ARCH_compat-arch = "${COMPAT_DISTRO_ARCH}"
> +
> DPKG_PREBUILD_ENV_FILE="${WORKDIR}/dpkg_prebuild.env"
>
> # bitbake variables that should be passed into sbuild env
> @@ -101,19 +107,21 @@ dpkg_runbuild() {
> sh -c "cd ${WORKDIR}; dpkg-source -q -b ${PPS}"
> DSC_FILE=$(find ${WORKDIR} -name "${DEB_SOURCE_NAME}*.dsc" -
> print)
>
> - sbuild -A -n -c ${SBUILD_CHROOT} --extra-
> repository="${ISAR_APT_REPO}" \
> - --host=${PACKAGE_ARCH} --build=${SBUILD_HOST_ARCH}
> ${profiles} \
> - --no-run-lintian --no-run-piuparts --no-run-autopkgtest --
> resolve-alternatives \
> - --no-apt-update \
> - --chroot-setup-commands="echo \"Package: *\nPin: release
> n=${DEBDISTRONAME}\nPin-Priority: 1000\" >
> /etc/apt/preferences.d/isar-apt" \
> - --chroot-setup-commands="echo \"APT::Get::allow-downgrades
> 1;\" > /etc/apt/apt.conf.d/50isar-apt" \
> - --chroot-setup-commands="rm -f /var/log/dpkg.log" \
> - --chroot-setup-commands="cp -n --no-preserve=owner
> ${ext_deb_dir}/*.deb -t ${deb_dir}/ || :" \
> - --finished-build-commands="rm -f ${deb_dir}/sbuild-build-
> depends-main-dummy_*.deb" \
> - --finished-build-commands="cp -n --no-preserve=owner
> ${deb_dir}/*.deb -t ${ext_deb_dir}/ || :" \
> - --finished-build-commands="cp /var/log/dpkg.log
> ${ext_root}/dpkg_partial.log" \
> - --debbuildopts="--source-option=-I" \
> - --build-dir=${WORKDIR} --dist="isar" ${DSC_FILE}
> + for arch in ${PACKAGE_ARCHS}; do
> + sbuild -A -n -c ${SBUILD_CHROOT} --extra-
> repository="${ISAR_APT_REPO}" \
> + --host=$arch --build=${SBUILD_HOST_ARCH} ${profiles} \
> + --no-run-lintian --no-run-piuparts --no-run-autopkgtest
> --resolve-alternatives \
> + --no-apt-update \
> + --chroot-setup-commands="echo \"Package: *\nPin: release
> n=${DEBDISTRONAME}\nPin-Priority: 1000\" >
> /etc/apt/preferences.d/isar-apt" \
> + --chroot-setup-commands="echo \"APT::Get::allow-
> downgrades 1;\" > /etc/apt/apt.conf.d/50isar-apt" \
> + --chroot-setup-commands="rm -f /var/log/dpkg.log" \
> + --chroot-setup-commands="cp -n --no-preserve=owner
> ${ext_deb_dir}/*.deb -t ${deb_dir}/ || :" \
> + --finished-build-commands="rm -f ${deb_dir}/sbuild-
> build-depends-main-dummy_*.deb" \
> + --finished-build-commands="cp -n --no-preserve=owner
> ${deb_dir}/*.deb -t ${ext_deb_dir}/ || :" \
> + --finished-build-commands="cp /var/log/dpkg.log
> ${ext_root}/dpkg_partial.log" \
> + --debbuildopts="--source-option=-I" \
> + --build-dir=${WORKDIR} --dist="isar" ${DSC_FILE}
> + done
>
> sbuild_dpkg_log_export "${WORKDIR}/rootfs/dpkg_partial.log"
> deb_dl_dir_export "${WORKDIR}/rootfs" "${distro}"
--
Stefan Koch
Siemens AG
www.siemens.com
^ permalink raw reply [flat|nested] 3+ messages in thread* Re: [PATCH] dpkg: Add support for building packages also for multi archs
2023-01-16 11:50 ` Koch, Stefan
@ 2023-01-16 13:09 ` Jan Kiszka
0 siblings, 0 replies; 3+ messages in thread
From: Jan Kiszka @ 2023-01-16 13:09 UTC (permalink / raw)
To: Koch, Stefan (DI PA DCP R&D 3), isar-users
Cc: Schmidt, Adriaan (T CED SES-DE)
On 16.01.23 12:50, Koch, Stefan (DI PA DCP R&D 3) wrote:
> Sounds principally good. Thanks.
>
> For the kbuild case I have submitted a similar patch:
> [PATCH v2 3/5] dpkg: Add support for additional target and host builds
> (Tue, 20 Dec 2022 18:09:19 +0100)
>
> It modifies dpkg-base.bbclass instead of dpkg.bbclass
>
> It uses the SBUILD_HOST variable for the same purpose as this patch
> fulfills with the new PACKAGE_ARCH(S) variable. Migration from
> SBUILD_HOST to PACKAGE_ARCHS should be possible. But I have not tested
> this yet.
>
> My patch will additionally set SCHROOT_DIR depending on
> ISAR_CROSS_COMPILE to SCHROOT_TARGET_DIR or SCHROOT_HOST_DIR. That's
> what I have not found inside this patch.
It's not needed in my model because schroot variations collapse if you
don't need them (host=target), and the host variant can not only build
for the target but naturally also the host - thus SCHROOT_HOST is
universal in practice.
> Since my patch does it in the
> dpkg-base.bbclass the schroot configuration is written before running
> dpkg_runbuild with the correct SCHROOT_DIR. Doing this in dpkg.bbclass
> would need a bit more code - it's not impossible I think.
There is in fact a 3rd proposal that Adriaan explained to me on Friday
(not sure if he shared it on the list before), and that would model the
variation via different (auto-generated) targets (mytarget,
mytarget-native, mytarget-compat). It has some critical point regarding
how to detect non-identical rebuilt arch-all and source packages, but
would otherwise be even more elegant. Adriaan wants to follow-up on that
soon.
Jan
--
Siemens AG, Technology
Competence Center Embedded Linux
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2023-01-16 13:09 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-01-11 20:05 [PATCH] dpkg: Add support for building packages also for multi archs Jan Kiszka
2023-01-16 11:50 ` Koch, Stefan
2023-01-16 13:09 ` Jan Kiszka
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox