Re: [PATCH v1] recipes-devtools/sbuild-chroot: install self-built packages in sbuild flavor

[Srinuvasan Arjunan <srinuvasanasv@gmail.com>]

[-- Attachment #1.1: Type: text/plain, Size: 14826 bytes --]



On Wednesday, January 3, 2024 at 4:06:20 PM UTC+5:30 Jan Kiszka wrote:

On 03.01.24 08:56, Srinuvasan Arjunan wrote: 
> 
> 
> On Wednesday, January 3, 2024 at 10:47:54 AM UTC+5:30 Jan Kiszka wrote: 
> 
> On 03.01.24 05:47, Srinuvasan Arjunan wrote: 
> > 
> > 
> > On Friday, December 15, 2023 at 5:15:50 PM UTC+5:30 Srinuvasan 
> Arjunan 
> > wrote: 
> > 
> > From: srinuvasan <srinuv...@siemens.com> 
> > 
> > In some use-cases we need to install custom packages in sbuildchroot 
> > and 
> > this can be referred later time during image generation part, 
> hence we 
> > additionally added the provision to install the custom packages in 
> > sbuild flavor. 
> > 
> > Introduced SBUILD_CHROOT_INSTALL variable, here you can define the 
> > custom packages that you need to install in sbuild flavor. 
> > 
> > Signed-off-by: srinuvasan <srinuv...@siemens.com> 
> > --- 
> > .../recipes-app/hello-isar/hello-isar.bb <http://hello-isar.bb> 
> <http://hello-isar.bb <http://hello-isar.bb>> | 2 +- 
> > meta-isar/recipes-app/libhello/libhello.bb <http://libhello.bb> 
> <http://libhello.bb <http://libhello.bb>> | 2 +- 
> > .../recipes-core/images/isar-image-base.bb 
> <http://isar-image-base.bb> 
> > <http://isar-image-base.bb <http://isar-image-base.bb>> | 3 + 
> > ...b => sbuild-chroot-host-extra-packages.bb 
> <http://sbuild-chroot-host-extra-packages.bb> 
> > <http://sbuild-chroot-host-extra-packages.bb 
> <http://sbuild-chroot-host-extra-packages.bb>>} | 2 +- 
> > ...=> sbuild-chroot-target-extra-packages.bb 
> <http://sbuild-chroot-target-extra-packages.bb> 
> > <http://sbuild-chroot-target-extra-packages.bb 
> <http://sbuild-chroot-target-extra-packages.bb>>} | 2 +- 
> > meta/classes/image.bbclass | 1 + 
> > ...tall-custom-packages-sbuild-flavor.bbclass | 64 
> +++++++++++++++++++ 
> > 7 files changed, 72 insertions(+), 4 deletions(-) 
> > rename 
> > 
> meta-isar/recipes-devtools/sbuild-chroot/{sbuild-chroot-host-db2m.bb 
> <http://sbuild-chroot-host-db2m.bb> 
> > <http://sbuild-chroot-host-db2m.bb 
> <http://sbuild-chroot-host-db2m.bb>> => 
> > sbuild-chroot-host-extra-packages.bb 
> <http://sbuild-chroot-host-extra-packages.bb> 
> > <http://sbuild-chroot-host-extra-packages.bb 
> <http://sbuild-chroot-host-extra-packages.bb>>} (91%) 
> > rename 
> > 
> meta-isar/recipes-devtools/sbuild-chroot/{sbuild-chroot-target-db2m.bb <
http://sbuild-chroot-target-db2m.bb> <http://sbuild-chroot-target-db2m.bb <
http://sbuild-chroot-target-db2m.bb>> => 
sbuild-chroot-target-extra-packages.bb <
http://sbuild-chroot-target-extra-packages.bb> <
http://sbuild-chroot-target-extra-packages.bb <
http://sbuild-chroot-target-extra-packages.bb>>} (91%) 
> > create mode 100644 
> > meta/classes/install-custom-packages-sbuild-flavor.bbclass 
> > 
> > diff --git a/meta-isar/recipes-app/hello-isar/hello-isar.bb 
> <http://hello-isar.bb> 
> > <http://hello-isar.bb <http://hello-isar.bb>> 
> > b/meta-isar/recipes-app/hello-isar/hello-isar.bb 
> <http://hello-isar.bb> <http://hello-isar.bb <http://hello-isar.bb>> 
> > index 51dac2b7..484e5704 100644 
> > --- a/meta-isar/recipes-app/hello-isar/hello-isar.bb 
> <http://hello-isar.bb> 
> > <http://hello-isar.bb <http://hello-isar.bb>> 
> > +++ b/meta-isar/recipes-app/hello-isar/hello-isar.bb 
> <http://hello-isar.bb> 
> > <http://hello-isar.bb <http://hello-isar.bb>> 
> > @@ -23,4 +23,4 @@ SRCREV = "a18c14cc11ce6b003f3469e89223cffb4016861d" 
> > inherit dpkg 
> > 
> > # Example of using alternative sbuild chroot 
> > -SBUILD_FLAVOR="db2m" 
> > +SBUILD_FLAVOR="extra-packages" 
> > diff --git a/meta-isar/recipes-app/libhello/libhello.bb 
> <http://libhello.bb> 
> > <http://libhello.bb <http://libhello.bb>> 
> b/meta-isar/recipes-app/libhello/libhello.bb <http://libhello.bb> 
> > <http://libhello.bb <http://libhello.bb>> 
> > index 71604bf5..4e6522fe 100644 
> > --- a/meta-isar/recipes-app/libhello/libhello.bb 
> <http://libhello.bb> <http://libhello.bb <http://libhello.bb>> 
> > +++ b/meta-isar/recipes-app/libhello/libhello.bb 
> <http://libhello.bb> <http://libhello.bb <http://libhello.bb>> 
> > @@ -16,4 +16,4 @@ SRCREV = "98f2e41e7d05ab8d19b0c5d160b104b725c8fd93" 
> > inherit dpkg 
> > 
> > # Example of using alternative sbuild chroot 
> > -SBUILD_FLAVOR="db2m" 
> > +SBUILD_FLAVOR="extra-packages" 
> > diff --git a/meta-isar/recipes-core/images/isar-image-base.bb 
> <http://isar-image-base.bb> 
> > <http://isar-image-base.bb <http://isar-image-base.bb>> 
> > b/meta-isar/recipes-core/images/isar-image-base.bb 
> <http://isar-image-base.bb> 
> > <http://isar-image-base.bb <http://isar-image-base.bb>> 
> > index b381d859..d635a3aa 100644 
> > --- a/meta-isar/recipes-core/images/isar-image-base.bb 
> <http://isar-image-base.bb> 
> > <http://isar-image-base.bb <http://isar-image-base.bb>> 
> > +++ b/meta-isar/recipes-core/images/isar-image-base.bb 
> <http://isar-image-base.bb> 
> > <http://isar-image-base.bb <http://isar-image-base.bb>> 
> > @@ -10,4 +10,7 @@ LIC_FILES_CHKSUM = 
> > "file://${LAYERDIR_core}/licenses/COPYING.GPLv2;md5=751419260 
> > 
> > PV = "1.0" 
> > 
> > +SBUILD_FLAVOR = "extra-packages" 
> > +SBUILD_CHROOT_INSTALL = "hello-isar" 
> > + 
> > inherit image 
> > diff --git 
> > 
> a/meta-isar/recipes-devtools/sbuild-chroot/sbuild-chroot-host-db2m.bb <
http://sbuild-chroot-host-db2m.bb> <http://sbuild-chroot-host-db2m.bb <
http://sbuild-chroot-host-db2m.bb>> 
b/meta-isar/recipes-devtools/sbuild-chroot/
sbuild-chroot-host-extra-packages.bb <
http://sbuild-chroot-host-extra-packages.bb> <
http://sbuild-chroot-host-extra-packages.bb <
http://sbuild-chroot-host-extra-packages.bb>> 
> > similarity index 91% 
> > rename from 
> > 
> meta-isar/recipes-devtools/sbuild-chroot/sbuild-chroot-host-db2m.bb 
> <http://sbuild-chroot-host-db2m.bb> 
> > <http://sbuild-chroot-host-db2m.bb 
> <http://sbuild-chroot-host-db2m.bb>> 
> > rename to 
> > 
> meta-isar/recipes-devtools/sbuild-chroot/
sbuild-chroot-host-extra-packages.bb <
http://sbuild-chroot-host-extra-packages.bb> <
http://sbuild-chroot-host-extra-packages.bb <
http://sbuild-chroot-host-extra-packages.bb>> 
> > index 1795c5a2..b035a926 100644 
> > --- 
> > 
> a/meta-isar/recipes-devtools/sbuild-chroot/sbuild-chroot-host-db2m.bb <
http://sbuild-chroot-host-db2m.bb> <http://sbuild-chroot-host-db2m.bb <
http://sbuild-chroot-host-db2m.bb>> 
> > +++ 
> > 
> b/meta-isar/recipes-devtools/sbuild-chroot/
sbuild-chroot-host-extra-packages.bb <
http://sbuild-chroot-host-extra-packages.bb> <
http://sbuild-chroot-host-extra-packages.bb <
http://sbuild-chroot-host-extra-packages.bb>> 
> > @@ -8,5 +8,5 @@ DESCRIPTION = "Isar sbuild/schroot filesystem for 
> > host (docbook-to-man)" 
> > 
> > require recipes-devtools/sbuild-chroot/sbuild-chroot-host.bb 
> <http://sbuild-chroot-host.bb> 
> > <http://sbuild-chroot-host.bb <http://sbuild-chroot-host.bb>> 
> > 
> > -SBUILD_FLAVOR = "db2m" 
> > +SBUILD_FLAVOR = "extra-packages" 
> > SBUILD_CHROOT_PREINSTALL_EXTRA ?= "docbook-to-man" 
> > diff --git 
> > 
> a/meta-isar/recipes-devtools/sbuild-chroot/sbuild-chroot-target-db2m.bb <
http://sbuild-chroot-target-db2m.bb> <http://sbuild-chroot-target-db2m.bb <
http://sbuild-chroot-target-db2m.bb>> 
b/meta-isar/recipes-devtools/sbuild-chroot/
sbuild-chroot-target-extra-packages.bb <
http://sbuild-chroot-target-extra-packages.bb> <
http://sbuild-chroot-target-extra-packages.bb <
http://sbuild-chroot-target-extra-packages.bb>> 
> > similarity index 91% 
> > rename from 
> > 
> meta-isar/recipes-devtools/sbuild-chroot/sbuild-chroot-target-db2m.bb <
http://sbuild-chroot-target-db2m.bb> <http://sbuild-chroot-target-db2m.bb <
http://sbuild-chroot-target-db2m.bb>> 
> > rename to 
> > 
> meta-isar/recipes-devtools/sbuild-chroot/
sbuild-chroot-target-extra-packages.bb <
http://sbuild-chroot-target-extra-packages.bb> <
http://sbuild-chroot-target-extra-packages.bb <
http://sbuild-chroot-target-extra-packages.bb>> 
> > index e2e10433..80440df2 100644 
> > --- 
> > 
> a/meta-isar/recipes-devtools/sbuild-chroot/sbuild-chroot-target-db2m.bb <
http://sbuild-chroot-target-db2m.bb> <http://sbuild-chroot-target-db2m.bb <
http://sbuild-chroot-target-db2m.bb>> 
> > +++ 
> > 
> b/meta-isar/recipes-devtools/sbuild-chroot/
sbuild-chroot-target-extra-packages.bb <
http://sbuild-chroot-target-extra-packages.bb> <
http://sbuild-chroot-target-extra-packages.bb <
http://sbuild-chroot-target-extra-packages.bb>> 
> > @@ -8,5 +8,5 @@ DESCRIPTION = "Isar sbuild/schroot filesystem for 
> > target (docbook-to-man)" 
> > 
> > require recipes-devtools/sbuild-chroot/sbuild-chroot-target.bb 
> <http://sbuild-chroot-target.bb> 
> > <http://sbuild-chroot-target.bb <http://sbuild-chroot-target.bb>> 
> > 
> > -SBUILD_FLAVOR = "db2m" 
> > +SBUILD_FLAVOR = "extra-packages" 
> > SBUILD_CHROOT_PREINSTALL_EXTRA ?= "docbook-to-man" 
> > diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass 
> > index 73f1d52c..5fea18b0 100644 
> > --- a/meta/classes/image.bbclass 
> > +++ b/meta/classes/image.bbclass 
> > @@ -85,6 +85,7 @@ inherit image-tools-extension 
> > inherit image-postproc-extension 
> > inherit image-locales-extension 
> > inherit image-account-extension 
> > +inherit install-custom-packages-sbuild-flavor 
> > 
> > # Extra space for rootfs in MB 
> > ROOTFS_EXTRA ?= "64" 
> > diff --git 
> > a/meta/classes/install-custom-packages-sbuild-flavor.bbclass 
> > b/meta/classes/install-custom-packages-sbuild-flavor.bbclass 
> > new file mode 100644 
> > index 00000000..04650f51 
> > --- /dev/null 
> > +++ b/meta/classes/install-custom-packages-sbuild-flavor.bbclass 
> > @@ -0,0 +1,64 @@ 
> > +# This software is a part of ISAR. 
> > +# Copyright (C) Siemens AG, 2023 
> > +# 
> > +# SPDX-License-Identifier: MIT 
> > +# 
> > +# This file extends the image.bbclass to supply tools for futher 
> > imager functions 
> > + 
> > +inherit sbuild 
> > + 
> > +SBUILD_FLAVOR = "extra-packages" 
> > +SBUILD_CHROOT_INSTALL ??= "" 
> > + 
> > +SCHROOT_MOUNTS = "${WORKDIR}:${PP_WORK} 
> > ${IMAGE_ROOTFS}:${PP_ROOTFS} ${DEPLOY_DIR_IMAGE}:${PP_DEPLOY}" 
> > +SCHROOT_MOUNTS += "${REPO_ISAR_DIR}/${DISTRO}:/isar-apt" 
> > + 
> > +ROOTFS_INSTALL_COMMAND:prepend = " 
> > install_custom_packages_sbuild_flavor" 
> > + 
> > +install_custom_packages_sbuild_flavor() { 
> > + 
> > + schroot_create_configs 
> > + insert_mounts 
> > + 
> > + if [ -n "${SBUILD_CHROOT_INSTALL}" ]; then 
> > + echo "Installing self built packages: ${SBUILD_CHROOT_INSTALL}" 
> > + 
> > + distro="${BASE_DISTRO}-${BASE_DISTRO_CODENAME}" 
> > + if [ ${ISAR_CROSS_COMPILE} -eq 1 ]; then 
> > + distro="${HOST_BASE_DISTRO}-${BASE_DISTRO_CODENAME}" 
> > + fi 
> > + 
> > + # prepare isar-apt 
> > + sudo -E chroot ${SCHROOT_DIR} sh -c " \ 
> > + mkdir -p '/etc/apt/sources.list.d' 
> > + echo 'deb [trusted=yes] file:///isar-apt ${DEBDISTRONAME} main' > \ 
> > + '/etc/apt/sources.list.d/isar-apt.list' 
> > + 
> > + mkdir -p '/etc/apt/preferences.d' 
> > + cat << EOF > '/etc/apt/preferences.d/isar-apt' 
> > +Package: ${SBUILD_CHROOT_INSTALL} 
> > +Pin: release n=${DEBDISTRONAME} 
> > +Pin-Priority: 1000 
> > +EOF" 
> > + 
> > + E="${@ isar_export_proxies(d)}" 
> > + deb_dl_dir_import ${schroot_dir} ${distro} 
> > + sudo -E chroot ${SCHROOT_DIR} sh -c " \ 
> > + apt-get update \ 
> > + -o Dir::Etc::SourceList='sources.list.d/isar-apt.list' \ 
> > + -o Dir::Etc::SourceParts='-' \ 
> > + -o APT::Get::List-Cleanup='0' 
> > + apt-get -o Debug::pkgProblemResolver=yes --no-install-recommends 
> -y \ 
> > + --allow-unauthenticated --allow-downgrades --download-only 
> install \ 
> > + ${SBUILD_CHROOT_INSTALL}" 
> > + 
> > + deb_dl_dir_export ${schroot_dir} ${distro} 
> > + sudo -E chroot ${SCHROOT_DIR} sh -c " \ 
> > + apt-get -o Debug::pkgProblemResolver=yes --no-install-recommends 
> -y \ 
> > + --allow-unauthenticated --allow-downgrades install \ 
> > + ${SBUILD_CHROOT_INSTALL}" 
> > + fi 
> > + 
> > + remove_mounts 
> > + schroot_delete_configs 
> > +} 
> > -- 
> > 2.39.2 
> > 
> > 
> > 
> >     Any improvements needed in this patch , Please provide your 
> thoughts. 
> > 
> 
> I still don't get all the complexity here, specificially as I just did 
> what you did in [1] with just a DEPENDS. 
> 
> Jan 
> 
> [1] 
> 
https://groups.google.com/d/msgid/isar-users/c1e5d483ee4fae79645d69ac95258a622c9b5a7d.1704190109.git.jan.kiszka%40siemens.com 
<
https://groups.google.com/d/msgid/isar-users/c1e5d483ee4fae79645d69ac95258a622c9b5a7d.1704190109.git.jan.kiszka%40siemens.com> 

> 
> 
> 
>    Hmm, Let me try with DEPENDS , but you added the dependencies via 
> DEPENDS in base sbuild-chroot [target/host], if i add all my 
> dependencies here it may increases the size of bare-metal sbuild-chroot 
> , i will check this DEPENDS for sbuild flavor too. 
> 

That pattern is not for direct copying. But, as discussed, if you add 
SBUILD_CHROOT_INSTALL_EXTRA which creates the necessary DEPENDS for the 
sbuild flavor recipe, it should Just Work from my perspective. 

Jan 

-- 
Siemens AG, Technology 
Linux Expert Center



      As per your changes in [1] 
https://groups.google.com/d/msgid/isar-users/c1e5d483ee4fae79645d69ac95258a622c9b5a7d.1704190109.git.jan.kiszka%40siemens.com
 <
https://groups.google.com/d/msgid/isar-users/c1e5d483ee4fae79645d69ac95258a622c9b5a7d.1704190109.git.jan.kiszka%40siemens.com
>
how this DEPENDS is working?, you added keyutils custom package as a 
dependency in sbuild-chroot recipe, but for building this custom keyutils
packages we should need sbuild-chroot-[host/target], then obviously we may 
face dependency loop issue.

In my case first i need to build all the custom packages first, then based 
on the need basis i may install any custom packages in sbuild-chroot flavor 
[ the persistent one] during image creation part.

what i implemented so far:

List out the custom packages that need to be install in sbuild-chroot via 
SBUILD_CHROOT_INSTALL_EXTRA or SBUILD_CHROOT_INSTALL analogously 
IMAGE_[PRE]INSTALL in the image recipe, for this purpose i added  the 
provision to access the isar-apt to install local packages in sbuild-chroot.

Presently isar-apt mount logic available in two places 1. during package 
build via SBUILD (dpkg.bbclass) and 2. Imager creation part (f.e : wic )

But my requirement is something need to install the custom packages in 
sbuildchroot to do some postprocess the meta data before image creation, 
for this reason we introduced another isar-apt mount logic in separate 
class.


Thanks,
Srinu

[-- Attachment #1.2: Type: text/html, Size: 31410 bytes --]