* Re: [PATCH] add support for rebuilding essential packages
2023-02-20 10:32 [PATCH] add support for rebuilding essential packages Adriaan Schmidt
@ 2023-02-20 11:24 ` Jan Kiszka
2023-02-28 7:06 ` Cedric Hombourger
2023-03-06 6:06 ` Uladzimir Bely
2 siblings, 0 replies; 5+ messages in thread
From: Jan Kiszka @ 2023-02-20 11:24 UTC (permalink / raw)
To: Adriaan Schmidt, isar-users
On 20.02.23 11:32, Adriaan Schmidt wrote:
> There are cases when we need to rebuild a package that is installed during
> bootstrap (we call those "essential package" here). This patch introduces
> `ISAR_REBUILD_ESSENTIAL_PKGS` which can be set in (distro/layer) config, and
> lists all essential packages.
>
> During build, Isar ensures that essential packages are built before any others,
> so that their locally built versions are available in isar-apt, and will
> be used in any subsequent package builds.
>
> Signed-off-by: Adriaan Schmidt <adriaan.schmidt@siemens.com>
>
> ---
> This has interdependencies with the proposed multiarch feature,
> and builds on v2 of that series.
>
> Adriaan
> ---
> meta/classes/dpkg-base.bbclass | 1 +
> meta/classes/essential.bbclass | 39 ++++++++++++++++++++++++++++++++++
> meta/classes/image.bbclass | 1 +
> 3 files changed, 41 insertions(+)
> create mode 100644 meta/classes/essential.bbclass
>
> diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass
> index 55cc6655..ce301346 100644
> --- a/meta/classes/dpkg-base.bbclass
> +++ b/meta/classes/dpkg-base.bbclass
> @@ -10,6 +10,7 @@ inherit debianize
> inherit terminal
> inherit repository
> inherit deb-dl-dir
> +inherit essential
>
> DEPENDS ?= ""
> RPROVIDES ?= "${PROVIDES}"
> diff --git a/meta/classes/essential.bbclass b/meta/classes/essential.bbclass
> new file mode 100644
> index 00000000..cb444674
> --- /dev/null
> +++ b/meta/classes/essential.bbclass
> @@ -0,0 +1,39 @@
> +# This software is a part of ISAR.
> +# Copyright (C) 2023 Siemens AG
> +
> +ISAR_REBUILD_ESSENTIAL_PKGS ?= ""
> +
> +python() {
> + isar_rebuild_essential_pkgs = (d.getVar('ISAR_REBUILD_ESSENTIAL_PKGS', True) or '').split()
> + build_compat = d.getVar('ISAR_ENABLE_COMPAT_ARCH', True) == "1"
> + build_native = not d.getVar('DISTRO_ARCH', True) == d.getVar('HOST_ARCH')
> +
> + # construct list of essential packages that should be rebuilt:
> + # if we can't build compat, don't include any -compat packages
> + # if we don't need native (because DISTRO_ARCH == HOST_ARCH), don't build native
> + # otherwise, automatically include compat/native when we can build them
> + essential_packages = []
> + for p in isar_rebuild_essential_pkgs:
> + if p.endswith('-compat') and build_compat:
> + essential_packages.append(p)
> + elif p.endswith('-native') and build_native:
> + essential_packages.append(p)
> + else:
> + essential_packages.append(p)
> + if build_compat:
> + essential_packages.append(f'{p}-compat')
> + if build_native:
> + essential_packages.append(f'{p}-native')
> +
> + # bail out if this recipe is in the essential list
> + if d.getVar('PN') in essential_packages:
> + return
> +
> + # add dependencies to all packages from the essential list
> + for p in essential_packages:
> + if d.getVar('do_prepare_build'):
> + d.appendVarFlag('do_prepare_build', 'depends', f' {p}:do_deploy_deb')
> + if d.getVar('do_install_rootfs'):
> + d.appendVarFlag('do_install_rootfs', 'depends', f' {p}:do_deploy_deb')
> +}
> +
.git/rebase-apply/patch:69: new blank line at EOF.
> diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
> index ce7c549c..550785ea 100644
> --- a/meta/classes/image.bbclass
> +++ b/meta/classes/image.bbclass
> @@ -80,6 +80,7 @@ image_do_mounts() {
> }
>
> inherit multiarch
> +inherit essential
>
> ROOTFSDIR = "${IMAGE_ROOTFS}"
> ROOTFS_FEATURES += "clean-package-cache clean-pycache generate-manifest export-dpkg-status clean-log-files clean-debconf-cache"
Under test...
Jan
--
Siemens AG, Technology
Competence Center Embedded Linux
^ permalink raw reply [flat|nested] 5+ messages in thread* Re: [PATCH] add support for rebuilding essential packages
2023-02-20 10:32 [PATCH] add support for rebuilding essential packages Adriaan Schmidt
2023-02-20 11:24 ` Jan Kiszka
@ 2023-02-28 7:06 ` Cedric Hombourger
2023-02-28 7:19 ` Jan Kiszka
2023-03-06 6:06 ` Uladzimir Bely
2 siblings, 1 reply; 5+ messages in thread
From: Cedric Hombourger @ 2023-02-28 7:06 UTC (permalink / raw)
To: isar-users
[-- Attachment #1.1: Type: text/plain, Size: 3665 bytes --]
On Monday, February 20, 2023 at 11:32:22 AM UTC+1 Adriaan Schmidt wrote:
There are cases when we need to rebuild a package that is installed during
bootstrap (we call those "essential package" here). This patch introduces
`ISAR_REBUILD_ESSENTIAL_PKGS` which can be set in (distro/layer) config,
and
lists all essential packages.
During build, Isar ensures that essential packages are built before any
others,
so that their locally built versions are available in isar-apt, and will
be used in any subsequent package builds.
Signed-off-by: Adriaan Schmidt <adriaan...@siemens.com>
---
This has interdependencies with the proposed multiarch feature,
and builds on v2 of that series.
Adriaan
---
meta/classes/dpkg-base.bbclass | 1 +
meta/classes/essential.bbclass | 39 ++++++++++++++++++++++++++++++++++
meta/classes/image.bbclass | 1 +
3 files changed, 41 insertions(+)
create mode 100644 meta/classes/essential.bbclass
diff --git a/meta/classes/dpkg-base.bbclass
b/meta/classes/dpkg-base.bbclass
index 55cc6655..ce301346 100644
--- a/meta/classes/dpkg-base.bbclass
+++ b/meta/classes/dpkg-base.bbclass
@@ -10,6 +10,7 @@ inherit debianize
inherit terminal
inherit repository
inherit deb-dl-dir
+inherit essential
DEPENDS ?= ""
RPROVIDES ?= "${PROVIDES}"
diff --git a/meta/classes/essential.bbclass
b/meta/classes/essential.bbclass
new file mode 100644
index 00000000..cb444674
--- /dev/null
+++ b/meta/classes/essential.bbclass
@@ -0,0 +1,39 @@
+# This software is a part of ISAR.
+# Copyright (C) 2023 Siemens AG
+
+ISAR_REBUILD_ESSENTIAL_PKGS ?= ""
+
+python() {
+ isar_rebuild_essential_pkgs = (d.getVar('ISAR_REBUILD_ESSENTIAL_PKGS',
True) or '').split()
",True" should not be necessary
here's getVar()'s signature:
def getVar(self, var, expand=True, noweakdefault=False, parsing=False)
+ build_compat = d.getVar('ISAR_ENABLE_COMPAT_ARCH', True) == "1"
+ build_native = not d.getVar('DISTRO_ARCH', True) == d.getVar('HOST_ARCH')
+
+ # construct list of essential packages that should be rebuilt:
+ # if we can't build compat, don't include any -compat packages
+ # if we don't need native (because DISTRO_ARCH == HOST_ARCH), don't build
native
+ # otherwise, automatically include compat/native when we can build them
+ essential_packages = []
+ for p in isar_rebuild_essential_pkgs:
+ if p.endswith('-compat') and build_compat:
+ essential_packages.append(p)
+ elif p.endswith('-native') and build_native:
+ essential_packages.append(p)
+ else:
+ essential_packages.append(p)
+ if build_compat:
+ essential_packages.append(f'{p}-compat')
+ if build_native:
+ essential_packages.append(f'{p}-native')
+
+ # bail out if this recipe is in the essential list
+ if d.getVar('PN') in essential_packages:
+ return
+
+ # add dependencies to all packages from the essential list
+ for p in essential_packages:
+ if d.getVar('do_prepare_build'):
+ d.appendVarFlag('do_prepare_build', 'depends', f' {p}:do_deploy_deb')
+ if d.getVar('do_install_rootfs'):
+ d.appendVarFlag('do_install_rootfs', 'depends', f' {p}:do_deploy_deb')
+}
+
diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index ce7c549c..550785ea 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -80,6 +80,7 @@ image_do_mounts() {
}
inherit multiarch
+inherit essential
ROOTFSDIR = "${IMAGE_ROOTFS}"
ROOTFS_FEATURES += "clean-package-cache clean-pycache generate-manifest
export-dpkg-status clean-log-files clean-debconf-cache"
--
2.30.2
[-- Attachment #1.2: Type: text/html, Size: 4732 bytes --]
^ permalink raw reply [flat|nested] 5+ messages in thread* Re: [PATCH] add support for rebuilding essential packages
2023-02-28 7:06 ` Cedric Hombourger
@ 2023-02-28 7:19 ` Jan Kiszka
0 siblings, 0 replies; 5+ messages in thread
From: Jan Kiszka @ 2023-02-28 7:19 UTC (permalink / raw)
To: Cedric Hombourger, isar-users
On 28.02.23 08:06, Cedric Hombourger wrote:
>
>
> On Monday, February 20, 2023 at 11:32:22 AM UTC+1 Adriaan Schmidt wrote:
>
> There are cases when we need to rebuild a package that is installed
> during
> bootstrap (we call those "essential package" here). This patch
> introduces
> `ISAR_REBUILD_ESSENTIAL_PKGS` which can be set in (distro/layer)
> config, and
> lists all essential packages.
>
> During build, Isar ensures that essential packages are built before
> any others,
> so that their locally built versions are available in isar-apt, and
> will
> be used in any subsequent package builds.
>
> Signed-off-by: Adriaan Schmidt <adriaan...@siemens.com>
>
> ---
> This has interdependencies with the proposed multiarch feature,
> and builds on v2 of that series.
>
> Adriaan
> ---
> meta/classes/dpkg-base.bbclass | 1 +
> meta/classes/essential.bbclass | 39 ++++++++++++++++++++++++++++++++++
> meta/classes/image.bbclass | 1 +
> 3 files changed, 41 insertions(+)
> create mode 100644 meta/classes/essential.bbclass
>
> diff --git a/meta/classes/dpkg-base.bbclass
> b/meta/classes/dpkg-base.bbclass
> index 55cc6655..ce301346 100644
> --- a/meta/classes/dpkg-base.bbclass
> +++ b/meta/classes/dpkg-base.bbclass
> @@ -10,6 +10,7 @@ inherit debianize
> inherit terminal
> inherit repository
> inherit deb-dl-dir
> +inherit essential
>
> DEPENDS ?= ""
> RPROVIDES ?= "${PROVIDES}"
> diff --git a/meta/classes/essential.bbclass
> b/meta/classes/essential.bbclass
> new file mode 100644
> index 00000000..cb444674
> --- /dev/null
> +++ b/meta/classes/essential.bbclass
> @@ -0,0 +1,39 @@
> +# This software is a part of ISAR.
> +# Copyright (C) 2023 Siemens AG
> +
> +ISAR_REBUILD_ESSENTIAL_PKGS ?= ""
> +
> +python() {
> + isar_rebuild_essential_pkgs =
> (d.getVar('ISAR_REBUILD_ESSENTIAL_PKGS', True) or '').split()
>
>
> ",True" should not be necessary
> here's getVar()'s signature:
>
> def getVar(self, var, expand=True, noweakdefault=False, parsing=False)
>
Yep, we still see this pattern popping up.
I fact, we have 85(!) left-overs of this legacy pattern in Isar. Let's
we write a quick patch...
Jan
--
Siemens AG, Technology
Competence Center Embedded Linux
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH] add support for rebuilding essential packages
2023-02-20 10:32 [PATCH] add support for rebuilding essential packages Adriaan Schmidt
2023-02-20 11:24 ` Jan Kiszka
2023-02-28 7:06 ` Cedric Hombourger
@ 2023-03-06 6:06 ` Uladzimir Bely
2 siblings, 0 replies; 5+ messages in thread
From: Uladzimir Bely @ 2023-03-06 6:06 UTC (permalink / raw)
To: isar-users
In the email from Monday, 20 February 2023 13:32:14 +03 user Adriaan Schmidt wrote:
> There are cases when we need to rebuild a package that is installed during
> bootstrap (we call those "essential package" here). This patch introduces
> `ISAR_REBUILD_ESSENTIAL_PKGS` which can be set in (distro/layer) config, and
> lists all essential packages.
>
> During build, Isar ensures that essential packages are built before any others,
> so that their locally built versions are available in isar-apt, and will
> be used in any subsequent package builds.
>
> Signed-off-by: Adriaan Schmidt <adriaan.schmidt@siemens.com>
>
> ---
> This has interdependencies with the proposed multiarch feature,
> and builds on v2 of that series.
>
> Adriaan
> ---
> meta/classes/dpkg-base.bbclass | 1 +
> meta/classes/essential.bbclass | 39 ++++++++++++++++++++++++++++++++++
> meta/classes/image.bbclass | 1 +
> 3 files changed, 41 insertions(+)
> create mode 100644 meta/classes/essential.bbclass
>
> diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass
> index 55cc6655..ce301346 100644
> --- a/meta/classes/dpkg-base.bbclass
> +++ b/meta/classes/dpkg-base.bbclass
> @@ -10,6 +10,7 @@ inherit debianize
> inherit terminal
> inherit repository
> inherit deb-dl-dir
> +inherit essential
>
> DEPENDS ?= ""
> RPROVIDES ?= "${PROVIDES}"
> diff --git a/meta/classes/essential.bbclass b/meta/classes/essential.bbclass
> new file mode 100644
> index 00000000..cb444674
> --- /dev/null
> +++ b/meta/classes/essential.bbclass
> @@ -0,0 +1,39 @@
> +# This software is a part of ISAR.
> +# Copyright (C) 2023 Siemens AG
> +
> +ISAR_REBUILD_ESSENTIAL_PKGS ?= ""
> +
> +python() {
> + isar_rebuild_essential_pkgs = (d.getVar('ISAR_REBUILD_ESSENTIAL_PKGS', True) or '').split()
> + build_compat = d.getVar('ISAR_ENABLE_COMPAT_ARCH', True) == "1"
> + build_native = not d.getVar('DISTRO_ARCH', True) == d.getVar('HOST_ARCH')
> +
> + # construct list of essential packages that should be rebuilt:
> + # if we can't build compat, don't include any -compat packages
> + # if we don't need native (because DISTRO_ARCH == HOST_ARCH), don't build native
> + # otherwise, automatically include compat/native when we can build them
> + essential_packages = []
> + for p in isar_rebuild_essential_pkgs:
> + if p.endswith('-compat') and build_compat:
> + essential_packages.append(p)
> + elif p.endswith('-native') and build_native:
> + essential_packages.append(p)
> + else:
> + essential_packages.append(p)
> + if build_compat:
> + essential_packages.append(f'{p}-compat')
> + if build_native:
> + essential_packages.append(f'{p}-native')
> +
> + # bail out if this recipe is in the essential list
> + if d.getVar('PN') in essential_packages:
> + return
> +
> + # add dependencies to all packages from the essential list
> + for p in essential_packages:
> + if d.getVar('do_prepare_build'):
> + d.appendVarFlag('do_prepare_build', 'depends', f' {p}:do_deploy_deb')
> + if d.getVar('do_install_rootfs'):
> + d.appendVarFlag('do_install_rootfs', 'depends', f' {p}:do_deploy_deb')
> +}
> +
> diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
> index ce7c549c..550785ea 100644
> --- a/meta/classes/image.bbclass
> +++ b/meta/classes/image.bbclass
> @@ -80,6 +80,7 @@ image_do_mounts() {
> }
>
> inherit multiarch
> +inherit essential
>
> ROOTFSDIR = "${IMAGE_ROOTFS}"
> ROOTFS_FEATURES += "clean-package-cache clean-pycache generate-manifest export-dpkg-status clean-log-files clean-debconf-cache"
>
Applied (with removing redundant expand=True) to next, thanks.
^ permalink raw reply [flat|nested] 5+ messages in thread