* [RFC 0/5] Debian dependencies investigation
@ 2022-02-25 7:40 Anton Mikanovich
2022-02-25 7:40 ` [RFC 1/5] dpkg-base: Cover do_apt_unpack task by lock Anton Mikanovich
` (4 more replies)
0 siblings, 5 replies; 7+ messages in thread
From: Anton Mikanovich @ 2022-02-25 7:40 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
Isar do not jet use Build-Depends field from control files to call
recipes building. DEPENDS field should be used instead in every single
case.
We can use additional source package management to fix that: build
source packages at first stage, obtain dependencies on second and build
final target on third.
It is not something to be used as-is, but just proof of concept to show
possible way for implementation. Proposed logic can't be implemented by
bitbake currently because all the dependencies are calculated before the
build, so we should try to work with upstream to find integration
possibilities.
Anton Mikanovich (5):
dpkg-base: Cover do_apt_unpack task by lock
meta: Implement two stage build
libhello: Declare provided packages
hello-isar: Remove duplicated dependency
isar: Add external builder
isar_builder.py | 142 ++++++++++++++++++
.../recipes-app/hello-isar/hello-isar.bb | 4 -
meta-isar/recipes-app/libhello/libhello.bb | 2 +
meta/classes/dpkg-base.bbclass | 36 ++++-
meta/classes/dpkg-gbp.bbclass | 6 +-
meta/classes/dpkg-prebuilt.bbclass | 7 +
meta/classes/dpkg.bbclass | 21 ++-
meta/classes/rootfs.bbclass | 2 +
.../buildchroot/buildchroot.inc | 2 +
.../buildchroot/files/build-source.sh | 16 ++
.../buildchroot/files/build.sh | 2 +-
11 files changed, 222 insertions(+), 18 deletions(-)
create mode 100755 isar_builder.py
create mode 100644 meta/recipes-devtools/buildchroot/files/build-source.sh
--
2.25.1
^ permalink raw reply [flat|nested] 7+ messages in thread
* [RFC 1/5] dpkg-base: Cover do_apt_unpack task by lock
2022-02-25 7:40 [RFC 0/5] Debian dependencies investigation Anton Mikanovich
@ 2022-02-25 7:40 ` Anton Mikanovich
2022-02-25 17:30 ` Jan Kiszka
2022-02-25 7:40 ` [RFC 2/5] meta: Implement two stage build Anton Mikanovich
` (3 subsequent siblings)
4 siblings, 1 reply; 7+ messages in thread
From: Anton Mikanovich @ 2022-02-25 7:40 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
After moving apt-get call from lock-protected do_apt_fetch to the
do_apt_unpack it should be also covered by isar.lock to avoid conflict.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
meta/classes/dpkg-base.bbclass | 2 ++
1 file changed, 2 insertions(+)
diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass
index 928856a..e3ec133 100644
--- a/meta/classes/dpkg-base.bbclass
+++ b/meta/classes/dpkg-base.bbclass
@@ -142,6 +142,8 @@ do_apt_unpack() {
dpkg_undo_mounts
}
+do_apt_unpack[lockfiles] += "${REPO_ISAR_DIR}/isar.lock"
+
addtask apt_unpack after do_apt_fetch before do_patch
addtask cleanall_apt before do_cleanall
--
2.25.1
^ permalink raw reply [flat|nested] 7+ messages in thread
* [RFC 2/5] meta: Implement two stage build
2022-02-25 7:40 [RFC 0/5] Debian dependencies investigation Anton Mikanovich
2022-02-25 7:40 ` [RFC 1/5] dpkg-base: Cover do_apt_unpack task by lock Anton Mikanovich
@ 2022-02-25 7:40 ` Anton Mikanovich
2022-02-25 7:40 ` [RFC 3/5] libhello: Declare provided packages Anton Mikanovich
` (2 subsequent siblings)
4 siblings, 0 replies; 7+ messages in thread
From: Anton Mikanovich @ 2022-02-25 7:40 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
Split package build into two stages: building source package and
building binary package. This allows to obtain Debian package
dependencies before any build steps will actually start.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
meta/classes/dpkg-base.bbclass | 34 +++++++++++++++++--
meta/classes/dpkg-gbp.bbclass | 6 +---
meta/classes/dpkg-prebuilt.bbclass | 7 ++++
meta/classes/dpkg.bbclass | 21 +++++++++---
meta/classes/rootfs.bbclass | 2 ++
.../buildchroot/buildchroot.inc | 2 ++
.../buildchroot/files/build-source.sh | 16 +++++++++
.../buildchroot/files/build.sh | 2 +-
8 files changed, 76 insertions(+), 14 deletions(-)
create mode 100644 meta/recipes-devtools/buildchroot/files/build-source.sh
diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass
index e3ec133..916bae7 100644
--- a/meta/classes/dpkg-base.bbclass
+++ b/meta/classes/dpkg-base.bbclass
@@ -75,7 +75,7 @@ addtask adjust_git after do_unpack before do_patch
do_adjust_git[lockfiles] += "${DL_DIR}/git/isar.lock"
inherit patch
-addtask patch after do_adjust_git before do_dpkg_build
+addtask patch after do_adjust_git before do_dpkg_source
SRC_APT ?= ""
@@ -175,7 +175,7 @@ do_prepare_build() {
true
}
-addtask prepare_build after do_patch do_transform_template before do_dpkg_build
+addtask prepare_build after do_patch do_transform_template before do_dpkg_source
# If Isar recipes depend on each other, they typically need the package
# deployed to isar-apt
do_prepare_build[deptask] = "do_deploy_deb"
@@ -238,6 +238,19 @@ python do_dpkg_build() {
addtask dpkg_build
+python do_dpkg_source() {
+ lock = bb.utils.lockfile(d.getVar("REPO_ISAR_DIR") + "/isar.lock",
+ shared=True)
+ bb.build.exec_func("dpkg_do_mounts", d)
+ try:
+ bb.build.exec_func("dpkg_runbuild_source", d)
+ finally:
+ bb.build.exec_func("dpkg_undo_mounts", d)
+ bb.utils.unlockfile(lock)
+}
+
+addtask dpkg_source before do_dpkg_build
+
SSTATETASKS += "do_dpkg_build"
DPKG_SSTATE = "${WORKDIR}/dpkg-sstate"
do_dpkg_build[dirs] += "${DPKG_SSTATE} ${S}/.."
@@ -300,6 +313,21 @@ do_deploy_deb[depends] += "isar-apt:do_cache_config"
do_deploy_deb[lockfiles] = "${REPO_ISAR_DIR}/isar.lock"
do_deploy_deb[dirs] = "${S}"
+do_deploy_source() {
+ #TODO: dsc_clean
+ find "${S}/../" -name '*\.dsc' | while read package; do
+ repo_add_srcpackage "${REPO_ISAR_DIR}"/"${DISTRO}" \
+ "${REPO_ISAR_DB_DIR}"/"${DISTRO}" \
+ "${DEBDISTRONAME}" \
+ "${package}"
+ done
+}
+
+addtask deploy_source after do_dpkg_source before do_dpkg_build
+do_deploy_source[depends] += "isar-apt:do_cache_config"
+do_deploy_source[lockfiles] = "${REPO_ISAR_DIR}/isar.lock"
+do_deploy_source[dirs] = "${S}"
+
python do_devshell() {
bb.build.exec_func('dpkg_do_mounts', d)
@@ -327,6 +355,6 @@ python do_devshell_nodeps() {
# devshell may be placed after do_instell_builddeps in downstream classes.
# devshell_nodeps will always stay right after do_prepare_build.
-addtask devshell_nodeps after do_prepare_build
+addtask devshell_nodeps after do_deploy_source
do_devshell_nodeps[dirs] = "${DEVSHELL_STARTDIR}"
do_devshell_nodeps[nostamp] = "1"
diff --git a/meta/classes/dpkg-gbp.bbclass b/meta/classes/dpkg-gbp.bbclass
index d956e8c..b02aed0 100644
--- a/meta/classes/dpkg-gbp.bbclass
+++ b/meta/classes/dpkg-gbp.bbclass
@@ -12,8 +12,7 @@ PATCHTOOL ?= "git"
GBP_DEPENDS ?= "git-buildpackage pristine-tar"
GBP_EXTRA_OPTIONS ?= "--git-pristine-tar"
-do_install_builddeps_append() {
- dpkg_do_mounts
+dpkg_runbuild_source_prepend() {
distro="${DISTRO}"
if [ ${ISAR_CROSS_COMPILE} -eq 1 ]; then
distro="${HOST_DISTRO}"
@@ -26,9 +25,6 @@ do_install_builddeps_append() {
sudo -E chroot ${BUILDCHROOT_DIR} \
apt-get install -y -o Debug::pkgProblemResolver=yes \
--no-install-recommends ${GBP_DEPENDS}
- dpkg_undo_mounts
-}
-dpkg_runbuild_prepend() {
export GBP_PREFIX="gbp buildpackage --git-ignore-new ${GBP_EXTRA_OPTIONS} --git-builder="
}
diff --git a/meta/classes/dpkg-prebuilt.bbclass b/meta/classes/dpkg-prebuilt.bbclass
index c32224b..c5d645c 100644
--- a/meta/classes/dpkg-prebuilt.bbclass
+++ b/meta/classes/dpkg-prebuilt.bbclass
@@ -18,3 +18,10 @@ python do_unpack_prepend() {
deltask dpkg_build
addtask unpack before do_deploy_deb
+
+deltask deploy_source
+# dummy task for 1st stage
+do_deploy_source() {
+ true
+}
+addtask deploy_source after do_deploy_deb before do_build
diff --git a/meta/classes/dpkg.bbclass b/meta/classes/dpkg.bbclass
index 320102b..e95420a 100644
--- a/meta/classes/dpkg.bbclass
+++ b/meta/classes/dpkg.bbclass
@@ -17,15 +17,17 @@ do_install_builddeps() {
fi
deb_dl_dir_import "${BUILDCHROOT_DIR}" "${distro}"
- sudo -E chroot ${BUILDCHROOT_DIR} /isar/deps.sh \
- ${PP}/${PPS} ${PACKAGE_ARCH} --download-only
+ sudo -E chroot ${BUILDCHROOT_DIR} sh -c "apt-get update && apt-get \
+ -o Debug::pkgProblemResolver=yes --no-install-recommends -y \
+ --allow-downgrades --download-only build-dep -a ${PACKAGE_ARCH} ${PN}"
deb_dl_dir_export "${BUILDCHROOT_DIR}" "${distro}"
- sudo -E chroot ${BUILDCHROOT_DIR} /isar/deps.sh \
- ${PP}/${PPS} ${PACKAGE_ARCH}
+ sudo -E chroot ${BUILDCHROOT_DIR} sh -c "apt-get update && apt-get \
+ -o Debug::pkgProblemResolver=yes --no-install-recommends -y \
+ --allow-downgrades build-dep -a ${PACKAGE_ARCH} ${PN}"
dpkg_undo_mounts
}
-addtask install_builddeps after do_prepare_build before do_dpkg_build
+addtask install_builddeps after do_deploy_source before do_dpkg_build
do_install_builddeps[depends] += "isar-apt:do_cache_config"
# apt and reprepro may not run in parallel, acquire the Isar lock
do_install_builddeps[lockfiles] += "${REPO_ISAR_DIR}/isar.lock"
@@ -42,3 +44,12 @@ dpkg_runbuild() {
sudo -E chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} \
/isar/build.sh ${PP}/${PPS} ${PACKAGE_ARCH}
}
+
+dpkg_runbuild_source() {
+ E="${@ isar_export_proxies(d)}"
+ export DEB_BUILD_OPTIONS="${@ isar_deb_build_options(d)}"
+ export DEB_BUILD_PROFILES="${@ isar_deb_build_profiles(d)}"
+ export PARALLEL_MAKE="${PARALLEL_MAKE}"
+ sudo -E chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} \
+ /isar/build-source.sh ${PP}/${PPS} ${PACKAGE_ARCH}
+}
diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass
index d578dd3..c5ea748 100644
--- a/meta/classes/rootfs.bbclass
+++ b/meta/classes/rootfs.bbclass
@@ -88,6 +88,8 @@ rootfs_configure_isar_apt() {
mkdir -p '${ROOTFSDIR}/etc/apt/sources.list.d'
echo 'deb [trusted=yes] file:///isar-apt ${DEBDISTRONAME} main' > \
'${ROOTFSDIR}/etc/apt/sources.list.d/isar-apt.list'
+ echo 'deb-src [trusted=yes] file:///isar-apt ${DEBDISTRONAME} main' >> \
+ '${ROOTFSDIR}/etc/apt/sources.list.d/isar-apt.list'
mkdir -p '${ROOTFSDIR}/etc/apt/preferences.d'
cat << EOF > '${ROOTFSDIR}/etc/apt/preferences.d/isar-apt'
diff --git a/meta/recipes-devtools/buildchroot/buildchroot.inc b/meta/recipes-devtools/buildchroot/buildchroot.inc
index aa190e9..2872680 100644
--- a/meta/recipes-devtools/buildchroot/buildchroot.inc
+++ b/meta/recipes-devtools/buildchroot/buildchroot.inc
@@ -9,6 +9,7 @@ LIC_FILES_CHKSUM = "file://${LAYERDIR_core}/licenses/COPYING.GPLv2;md5=751419260
FILESPATH_prepend := "${THISDIR}/files:"
SRC_URI = "file://configscript.sh \
file://build.sh \
+ file://build-source.sh \
file://common.sh \
file://deps.sh"
PV = "1.0"
@@ -58,6 +59,7 @@ buildchroot_install_files() {
sudo chmod -R a+rw "${BUILDCHROOT_DIR}/home/builder"
sudo install -m 755 -d ${BUILDCHROOT_DIR}/isar
sudo install -m 755 ${WORKDIR}/build.sh ${BUILDCHROOT_DIR}/isar/
+ sudo install -m 755 ${WORKDIR}/build-source.sh ${BUILDCHROOT_DIR}/isar/
sudo install -m 755 ${WORKDIR}/common.sh ${BUILDCHROOT_DIR}/isar/
sudo install -m 755 ${WORKDIR}/deps.sh ${BUILDCHROOT_DIR}/isar/
diff --git a/meta/recipes-devtools/buildchroot/files/build-source.sh b/meta/recipes-devtools/buildchroot/files/build-source.sh
new file mode 100644
index 0000000..a327c9b
--- /dev/null
+++ b/meta/recipes-devtools/buildchroot/files/build-source.sh
@@ -0,0 +1,16 @@
+#!/bin/bash
+#
+# This software is a part of ISAR.
+# Copyright (C) 2022 ilbers GmbH
+
+source /isar/common.sh
+
+# If autotools files have been created, update their timestamp to
+# prevent them from being regenerated
+for i in configure aclocal.m4 Makefile.am Makefile.in; do
+ if [ -f "${i}" ]; then
+ touch "${i}"
+ fi
+done
+
+${GBP_PREFIX}dpkg-buildpackage -a$target_arch -d --source-option=-I -S
diff --git a/meta/recipes-devtools/buildchroot/files/build.sh b/meta/recipes-devtools/buildchroot/files/build.sh
index d4709cf..c19221b 100644
--- a/meta/recipes-devtools/buildchroot/files/build.sh
+++ b/meta/recipes-devtools/buildchroot/files/build.sh
@@ -14,4 +14,4 @@ for i in configure aclocal.m4 Makefile.am Makefile.in; do
fi
done
-${GBP_PREFIX}dpkg-buildpackage -a$target_arch --source-option=-I
+${GBP_PREFIX}dpkg-buildpackage -a$target_arch --source-option=-I -b
--
2.25.1
^ permalink raw reply [flat|nested] 7+ messages in thread
* [RFC 3/5] libhello: Declare provided packages
2022-02-25 7:40 [RFC 0/5] Debian dependencies investigation Anton Mikanovich
2022-02-25 7:40 ` [RFC 1/5] dpkg-base: Cover do_apt_unpack task by lock Anton Mikanovich
2022-02-25 7:40 ` [RFC 2/5] meta: Implement two stage build Anton Mikanovich
@ 2022-02-25 7:40 ` Anton Mikanovich
2022-02-25 7:40 ` [RFC 4/5] hello-isar: Remove duplicated dependency Anton Mikanovich
2022-02-25 7:40 ` [RFC 5/5] isar: Add external builder Anton Mikanovich
4 siblings, 0 replies; 7+ messages in thread
From: Anton Mikanovich @ 2022-02-25 7:40 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
To make bitbake know about libhello-dev package provided by libhello.bb
recipe we need to add PROVIDES variable.
This allow running 'bitbake libhello-dev' without care about actual
recipe.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
meta-isar/recipes-app/libhello/libhello.bb | 2 ++
1 file changed, 2 insertions(+)
diff --git a/meta-isar/recipes-app/libhello/libhello.bb b/meta-isar/recipes-app/libhello/libhello.bb
index 5c44de5..c7f1fe4 100644
--- a/meta-isar/recipes-app/libhello/libhello.bb
+++ b/meta-isar/recipes-app/libhello/libhello.bb
@@ -10,6 +10,8 @@ LIC_FILES_CHKSUM = "file://${LAYERDIR_core}/licenses/COPYING.GPLv2;md5=751419260
PV = "0.1-98f2e41"
+PROVIDES = "libhello-dev"
+
SRC_URI = "git://github.com/ilbers/libhello.git;protocol=https;destsuffix=${P}"
SRCREV = "98f2e41e7d05ab8d19b0c5d160b104b725c8fd93"
--
2.25.1
^ permalink raw reply [flat|nested] 7+ messages in thread
* [RFC 4/5] hello-isar: Remove duplicated dependency
2022-02-25 7:40 [RFC 0/5] Debian dependencies investigation Anton Mikanovich
` (2 preceding siblings ...)
2022-02-25 7:40 ` [RFC 3/5] libhello: Declare provided packages Anton Mikanovich
@ 2022-02-25 7:40 ` Anton Mikanovich
2022-02-25 7:40 ` [RFC 5/5] isar: Add external builder Anton Mikanovich
4 siblings, 0 replies; 7+ messages in thread
From: Anton Mikanovich @ 2022-02-25 7:40 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
Remove build dependency already mentioned in debian/control.
We will parse all the deps automatically without need in manual
dependency duplication.
Warning! Normal Isar build will fail after that!
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
meta-isar/recipes-app/hello-isar/hello-isar.bb | 4 ----
1 file changed, 4 deletions(-)
diff --git a/meta-isar/recipes-app/hello-isar/hello-isar.bb b/meta-isar/recipes-app/hello-isar/hello-isar.bb
index 8c3ba8b..20794c2 100644
--- a/meta-isar/recipes-app/hello-isar/hello-isar.bb
+++ b/meta-isar/recipes-app/hello-isar/hello-isar.bb
@@ -10,10 +10,6 @@ LIC_FILES_CHKSUM = "file://${LAYERDIR_core}/licenses/COPYING.GPLv2;md5=751419260
PV = "0.3-a18c14c"
-# NOTE: the following line duplicates the content in 'debian/control', but
-# for now it's the only way to correctly build bitbake pipeline.
-DEPENDS += "libhello"
-
SRC_URI = " \
git://github.com/ilbers/hello.git;protocol=https;destsuffix=${P} \
file://subdir/0001-Add-some-help.patch \
--
2.25.1
^ permalink raw reply [flat|nested] 7+ messages in thread
* [RFC 5/5] isar: Add external builder
2022-02-25 7:40 [RFC 0/5] Debian dependencies investigation Anton Mikanovich
` (3 preceding siblings ...)
2022-02-25 7:40 ` [RFC 4/5] hello-isar: Remove duplicated dependency Anton Mikanovich
@ 2022-02-25 7:40 ` Anton Mikanovich
4 siblings, 0 replies; 7+ messages in thread
From: Anton Mikanovich @ 2022-02-25 7:40 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
Build Isar with external tool to parce all Debian package dependencies.
Support only one target build for now,
mc:qemuamd64-bullseye:isar-image-base is used by default.
Usage (inside build dir):
$ ../isar_builder.py
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
isar_builder.py | 142 ++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 142 insertions(+)
create mode 100755 isar_builder.py
diff --git a/isar_builder.py b/isar_builder.py
new file mode 100755
index 0000000..cf99c52
--- /dev/null
+++ b/isar_builder.py
@@ -0,0 +1,142 @@
+#!/usr/bin/env python3
+
+import os
+import sys
+import apt_pkg
+import apt.progress.base
+
+sys.path.insert(0, os.path.dirname(os.path.realpath(__file__)) + '/bitbake/lib')
+
+import bb.tinfoil
+
+class IsarBuilder:
+ def __init__(self):
+ self.tinfoil = bb.tinfoil.Tinfoil()
+ self.tinfoil.prepare()
+
+ def create_apt_rootfs(self, rootfs):
+ repo_isar_dir = '%s/tmp/deploy/isar-apt/%s-%s/apt' % (os.path.abspath(os.getcwd()), self.distro, self.arch)
+ mirror = 'file://%s/%s' % (repo_isar_dir, self.distro)
+ codename = 'isar'
+
+ if not os.path.exists(rootfs + '/var/lib/dpkg'):
+ os.makedirs(rootfs + '/var/lib/dpkg')
+ open(rootfs + '/var/lib/dpkg/status', 'w').close()
+
+ if not os.path.exists(rootfs + '/etc/apt/preferences.d'):
+ os.makedirs(rootfs + '/etc/apt/preferences.d')
+ with open(rootfs + '/etc/apt/sources.list', 'w') as f:
+ f.write('deb [arch=%s] %s %s main\n' % (self.arch, mirror, codename))
+ f.write('deb-src [arch=%s] %s %s main\n' % (self.arch, mirror, codename))
+
+ if not os.path.exists(rootfs + '/var/cache/apt/archives/partial'):
+ os.makedirs(rootfs + '/var/cache/apt/archives/partial')
+
+ def apt_config(self):
+ apt_pkg.init()
+
+ rootfs = os.path.abspath(os.getcwd()) + '/tmp/deps_poc_rootfs/%s-%s' % (self.distro, self.arch)
+
+ if not os.path.isdir(rootfs):
+ self.create_apt_rootfs(rootfs)
+
+ apt_pkg.config.set('APT::Architecture', self.arch)
+ apt_pkg.config.set('Dir', rootfs)
+ apt_pkg.config.set('Dir::Cache', rootfs + '/var/cache/apt')
+ apt_pkg.config.set('Dir::State::status', rootfs + '/var/lib/dpkg/status')
+
+ apt_pkg.config.set("Acquire::AllowInsecureRepositories", "1")
+
+ def isar_apt_update(self):
+ sources = apt_pkg.SourceList()
+ sources.read_main_list()
+
+ progress = apt.progress.text.AcquireProgress()
+
+ self.cache = apt_pkg.Cache()
+ self.cache.update(progress, sources)
+ self.cache = apt_pkg.Cache()
+
+ def srcpackage_lookup(self, pkg):
+ if pkg in self.cache:
+ return
+ if self.tinfoil.find_best_provider(pkg)[3] is None:
+ return
+ if pkg not in self.target_deps:
+ self.need_deb.append(pkg)
+ self.sr.restart()
+ if self.sr.lookup(pkg) is None:
+ self.need_source.append(pkg)
+ return
+ if 'Build-Depends' not in self.sr.build_depends:
+ return
+ for dep in self.sr.build_depends['Build-Depends']:
+ child = str(dep[0][0])
+ if child not in self.checkdeps:
+ self.checkdeps.append(child)
+ self.srcpackage_lookup(child)
+
+ def bitbake(self, mc, targets, task='', justbuild=[]):
+ targets = [targets] if isinstance(targets, str) else targets
+ target_str = ''
+ for pn in targets:
+ target_str += 'mc:%s:%s ' % (mc, pn if not task else ':'.join([pn, task]))
+ targets.clear()
+ for pn in justbuild:
+ target_str += 'mc:%s:%s ' % (mc, pn)
+ justbuild.clear()
+ if target_str != '':
+ print('Building %s' % target_str)
+ self.tinfoil.build_targets(target_str)
+
+ def build(self, mc, target):
+ d = self.tinfoil.parse_recipe('multiconfig:%s:%s' % (mc, target))
+ self.distro = d.getVar('DISTRO', expand=True)
+ self.arch = d.getVar('DISTRO_ARCH', expand=True)
+
+ recipecache = self.tinfoil.cooker.recipecaches[mc]
+ provider = self.tinfoil.find_best_provider('multiconfig:%s:%s' % (mc, target))
+ self.target_deps = recipecache.deps[provider[3]]
+
+ kernel_image = d.getVar('KERNEL_IMAGE_PKG', expand=True) or ''
+ if kernel_image in self.target_deps:
+ kernel = self.tinfoil.find_best_provider('multiconfig:%s:%s' % (mc, kernel_image))
+ if kernel[3].endswith('linux-distro.bb'):
+ self.target_deps.remove(kernel_image)
+ print('Remove %s from checking' % kernel_image)
+
+ self.need_source = self.target_deps.copy()
+ self.need_deb = []
+
+ self.apt_config()
+
+ maxdepth = 3
+ while maxdepth > 0:
+ maxdepth -= 1
+
+ self.bitbake(mc, self.need_source, 'do_deploy_source', self.need_deb)
+
+ self.isar_apt_update()
+ self.sr = apt_pkg.SourceRecords()
+
+ self.checkdeps = []
+ for dep in self.target_deps:
+ if dep not in self.checkdeps:
+ self.checkdeps.append(dep)
+ self.srcpackage_lookup(dep)
+
+ if not self.need_source:
+ break
+
+ if self.need_source:
+ print('Following packages still left unchecked: ' + str(self.need_source))
+
+ if self.need_deb:
+ print('Additional Debian dependencies found:' + str(self.need_deb))
+ self.bitbake(mc, self.need_deb)
+
+ # start final build
+ self.bitbake(mc, target)
+
+ib = IsarBuilder()
+ib.build('qemuamd64-bullseye', 'isar-image-base')
--
2.25.1
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [RFC 1/5] dpkg-base: Cover do_apt_unpack task by lock
2022-02-25 7:40 ` [RFC 1/5] dpkg-base: Cover do_apt_unpack task by lock Anton Mikanovich
@ 2022-02-25 17:30 ` Jan Kiszka
0 siblings, 0 replies; 7+ messages in thread
From: Jan Kiszka @ 2022-02-25 17:30 UTC (permalink / raw)
To: Anton Mikanovich, isar-users
On 25.02.22 08:40, Anton Mikanovich wrote:
> After moving apt-get call from lock-protected do_apt_fetch to the
> do_apt_unpack it should be also covered by isar.lock to avoid conflict.
>
Mind to tell us when this happened (Fixes: ...)? Did you see real
problems? How often? All valuable information to assess the relevance
and criticality for downstream possibly using an affected version.
Jan
> Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
> ---
> meta/classes/dpkg-base.bbclass | 2 ++
> 1 file changed, 2 insertions(+)
>
> diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass
> index 928856a..e3ec133 100644
> --- a/meta/classes/dpkg-base.bbclass
> +++ b/meta/classes/dpkg-base.bbclass
> @@ -142,6 +142,8 @@ do_apt_unpack() {
> dpkg_undo_mounts
> }
>
> +do_apt_unpack[lockfiles] += "${REPO_ISAR_DIR}/isar.lock"
> +
> addtask apt_unpack after do_apt_fetch before do_patch
>
> addtask cleanall_apt before do_cleanall
--
Siemens AG, Technology
Competence Center Embedded Linux
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2022-02-25 17:30 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-02-25 7:40 [RFC 0/5] Debian dependencies investigation Anton Mikanovich
2022-02-25 7:40 ` [RFC 1/5] dpkg-base: Cover do_apt_unpack task by lock Anton Mikanovich
2022-02-25 17:30 ` Jan Kiszka
2022-02-25 7:40 ` [RFC 2/5] meta: Implement two stage build Anton Mikanovich
2022-02-25 7:40 ` [RFC 3/5] libhello: Declare provided packages Anton Mikanovich
2022-02-25 7:40 ` [RFC 4/5] hello-isar: Remove duplicated dependency Anton Mikanovich
2022-02-25 7:40 ` [RFC 5/5] isar: Add external builder Anton Mikanovich
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox