* [PATCH v5 0/9] Imager schroot migration
@ 2023-05-31 11:12 Anton Mikanovich
2023-05-31 11:12 ` [PATCH v5 1/9] isar-apt: Move cleanup to postprocessing Anton Mikanovich
` (9 more replies)
0 siblings, 10 replies; 11+ messages in thread
From: Anton Mikanovich @ 2023-05-31 11:12 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
This patchset moves Isar imager from buildchroot to schroot usage as the next
step after moving to sbuild.
Imagers of every target are now running in separate schroot sessions with
independent overlays on top of basic unchanged schroot. Any changes made inside
overlay will be dropped after the usage to keep basic schroot clean.
Changes since v4:
- Rebased on next
- Add RECIPE-API-CHANGELOG entry
- Remove unused mounts from dpkg-gbp
Changes since v3:
- Rebased on next
- Fixed full CI build issues
- Add documentation updates
- Fix task name in CI
- Some code cleanup
Changes since v2:
- Rebased on next
- Do not keep schroot session open outside tasks
- Get rid of final cleanup code
- Fix sstate test case
- Get back buildchroot removing
Changes since v1:
- Rebased on next and bitbake update
- Fix some build cases
Anton Mikanovich (9):
isar-apt: Move cleanup to postprocessing
imager: Migrate from buildchroot to schroot
imager: Move image types to schroot
imager: Split imager deps between image types
CI: Fix used chroot recipe name
meta: Remove buildchroot
dpkg-gbp: Remove unused mounts
doc: Update chroot documentation
RECIPE-API-CHANGELOG.md: Describe imager changes
RECIPE-API-CHANGELOG.md | 20 ++++
doc/offline.md | 2 +-
doc/technical_overview.md | 107 +++++++++---------
doc/user_manual.md | 53 +++++----
meta-isar/conf/local.conf.sample | 2 +-
meta/classes/buildchroot.bbclass | 59 ----------
meta/classes/crossvars.bbclass | 6 -
meta/classes/deb-dl-dir.bbclass | 4 +-
meta/classes/dpkg-gbp.bbclass | 2 -
meta/classes/image-tools-extension.bbclass | 99 +++++++++++-----
meta/classes/image.bbclass | 35 +++---
meta/classes/imagetypes_vm.bbclass | 9 +-
meta/classes/imagetypes_wic.bbclass | 89 +++++----------
meta/classes/isar-events.bbclass | 1 -
meta/classes/rootfs.bbclass | 12 ++
meta/conf/bitbake.conf | 3 -
.../buildchroot/buildchroot-host.bb | 13 ---
.../buildchroot/buildchroot-target.bb | 10 --
.../buildchroot/buildchroot.inc | 73 ------------
.../buildchroot/files/build.sh | 17 ---
.../buildchroot/files/common.sh | 32 ------
.../buildchroot/files/configscript.sh | 15 ---
.../buildchroot/files/deps.sh | 47 --------
.../sbuild-chroot/sbuild-chroot.inc | 1 +
scripts/contrib/convert-overrides.py | 2 +-
testsuite/cibase.py | 8 +-
26 files changed, 238 insertions(+), 483 deletions(-)
delete mode 100644 meta/classes/buildchroot.bbclass
delete mode 100644 meta/recipes-devtools/buildchroot/buildchroot-host.bb
delete mode 100644 meta/recipes-devtools/buildchroot/buildchroot-target.bb
delete mode 100644 meta/recipes-devtools/buildchroot/buildchroot.inc
delete mode 100644 meta/recipes-devtools/buildchroot/files/build.sh
delete mode 100644 meta/recipes-devtools/buildchroot/files/common.sh
delete mode 100644 meta/recipes-devtools/buildchroot/files/configscript.sh
delete mode 100644 meta/recipes-devtools/buildchroot/files/deps.sh
--
2.34.1
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH v5 1/9] isar-apt: Move cleanup to postprocessing
2023-05-31 11:12 [PATCH v5 0/9] Imager schroot migration Anton Mikanovich
@ 2023-05-31 11:12 ` Anton Mikanovich
2023-05-31 11:12 ` [PATCH v5 2/9] imager: Migrate from buildchroot to schroot Anton Mikanovich
` (8 subsequent siblings)
9 siblings, 0 replies; 11+ messages in thread
From: Anton Mikanovich @ 2023-05-31 11:12 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
As isar-apt now used only for packages install move its cleanup stage
from finalize task to postprocessing.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
meta/classes/image.bbclass | 5 -----
meta/classes/rootfs.bbclass | 12 ++++++++++++
.../recipes-devtools/sbuild-chroot/sbuild-chroot.inc | 1 +
3 files changed, 13 insertions(+), 5 deletions(-)
diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index 08b6d3d6..53859eb9 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -450,11 +450,6 @@ do_rootfs_finalize() {
mountpoint -q '${ROOTFSDIR}/sys' && \
umount -l ${ROOTFSDIR}/sys
- rm -f "${ROOTFSDIR}/etc/apt/sources.list.d/isar-apt.list"
- rm -f "${ROOTFSDIR}/etc/apt/preferences.d/isar-apt"
- rm -f "${ROOTFSDIR}/etc/apt/sources.list.d/base-apt.list"
- rm -f "${ROOTFSDIR}/etc/apt/apt.conf.d/50isar"
-
if [ -e "${ROOTFSDIR}/etc/apt/sources-list" ]; then
mv "${ROOTFSDIR}/etc/apt/sources-list" \
"${ROOTFSDIR}/etc/apt/sources.list.d/bootstrap.list"
diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass
index 22449d71..7dec7b36 100644
--- a/meta/classes/rootfs.bbclass
+++ b/meta/classes/rootfs.bbclass
@@ -293,6 +293,18 @@ rootfs_export_dpkg_status() {
'${ROOTFS_DPKGSTATUS_DEPLOY_DIR}'/'${ROOTFS_PACKAGE_SUFFIX}'.dpkg_status
}
+ROOTFS_POSTPROCESS_COMMAND += "rootfs_cleanup_isar_apt"
+rootfs_cleanup_isar_apt[weight] = "2"
+rootfs_cleanup_isar_apt() {
+ sudo -s <<'EOSUDO'
+ set -e
+ rm -f "${ROOTFSDIR}/etc/apt/sources.list.d/isar-apt.list"
+ rm -f "${ROOTFSDIR}/etc/apt/preferences.d/isar-apt"
+ rm -f "${ROOTFSDIR}/etc/apt/sources.list.d/base-apt.list"
+ rm -f "${ROOTFSDIR}/etc/apt/apt.conf.d/50isar"
+EOSUDO
+}
+
do_rootfs_postprocess[vardeps] = "${ROOTFS_POSTPROCESS_COMMAND}"
do_rootfs_postprocess[network] = "${TASK_USE_SUDO}"
python do_rootfs_postprocess() {
diff --git a/meta/recipes-devtools/sbuild-chroot/sbuild-chroot.inc b/meta/recipes-devtools/sbuild-chroot/sbuild-chroot.inc
index fd8bb648..49992db0 100644
--- a/meta/recipes-devtools/sbuild-chroot/sbuild-chroot.inc
+++ b/meta/recipes-devtools/sbuild-chroot/sbuild-chroot.inc
@@ -51,6 +51,7 @@ ROOTFS_PACKAGES = "${SBUILD_CHROOT_PREINSTALL}"
# We don't need /etc/apt/sources.list.d/isar-apt.list' while it's handled by sbuild
ROOTFS_CONFIGURE_COMMAND:remove = "rootfs_configure_isar_apt"
+ROOTFS_POSTPROCESS_COMMAND:remove = "rootfs_cleanup_isar_apt"
DEPLOY_SCHROOT = "${@d.getVar('SCHROOT_' + d.getVar('SBUILD_VARIANT').upper() + '_DIR')}"
--
2.34.1
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH v5 2/9] imager: Migrate from buildchroot to schroot
2023-05-31 11:12 [PATCH v5 0/9] Imager schroot migration Anton Mikanovich
2023-05-31 11:12 ` [PATCH v5 1/9] isar-apt: Move cleanup to postprocessing Anton Mikanovich
@ 2023-05-31 11:12 ` Anton Mikanovich
2023-05-31 11:12 ` [PATCH v5 3/9] imager: Move image types " Anton Mikanovich
` (7 subsequent siblings)
9 siblings, 0 replies; 11+ messages in thread
From: Anton Mikanovich @ 2023-05-31 11:12 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
Install dependencies and perform all imager actions using schroot
instead of buildchroot. All the changes made during imager task will be
lost after the finish. This requires installing imager dependencies
during every run.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
meta/classes/image-tools-extension.bbclass | 97 +++++++++++++++-------
1 file changed, 66 insertions(+), 31 deletions(-)
diff --git a/meta/classes/image-tools-extension.bbclass b/meta/classes/image-tools-extension.bbclass
index 7c84505f..04ed1f84 100644
--- a/meta/classes/image-tools-extension.bbclass
+++ b/meta/classes/image-tools-extension.bbclass
@@ -5,43 +5,78 @@
#
# This file extends the image.bbclass to supply tools for futher imager functions
-inherit buildchroot
+inherit sbuild
IMAGER_INSTALL ??= ""
IMAGER_BUILD_DEPS ??= ""
DEPENDS += "${IMAGER_BUILD_DEPS}"
-do_install_imager_deps[depends] = "${BUILDCHROOT_DEP} isar-apt:do_cache_config"
-do_install_imager_deps[deptask] = "do_deploy_deb"
-do_install_imager_deps[lockfiles] += "${REPO_ISAR_DIR}/isar.lock"
-do_install_imager_deps[network] = "${TASK_USE_NETWORK_AND_SUDO}"
-do_install_imager_deps() {
- if [ -z "${@d.getVar("IMAGER_INSTALL").strip()}" ]; then
- exit
- fi
+SCHROOT_MOUNTS = "${WORKDIR}:${PP_WORK} ${IMAGE_ROOTFS}:${PP_ROOTFS} ${DEPLOY_DIR_IMAGE}:${PP_DEPLOY}"
+SCHROOT_MOUNTS += "${REPO_ISAR_DIR}/${DISTRO}:/isar-apt"
+
+imager_run() {
+ schroot_create_configs
+ insert_mounts
+
+ session_id=$(schroot -q -b -c ${SBUILD_CHROOT})
+ echo "Started session: ${session_id}"
+
+ # Schroot session mountpoint for deb downloads import/export
+ schroot_dir="/var/run/schroot/mount/${session_id}"
+
+ # setting up error handler
+ imager_cleanup() {
+ set +e
+ schroot -q -f -e -c ${session_id} > /dev/null 2>&1
+ remove_mounts > /dev/null 2>&1
+ schroot_delete_configs > /dev/null 2>&1
+ }
+ trap 'exit 1' INT HUP QUIT TERM ALRM USR1
+ trap 'imager_cleanup' EXIT
+
+ if [ -n "${@d.getVar("IMAGER_INSTALL").strip()}" ]; then
+ echo "Installing deps: ${IMAGER_INSTALL}"
- distro="${BASE_DISTRO}-${BASE_DISTRO_CODENAME}"
- if [ ${ISAR_CROSS_COMPILE} -eq 1 ]; then
- distro="${HOST_BASE_DISTRO}-${BASE_DISTRO_CODENAME}"
+ distro="${BASE_DISTRO}-${BASE_DISTRO_CODENAME}"
+ if [ ${ISAR_CROSS_COMPILE} -eq 1 ]; then
+ distro="${HOST_BASE_DISTRO}-${BASE_DISTRO_CODENAME}"
+ fi
+
+ # prepare isar-apt
+ schroot -r -c ${session_id} -d / -u root -- sh -c " \
+ mkdir -p '/etc/apt/sources.list.d'
+ echo 'deb [trusted=yes] file:///isar-apt ${DEBDISTRONAME} main' > \
+ '/etc/apt/sources.list.d/isar-apt.list'
+
+ mkdir -p '/etc/apt/preferences.d'
+ cat << EOF > '/etc/apt/preferences.d/isar-apt'
+Package: *
+Pin: release n=${DEBDISTRONAME}
+Pin-Priority: 1000
+EOF"
+
+ E="${@ isar_export_proxies(d)}"
+ deb_dl_dir_import ${schroot_dir} ${distro}
+ schroot -r -c ${session_id} -d / -u root -- sh -c " \
+ apt-get update \
+ -o Dir::Etc::SourceList='sources.list.d/isar-apt.list' \
+ -o Dir::Etc::SourceParts='-' \
+ -o APT::Get::List-Cleanup='0'
+ apt-get -o Debug::pkgProblemResolver=yes --no-install-recommends -y \
+ --allow-unauthenticated --allow-downgrades --download-only install \
+ ${IMAGER_INSTALL}"
+
+ deb_dl_dir_export ${schroot_dir} ${distro}
+ schroot -r -c ${session_id} -d / -u root -- sh -c " \
+ apt-get -o Debug::pkgProblemResolver=yes --no-install-recommends -y \
+ --allow-unauthenticated --allow-downgrades install \
+ ${IMAGER_INSTALL}"
fi
- buildchroot_do_mounts
-
- E="${@ isar_export_proxies(d)}"
- deb_dl_dir_import ${BUILDCHROOT_DIR} ${distro}
- sudo -E chroot ${BUILDCHROOT_DIR} sh -c ' \
- apt-get update \
- -o Dir::Etc::SourceList="sources.list.d/isar-apt.list" \
- -o Dir::Etc::SourceParts="-" \
- -o APT::Get::List-Cleanup="0"
- apt-get -o Debug::pkgProblemResolver=yes --no-install-recommends -y \
- --allow-unauthenticated --allow-downgrades --download-only install \
- ${IMAGER_INSTALL}'
-
- deb_dl_dir_export ${BUILDCHROOT_DIR} ${distro}
- sudo -E chroot ${BUILDCHROOT_DIR} sh -c ' \
- apt-get -o Debug::pkgProblemResolver=yes --no-install-recommends -y \
- --allow-unauthenticated --allow-downgrades install \
- ${IMAGER_INSTALL}'
+ schroot -r -c ${session_id} "$@"
+
+ schroot -e -c ${session_id}
+
+ remove_mounts
+ schroot_delete_configs
}
-addtask install_imager_deps before do_image_tools
--
2.34.1
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH v5 3/9] imager: Move image types to schroot
2023-05-31 11:12 [PATCH v5 0/9] Imager schroot migration Anton Mikanovich
2023-05-31 11:12 ` [PATCH v5 1/9] isar-apt: Move cleanup to postprocessing Anton Mikanovich
2023-05-31 11:12 ` [PATCH v5 2/9] imager: Migrate from buildchroot to schroot Anton Mikanovich
@ 2023-05-31 11:12 ` Anton Mikanovich
2023-05-31 11:12 ` [PATCH v5 4/9] imager: Split imager deps between image types Anton Mikanovich
` (6 subsequent siblings)
9 siblings, 0 replies; 11+ messages in thread
From: Anton Mikanovich @ 2023-05-31 11:12 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
Run imagetype-related commands inside schroot session during images
prepare steps.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
meta/classes/image.bbclass | 7 ++-
meta/classes/imagetypes_vm.bbclass | 9 +--
meta/classes/imagetypes_wic.bbclass | 89 ++++++++++-------------------
3 files changed, 38 insertions(+), 67 deletions(-)
diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index 53859eb9..42e4b0c9 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -163,7 +163,7 @@ inherit ${IMGCLASSES}
# convenience variables to be used by CMDs
IMAGE_FILE_HOST = "${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.${type}"
IMAGE_FILE_CHROOT = "${PP_DEPLOY}/${IMAGE_FULLNAME}.${type}"
-SUDO_CHROOT = "sudo chroot ${BUILDCHROOT_DIR}"
+SUDO_CHROOT = "imager_run -d ${PP_ROOTFS} -u root --"
# hook up IMAGE_CMD_*
python() {
@@ -235,7 +235,6 @@ python() {
imager_build_deps.add(dep)
# construct image command
- cmds.append('\timage_do_mounts')
image_cmd = localdata.getVar('IMAGE_CMD:' + bt_clean)
if image_cmd:
localdata.setVar('type', bt)
@@ -409,6 +408,10 @@ python do_image_tools() {
}
addtask image_tools before do_build after do_rootfs
+# all imagetypes are depend on schroot and isar-apt
+do_image_tools[depends] = "${SCHROOT_DEP} isar-apt:do_cache_config"
+do_image_tools[deptask] = "do_deploy_deb"
+
python do_image() {
"""Virtual task"""
pass
diff --git a/meta/classes/imagetypes_vm.bbclass b/meta/classes/imagetypes_vm.bbclass
index 41f2af06..782b60c1 100644
--- a/meta/classes/imagetypes_vm.bbclass
+++ b/meta/classes/imagetypes_vm.bbclass
@@ -4,8 +4,6 @@
# This class allows to generate images for VMware and VirtualBox
#
-inherit buildchroot
-
OVF_TEMPLATE_FILE ?= "vm-img-virtualbox.ovf.tmpl"
IMAGE_SRC_URI:ova = "file://${OVF_TEMPLATE_FILE}"
@@ -35,9 +33,8 @@ CONVERSION_OPTIONS = "${@set_convert_options(d)}"
convert_wic() {
rm -f '${DEPLOY_DIR_IMAGE}/${VIRTUAL_MACHINE_IMAGE_FILE}'
- image_do_mounts
bbnote "Creating ${VIRTUAL_MACHINE_IMAGE_FILE} from ${SOURCE_IMAGE_FILE}"
- sudo -E chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} \
+ imager_run -p -d ${PP_WORK} -- \
/usr/bin/qemu-img convert -f raw -O ${VIRTUAL_MACHINE_IMAGE_TYPE} ${CONVERSION_OPTIONS} \
'${PP_DEPLOY}/${SOURCE_IMAGE_FILE}' '${VIRTUAL_MACHINE_DISK}'
}
@@ -80,7 +77,7 @@ IMAGE_CMD:ova() {
export LAST_CHANGE=$(date -u "+%Y-%m-%dT%H:%M:%SZ")
export OVA_FIRMWARE_UPPERCASE=$(echo ${OVA_FIRMWARE} | tr '[a-z]' '[A-Z]')
export OVF_TEMPLATE_STAGE2=$(echo ${OVF_TEMPLATE_FILE} | sed 's/.tmpl$//' )
- sudo -Es chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} <<'EOSUDO'
+ imager_run -p -d ${PP_WORK} <<'EOIMAGER'
set -e
export DISK_SIZE_BYTES=$(qemu-img info -f vmdk "${VIRTUAL_MACHINE_DISK}" \
| gawk 'match($0, /^virtual size:.*\(([0-9]+) bytes\)/, a) {print a[1]}')
@@ -97,6 +94,6 @@ IMAGE_CMD:ova() {
tar -uvf ${PP_DEPLOY}/${OVA_NAME}.ova -C ${PP_DEPLOY} ${OVA_NAME}.mf
fi
tar -uvf ${PP_DEPLOY}/${OVA_NAME}.ova -C ${PP_DEPLOY} ${VIRTUAL_MACHINE_IMAGE_FILE}
-EOSUDO
+EOIMAGER
}
IMAGE_CMD:ova[depends] = "${PN}:do_transform_template"
diff --git a/meta/classes/imagetypes_wic.bbclass b/meta/classes/imagetypes_wic.bbclass
index 5be53f60..9a0bc2b7 100644
--- a/meta/classes/imagetypes_wic.bbclass
+++ b/meta/classes/imagetypes_wic.bbclass
@@ -83,8 +83,6 @@ python () {
bb.build.addtask('do_transform_template', 'do_image_wic', None, d)
}
-inherit buildchroot
-
IMAGER_INSTALL:wic += "${WIC_IMAGER_INSTALL}"
# wic comes with reasonable defaults, and the proper interface is the wks file
ROOTFS_EXTRA ?= "0"
@@ -95,7 +93,6 @@ STAGING_DIR ?= "${TMPDIR}"
IMAGE_BASENAME ?= "${PN}-${DISTRO}"
FAKEROOTCMD ?= "${SCRIPTSDIR}/wic_fakeroot"
RECIPE_SYSROOT_NATIVE ?= "/"
-BUILDCHROOT_DIR = "${BUILDCHROOT_TARGET_DIR}"
WIC_CREATE_EXTRA_ARGS ?= ""
WIC_DEPLOY_PARTITIONS ?= "0"
@@ -145,26 +142,11 @@ check_for_wic_warnings() {
do_image_wic[file-checksums] += "${WKS_FILE_CHECKSUM}"
IMAGE_CMD:wic() {
- wic_do_mounts
generate_wic_image
check_for_wic_warnings
}
-wic_do_mounts[vardepsexclude] += "BITBAKEDIR"
-wic_do_mounts() {
- buildchroot_do_mounts
- sudo -s <<'EOSUDO'
- ( flock 9
- set -e
- for dir in ${BBLAYERS} ${STAGING_DIR} ${SCRIPTSDIR} ${BITBAKEDIR}; do
- mkdir -p ${BUILDCHROOT_DIR}/$dir
- if ! mountpoint ${BUILDCHROOT_DIR}/$dir >/dev/null 2>&1; then
- mount --bind --make-private $dir ${BUILDCHROOT_DIR}/$dir
- fi
- done
- ) 9>${MOUNT_LOCKFILE}
-EOSUDO
-}
+SCHROOT_MOUNTS += "${BBLAYERS} ${STAGING_DIR} ${SCRIPTSDIR} ${BITBAKEDIR}"
generate_wic_image[vardepsexclude] += "WKS_FULL_PATH BITBAKEDIR TOPDIR"
generate_wic_image() {
@@ -177,50 +159,39 @@ generate_wic_image() {
mkdir -p ${IMAGE_ROOTFS}/../pseudo
touch ${IMAGE_ROOTFS}/../pseudo/files.db
- # create the temp dir in the buildchroot to ensure uniqueness
- WICTMP=$(cd ${BUILDCHROOT_DIR}; mktemp -d -p tmp)
-
- sudo -E chroot ${BUILDCHROOT_DIR} \
- sh -c ' \
- BITBAKEDIR="$1"
- SCRIPTSDIR="$2"
- WKS_FULL_PATH="$3"
- STAGING_DIR="$4"
- MACHINE="$5"
- WICTMP="$6"
- IMAGE_FULLNAME="$7"
- IMAGE_BASENAME="$8"
- shift 8
- # The python path is hard-coded as /usr/bin/python3-native/python3 in wic. Handle that.
- mkdir -p /usr/bin/python3-native/
- if [ $(head -1 $(which bmaptool) | grep python3) ];then
+ imager_run -p -d ${PP_WORK} -u root <<'EOIMAGER'
+ set -e
+
+ # The python path is hard-coded as /usr/bin/python3-native/python3 in wic. Handle that.
+ mkdir -p /usr/bin/python3-native/
+ if [ $(head -1 $(which bmaptool) | grep python3) ];then
ln -fs /usr/bin/python3 /usr/bin/python3-native/python3
- else
+ else
ln -fs /usr/bin/python2 /usr/bin/python3-native/python3
- fi
- export PATH="$BITBAKEDIR/bin:$PATH"
- "$SCRIPTSDIR"/wic create "$WKS_FULL_PATH" \
- --vars "$STAGING_DIR/$MACHINE/imgdata/" \
- -o "/$WICTMP/${IMAGE_FULLNAME}.wic/" \
+ fi
+
+ export PATH="${BITBAKEDIR}/bin:$PATH"
+
+ "${SCRIPTSDIR}"/wic create "${WKS_FULL_PATH}" \
+ --vars "${STAGING_DIR}/${MACHINE}/imgdata/" \
+ -o "/tmp/${IMAGE_FULLNAME}.wic/" \
--bmap \
- -e "$IMAGE_BASENAME" $@' \
- my_script "${BITBAKEDIR}" "${SCRIPTSDIR}" "${WKS_FULL_PATH}" "${STAGING_DIR}" \
- "${MACHINE}" "${WICTMP}" "${IMAGE_FULLNAME}" "${IMAGE_BASENAME}" \
- ${WIC_CREATE_EXTRA_ARGS}
+ -e "${IMAGE_BASENAME}" ${WIC_CREATE_EXTRA_ARGS}
+
+ WIC_DIRECT=$(ls -t -1 /tmp/${IMAGE_FULLNAME}.wic/*.direct | head -1)
+ mv -f ${WIC_DIRECT} ${PP_DEPLOY}/${IMAGE_FULLNAME}.wic
+ mv -f ${WIC_DIRECT}.bmap ${PP_DEPLOY}/${IMAGE_FULLNAME}.wic.bmap
+ # deploy partition files if requested (ending with .p<x>)
+ if [ "${WIC_DEPLOY_PARTITIONS}" -eq "1" ]; then
+ # locate *.direct.p<x> partition files
+ find "/tmp/${IMAGE_FULLNAME}.wic/" -type f -regextype sed -regex ".*\.direct.*\.p[0-9]\{1,\}" | while read f; do
+ suffix=$(basename $f | sed 's/.*\.direct\(.*\)/\1/')
+ mv -f ${f} ${PP_DEPLOY}/${IMAGE_FULLNAME}.wic${suffix}
+ done
+ fi
+EOIMAGER
sudo chown -R $(stat -c "%U" ${LAYERDIR_core}) ${LAYERDIR_core} ${LAYERDIR_isar} ${SCRIPTSDIR} || true
- WIC_DIRECT=$(ls -t -1 ${BUILDCHROOT_DIR}/$WICTMP/${IMAGE_FULLNAME}.wic/*.direct | head -1)
- sudo chown -R $(id -u):$(id -g) ${BUILDCHROOT_DIR}/${WICTMP}
- mv -f ${WIC_DIRECT} ${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.wic
- mv -f ${WIC_DIRECT}.bmap ${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.wic.bmap
- # deploy partition files if requested (ending with .p<x>)
- if [ "${WIC_DEPLOY_PARTITIONS}" -eq "1" ]; then
- # locate *.direct.p<x> partition files
- find ${BUILDCHROOT_DIR}/${WICTMP} -type f -regextype sed -regex ".*\.direct.*\.p[0-9]\{1,\}" | while read f; do
- suffix=$(basename $f | sed 's/.*\.direct\(.*\)/\1/')
- mv -f ${f} ${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.wic${suffix}
- done
- fi
- rm -rf ${BUILDCHROOT_DIR}/${WICTMP}
+ sudo chown -R $(id -u):$(id -g) "${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.wic"*
rm -rf ${IMAGE_ROOTFS}/../pseudo
}
--
2.34.1
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH v5 4/9] imager: Split imager deps between image types
2023-05-31 11:12 [PATCH v5 0/9] Imager schroot migration Anton Mikanovich
` (2 preceding siblings ...)
2023-05-31 11:12 ` [PATCH v5 3/9] imager: Move image types " Anton Mikanovich
@ 2023-05-31 11:12 ` Anton Mikanovich
2023-05-31 11:12 ` [PATCH v5 5/9] CI: Fix used chroot recipe name Anton Mikanovich
` (5 subsequent siblings)
9 siblings, 0 replies; 11+ messages in thread
From: Anton Mikanovich @ 2023-05-31 11:12 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
To reduce dependencies install time and disk space consumption we need
to install only those imager dependencies which used in every image
type prepare logic. So split variables between tasks.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
meta/classes/image-tools-extension.bbclass | 10 ++++++----
meta/classes/image.bbclass | 8 ++++++++
2 files changed, 14 insertions(+), 4 deletions(-)
diff --git a/meta/classes/image-tools-extension.bbclass b/meta/classes/image-tools-extension.bbclass
index 04ed1f84..58799fcf 100644
--- a/meta/classes/image-tools-extension.bbclass
+++ b/meta/classes/image-tools-extension.bbclass
@@ -15,6 +15,8 @@ SCHROOT_MOUNTS = "${WORKDIR}:${PP_WORK} ${IMAGE_ROOTFS}:${PP_ROOTFS} ${DEPLOY_DI
SCHROOT_MOUNTS += "${REPO_ISAR_DIR}/${DISTRO}:/isar-apt"
imager_run() {
+ local_install="${@(d.getVar("INSTALL_%s" % d.getVar("BB_CURRENTTASK")) or '').strip()}"
+
schroot_create_configs
insert_mounts
@@ -34,8 +36,8 @@ imager_run() {
trap 'exit 1' INT HUP QUIT TERM ALRM USR1
trap 'imager_cleanup' EXIT
- if [ -n "${@d.getVar("IMAGER_INSTALL").strip()}" ]; then
- echo "Installing deps: ${IMAGER_INSTALL}"
+ if [ -n "${local_install}" ]; then
+ echo "Installing imager deps: ${local_install}"
distro="${BASE_DISTRO}-${BASE_DISTRO_CODENAME}"
if [ ${ISAR_CROSS_COMPILE} -eq 1 ]; then
@@ -64,13 +66,13 @@ EOF"
-o APT::Get::List-Cleanup='0'
apt-get -o Debug::pkgProblemResolver=yes --no-install-recommends -y \
--allow-unauthenticated --allow-downgrades --download-only install \
- ${IMAGER_INSTALL}"
+ ${local_install}"
deb_dl_dir_export ${schroot_dir} ${distro}
schroot -r -c ${session_id} -d / -u root -- sh -c " \
apt-get -o Debug::pkgProblemResolver=yes --no-install-recommends -y \
--allow-unauthenticated --allow-downgrades install \
- ${IMAGER_INSTALL}"
+ ${local_install}"
fi
schroot -r -c ${session_id} "$@"
diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index 42e4b0c9..b9d3d411 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -208,6 +208,8 @@ python() {
imager_build_deps = set()
conversion_install = set()
for bt in basetypes:
+ local_imager_install = set()
+ local_conversion_install = set()
vardeps = set()
cmds = []
bt_clean = bt.replace('-', '_').replace('.', '_')
@@ -231,6 +233,7 @@ python() {
# imager install
for dep in (d.getVar('IMAGER_INSTALL:' + bt_clean) or '').split():
imager_install.add(dep)
+ local_imager_install.add(dep)
for dep in (d.getVar('IMAGER_BUILD_DEPS:' + bt_clean) or '').split():
imager_build_deps.add(dep)
@@ -273,6 +276,7 @@ python() {
vardeps.add('CONVERSION_CMD:' + c)
for dep in (localdata.getVar('CONVERSION_DEPS:' + c) or '').split():
conversion_install.add(dep)
+ local_conversion_install.add(dep)
# remove temporary image files
if t not in image_types:
rm_images.add(localdata.expand('${IMAGE_FILE_HOST}'))
@@ -305,6 +309,10 @@ python() {
d.appendVarFlag(task, 'depends', task_deps)
bb.build.addtask(task, 'do_image', after, d)
+ # set per type imager dependencies
+ d.setVar('INSTALL_image_%s' % bt_clean, d.getVar('IMAGER_INSTALL'))
+ d.appendVar('INSTALL_image_%s' % bt_clean, ' ' + ' '.join(sorted(local_imager_install | local_conversion_install)))
+
d.appendVar('IMAGER_INSTALL', ' ' + ' '.join(sorted(imager_install | conversion_install)))
d.appendVar('IMAGER_BUILD_DEPS', ' ' + ' '.join(sorted(imager_build_deps)))
}
--
2.34.1
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH v5 5/9] CI: Fix used chroot recipe name
2023-05-31 11:12 [PATCH v5 0/9] Imager schroot migration Anton Mikanovich
` (3 preceding siblings ...)
2023-05-31 11:12 ` [PATCH v5 4/9] imager: Split imager deps between image types Anton Mikanovich
@ 2023-05-31 11:12 ` Anton Mikanovich
2023-05-31 11:12 ` [PATCH v5 6/9] meta: Remove buildchroot Anton Mikanovich
` (4 subsequent siblings)
9 siblings, 0 replies; 11+ messages in thread
From: Anton Mikanovich @ 2023-05-31 11:12 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
We've moved to schroot, so need to adjust recipe names for sstate test
case in CI.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
testsuite/cibase.py | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/testsuite/cibase.py b/testsuite/cibase.py
index df8b350e..d9641377 100755
--- a/testsuite/cibase.py
+++ b/testsuite/cibase.py
@@ -151,7 +151,7 @@ class CIBaseTest(CIBuilder):
if not all([
check_executed_tasks('isar-bootstrap-target',
['do_bootstrap_setscene', '!do_bootstrap']),
- check_executed_tasks('buildchroot-target',
+ check_executed_tasks('sbuild-chroot-target',
['do_rootfs_install_setscene', '!do_rootfs_install']),
check_executed_tasks('isar-image-base-*',
['do_rootfs_install_setscene', '!do_rootfs_install'])
@@ -175,8 +175,8 @@ class CIBaseTest(CIBuilder):
if not all([
check_executed_tasks('isar-bootstrap-target',
['do_bootstrap_setscene']),
- check_executed_tasks('buildchroot-target',
- ['!do_buildchroot_deploy']),
+ check_executed_tasks('sbuild-chroot-target',
+ ['!do_sbuildchroot_deploy']),
check_executed_tasks('hello',
['do_dpkg_build_setscene', 'do_deploy_deb', '!do_dpkg_build'])
]):
@@ -189,7 +189,7 @@ class CIBaseTest(CIBuilder):
if not all([
check_executed_tasks('isar-bootstrap-target',
['do_bootstrap_setscene', '!do_bootstrap']),
- check_executed_tasks('buildchroot-target',
+ check_executed_tasks('sbuild-chroot-target',
['do_rootfs_install_setscene', '!do_rootfs_install']),
check_executed_tasks('hello',
['do_fetch', 'do_dpkg_build']),
--
2.34.1
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH v5 6/9] meta: Remove buildchroot
2023-05-31 11:12 [PATCH v5 0/9] Imager schroot migration Anton Mikanovich
` (4 preceding siblings ...)
2023-05-31 11:12 ` [PATCH v5 5/9] CI: Fix used chroot recipe name Anton Mikanovich
@ 2023-05-31 11:12 ` Anton Mikanovich
2023-05-31 11:12 ` [PATCH v5 7/9] dpkg-gbp: Remove unused mounts Anton Mikanovich
` (3 subsequent siblings)
9 siblings, 0 replies; 11+ messages in thread
From: Anton Mikanovich @ 2023-05-31 11:12 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
Buildchroot is not used anywhere so remove it.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
meta-isar/conf/local.conf.sample | 2 +-
meta/classes/buildchroot.bbclass | 59 ---------------
meta/classes/crossvars.bbclass | 6 --
meta/classes/deb-dl-dir.bbclass | 4 +-
meta/classes/image.bbclass | 15 ----
meta/classes/isar-events.bbclass | 1 -
meta/conf/bitbake.conf | 3 -
.../buildchroot/buildchroot-host.bb | 13 ----
.../buildchroot/buildchroot-target.bb | 10 ---
.../buildchroot/buildchroot.inc | 73 -------------------
.../buildchroot/files/build.sh | 17 -----
.../buildchroot/files/common.sh | 32 --------
.../buildchroot/files/configscript.sh | 15 ----
.../buildchroot/files/deps.sh | 47 ------------
scripts/contrib/convert-overrides.py | 2 +-
15 files changed, 3 insertions(+), 296 deletions(-)
delete mode 100644 meta/classes/buildchroot.bbclass
delete mode 100644 meta/recipes-devtools/buildchroot/buildchroot-host.bb
delete mode 100644 meta/recipes-devtools/buildchroot/buildchroot-target.bb
delete mode 100644 meta/recipes-devtools/buildchroot/buildchroot.inc
delete mode 100644 meta/recipes-devtools/buildchroot/files/build.sh
delete mode 100644 meta/recipes-devtools/buildchroot/files/common.sh
delete mode 100644 meta/recipes-devtools/buildchroot/files/configscript.sh
delete mode 100644 meta/recipes-devtools/buildchroot/files/deps.sh
diff --git a/meta-isar/conf/local.conf.sample b/meta-isar/conf/local.conf.sample
index c5d50931..a9b6e5e8 100644
--- a/meta-isar/conf/local.conf.sample
+++ b/meta-isar/conf/local.conf.sample
@@ -24,7 +24,7 @@ MACHINE ??= "qemuarm"
# Isar Configuration Selection
#
# You need to select a specific distribution configuration which will used for both:
-# generation of buildchroot environment and target root filesystem.
+# generation of schroot environment and target root filesystem.
#
# This sets the default distribution configuration:
DISTRO ??= "debian-bullseye"
diff --git a/meta/classes/buildchroot.bbclass b/meta/classes/buildchroot.bbclass
deleted file mode 100644
index e4a7a571..00000000
--- a/meta/classes/buildchroot.bbclass
+++ /dev/null
@@ -1,59 +0,0 @@
-# This software is a part of ISAR.
-# Copyright (C) 2018 Siemens AG
-#
-# SPDX-License-Identifier: MIT
-
-inherit crossvars
-
-MOUNT_LOCKFILE = "${BUILDCHROOT_DIR}.lock"
-
-buildchroot_do_mounts() {
- if [ "${USE_CCACHE}" = "1" ]; then
- mkdir -p "${CCACHE_DIR}"
- if [ "${CCACHE_DEBUG}" = "1" ]; then
- mkdir -p "${CCACHE_DIR}/debug"
- fi
- fi
-
- sudo -s <<'EOSUDO'
- ( flock 9
- set -e
-
- mountpoint -q '${BUILDCHROOT_DIR}/isar-apt' ||
- mount --bind '${REPO_ISAR_DIR}/${DISTRO}' '${BUILDCHROOT_DIR}/isar-apt'
- mountpoint -q '${BUILDCHROOT_DIR}/downloads' ||
- mount --bind '${DL_DIR}' '${BUILDCHROOT_DIR}/downloads'
- if [ "${USE_CCACHE}" = "1" ]; then
- mkdir -p '${BUILDCHROOT_DIR}/ccache'
- mountpoint -q '${BUILDCHROOT_DIR}/ccache' ||
- mount --bind '${CCACHE_DIR}' '${BUILDCHROOT_DIR}/ccache'
- fi
- mountpoint -q '${BUILDCHROOT_DIR}/dev' ||
- ( mount -o bind,private /dev '${BUILDCHROOT_DIR}/dev' &&
- mount -t tmpfs none '${BUILDCHROOT_DIR}/dev/shm' &&
- mount --bind /dev/pts '${BUILDCHROOT_DIR}/dev/pts' )
- mountpoint -q '${BUILDCHROOT_DIR}/proc' ||
- mount -t proc none '${BUILDCHROOT_DIR}/proc'
- mountpoint -q '${BUILDCHROOT_DIR}/sys' ||
- mount --rbind /sys '${BUILDCHROOT_DIR}/sys'
- mount --make-rslave '${BUILDCHROOT_DIR}/sys'
-
- # Mount base-apt if 'ISAR_USE_CACHED_BASE_REPO' is set
- if [ "${@repr(bb.utils.to_boolean(d.getVar('ISAR_USE_CACHED_BASE_REPO')))}" = 'True' ]
- then
- mkdir -p '${BUILDCHROOT_DIR}/base-apt'
- mountpoint -q '${BUILDCHROOT_DIR}/base-apt' || \
- mount --bind '${REPO_BASE_DIR}' '${BUILDCHROOT_DIR}/base-apt'
- fi
-
- # Refresh or remove /etc/resolv.conf at this chance
- if [ "${@repr(bb.utils.to_boolean(d.getVar('BB_NO_NETWORK')))}" = 'True' ]
- then
- rm -rf '${BUILDCHROOT_DIR}/etc/resolv.conf'
- else
- cp -L /etc/resolv.conf '${BUILDCHROOT_DIR}/etc'
- fi
-
- ) 9>'${MOUNT_LOCKFILE}'
-EOSUDO
-}
diff --git a/meta/classes/crossvars.bbclass b/meta/classes/crossvars.bbclass
index 201d460f..e6a085b1 100644
--- a/meta/classes/crossvars.bbclass
+++ b/meta/classes/crossvars.bbclass
@@ -15,20 +15,14 @@ python __anonymous() {
d.setVar('BUILD_HOST_ARCH', distro_arch)
schroot_dir = d.getVar('SCHROOT_TARGET_DIR', False)
sbuild_dep = "sbuild-chroot-target:do_build"
- buildchroot_dir = d.getVar('BUILDCHROOT_TARGET_DIR', False)
- buildchroot_dep = "buildchroot-target:do_build"
sdk_toolchain = "build-essential"
else:
d.setVar('BUILD_HOST_ARCH', d.getVar('HOST_ARCH'))
schroot_dir = d.getVar('SCHROOT_HOST_DIR', False)
sbuild_dep = "sbuild-chroot-host:do_build"
- buildchroot_dir = d.getVar('BUILDCHROOT_HOST_DIR', False)
- buildchroot_dep = "buildchroot-host:do_build"
sdk_toolchain = "crossbuild-essential-" + distro_arch
d.setVar('SCHROOT_DIR', schroot_dir)
d.setVar('SCHROOT_DEP', sbuild_dep)
- d.setVar('BUILDCHROOT_DIR', buildchroot_dir)
- d.setVar('BUILDCHROOT_DEP', buildchroot_dep)
if isar_can_build_compat(d):
sdk_toolchain += " crossbuild-essential-" + d.getVar('COMPAT_DISTRO_ARCH')
d.setVar('SDK_TOOLCHAIN', sdk_toolchain)
diff --git a/meta/classes/deb-dl-dir.bbclass b/meta/classes/deb-dl-dir.bbclass
index f0ab9714..d36b7190 100644
--- a/meta/classes/deb-dl-dir.bbclass
+++ b/meta/classes/deb-dl-dir.bbclass
@@ -17,9 +17,7 @@ is_not_part_of_current_build() {
"${SCHROOT_HOST_DIR}"/var/log/dpkg.log \
"${SCHROOT_TARGET_DIR}"/var/log/dpkg.log \
"${SCHROOT_HOST_DIR}"/tmp/dpkg_common.log \
- "${SCHROOT_TARGET_DIR}"/tmp/dpkg_common.log \
- "${BUILDCHROOT_HOST_DIR}"/var/log/dpkg.log \
- "${BUILDCHROOT_TARGET_DIR}"/var/log/dpkg.log | head -1 )"
+ "${SCHROOT_TARGET_DIR}"/tmp/dpkg_common.log | head -1 )"
[ -z "${output}" ]
}
diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index b9d3d411..e0c0a3cf 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -35,11 +35,6 @@ PP_DEPLOY = "${PP}/deploy"
PP_ROOTFS = "${PP}/rootfs"
PP_WORK = "${PP}/work"
-BUILDROOT = "${BUILDCHROOT_DIR}${PP}"
-BUILDROOT_DEPLOY = "${BUILDCHROOT_DIR}${PP_DEPLOY}"
-BUILDROOT_ROOTFS = "${BUILDCHROOT_DIR}${PP_ROOTFS}"
-BUILDROOT_WORK = "${BUILDCHROOT_DIR}${PP_WORK}"
-
python(){
if (d.getVar('IMAGE_TRANSIENT_PACKAGES')):
bb.warn("IMAGE_TRANSIENT_PACKAGES is set and no longer supported")
@@ -72,16 +67,6 @@ DEPENDS += "${IMAGE_INSTALL}"
ISAR_RELEASE_CMD_DEFAULT = "git -C ${LAYERDIR_core} describe --tags --dirty --match 'v[0-9].[0-9]*'"
ISAR_RELEASE_CMD ?= "${ISAR_RELEASE_CMD_DEFAULT}"
-image_do_mounts() {
- sudo flock ${MOUNT_LOCKFILE} -c ' \
- mkdir -p "${BUILDROOT_DEPLOY}" "${BUILDROOT_ROOTFS}" "${BUILDROOT_WORK}"
- mount --bind "${DEPLOY_DIR_IMAGE}" "${BUILDROOT_DEPLOY}"
- mount --bind "${IMAGE_ROOTFS}" "${BUILDROOT_ROOTFS}"
- mount --bind "${WORKDIR}" "${BUILDROOT_WORK}"
- '
- buildchroot_do_mounts
-}
-
inherit multiarch
inherit essential
diff --git a/meta/classes/isar-events.bbclass b/meta/classes/isar-events.bbclass
index 4f3c87dd..a6ba0a9e 100644
--- a/meta/classes/isar-events.bbclass
+++ b/meta/classes/isar-events.bbclass
@@ -9,7 +9,6 @@ addhandler build_started
python build_started() {
bb.utils.remove(d.getVar('TMPDIR') + "/work/*/*/*/temp/once.*")
bb.utils.remove(d.getVar('TMPDIR') + "/work/*/*/*/rootfs.mount")
- bb.utils.remove(d.getVar('TMPDIR') + "/deploy/buildchroot-*/*.mount")
}
build_started[eventmask] = "bb.event.BuildStarted"
diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index a85fea77..a95cc9fd 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -54,14 +54,11 @@ GITPKGV = "${@bb.fetch2.get_srcrev(d, 'gitpkgv_revision')}"
WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}/${PV}-${PR}"
GIT_DL_LINK_DIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}"
DEPLOY_DIR_BOOTSTRAP = "${DEPLOY_DIR}/bootstrap"
-DEPLOY_DIR_BUILDCHROOT = "${DEPLOY_DIR}/buildchroot"
DEPLOY_DIR_SDKCHROOT = "${DEPLOY_DIR}/sdkchroot"
DEPLOY_DIR_IMAGE = "${DEPLOY_DIR}/images/${MACHINE}"
DL_DIR ?= "${TOPDIR}/downloads"
SSTATE_DIR ?= "${TOPDIR}/sstate-cache"
SSTATE_MANIFESTS = "${TMPDIR}/sstate-control/${DISTRO}-${DISTRO_ARCH}"
-BUILDCHROOT_HOST_DIR = "${DEPLOY_DIR_BUILDCHROOT}-host/${HOST_DISTRO}-${HOST_ARCH}_${DISTRO}-${DISTRO_ARCH}"
-BUILDCHROOT_TARGET_DIR = "${DEPLOY_DIR_BUILDCHROOT}-target/${DISTRO}-${DISTRO_ARCH}"
SCHROOT_HOST_DIR = "${DEPLOY_DIR}/schroot-host/${HOST_DISTRO}-${HOST_ARCH}_${DISTRO}-${DISTRO_ARCH}"
SCHROOT_TARGET_DIR = "${DEPLOY_DIR}/schroot-target/${DISTRO}-${DISTRO_ARCH}"
SDKCHROOT_DIR = "${DEPLOY_DIR_SDKCHROOT}/${BPN}-${DISTRO}-${MACHINE}"
diff --git a/meta/recipes-devtools/buildchroot/buildchroot-host.bb b/meta/recipes-devtools/buildchroot/buildchroot-host.bb
deleted file mode 100644
index e5ce248e..00000000
--- a/meta/recipes-devtools/buildchroot/buildchroot-host.bb
+++ /dev/null
@@ -1,13 +0,0 @@
-# Root filesystem for packages building
-#
-# This software is a part of ISAR.
-# Copyright (C) 2015-2018 ilbers GmbH
-
-DESCRIPTION = "Isar development filesystem for host"
-
-BUILDCHROOT_VARIANT = "host"
-
-require buildchroot.inc
-ROOTFS_ARCH = "${HOST_ARCH}"
-ROOTFS_DISTRO = "${HOST_DISTRO}"
-ROOTFS_BASE_DISTRO = "${HOST_BASE_DISTRO}"
diff --git a/meta/recipes-devtools/buildchroot/buildchroot-target.bb b/meta/recipes-devtools/buildchroot/buildchroot-target.bb
deleted file mode 100644
index 6613de63..00000000
--- a/meta/recipes-devtools/buildchroot/buildchroot-target.bb
+++ /dev/null
@@ -1,10 +0,0 @@
-# Root filesystem for packages building
-#
-# This software is a part of ISAR.
-# Copyright (C) 2015-2018 ilbers GmbH
-
-DESCRIPTION = "Isar development filesystem for target"
-
-BUILDCHROOT_VARIANT = "target"
-
-require buildchroot.inc
diff --git a/meta/recipes-devtools/buildchroot/buildchroot.inc b/meta/recipes-devtools/buildchroot/buildchroot.inc
deleted file mode 100644
index f74896fb..00000000
--- a/meta/recipes-devtools/buildchroot/buildchroot.inc
+++ /dev/null
@@ -1,73 +0,0 @@
-# Common part for build chroot filesystem.
-#
-# This software is a part of ISAR.
-# Copyright (C) 2015-2018 ilbers GmbH
-
-LICENSE = "gpl-2.0"
-LIC_FILES_CHKSUM = "file://${LAYERDIR_core}/licenses/COPYING.GPLv2;md5=751419260aa954499f7abaabaa882bbe"
-
-FILESPATH:prepend := "${THISDIR}/files:"
-SRC_URI = "file://configscript.sh \
- file://build.sh \
- file://common.sh \
- file://deps.sh"
-PV = "1.0"
-
-inherit rootfs
-
-BUILDCHROOT_DIR = "${WORKDIR}/rootfs"
-ROOTFSDIR = "${BUILDCHROOT_DIR}"
-ROOTFS_PACKAGES = "${BUILDCHROOT_PREINSTALL}"
-ROOTFS_CLEAN_FILES = ""
-ROOTFS_MANIFEST_DEPLOY_DIR = "${DEPLOY_DIR_BUILDCHROOT}"
-ROOTFS_DPKGSTATUS_DEPLOY_DIR = "${DEPLOY_DIR_BUILDCHROOT}"
-ROOTFS_FEATURES += "generate-manifest export-dpkg-status"
-
-BUILDCHROOT_PREINSTALL ?= " \
- dpkg \
- locales \
- apt \
- equivs \
- adduser"
-
-rootfs_do_mounts:append() {
- sudo -s <<'EOSUDO'
- set -e
- mkdir -p '${BUILDCHROOT_DIR}/downloads'
- mountpoint -q '${BUILDCHROOT_DIR}/downloads' || \
- mount --bind '${DL_DIR}' '${BUILDCHROOT_DIR}/downloads'
-EOSUDO
-}
-
-ROOTFS_POSTPROCESS_COMMAND =+ "buildchroot_install_files"
-buildchroot_install_files() {
- sudo mkdir -p "${BUILDCHROOT_DIR}/home/builder"
- # Install package builder script
- sudo chmod -R a+rw "${BUILDCHROOT_DIR}/home/builder"
- sudo install -m 755 -d ${BUILDCHROOT_DIR}/isar
- sudo install -m 755 ${WORKDIR}/build.sh ${BUILDCHROOT_DIR}/isar/
- sudo install -m 755 ${WORKDIR}/common.sh ${BUILDCHROOT_DIR}/isar/
- sudo install -m 755 ${WORKDIR}/deps.sh ${BUILDCHROOT_DIR}/isar/
-
- sudo ln -sf /downloads/git "${BUILDCHROOT_DIR}/home/.git-downloads"
-
- # Configure root filesystem
- sudo install -m 755 ${WORKDIR}/configscript.sh ${BUILDCHROOT_DIR}
- USER_ID=$(id -u)
- GROUP_ID=$(id -g)
- sudo chroot ${BUILDCHROOT_DIR} /configscript.sh $USER_ID $GROUP_ID
-
-}
-
-DEPLOY_BUILDCHROOT = "${@d.getVar('BUILDCHROOT_' + d.getVar('BUILDCHROOT_VARIANT').upper() + '_DIR')}"
-
-do_buildchroot_deploy[dirs] = "${DEPLOY_DIR_BUILDCHROOT}-${BUILDCHROOT_VARIANT}"
-do_buildchroot_deploy() {
- ln -Tfsr "${ROOTFSDIR}" "${DEPLOY_BUILDCHROOT}"
-}
-addtask buildchroot_deploy before do_build after do_rootfs
-
-CLEANFUNCS = "clean_deploy"
-clean_deploy() {
- rm -f "${DEPLOY_BUILDCHROOT}"
-}
diff --git a/meta/recipes-devtools/buildchroot/files/build.sh b/meta/recipes-devtools/buildchroot/files/build.sh
deleted file mode 100644
index d4709cf5..00000000
--- a/meta/recipes-devtools/buildchroot/files/build.sh
+++ /dev/null
@@ -1,17 +0,0 @@
-#!/bin/bash
-#
-# This software is a part of ISAR.
-# Copyright (C) 2015-2017 ilbers GmbH
-# Copyright (c) 2018 Siemens AG
-
-source /isar/common.sh
-
-# If autotools files have been created, update their timestamp to
-# prevent them from being regenerated
-for i in configure aclocal.m4 Makefile.am Makefile.in; do
- if [ -f "${i}" ]; then
- touch "${i}"
- fi
-done
-
-${GBP_PREFIX}dpkg-buildpackage -a$target_arch --source-option=-I
diff --git a/meta/recipes-devtools/buildchroot/files/common.sh b/meta/recipes-devtools/buildchroot/files/common.sh
deleted file mode 100644
index 73226d8d..00000000
--- a/meta/recipes-devtools/buildchroot/files/common.sh
+++ /dev/null
@@ -1,32 +0,0 @@
-#!/bin/bash
-#
-# This software is a part of ISAR.
-# Copyright (C) 2015-2017 ilbers GmbH
-# Copyright (c) 2018 Siemens AG
-
-set -e
-printenv | grep -q BB_VERBOSE_LOGS && set -x
-
-# assert we are either "root:root" or "builder:<gid of builder>"
-if ([ "$(id -un)" != "builder" ] || [ "$(id -g)" != "$(id -g builder)" ]) &&
- ([ "$(id -un)" != "root" ] || [ "$(id -gn)" != "root" ]); then
- echo "This script can only be run as root:root or builder:<gid of builder>!" >&2
- echo "(Currently running as $(id -un)($(id -u)):$(id -gn)($(id -g)))" >&2
- exit 1
-fi
-
-# Create human-readable names
-target_arch=$2
-
-set_arch="--host-arch $target_arch"
-
-# Go to build directory
-cd "$1"
-
-# To avoid Perl locale warnings:
-export LC_ALL=C
-export LANG=C
-export LANGUAGE=C
-
-# allow for changes to the PATH variable
-export PATH=$PATH_PREPEND:$PATH
diff --git a/meta/recipes-devtools/buildchroot/files/configscript.sh b/meta/recipes-devtools/buildchroot/files/configscript.sh
deleted file mode 100644
index c4d8cf83..00000000
--- a/meta/recipes-devtools/buildchroot/files/configscript.sh
+++ /dev/null
@@ -1,15 +0,0 @@
-#!/bin/sh
-#
-# This software is a part of ISAR.
-# Copyright (C) 2015-2018 ilbers GmbH
-
-set -e
-
-debconf-set-selections <<END
-locales locales/locales_to_be_generated multiselect en_US.UTF-8 UTF-8
-locales locales/default_environment_locale select en_US.UTF-8
-END
-
-grep -q "builder:x:$2" /etc/group || groupadd -f --system builder -o --gid $2
-grep -q "builder:x:$1" /etc/passwd || useradd --system -o --uid $1 --gid builder --no-create-home --home /home/builder --no-user-group --comment "Isar buildchroot build user" builder
-chown -R builder:builder /home/builder
diff --git a/meta/recipes-devtools/buildchroot/files/deps.sh b/meta/recipes-devtools/buildchroot/files/deps.sh
deleted file mode 100644
index ccfc460c..00000000
--- a/meta/recipes-devtools/buildchroot/files/deps.sh
+++ /dev/null
@@ -1,47 +0,0 @@
-#!/bin/bash
-#
-# This software is a part of ISAR.
-# Copyright (C) 2015-2017 ilbers GmbH
-# Copyright (c) 2018 Siemens AG
-
-source /isar/common.sh
-
-# Install command to be used by mk-build-deps
-# Notes:
-# 1) everything before the -y switch is unchanged from the defaults
-# 2) we add -y to go non-interactive
-# 3) downgrades shall be allowed in case a package recipe was changed
-install_cmd="apt-get -o Debug::pkgProblemResolver=yes --no-install-recommends \
- -y --allow-downgrades $3"
-
-if [ "$3" != "--download-only" ]; then
- # Make sure that we have latest isar-apt content.
- # Options meaning:
- # Dir::Etc::SourceList - specifies which source to be used
- # Dir::Etc::SourceParts - disables looking for the other sources
- # APT::Get::List-Cleanup - do not erase obsolete packages list for
- # upstream in '/var/lib/apt/lists'
- apt-get update \
- -o Dir::Etc::SourceList="sources.list.d/isar-apt.list" \
- -o Dir::Etc::SourceParts="-" \
- -o APT::Get::List-Cleanup="0"
-fi
-
-control_file=$(pwd)/debian/control
-cd ..
-
-# Install all build deps
-if [ "$3" = "--download-only" ]; then
- # this will not return 0 even when it worked
- mk-build-deps $set_arch -t "${install_cmd}" -i -r $control_file &> \
- mk-build-deps.output || true
- cat mk-build-deps.output
- # we assume success when we find this
- grep "mk-build-deps: Unable to install all build-dep packages" mk-build-deps.output
- rm -f mk-build-deps.output
-else
- mk-build-deps $set_arch -t "${install_cmd}" -i -r $control_file
-
- # Upgrade any already installed packages in case we are partially rebuilding
- apt-get upgrade -y --allow-downgrades
-fi
diff --git a/scripts/contrib/convert-overrides.py b/scripts/contrib/convert-overrides.py
index 6551aaa8..57f166ca 100755
--- a/scripts/contrib/convert-overrides.py
+++ b/scripts/contrib/convert-overrides.py
@@ -65,7 +65,7 @@ skips = skips + ["self._removePatch", "self._removePatchFile", "self._appendPatc
imagevars = ["IMAGE_CMD", "EXTRA_IMAGECMD", "IMAGE_TYPEDEP", "CONVERSION_CMD", "COMPRESS_CMD"]
imagevars = imagevars + ["TOOLCHAIN", "BOOTSTRAP_SRC", "IMAGE_FSTYPES", "IMAGE_SRC_URI", "IMAGE_TEMPLATE_FILES", "IMAGE_TEMPLATE_VARS", "IMAGE_CMD_REQUIRED_ARGS"]
-imagevars = imagevars + ["IMAGE_FULLNAME", "CONVERSION_DEPS", "BUILDCHROOT_PREINSTALL", "IMAGER_BUILD_DEPS", "IMAGER_INSTALL"]
+imagevars = imagevars + ["IMAGE_FULLNAME", "CONVERSION_DEPS", "IMAGER_BUILD_DEPS", "IMAGER_INSTALL"]
packagevars = packagevars + imagevars
vars_re = {}
--
2.34.1
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH v5 7/9] dpkg-gbp: Remove unused mounts
2023-05-31 11:12 [PATCH v5 0/9] Imager schroot migration Anton Mikanovich
` (5 preceding siblings ...)
2023-05-31 11:12 ` [PATCH v5 6/9] meta: Remove buildchroot Anton Mikanovich
@ 2023-05-31 11:12 ` Anton Mikanovich
2023-05-31 11:12 ` [PATCH v5 8/9] doc: Update chroot documentation Anton Mikanovich
` (2 subsequent siblings)
9 siblings, 0 replies; 11+ messages in thread
From: Anton Mikanovich @ 2023-05-31 11:12 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
Cleanup unused mountpoints left from previous sbuild implementations.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
meta/classes/dpkg-gbp.bbclass | 2 --
1 file changed, 2 deletions(-)
diff --git a/meta/classes/dpkg-gbp.bbclass b/meta/classes/dpkg-gbp.bbclass
index 16473749..2bd09eaa 100644
--- a/meta/classes/dpkg-gbp.bbclass
+++ b/meta/classes/dpkg-gbp.bbclass
@@ -11,8 +11,6 @@ PATCHTOOL ?= "git"
GBP_EXTRA_OPTIONS ?= "--git-pristine-tar"
-SCHROOT_MOUNTS = "${WORKDIR}:${PP} ${GITDIR}:/home/.git-downloads"
-
dpkg_runbuild:prepend() {
sh -c "
cd ${WORKDIR}/${PPS}
--
2.34.1
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH v5 8/9] doc: Update chroot documentation
2023-05-31 11:12 [PATCH v5 0/9] Imager schroot migration Anton Mikanovich
` (6 preceding siblings ...)
2023-05-31 11:12 ` [PATCH v5 7/9] dpkg-gbp: Remove unused mounts Anton Mikanovich
@ 2023-05-31 11:12 ` Anton Mikanovich
2023-05-31 11:12 ` [PATCH v5 9/9] RECIPE-API-CHANGELOG.md: Describe imager changes Anton Mikanovich
2023-07-13 13:33 ` [PATCH v5 0/9] Imager schroot migration Uladzimir Bely
9 siblings, 0 replies; 11+ messages in thread
From: Anton Mikanovich @ 2023-05-31 11:12 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
Describe schroot rootfs instead of buildchroot.
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
doc/offline.md | 2 +-
doc/technical_overview.md | 107 ++++++++++++++++++--------------------
doc/user_manual.md | 53 +++++++++++--------
3 files changed, 84 insertions(+), 78 deletions(-)
diff --git a/doc/offline.md b/doc/offline.md
index cc1e4e39..f9cb479b 100644
--- a/doc/offline.md
+++ b/doc/offline.md
@@ -70,7 +70,7 @@ Repeat for `/etc/apache2/sites-available/default-ssl.conf`.
== Set Up BIND
-A DNS server is necessary to resolve `localhost` from `buildchroot-target`.
+A DNS server is necessary to resolve `localhost` from `sbuild-chroot-target`.
```
sudo apt-get install bind9
diff --git a/doc/technical_overview.md b/doc/technical_overview.md
index ce9cee18..0f491c1a 100644
--- a/doc/technical_overview.md
+++ b/doc/technical_overview.md
@@ -62,41 +62,39 @@ hello-isar demo application will be processed 3 times for each
environment. Three different sets of stamps will be used to distinguish
build progress between different bitbake configurations.
-## 2.3 Buildchroot
+## 2.3 Schroot rootfs
+Also called sbuild chroot rootfs, because of the usage by sbuild tool.
One of the key aspect of Debian philosophy claims the fact, that everything in
Debian should be built within Debian environment. To follow this rule, Isar
-introduces the new component - buildchroot. Bulidchroot is typical Debian
-filesystem that is created using standard Debian tools: debootstrap, apt. The
-source of packages can be either official Debian repositories or custom
-repositories created by user. Buildchroot could be one of the following type:
+introduces the new component - schroot rootfs. Schroot rootfs is typical Debian
+filesystem that is created using standard Debian tools: debootstrap, apt-get.
+The source of packages can be either official Debian repositories or custom
+repositories created by user. It could be one of the following type:
- - `buildchroot-host` with the host architecture for cross-compilation.
+ - `sbuild-chroot-host` with the host architecture for cross-compilation.
- - `buildchroot-target` with the target architecture for target native
+ - `sbuild-chroot-target` with the target architecture for target native
compilation.
-Buildchroot lifecycle can be described as following:
+Schroot rootfs lifecycle can be described as following:
- - Buildchroot starts out with a minimal base system that was generated by the
- isar-bootstrap-[host,target] recipe that uses debootstrap and apt.
+ - Schroot rootfs starts out with a minimal base system that was generated by
+ the isar-bootstrap-{host,target} recipe that uses debootstrap and apt-get.
- - During building custom Debian package, list of its build dependencies is
- installed to buildchroot.
-
- - When package has been built, it's installed to current buildchroot to
- satisfy further packages build dependencies.
+ - Common development packages needed for packages building are installed on
+ top of this rootfs.
## 2.4 Target Root Filesystem
-Target filesystem is quite similar to buildchroot. The only difference is that
-it doesn't have development packages installed and always has target
+Target filesystem is quite similar to schroot rootfs. The only difference is
+that it doesn't have development packages installed and always has target
architecture.
Target filesystem lifecycle can be described as following:
- - Target filesystem starts, like the buildchroot, with the same minimal base
- system generated by the isar-bootstrap recipe.
+ - Target filesystem starts, like the schroot rootfs, with the same minimal
+ base system generated by the isar-bootstrap recipe.
- According to the list of custom packages in bitbake recipes, the initial
filesystem will be populated by successfully built packages from locally
@@ -112,7 +110,7 @@ Whole Isar build process can be split into the following steps:
apt for each configuration requested by the user (Debian distro, machine and
architecture).
- - Generation of initial buildchroots for each configuration.
+ - Generation of initial schroot rootfs for each configuration.
- Generation of initial target filesystems for each configuration.
@@ -157,45 +155,42 @@ This recipe implements several tasks which are executed in this order:
minimal base system, that can be used by the other recipes, that depend on
isar-bootstrap.
-## 3.3 Initial Buildchroot Generation
+## 3.3 Initial Schroot Rootfs Generation
-As mentioned above, initial buildchroot is generated using the minimal base
+As mentioned above, initial schroot rootfs is generated using the minimal base
system generated by isar-bootstrap. The bitbake recipe which is responsible for
-buildchroot can be found here:
-`meta/recipes-devtools/buildchroot/buildchroot-[host,target].bb`
+schroot rootfs can be found here:
+`meta/recipes-devtools/sbuild-chroot/sbuild-chroot-{host,target}.bb`
Section 3.8 describes when host or target architecture is used, please refer
to it for more details.
-This recipe implements `do_build` task which performs the following:
+This recipe implements rootfs preparation for the next schroot usage and
+performs the following:
+
+1. Calls `rootfs_prepare` helper function that copies the minimal base system
+ to the schroot rootfs working directory.
-1. Calls the `setup_root_file_system` helper function defined in
- `meta/classes/isar-bootstrap-helper.bbclass` that copies the minimal base
- system to the buildchroot working directory, sets up the isar-apt repository
- which will later contain all isar-build packages, calls update on just this
- repository and installs all packages given via a parameter to this system.
- Those are, in buildchroots case, the basic package building toolchain.
+2. Sets up the isar-apt repository which will later contain all isar-build
+ packages. Also configure apt to retry and not install recommended packages.
-2. Install script for building custom Debian packages:
- `meta/recipes-devtools/buildchroot/files/build.sh`.
+3. Mounts dev, proc, sys and also isar-apt, base-apt into schroot rootfs.
-3. Installs and executes a configuration script, that further customizes the
- buildchroot. For instance it sets the locale settings.
+4. Calls apt-get update, imports package download cache and installs the
+ packages specified as an argument to this rootfs. Those are, in schroot
+ rootfs case, the basic package building toolchain.
-The single stamp is created for each user buildchroot configuration.
+The single stamp is created for each user schroot rootfs configuration.
## 3.4 Initial Target Filesystem Generation
-Initial target filesystem generation process is very similar to buildchroot
+Initial target filesystem generation process is very similar to schroot rootfs
creating, the difference is only in initial packages list.
Target image recipes are the part of Isar core. There is a sample of typical
Isar image that can be customized according to the user requirements:
`meta-isar/recipes-core/images/isar-image-base.bb`.
-Like for buildchroot, the configuration files for image can be found
-here: `meta-isar/recipes-core/images/files`, and it implements `do_build` task.
-
## 3.5 Building Custom Packages
Isar provides the possibility to build Debian packages from sources. This
@@ -217,12 +212,12 @@ Both consist of the following steps:
3. Task `do_install` _only_ for `dpkg-raw`: copy all you want in your
debian package to `${D}`, install hooks in `${D}/DEBIAN`
-4. Task `do_prepare': perform any preparation steps to the unpacked/patched
+4. Task `do_prepare`: Perform any preparation steps to the unpacked/patched
sources before the build. This task calls the dpkg_prepare shell function
- with the buildchroot mounts in place (`dpkg_do_mounts')
+ with the schroot rootfs mounts in place (`dpkg_do_mounts`).
4.1. the `dpkg_prepare` function of `dpkg.bbclass` runs `/isar/deps.sh` in
- the buildchroot. That performs the following:
+ the chroot rootfs. That performs the following:
1. Go to `/home/build/${PN}`
@@ -232,12 +227,12 @@ Both consist of the following steps:
recipe meta-data into a debian/control file suitable for packaging
with dpkg-deb
-5. Task `do_build`: mount folder (`dpkg_do_mounts') with unpacked files to buildchroot,
- execute the actual build function `dpkg_runbuild`, and finally umount again
- (`dpkg_undo_mounts')
+5. Task `do_build`: Mount folder (`dpkg_do_mounts`) with unpacked files to
+ schroot rootfs, execute the actual build function `dpkg_runbuild`, and
+ finally unmount again (`dpkg_undo_mounts`).
5.1. the `dpkg_runbuild` function of `dpkg.bbclass` runs `build.sh` in the
- buildchroot. That performs the following:
+ schroot rootfs. That performs the following:
1. Go to `/home/build/${PN}`
@@ -274,15 +269,15 @@ This process contains the following steps:
## 3.8 Cross-compilation
-Isar provides possibility to cross-compile all or individual packages. There is
-variable ISAR_CROSS_COMPILE that could be set globaly for whole Isar build,
-but also it could be overwritten from the individual recipe to force specific
-compilation mode for it. This variable influences on which buildchroot will be
-used for recipe compilation. Isar is free from packages modification to perform
-cross-compilation, this is done by the same tools as for native compilation.
-Depending on ISAR_CROSS_COMPILE value, additional architecture specifiers are
-passed to build tools automatically, so this is absolutely transparent from
-the user point of view.
+Isar provides the possibility to cross-compile all or individual packages. If
+`ISAR_CROSS_COMPILE` variable is set to 0, the native toolchain will be used
+(e.g., when building for arm64 on amd64, the arm64 gcc binaries will be
+executed in an emulated environment). If set to 1, the cross-toolchain will be
+used (the amd64 cross-gcc binaries will be executed). The default is 0. The
+variable can be set globally for the whole build (e.g., in `local.conf` or
+multiconfig files) or overridden in individual recipes. Cross-compilation
+support must be implemented in source packages. Debian's standard
+cross-toolchains are used for cross-building.
## 3.9 Additional features
diff --git a/doc/user_manual.md b/doc/user_manual.md
index e07b76d8..cf818e59 100644
--- a/doc/user_manual.md
+++ b/doc/user_manual.md
@@ -34,7 +34,9 @@ Isar provides:
- Fast target image generation: About 10 minutes to get base system image for one machine.
- Use any apt package provider, including open-source communities like `Debian`, `Raspbian`, etc. and proprietary ones created manually.
- - Native compilation: Packages are compiled in a `chroot` environment using the same toolchain and libraries that will be installed to the target filesystem.
+ - Native compilation: Packages are compiled in a `schroot` environment using
+ the same toolchain and libraries that will be installed to the target
+ filesystem.
- Cross compilation: Could be enabled, when native compilation from the sources takes a lot of time f.e. for Linux kernel.
- Product templates that can be quickly re-used for real projects.
@@ -111,7 +113,7 @@ unstable/bullseye included in `/etc/apt/sources.list[.d]`).
Notes:
-* BitBake requires Python 3.4+.
+* BitBake requires Python 3.6+.
* The python3 package is required for the correct `alternatives` setting.
* If you'd like to run bitbake in a container (chroot, docker, etc.), install
the above in the container, and also perform `sudo apt-get install
@@ -322,6 +324,11 @@ docker run --rm -ti --volume "$(pwd):/build" isar-image-base-debian-buster-armhf
`chroot`(8) runs a command within a specified root directory. Please refer to GNU coreutils online help: <http://www.gnu.org/software/coreutils/> for more information.
+### Schroot
+
+Schroot allows the user to run a command in a chroot environment specified by
+root directory or previously opened session.
+
### QEMU
QEMU is a generic and open source machine emulator and virtualizer. Please refer to <http://wiki.qemu.org/Main_Page> for more information.
@@ -344,7 +351,7 @@ BitBake is a generic task execution engine for efficient execution of shell and
Isar workflow consists of stages described below.
-### Generation of Buildchroot Filesystem
+### Generation of Schroot Filesystem
This filesystem is used as a build environment to compile custom packages. It is generated using `apt` binaries repository, selected by the user in configuration file. Please refer to distro configuration chapter for more information.
@@ -354,7 +361,9 @@ During this stage Isar processes custom packages selected by the user and genera
### Generation of Basic Target Filesystem
-This filesystem is generated similarly to the `buildchroot` one using the `apt` binaries repository. Please refer to distro configuration chapter for more information.
+This filesystem is generated similarly to the `schroot` one using the `apt`
+binaries repository. Please refer to distro configuration chapter for more
+information.
### Install Custom Packages
@@ -426,7 +435,9 @@ Some other variables include:
## Isar Distro Configuration
-In Isar, each machine can use its specific Linux distro to generate `buildchroot` and target filesystem. By default, Isar provides configuration files for the following distros:
+In Isar, each machine can use its specific Linux distro to generate `schroot`
+and target filesystem. By default, Isar provides configuration files for the
+following distros:
- debian-buster
- debian-bullseye
@@ -604,12 +615,12 @@ The IMAGE_CMD is a shell function, and the environment has some pre-set
variables:
- `IMAGE_FILE_HOST` and `IMAGE_FILE_CHROOT` are the paths of the output image
- (including extension) in the host or buildchroot.
+ (including extension) in the host or schroot rootfs.
- `SUDO_CHROOT` is a prefix you can use to have a command run inside the
- buildchroot.
+ imager schroot rootfs.
If the code you provide in `IMAGE_CMD` requires the building and/or installation
-of additional packages in the buildchroot, you can specify this:
+of additional packages in the imager schroot rootfs, you can specify this:
```
IMAGER_BULID_DEPS:my_image = "my_command"
IMAGER_INSTALL:my_image = "my_command"
@@ -973,7 +984,7 @@ concept.
Isar can build 32-bit packages as part of a 64-bit image build and also enable
the image with the necessary packages. To activate compat support,
set `ISAR_ENABLE_COMPAT_ARCH = "1"` in `local.conf`. This will install necessary
-build dependencies in the buildchroot.
+build dependencies in the schroot rootfs.
For all dpkg package recipes, Isar automatically provides a `<package>-compat`
target that builds the package for the `COMPAT_DISTRO_ARCH`. This can be
@@ -1043,7 +1054,7 @@ Now the image should be up again and `modprobe example-module` should work.
### Cross Support for Imagers
If `ISAR_CROSS_COMPILE = "1"`, the imager and optional compression tasks
-run in the host buildchroot instead of the target buildchroot.
+run in the host schroot rootfs instead of the target one.
This gives a significant speedup when compressing the generated image,
as the compression is not emulated.
@@ -1051,10 +1062,10 @@ In case your setup does not support cross-imaging, you can disable this
just for the particular image by adding `ISAR_CROSS_COMPILE = "0"` to your
image recipe.
-## Examining and debugging package generation inside their buildchroot
+## Examining and debugging package generation inside their schroot rootfs
Just like OpenEmbedded, Isar supports a devshell target for all dpkg package
-recipes. This target opens a terminal inside the buildchroot that runs the
+recipes. This target opens a terminal inside the schroot rootfs that runs the
package build. To invoke it, just call
`bitbake mc:${MACHINE}-${DISTRO}:<package_name> -c devshell`.
@@ -1087,12 +1098,12 @@ feature known from OpenEmbedded. Isar caches
* the Debian bootstrap (`isar-bootstrap` recipe)
* Debian packages (built with the `dpkg` or `dpkg-raw` classes)
- * root file systems (buildchroot and image rootfs)
+ * root file systems (schroot and image rootfs)
The location of the sstate-cache is controlled by the variable `SSTATE_DIR`
and defaults to `${TMPDIR}/sstate-cache`.
-Note that cached rootfs artifacts (bootstrap and buildchroot) have a limited
+Note that cached rootfs artifacts (bootstrap and schroot rootfs) have a limited
"lifetime": Isar updates their package lists for the upstream package sources
only once, when they are initially created. So as packages on the upstream
mirrors change, those lists will be out-of-date and the rootfs becomes useless.
@@ -1392,14 +1403,14 @@ to have Debian source packages as well.
- Trigger download of Debian source packages as part of rootfs postprocess.
-With the current base-apt implementation, we already cache all the binary packages that
-we download and install onto the target rootfs and buildchroot. This is then used to
-generate a local-apt for offline build.
+With the current base-apt implementation, we already cache all the binary
+packages that we download and install onto the target rootfs and schroot
+rootfs. This is then used to generate a local-apt for offline build.
-Use rootfs postprocessing to parse through the list of deb files in ${DEBDIR} and
-download the corresponding Debian source file using "apt-get source" command.
-This caches the sources of all the Debian packages that are downloaded and installed onto
-the target rootfs and buildchroots.
+Use rootfs postprocessing to parse through the list of deb files in ${DEBDIR}
+and download the corresponding Debian source file using "apt-get source"
+command. This caches the sources of all the Debian packages that are downloaded
+and installed onto the target rootfs and schroot rootfs.
By default, the Debian source caching is not enabled.
To enable it, add the below line to your local.conf file.
--
2.34.1
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH v5 9/9] RECIPE-API-CHANGELOG.md: Describe imager changes
2023-05-31 11:12 [PATCH v5 0/9] Imager schroot migration Anton Mikanovich
` (7 preceding siblings ...)
2023-05-31 11:12 ` [PATCH v5 8/9] doc: Update chroot documentation Anton Mikanovich
@ 2023-05-31 11:12 ` Anton Mikanovich
2023-07-13 13:33 ` [PATCH v5 0/9] Imager schroot migration Uladzimir Bely
9 siblings, 0 replies; 11+ messages in thread
From: Anton Mikanovich @ 2023-05-31 11:12 UTC (permalink / raw)
To: isar-users; +Cc: Anton Mikanovich
---
RECIPE-API-CHANGELOG.md | 20 ++++++++++++++++++++
1 file changed, 20 insertions(+)
diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md
index 7a16b5c2..55fd9037 100644
--- a/RECIPE-API-CHANGELOG.md
+++ b/RECIPE-API-CHANGELOG.md
@@ -492,3 +492,23 @@ Recipes inheriting dpkg-base now automatically have a bitbake target
exists for the current `DISTRO_ARCH`.
In that case the compat package can be built by adding `<foo>-compat`
to `DEPENDS` or `IMAGE_INSTALL`.
+
+### Imager is now executed inside schroot
+
+Buildchroot is completely removed and can't be used any more.
+To execute imager code new `imager_run` API was created.
+
+So older style call:
+```
+sudo chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} cmd_to_execute
+```
+Can now be performed by:
+```
+imager_run -p -d ${PP_WORK} cmd_to_execute
+```
+If privileged execution is required `-u root` option can be added.
+
+`image_do_mounts` is removed, additional mountpoints can be added like:
+```
+SCHROOT_MOUNTS += "${OUT_PATH1}:${IN_PATH1} ${OUT_PATH2}"
+```
--
2.34.1
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v5 0/9] Imager schroot migration
2023-05-31 11:12 [PATCH v5 0/9] Imager schroot migration Anton Mikanovich
` (8 preceding siblings ...)
2023-05-31 11:12 ` [PATCH v5 9/9] RECIPE-API-CHANGELOG.md: Describe imager changes Anton Mikanovich
@ 2023-07-13 13:33 ` Uladzimir Bely
9 siblings, 0 replies; 11+ messages in thread
From: Uladzimir Bely @ 2023-07-13 13:33 UTC (permalink / raw)
To: isar-users; +Cc: Jan Kiszka
On Wed, 2023-05-31 at 14:12 +0300, Anton Mikanovich wrote:
> This patchset moves Isar imager from buildchroot to schroot usage as
> the next
> step after moving to sbuild.
>
> Imagers of every target are now running in separate schroot sessions
> with
> independent overlays on top of basic unchanged schroot. Any changes
> made inside
> overlay will be dropped after the usage to keep basic schroot clean.
>
> Changes since v4:
> - Rebased on next
> - Add RECIPE-API-CHANGELOG entry
> - Remove unused mounts from dpkg-gbp
>
> Changes since v3:
> - Rebased on next
> - Fixed full CI build issues
> - Add documentation updates
> - Fix task name in CI
> - Some code cleanup
>
> Changes since v2:
> - Rebased on next
> - Do not keep schroot session open outside tasks
> - Get rid of final cleanup code
> - Fix sstate test case
> - Get back buildchroot removing
>
> Changes since v1:
> - Rebased on next and bitbake update
> - Fix some build cases
>
> Anton Mikanovich (9):
> isar-apt: Move cleanup to postprocessing
> imager: Migrate from buildchroot to schroot
> imager: Move image types to schroot
> imager: Split imager deps between image types
> CI: Fix used chroot recipe name
> meta: Remove buildchroot
> dpkg-gbp: Remove unused mounts
> doc: Update chroot documentation
> RECIPE-API-CHANGELOG.md: Describe imager changes
>
> RECIPE-API-CHANGELOG.md | 20 ++++
> doc/offline.md | 2 +-
> doc/technical_overview.md | 107 +++++++++-------
> --
> doc/user_manual.md | 53 +++++----
> meta-isar/conf/local.conf.sample | 2 +-
> meta/classes/buildchroot.bbclass | 59 ----------
> meta/classes/crossvars.bbclass | 6 -
> meta/classes/deb-dl-dir.bbclass | 4 +-
> meta/classes/dpkg-gbp.bbclass | 2 -
> meta/classes/image-tools-extension.bbclass | 99 +++++++++++-----
> meta/classes/image.bbclass | 35 +++---
> meta/classes/imagetypes_vm.bbclass | 9 +-
> meta/classes/imagetypes_wic.bbclass | 89 +++++----------
> meta/classes/isar-events.bbclass | 1 -
> meta/classes/rootfs.bbclass | 12 ++
> meta/conf/bitbake.conf | 3 -
> .../buildchroot/buildchroot-host.bb | 13 ---
> .../buildchroot/buildchroot-target.bb | 10 --
> .../buildchroot/buildchroot.inc | 73 ------------
> .../buildchroot/files/build.sh | 17 ---
> .../buildchroot/files/common.sh | 32 ------
> .../buildchroot/files/configscript.sh | 15 ---
> .../buildchroot/files/deps.sh | 47 --------
> .../sbuild-chroot/sbuild-chroot.inc | 1 +
> scripts/contrib/convert-overrides.py | 2 +-
> testsuite/cibase.py | 8 +-
> 26 files changed, 238 insertions(+), 483 deletions(-)
> delete mode 100644 meta/classes/buildchroot.bbclass
> delete mode 100644 meta/recipes-devtools/buildchroot/buildchroot-
> host.bb
> delete mode 100644 meta/recipes-devtools/buildchroot/buildchroot-
> target.bb
> delete mode 100644 meta/recipes-devtools/buildchroot/buildchroot.inc
> delete mode 100644 meta/recipes-devtools/buildchroot/files/build.sh
> delete mode 100644 meta/recipes-devtools/buildchroot/files/common.sh
> delete mode 100644 meta/recipes-
> devtools/buildchroot/files/configscript.sh
> delete mode 100644 meta/recipes-devtools/buildchroot/files/deps.sh
>
> --
> 2.34.1
>
Before sendingv6 of the patchset, we would like to propose a patchset
for meta-iot2050 that makes it compatible with new schroot-based
imager. The patchset relays on SBUILD_FLAVOR feature that was recently
added in isar.
The WIP patchset for meta-iot2050 can be found at
https://github.com/WiseLord/meta-iot2050/commits/master
For now I don't send it as a pull request since p1 "Update isar
revision to next" will be changed later since meta-iot2050 is temporary
not compatible with recent isar 'next'. This is due to missing patch
https://groups.google.com/g/isar-users/c/Sfd2fYMLXxg that is going to
be merged soon.
^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2023-07-13 13:33 UTC | newest]
Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-05-31 11:12 [PATCH v5 0/9] Imager schroot migration Anton Mikanovich
2023-05-31 11:12 ` [PATCH v5 1/9] isar-apt: Move cleanup to postprocessing Anton Mikanovich
2023-05-31 11:12 ` [PATCH v5 2/9] imager: Migrate from buildchroot to schroot Anton Mikanovich
2023-05-31 11:12 ` [PATCH v5 3/9] imager: Move image types " Anton Mikanovich
2023-05-31 11:12 ` [PATCH v5 4/9] imager: Split imager deps between image types Anton Mikanovich
2023-05-31 11:12 ` [PATCH v5 5/9] CI: Fix used chroot recipe name Anton Mikanovich
2023-05-31 11:12 ` [PATCH v5 6/9] meta: Remove buildchroot Anton Mikanovich
2023-05-31 11:12 ` [PATCH v5 7/9] dpkg-gbp: Remove unused mounts Anton Mikanovich
2023-05-31 11:12 ` [PATCH v5 8/9] doc: Update chroot documentation Anton Mikanovich
2023-05-31 11:12 ` [PATCH v5 9/9] RECIPE-API-CHANGELOG.md: Describe imager changes Anton Mikanovich
2023-07-13 13:33 ` [PATCH v5 0/9] Imager schroot migration Uladzimir Bely
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox