* [PATCH v3 0/7] Migrate to mmdebstrap @ 2023-05-31 10:10 Anton Mikanovich 2023-05-31 10:10 ` [PATCH v3 1/7] meta: Switch " Anton Mikanovich ` (7 more replies) 0 siblings, 8 replies; 16+ messages in thread From: Anton Mikanovich @ 2023-05-31 10:10 UTC (permalink / raw) To: isar-users; +Cc: Anton Mikanovich Switch Isar from debootstrap to mmdebstrap. It makes isar-bootstrap target ~10% faster on both cross and native rootfs types. Also mmdebstrap support using SOURCE_DATE_EPOCH for images reproducibly out of the box. But the main advantage is an ability to remove sudo for debootstrapping later on. This change brakes Debian Buster support as host distro. Changes since v2: - Rebase on next - Fix commit messages - Update host requirements Changes since v1: - Move mmdebstrap tmpdir to workdir - Fix kas building - Improve mounts cleanup Anton Mikanovich (5): isar-bootstrap: Move preparations to hooks isar-bootstrap: Use tar output instead of directory user_manual.md: Replace debootstrap with mmdebstrap user_manual.md: Update minimal host requirements isar-bootstrap: Remove unused code Uladzimir Bely (2): meta: Switch to mmdebstrap CI: Install mmdebstrap doc/user_manual.md | 41 +-- meta-isar/conf/distro/ubuntu-common.inc | 4 - meta/classes/rootfs.bbclass | 11 +- .../isar-bootstrap/isar-bootstrap.inc | 274 ++++++++---------- scripts/ci_build.sh | 6 + 5 files changed, 152 insertions(+), 184 deletions(-) -- 2.34.1 ^ permalink raw reply [flat|nested] 16+ messages in thread
* [PATCH v3 1/7] meta: Switch to mmdebstrap 2023-05-31 10:10 [PATCH v3 0/7] Migrate to mmdebstrap Anton Mikanovich @ 2023-05-31 10:10 ` Anton Mikanovich 2023-05-31 10:10 ` [PATCH v3 2/7] isar-bootstrap: Move preparations to hooks Anton Mikanovich ` (6 subsequent siblings) 7 siblings, 0 replies; 16+ messages in thread From: Anton Mikanovich @ 2023-05-31 10:10 UTC (permalink / raw) To: isar-users; +Cc: Uladzimir Bely, Anton Mikanovich From: Uladzimir Bely <ubely@ilbers.de> Instead of debootstrap, use mmdebstrap alternative. Internally, it uses apt and allows to debootstrap the distro from multiple repositories. Signed-off-by: Uladzimir Bely <ubely@ilbers.de> Signed-off-by: Anton Mikanovich <amikan@ilbers.de> --- meta-isar/conf/distro/ubuntu-common.inc | 4 -- .../isar-bootstrap/isar-bootstrap.inc | 57 ++++++++++--------- 2 files changed, 30 insertions(+), 31 deletions(-) diff --git a/meta-isar/conf/distro/ubuntu-common.inc b/meta-isar/conf/distro/ubuntu-common.inc index becb7e0a..510e302c 100644 --- a/meta-isar/conf/distro/ubuntu-common.inc +++ b/meta-isar/conf/distro/ubuntu-common.inc @@ -16,10 +16,6 @@ BOOTSTRAP_KEY = "file://${LAYERDIR_isar}/conf/distro/ubuntu.public.key;sha256sum DISTRO_BOOTSTRAP_KEYS += "${BOOTSTRAP_KEY}" HOST_DISTRO_BOOTSTRAP_KEYS += "${BOOTSTRAP_KEY}" - -# that is what debootstrap_1.0.118ubuntu1 does anyways -DISTRO_DEBOOTSTRAP_SCRIPT = "/usr/share/debootstrap/scripts/gutsy" - DISTRO_CONFIG_SCRIPT ?= "debian-configscript.sh" DISTRO_KERNELS ?= " \ diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc index b94ae0bd..f89c284e 100644 --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc @@ -15,7 +15,7 @@ PV = "1.0" BOOTSTRAP_FOR_HOST ?= "0" -DEBOOTSTRAP ?= "qemu-debootstrap" +DEBOOTSTRAP = "mmdebstrap" ROOTFSDIR = "${WORKDIR}/rootfs" APTPREFS = "${WORKDIR}/apt-preferences" APTSRCS = "${WORKDIR}/apt-sources" @@ -23,12 +23,14 @@ APTSRCS_INIT = "${WORKDIR}/apt-sources-init" DISTRO_BOOTSTRAP_KEYFILES = "" THIRD_PARTY_APT_KEYFILES = "" DEPLOY_ISAR_BOOTSTRAP ?= "" -DISTRO_BOOTSTRAP_BASE_PACKAGES = "locales" +DISTRO_BOOTSTRAP_BASE_PACKAGES = "locales,usrmerge" DISTRO_BOOTSTRAP_BASE_PACKAGES:append:gnupg = ",gnupg" DISTRO_BOOTSTRAP_BASE_PACKAGES:append:https-support = ",ca-certificates" DISTRO_VARS_PREFIX ?= "${@'HOST_' if d.getVar('BOOTSTRAP_FOR_HOST') == '1' else ''}" BOOTSTRAP_DISTRO = "${@d.getVar('HOST_DISTRO' if d.getVar('BOOTSTRAP_FOR_HOST') == '1' else 'DISTRO')}" BOOTSTRAP_BASE_DISTRO = "${@d.getVar('HOST_BASE_DISTRO' if d.getVar('BOOTSTRAP_FOR_HOST') == '1' else 'BASE_DISTRO')}" +BOOTSTRAP_DISTRO_ARCH = "${@d.getVar('HOST_ARCH' if d.getVar('BOOTSTRAP_FOR_HOST') == '1' else 'DISTRO_ARCH')}" +BOOTSTRAP_TMPDIR = "${WORKDIR}/tempdir" FILESEXTRAPATHS:append = ":${BBPATH}" inherit deb-dl-dir @@ -270,7 +272,7 @@ do_bootstrap[vardeps] += " \ ISAR_ENABLE_COMPAT_ARCH \ ${DISTRO_VARS_PREFIX}DISTRO_APT_SOURCES \ " -do_bootstrap[dirs] = "${DEPLOY_DIR_BOOTSTRAP}" +do_bootstrap[dirs] = "${DEPLOY_DIR_BOOTSTRAP} ${BOOTSTRAP_TMPDIR}" do_bootstrap[depends] = "base-apt:do_cache isar-apt:do_cache_config" do_bootstrap[network] = "${TASK_USE_NETWORK_AND_SUDO}" @@ -286,27 +288,31 @@ do_bootstrap() { if [ -f "${DISTRO_BOOTSTRAP_KEYRING}" ]; then debootstrap_args="$debootstrap_args --keyring=${DISTRO_BOOTSTRAP_KEYRING}" fi - if [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" -a -z "${BASE_REPO_KEY}" ]; then - debootstrap_args="$debootstrap_args --no-check-gpg" - fi E="${@ isar_export_proxies(d)}" - export BOOTSTRAP_FOR_HOST debootstrap_args E + export BOOTSTRAP_FOR_HOST - sudo rm -rf --one-file-system "${ROOTFSDIR}" deb_dl_dir_import "${ROOTFSDIR}" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}" + sudo rm -rf --one-file-system "${ROOTFSDIR}" + mkdir -p "${ROOTFSDIR}" + + if [ ! -z "${SOURCE_DATE_EPOCH}" ]; then + export SOURCE_DATE_EPOCH="${SOURCE_DATE_EPOCH}" + fi + + arch_param="--arch=${BOOTSTRAP_DISTRO_ARCH},${DISTRO_ARCH}" + + sudo TMPDIR="${BOOTSTRAP_TMPDIR}" ${DEBOOTSTRAP} $debootstrap_args \ + $arch_param \ + --mode=unshare \ + ${@get_distro_components_argument(d)} \ + "${@get_distro_suite(d)}" \ + "${WORKDIR}/rootfs.tar.zst" \ + "${@get_distro_source(d)}" sudo -E -s <<'EOSUDO' set -e - if [ "${BOOTSTRAP_FOR_HOST}" = "0" ]; then - arch_param="--arch=${DISTRO_ARCH}" - fi - ${DEBOOTSTRAP} $debootstrap_args \ - $arch_param \ - ${@get_distro_components_argument(d)} \ - "${@get_distro_suite(d)}" \ - "${ROOTFSDIR}" \ - "${@get_distro_source(d)}" \ - ${DISTRO_DEBOOTSTRAP_SCRIPT} + + tar -xf "${WORKDIR}/rootfs.tar.zst" -C "${ROOTFSDIR}" --exclude="./dev/console" # Install apt config mkdir -p "${ROOTFSDIR}/etc/apt/preferences.d" @@ -409,19 +415,16 @@ SSTATEPOSTINSTFUNCS += "bootstrap_sstate_finalize" bootstrap_sstate_prepare() { # this runs in SSTATE_BUILDDIR, which will be deleted automatically - lopts="--one-file-system --exclude=var/cache/apt/archives" - sudo tar -C $(dirname "${ROOTFSDIR}") -cpSf bootstrap.tar $lopts $(basename "${ROOTFSDIR}") - sudo chown $(id -u):$(id -g) bootstrap.tar + sudo cp -a "$(dirname "${ROOTFSDIR}")/rootfs.tar.zst" ./bootstrap.tar.zst + sudo chown $(id -u):$(id -g) bootstrap.tar.zst } bootstrap_sstate_finalize() { # this runs in SSTATE_INSTDIR - # - after building the bootstrap, the tar won't be there, but we also don't need to unpack - # - after restoring from cache, there will be a tar which we unpack and then delete - if [ -f bootstrap.tar ]; then - sudo tar -C $(dirname "${ROOTFSDIR}") -xpf bootstrap.tar - sudo ln -Tfsr "${ROOTFSDIR}" "${DEPLOY_ISAR_BOOTSTRAP}" - rm bootstrap.tar + if [ -f bootstrap.tar.zst ]; then + mv bootstrap.tar.zst "$(dirname "${ROOTFSDIR}")/rootfs.tar.zst" + sudo ln -Tfsr "$(dirname "${ROOTFSDIR}")/rootfs.tar.zst" \ + "${DEPLOY_ISAR_BOOTSTRAP}.tar.zst" fi } -- 2.34.1 ^ permalink raw reply [flat|nested] 16+ messages in thread
* [PATCH v3 2/7] isar-bootstrap: Move preparations to hooks 2023-05-31 10:10 [PATCH v3 0/7] Migrate to mmdebstrap Anton Mikanovich 2023-05-31 10:10 ` [PATCH v3 1/7] meta: Switch " Anton Mikanovich @ 2023-05-31 10:10 ` Anton Mikanovich 2023-05-31 10:10 ` [PATCH v3 3/7] isar-bootstrap: Use tar output instead of directory Anton Mikanovich ` (5 subsequent siblings) 7 siblings, 0 replies; 16+ messages in thread From: Anton Mikanovich @ 2023-05-31 10:10 UTC (permalink / raw) To: isar-users; +Cc: Anton Mikanovich Move all bootstrap rootfs prepare logic into mmdebstrap hooks. Signed-off-by: Anton Mikanovich <amikan@ilbers.de> --- .../isar-bootstrap/isar-bootstrap.inc | 196 +++++++++--------- 1 file changed, 102 insertions(+), 94 deletions(-) diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc index f89c284e..fba876ea 100644 --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc @@ -212,24 +212,21 @@ def get_distro_components_argument(d): else: return "" -APT_KEYS_DIR = "${WORKDIR}/aptkeys" DISTRO_BOOTSTRAP_KEYRING = "${WORKDIR}/distro-keyring.gpg" -do_generate_keyrings[cleandirs] = "${APT_KEYS_DIR}" -do_generate_keyrings[dirs] = "${DL_DIR}" +do_generate_keyrings[cleandirs] = "${WORKDIR}/trusted.gpg.d" +do_generate_keyrings[dirs] = "${DEBDIR}" do_generate_keyrings[vardeps] += "DISTRO_BOOTSTRAP_KEYS THIRD_PARTY_APT_KEYS" do_generate_keyrings[network] = "${TASK_USE_SUDO}" do_generate_keyrings() { if [ -n "${@d.getVar("THIRD_PARTY_APT_KEYFILES") or ""}" ]; then - chmod 777 "${APT_KEYS_DIR}" for keyfile in ${@d.getVar("THIRD_PARTY_APT_KEYFILES")}; do - cp "$keyfile" "${APT_KEYS_DIR}"/"$(basename "$keyfile")" + sudo apt-key --keyring "${DISTRO_BOOTSTRAP_KEYRING}" add $keyfile done fi if [ -n "${@d.getVar("DISTRO_BOOTSTRAP_KEYFILES") or ""}" ]; then for keyfile in ${@d.getVar("DISTRO_BOOTSTRAP_KEYFILES")}; do sudo apt-key --keyring "${DISTRO_BOOTSTRAP_KEYRING}" add $keyfile - cp "$keyfile" "${APT_KEYS_DIR}"/"$(basename "$keyfile")" done fi } @@ -272,12 +269,14 @@ do_bootstrap[vardeps] += " \ ISAR_ENABLE_COMPAT_ARCH \ ${DISTRO_VARS_PREFIX}DISTRO_APT_SOURCES \ " -do_bootstrap[dirs] = "${DEPLOY_DIR_BOOTSTRAP} ${BOOTSTRAP_TMPDIR}" +do_bootstrap[dirs] = "${DEPLOY_DIR_BOOTSTRAP} ${BOOTSTRAP_TMPDIR} ${WORKDIR}/trusted.gpg.d ${WORKDIR}/sources.list.d" do_bootstrap[depends] = "base-apt:do_cache isar-apt:do_cache_config" do_bootstrap[network] = "${TASK_USE_NETWORK_AND_SUDO}" inherit compat +DEB_DL_LOCK ?= "${DEBDIR}/${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}.lock" + do_bootstrap() { if [ "${ISAR_ENABLE_COMPAT_ARCH}" = "1" ]; then if [ -z "${COMPAT_DISTRO_ARCH}" ]; then @@ -287,124 +286,133 @@ do_bootstrap() { debootstrap_args="--verbose --variant=minbase --include=${DISTRO_BOOTSTRAP_BASE_PACKAGES}" if [ -f "${DISTRO_BOOTSTRAP_KEYRING}" ]; then debootstrap_args="$debootstrap_args --keyring=${DISTRO_BOOTSTRAP_KEYRING}" + cp "${DISTRO_BOOTSTRAP_KEYRING}" "${WORKDIR}/trusted.gpg.d/" fi E="${@ isar_export_proxies(d)}" - export BOOTSTRAP_FOR_HOST - deb_dl_dir_import "${ROOTFSDIR}" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}" sudo rm -rf --one-file-system "${ROOTFSDIR}" mkdir -p "${ROOTFSDIR}" + if [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" ]; then + base_apt_tmp="$(mktemp -d /tmp/isar-base-aptXXXXXXXXXX)" + bootstrap_list="${WORKDIR}/sources.list.d/base-apt.list" + line="copy://$base_apt_tmp/${BOOTSTRAP_BASE_DISTRO} ${BASE_DISTRO_CODENAME} main" + if [ -z "${BASE_REPO_KEY}" ]; then + line="[trusted=yes] ${line}" + fi + echo "deb ${line}" > "${WORKDIR}/sources.list.d/base-apt.list" + line="copy://$base_apt_tmp/${BASE_DISTRO} ${BASE_DISTRO_CODENAME} main" + if [ -z "${BASE_REPO_KEY}" ]; then + line="[trusted=yes] ${line}" + fi + echo "deb-src ${line}" >> "${WORKDIR}/sources.list.d/base-apt.list" + + # no need to sync /var/cache/apt/archives if base-apt used + syncin='echo skip sync-in' + syncout='echo skip sync-out' + extra_setup="mount --bind '${REPO_BASE_DIR}' $base_apt_tmp" + extra_extract="$syncout" + # save mmdebstrap tempdir for cleanup + extra_essential="mkdir -p \$1/$base_apt_tmp && \ + echo \$1 > ${WORKDIR}/mmtmpdir && \ + mount --bind '${REPO_BASE_DIR}' \$1/$base_apt_tmp" + # replace base-apt mount in tmp with /base-apt mount + extra_customize="sed -i \"s|copy://$base_apt_tmp|file:///base-apt|g\" \ + \$1/etc/apt/sources.list.d/*.list && \ + mkdir -p \$1/base-apt && \ + mount --bind '${REPO_BASE_DIR}' \$1/base-apt && \ + chroot \$1 apt-get update -y \ + -o APT::Update::Error-Mode=any && \ + umount \$1/base-apt && \ + umount \$1/$base_apt_tmp && rm ${WORKDIR}/mmtmpdir && \ + umount $base_apt_tmp && rm -rf --one-file-system $base_apt_tmp" + else + deb_dl_dir_import "${WORKDIR}/dl_dir" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}" + + bootstrap_list="${WORKDIR}/sources.list.d/bootstrap.list" + install -v -m644 "${APTSRCS}" \ + "${WORKDIR}/sources.list.d/bootstrap.list" + + syncin='flock -s ${DEB_DL_LOCK} cp -n --no-preserve=owner \ + "${WORKDIR}/dl_dir/var/cache/apt/archives/"*.deb \ + "$1/var/cache/apt/archives/" || true' + syncout='flock -s ${DEB_DL_LOCK} cp -n --no-preserve=owner \ + "$1/var/cache/apt/archives/"*.deb \ + "${WORKDIR}/dl_dir/var/cache/apt/archives/"' + extra_setup="$syncin" + extra_extract="$syncout" + # prefetch apt debs because mmdebstrap will clean them on next stage + extra_essential='apt-get install apt -y -d \ + -o Dir::State="$1/var/lib/apt" \ + -o Dir::Etc="$1/etc/apt" \ + -o Dir::Cache="$1/var/cache/apt" \ + -o Apt::Architecture="${BOOTSTRAP_DISTRO_ARCH}"' + extra_essential="$extra_essential && $syncout" + extra_customize="$syncout" + fi + if [ ! -z "${SOURCE_DATE_EPOCH}" ]; then export SOURCE_DATE_EPOCH="${SOURCE_DATE_EPOCH}" fi arch_param="--arch=${BOOTSTRAP_DISTRO_ARCH},${DISTRO_ARCH}" + if [ "${ISAR_ENABLE_COMPAT_ARCH}" = "1" ]; then + arch_param="$arch_param,${COMPAT_DISTRO_ARCH}" + fi + + # Cleanup mounts if fails + trap 'exit 1' INT HUP QUIT TERM ALRM USR1 + trap '[ -r "${WORKDIR}/mmtmpdir" ] && tmpdir=$(cat "${WORKDIR}/mmtmpdir") \ + && rm "${WORKDIR}/mmtmpdir"; \ + [ -d "$tmpdir" ] && mountpoint -q $tmpdir/$base_apt_tmp \ + && sudo umount $tmpdir/$base_apt_tmp; \ + [ -d "$tmpdir" ] && mountpoint -q $tmpdir/base-apt \ + && sudo umount $tmpdir/base-apt; \ + [ -d "$tmpdir" ] && sudo rm -rf --one-file-system $tmpdir; \ + [ -n "$base_apt_tmp" ] && mountpoint -q $base_apt_tmp \ + && sudo umount $base_apt_tmp \ + && rm -rf --one-file-system $base_apt_tmp' EXIT sudo TMPDIR="${BOOTSTRAP_TMPDIR}" ${DEBOOTSTRAP} $debootstrap_args \ $arch_param \ --mode=unshare \ + --setup-hook='mkdir -p "$1/var/cache/apt/archives/"' \ + --setup-hook="$extra_setup" \ + --setup-hook='upload "${APTPREFS}" /etc/apt/preferences.d/bootstrap' \ + --setup-hook='upload "${APTSRCS_INIT}" /etc/apt/sources-list' \ + --setup-hook='upload "${WORKDIR}/locale" /etc/locale' \ + --setup-hook='mkdir -p "$1/etc/apt/trusted.gpg.d"' \ + --setup-hook='sync-in "${WORKDIR}/trusted.gpg.d" /etc/apt/trusted.gpg.d' \ + --setup-hook='install -v -m755 "${WORKDIR}/chroot-setup.sh" "$1/chroot-setup.sh"' \ + --extract-hook="$extra_extract" \ + --essential-hook="$extra_essential" \ + --customize-hook="$extra_customize" \ + --customize-hook='sed -i "/en_US.UTF-8 UTF-8/s/^#//g" "$1/etc/locale.gen"' \ + --customize-hook='chroot "$1" /usr/sbin/locale-gen' \ + --customize-hook='chroot "$1" /usr/bin/apt-get -y clean' \ + --skip=cleanup/apt \ + --skip=download/empty \ ${@get_distro_components_argument(d)} \ "${@get_distro_suite(d)}" \ "${WORKDIR}/rootfs.tar.zst" \ - "${@get_distro_source(d)}" + "$bootstrap_list" sudo -E -s <<'EOSUDO' set -e tar -xf "${WORKDIR}/rootfs.tar.zst" -C "${ROOTFSDIR}" --exclude="./dev/console" - # Install apt config - mkdir -p "${ROOTFSDIR}/etc/apt/preferences.d" - install -v -m644 "${APTPREFS}" \ - "${ROOTFSDIR}/etc/apt/preferences.d/bootstrap" - mkdir -p "${ROOTFSDIR}/etc/apt/sources.list.d" - if [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" ]; then - line="file:///base-apt/${BOOTSTRAP_BASE_DISTRO} ${BASE_DISTRO_CODENAME} main" - if [ -z "${BASE_REPO_KEY}" ]; then - line="[trusted=yes] ${line}" - fi - echo "deb ${line}" > "${ROOTFSDIR}/etc/apt/sources.list.d/base-apt.list" - line="file:///base-apt/${BASE_DISTRO} ${BASE_DISTRO_CODENAME} main" - if [ -z "${BASE_REPO_KEY}" ]; then - line="[trusted=yes] ${line}" - fi - echo "deb-src ${line}" >> "${ROOTFSDIR}/etc/apt/sources.list.d/base-apt.list" - - mkdir -p ${ROOTFSDIR}/base-apt - mount --bind ${REPO_BASE_DIR} ${ROOTFSDIR}/base-apt - else - install -v -m644 "${APTSRCS}" \ - "${ROOTFSDIR}/etc/apt/sources.list.d/bootstrap.list" - fi - install -v -m644 "${APTSRCS_INIT}" "${ROOTFSDIR}/etc/apt/sources-list" - rm -f "${ROOTFSDIR}/etc/apt/sources.list" - rm -rf "${ROOTFSDIR}/var/lib/apt/lists/"* - find ${APT_KEYS_DIR}/ -type f | while read keyfile - do - MY_GPGHOME="$(chroot "${ROOTFSDIR}" mktemp -d /tmp/gpghomeXXXXXXXXXX)" - echo "Created temporary directory ${MY_GPGHOME} for gpg-agent" - export GNUPGHOME="${MY_GPGHOME}" - APT_KEY_APPEND="--homedir ${MY_GPGHOME}" - - kfn="$(basename $keyfile)" - cp $keyfile "${ROOTFSDIR}/tmp/$kfn" - chroot "${ROOTFSDIR}" /usr/bin/gpg-agent --daemon -- /usr/bin/apt-key \ - --keyring ${THIRD_PARTY_APT_KEYRING} ${APT_KEY_APPEND} add "/tmp/$kfn" - rm "${ROOTFSDIR}/tmp/$kfn" - - echo "Removing ${MY_GPGHOME}" - rm -rf "${ROOTFSDIR}${MY_GPGHOME}" - done - - # Set locale - install -v -m644 "${WORKDIR}/locale" "${ROOTFSDIR}/etc/locale" - - sed -i '/en_US.UTF-8 UTF-8/s/^#//g' "${ROOTFSDIR}/etc/locale.gen" - chroot "${ROOTFSDIR}" /usr/sbin/locale-gen - # setup chroot install -v -m755 "${WORKDIR}/chroot-setup.sh" "${ROOTFSDIR}/chroot-setup.sh" "${ROOTFSDIR}/chroot-setup.sh" "setup" "${ROOTFSDIR}" - # update APT - mount -o bind,private /dev ${ROOTFSDIR}/dev - mount --bind /dev/pts ${ROOTFSDIR}/dev/pts - mount -t tmpfs none "${ROOTFSDIR}/dev/shm" - mount -t proc none ${ROOTFSDIR}/proc - mount --rbind /sys ${ROOTFSDIR}/sys - mount --make-rslave ${ROOTFSDIR}/sys - - export DEBIAN_FRONTEND=noninteractive - - if [ "${BOOTSTRAP_FOR_HOST}" = "1" ]; then - chroot "${ROOTFSDIR}" /usr/bin/dpkg --add-architecture ${DISTRO_ARCH} - fi - - if [ "${ISAR_ENABLE_COMPAT_ARCH}" = "1" ]; then - chroot "${ROOTFSDIR}" /usr/bin/dpkg --add-architecture ${COMPAT_DISTRO_ARCH} - fi - - chroot "${ROOTFSDIR}" /usr/bin/apt-get update -y \ - -o APT::Update::Error-Mode=any - chroot "${ROOTFSDIR}" /usr/bin/apt-get install -y -f - chroot "${ROOTFSDIR}" /usr/bin/apt-get dist-upgrade -y \ - -o Debug::pkgProblemResolver=yes - - umount -l "${ROOTFSDIR}/dev/shm" - umount -l "${ROOTFSDIR}/dev/pts" - umount -l "${ROOTFSDIR}/dev" - umount -l "${ROOTFSDIR}/proc" - umount -l "${ROOTFSDIR}/sys" - umount -l "${ROOTFSDIR}/base-apt" || true - # Finalize debootstrap by setting the link in deploy ln -Tfsr "${ROOTFSDIR}" "${DEPLOY_ISAR_BOOTSTRAP}" EOSUDO - deb_dl_dir_export "${ROOTFSDIR}" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}" - - # Cleanup apt cache - sudo -Es chroot "${ROOTFSDIR}" /usr/bin/apt-get -y clean + if [ "${ISAR_USE_CACHED_BASE_REPO}" != "1" ]; then + deb_dl_dir_export "${WORKDIR}/dl_dir" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}" + sudo rm -rf --one-file-system "${WORKDIR}/dl_dir" + fi } addtask bootstrap before do_build after do_generate_keyrings -- 2.34.1 ^ permalink raw reply [flat|nested] 16+ messages in thread
* [PATCH v3 3/7] isar-bootstrap: Use tar output instead of directory 2023-05-31 10:10 [PATCH v3 0/7] Migrate to mmdebstrap Anton Mikanovich 2023-05-31 10:10 ` [PATCH v3 1/7] meta: Switch " Anton Mikanovich 2023-05-31 10:10 ` [PATCH v3 2/7] isar-bootstrap: Move preparations to hooks Anton Mikanovich @ 2023-05-31 10:10 ` Anton Mikanovich 2023-05-31 10:10 ` [PATCH v3 4/7] user_manual.md: Replace debootstrap with mmdebstrap Anton Mikanovich ` (4 subsequent siblings) 7 siblings, 0 replies; 16+ messages in thread From: Anton Mikanovich @ 2023-05-31 10:10 UTC (permalink / raw) To: isar-users; +Cc: Anton Mikanovich Do not unpack tar with rootfs content inside bootstrap recipe. This also requires moving chroot prepare step to the later unpack step. Signed-off-by: Anton Mikanovich <amikan@ilbers.de> --- meta/classes/rootfs.bbclass | 11 +++++---- .../isar-bootstrap/isar-bootstrap.inc | 23 ++++--------------- 2 files changed, 12 insertions(+), 22 deletions(-) diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass index 22449d71..29d5fd53 100644 --- a/meta/classes/rootfs.bbclass +++ b/meta/classes/rootfs.bbclass @@ -73,12 +73,15 @@ rootfs_do_qemu() { fi } -BOOTSTRAP_SRC = "${DEPLOY_DIR_BOOTSTRAP}/${ROOTFS_DISTRO}-host_${DISTRO}-${DISTRO_ARCH}" -BOOTSTRAP_SRC:${ROOTFS_ARCH} = "${DEPLOY_DIR_BOOTSTRAP}/${ROOTFS_DISTRO}-${ROOTFS_ARCH}" +BOOTSTRAP_SRC = "${DEPLOY_DIR_BOOTSTRAP}/${ROOTFS_DISTRO}-host_${DISTRO}-${DISTRO_ARCH}.tar.zst" +BOOTSTRAP_SRC:${ROOTFS_ARCH} = "${DEPLOY_DIR_BOOTSTRAP}/${ROOTFS_DISTRO}-${ROOTFS_ARCH}.tar.zst" rootfs_prepare[weight] = "25" rootfs_prepare(){ - sudo cp -Trpfx --reflink=auto '${BOOTSTRAP_SRC}/' '${ROOTFSDIR}' + sudo tar -xf "${BOOTSTRAP_SRC}" -C "${ROOTFSDIR}" --exclude="./dev/console" + + # setup chroot + sudo "${ROOTFSDIR}/chroot-setup.sh" "setup" "${ROOTFSDIR}" } ROOTFS_CONFIGURE_COMMAND += "rootfs_configure_isar_apt" @@ -222,7 +225,7 @@ cache_deb_src() { # Note: ISAR updates the apt state information(apt-get update) only once during bootstrap and # relies on that through out the build. Copy that state information instead of apt-get update # which generates a new state from upstream. - sudo cp -Trpn --reflink=auto "${BOOTSTRAP_SRC}/var/lib/apt/lists/" "${ROOTFSDIR}/var/lib/apt/lists/" + sudo tar -xf "${BOOTSTRAP_SRC}" ./var/lib/apt/lists --one-top-level="${ROOTFSDIR}" deb_dl_dir_import ${ROOTFSDIR} ${ROOTFS_BASE_DISTRO}-${BASE_DISTRO_CODENAME} debsrc_download ${ROOTFSDIR} ${ROOTFS_BASE_DISTRO}-${BASE_DISTRO_CODENAME} diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc index fba876ea..2aeceb58 100644 --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc @@ -16,7 +16,6 @@ PV = "1.0" BOOTSTRAP_FOR_HOST ?= "0" DEBOOTSTRAP = "mmdebstrap" -ROOTFSDIR = "${WORKDIR}/rootfs" APTPREFS = "${WORKDIR}/apt-preferences" APTSRCS = "${WORKDIR}/apt-sources" APTSRCS_INIT = "${WORKDIR}/apt-sources-init" @@ -290,9 +289,6 @@ do_bootstrap() { fi E="${@ isar_export_proxies(d)}" - sudo rm -rf --one-file-system "${ROOTFSDIR}" - mkdir -p "${ROOTFSDIR}" - if [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" ]; then base_apt_tmp="$(mktemp -d /tmp/isar-base-aptXXXXXXXXXX)" bootstrap_list="${WORKDIR}/sources.list.d/base-apt.list" @@ -397,18 +393,9 @@ do_bootstrap() { "${WORKDIR}/rootfs.tar.zst" \ "$bootstrap_list" - sudo -E -s <<'EOSUDO' - set -e - - tar -xf "${WORKDIR}/rootfs.tar.zst" -C "${ROOTFSDIR}" --exclude="./dev/console" - - # setup chroot - install -v -m755 "${WORKDIR}/chroot-setup.sh" "${ROOTFSDIR}/chroot-setup.sh" - "${ROOTFSDIR}/chroot-setup.sh" "setup" "${ROOTFSDIR}" + # Finalize debootstrap by setting the link in deploy + sudo ln -Tfsr "${WORKDIR}/rootfs.tar.zst" "${DEPLOY_ISAR_BOOTSTRAP}.tar.zst" - # Finalize debootstrap by setting the link in deploy - ln -Tfsr "${ROOTFSDIR}" "${DEPLOY_ISAR_BOOTSTRAP}" -EOSUDO if [ "${ISAR_USE_CACHED_BASE_REPO}" != "1" ]; then deb_dl_dir_export "${WORKDIR}/dl_dir" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}" sudo rm -rf --one-file-system "${WORKDIR}/dl_dir" @@ -423,15 +410,15 @@ SSTATEPOSTINSTFUNCS += "bootstrap_sstate_finalize" bootstrap_sstate_prepare() { # this runs in SSTATE_BUILDDIR, which will be deleted automatically - sudo cp -a "$(dirname "${ROOTFSDIR}")/rootfs.tar.zst" ./bootstrap.tar.zst + sudo cp -a "${WORKDIR}/rootfs.tar.zst" ./bootstrap.tar.zst sudo chown $(id -u):$(id -g) bootstrap.tar.zst } bootstrap_sstate_finalize() { # this runs in SSTATE_INSTDIR if [ -f bootstrap.tar.zst ]; then - mv bootstrap.tar.zst "$(dirname "${ROOTFSDIR}")/rootfs.tar.zst" - sudo ln -Tfsr "$(dirname "${ROOTFSDIR}")/rootfs.tar.zst" \ + mv bootstrap.tar.zst "${WORKDIR}/rootfs.tar.zst" + sudo ln -Tfsr "${WORKDIR}/rootfs.tar.zst" \ "${DEPLOY_ISAR_BOOTSTRAP}.tar.zst" fi } -- 2.34.1 ^ permalink raw reply [flat|nested] 16+ messages in thread
* [PATCH v3 4/7] user_manual.md: Replace debootstrap with mmdebstrap 2023-05-31 10:10 [PATCH v3 0/7] Migrate to mmdebstrap Anton Mikanovich ` (2 preceding siblings ...) 2023-05-31 10:10 ` [PATCH v3 3/7] isar-bootstrap: Use tar output instead of directory Anton Mikanovich @ 2023-05-31 10:10 ` Anton Mikanovich 2023-05-31 10:11 ` [PATCH v3 5/7] user_manual.md: Update minimal host requirements Anton Mikanovich ` (3 subsequent siblings) 7 siblings, 0 replies; 16+ messages in thread From: Anton Mikanovich @ 2023-05-31 10:10 UTC (permalink / raw) To: isar-users; +Cc: Anton Mikanovich Update documentation in user manual. Signed-off-by: Anton Mikanovich <amikan@ilbers.de> --- doc/user_manual.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/doc/user_manual.md b/doc/user_manual.md index e07b76d8..cdf50e35 100644 --- a/doc/user_manual.md +++ b/doc/user_manual.md @@ -65,7 +65,9 @@ Install the following packages: apt install \ binfmt-support \ bzip2 \ - debootstrap \ + mmdebstrap \ + arch-test \ + apt-utils \ dosfstools \ dpkg-dev \ gettext-base \ @@ -128,7 +130,7 @@ apt install qemu ### Setup Sudo -Isar requires `sudo` rights without password to work with `chroot` and `debootstrap`. To add them, use the following steps: +Isar requires `sudo` rights without password to work with `chroot`. To add them, use the following steps: ``` # visudo ``` -- 2.34.1 ^ permalink raw reply [flat|nested] 16+ messages in thread
* [PATCH v3 5/7] user_manual.md: Update minimal host requirements 2023-05-31 10:10 [PATCH v3 0/7] Migrate to mmdebstrap Anton Mikanovich ` (3 preceding siblings ...) 2023-05-31 10:10 ` [PATCH v3 4/7] user_manual.md: Replace debootstrap with mmdebstrap Anton Mikanovich @ 2023-05-31 10:11 ` Anton Mikanovich 2023-05-31 10:11 ` [PATCH v3 6/7] isar-bootstrap: Remove unused code Anton Mikanovich ` (2 subsequent siblings) 7 siblings, 0 replies; 16+ messages in thread From: Anton Mikanovich @ 2023-05-31 10:11 UTC (permalink / raw) To: isar-users; +Cc: Anton Mikanovich Using mmdebstrap requires at least Debian Bullseye on host. Update user manual with the new minimal host distro. Also remove all Buster-related notes. Signed-off-by: Anton Mikanovich <amikan@ilbers.de> --- doc/user_manual.md | 35 ++++++++++++----------------------- 1 file changed, 12 insertions(+), 23 deletions(-) diff --git a/doc/user_manual.md b/doc/user_manual.md index cdf50e35..ed768ba7 100644 --- a/doc/user_manual.md +++ b/doc/user_manual.md @@ -46,7 +46,7 @@ For demonstration purposes, Isar provides support for the following configurations: - QEMU ARM with Debian Buster - - QEMU ARM64 with Debian Buster (for host >= buster) + - QEMU ARM64 with Debian Buster - QEMU i386 with Debian Buster - QEMU amd64 with Debian Buster - Raspberry Pi various models with Raspberry OS Bullseye @@ -58,7 +58,7 @@ The steps below describe how to build the images provided by default. ### Install Host Tools -The supported host system is >= buster. +The minimal supported host system is Debian Bullseye. Install the following packages: ``` @@ -75,6 +75,7 @@ apt install \ mtools \ parted \ python3 \ + python3-distutils \ quilt \ qemu-user-static \ reprepro \ @@ -88,28 +89,19 @@ apt install \ zstd ``` -If your host is >= buster, also install the following package. -``` -apt install python3-distutils -``` - -**NOTE:** sbuild version (<=0.78.1) packaged in Debian Buster doesn't support -`$apt_keep_downloaded_packages` option which is required in Isar for -populating `${DL_DIR}/deb`. So, host `sbuild` in this case should be manually -upgraded to >=0.81.2 version from Debian Bullseye. - Next, the user who should run Isar needs to be added to `sbuild` group. ``` sudo gpasswd -a <username> sbuild ``` If you want to generate containerized SDKs, also install the following -packages: `umoci` and `skopeo`. -Umoci is provided by Debian Buster and can be installed with -`apt install umoci`, Skopeo is provided by Debian Bullseye/Unstable and has to -be installed either manually downloading the DEB and installing it (no other -packages required) or with `apt install -t bullseye skopeo` (if -unstable/bullseye included in `/etc/apt/sources.list[.d]`). +packages: + +``` +apt install \ + umoci \ + skopeo +``` Notes: @@ -235,9 +227,6 @@ Variables may be used in `.wks.in` files; Isar will expand them and generate a r In order to run the EFI images with `qemu`, an EFI firmware is required and available at the following address: https://github.com/tianocore/edk2/tree/3858b4a1ff09d3243fea8d07bd135478237cb8f7 -Note that the `ovmf` package in Debian Buster contains a pre-compiled firmware, but doesn't seem to be recent -enough to allow images to be testable under `qemu`. - ``` # AMD64 image, EFI qemu-system-x86_64 -m 256M -nographic -bios edk2/Build/OvmfX64/RELEASE_*/FV/OVMF.fd -hda tmp/deploy/images/qemuamd64/isar-image-base-debian-buster-qemuamd64.wic @@ -955,8 +944,8 @@ put into Isar apt. Debian cross-compilation works out of the box. Currently the following build configurations are supported in Isar: - buster armhf - - buster arm64 (for host >= buster) - - buster mipsel (for host >= buster) + - buster arm64 + - buster mipsel - bullseye armhf - bullseye arm64 - bullseye mipsel -- 2.34.1 ^ permalink raw reply [flat|nested] 16+ messages in thread
* [PATCH v3 6/7] isar-bootstrap: Remove unused code 2023-05-31 10:10 [PATCH v3 0/7] Migrate to mmdebstrap Anton Mikanovich ` (4 preceding siblings ...) 2023-05-31 10:11 ` [PATCH v3 5/7] user_manual.md: Update minimal host requirements Anton Mikanovich @ 2023-05-31 10:11 ` Anton Mikanovich 2023-05-31 10:11 ` [PATCH v3 7/7] CI: Install mmdebstrap Anton Mikanovich 2023-06-20 5:30 ` [PATCH v3 0/7] Migrate to mmdebstrap Jan Kiszka 7 siblings, 0 replies; 16+ messages in thread From: Anton Mikanovich @ 2023-05-31 10:11 UTC (permalink / raw) To: isar-users; +Cc: Anton Mikanovich Some external logic like installing gpg and ca-certificates is now doing by mmdebstrap inself. So remove duplications. Signed-off-by: Anton Mikanovich <amikan@ilbers.de> --- .../isar-bootstrap/isar-bootstrap.inc | 26 ------------------- 1 file changed, 26 deletions(-) diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc index 2aeceb58..b52edc05 100644 --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc @@ -23,8 +23,6 @@ DISTRO_BOOTSTRAP_KEYFILES = "" THIRD_PARTY_APT_KEYFILES = "" DEPLOY_ISAR_BOOTSTRAP ?= "" DISTRO_BOOTSTRAP_BASE_PACKAGES = "locales,usrmerge" -DISTRO_BOOTSTRAP_BASE_PACKAGES:append:gnupg = ",gnupg" -DISTRO_BOOTSTRAP_BASE_PACKAGES:append:https-support = ",ca-certificates" DISTRO_VARS_PREFIX ?= "${@'HOST_' if d.getVar('BOOTSTRAP_FOR_HOST') == '1' else ''}" BOOTSTRAP_DISTRO = "${@d.getVar('HOST_DISTRO' if d.getVar('BOOTSTRAP_FOR_HOST') == '1' else 'DISTRO')}" BOOTSTRAP_BASE_DISTRO = "${@d.getVar('HOST_BASE_DISTRO' if d.getVar('BOOTSTRAP_FOR_HOST') == '1' else 'BASE_DISTRO')}" @@ -177,30 +175,6 @@ def get_distro_primary_source_entry(d): return source[2:] bb.fatal('Invalid apt sources list') -def get_distro_have_https_source(d): - return any(source[2].startswith("https://") for source in generate_distro_sources(d)) - -def get_distro_needs_https_support(d): - if get_distro_have_https_source(d): - return "https-support" - else: - return "" - -OVERRIDES:append = ":${@get_distro_needs_https_support(d)}" - -def get_distro_needs_gpg_support(d): - if d.getVar("DISTRO_BOOTSTRAP_KEYS") or \ - d.getVar("THIRD_PARTY_APT_KEYS") or \ - d.getVar("BASE_REPO_KEY"): - return "gnupg" - else: - return "" - -OVERRIDES:append = ":${@get_distro_needs_gpg_support(d)}" - -def get_distro_source(d): - return get_distro_primary_source_entry(d)[0] - def get_distro_suite(d): return get_distro_primary_source_entry(d)[1] -- 2.34.1 ^ permalink raw reply [flat|nested] 16+ messages in thread
* [PATCH v3 7/7] CI: Install mmdebstrap 2023-05-31 10:10 [PATCH v3 0/7] Migrate to mmdebstrap Anton Mikanovich ` (5 preceding siblings ...) 2023-05-31 10:11 ` [PATCH v3 6/7] isar-bootstrap: Remove unused code Anton Mikanovich @ 2023-05-31 10:11 ` Anton Mikanovich 2023-06-20 5:30 ` [PATCH v3 0/7] Migrate to mmdebstrap Jan Kiszka 7 siblings, 0 replies; 16+ messages in thread From: Anton Mikanovich @ 2023-05-31 10:11 UTC (permalink / raw) To: isar-users; +Cc: Uladzimir Bely, Anton Mikanovich From: Uladzimir Bely <ubely@ilbers.de> Temporary fix for old kas versions: install mmdebstrap and required tools if not present. Signed-off-by: Anton Mikanovich <amikan@ilbers.de> --- scripts/ci_build.sh | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh index 2fa5cccd..d26f3cdc 100755 --- a/scripts/ci_build.sh +++ b/scripts/ci_build.sh @@ -127,6 +127,12 @@ if echo "$TAGS" | grep -Fqive "-startvm"; then fi fi +# install mmdebstrap and tools +if ! command -v mmdebstrap > /dev/null; then + sudo apt-get update -qq + sudo apt-get install -y mmdebstrap arch-test apt-utils +fi + # Provide working path mkdir -p .config/avocado cat <<EOF > .config/avocado/avocado.conf -- 2.34.1 ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH v3 0/7] Migrate to mmdebstrap 2023-05-31 10:10 [PATCH v3 0/7] Migrate to mmdebstrap Anton Mikanovich ` (6 preceding siblings ...) 2023-05-31 10:11 ` [PATCH v3 7/7] CI: Install mmdebstrap Anton Mikanovich @ 2023-06-20 5:30 ` Jan Kiszka 2023-06-20 10:00 ` Uladzimir Bely 7 siblings, 1 reply; 16+ messages in thread From: Jan Kiszka @ 2023-06-20 5:30 UTC (permalink / raw) To: Anton Mikanovich, isar-users, Uladzimir Bely On 31.05.23 12:10, Anton Mikanovich wrote: > Switch Isar from debootstrap to mmdebstrap. > It makes isar-bootstrap target ~10% faster on both cross and native rootfs > types. Also mmdebstrap support using SOURCE_DATE_EPOCH for images reproducibly > out of the box. But the main advantage is an ability to remove sudo for > debootstrapping later on. > This change brakes Debian Buster support as host distro. > > Changes since v2: > - Rebase on next > - Fix commit messages > - Update host requirements > > Changes since v1: > - Move mmdebstrap tmpdir to workdir > - Fix kas building > - Improve mounts cleanup > > Anton Mikanovich (5): > isar-bootstrap: Move preparations to hooks > isar-bootstrap: Use tar output instead of directory > user_manual.md: Replace debootstrap with mmdebstrap > user_manual.md: Update minimal host requirements > isar-bootstrap: Remove unused code > > Uladzimir Bely (2): > meta: Switch to mmdebstrap > CI: Install mmdebstrap > > doc/user_manual.md | 41 +-- > meta-isar/conf/distro/ubuntu-common.inc | 4 - > meta/classes/rootfs.bbclass | 11 +- > .../isar-bootstrap/isar-bootstrap.inc | 274 ++++++++---------- > scripts/ci_build.sh | 6 + > 5 files changed, 152 insertions(+), 184 deletions(-) > Is this now settled? Asking as I'm planning to release kas 4.0 with the dependencies in patch 7 resolved these days, and I'd like to see this targeting the final version of this feature. Jan -- Siemens AG, Technology Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH v3 0/7] Migrate to mmdebstrap 2023-06-20 5:30 ` [PATCH v3 0/7] Migrate to mmdebstrap Jan Kiszka @ 2023-06-20 10:00 ` Uladzimir Bely 2023-06-20 10:22 ` Jan Kiszka ` (2 more replies) 0 siblings, 3 replies; 16+ messages in thread From: Uladzimir Bely @ 2023-06-20 10:00 UTC (permalink / raw) To: Jan Kiszka, Anton Mikanovich, isar-users On Tue, 2023-06-20 at 07:30 +0200, Jan Kiszka wrote: > On 31.05.23 12:10, Anton Mikanovich wrote: > > Switch Isar from debootstrap to mmdebstrap. > > It makes isar-bootstrap target ~10% faster on both cross and native > > rootfs > > types. Also mmdebstrap support using SOURCE_DATE_EPOCH for images > > reproducibly > > out of the box. But the main advantage is an ability to remove sudo > > for > > debootstrapping later on. > > This change brakes Debian Buster support as host distro. > > > > Changes since v2: > > - Rebase on next > > - Fix commit messages > > - Update host requirements > > > > Changes since v1: > > - Move mmdebstrap tmpdir to workdir > > - Fix kas building > > - Improve mounts cleanup > > > > Anton Mikanovich (5): > > isar-bootstrap: Move preparations to hooks > > isar-bootstrap: Use tar output instead of directory > > user_manual.md: Replace debootstrap with mmdebstrap > > user_manual.md: Update minimal host requirements > > isar-bootstrap: Remove unused code > > > > Uladzimir Bely (2): > > meta: Switch to mmdebstrap > > CI: Install mmdebstrap > > > > doc/user_manual.md | 41 +-- > > meta-isar/conf/distro/ubuntu-common.inc | 4 - > > meta/classes/rootfs.bbclass | 11 +- > > .../isar-bootstrap/isar-bootstrap.inc | 274 ++++++++------ > > ---- > > scripts/ci_build.sh | 6 + > > 5 files changed, 152 insertions(+), 184 deletions(-) > > > > Is this now settled? Asking as I'm planning to release kas 4.0 with > the > dependencies in patch 7 resolved these days, and I'd like to see this > targeting the final version of this feature. > > Jan > We are currently upgrading our interlnal CI and need to run more jobs before merging it. But anyway, if mmdebstrap (and other packages mentioned in patch 3) is preinstalled in kas 4.0 image, it would be nice. ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH v3 0/7] Migrate to mmdebstrap 2023-06-20 10:00 ` Uladzimir Bely @ 2023-06-20 10:22 ` Jan Kiszka 2023-07-14 16:40 ` vijai kumar 2023-09-19 9:14 ` Schaffner, Tobias 2 siblings, 0 replies; 16+ messages in thread From: Jan Kiszka @ 2023-06-20 10:22 UTC (permalink / raw) To: Uladzimir Bely, Anton Mikanovich, isar-users On 20.06.23 12:00, Uladzimir Bely wrote: > On Tue, 2023-06-20 at 07:30 +0200, Jan Kiszka wrote: >> On 31.05.23 12:10, Anton Mikanovich wrote: >>> Switch Isar from debootstrap to mmdebstrap. >>> It makes isar-bootstrap target ~10% faster on both cross and native >>> rootfs >>> types. Also mmdebstrap support using SOURCE_DATE_EPOCH for images >>> reproducibly >>> out of the box. But the main advantage is an ability to remove sudo >>> for >>> debootstrapping later on. >>> This change brakes Debian Buster support as host distro. >>> >>> Changes since v2: >>> - Rebase on next >>> - Fix commit messages >>> - Update host requirements >>> >>> Changes since v1: >>> - Move mmdebstrap tmpdir to workdir >>> - Fix kas building >>> - Improve mounts cleanup >>> >>> Anton Mikanovich (5): >>> isar-bootstrap: Move preparations to hooks >>> isar-bootstrap: Use tar output instead of directory >>> user_manual.md: Replace debootstrap with mmdebstrap >>> user_manual.md: Update minimal host requirements >>> isar-bootstrap: Remove unused code >>> >>> Uladzimir Bely (2): >>> meta: Switch to mmdebstrap >>> CI: Install mmdebstrap >>> >>> doc/user_manual.md | 41 +-- >>> meta-isar/conf/distro/ubuntu-common.inc | 4 - >>> meta/classes/rootfs.bbclass | 11 +- >>> .../isar-bootstrap/isar-bootstrap.inc | 274 ++++++++------ >>> ---- >>> scripts/ci_build.sh | 6 + >>> 5 files changed, 152 insertions(+), 184 deletions(-) >>> >> >> Is this now settled? Asking as I'm planning to release kas 4.0 with >> the >> dependencies in patch 7 resolved these days, and I'd like to see this >> targeting the final version of this feature. >> >> Jan >> > > We are currently upgrading our interlnal CI and need to run more jobs > before merging it. > > But anyway, if mmdebstrap (and other packages mentioned in patch 3) is > preinstalled in kas 4.0 image, it would be nice. You can already check that by using kas-isar:next which contains [1]. Jan [1] https://github.com/siemens/kas/commit/9110175839729cd5dbd3047e7a52e6bd43786ecc -- Siemens AG, Technology Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH v3 0/7] Migrate to mmdebstrap 2023-06-20 10:00 ` Uladzimir Bely 2023-06-20 10:22 ` Jan Kiszka @ 2023-07-14 16:40 ` vijai kumar 2023-09-19 9:14 ` Schaffner, Tobias 2 siblings, 0 replies; 16+ messages in thread From: vijai kumar @ 2023-07-14 16:40 UTC (permalink / raw) To: isar-users [-- Attachment #1.1: Type: text/plain, Size: 2184 bytes --] Is this series ready to get in now? On Tuesday, June 20, 2023 at 3:30:43 PM UTC+5:30 Uladzimir Bely wrote: > On Tue, 2023-06-20 at 07:30 +0200, Jan Kiszka wrote: > > On 31.05.23 12:10, Anton Mikanovich wrote: > > > Switch Isar from debootstrap to mmdebstrap. > > > It makes isar-bootstrap target ~10% faster on both cross and native > > > rootfs > > > types. Also mmdebstrap support using SOURCE_DATE_EPOCH for images > > > reproducibly > > > out of the box. But the main advantage is an ability to remove sudo > > > for > > > debootstrapping later on. > > > This change brakes Debian Buster support as host distro. > > > > > > Changes since v2: > > > - Rebase on next > > > - Fix commit messages > > > - Update host requirements > > > > > > Changes since v1: > > > - Move mmdebstrap tmpdir to workdir > > > - Fix kas building > > > - Improve mounts cleanup > > > > > > Anton Mikanovich (5): > > > isar-bootstrap: Move preparations to hooks > > > isar-bootstrap: Use tar output instead of directory > > > user_manual.md: Replace debootstrap with mmdebstrap > > > user_manual.md: Update minimal host requirements > > > isar-bootstrap: Remove unused code > > > > > > Uladzimir Bely (2): > > > meta: Switch to mmdebstrap > > > CI: Install mmdebstrap > > > > > > doc/user_manual.md | 41 +-- > > > meta-isar/conf/distro/ubuntu-common.inc | 4 - > > > meta/classes/rootfs.bbclass | 11 +- > > > .../isar-bootstrap/isar-bootstrap.inc | 274 ++++++++------ > > > ---- > > > scripts/ci_build.sh | 6 + > > > 5 files changed, 152 insertions(+), 184 deletions(-) > > > > > > > Is this now settled? Asking as I'm planning to release kas 4.0 with > > the > > dependencies in patch 7 resolved these days, and I'd like to see this > > targeting the final version of this feature. > > > > Jan > > > > We are currently upgrading our interlnal CI and need to run more jobs > before merging it. > > But anyway, if mmdebstrap (and other packages mentioned in patch 3) is > preinstalled in kas 4.0 image, it would be nice. > [-- Attachment #1.2: Type: text/html, Size: 2888 bytes --] ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH v3 0/7] Migrate to mmdebstrap 2023-06-20 10:00 ` Uladzimir Bely 2023-06-20 10:22 ` Jan Kiszka 2023-07-14 16:40 ` vijai kumar @ 2023-09-19 9:14 ` Schaffner, Tobias 2023-09-20 7:24 ` Uladzimir Bely 2 siblings, 1 reply; 16+ messages in thread From: Schaffner, Tobias @ 2023-09-19 9:14 UTC (permalink / raw) To: Uladzimir Bely, isar-users; +Cc: Anton Mikanovich, Kiszka, Jan [-- Attachment #1.1: Type: text/plain, Size: 698 bytes --] Hi Uladzimir, some of the patches of this story needed a rebase. Find them attached. On 20.06.23 12:00, Uladzimir Bely wrote: ... >> >> Is this now settled? Asking as I'm planning to release kas 4.0 with >> the >> dependencies in patch 7 resolved these days, and I'd like to see this >> targeting the final version of this feature. >> >> Jan >> > > We are currently upgrading our interlnal CI and need to run more jobs > before merging it. > > But anyway, if mmdebstrap (and other packages mentioned in patch 3) is > preinstalled in kas 4.0 image, it would be nice. The dependencies needed are already included in KAS 4.0. Is there something else missing to finish this? All the best, Tobias [-- Attachment #1.2: 0005-user_manual-md-Update-minimal-host-requirements.patch --] [-- Type: text/x-patch, Size: 3439 bytes --] From: Anton Mikanovich <amikan@ilbers.de> Subject: [PATCH v3 5/7] user_manual.md: Update minimal host requirements Date: Wed, 31 May 2023 13:11:00 +0300 Content-Type: text/plain; charset="UTF-8" Using mmdebstrap requires at least Debian Bullseye on host. Update user manual with the new minimal host distro. Also remove all Buster-related notes. Signed-off-by: Anton Mikanovich <amikan@ilbers.de> --- doc/user_manual.md | 35 ++++++++++++----------------------- 1 file changed, 12 insertions(+), 23 deletions(-) diff --git a/doc/user_manual.md b/doc/user_manual.md index dc7ed4c8..72dd49e3 100644 --- a/doc/user_manual.md +++ b/doc/user_manual.md @@ -48,7 +48,7 @@ For demonstration purposes, Isar provides support for the following configurations: - QEMU ARM with Debian Buster - - QEMU ARM64 with Debian Buster (for host >= buster) + - QEMU ARM64 with Debian Buster - QEMU i386 with Debian Buster - QEMU amd64 with Debian Buster - Raspberry Pi various models with Raspberry OS Bullseye @@ -60,7 +60,7 @@ The steps below describe how to build the images provided by default. ### Install Host Tools -The supported host system is >= buster. +The minimal supported host system is Debian Bullseye. Install the following packages: ``` @@ -74,6 +74,7 @@ apt install \ gettext-base \ git \ python3 \ + python3-distutils \ quilt \ qemu-user-static \ reprepro \ @@ -87,28 +88,13 @@ apt install \ zstd ``` -If your host is >= buster, also install the following package. -``` -apt install python3-distutils -``` - -**NOTE:** sbuild version (<=0.78.1) packaged in Debian Buster doesn't support -`$apt_keep_downloaded_packages` option which is required in Isar for -populating `${DL_DIR}/deb`. So, host `sbuild` in this case should be manually -upgraded to >=0.81.2 version from Debian Bullseye. - Next, the user who should run Isar needs to be added to `sbuild` group. ``` sudo gpasswd -a <username> sbuild ``` If you want to generate containerized SDKs, also install the following -packages: `umoci` and `skopeo`. -Umoci is provided by Debian Buster and can be installed with -`apt install umoci`, Skopeo is provided by Debian Bullseye/Unstable and has to -be installed either manually downloading the DEB and installing it (no other -packages required) or with `apt install -t bullseye skopeo` (if -unstable/bullseye included in `/etc/apt/sources.list[.d]`). +packages: Notes: @@ -234,9 +220,6 @@ Variables may be used in `.wks.in` files; Isar will expand them and generate a r In order to run the EFI images with `qemu`, an EFI firmware is required and available at the following address: https://github.com/tianocore/edk2/tree/3858b4a1ff09d3243fea8d07bd135478237cb8f7 -Note that the `ovmf` package in Debian Buster contains a pre-compiled firmware, but doesn't seem to be recent -enough to allow images to be testable under `qemu`. - ``` # AMD64 image, EFI qemu-system-x86_64 -m 256M -nographic -bios edk2/Build/OvmfX64/RELEASE_*/FV/OVMF.fd -hda tmp/deploy/images/qemuamd64/isar-image-base-debian-buster-qemuamd64.wic @@ -963,8 +946,8 @@ put into Isar apt. Debian cross-compilation works out of the box. Currently the following build configurations are supported in Isar: - buster armhf - - buster arm64 (for host >= buster) - - buster mipsel (for host >= buster) + - buster arm64 + - buster mipsel - bullseye armhf - bullseye arm64 - bullseye mipsel -- 2.34.1 [-- Attachment #1.3: 0004-user_manual-md-Replace-debootstrap-with-mmdebstrap.patch --] [-- Type: text/x-patch, Size: 1029 bytes --] From: Anton Mikanovich <amikan@ilbers.de> Subject: [PATCH v3 4/7] user_manual.md: Replace debootstrap with mmdebstrap Date: Wed, 31 May 2023 13:10:59 +0300 Content-Type: text/plain; charset="UTF-8" Update documentation in user manual. Signed-off-by: Anton Mikanovich <amikan@ilbers.de> --- doc/user_manual.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/doc/user_manual.md b/doc/user_manual.md index be47fdcc..dc7ed4c8 100644 --- a/doc/user_manual.md +++ b/doc/user_manual.md @@ -67,7 +67,9 @@ Install the following packages: apt install \ binfmt-support \ bzip2 \ - debootstrap \ + mmdebstrap \ + arch-test \ + apt-utils \ dpkg-dev \ gettext-base \ git \ @@ -127,7 +129,7 @@ apt install qemu ### Setup Sudo -Isar requires `sudo` rights without password to work with `chroot` and `debootstrap`. To add them, use the following steps: +Isar requires `sudo` rights without password to work with `chroot`. To add them, use the following steps: ``` # visudo ``` -- 2.34.1 [-- Attachment #1.4: 0001-meta-Switch-to-mmdebstrap.patch --] [-- Type: text/x-patch, Size: 6108 bytes --] From: Anton Mikanovich <amikan@ilbers.de> Subject: [PATCH v3 1/7] meta: Switch to mmdebstrap Date: Wed, 31 May 2023 13:10:56 +0300 Content-Type: text/plain; charset="UTF-8" From: Uladzimir Bely <ubely@ilbers.de> Instead of debootstrap, use mmdebstrap alternative. Internally, it uses apt and allows to debootstrap the distro from multiple repositories. Signed-off-by: Uladzimir Bely <ubely@ilbers.de> Signed-off-by: Anton Mikanovich <amikan@ilbers.de> --- meta-isar/conf/distro/ubuntu-common.inc | 4 -- .../isar-bootstrap/isar-bootstrap.inc | 57 ++++++++++--------- 2 files changed, 30 insertions(+), 31 deletions(-) diff --git a/meta-isar/conf/distro/ubuntu-common.inc b/meta-isar/conf/distro/ubuntu-common.inc index 6a990a89..9ca1273d 100644 --- a/meta-isar/conf/distro/ubuntu-common.inc +++ b/meta-isar/conf/distro/ubuntu-common.inc @@ -19,9 +19,6 @@ HOST_DISTRO_BOOTSTRAP_KEYS += "${BOOTSTRAP_KEY}" # kernel package name is linux-image-generic in Ubuntu KERNEL_NAME = "generic" -# that is what debootstrap_1.0.118ubuntu1 does anyways -DISTRO_DEBOOTSTRAP_SCRIPT = "/usr/share/debootstrap/scripts/gutsy" - DISTRO_CONFIG_SCRIPT ?= "debian-configscript.sh" DISTRO_KERNELS ?= " \ diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc index b94ae0bd..f89c284e 100644 --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc @@ -15,7 +15,7 @@ PV = "1.0" BOOTSTRAP_FOR_HOST ?= "0" -DEBOOTSTRAP ?= "qemu-debootstrap" +DEBOOTSTRAP = "mmdebstrap" ROOTFSDIR = "${WORKDIR}/rootfs" APTPREFS = "${WORKDIR}/apt-preferences" APTSRCS = "${WORKDIR}/apt-sources" @@ -23,12 +23,14 @@ APTSRCS_INIT = "${WORKDIR}/apt-sources-init" DISTRO_BOOTSTRAP_KEYFILES = "" THIRD_PARTY_APT_KEYFILES = "" DEPLOY_ISAR_BOOTSTRAP ?= "" -DISTRO_BOOTSTRAP_BASE_PACKAGES = "locales" +DISTRO_BOOTSTRAP_BASE_PACKAGES = "locales,usrmerge" DISTRO_BOOTSTRAP_BASE_PACKAGES:append:gnupg = ",gnupg" DISTRO_BOOTSTRAP_BASE_PACKAGES:append:https-support = ",ca-certificates" DISTRO_VARS_PREFIX ?= "${@'HOST_' if d.getVar('BOOTSTRAP_FOR_HOST') == '1' else ''}" BOOTSTRAP_DISTRO = "${@d.getVar('HOST_DISTRO' if d.getVar('BOOTSTRAP_FOR_HOST') == '1' else 'DISTRO')}" BOOTSTRAP_BASE_DISTRO = "${@d.getVar('HOST_BASE_DISTRO' if d.getVar('BOOTSTRAP_FOR_HOST') == '1' else 'BASE_DISTRO')}" +BOOTSTRAP_DISTRO_ARCH = "${@d.getVar('HOST_ARCH' if d.getVar('BOOTSTRAP_FOR_HOST') == '1' else 'DISTRO_ARCH')}" +BOOTSTRAP_TMPDIR = "${WORKDIR}/tempdir" FILESEXTRAPATHS:append = ":${BBPATH}" inherit deb-dl-dir @@ -270,7 +272,7 @@ do_bootstrap[vardeps] += " \ ISAR_ENABLE_COMPAT_ARCH \ ${DISTRO_VARS_PREFIX}DISTRO_APT_SOURCES \ " -do_bootstrap[dirs] = "${DEPLOY_DIR_BOOTSTRAP}" +do_bootstrap[dirs] = "${DEPLOY_DIR_BOOTSTRAP} ${BOOTSTRAP_TMPDIR}" do_bootstrap[depends] = "base-apt:do_cache isar-apt:do_cache_config" do_bootstrap[network] = "${TASK_USE_NETWORK_AND_SUDO}" @@ -286,27 +288,31 @@ do_bootstrap() { if [ -f "${DISTRO_BOOTSTRAP_KEYRING}" ]; then debootstrap_args="$debootstrap_args --keyring=${DISTRO_BOOTSTRAP_KEYRING}" fi - if [ "${ISAR_USE_CACHED_BASE_REPO}" = "1" -a -z "${BASE_REPO_KEY}" ]; then - debootstrap_args="$debootstrap_args --no-check-gpg" - fi E="${@ isar_export_proxies(d)}" - export BOOTSTRAP_FOR_HOST debootstrap_args E + export BOOTSTRAP_FOR_HOST - sudo rm -rf --one-file-system "${ROOTFSDIR}" deb_dl_dir_import "${ROOTFSDIR}" "${BOOTSTRAP_BASE_DISTRO}-${BASE_DISTRO_CODENAME}" + sudo rm -rf --one-file-system "${ROOTFSDIR}" + mkdir -p "${ROOTFSDIR}" + + if [ ! -z "${SOURCE_DATE_EPOCH}" ]; then + export SOURCE_DATE_EPOCH="${SOURCE_DATE_EPOCH}" + fi + + arch_param="--arch=${BOOTSTRAP_DISTRO_ARCH},${DISTRO_ARCH}" + + sudo TMPDIR="${BOOTSTRAP_TMPDIR}" ${DEBOOTSTRAP} $debootstrap_args \ + $arch_param \ + --mode=unshare \ + ${@get_distro_components_argument(d)} \ + "${@get_distro_suite(d)}" \ + "${WORKDIR}/rootfs.tar.zst" \ + "${@get_distro_source(d)}" sudo -E -s <<'EOSUDO' set -e - if [ "${BOOTSTRAP_FOR_HOST}" = "0" ]; then - arch_param="--arch=${DISTRO_ARCH}" - fi - ${DEBOOTSTRAP} $debootstrap_args \ - $arch_param \ - ${@get_distro_components_argument(d)} \ - "${@get_distro_suite(d)}" \ - "${ROOTFSDIR}" \ - "${@get_distro_source(d)}" \ - ${DISTRO_DEBOOTSTRAP_SCRIPT} + + tar -xf "${WORKDIR}/rootfs.tar.zst" -C "${ROOTFSDIR}" --exclude="./dev/console" # Install apt config mkdir -p "${ROOTFSDIR}/etc/apt/preferences.d" @@ -409,19 +415,16 @@ SSTATEPOSTINSTFUNCS += "bootstrap_sstate_finalize" bootstrap_sstate_prepare() { # this runs in SSTATE_BUILDDIR, which will be deleted automatically - lopts="--one-file-system --exclude=var/cache/apt/archives" - sudo tar -C $(dirname "${ROOTFSDIR}") -cpSf bootstrap.tar $lopts $(basename "${ROOTFSDIR}") - sudo chown $(id -u):$(id -g) bootstrap.tar + sudo cp -a "$(dirname "${ROOTFSDIR}")/rootfs.tar.zst" ./bootstrap.tar.zst + sudo chown $(id -u):$(id -g) bootstrap.tar.zst } bootstrap_sstate_finalize() { # this runs in SSTATE_INSTDIR - # - after building the bootstrap, the tar won't be there, but we also don't need to unpack - # - after restoring from cache, there will be a tar which we unpack and then delete - if [ -f bootstrap.tar ]; then - sudo tar -C $(dirname "${ROOTFSDIR}") -xpf bootstrap.tar - sudo ln -Tfsr "${ROOTFSDIR}" "${DEPLOY_ISAR_BOOTSTRAP}" - rm bootstrap.tar + if [ -f bootstrap.tar.zst ]; then + mv bootstrap.tar.zst "$(dirname "${ROOTFSDIR}")/rootfs.tar.zst" + sudo ln -Tfsr "$(dirname "${ROOTFSDIR}")/rootfs.tar.zst" \ + "${DEPLOY_ISAR_BOOTSTRAP}.tar.zst" fi } -- 2.34.1 [-- Attachment #2: S/MIME Cryptographic Signature --] [-- Type: application/pkcs7-signature, Size: 10139 bytes --] ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH v3 0/7] Migrate to mmdebstrap 2023-09-19 9:14 ` Schaffner, Tobias @ 2023-09-20 7:24 ` Uladzimir Bely 2023-09-20 7:30 ` Uladzimir Bely 0 siblings, 1 reply; 16+ messages in thread From: Uladzimir Bely @ 2023-09-20 7:24 UTC (permalink / raw) To: Schaffner, Tobias, isar-users On Tue, 2023-09-19 at 09:14 +0000, Schaffner, Tobias wrote: > Hi Uladzimir, > > some of the patches of this story needed a rebase. Find them > attached. > > On 20.06.23 12:00, Uladzimir Bely wrote: > ... > > > > > > Is this now settled? Asking as I'm planning to release kas 4.0 > > > with > > > the > > > dependencies in patch 7 resolved these days, and I'd like to see > > > this > > > targeting the final version of this feature. > > > > > > Jan > > > > > > > We are currently upgrading our interlnal CI and need to run more > > jobs > > before merging it. > > > > But anyway, if mmdebstrap (and other packages mentioned in patch 3) > > is > > preinstalled in kas 4.0 image, it would be nice. > > The dependencies needed are already included in KAS 4.0. > > Is there something else missing to finish this? > Actually, there are some more things that could be included in KAS. The patch uses "--mode=unshare" for mmdebstrap. It's currently runs under sudo, but also can be run under user. But this requires "newuidmap" utility provided by "uidmap" packages. Also, we have "fakeroot" preinstalled, but not "fakechroot". Currently it's not used, but potentially could be used in order to reduce "sudo" calls number. So, having "uidmap" and "fakechroot" preinstalled with KAS would be nice. > All the best, > Tobias ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH v3 0/7] Migrate to mmdebstrap 2023-09-20 7:24 ` Uladzimir Bely @ 2023-09-20 7:30 ` Uladzimir Bely 2023-09-21 4:36 ` Jan Kiszka 0 siblings, 1 reply; 16+ messages in thread From: Uladzimir Bely @ 2023-09-20 7:30 UTC (permalink / raw) To: Schaffner, Tobias, isar-users On Wed, 2023-09-20 at 10:24 +0300, Uladzimir Bely wrote: > On Tue, 2023-09-19 at 09:14 +0000, Schaffner, Tobias wrote: > > Hi Uladzimir, > > > > some of the patches of this story needed a rebase. Find them > > attached. > > > > On 20.06.23 12:00, Uladzimir Bely wrote: > > ... > > > > > > > > Is this now settled? Asking as I'm planning to release kas 4.0 > > > > with > > > > the > > > > dependencies in patch 7 resolved these days, and I'd like to > > > > see > > > > this > > > > targeting the final version of this feature. > > > > > > > > Jan > > > > > > > > > > We are currently upgrading our interlnal CI and need to run more > > > jobs > > > before merging it. > > > > > > But anyway, if mmdebstrap (and other packages mentioned in patch > > > 3) > > > is > > > preinstalled in kas 4.0 image, it would be nice. > > > > The dependencies needed are already included in KAS 4.0. > > > > Is there something else missing to finish this? > > > > Actually, there are some more things that could be included in KAS. > > The patch uses "--mode=unshare" for mmdebstrap. It's currently runs > under sudo, but also can be run under user. But this requires > "newuidmap" utility provided by "uidmap" packages. > > Also, we have "fakeroot" preinstalled, but not "fakechroot". > Currently > it's not used, but potentially could be used in order to reduce > "sudo" > calls number. > > So, having "uidmap" and "fakechroot" preinstalled with KAS would be > nice. Also, forget to mention... It's not directly related to this patchset, but there is one more ("Improving base-apt usage") under development. It requires "python3- apt" package preinstalled. Currently it's used by some standalone script, but potentially could also be used directly from isar bbclasses/recipes, instead of calling "apt-get" commands. The package is quite small and if nobody against, it would be nice to have it also preinstalled with newer KAS> > > > All the best, > > Tobias > ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH v3 0/7] Migrate to mmdebstrap 2023-09-20 7:30 ` Uladzimir Bely @ 2023-09-21 4:36 ` Jan Kiszka 0 siblings, 0 replies; 16+ messages in thread From: Jan Kiszka @ 2023-09-21 4:36 UTC (permalink / raw) To: Uladzimir Bely, Schaffner, Tobias, isar-users On 20.09.23 09:30, Uladzimir Bely wrote: > On Wed, 2023-09-20 at 10:24 +0300, Uladzimir Bely wrote: >> On Tue, 2023-09-19 at 09:14 +0000, Schaffner, Tobias wrote: >>> Hi Uladzimir, >>> >>> some of the patches of this story needed a rebase. Find them >>> attached. >>> >>> On 20.06.23 12:00, Uladzimir Bely wrote: >>> ... >>>>> >>>>> Is this now settled? Asking as I'm planning to release kas 4.0 >>>>> with >>>>> the >>>>> dependencies in patch 7 resolved these days, and I'd like to >>>>> see >>>>> this >>>>> targeting the final version of this feature. >>>>> >>>>> Jan >>>>> >>>> >>>> We are currently upgrading our interlnal CI and need to run more >>>> jobs >>>> before merging it. >>>> >>>> But anyway, if mmdebstrap (and other packages mentioned in patch >>>> 3) >>>> is >>>> preinstalled in kas 4.0 image, it would be nice. >>> >>> The dependencies needed are already included in KAS 4.0. >>> >>> Is there something else missing to finish this? >>> >> >> Actually, there are some more things that could be included in KAS. >> >> The patch uses "--mode=unshare" for mmdebstrap. It's currently runs >> under sudo, but also can be run under user. But this requires >> "newuidmap" utility provided by "uidmap" packages. >> >> Also, we have "fakeroot" preinstalled, but not "fakechroot". >> Currently >> it's not used, but potentially could be used in order to reduce >> "sudo" >> calls number. >> >> So, having "uidmap" and "fakechroot" preinstalled with KAS would be >> nice. > > Also, forget to mention... > > It's not directly related to this patchset, but there is one more > ("Improving base-apt usage") under development. It requires "python3- > apt" package preinstalled. > > Currently it's used by some standalone script, but potentially could > also be used directly from isar bbclasses/recipes, instead of calling > "apt-get" commands. > > The package is quite small and if nobody against, it would be nice to > have it also preinstalled with newer KAS> > Well, I asked back then for the exact list of requirements... Jan -- Siemens AG, Technology Linux Expert Center ^ permalink raw reply [flat|nested] 16+ messages in thread
end of thread, other threads:[~2023-09-21 4:36 UTC | newest] Thread overview: 16+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2023-05-31 10:10 [PATCH v3 0/7] Migrate to mmdebstrap Anton Mikanovich 2023-05-31 10:10 ` [PATCH v3 1/7] meta: Switch " Anton Mikanovich 2023-05-31 10:10 ` [PATCH v3 2/7] isar-bootstrap: Move preparations to hooks Anton Mikanovich 2023-05-31 10:10 ` [PATCH v3 3/7] isar-bootstrap: Use tar output instead of directory Anton Mikanovich 2023-05-31 10:10 ` [PATCH v3 4/7] user_manual.md: Replace debootstrap with mmdebstrap Anton Mikanovich 2023-05-31 10:11 ` [PATCH v3 5/7] user_manual.md: Update minimal host requirements Anton Mikanovich 2023-05-31 10:11 ` [PATCH v3 6/7] isar-bootstrap: Remove unused code Anton Mikanovich 2023-05-31 10:11 ` [PATCH v3 7/7] CI: Install mmdebstrap Anton Mikanovich 2023-06-20 5:30 ` [PATCH v3 0/7] Migrate to mmdebstrap Jan Kiszka 2023-06-20 10:00 ` Uladzimir Bely 2023-06-20 10:22 ` Jan Kiszka 2023-07-14 16:40 ` vijai kumar 2023-09-19 9:14 ` Schaffner, Tobias 2023-09-20 7:24 ` Uladzimir Bely 2023-09-20 7:30 ` Uladzimir Bely 2023-09-21 4:36 ` Jan Kiszka
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox