From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7239302955540676608 X-Received: by 2002:adf:dd4d:0:b0:314:350a:6912 with SMTP id u13-20020adfdd4d000000b00314350a6912mr724124wrm.36.1689918846808; Thu, 20 Jul 2023 22:54:06 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:600c:5113:b0:3fa:8731:d752 with SMTP id o19-20020a05600c511300b003fa8731d752ls27837wms.0.-pod-prod-08-eu; Thu, 20 Jul 2023 22:54:05 -0700 (PDT) X-Google-Smtp-Source: APBJJlG3/AqlYCNF3reNVAYjTAxYnQElB/p8in7i2yzOAmRzXwQLrZX9WnBEqHTk8GFdXl+c1MaN X-Received: by 2002:adf:f145:0:b0:314:d31:f7 with SMTP id y5-20020adff145000000b003140d3100f7mr592712wro.63.1689918845023; Thu, 20 Jul 2023 22:54:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1689918845; cv=none; d=google.com; s=arc-20160816; b=thudqyTAoKUhqCwGJT4Ri+Msy06jZ8RH5YcsciogHQFoIXhMd8QQrFW7XeDSnP/T7s j70aAWIaUSMp8JPE+h1Vx5t8zWSnrvRPcO0lhxtBymp+Zisn1sHqfzjrtzDNN+BbM582 9KxiWglCOfU8CzBwsdME2IJnsVKO8oEXegenwYJ7ooZA81wFNLWM00GgNHcFg54zWUUv NW8NQQ4sHcgF7dHGeQHaG+U4VgPYeokFte/FLDmPvL/FQkA5RwemTLubRlQTj1q4ZhSl /Vj0WaHE7ZeMS2GA3CUyjg1LevX6VJ4dSmhBqbFg4VbhHGvcNctYWkH1DoFaqxq2Xe52 MBRg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from; bh=j70fFQ/RwDV/l4DmqnFDJDhXMNPicRdbYe8cnfzcbzY=; fh=swRs1+OhWjovx1IrvOlKBWsdq9manB5qdcuGoHoh5Ak=; b=AUD42ysRezDfKxAU3KeQrZ6KxjAnC1sX0DqrJSE/bQCo8+RGjcV65SuybT50proVG7 UDlQMfw34vK302hML9SXzzH8wP1pNKXYSf6aEJhErQT7dDrKzzpaXuC6hN3Mq9226UtX LE2Kj1zu8IMBObWIWCzjPCEneRM++SLiIc2e89Q3WKQ3tZ1Mvtt+rl3WikieooXOIB43 lZxojKQ1xqK35fxt2LGL4sz/0LSn+mklzZu1YQsGR0CZu5CSxz/LJJc5I3LK6BfXCdTn P7lZ3j+iq1IXd0gvvEme6+Ug6+rIgELRQW4OT7Ds3U+hVmS0zsc6neHoACw3Ha4BV45I x+bw== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Return-Path: Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id bq2-20020a5d5a02000000b0031597ce1881si165486wrb.6.2023.07.20.22.54.04 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Thu, 20 Jul 2023 22:54:04 -0700 (PDT) Received-SPF: pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Received: from baighyz.m.ilbers.de (host-80-81-17-52.static.customer.m-online.net [80.81.17.52]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 36L5s2TC027957 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 21 Jul 2023 07:54:04 +0200 From: Uladzimir Bely To: isar-users@googlegroups.com Subject: [PATCH v6 3/9] imager: Move image types to schroot Date: Fri, 21 Jul 2023 07:53:56 +0200 Message-Id: <20230721055402.13764-4-ubely@ilbers.de> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20230721055402.13764-1-ubely@ilbers.de> References: <20230721055402.13764-1-ubely@ilbers.de> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-TUID: UwIc6YGEc3J/ From: Anton Mikanovich Run imagetype-related commands inside schroot session during images prepare steps. Signed-off-by: Anton Mikanovich --- meta/classes/image.bbclass | 7 ++- meta/classes/imagetypes_vm.bbclass | 9 +-- meta/classes/imagetypes_wic.bbclass | 89 ++++++++++------------------- 3 files changed, 38 insertions(+), 67 deletions(-) diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass index 53859eb9..42e4b0c9 100644 --- a/meta/classes/image.bbclass +++ b/meta/classes/image.bbclass @@ -163,7 +163,7 @@ inherit ${IMGCLASSES} # convenience variables to be used by CMDs IMAGE_FILE_HOST = "${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.${type}" IMAGE_FILE_CHROOT = "${PP_DEPLOY}/${IMAGE_FULLNAME}.${type}" -SUDO_CHROOT = "sudo chroot ${BUILDCHROOT_DIR}" +SUDO_CHROOT = "imager_run -d ${PP_ROOTFS} -u root --" # hook up IMAGE_CMD_* python() { @@ -235,7 +235,6 @@ python() { imager_build_deps.add(dep) # construct image command - cmds.append('\timage_do_mounts') image_cmd = localdata.getVar('IMAGE_CMD:' + bt_clean) if image_cmd: localdata.setVar('type', bt) @@ -409,6 +408,10 @@ python do_image_tools() { } addtask image_tools before do_build after do_rootfs +# all imagetypes are depend on schroot and isar-apt +do_image_tools[depends] = "${SCHROOT_DEP} isar-apt:do_cache_config" +do_image_tools[deptask] = "do_deploy_deb" + python do_image() { """Virtual task""" pass diff --git a/meta/classes/imagetypes_vm.bbclass b/meta/classes/imagetypes_vm.bbclass index cd56d31b..56410de9 100644 --- a/meta/classes/imagetypes_vm.bbclass +++ b/meta/classes/imagetypes_vm.bbclass @@ -4,8 +4,6 @@ # This class allows to generate images for VMware and VirtualBox # -inherit buildchroot - FILESEXTRAPATHS:prepend = "${LAYERDIR_core}/recipes-core/images:" OVF_TEMPLATE_FILE ?= "vm-img-virtualbox.ovf.tmpl" IMAGE_SRC_URI:ova = "file://${OVF_TEMPLATE_FILE}" @@ -36,9 +34,8 @@ CONVERSION_OPTIONS = "${@set_convert_options(d)}" convert_wic() { rm -f '${DEPLOY_DIR_IMAGE}/${VIRTUAL_MACHINE_IMAGE_FILE}' - image_do_mounts bbnote "Creating ${VIRTUAL_MACHINE_IMAGE_FILE} from ${SOURCE_IMAGE_FILE}" - sudo -E chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} \ + imager_run -p -d ${PP_WORK} -- \ /usr/bin/qemu-img convert -f raw -O ${VIRTUAL_MACHINE_IMAGE_TYPE} ${CONVERSION_OPTIONS} \ '${PP_DEPLOY}/${SOURCE_IMAGE_FILE}' '${VIRTUAL_MACHINE_DISK}' } @@ -81,7 +78,7 @@ IMAGE_CMD:ova() { export LAST_CHANGE=$(date -u "+%Y-%m-%dT%H:%M:%SZ") export OVA_FIRMWARE_UPPERCASE=$(echo ${OVA_FIRMWARE} | tr '[a-z]' '[A-Z]') export OVF_TEMPLATE_STAGE2=$(echo ${OVF_TEMPLATE_FILE} | sed 's/.tmpl$//' ) - sudo -Es chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} <<'EOSUDO' + imager_run -p -d ${PP_WORK} <<'EOIMAGER' set -e export DISK_SIZE_BYTES=$(qemu-img info -f vmdk "${VIRTUAL_MACHINE_DISK}" \ | gawk 'match($0, /^virtual size:.*\(([0-9]+) bytes\)/, a) {print a[1]}') @@ -98,6 +95,6 @@ IMAGE_CMD:ova() { tar -uvf ${PP_DEPLOY}/${OVA_NAME}.ova -C ${PP_DEPLOY} ${OVA_NAME}.mf fi tar -uvf ${PP_DEPLOY}/${OVA_NAME}.ova -C ${PP_DEPLOY} ${VIRTUAL_MACHINE_IMAGE_FILE} -EOSUDO +EOIMAGER } IMAGE_CMD:ova[depends] = "${PN}:do_transform_template" diff --git a/meta/classes/imagetypes_wic.bbclass b/meta/classes/imagetypes_wic.bbclass index 5be53f60..9a0bc2b7 100644 --- a/meta/classes/imagetypes_wic.bbclass +++ b/meta/classes/imagetypes_wic.bbclass @@ -83,8 +83,6 @@ python () { bb.build.addtask('do_transform_template', 'do_image_wic', None, d) } -inherit buildchroot - IMAGER_INSTALL:wic += "${WIC_IMAGER_INSTALL}" # wic comes with reasonable defaults, and the proper interface is the wks file ROOTFS_EXTRA ?= "0" @@ -95,7 +93,6 @@ STAGING_DIR ?= "${TMPDIR}" IMAGE_BASENAME ?= "${PN}-${DISTRO}" FAKEROOTCMD ?= "${SCRIPTSDIR}/wic_fakeroot" RECIPE_SYSROOT_NATIVE ?= "/" -BUILDCHROOT_DIR = "${BUILDCHROOT_TARGET_DIR}" WIC_CREATE_EXTRA_ARGS ?= "" WIC_DEPLOY_PARTITIONS ?= "0" @@ -145,26 +142,11 @@ check_for_wic_warnings() { do_image_wic[file-checksums] += "${WKS_FILE_CHECKSUM}" IMAGE_CMD:wic() { - wic_do_mounts generate_wic_image check_for_wic_warnings } -wic_do_mounts[vardepsexclude] += "BITBAKEDIR" -wic_do_mounts() { - buildchroot_do_mounts - sudo -s <<'EOSUDO' - ( flock 9 - set -e - for dir in ${BBLAYERS} ${STAGING_DIR} ${SCRIPTSDIR} ${BITBAKEDIR}; do - mkdir -p ${BUILDCHROOT_DIR}/$dir - if ! mountpoint ${BUILDCHROOT_DIR}/$dir >/dev/null 2>&1; then - mount --bind --make-private $dir ${BUILDCHROOT_DIR}/$dir - fi - done - ) 9>${MOUNT_LOCKFILE} -EOSUDO -} +SCHROOT_MOUNTS += "${BBLAYERS} ${STAGING_DIR} ${SCRIPTSDIR} ${BITBAKEDIR}" generate_wic_image[vardepsexclude] += "WKS_FULL_PATH BITBAKEDIR TOPDIR" generate_wic_image() { @@ -177,50 +159,39 @@ generate_wic_image() { mkdir -p ${IMAGE_ROOTFS}/../pseudo touch ${IMAGE_ROOTFS}/../pseudo/files.db - # create the temp dir in the buildchroot to ensure uniqueness - WICTMP=$(cd ${BUILDCHROOT_DIR}; mktemp -d -p tmp) - - sudo -E chroot ${BUILDCHROOT_DIR} \ - sh -c ' \ - BITBAKEDIR="$1" - SCRIPTSDIR="$2" - WKS_FULL_PATH="$3" - STAGING_DIR="$4" - MACHINE="$5" - WICTMP="$6" - IMAGE_FULLNAME="$7" - IMAGE_BASENAME="$8" - shift 8 - # The python path is hard-coded as /usr/bin/python3-native/python3 in wic. Handle that. - mkdir -p /usr/bin/python3-native/ - if [ $(head -1 $(which bmaptool) | grep python3) ];then + imager_run -p -d ${PP_WORK} -u root <<'EOIMAGER' + set -e + + # The python path is hard-coded as /usr/bin/python3-native/python3 in wic. Handle that. + mkdir -p /usr/bin/python3-native/ + if [ $(head -1 $(which bmaptool) | grep python3) ];then ln -fs /usr/bin/python3 /usr/bin/python3-native/python3 - else + else ln -fs /usr/bin/python2 /usr/bin/python3-native/python3 - fi - export PATH="$BITBAKEDIR/bin:$PATH" - "$SCRIPTSDIR"/wic create "$WKS_FULL_PATH" \ - --vars "$STAGING_DIR/$MACHINE/imgdata/" \ - -o "/$WICTMP/${IMAGE_FULLNAME}.wic/" \ + fi + + export PATH="${BITBAKEDIR}/bin:$PATH" + + "${SCRIPTSDIR}"/wic create "${WKS_FULL_PATH}" \ + --vars "${STAGING_DIR}/${MACHINE}/imgdata/" \ + -o "/tmp/${IMAGE_FULLNAME}.wic/" \ --bmap \ - -e "$IMAGE_BASENAME" $@' \ - my_script "${BITBAKEDIR}" "${SCRIPTSDIR}" "${WKS_FULL_PATH}" "${STAGING_DIR}" \ - "${MACHINE}" "${WICTMP}" "${IMAGE_FULLNAME}" "${IMAGE_BASENAME}" \ - ${WIC_CREATE_EXTRA_ARGS} + -e "${IMAGE_BASENAME}" ${WIC_CREATE_EXTRA_ARGS} + + WIC_DIRECT=$(ls -t -1 /tmp/${IMAGE_FULLNAME}.wic/*.direct | head -1) + mv -f ${WIC_DIRECT} ${PP_DEPLOY}/${IMAGE_FULLNAME}.wic + mv -f ${WIC_DIRECT}.bmap ${PP_DEPLOY}/${IMAGE_FULLNAME}.wic.bmap + # deploy partition files if requested (ending with .p) + if [ "${WIC_DEPLOY_PARTITIONS}" -eq "1" ]; then + # locate *.direct.p partition files + find "/tmp/${IMAGE_FULLNAME}.wic/" -type f -regextype sed -regex ".*\.direct.*\.p[0-9]\{1,\}" | while read f; do + suffix=$(basename $f | sed 's/.*\.direct\(.*\)/\1/') + mv -f ${f} ${PP_DEPLOY}/${IMAGE_FULLNAME}.wic${suffix} + done + fi +EOIMAGER sudo chown -R $(stat -c "%U" ${LAYERDIR_core}) ${LAYERDIR_core} ${LAYERDIR_isar} ${SCRIPTSDIR} || true - WIC_DIRECT=$(ls -t -1 ${BUILDCHROOT_DIR}/$WICTMP/${IMAGE_FULLNAME}.wic/*.direct | head -1) - sudo chown -R $(id -u):$(id -g) ${BUILDCHROOT_DIR}/${WICTMP} - mv -f ${WIC_DIRECT} ${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.wic - mv -f ${WIC_DIRECT}.bmap ${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.wic.bmap - # deploy partition files if requested (ending with .p) - if [ "${WIC_DEPLOY_PARTITIONS}" -eq "1" ]; then - # locate *.direct.p partition files - find ${BUILDCHROOT_DIR}/${WICTMP} -type f -regextype sed -regex ".*\.direct.*\.p[0-9]\{1,\}" | while read f; do - suffix=$(basename $f | sed 's/.*\.direct\(.*\)/\1/') - mv -f ${f} ${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.wic${suffix} - done - fi - rm -rf ${BUILDCHROOT_DIR}/${WICTMP} + sudo chown -R $(id -u):$(id -g) "${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.wic"* rm -rf ${IMAGE_ROOTFS}/../pseudo } -- 2.20.1