From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6790334981638979584 X-Received: by 2002:ac2:5467:: with SMTP id e7mr3709778lfn.74.1581434443965; Tue, 11 Feb 2020 07:20:43 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:6512:3b2:: with SMTP id v18ls1547677lfp.8.gmail; Tue, 11 Feb 2020 07:20:43 -0800 (PST) X-Google-Smtp-Source: APXvYqyh6eNPhktVEKhwYRhgAyd6N5TsCBaXehYhiEOgr0OPuNuFvGtRpF7xCqI1d9yRYpsiF7RF X-Received: by 2002:a19:740a:: with SMTP id v10mr3804718lfe.65.1581434443327; Tue, 11 Feb 2020 07:20:43 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1581434443; cv=none; d=google.com; s=arc-20160816; b=BIoRKunODWAPnPgJq/C/P21ThDBioLkts4ep5paXlsqb0Wf9RCsSJWSS26hoz7uTuS xo3pDLhFdiEiv1NhrQDVfxMdkMfRgNmyJkQiAt2GVDMTMIuCDR+DYHy/Bu+/wTc9JYH0 uG9X7Hqp1uU+JV5GLHINB/sdS82R3cG3edR3a5XczjBqBwOLaF8FHvOv2W2OWQyb3LlC GP9d4NkNncZ1pUqsnRmAov2NaKAubHa5H8N90RY1zpjubigEqkQD6XEINCs6HQRmlPdl MLYtYYR2MTnpxeuGQ4pAmL8uH9uycvuJMU3Q1fj39fiW0FI8pyV3eKrO8MTsYpBtBQuW zXGQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date; bh=O139vYjPHEcmbd5bvly77FXIQZHssoBw1VwqZ3KHtjU=; b=JKdt7ePoXVvtoAcxsCS5by3K5QlhhdEdLxUgTYIMs8QvhbVGMWwivVg377yb7ELssY swnhtZDgMbTTyZRRe/wuVR6w/Y8oxKuNnFVhomc3KM9D7sllTx8EQj5cSpd0B4sru48j jpGAP0Qu+PQugojDq2LY7+H/Zd3+lBmtQuxVFMPoVN1dFhTWNyH7qMYS76jCAv3oQse3 7gBM/dYTggrTKOZKkKj3av36KH7+lsTgG9pKF2G8FlSnu0JkfV0vo9/Y+vyji0SwuJ0E LIACUkIdoLvd+F7I8USB++wR4HMqZ4pk4oQul7Wx9HRsqO6YNrWJOI9YTJiqYOpBf/ts xTUg== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.14 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from david.siemens.de (david.siemens.de. [192.35.17.14]) by gmr-mx.google.com with ESMTPS id p12si186077lji.1.2020.02.11.07.20.43 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 11 Feb 2020 07:20:43 -0800 (PST) Received-SPF: pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.14 as permitted sender) client-ip=192.35.17.14; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.14 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: from mail2.sbs.de (mail2.sbs.de [192.129.41.66]) by david.siemens.de (8.15.2/8.15.2) with ESMTPS id 01BFKfmj005940 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 11 Feb 2020 16:20:42 +0100 Received: from md1za8fc.ad001.siemens.net ([139.25.69.193]) by mail2.sbs.de (8.15.2/8.15.2) with ESMTP id 01BFKf1m029618; Tue, 11 Feb 2020 16:20:41 +0100 Date: Tue, 11 Feb 2020 16:20:41 +0100 From: Henning Schild To: vijai kumar Cc: isar-users Subject: Re: [PATCH v2] rootfs: Make rootfs finalize a separate task Message-ID: <20200211162041.3c470504@md1za8fc.ad001.siemens.net> In-Reply-To: References: <20200210053753.32341-1-Vijaikumar_Kanagarajan@mentor.com> <20200211123841.1a72daed@md1za8fc.ad001.siemens.net> X-Mailer: Claws Mail 3.17.4 (GTK+ 2.24.32; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-TUID: 6MhitzIOQTgg On Tue, 11 Feb 2020 06:14:25 -0800 vijai kumar wrote: > On Tuesday, February 11, 2020 at 5:08:43 PM UTC+5:30, Henning Schild > wrote: > > > > This patch will allow you to keep your apt downloading feature > > downstream for yourself. I would say - propose it again together > > with the feature. > > > > I can push that feature if it is needed upstream, which I doubt, > since I don't vision an upstream use-case where-in one would need to > download all the sources. The motivation is exactly yours, building a serious product, with an eye on possibly maintaining packages longer than upstream and OSS license clearing. > Also, It would need your base-apt series. Even better, that needs attention ;). > > > > In fact if the feature was in Isar the whole problem would go away, > > unless you have more postprocess functions. > > > > Yes. We have one per se for our QA layer. To export dpkg status file > to deploy directory. This will be used by debsecan. > > > > We discussed postprocessing a couple of times, it is really bad > > style and enabling it as a feature that is easy to use we provoke > > downstream layers making mistakes by implementing their stuff as > > such postprocess functions. > > > > I see the ability to add custom post-processing as a useful feature. > Not sure if anyone actually uses them in their downstream layers. It > is good to have if you know what you are doing. I have seen some downstream layers where clearly not everyone knows what they are doing. > As long as this provision is there, people would use it. If we feel > that this > provision is unnecessary and would lead to issues, well, we could go > ahead and remove it. Removing it later on would break downstream layers. So it is easy and risky to add it, and hard to remove it again. Henning > Thanks, > Vijai Kumar K > > > Henning > > > > On Mon, 10 Feb 2020 11:07:53 +0530 > > Vijai Kumar K > wrote: > > > > > With the current implementation it is difficult to append a > > > postprocess function which requires a chroot environment. > > > For example, to add a postprocess function which runs apt-get to > > > download all source of packages installed in the target. > > > > > > rootfs_postprocess_finalize is not actually an optional feature > > > but instead a necessary cleanup function for image class. > > > So, move the implementation to image class and make it as a task. > > > > > > Signed-off-by: Vijai Kumar K > > > --- > > > > > > Changes in v2: > > > - The solution is changed to remove rootfs_finalize from > > > ROOTFS_POSTPROCESS_COMMAND. > > > > > > meta/classes/image.bbclass | 41 > > > ++++++++++++++++++++++++++++++++++++- meta/classes/rootfs.bbclass > > > | 39 ----------------------------------- 2 files changed, 40 > > > insertions(+), 40 deletions(-) > > > > > > diff --git a/meta/classes/image.bbclass > > > b/meta/classes/image.bbclass index 725bc04..98338ac 100644 > > > --- a/meta/classes/image.bbclass > > > +++ b/meta/classes/image.bbclass > > > @@ -60,7 +60,7 @@ image_do_mounts() { > > > } > > > > > > ROOTFSDIR = "${IMAGE_ROOTFS}" > > > -ROOTFS_FEATURES += "copy-package-cache clean-package-cache > > > finalize-rootfs generate-manifest" +ROOTFS_FEATURES += > > > "copy-package-cache clean-package-cache generate-manifest" > > > ROOTFS_PACKAGES += "${IMAGE_PREINSTALL} ${IMAGE_INSTALL}" > > > ROOTFS_MANIFEST_DEPLOY_DIR ?= "${DEPLOY_DIR_IMAGE}" > > > @@ -169,5 +169,44 @@ python do_deploy() { > > > } > > > addtask deploy before do_build after do_image > > > > > > +do_rootfs_finalize() { > > > + sudo -s <<'EOSUDO' > > > + test -e "${ROOTFSDIR}/chroot-setup.sh" && \ > > > + "${ROOTFSDIR}/chroot-setup.sh" "cleanup" > > > "${ROOTFSDIR}" > > > + rm -f "${ROOTFSDIR}/chroot-setup.sh" > > > + > > > + test ! -e "${ROOTFSDIR}/usr/share/doc/qemu-user-static" > > > && \ > > > + find "${ROOTFSDIR}/usr/bin" \ > > > + -maxdepth 1 -name 'qemu-*-static' -type f > > > -delete > > > + > > > + mountpoint -q '${ROOTFSDIR}/isar-apt' && \ > > > + umount -l ${ROOTFSDIR}/isar-apt > > > + rmdir --ignore-fail-on-non-empty ${ROOTFSDIR}/isar-apt > > > + > > > + mountpoint -q '${ROOTFSDIR}/base-apt' && \ > > > + umount -l ${ROOTFSDIR}/base-apt > > > + rmdir --ignore-fail-on-non-empty ${ROOTFSDIR}/base-apt > > > + > > > + mountpoint -q '${ROOTFSDIR}/dev' && \ > > > + umount -l ${ROOTFSDIR}/dev > > > + mountpoint -q '${ROOTFSDIR}/sys' && \ > > > + umount -l ${ROOTFSDIR}/proc > > > + mountpoint -q '${ROOTFSDIR}/sys' && \ > > > + umount -l ${ROOTFSDIR}/sys > > > + > > > + rm -f > > > "${ROOTFSDIR}/etc/apt/apt.conf.d/55isar-fallback.conf" > > > + > > > + rm -f > > > "${ROOTFSDIR}/etc/apt/sources.list.d/isar-apt.list" > > > + rm -f "${ROOTFSDIR}/etc/apt/preferences.d/isar-apt" > > > + rm -f > > > "${ROOTFSDIR}/etc/apt/sources.list.d/base-apt.list" > > > + > > > + mv "${ROOTFSDIR}/etc/apt/sources-list" \ > > > + "${ROOTFSDIR}/etc/apt/sources.list.d/bootstrap.list" > > > + > > > + rm -f "${ROOTFSDIR}/etc/apt/sources-list" > > > +EOSUDO > > > +} > > > +addtask rootfs_finalize before do_rootfs after > > > do_rootfs_postprocess > > > + > > > # Last so that the image type can overwrite tasks if needed > > > inherit ${IMAGE_TYPE} > > > diff --git a/meta/classes/rootfs.bbclass > > > b/meta/classes/rootfs.bbclass index 64eaef7..153038a 100644 > > > --- a/meta/classes/rootfs.bbclass > > > +++ b/meta/classes/rootfs.bbclass > > > @@ -197,45 +197,6 @@ rootfs_generate_manifest () { > > > ${ROOTFS_MANIFEST_DEPLOY_DIR}/"${PF}".manifest > > > } > > > > > > -ROOTFS_POSTPROCESS_COMMAND += > > > "${@bb.utils.contains('ROOTFS_FEATURES', 'finalize-rootfs', > > > 'rootfs_postprocess_finalize', '', d)}" > > > -rootfs_postprocess_finalize() { > > > - sudo -s <<'EOSUDO' > > > - test -e "${ROOTFSDIR}/chroot-setup.sh" && \ > > > - "${ROOTFSDIR}/chroot-setup.sh" "cleanup" > > > "${ROOTFSDIR}" > > > - rm -f "${ROOTFSDIR}/chroot-setup.sh" > > > - > > > - test ! -e "${ROOTFSDIR}/usr/share/doc/qemu-user-static" > > > && \ > > > - find "${ROOTFSDIR}/usr/bin" \ > > > - -maxdepth 1 -name 'qemu-*-static' -type f > > > -delete > > > - > > > - mountpoint -q '${ROOTFSDIR}/isar-apt' && \ > > > - umount -l ${ROOTFSDIR}/isar-apt > > > - rmdir --ignore-fail-on-non-empty ${ROOTFSDIR}/isar-apt > > > - > > > - mountpoint -q '${ROOTFSDIR}/base-apt' && \ > > > - umount -l ${ROOTFSDIR}/base-apt > > > - rmdir --ignore-fail-on-non-empty ${ROOTFSDIR}/base-apt > > > - > > > - mountpoint -q '${ROOTFSDIR}/dev' && \ > > > - umount -l ${ROOTFSDIR}/dev > > > - mountpoint -q '${ROOTFSDIR}/sys' && \ > > > - umount -l ${ROOTFSDIR}/proc > > > - mountpoint -q '${ROOTFSDIR}/sys' && \ > > > - umount -l ${ROOTFSDIR}/sys > > > - > > > - rm -f > > > "${ROOTFSDIR}/etc/apt/apt.conf.d/55isar-fallback.conf" > > > - > > > - rm -f > > > "${ROOTFSDIR}/etc/apt/sources.list.d/isar-apt.list" > > > - rm -f "${ROOTFSDIR}/etc/apt/preferences.d/isar-apt" > > > - rm -f > > > "${ROOTFSDIR}/etc/apt/sources.list.d/base-apt.list" > > > - > > > - mv "${ROOTFSDIR}/etc/apt/sources-list" \ > > > - "${ROOTFSDIR}/etc/apt/sources.list.d/bootstrap.list" > > > - > > > - rm -f "${ROOTFSDIR}/etc/apt/sources-list" > > > -EOSUDO > > > -} > > > - > > > do_rootfs_postprocess[vardeps] = "${ROOTFS_POSTPROCESS_COMMAND}" > > > python do_rootfs_postprocess() { > > > # Take care that its correctly mounted: > > > > >