From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6842309179660566528 X-Received: by 2002:a17:906:f911:: with SMTP id lc17mr14226140ejb.330.1593433327000; Mon, 29 Jun 2020 05:22:07 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:6402:1d37:: with SMTP id dh23ls3217072edb.1.gmail; Mon, 29 Jun 2020 05:22:06 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzytQ0Khhz0IIV4JG3fDDzDy38AoPIyOb+yuD8Z5TGZwJ1kAp+Pw1eiklqT1bN4jKeV/Au9 X-Received: by 2002:a50:9f22:: with SMTP id b31mr17590479edf.24.1593433326431; Mon, 29 Jun 2020 05:22:06 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1593433326; cv=none; d=google.com; s=arc-20160816; b=aIqM/g5sPXAgnythfKF5cg8QTWVh18Ud38WzuG0ulblvzD6YV6ek/PmK4IVOYjWpXX k5Dx8cj/L0iMteq3J7lYVxXRQA3XNBA1iflBCFokAQFiKA0EotPBfb73O5MijqCn4hPc rr3Err37uPFM5OgPUdYIe0npFtsxFIKYkxBfh9mCKnfSsdw1dYiTaumQyFco3nI0K/ru GhgEtZuOfvEnYprNs2MMp7xmF6t8fuK60DDBZc9Hzw3wTatbZJfyuS20uk0Zjg40bHp8 XtwArcLDip+G9Wb+Jcv8PYE6OecQeuBXTZcf3sAjCSW1chrX3WKE6vwmlVvof4bOq2Gm AU4Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:date:cc:to:from:subject:message-id; bh=t9ARtiCcPsUU/BQxJE7utVDGQY4d9ypLpbgUHj6CXF8=; b=RO8H+ftLjl7JlhQXF8GkQzwQkt5/dk7va7oGwoNy1mp7ottLQYe2/f4zcZxbeSq4MJ 7VlFT76I+a9EUKAbwtt9yaV5X/xEuOwuD3R3RkWZmKa4dYA0eh4irNfxsURtrHNcWVT0 IFqLT9XDKgS8VD5JHdsjsDrbEmKRWJVstnVIF4diRwL0m9LP4sBD1E8CBst0tEw1KY5I THVkYAZxiwMYfTWJm2w15L6wNdmsQS/kR6K3YB9QgnqL4Cwi+QneSbFS2MzqmC+DOq18 yDa7xLxwLS7JlBAedWorD5nGxlfve7cm0gY1eK5rPaSBCgppraCCnRnk15M6oj14d7s5 gNjw== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=neutral (google.com: 2001:a60:0:28:0:1:25:1 is neither permitted nor denied by best guess record for domain of hws@denx.de) smtp.mailfrom=hws@denx.de Return-Path: Received: from mail-out.m-online.net (mail-out.m-online.net. [2001:a60:0:28:0:1:25:1]) by gmr-mx.google.com with ESMTPS id a23si1997827edn.0.2020.06.29.05.22.06 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 29 Jun 2020 05:22:06 -0700 (PDT) Received-SPF: neutral (google.com: 2001:a60:0:28:0:1:25:1 is neither permitted nor denied by best guess record for domain of hws@denx.de) client-ip=2001:a60:0:28:0:1:25:1; Authentication-Results: gmr-mx.google.com; spf=neutral (google.com: 2001:a60:0:28:0:1:25:1 is neither permitted nor denied by best guess record for domain of hws@denx.de) smtp.mailfrom=hws@denx.de Received: from frontend01.mail.m-online.net (unknown [192.168.8.182]) by mail-out.m-online.net (Postfix) with ESMTP id 49wRSZ1k8Tz1rv9l; Mon, 29 Jun 2020 14:22:06 +0200 (CEST) Received: from localhost (dynscan1.mnet-online.de [192.168.6.70]) by mail.m-online.net (Postfix) with ESMTP id 49wRSZ1CXhz1qqkn; Mon, 29 Jun 2020 14:22:06 +0200 (CEST) X-Virus-Scanned: amavisd-new at mnet-online.de Received: from mail.mnet-online.de ([192.168.8.182]) by localhost (dynscan1.mail.m-online.net [192.168.6.70]) (amavisd-new, port 10024) with ESMTP id F6NO6GTnys-U; Mon, 29 Jun 2020 14:22:03 +0200 (CEST) X-Auth-Info: pcYttFw8uOCo7MPqDyXWgjxu6w8ip7kY//54rCnXAgQ= Received: from maia.denx.de (p578adb1c.dip0.t-ipconnect.de [87.138.219.28]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.mnet-online.de (Postfix) with ESMTPSA; Mon, 29 Jun 2020 14:22:03 +0200 (CEST) Message-ID: <6bd78ab80bde335821d792046ad2803e4bc9bb57.camel@denx.de> Subject: Re: [PATCH] image: Run copy_boot_files after rootfs postprocessing From: Harald Seiler To: Jan Kiszka , Claudius Heine Cc: isar-users@googlegroups.com, Henning Schild Date: Mon, 29 Jun 2020 14:22:03 +0200 In-Reply-To: <5a86e555-416e-d788-2655-003403f1d190@siemens.com> References: <20200625153351.3402179-1-hws@denx.de> <20200625184822.236ff069@md1za8fc.ad001.siemens.net> <91ce92c15d267e4836ab4d9de2870bc8e6f6dfa1.camel@denx.de> <5a86e555-416e-d788-2655-003403f1d190@siemens.com> Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.34.4 (3.34.4-1.fc31) MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-TUID: FBUYVpEK/mfc Hi Jan, On Fri, 2020-06-26 at 11:12 +0200, Jan Kiszka wrote: > On 26.06.20 09:17, Claudius Heine wrote: > > Hi Harald, > > > > On 2020-06-25 19:24, Harald Seiler wrote: > > > Hello Henning, > > > > > > On Thu, 2020-06-25 at 19:02 +0200, Jan Kiszka wrote: > > > > On 25.06.20 18:48, [ext] Henning Schild wrote: > > > > > Hi Harald, > > > > > > > > > > can you elaborate on those cases? The postprocessing is hacky, if the > > > > > problem is coming from your layer you should probably keep this patch > > > > > in you layer. > > > > > > > > Basically do_generate_image_uuid from > > > > https://lore.kernel.org/cip-dev/20200625141015.31719-4-Quirin.Gylstorff@siemens.com/T/#u, > > > > just modeled as post-processing hook, rather than a task. > > > > > > For reference, this is the exact code: > > > > > > ROOTFS_POSTPROCESS_COMMAND =+ "image_postprocess_generate_uuid" > > > image_postprocess_generate_uuid() { > > > sudo sed -i '/^IMAGE_UUID=.*/d' '${IMAGE_ROOTFS}/etc/os-release' > > > echo "IMAGE_UUID=\"${IMAGE_UUID}\"" | \ > > > sudo tee -a '${IMAGE_ROOTFS}/etc/os-release' > > > > > > sudo -E chroot '${ROOTFSDIR}' \ > > > update-initramfs -u > > > } > > > > > > > Jan > > > > > > > > > Maybe you can point out an issue in isar itself, or explain how you got > > > > > into this situation? We can then see if your change is generic enough > > > > > for upstream. You could also provide the error-case from your layer as > > > > > an upstream feature, if that is generic enough. > > > > > > I think this patch addresses an issue in isar itself. There is no reason > > > for copy_boot_files() to run before the postprocessing does. I've checked > > > through the git history and the reason this relationship was introduced > > > was a bigger refactor of the task dependency chain. It does not seem to > > > be intentionally this way from what I can tell. > > > > > > The other way around makes more sense, in my opinion. As stated in the > > > commit message, postprocessing might do an update to the initramfs (as > > > seen above) and this change needs to be reflected in the deployed > > > initramfs as well, instead of silently only living in the version that is > > > part of the rootfs. > > > > > > I also checked all existing postprocessing commands and did not see any > > > that assume to be run after the boot files have been deployed. > > > > Its been a while when I implemented this, but I also thought of the > > scenario where someone would like to 'minimize' a image via the root fs > > postprocessing by deleting everything that is not needed, and that could > > possible include the kernel + initramfs, if those are stored somewhere > > else outside the root file system. So the idea was, IIRC, to move the > > kernel and initrd to the deploy dir, out of harms way, before > > postprocessing does its rootfs manipulation. > > > > So by ordering the copy_boot_files behind the root fs post processing, > > you might break other layers that rely on this ordering and have such > > 'minimization' procedures, that remove the kernel package and specific > > files. > > > > We don't have such 'minimization' stuff in upstream isar, since it > > pretty much breaks apt and dpkg, but if you do image based update, you > > might not care. > > I think the problem with this pattern is elsewhere: We should not > install stuff on the rootfs in the first place that shall not end up in > the rootfs. That this copy_boot_files thing depends on the installation > on the rootfs is actually a bug. It should use the chroot for its work, > like the imager does (for the bootloader e.g.). For kernel and DTB I am totally on your side but I'm not sure how you would want to do this for the initramfs. I think the initramfs should definitely be generated from the 'real' rootfs because otherwise you might get packages from chroot 'polluting' it in unexpected ways. Also, considering the IMAGE_UUID use-case in particular, how would you get the ID from the rootfs into the chroot for the hook to pick it up? Not sure whether there is a clean solution for this ... -- Harald DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: +49-8142-66989-62 Fax: +49-8142-66989-80 Email: hws@denx.de