From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6842309179660566528 X-Received: by 2002:adf:e4cc:: with SMTP id v12mr2776193wrm.92.1593162758369; Fri, 26 Jun 2020 02:12:38 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a1c:1fc9:: with SMTP id f192ls1353506wmf.1.gmail; Fri, 26 Jun 2020 02:12:37 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwcPzO4vDXvKT1y9kE2gtdtjFAEEK9X7rdFY+iWulkKFDqVjpLhqfVA4SneJLL6UDlWyP5V X-Received: by 2002:a1c:2044:: with SMTP id g65mr2577013wmg.16.1593162757655; Fri, 26 Jun 2020 02:12:37 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1593162757; cv=none; d=google.com; s=arc-20160816; b=Hum7motR3xAGQEdYDZG3lq1OD1/1G5nBDYnVWaQcV9BCctk+xl2jZ06IKxUkSCLXMo tZS/5FiBn3WNNTzcqqPmfNmpcMsCinKvZXXKE5TJHGblGpKCEl3wAd9RFrnEZrwQWGxS h2+hVtyh8xlDVhsyA5IQcciNr5ER7JBRd95uPnA1L6fg3UdRgtOzt5F3zgMPQPw7UwRs 4IDyVgWXfbinzMh9AKEhxgr02B4Gw98FMILKQ44WcQhXJBfYLoH+P0TeFRohXx66PTSE vR8WYZCOh91ellYs+aKR6yPe5CGpnJC7ExmQ6sBHPuuvDCBcUDQU9p4ispjrDjdCvTmN G8Ww== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:cc:to:subject; bh=jpGtnAzok1iXu17lA5sZovp5YU+nPNK7RP4nUU6CVzM=; b=UlFl7EKdZPfCR0coBKwzbD7hrS0bNapYOUA2OntAvfoT3XT/WPpYu2RxmR0NVvJSk/ KwoyQGaSc1POxlaAiB+tqAI0hTzrsWwyQqoFnzpXNUYUpoUNFqtOdlkyXQ0zhG90OjBQ cSIkwHbhXFeVwFgkIBvyp7NV01bOha9vpn6zONVb4l5nsUFvPN+M6qA0LPGFWYGOVSpx OhprNV68hx3T30NtmR1WvqlzclV+YreR6bkTLY8LG81nCgAxHYes1Xd9JSHx5sZStaPi UsyGSWsQQblgYli0eJP3BYml/StHTKNtchOivuIeWsOEp78vEm5VhOw36Gdv8YneQ05m Jp6w== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 194.138.37.40 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from gecko.sbs.de (gecko.sbs.de. [194.138.37.40]) by gmr-mx.google.com with ESMTPS id g141si678178wmg.0.2020.06.26.02.12.37 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 26 Jun 2020 02:12:37 -0700 (PDT) Received-SPF: pass (google.com: domain of jan.kiszka@siemens.com designates 194.138.37.40 as permitted sender) client-ip=194.138.37.40; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 194.138.37.40 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: from mail2.sbs.de (mail2.sbs.de [192.129.41.66]) by gecko.sbs.de (8.15.2/8.15.2) with ESMTPS id 05Q9CaJF007934 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 26 Jun 2020 11:12:37 +0200 Received: from [167.87.241.222] ([167.87.241.222]) by mail2.sbs.de (8.15.2/8.15.2) with ESMTP id 05Q9CaGi002249; Fri, 26 Jun 2020 11:12:36 +0200 Subject: Re: [PATCH] image: Run copy_boot_files after rootfs postprocessing To: Claudius Heine , Harald Seiler , Henning Schild Cc: isar-users@googlegroups.com References: <20200625153351.3402179-1-hws@denx.de> <20200625184822.236ff069@md1za8fc.ad001.siemens.net> <91ce92c15d267e4836ab4d9de2870bc8e6f6dfa1.camel@denx.de> From: Jan Kiszka Message-ID: <5a86e555-416e-d788-2655-003403f1d190@siemens.com> Date: Fri, 26 Jun 2020 11:12:36 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.9.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-TUID: dd4ceeBj5S/l On 26.06.20 09:17, Claudius Heine wrote: > Hi Harald, > > On 2020-06-25 19:24, Harald Seiler wrote: >> Hello Henning, >> >> On Thu, 2020-06-25 at 19:02 +0200, Jan Kiszka wrote: >>> On 25.06.20 18:48, [ext] Henning Schild wrote: >>>> Hi Harald, >>>> >>>> can you elaborate on those cases? The postprocessing is hacky, if the >>>> problem is coming from your layer you should probably keep this patch >>>> in you layer. >>> >>> Basically do_generate_image_uuid from >>> https://lore.kernel.org/cip-dev/20200625141015.31719-4-Quirin.Gylstorff@siemens.com/T/#u, >>> just modeled as post-processing hook, rather than a task. >> >> For reference, this is the exact code: >> >> ROOTFS_POSTPROCESS_COMMAND =+ "image_postprocess_generate_uuid" >> image_postprocess_generate_uuid() { >> sudo sed -i '/^IMAGE_UUID=.*/d' '${IMAGE_ROOTFS}/etc/os-release' >> echo "IMAGE_UUID=\"${IMAGE_UUID}\"" | \ >> sudo tee -a '${IMAGE_ROOTFS}/etc/os-release' >> >> sudo -E chroot '${ROOTFSDIR}' \ >> update-initramfs -u >> } >> >>> Jan >>> >>>> Maybe you can point out an issue in isar itself, or explain how you got >>>> into this situation? We can then see if your change is generic enough >>>> for upstream. You could also provide the error-case from your layer as >>>> an upstream feature, if that is generic enough. >> >> I think this patch addresses an issue in isar itself. There is no reason >> for copy_boot_files() to run before the postprocessing does. I've checked >> through the git history and the reason this relationship was introduced >> was a bigger refactor of the task dependency chain. It does not seem to >> be intentionally this way from what I can tell. >> >> The other way around makes more sense, in my opinion. As stated in the >> commit message, postprocessing might do an update to the initramfs (as >> seen above) and this change needs to be reflected in the deployed >> initramfs as well, instead of silently only living in the version that is >> part of the rootfs. >> >> I also checked all existing postprocessing commands and did not see any >> that assume to be run after the boot files have been deployed. > > Its been a while when I implemented this, but I also thought of the > scenario where someone would like to 'minimize' a image via the root fs > postprocessing by deleting everything that is not needed, and that could > possible include the kernel + initramfs, if those are stored somewhere > else outside the root file system. So the idea was, IIRC, to move the > kernel and initrd to the deploy dir, out of harms way, before > postprocessing does its rootfs manipulation. > > So by ordering the copy_boot_files behind the root fs post processing, > you might break other layers that rely on this ordering and have such > 'minimization' procedures, that remove the kernel package and specific > files. > > We don't have such 'minimization' stuff in upstream isar, since it > pretty much breaks apt and dpkg, but if you do image based update, you > might not care. I think the problem with this pattern is elsewhere: We should not install stuff on the rootfs in the first place that shall not end up in the rootfs. That this copy_boot_files thing depends on the installation on the rootfs is actually a bug. It should use the chroot for its work, like the imager does (for the bootloader e.g.). Jan -- Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux