From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6842309179660566528 X-Received: by 2002:a2e:b541:: with SMTP id a1mr766570ljn.4.1593158534655; Fri, 26 Jun 2020 01:02:14 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a2e:8843:: with SMTP id z3ls1670045ljj.11.gmail; Fri, 26 Jun 2020 01:02:14 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxCupiSXlAg9Tr9ODM5MKFdxVDyP3f6lbL16Pbx1/kb30MNh3TsUztFr7wSifKa/TqqtCBn X-Received: by 2002:a2e:850b:: with SMTP id j11mr349556lji.30.1593158534033; Fri, 26 Jun 2020 01:02:14 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1593158534; cv=none; d=google.com; s=arc-20160816; b=SFcv44dYedW1Uh7D8oiDElxa2XdnZZyUUk43IJv6lPHdAgTz0OyToCKLvdnCHHl7vd jwIRLRqpKqwpAgLWAbEJy5KvWzNCN0SZjYRLsw7NvmIGeAvZcnr2IzKCnd2VICUB42WR rbKAOFbr+1hk/Uy9u9/s76/Ynx/b+adC1n0gyN4us/PzKpSXpHuOyF+RWAcIzEaSDUzL UFkD4k66poxiwf2dG+NioRDkjOFdfwd77leVUsf25xNITqCsxQoOsch4nX+DMrpYeM9J yygHSg7i8zMB1iOXjUkdMDkTUFuwu8VPIM7O8sHUCR965I9WrBBKYoLmkU/ZqsIFzCDi pImQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:user-agent:references:in-reply-to:date:cc:to:from :subject:message-id; bh=WlqiLzCUA/w5S+z+tqzPfzMbY7dLPy70WHtkZdQhsVQ=; b=0SSJ0EJQYQbCsAw8qV90ZRrTKJZvIqziNn5rsLhU7FuyZbO0VgVyNHE0ZM9GwnFqZL 26N3E65rLmucgtIZFFXVwvBBdimOZCYaUSBYTD8xbmAckzfcJrUlNuAoV3o+ENCnU9XC CQW4+0AGIp28Y3fbLVwCBNd68JbmTSmmACt7t3SKYekNi9cbnFfZeL4vrJqkkVz6EF6S 7KlyeXaONGBcdZOClDPGVTQy7qWE8vasgxPomcoPZPf7CsHVlJ0G90a8Z5JD3JnnKioE +dji/pLX+EWuqJEzDPk+xkXmeozv2Rj2wMtXZsE4SHhLBHRRmVtMBgvtrBslRqoCJ2eF ayAQ== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=neutral (google.com: 2001:a60:0:28:0:1:25:1 is neither permitted nor denied by best guess record for domain of hws@denx.de) smtp.mailfrom=hws@denx.de Return-Path: Received: from mail-out.m-online.net (mail-out.m-online.net. [2001:a60:0:28:0:1:25:1]) by gmr-mx.google.com with ESMTPS id 7si913414lfk.0.2020.06.26.01.02.13 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 26 Jun 2020 01:02:13 -0700 (PDT) Received-SPF: neutral (google.com: 2001:a60:0:28:0:1:25:1 is neither permitted nor denied by best guess record for domain of hws@denx.de) client-ip=2001:a60:0:28:0:1:25:1; Authentication-Results: gmr-mx.google.com; spf=neutral (google.com: 2001:a60:0:28:0:1:25:1 is neither permitted nor denied by best guess record for domain of hws@denx.de) smtp.mailfrom=hws@denx.de Received: from frontend01.mail.m-online.net (unknown [192.168.8.182]) by mail-out.m-online.net (Postfix) with ESMTP id 49tTr527mFz1rxb9; Fri, 26 Jun 2020 10:02:13 +0200 (CEST) Received: from localhost (dynscan1.mnet-online.de [192.168.6.70]) by mail.m-online.net (Postfix) with ESMTP id 49tTr51jyPz1qqkt; Fri, 26 Jun 2020 10:02:13 +0200 (CEST) X-Virus-Scanned: amavisd-new at mnet-online.de Received: from mail.mnet-online.de ([192.168.8.182]) by localhost (dynscan1.mail.m-online.net [192.168.6.70]) (amavisd-new, port 10024) with ESMTP id Po9WbIX3AhII; Fri, 26 Jun 2020 10:02:11 +0200 (CEST) X-Auth-Info: 2G3oIXyBXHezVFIjVjam9Ze9MtCngT9qorgKNgRjVRg= Received: from maia.denx.de (p578adb1c.dip0.t-ipconnect.de [87.138.219.28]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.mnet-online.de (Postfix) with ESMTPSA; Fri, 26 Jun 2020 10:02:11 +0200 (CEST) Message-ID: <65d916368b551945611f2bb4348bb53aca4ef88d.camel@denx.de> Subject: Re: [PATCH] image: Run copy_boot_files after rootfs postprocessing From: Harald Seiler To: Claudius Heine , Jan Kiszka , Henning Schild Cc: isar-users@googlegroups.com Date: Fri, 26 Jun 2020 10:02:05 +0200 In-Reply-To: References: <20200625153351.3402179-1-hws@denx.de> <20200625184822.236ff069@md1za8fc.ad001.siemens.net> <91ce92c15d267e4836ab4d9de2870bc8e6f6dfa1.camel@denx.de> Content-Type: multipart/signed; micalg="pgp-sha256"; protocol="application/pgp-signature"; boundary="=-cQle51zrGOM59koMv8N8" User-Agent: Evolution 3.34.4 (3.34.4-1.fc31) MIME-Version: 1.0 X-TUID: oniDpVj2mxfd --=-cQle51zrGOM59koMv8N8 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hello Claudius, On Fri, 2020-06-26 at 09:17 +0200, Claudius Heine wrote: > Hi Harald, >=20 > On 2020-06-25 19:24, Harald Seiler wrote: > > Hello Henning, > >=20 > > On Thu, 2020-06-25 at 19:02 +0200, Jan Kiszka wrote: > > > On 25.06.20 18:48, [ext] Henning Schild wrote: > > > > Hi Harald, > > > >=20 > > > > can you elaborate on those cases? The postprocessing is hacky, if t= he > > > > problem is coming from your layer you should probably keep this pat= ch > > > > in you layer. > > >=20 > > > Basically do_generate_image_uuid from=20 > > > https://lore.kernel.org/cip-dev/20200625141015.31719-4-Quirin.Gylstor= ff@siemens.com/T/#u,=20 > > > just modeled as post-processing hook, rather than a task. > >=20 > > For reference, this is the exact code: > >=20 > > ROOTFS_POSTPROCESS_COMMAND =3D+ "image_postprocess_generate_uuid" > > image_postprocess_generate_uuid() { > > sudo sed -i '/^IMAGE_UUID=3D.*/d' '${IMAGE_ROOTFS}/etc/os-relea= se' > > echo "IMAGE_UUID=3D\"${IMAGE_UUID}\"" | \ > > sudo tee -a '${IMAGE_ROOTFS}/etc/os-release' > >=20 > > sudo -E chroot '${ROOTFSDIR}' \ > > update-initramfs -u > > } > >=20 > > > Jan > > >=20 > > > > Maybe you can point out an issue in isar itself, or explain how you= got > > > > into this situation? We can then see if your change is generic enou= gh > > > > for upstream. You could also provide the error-case from your layer= as > > > > an upstream feature, if that is generic enough. > >=20 > > I think this patch addresses an issue in isar itself. There is no reas= on > > for copy_boot_files() to run before the postprocessing does. I've chec= ked > > through the git history and the reason this relationship was introduced > > was a bigger refactor of the task dependency chain. It does not seem t= o > > be intentionally this way from what I can tell. > >=20 > > The other way around makes more sense, in my opinion. As stated in the > > commit message, postprocessing might do an update to the initramfs (as > > seen above) and this change needs to be reflected in the deployed > > initramfs as well, instead of silently only living in the version that = is > > part of the rootfs. > >=20 > > I also checked all existing postprocessing commands and did not see any > > that assume to be run after the boot files have been deployed. >=20 > Its been a while when I implemented this, but I also thought of the > scenario where someone would like to 'minimize' a image via the root fs > postprocessing by deleting everything that is not needed, and that could > possible include the kernel + initramfs, if those are stored somewhere > else outside the root file system. So the idea was, IIRC, to move the > kernel and initrd to the deploy dir, out of harms way, before > postprocessing does its rootfs manipulation. >=20 > So by ordering the copy_boot_files behind the root fs post processing, > you might break other layers that rely on this ordering and have such > 'minimization' procedures, that remove the kernel package and specific > files. >=20 > We don't have such 'minimization' stuff in upstream isar, since it > pretty much breaks apt and dpkg, but if you do image based update, you > might not care. I can see the use-case you are describing though I think mine (or rather the one for IMAGE_UUID) is just as valid. Both are things that need to happen after the rootfs has been created but before the imager is invoked. The difference is that one needs to happen before copy_boot_files() and one needs to happen after. I see two solutions: - Either we split postprocessing into 'early' (before boot files are deployed) and 'late' (after boot files have been deployed). This has the disadvantage of further convoluting the task chain and isn't really pretty in my opinion. - The cleanup 'postprocessing' is treated special and a different solution is created for it; one that runs after deploying the boot files. E.g. a ROOTFS_CLEANUP_FILES variable which is filled with files and directories to be removed before invoking the imager. Maybe you have another idea for this? --=20 Harald DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: +49-8142-66989-62 Fax: +49-8142-66989-80 Email: hws@denx.de --=-cQle51zrGOM59koMv8N8 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- iQJABAABCAAqFiEETcH23ThHylbMjihC+a//80/kShkFAl71q30MHGh3c0BkZW54 LmRlAAoJEPmv//NP5EoZE6MP/il7c7Q8xjO1PYnzewI4ntI18mv/NhsY9YogB5be sMYZrfO5yAoP8EVkwuKTGo69z7MtPfOE74jU4CAAtHy+CMrSv8VIzjrBUUolx3IA 3KsmDOzilTRldWi5xkDKnbeI8+Jdt/Y3QkqckMOMatxYaZE4PLCRq4DWIVA1+hn9 io8lQU2XizKzi5KIWC1lJIkzNWbfg6bxF6oqVXgt4ven+EZ/HhcoNETHorh4pekR t4mp5qvRhie4mvx2NAvpzFzotn7W4uP/e/+7zYS4ge9HSEFcnZiH3BgmEfFYBy0a CKRY3cdFXW7gZofbigrdghUDuywShomdJEaeLDocNE1IZ+mREfqxARWbvz8Ktpqe zdHp7f4C55W04u36UBLMpvkNX4Mtt2STEdhdO0is5w+9m+Yvyq9p8eLNsTd1xN+5 Z7673r9Gyj3odUa73CrvcjvfKrwtm32G0lilbqEV12i+KeBmTuX2xoXZjTQYqX0h MFeHeO3+rH0era80jm4vLnpwfm9JLKRs9xMaY+3gTpz/5JcesLoTgit7NRm9BVuL mL1MIMYJOP/AeNf1iy6zMMkqUkiSYysmD6m8bXrnu2E388aeikU87579O3cLjjEL 8i9BETj9/2cswelAyMoucRCy/z5EkRRb+chMKejDiri+b6QF/SCZDlaKcZBud5hd Br6Q =8QXc -----END PGP SIGNATURE----- --=-cQle51zrGOM59koMv8N8--