From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6818448905053667328 X-Received: by 2002:a5d:488a:: with SMTP id g10mr28539447wrq.262.1587553616991; Wed, 22 Apr 2020 04:06:56 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:6000:128f:: with SMTP id f15ls1218737wrx.10.gmail; Wed, 22 Apr 2020 04:06:56 -0700 (PDT) X-Google-Smtp-Source: APiQypKCeGWnc/kqNy2ZAupBZgFRCRKbyvMsD813oagMmAyJgBkahRJjWo+DhT3yQ7uMrz/n17j0 X-Received: by 2002:adf:dfc2:: with SMTP id q2mr31415970wrn.390.1587553616339; Wed, 22 Apr 2020 04:06:56 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1587553616; cv=none; d=google.com; s=arc-20160816; b=Voo2mqIk5kx8iiP1qPW7L3uMmAQIXSdFwY+AgbxyS+oWtqMufVUni+q+JDhTFWkC+H MuOCfYEY8jcKzx74XX/5AgU533OZLP4sU0kBNryWxJGhBXIXBq/zJcUPA+OcPka0N8Eu pPq5zpHhy6WGDTQTyZcmGqQRklBIVld+7YYDRyyjeNwYFDLiYYk8gRDfsCPu92TxDtSp 6tfcFl93mHI3d4OF10Zl7lrPRHn9dcGzKNlvKZHBNuBtKqmLr/udgrVe23bbVwlXQBJM ny2UnuSr8fy5QZtIWIFYXT0Ee3Sq6Mbqg2AQbMNh4WYxgO0lg8JOUqi263y3wAodZG4u gSyg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:to:subject; bh=bJn59Oo3epk68O65gVVsG7dcJ9/uX0qZk4ZaVIhJE8M=; b=lRFQEWZ1tYmUvRr7WnnSu/TX+6bK+FfSNNQdfRxML8og8J+yFhOwO2KJ/AUngX93fa fgszRXET+GwF+N9OaHoMk51jm1NzRJjxXI84NmDRRVMWPQZS4ctaBf0KMt1pXo2Crvml AAkopNSDMa/oybgvTHtq8wGApw24YSvIbkPrIczd+kGTYnAt+B3NsrM/97mQAN6uW+oQ nL3cMf24Qt/rOg2GRTKnZJTgdAIMXyObzXRt9P1iraPvEUzQxPnSBW3l+HI8kzYhV2ij TZGl75D0v3c7mqks6/FvyBhEBvjngnzXRwSOUIaL6StLEeHc4UkUj9dCsWiChljDdUtu 2dbg== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.28 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from goliath.siemens.de (goliath.siemens.de. [192.35.17.28]) by gmr-mx.google.com with ESMTPS id u23si499722wmn.0.2020.04.22.04.06.56 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 22 Apr 2020 04:06:56 -0700 (PDT) Received-SPF: pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.28 as permitted sender) client-ip=192.35.17.28; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.28 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: from mail2.sbs.de (mail2.sbs.de [192.129.41.66]) by goliath.siemens.de (8.15.2/8.15.2) with ESMTPS id 03MB6tAm014743 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Wed, 22 Apr 2020 13:06:56 +0200 Received: from [167.87.33.221] ([167.87.33.221]) by mail2.sbs.de (8.15.2/8.15.2) with ESMTP id 03MB6sHn002693; Wed, 22 Apr 2020 13:06:55 +0200 Subject: Re: [PATCH] meta/recipes-support: Mount overlay-fs for /etc To: Gylstorff Quirin , isar-users@googlegroups.com, Henning.Schild@siemens.com References: <20200422082403.7392-1-Quirin.Gylstorff@siemens.com> <143382b8-69d0-2449-2c56-eb28c23a80df@siemens.com> <53549f63-ead6-411c-6259-2f3f136c1f67@siemens.com> From: Jan Kiszka Message-ID: <2afaf79b-e999-af10-f051-cb77b115f644@siemens.com> Date: Wed, 22 Apr 2020 13:06:54 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.7.0 MIME-Version: 1.0 In-Reply-To: <53549f63-ead6-411c-6259-2f3f136c1f67@siemens.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-TUID: Rd2j51kxZtu0 On 22.04.20 13:00, Gylstorff Quirin wrote: > > > On 4/22/20 10:30 AM, [ext] Jan Kiszka wrote: >> On 22.04.20 10:24, Q. Gylstorff wrote: >>> From: Quirin Gylstorff >>> >>> Add an overlay fs[1] mount for /etc. This overlay allows an image >>> update of >>> root file-system without overwriting the user settings in /etc. The >>> overlay >>> mounts during the systemd target `local-fs-pre.target.` Due to this >>> target >>> the by systemd generated fsck service is not a dependency for the >>> overlay. >>> >>> [1]: https://www.kernel.org/doc/Documentation/filesystems/overlayfs.txt >>> >>> Signed-off-by: Quirin Gylstorff >>> --- >>>   doc/user_manual.md                            | 26 ++++++ >>>   meta-isar/conf/machine/qemuamd64.conf         |  4 +- >>>   .../etc-overlay/etc-overlay_0.1.bb            | 39 +++++++++ >>>   .../etc-overlay/files/etc-hostname.service    | 14 ++++ >>>   .../etc-overlay/files/etc.mount               | 15 ++++ >>>   .../files/overlay-parse-etc.service           | 14 ++++ >>>   .../etc-overlay/files/ovl.mount.tmpl          | 14 ++++ >>>   .../etc-overlay/files/postinst                | 32 +++++++ >>>   meta/recipes-support/etc-overlay/files/postrm | 29 +++++++ >>>   scripts/lib/wic/canned-wks/etc-overlay.inc    |  5 ++ >>>   scripts/lib/wic/plugins/source/etc-overlay.py | 84 +++++++++++++++++++ >>>   11 files changed, 275 insertions(+), 1 deletion(-) >>>   create mode 100644 meta/recipes-support/etc-overlay/etc-overlay_0.1.bb >>>   create mode 100644 >>> meta/recipes-support/etc-overlay/files/etc-hostname.service >>>   create mode 100644 meta/recipes-support/etc-overlay/files/etc.mount >>>   create mode 100644 >>> meta/recipes-support/etc-overlay/files/overlay-parse-etc.service >>>   create mode 100644 >>> meta/recipes-support/etc-overlay/files/ovl.mount.tmpl >>>   create mode 100755 meta/recipes-support/etc-overlay/files/postinst >>>   create mode 100644 meta/recipes-support/etc-overlay/files/postrm >>>   create mode 100644 scripts/lib/wic/canned-wks/etc-overlay.inc >>>   create mode 100644 scripts/lib/wic/plugins/source/etc-overlay.py >>> >>> diff --git a/doc/user_manual.md b/doc/user_manual.md >>> index d13a74e..5ec7003 100644 >>> --- a/doc/user_manual.md >>> +++ b/doc/user_manual.md >>> @@ -901,3 +901,29 @@ And build the corresponding image target: >>>   ``` >>>   bitbake mc:qemuarm64-buster:isar-image-base >>>   ``` >>> +## Mount overlay for /etc >>> + >>> +### Motivation >>> + >>> +If a software update solution updates the complete image, the >>> content of `/etc` >>> +is also set to new image content. In this case, the user needs to >>> reconfigure >>> +her system. >>> + >>> +### Approach/Solution >>> + >>> +A possible solution is to create an additional partition, which >>> superimposes >>> +`/etc` with an [overlay >>> file-system](https://www.kernel.org/doc/Documentation/filesystems/overlayfs.txt). >>> >>> + >>> +### Example >>> + >>> +Add the following lines to local.conf: >>> +``` >>> +IMAGE_TYPE = "wic-img" >>> +WKS_FILE = "sdimage-efi-overlay" >>> +IMAGE_INSTALL += "etc-overlay" >>> +``` >>> +And build the corresponding image target: >>> + >>> +``` >>> +bitbake mc:qemuamd64-buster:isar-image-base >>> +``` >>> diff --git a/meta-isar/conf/machine/qemuamd64.conf >>> b/meta-isar/conf/machine/qemuamd64.conf >>> index 028b738..d429d96 100644 >>> --- a/meta-isar/conf/machine/qemuamd64.conf >>> +++ b/meta-isar/conf/machine/qemuamd64.conf >>> @@ -6,11 +6,13 @@ DISTRO_ARCH ?= "amd64" >>>   KERNEL_NAME ?= "amd64" >>>   IMAGE_TYPE ?= "wic-img" >>> -WKS_FILE ?= "sdimage-efi" >>> +WKS_FILE ?= "sdimage-efi-overlay" >>>   IMAGER_INSTALL += "${GRUB_BOOTLOADER_INSTALL}" >>>   IMAGE_INSTALL += "sshd-regen-keys" >>> +IMAGE_INSTALL += "etc-overlay" >>> + >>>   QEMU_ARCH ?= "x86_64" >>>   QEMU_MACHINE ?= "q35" >>>   QEMU_CPU ?= "" >>> diff --git a/meta/recipes-support/etc-overlay/etc-overlay_0.1.bb >>> b/meta/recipes-support/etc-overlay/etc-overlay_0.1.bb >>> new file mode 100644 >>> index 0000000..cbed71d >>> --- /dev/null >>> +++ b/meta/recipes-support/etc-overlay/etc-overlay_0.1.bb >>> @@ -0,0 +1,39 @@ >>> +# Create a overlay for /etc to freeze a default configuration >>> +# >>> +# This software is a part of ISAR. >>> +# Copyright (c) Siemens AG, 2020 >>> +# >>> +# SPDX-License-Identifier: MIT >>> + >>> + >>> +DESCRIPTION = "overlay systemd-mount" >>> + >>> +DEBIAN_DEPENDS = "systemd" >>> + >>> +SRC_URI = "file://postinst \ >>> +           file://postrm \ >>> +           file://etc.mount \ >>> +           file://ovl.mount.tmpl \ >>> +           file://overlay-parse-etc.service \ >>> +           file://etc-hostname.service" >>> + >>> +FS_COMMIT_INTERVAL ?= "20" >>> + >>> +TEMPLATE_VARS  += "FS_COMMIT_INTERVAL" >>> +TEMPLATE_FILES += "ovl.mount.tmpl" >>> + >>> +inherit dpkg-raw >>> + >>> +do_install() { >>> +    install -m 0755 -d ${D}/ovl >>> +    touch ${D}/ovl/.keep >>> + >>> +    TARGET=${D}/lib/systemd/system >>> +    install -m 0755 -d ${TARGET} >>> +    install -m 0644 ${WORKDIR}/etc.mount ${TARGET}/etc.mount >>> +    install -m 0644 ${WORKDIR}/ovl.mount ${TARGET}/ovl.mount >>> +    install -m 0644 ${WORKDIR}/overlay-parse-etc.service >>> ${TARGET}/overlay-parse-etc.service >>> +    install -m 0644 ${WORKDIR}/etc-hostname.service >>> ${TARGET}/etc-hostname.service >>> +} >>> + >>> +addtask do_install after do_transform_template >>> diff --git >>> a/meta/recipes-support/etc-overlay/files/etc-hostname.service >>> b/meta/recipes-support/etc-overlay/files/etc-hostname.service >>> new file mode 100644 >>> index 0000000..2306b9f >>> --- /dev/null >>> +++ b/meta/recipes-support/etc-overlay/files/etc-hostname.service >>> @@ -0,0 +1,14 @@ >>> +[Unit] >>> +Description=set hostname /etc overlay-aware >>> +Before=network-pre.target >>> +Wants=network-pre.target >>> +Requires=etc.mount >>> +After=etc.mount >>> + >>> +[Service] >>> +Type=oneshot >>> +RemainAfterExit=yes >>> +ExecStart=/bin/hostname --boot --file /etc/hostname >>> + >>> +[Install] >>> +WantedBy=basic.target >>> diff --git a/meta/recipes-support/etc-overlay/files/etc.mount >>> b/meta/recipes-support/etc-overlay/files/etc.mount >>> new file mode 100644 >>> index 0000000..59ee0d7 >>> --- /dev/null >>> +++ b/meta/recipes-support/etc-overlay/files/etc.mount >>> @@ -0,0 +1,15 @@ >>> +[Unit] >>> +Description=Overlay-mount /etc >>> +DefaultDependencies=no >>> +Before=local-fs-pre.target >>> +Requires=ovl.mount >>> +After=ovl.mount >>> + >>> +[Mount] >>> +What=overlay >>> +Where=/etc >>> +Type=overlay >>> +Options=noauto,x-systemd.automount,lowerdir=/etc,upperdir=/ovl/etc,workdir=/ovl/.atomic >>> >>> + >>> +[Install] >>> +WantedBy=local-fs-pre.target >>> diff --git >>> a/meta/recipes-support/etc-overlay/files/overlay-parse-etc.service >>> b/meta/recipes-support/etc-overlay/files/overlay-parse-etc.service >>> new file mode 100644 >>> index 0000000..eb5d422 >>> --- /dev/null >>> +++ b/meta/recipes-support/etc-overlay/files/overlay-parse-etc.service >>> @@ -0,0 +1,14 @@ >>> +[Unit] >>> +Description=Reload Configuration from the etc overlay >>> +DefaultDependencies=no >>> +Requires=etc.mount >>> +After=etc.mount >>> +Before=local-fs-pre.target >>> + >>> +[Service] >>> +Type=oneshot >>> +RemainAfterExit=yes >>> +ExecStartPre=!/bin/systemctl daemon-reload >>> +ExecStart=!/bin/systemctl --no-block isolate multi-user.target >>> +[Install] >>> +WantedBy=local-fs-pre.target >>> diff --git a/meta/recipes-support/etc-overlay/files/ovl.mount.tmpl >>> b/meta/recipes-support/etc-overlay/files/ovl.mount.tmpl >>> new file mode 100644 >>> index 0000000..3593c55 >>> --- /dev/null >>> +++ b/meta/recipes-support/etc-overlay/files/ovl.mount.tmpl >>> @@ -0,0 +1,14 @@ >>> +[Unit] >>> +Description=Mount /etc overlay backing store >>> +DefaultDependencies=no >>> +Before=local-fs-pre.target >>> +Before=etc.mount >>> + >>> +[Mount] >>> +What=/dev/disk/by-partlabel/etcovl >>> +Where=/ovl >>> +Type=ext4 >>> +Options=noatime,nodiratime,data=journal,commit=${FS_COMMIT_INTERVAL},nodelalloc >>> >>> + >>> +[Install] >>> +WantedBy=local-fs-pre.target >>> diff --git a/meta/recipes-support/etc-overlay/files/postinst >>> b/meta/recipes-support/etc-overlay/files/postinst >>> new file mode 100755 >>> index 0000000..7a10d68 >>> --- /dev/null >>> +++ b/meta/recipes-support/etc-overlay/files/postinst >>> @@ -0,0 +1,32 @@ >>> +#!/bin/sh >>> +# postinst script for etc-overlay >>> +# >>> +# see: dh_installdeb(1) >>> + >>> +set -e >>> + >>> +case "$1" in >>> +    configure) >>> +        deb-systemd-helper enable etc.mount  || true >>> +        deb-systemd-helper enable ovl.mount  || true >>> +        deb-systemd-helper enable overlay-parse-etc.service || true >>> +        deb-systemd-helper enable systemd-remount-fs.service || true >>> +        deb-systemd-helper enable etc-hostname.service || true >>> +    ;; >>> + >>> +    abort-upgrade|abort-remove|abort-deconfigure) >>> +    ;; >>> + >>> +    *) >>> +        echo "postinst called with unknown argument \`$1'" >&2 >>> +        exit 1 >>> +        ;; >>> +esac >>> + >>> +# dh_installdeb will replace this with shell code automatically >>> +# generated by other debhelper scripts. >>> + >>> +#DEBHELPER# >>> + >>> +exit 0 >>> + >>> diff --git a/meta/recipes-support/etc-overlay/files/postrm >>> b/meta/recipes-support/etc-overlay/files/postrm >>> new file mode 100644 >>> index 0000000..7a3defb >>> --- /dev/null >>> +++ b/meta/recipes-support/etc-overlay/files/postrm >>> @@ -0,0 +1,29 @@ >>> +#!/bin/sh >>> +# postrm script for etc-overlay >>> +# >>> +# see: dh_installdeb(1) >>> + >>> +set -e >>> + >>> + >>> +case "$1" in >>> + >>> purge|remove|upgrade|failed-upgrade|abort-install|abort-upgrade|disappear) >>> >>> +        deb-systemd-helper disable etc.mount  || true >>> +        deb-systemd-helper disable ovl.mount  || true >>> +        deb-systemd-helper disable overlay-parse-etc.service || true >>> +        deb-systemd-helper disable systemd-remount-fs.service || true >>> +        deb-systemd-helper disable etc-hostname.service || true >>> +     ;; >>> + >>> +    *) >>> +        echo "postrm called with unknown argument \`$1'" >&2 >>> +        exit 1 >>> +    ;; >>> +esac >>> + >>> +# dh_installdeb will replace this with shell code automatically >>> +# generated by other debhelper scripts. >>> + >>> +#DEBHELPER# >>> + >>> +exit 0 >>> diff --git a/scripts/lib/wic/canned-wks/etc-overlay.inc >>> b/scripts/lib/wic/canned-wks/etc-overlay.inc >>> new file mode 100644 >>> index 0000000..7b40854 >>> --- /dev/null >>> +++ b/scripts/lib/wic/canned-wks/etc-overlay.inc >>> @@ -0,0 +1,5 @@ >>> +# add a overlay partition to the image >>> + >>> +# overlay partition >>> +part --source etc-overlay --ondisk sda --size 100M --extra-space >>> 128M --overhead-factor 1 --label etcovl --align 1024 --fstype=ext4 >>> + >> >> Where is this used/tested? I think you should create a test/demo >> target that uses a read-only rootfs and such an overlay partition via >> the include. >> >> Jan > > In the patch series it is added to the qemuamd64 target as test. It > replaces the sdimage-efi with sdimage-efi-overlay. But a read only > rootfs/target is better. > You forgot to include sdimage-efi-overlay.wks - that's why I got confused. Or you didn't send all patches? Jan -- Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux