From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6818448905053667328 X-Received: by 2002:a2e:9b0f:: with SMTP id u15mr1123163lji.272.1587544255198; Wed, 22 Apr 2020 01:30:55 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a19:c310:: with SMTP id t16ls213427lff.2.gmail; Wed, 22 Apr 2020 01:30:53 -0700 (PDT) X-Google-Smtp-Source: APiQypJJdy0iSKmpMK5bPXNnwW6r5+K5nNo1ZEMJag+uSEg0lEmKp0xzWvRY9Em2LAOmVFmzlJ84 X-Received: by 2002:ac2:4143:: with SMTP id c3mr16304103lfi.131.1587544253861; Wed, 22 Apr 2020 01:30:53 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1587544253; cv=none; d=google.com; s=arc-20160816; b=yLKUc+1gIT+lI+T5bEgqLSBaXZYr6e1B4IljsLfqEoIc5L8X/sHAcJsWNC8eK3nK0B 8TOCzn6kwiM4VuXPmHi3DDsgMEX6ZZ/JOFJtzW1zvs8/GA9wTPDTVYziRek2a4LWgzGm vY5DU0m0hXC+wn0EsA21/PyCAWwe+yzZuMNwAVVb+f6dj5K/QKBaFC+ZcCyK+fdCSDxs WSabe20n3USXPLR1rEClSSqvaV9jh/Kps4Z13xRtqLSSPWPydqCusuhMJ/Vb7t3qflHh OPZKnjFWL2KYj3uhCo7zVB9Xn3V6M6HbklWxAJh5fBMyNyobOVA01+a2N9EWPCcnsrWE 4OnA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:to:subject; bh=PWEafi7//NOsRLKn04v2BSuwQZ0wpamo8tvb93JR6aI=; b=qXL3hPhxcpBsiyrTXh7cantiQp3H4/1pBrLOxf19CzbBWQz9y3S71Gx4iL0gSGmalB m3tmjjRNbdMp8hIWoIzyOGHJ1IsXAzbOecsPWq8x0WRcHLAMy3drevAkkrzrlKNnzcbs Zw/KOx3BxR5VDiN8PU910cBaU+TpjJv0nhpaM7MpxXv4fCaeSLFxnXe3R0JmUD6utKXa 1P0qJv30B28EcNRon2SufEHS1+LRGPfH3fKJZ/iu+aucdw85QIamMg1mvSRyQr9HYCvW m3GXLLthExe/a939/WEwuXuE9PPmSdfJvp4TNgRuRWa+2hrtNGlSsDt1QcLTFJMqkSXy qZ0w== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.28 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from goliath.siemens.de (goliath.siemens.de. [192.35.17.28]) by gmr-mx.google.com with ESMTPS id k13si349358lfg.5.2020.04.22.01.30.53 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 22 Apr 2020 01:30:53 -0700 (PDT) Received-SPF: pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.28 as permitted sender) client-ip=192.35.17.28; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.28 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: from mail2.sbs.de (mail2.sbs.de [192.129.41.66]) by goliath.siemens.de (8.15.2/8.15.2) with ESMTPS id 03M8UqOX001909 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Wed, 22 Apr 2020 10:30:52 +0200 Received: from [167.87.33.221] ([167.87.33.221]) by mail2.sbs.de (8.15.2/8.15.2) with ESMTP id 03M8UqJp018493; Wed, 22 Apr 2020 10:30:52 +0200 Subject: Re: [PATCH] meta/recipes-support: Mount overlay-fs for /etc To: "Q. Gylstorff" , isar-users@googlegroups.com, Henning.Schild@siemens.com References: <20200422082403.7392-1-Quirin.Gylstorff@siemens.com> From: Jan Kiszka Message-ID: <143382b8-69d0-2449-2c56-eb28c23a80df@siemens.com> Date: Wed, 22 Apr 2020 10:30:51 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.7.0 MIME-Version: 1.0 In-Reply-To: <20200422082403.7392-1-Quirin.Gylstorff@siemens.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-TUID: ZUNa07UAZbXu On 22.04.20 10:24, Q. Gylstorff wrote: > From: Quirin Gylstorff > > Add an overlay fs[1] mount for /etc. This overlay allows an image update of > root file-system without overwriting the user settings in /etc. The overlay > mounts during the systemd target `local-fs-pre.target.` Due to this target > the by systemd generated fsck service is not a dependency for the overlay. > > [1]: https://www.kernel.org/doc/Documentation/filesystems/overlayfs.txt > > Signed-off-by: Quirin Gylstorff > --- > doc/user_manual.md | 26 ++++++ > meta-isar/conf/machine/qemuamd64.conf | 4 +- > .../etc-overlay/etc-overlay_0.1.bb | 39 +++++++++ > .../etc-overlay/files/etc-hostname.service | 14 ++++ > .../etc-overlay/files/etc.mount | 15 ++++ > .../files/overlay-parse-etc.service | 14 ++++ > .../etc-overlay/files/ovl.mount.tmpl | 14 ++++ > .../etc-overlay/files/postinst | 32 +++++++ > meta/recipes-support/etc-overlay/files/postrm | 29 +++++++ > scripts/lib/wic/canned-wks/etc-overlay.inc | 5 ++ > scripts/lib/wic/plugins/source/etc-overlay.py | 84 +++++++++++++++++++ > 11 files changed, 275 insertions(+), 1 deletion(-) > create mode 100644 meta/recipes-support/etc-overlay/etc-overlay_0.1.bb > create mode 100644 meta/recipes-support/etc-overlay/files/etc-hostname.service > create mode 100644 meta/recipes-support/etc-overlay/files/etc.mount > create mode 100644 meta/recipes-support/etc-overlay/files/overlay-parse-etc.service > create mode 100644 meta/recipes-support/etc-overlay/files/ovl.mount.tmpl > create mode 100755 meta/recipes-support/etc-overlay/files/postinst > create mode 100644 meta/recipes-support/etc-overlay/files/postrm > create mode 100644 scripts/lib/wic/canned-wks/etc-overlay.inc > create mode 100644 scripts/lib/wic/plugins/source/etc-overlay.py > > diff --git a/doc/user_manual.md b/doc/user_manual.md > index d13a74e..5ec7003 100644 > --- a/doc/user_manual.md > +++ b/doc/user_manual.md > @@ -901,3 +901,29 @@ And build the corresponding image target: > ``` > bitbake mc:qemuarm64-buster:isar-image-base > ``` > +## Mount overlay for /etc > + > +### Motivation > + > +If a software update solution updates the complete image, the content of `/etc` > +is also set to new image content. In this case, the user needs to reconfigure > +her system. > + > +### Approach/Solution > + > +A possible solution is to create an additional partition, which superimposes > +`/etc` with an [overlay file-system](https://www.kernel.org/doc/Documentation/filesystems/overlayfs.txt). > + > +### Example > + > +Add the following lines to local.conf: > +``` > +IMAGE_TYPE = "wic-img" > +WKS_FILE = "sdimage-efi-overlay" > +IMAGE_INSTALL += "etc-overlay" > +``` > +And build the corresponding image target: > + > +``` > +bitbake mc:qemuamd64-buster:isar-image-base > +``` > diff --git a/meta-isar/conf/machine/qemuamd64.conf b/meta-isar/conf/machine/qemuamd64.conf > index 028b738..d429d96 100644 > --- a/meta-isar/conf/machine/qemuamd64.conf > +++ b/meta-isar/conf/machine/qemuamd64.conf > @@ -6,11 +6,13 @@ DISTRO_ARCH ?= "amd64" > KERNEL_NAME ?= "amd64" > > IMAGE_TYPE ?= "wic-img" > -WKS_FILE ?= "sdimage-efi" > +WKS_FILE ?= "sdimage-efi-overlay" > IMAGER_INSTALL += "${GRUB_BOOTLOADER_INSTALL}" > > IMAGE_INSTALL += "sshd-regen-keys" > > +IMAGE_INSTALL += "etc-overlay" > + > QEMU_ARCH ?= "x86_64" > QEMU_MACHINE ?= "q35" > QEMU_CPU ?= "" > diff --git a/meta/recipes-support/etc-overlay/etc-overlay_0.1.bb b/meta/recipes-support/etc-overlay/etc-overlay_0.1.bb > new file mode 100644 > index 0000000..cbed71d > --- /dev/null > +++ b/meta/recipes-support/etc-overlay/etc-overlay_0.1.bb > @@ -0,0 +1,39 @@ > +# Create a overlay for /etc to freeze a default configuration > +# > +# This software is a part of ISAR. > +# Copyright (c) Siemens AG, 2020 > +# > +# SPDX-License-Identifier: MIT > + > + > +DESCRIPTION = "overlay systemd-mount" > + > +DEBIAN_DEPENDS = "systemd" > + > +SRC_URI = "file://postinst \ > + file://postrm \ > + file://etc.mount \ > + file://ovl.mount.tmpl \ > + file://overlay-parse-etc.service \ > + file://etc-hostname.service" > + > +FS_COMMIT_INTERVAL ?= "20" > + > +TEMPLATE_VARS += "FS_COMMIT_INTERVAL" > +TEMPLATE_FILES += "ovl.mount.tmpl" > + > +inherit dpkg-raw > + > +do_install() { > + install -m 0755 -d ${D}/ovl > + touch ${D}/ovl/.keep > + > + TARGET=${D}/lib/systemd/system > + install -m 0755 -d ${TARGET} > + install -m 0644 ${WORKDIR}/etc.mount ${TARGET}/etc.mount > + install -m 0644 ${WORKDIR}/ovl.mount ${TARGET}/ovl.mount > + install -m 0644 ${WORKDIR}/overlay-parse-etc.service ${TARGET}/overlay-parse-etc.service > + install -m 0644 ${WORKDIR}/etc-hostname.service ${TARGET}/etc-hostname.service > +} > + > +addtask do_install after do_transform_template > diff --git a/meta/recipes-support/etc-overlay/files/etc-hostname.service b/meta/recipes-support/etc-overlay/files/etc-hostname.service > new file mode 100644 > index 0000000..2306b9f > --- /dev/null > +++ b/meta/recipes-support/etc-overlay/files/etc-hostname.service > @@ -0,0 +1,14 @@ > +[Unit] > +Description=set hostname /etc overlay-aware > +Before=network-pre.target > +Wants=network-pre.target > +Requires=etc.mount > +After=etc.mount > + > +[Service] > +Type=oneshot > +RemainAfterExit=yes > +ExecStart=/bin/hostname --boot --file /etc/hostname > + > +[Install] > +WantedBy=basic.target > diff --git a/meta/recipes-support/etc-overlay/files/etc.mount b/meta/recipes-support/etc-overlay/files/etc.mount > new file mode 100644 > index 0000000..59ee0d7 > --- /dev/null > +++ b/meta/recipes-support/etc-overlay/files/etc.mount > @@ -0,0 +1,15 @@ > +[Unit] > +Description=Overlay-mount /etc > +DefaultDependencies=no > +Before=local-fs-pre.target > +Requires=ovl.mount > +After=ovl.mount > + > +[Mount] > +What=overlay > +Where=/etc > +Type=overlay > +Options=noauto,x-systemd.automount,lowerdir=/etc,upperdir=/ovl/etc,workdir=/ovl/.atomic > + > +[Install] > +WantedBy=local-fs-pre.target > diff --git a/meta/recipes-support/etc-overlay/files/overlay-parse-etc.service b/meta/recipes-support/etc-overlay/files/overlay-parse-etc.service > new file mode 100644 > index 0000000..eb5d422 > --- /dev/null > +++ b/meta/recipes-support/etc-overlay/files/overlay-parse-etc.service > @@ -0,0 +1,14 @@ > +[Unit] > +Description=Reload Configuration from the etc overlay > +DefaultDependencies=no > +Requires=etc.mount > +After=etc.mount > +Before=local-fs-pre.target > + > +[Service] > +Type=oneshot > +RemainAfterExit=yes > +ExecStartPre=!/bin/systemctl daemon-reload > +ExecStart=!/bin/systemctl --no-block isolate multi-user.target > +[Install] > +WantedBy=local-fs-pre.target > diff --git a/meta/recipes-support/etc-overlay/files/ovl.mount.tmpl b/meta/recipes-support/etc-overlay/files/ovl.mount.tmpl > new file mode 100644 > index 0000000..3593c55 > --- /dev/null > +++ b/meta/recipes-support/etc-overlay/files/ovl.mount.tmpl > @@ -0,0 +1,14 @@ > +[Unit] > +Description=Mount /etc overlay backing store > +DefaultDependencies=no > +Before=local-fs-pre.target > +Before=etc.mount > + > +[Mount] > +What=/dev/disk/by-partlabel/etcovl > +Where=/ovl > +Type=ext4 > +Options=noatime,nodiratime,data=journal,commit=${FS_COMMIT_INTERVAL},nodelalloc > + > +[Install] > +WantedBy=local-fs-pre.target > diff --git a/meta/recipes-support/etc-overlay/files/postinst b/meta/recipes-support/etc-overlay/files/postinst > new file mode 100755 > index 0000000..7a10d68 > --- /dev/null > +++ b/meta/recipes-support/etc-overlay/files/postinst > @@ -0,0 +1,32 @@ > +#!/bin/sh > +# postinst script for etc-overlay > +# > +# see: dh_installdeb(1) > + > +set -e > + > +case "$1" in > + configure) > + deb-systemd-helper enable etc.mount || true > + deb-systemd-helper enable ovl.mount || true > + deb-systemd-helper enable overlay-parse-etc.service || true > + deb-systemd-helper enable systemd-remount-fs.service || true > + deb-systemd-helper enable etc-hostname.service || true > + ;; > + > + abort-upgrade|abort-remove|abort-deconfigure) > + ;; > + > + *) > + echo "postinst called with unknown argument \`$1'" >&2 > + exit 1 > + ;; > +esac > + > +# dh_installdeb will replace this with shell code automatically > +# generated by other debhelper scripts. > + > +#DEBHELPER# > + > +exit 0 > + > diff --git a/meta/recipes-support/etc-overlay/files/postrm b/meta/recipes-support/etc-overlay/files/postrm > new file mode 100644 > index 0000000..7a3defb > --- /dev/null > +++ b/meta/recipes-support/etc-overlay/files/postrm > @@ -0,0 +1,29 @@ > +#!/bin/sh > +# postrm script for etc-overlay > +# > +# see: dh_installdeb(1) > + > +set -e > + > + > +case "$1" in > + purge|remove|upgrade|failed-upgrade|abort-install|abort-upgrade|disappear) > + deb-systemd-helper disable etc.mount || true > + deb-systemd-helper disable ovl.mount || true > + deb-systemd-helper disable overlay-parse-etc.service || true > + deb-systemd-helper disable systemd-remount-fs.service || true > + deb-systemd-helper disable etc-hostname.service || true > + ;; > + > + *) > + echo "postrm called with unknown argument \`$1'" >&2 > + exit 1 > + ;; > +esac > + > +# dh_installdeb will replace this with shell code automatically > +# generated by other debhelper scripts. > + > +#DEBHELPER# > + > +exit 0 > diff --git a/scripts/lib/wic/canned-wks/etc-overlay.inc b/scripts/lib/wic/canned-wks/etc-overlay.inc > new file mode 100644 > index 0000000..7b40854 > --- /dev/null > +++ b/scripts/lib/wic/canned-wks/etc-overlay.inc > @@ -0,0 +1,5 @@ > +# add a overlay partition to the image > + > +# overlay partition > +part --source etc-overlay --ondisk sda --size 100M --extra-space 128M --overhead-factor 1 --label etcovl --align 1024 --fstype=ext4 > + Where is this used/tested? I think you should create a test/demo target that uses a read-only rootfs and such an overlay partition via the include. Jan > diff --git a/scripts/lib/wic/plugins/source/etc-overlay.py b/scripts/lib/wic/plugins/source/etc-overlay.py > new file mode 100644 > index 0000000..55189a2 > --- /dev/null > +++ b/scripts/lib/wic/plugins/source/etc-overlay.py > @@ -0,0 +1,84 @@ > +# ex:ts=4:sw=4:sts=4:et > +# -*- tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*- > +# > +# Copyright (c) 2014, Intel Corporation. > +# Copyright (c) 2018, Siemens AG. > +# All rights reserved. > +# > +# This program is free software; you can redistribute it and/or modify > +# it under the terms of the GNU General Public License version 2 as > +# published by the Free Software Foundation. > +# > +# This program is distributed in the hope that it will be useful, > +# but WITHOUT ANY WARRANTY; without even the implied warranty of > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > +# GNU General Public License for more details. > +# > +# You should have received a copy of the GNU General Public License along > +# with this program; if not, write to the Free Software Foundation, Inc., > +# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. > +# > +# DESCRIPTION > +# This implements the 'etc-overlay' source plugin class for 'wic' > +# > +# AUTHORS > +# Tom Zanussi > +# Andreas Reichel > +# Quirin Gylstorff > + > + > +import logging > + > +msger = logging.getLogger('wic') > + > +from wic.pluginbase import SourcePlugin > +from wic.utils.misc import exec_cmd,BOOTDD_EXTRA_SPACE > + > +class EtcOverlayPlugin(SourcePlugin): > + """ > + Create an overlay file system scheme for etc > + """ > + > + name = 'etc-overlay' > + > + @classmethod > + def do_prepare_partition(cls, part, source_params, creator, cr_workdir, > + oe_builddir, deploy_dir, kernel_dir, > + rootfs_dir, native_sysroot): > + > + part_rootfs_dir = "%s/disk/%s.%s" % (cr_workdir, > + part.label, > + part.lineno) > + create_dir_cmd = "install -d %s" % part_rootfs_dir > + exec_cmd(create_dir_cmd) > + > + exec_cmd("install -m 0755 -d %s/etc" % part_rootfs_dir) > + exec_cmd("install -m 0755 -d %s/.atomic" % part_rootfs_dir) > + > + blocks = 16 > + extra_blocks = part.get_extra_block_count(blocks) > + if extra_blocks < BOOTDD_EXTRA_SPACE: > + extra_blocks = BOOTDD_EXTRA_SPACE > + blocks += extra_blocks > + blocks = blocks + (16 - (blocks % 16)) > + > + msger.debug("Added %d extra blocks to %s to get to %d total blocks", > + extra_blocks, part.mountpoint, blocks) > + > + # ext4 image, created by mkfs.ext4 > + etcovlimg = "%s/%s.%s.img" % (cr_workdir, part.label, part.lineno) > + partfs_cmd = "dd if=/dev/zero of=%s bs=512 count=%d" % (etcovlimg, > + blocks) > + exec_cmd(partfs_cmd) > + > + partfs_cmd = "mkfs.ext4 %s -d %s" % (etcovlimg, part_rootfs_dir) > + exec_cmd(partfs_cmd) > + > + chmod_cmd = "chmod 644 %s" % etcovlimg > + exec_cmd(chmod_cmd) > + > + du_cmd = "du -Lbks %s" % etcovlimg > + etcovlimg_size = int(exec_cmd(du_cmd).split()[0]) > + > + part.size = etcovlimg_size > + part.source_file = etcovlimg > -- Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux