From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6818448905053667328 X-Received: by 2002:a1c:2d14:: with SMTP id t20mr3199354wmt.28.1588254602728; Thu, 30 Apr 2020 06:50:02 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:600c:224f:: with SMTP id a15ls1954474wmm.3.gmail; Thu, 30 Apr 2020 06:50:02 -0700 (PDT) X-Google-Smtp-Source: APiQypLp5isHG7oBY6Ljbv6LW2hCWerp7P8/Ctw/ypJHTbjjz71Q4VCFa8rwYPwlfxMFFexeITD3 X-Received: by 2002:a1c:3182:: with SMTP id x124mr3330320wmx.54.1588254601951; Thu, 30 Apr 2020 06:50:01 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1588254601; cv=none; d=google.com; s=arc-20160816; b=fNh458h7aotTCAGm2IDVp2pXpHb/cMYK8+sRG/CRxkADwzRNQmL4ziJn6jYZtSb7H1 Shm1KKvpPwESGG+0BSr5m9jk1o4MjddctfB2TR7zHdbQixpAmtW/Z0q0er22cYFpR5FG PFAtn6sXhU0mv5Y33QZI9Q9sHqZ88B3i/XKlTMLB23hkDw5Zz4y/QkCx1QtGisfRBcpZ drW29zY4d82/WERMCK8buOp2pXwUPtj87B2IL0TwwETBGDVf2D530EvGoCybYD/zL38l QE2L53d3hVMnOsDYriW6dsLJlMP0muUayFVXcslyu382Z13aJLlV1LX/Z5iCWJa+ixhd lrsA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=iyBgfjeuZv8U5NvRNAmItPXZJS8RwGSQsA9nLmc1+YE=; b=Ha+fi2yXX4LSD2aXBqfnDDRS8V4ThTd2cOVj/M7WzZ4cbowx69g63yWCplOKGOgpk+ 1Um/N0tTQi0cUtSOgtzsX62XZiMGU/rFd5GMhc6bqF9HlYesEastisK9jg3SGY/o+w0S Eu7q59Z9RbSCNaomxFY2Ebo7ffchNMsU2fJZlpudGz0cZrdEgnpQjOFvZYyhVDs6hp75 fwCAw2VRMHGOxHAgZv2B+rv63ldgVmVl4PSbD8s6g58ZhG7kZN9//Bz/nFCJs1xyH5to fNEm3FgN4qfBIKNYa3T5vU3ZvxcHAGse7H2aLBhibAQ6T3Ka65oA3e6NB6sMzgXDn4lD YTDg== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of quirin.gylstorff@siemens.com designates 192.35.17.2 as permitted sender) smtp.mailfrom=Quirin.Gylstorff@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from thoth.sbs.de (thoth.sbs.de. [192.35.17.2]) by gmr-mx.google.com with ESMTPS id o136si101361wme.0.2020.04.30.06.50.01 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 30 Apr 2020 06:50:01 -0700 (PDT) Received-SPF: pass (google.com: domain of quirin.gylstorff@siemens.com designates 192.35.17.2 as permitted sender) client-ip=192.35.17.2; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of quirin.gylstorff@siemens.com designates 192.35.17.2 as permitted sender) smtp.mailfrom=Quirin.Gylstorff@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: from mail1.sbs.de (mail1.sbs.de [192.129.41.35]) by thoth.sbs.de (8.15.2/8.15.2) with ESMTPS id 03UDo1SH006281 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Thu, 30 Apr 2020 15:50:01 +0200 Received: from md2dvrtc.ad001.siemens.net ([139.22.46.171]) by mail1.sbs.de (8.15.2/8.15.2) with ESMTP id 03UDo1M2018012; Thu, 30 Apr 2020 15:50:01 +0200 From: "Q. Gylstorff" To: isar-users@googlegroups.com Cc: Quirin Gylstorff Subject: [PATCH v2] meta/recipes-support: Mount overlay-fs for /etc Date: Thu, 30 Apr 2020 15:50:01 +0200 Message-Id: <20200430135001.13357-1-Quirin.Gylstorff@siemens.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200422082403.7392-1-Quirin.Gylstorff@siemens.com> References: <20200422082403.7392-1-Quirin.Gylstorff@siemens.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-TUID: MHkLT209N3W2 From: Quirin Gylstorff Add an overlay fs[1] mount for /etc. This overlay allows an image update of root file-system without overwriting the user settings in /etc. The overlay mounts during the systemd target `local-fs-pre.target.` Due to this target the by systemd generated fsck service is not a dependency for the overlay. [1]: https://www.kernel.org/doc/Documentation/filesystems/overlayfs.txt Signed-off-by: Quirin Gylstorff --- Changes V2: - add missing file sdimage-efi-overlay.wks - fix whitespaces doc/user_manual.md | 26 ++++++ meta-isar/conf/machine/qemuamd64.conf | 4 +- .../wic/canned-wks/sdimage-efi-overlay.wks | 4 + .../etc-overlay/etc-overlay_0.1.bb | 39 +++++++++ .../etc-overlay/files/etc-hostname.service | 14 ++++ .../etc-overlay/files/etc.mount | 15 ++++ .../files/overlay-parse-etc.service | 14 ++++ .../etc-overlay/files/ovl.mount.tmpl | 14 ++++ .../etc-overlay/files/postinst | 31 +++++++ meta/recipes-support/etc-overlay/files/postrm | 28 +++++++ scripts/lib/wic/canned-wks/etc-overlay.inc | 4 + scripts/lib/wic/plugins/source/etc-overlay.py | 84 +++++++++++++++++++ 12 files changed, 276 insertions(+), 1 deletion(-) create mode 100644 meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-overlay.wks create mode 100644 meta/recipes-support/etc-overlay/etc-overlay_0.1.bb create mode 100644 meta/recipes-support/etc-overlay/files/etc-hostname.service create mode 100644 meta/recipes-support/etc-overlay/files/etc.mount create mode 100644 meta/recipes-support/etc-overlay/files/overlay-parse-etc.service create mode 100644 meta/recipes-support/etc-overlay/files/ovl.mount.tmpl create mode 100755 meta/recipes-support/etc-overlay/files/postinst create mode 100644 meta/recipes-support/etc-overlay/files/postrm create mode 100644 scripts/lib/wic/canned-wks/etc-overlay.inc create mode 100644 scripts/lib/wic/plugins/source/etc-overlay.py diff --git a/doc/user_manual.md b/doc/user_manual.md index d13a74e..5ec7003 100644 --- a/doc/user_manual.md +++ b/doc/user_manual.md @@ -901,3 +901,29 @@ And build the corresponding image target: ``` bitbake mc:qemuarm64-buster:isar-image-base ``` +## Mount overlay for /etc + +### Motivation + +If a software update solution updates the complete image, the content of `/etc` +is also set to new image content. In this case, the user needs to reconfigure +her system. + +### Approach/Solution + +A possible solution is to create an additional partition, which superimposes +`/etc` with an [overlay file-system](https://www.kernel.org/doc/Documentation/filesystems/overlayfs.txt). + +### Example + +Add the following lines to local.conf: +``` +IMAGE_TYPE = "wic-img" +WKS_FILE = "sdimage-efi-overlay" +IMAGE_INSTALL += "etc-overlay" +``` +And build the corresponding image target: + +``` +bitbake mc:qemuamd64-buster:isar-image-base +``` diff --git a/meta-isar/conf/machine/qemuamd64.conf b/meta-isar/conf/machine/qemuamd64.conf index 028b738..d429d96 100644 --- a/meta-isar/conf/machine/qemuamd64.conf +++ b/meta-isar/conf/machine/qemuamd64.conf @@ -6,11 +6,13 @@ DISTRO_ARCH ?= "amd64" KERNEL_NAME ?= "amd64" IMAGE_TYPE ?= "wic-img" -WKS_FILE ?= "sdimage-efi" +WKS_FILE ?= "sdimage-efi-overlay" IMAGER_INSTALL += "${GRUB_BOOTLOADER_INSTALL}" IMAGE_INSTALL += "sshd-regen-keys" +IMAGE_INSTALL += "etc-overlay" + QEMU_ARCH ?= "x86_64" QEMU_MACHINE ?= "q35" QEMU_CPU ?= "" diff --git a/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-overlay.wks b/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-overlay.wks new file mode 100644 index 0000000..96f756f --- /dev/null +++ b/meta-isar/scripts/lib/wic/canned-wks/sdimage-efi-overlay.wks @@ -0,0 +1,4 @@ +# sdimage with etc overlay + +include sdimage-efi.wks +include etc-overlay.inc diff --git a/meta/recipes-support/etc-overlay/etc-overlay_0.1.bb b/meta/recipes-support/etc-overlay/etc-overlay_0.1.bb new file mode 100644 index 0000000..cbed71d --- /dev/null +++ b/meta/recipes-support/etc-overlay/etc-overlay_0.1.bb @@ -0,0 +1,39 @@ +# Create a overlay for /etc to freeze a default configuration +# +# This software is a part of ISAR. +# Copyright (c) Siemens AG, 2020 +# +# SPDX-License-Identifier: MIT + + +DESCRIPTION = "overlay systemd-mount" + +DEBIAN_DEPENDS = "systemd" + +SRC_URI = "file://postinst \ + file://postrm \ + file://etc.mount \ + file://ovl.mount.tmpl \ + file://overlay-parse-etc.service \ + file://etc-hostname.service" + +FS_COMMIT_INTERVAL ?= "20" + +TEMPLATE_VARS += "FS_COMMIT_INTERVAL" +TEMPLATE_FILES += "ovl.mount.tmpl" + +inherit dpkg-raw + +do_install() { + install -m 0755 -d ${D}/ovl + touch ${D}/ovl/.keep + + TARGET=${D}/lib/systemd/system + install -m 0755 -d ${TARGET} + install -m 0644 ${WORKDIR}/etc.mount ${TARGET}/etc.mount + install -m 0644 ${WORKDIR}/ovl.mount ${TARGET}/ovl.mount + install -m 0644 ${WORKDIR}/overlay-parse-etc.service ${TARGET}/overlay-parse-etc.service + install -m 0644 ${WORKDIR}/etc-hostname.service ${TARGET}/etc-hostname.service +} + +addtask do_install after do_transform_template diff --git a/meta/recipes-support/etc-overlay/files/etc-hostname.service b/meta/recipes-support/etc-overlay/files/etc-hostname.service new file mode 100644 index 0000000..2306b9f --- /dev/null +++ b/meta/recipes-support/etc-overlay/files/etc-hostname.service @@ -0,0 +1,14 @@ +[Unit] +Description=set hostname /etc overlay-aware +Before=network-pre.target +Wants=network-pre.target +Requires=etc.mount +After=etc.mount + +[Service] +Type=oneshot +RemainAfterExit=yes +ExecStart=/bin/hostname --boot --file /etc/hostname + +[Install] +WantedBy=basic.target diff --git a/meta/recipes-support/etc-overlay/files/etc.mount b/meta/recipes-support/etc-overlay/files/etc.mount new file mode 100644 index 0000000..59ee0d7 --- /dev/null +++ b/meta/recipes-support/etc-overlay/files/etc.mount @@ -0,0 +1,15 @@ +[Unit] +Description=Overlay-mount /etc +DefaultDependencies=no +Before=local-fs-pre.target +Requires=ovl.mount +After=ovl.mount + +[Mount] +What=overlay +Where=/etc +Type=overlay +Options=noauto,x-systemd.automount,lowerdir=/etc,upperdir=/ovl/etc,workdir=/ovl/.atomic + +[Install] +WantedBy=local-fs-pre.target diff --git a/meta/recipes-support/etc-overlay/files/overlay-parse-etc.service b/meta/recipes-support/etc-overlay/files/overlay-parse-etc.service new file mode 100644 index 0000000..eb5d422 --- /dev/null +++ b/meta/recipes-support/etc-overlay/files/overlay-parse-etc.service @@ -0,0 +1,14 @@ +[Unit] +Description=Reload Configuration from the etc overlay +DefaultDependencies=no +Requires=etc.mount +After=etc.mount +Before=local-fs-pre.target + +[Service] +Type=oneshot +RemainAfterExit=yes +ExecStartPre=!/bin/systemctl daemon-reload +ExecStart=!/bin/systemctl --no-block isolate multi-user.target +[Install] +WantedBy=local-fs-pre.target diff --git a/meta/recipes-support/etc-overlay/files/ovl.mount.tmpl b/meta/recipes-support/etc-overlay/files/ovl.mount.tmpl new file mode 100644 index 0000000..3593c55 --- /dev/null +++ b/meta/recipes-support/etc-overlay/files/ovl.mount.tmpl @@ -0,0 +1,14 @@ +[Unit] +Description=Mount /etc overlay backing store +DefaultDependencies=no +Before=local-fs-pre.target +Before=etc.mount + +[Mount] +What=/dev/disk/by-partlabel/etcovl +Where=/ovl +Type=ext4 +Options=noatime,nodiratime,data=journal,commit=${FS_COMMIT_INTERVAL},nodelalloc + +[Install] +WantedBy=local-fs-pre.target diff --git a/meta/recipes-support/etc-overlay/files/postinst b/meta/recipes-support/etc-overlay/files/postinst new file mode 100755 index 0000000..5b00d9d --- /dev/null +++ b/meta/recipes-support/etc-overlay/files/postinst @@ -0,0 +1,31 @@ +#!/bin/sh +# postinst script for etc-overlay +# +# see: dh_installdeb(1) + +set -e + +case "$1" in + configure) + deb-systemd-helper enable etc.mount || true + deb-systemd-helper enable ovl.mount || true + deb-systemd-helper enable overlay-parse-etc.service || true + deb-systemd-helper enable systemd-remount-fs.service || true + deb-systemd-helper enable etc-hostname.service || true + ;; + + abort-upgrade|abort-remove|abort-deconfigure) + ;; + + *) + echo "postinst called with unknown argument \`$1'" >&2 + exit 1 + ;; +esac + +# dh_installdeb will replace this with shell code automatically +# generated by other debhelper scripts. + +#DEBHELPER# + +exit 0 diff --git a/meta/recipes-support/etc-overlay/files/postrm b/meta/recipes-support/etc-overlay/files/postrm new file mode 100644 index 0000000..8e69c4d --- /dev/null +++ b/meta/recipes-support/etc-overlay/files/postrm @@ -0,0 +1,28 @@ +#!/bin/sh +# postrm script for etc-overlay +# +# see: dh_installdeb(1) + +set -e + +case "$1" in + purge|remove|upgrade|failed-upgrade|abort-install|abort-upgrade|disappear) + deb-systemd-helper disable etc.mount || true + deb-systemd-helper disable ovl.mount || true + deb-systemd-helper disable overlay-parse-etc.service || true + deb-systemd-helper disable systemd-remount-fs.service || true + deb-systemd-helper disable etc-hostname.service || true + ;; + + *) + echo "postrm called with unknown argument \`$1'" >&2 + exit 1 + ;; +esac + +# dh_installdeb will replace this with shell code automatically +# generated by other debhelper scripts. + +#DEBHELPER# + +exit 0 diff --git a/scripts/lib/wic/canned-wks/etc-overlay.inc b/scripts/lib/wic/canned-wks/etc-overlay.inc new file mode 100644 index 0000000..2ad4ff4 --- /dev/null +++ b/scripts/lib/wic/canned-wks/etc-overlay.inc @@ -0,0 +1,4 @@ +# add a overlay partition to the image + +# overlay partition +part --source etc-overlay --ondisk sda --size 100M --extra-space 128M --overhead-factor 1 --label etcovl --align 1024 --fstype=ext4 diff --git a/scripts/lib/wic/plugins/source/etc-overlay.py b/scripts/lib/wic/plugins/source/etc-overlay.py new file mode 100644 index 0000000..55189a2 --- /dev/null +++ b/scripts/lib/wic/plugins/source/etc-overlay.py @@ -0,0 +1,84 @@ +# ex:ts=4:sw=4:sts=4:et +# -*- tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*- +# +# Copyright (c) 2014, Intel Corporation. +# Copyright (c) 2018, Siemens AG. +# All rights reserved. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License version 2 as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License along +# with this program; if not, write to the Free Software Foundation, Inc., +# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. +# +# DESCRIPTION +# This implements the 'etc-overlay' source plugin class for 'wic' +# +# AUTHORS +# Tom Zanussi +# Andreas Reichel +# Quirin Gylstorff + + +import logging + +msger = logging.getLogger('wic') + +from wic.pluginbase import SourcePlugin +from wic.utils.misc import exec_cmd,BOOTDD_EXTRA_SPACE + +class EtcOverlayPlugin(SourcePlugin): + """ + Create an overlay file system scheme for etc + """ + + name = 'etc-overlay' + + @classmethod + def do_prepare_partition(cls, part, source_params, creator, cr_workdir, + oe_builddir, deploy_dir, kernel_dir, + rootfs_dir, native_sysroot): + + part_rootfs_dir = "%s/disk/%s.%s" % (cr_workdir, + part.label, + part.lineno) + create_dir_cmd = "install -d %s" % part_rootfs_dir + exec_cmd(create_dir_cmd) + + exec_cmd("install -m 0755 -d %s/etc" % part_rootfs_dir) + exec_cmd("install -m 0755 -d %s/.atomic" % part_rootfs_dir) + + blocks = 16 + extra_blocks = part.get_extra_block_count(blocks) + if extra_blocks < BOOTDD_EXTRA_SPACE: + extra_blocks = BOOTDD_EXTRA_SPACE + blocks += extra_blocks + blocks = blocks + (16 - (blocks % 16)) + + msger.debug("Added %d extra blocks to %s to get to %d total blocks", + extra_blocks, part.mountpoint, blocks) + + # ext4 image, created by mkfs.ext4 + etcovlimg = "%s/%s.%s.img" % (cr_workdir, part.label, part.lineno) + partfs_cmd = "dd if=/dev/zero of=%s bs=512 count=%d" % (etcovlimg, + blocks) + exec_cmd(partfs_cmd) + + partfs_cmd = "mkfs.ext4 %s -d %s" % (etcovlimg, part_rootfs_dir) + exec_cmd(partfs_cmd) + + chmod_cmd = "chmod 644 %s" % etcovlimg + exec_cmd(chmod_cmd) + + du_cmd = "du -Lbks %s" % etcovlimg + etcovlimg_size = int(exec_cmd(du_cmd).split()[0]) + + part.size = etcovlimg_size + part.source_file = etcovlimg -- 2.20.1