* [PATCH v2 0/2] bump bitbake and wic for python 3.10 support
@ 2022-04-21 8:52 Henning Schild
2022-04-21 8:52 ` [PATCH v2 1/2] bitbake: Update to 1.50.5 release Henning Schild
` (2 more replies)
0 siblings, 3 replies; 22+ messages in thread
From: Henning Schild @ 2022-04-21 8:52 UTC (permalink / raw)
To: isar-users; +Cc: Florian Bezdeka, Felix Moessbauer, Henning Schild
debian bookworm switched to python 3.10 recently, our versions of
bitbake and wic both have some issues with that new interpreter. All
these issues have been fixes in bitbake and OE, so all we need to do is
update the two tools in our tree.
v1 was send as "[PATCH] bitbake: Update to 1.50.5 release", which is p1
here, only later i found that wic also needs a bump to v2 added p2
If this series is applied we do not need "[PATCH] wic-img: use
python3.9 in bookworm", in case this one is applied before it should be
reverted after the version bumps.
Henning Schild (2):
bitbake: Update to 1.50.5 release
wic: Update to the latest revision
bitbake/lib/bb/cache.py | 3 +-
bitbake/lib/bb/cooker.py | 30 +++++++-
bitbake/lib/bb/data_smart.py | 4 +-
bitbake/lib/bb/fetch2/__init__.py | 4 +
bitbake/lib/bb/fetch2/perforce.py | 2 +-
bitbake/lib/bb/fetch2/wget.py | 2 +-
bitbake/lib/bb/persist_data.py | 5 +-
bitbake/lib/bb/process.py | 2 +-
bitbake/lib/bb/runqueue.py | 34 ++++----
bitbake/lib/bb/server/process.py | 2 +-
bitbake/lib/bb/tests/fetch.py | 35 +++++----
bitbake/lib/bb/utils.py | 13 +++-
bitbake/lib/hashserv/server.py | 4 +-
bitbake/lib/toaster/tests/builds/buildtest.py | 2 +-
.../wic/plugins/source/bootimg-efi-isar.py | 77 ++++++++++++++++---
.../wic/plugins/source/bootimg-pcbios-isar.py | 6 +-
scripts/lib/wic/canned-wks/common.wks.inc | 2 +-
scripts/lib/wic/canned-wks/directdisk-gpt.wks | 2 +-
scripts/lib/wic/canned-wks/mkefidisk.wks | 2 +-
scripts/lib/wic/engine.py | 6 +-
scripts/lib/wic/help.py | 10 ++-
scripts/lib/wic/ksparser.py | 8 +-
scripts/lib/wic/misc.py | 4 +-
scripts/lib/wic/partition.py | 25 ++++--
scripts/lib/wic/pluginbase.py | 8 +-
scripts/lib/wic/plugins/imager/direct.py | 11 ++-
scripts/lib/wic/plugins/source/bootimg-efi.py | 74 +++++++++++++++---
.../lib/wic/plugins/source/bootimg-pcbios.py | 6 +-
scripts/lib/wic/plugins/source/rawcopy.py | 35 ++++++++-
scripts/lib/wic/plugins/source/rootfs.py | 2 +-
scripts/wic | 9 ++-
31 files changed, 320 insertions(+), 109 deletions(-)
--
2.35.1
^ permalink raw reply [flat|nested] 22+ messages in thread* [PATCH v2 1/2] bitbake: Update to 1.50.5 release 2022-04-21 8:52 [PATCH v2 0/2] bump bitbake and wic for python 3.10 support Henning Schild @ 2022-04-21 8:52 ` Henning Schild 2022-04-21 8:52 ` [PATCH v2 2/2] wic: Update to the latest revision Henning Schild 2022-04-22 7:59 ` [PATCH v2 0/2] bump bitbake and wic for python 3.10 support Anton Mikanovich 2 siblings, 0 replies; 22+ messages in thread From: Henning Schild @ 2022-04-21 8:52 UTC (permalink / raw) To: isar-users; +Cc: Florian Bezdeka, Felix Moessbauer, Henning Schild Update bitbake to the latest release in 1.50 branch. This release is tagged to the commit ID aaa7f7af23d5f89fe4a5ed48c57ea3dfca07c79d in the bitbake upstream. Signed-off-by: Henning Schild <henning.schild@siemens.com> --- bitbake/lib/bb/cache.py | 3 +- bitbake/lib/bb/cooker.py | 30 ++++++++++++++-- bitbake/lib/bb/data_smart.py | 4 +-- bitbake/lib/bb/fetch2/__init__.py | 4 +++ bitbake/lib/bb/fetch2/perforce.py | 2 +- bitbake/lib/bb/fetch2/wget.py | 2 +- bitbake/lib/bb/persist_data.py | 5 +-- bitbake/lib/bb/process.py | 2 +- bitbake/lib/bb/runqueue.py | 34 +++++++++--------- bitbake/lib/bb/server/process.py | 2 +- bitbake/lib/bb/tests/fetch.py | 35 ++++++++++--------- bitbake/lib/bb/utils.py | 13 +++++-- bitbake/lib/hashserv/server.py | 4 +-- bitbake/lib/toaster/tests/builds/buildtest.py | 2 +- 14 files changed, 89 insertions(+), 53 deletions(-) diff --git a/bitbake/lib/bb/cache.py b/bitbake/lib/bb/cache.py index 27eb271798e8..5f9c0a779d75 100644 --- a/bitbake/lib/bb/cache.py +++ b/bitbake/lib/bb/cache.py @@ -19,7 +19,8 @@ import os import logging import pickle -from collections import defaultdict, Mapping +from collections import defaultdict +from collections.abc import Mapping import bb.utils from bb import PrefixLoggerAdapter import re diff --git a/bitbake/lib/bb/cooker.py b/bitbake/lib/bb/cooker.py index 89f1fad08310..c946800a8c62 100644 --- a/bitbake/lib/bb/cooker.py +++ b/bitbake/lib/bb/cooker.py @@ -388,12 +388,22 @@ class BBCooker: # Create a new hash server bound to a unix domain socket if not self.hashserv: dbfile = (self.data.getVar("PERSISTENT_DIR") or self.data.getVar("CACHE")) + "/hashserv.db" + upstream = self.data.getVar("BB_HASHSERVE_UPSTREAM") or None + if upstream: + import socket + try: + sock = socket.create_connection(upstream.split(":"), 5) + sock.close() + except socket.error as e: + bb.warn("BB_HASHSERVE_UPSTREAM is not valid, unable to connect hash equivalence server at '%s': %s" + % (upstream, repr(e))) + self.hashservaddr = "unix://%s/hashserve.sock" % self.data.getVar("TOPDIR") self.hashserv = hashserv.create_server( self.hashservaddr, dbfile, sync=False, - upstream=self.data.getVar("BB_HASHSERVE_UPSTREAM") or None, + upstream=upstream, ) self.hashserv.process = multiprocessing.Process(target=self.hashserv.serve_forever) self.hashserv.process.start() @@ -805,7 +815,9 @@ class BBCooker: for dep in rq.rqdata.runtaskentries[tid].depends: (depmc, depfn, _, deptaskfn) = bb.runqueue.split_tid_mcfn(dep) deppn = self.recipecaches[depmc].pkg_fn[deptaskfn] - depend_tree["tdepends"][dotname].append("%s.%s" % (deppn, bb.runqueue.taskname_from_tid(dep))) + if depmc: + depmc = "mc:" + depmc + ":" + depend_tree["tdepends"][dotname].append("%s%s.%s" % (depmc, deppn, bb.runqueue.taskname_from_tid(dep))) if taskfn not in seen_fns: seen_fns.append(taskfn) packages = [] @@ -2204,21 +2216,33 @@ class CookerParser(object): yield not cached, mc, infos def parse_generator(self): - while True: + empty = False + while self.processes or not empty: + for process in self.processes.copy(): + if not process.is_alive(): + process.join() + self.processes.remove(process) + if self.parsed >= self.toparse: break try: result = self.result_queue.get(timeout=0.25) except queue.Empty: + empty = True pass else: + empty = False value = result[1] if isinstance(value, BaseException): raise value else: yield result + if not (self.parsed >= self.toparse): + raise bb.parse.ParseError("Not all recipes parsed, parser thread killed/died? Exiting.", None) + + def parse_next(self): result = [] parsed = None diff --git a/bitbake/lib/bb/data_smart.py b/bitbake/lib/bb/data_smart.py index 8291ca65e309..65857a9c7941 100644 --- a/bitbake/lib/bb/data_smart.py +++ b/bitbake/lib/bb/data_smart.py @@ -17,7 +17,7 @@ BitBake build tools. # Based on functions from the base bb module, Copyright 2003 Holger Schurig import copy, re, sys, traceback -from collections import MutableMapping +from collections.abc import MutableMapping import logging import hashlib import bb, bb.codeparser @@ -403,7 +403,7 @@ class DataSmart(MutableMapping): s = __expand_python_regexp__.sub(varparse.python_sub, s) except SyntaxError as e: # Likely unmatched brackets, just don't expand the expression - if e.msg != "EOL while scanning string literal": + if e.msg != "EOL while scanning string literal" and not e.msg.startswith("unterminated string literal"): raise if s == olds: break diff --git a/bitbake/lib/bb/fetch2/__init__.py b/bitbake/lib/bb/fetch2/__init__.py index dbf8b50e68a7..1005ec10c639 100644 --- a/bitbake/lib/bb/fetch2/__init__.py +++ b/bitbake/lib/bb/fetch2/__init__.py @@ -430,6 +430,7 @@ def uri_replace(ud, uri_find, uri_replace, replacements, d, mirrortarball=None): uri_replace_decoded = list(decodeurl(uri_replace)) logger.debug2("For url %s comparing %s to %s" % (uri_decoded, uri_find_decoded, uri_replace_decoded)) result_decoded = ['', '', '', '', '', {}] + # 0 - type, 1 - host, 2 - path, 3 - user, 4- pswd, 5 - params for loc, i in enumerate(uri_find_decoded): result_decoded[loc] = uri_decoded[loc] regexp = i @@ -449,6 +450,9 @@ def uri_replace(ud, uri_find, uri_replace, replacements, d, mirrortarball=None): for l in replacements: uri_replace_decoded[loc][k] = uri_replace_decoded[loc][k].replace(l, replacements[l]) result_decoded[loc][k] = uri_replace_decoded[loc][k] + elif (loc == 3 or loc == 4) and uri_replace_decoded[loc]: + # User/password in the replacement is just a straight replacement + result_decoded[loc] = uri_replace_decoded[loc] elif (re.match(regexp, uri_decoded[loc])): if not uri_replace_decoded[loc]: result_decoded[loc] = "" diff --git a/bitbake/lib/bb/fetch2/perforce.py b/bitbake/lib/bb/fetch2/perforce.py index e2a41a4a1287..3b6fa4b1ec9a 100644 --- a/bitbake/lib/bb/fetch2/perforce.py +++ b/bitbake/lib/bb/fetch2/perforce.py @@ -134,7 +134,7 @@ class Perforce(FetchMethod): ud.setup_revisions(d) - ud.localfile = d.expand('%s_%s_%s_%s.tar.gz' % (cleanedhost, cleanedpath, cleandedmodule, ud.revision)) + ud.localfile = d.expand('%s_%s_%s_%s.tar.gz' % (cleanedhost, cleanedpath, cleanedmodule, ud.revision)) def _buildp4command(self, ud, d, command, depot_filename=None): """ diff --git a/bitbake/lib/bb/fetch2/wget.py b/bitbake/lib/bb/fetch2/wget.py index 784df70c9f62..7fa2a87ffde5 100644 --- a/bitbake/lib/bb/fetch2/wget.py +++ b/bitbake/lib/bb/fetch2/wget.py @@ -322,7 +322,7 @@ class Wget(FetchMethod): except (TypeError, ImportError, IOError, netrc.NetrcParseError): pass - with opener.open(r) as response: + with opener.open(r, timeout=30) as response: pass except urllib.error.URLError as e: if try_again: diff --git a/bitbake/lib/bb/persist_data.py b/bitbake/lib/bb/persist_data.py index c6a209fb3fc1..6f32d81afe80 100644 --- a/bitbake/lib/bb/persist_data.py +++ b/bitbake/lib/bb/persist_data.py @@ -12,6 +12,7 @@ currently, providing a key/value store accessed by 'domain'. # import collections +import collections.abc import contextlib import functools import logging @@ -19,7 +20,7 @@ import os.path import sqlite3 import sys import warnings -from collections import Mapping +from collections.abc import Mapping sqlversion = sqlite3.sqlite_version_info if sqlversion[0] < 3 or (sqlversion[0] == 3 and sqlversion[1] < 3): @@ -29,7 +30,7 @@ if sqlversion[0] < 3 or (sqlversion[0] == 3 and sqlversion[1] < 3): logger = logging.getLogger("BitBake.PersistData") @functools.total_ordering -class SQLTable(collections.MutableMapping): +class SQLTable(collections.abc.MutableMapping): class _Decorators(object): @staticmethod def retry(*, reconnect=True): diff --git a/bitbake/lib/bb/process.py b/bitbake/lib/bb/process.py index d5a1775fcec0..af5d804a1d59 100644 --- a/bitbake/lib/bb/process.py +++ b/bitbake/lib/bb/process.py @@ -60,7 +60,7 @@ class Popen(subprocess.Popen): "close_fds": True, "preexec_fn": subprocess_setup, "stdout": subprocess.PIPE, - "stderr": subprocess.STDOUT, + "stderr": subprocess.PIPE, "stdin": subprocess.PIPE, "shell": False, } diff --git a/bitbake/lib/bb/runqueue.py b/bitbake/lib/bb/runqueue.py index 10511a09dc1c..cd10da8b3a6f 100644 --- a/bitbake/lib/bb/runqueue.py +++ b/bitbake/lib/bb/runqueue.py @@ -926,38 +926,36 @@ class RunQueueData: # # Once all active tasks are marked, prune the ones we don't need. - delcount = {} - for tid in list(self.runtaskentries.keys()): - if tid not in runq_build: - delcount[tid] = self.runtaskentries[tid] - del self.runtaskentries[tid] - # Handle --runall if self.cooker.configuration.runall: # re-run the mark_active and then drop unused tasks from new list + reduced_tasklist = set(self.runtaskentries.keys()) + for tid in list(self.runtaskentries.keys()): + if tid not in runq_build: + reduced_tasklist.remove(tid) runq_build = {} for task in self.cooker.configuration.runall: if not task.startswith("do_"): task = "do_{0}".format(task) runall_tids = set() - for tid in list(self.runtaskentries): + for tid in reduced_tasklist: wanttid = "{0}:{1}".format(fn_from_tid(tid), task) - if wanttid in delcount: - self.runtaskentries[wanttid] = delcount[wanttid] if wanttid in self.runtaskentries: runall_tids.add(wanttid) for tid in list(runall_tids): - mark_active(tid,1) + mark_active(tid, 1) if self.cooker.configuration.force: invalidate_task(tid, False) - for tid in list(self.runtaskentries.keys()): - if tid not in runq_build: - delcount[tid] = self.runtaskentries[tid] - del self.runtaskentries[tid] + delcount = set() + for tid in list(self.runtaskentries.keys()): + if tid not in runq_build: + delcount.add(tid) + del self.runtaskentries[tid] + if self.cooker.configuration.runall: if len(self.runtaskentries) == 0: bb.msg.fatal("RunQueue", "Could not find any tasks with the tasknames %s to run within the recipes of the taskgraphs of the targets %s" % (str(self.cooker.configuration.runall), str(self.targets))) @@ -971,16 +969,16 @@ class RunQueueData: for task in self.cooker.configuration.runonly: if not task.startswith("do_"): task = "do_{0}".format(task) - runonly_tids = { k: v for k, v in self.runtaskentries.items() if taskname_from_tid(k) == task } + runonly_tids = [k for k in self.runtaskentries.keys() if taskname_from_tid(k) == task] - for tid in list(runonly_tids): - mark_active(tid,1) + for tid in runonly_tids: + mark_active(tid, 1) if self.cooker.configuration.force: invalidate_task(tid, False) for tid in list(self.runtaskentries.keys()): if tid not in runq_build: - delcount[tid] = self.runtaskentries[tid] + delcount.add(tid) del self.runtaskentries[tid] if len(self.runtaskentries) == 0: diff --git a/bitbake/lib/bb/server/process.py b/bitbake/lib/bb/server/process.py index 07bb785a1822..fcdce19717d2 100644 --- a/bitbake/lib/bb/server/process.py +++ b/bitbake/lib/bb/server/process.py @@ -659,7 +659,7 @@ class BBUIEventQueue: self.reader = ConnectionReader(readfd) self.t = threading.Thread() - self.t.setDaemon(True) + self.t.daemon = True self.t.run = self.startCallbackHandler self.t.start() diff --git a/bitbake/lib/bb/tests/fetch.py b/bitbake/lib/bb/tests/fetch.py index f5d557e8c000..3b64584da0e9 100644 --- a/bitbake/lib/bb/tests/fetch.py +++ b/bitbake/lib/bb/tests/fetch.py @@ -431,6 +431,10 @@ class MirrorUriTest(FetcherTest): ("git://someserver.org/bitbake;tag=1234567890123456789012345678901234567890;branch=master", "git://someserver.org/bitbake;branch=master", "git://git.openembedded.org/bitbake;protocol=http") : "git://git.openembedded.org/bitbake;tag=1234567890123456789012345678901234567890;branch=master;protocol=http", + ("git://user1@someserver.org/bitbake;tag=1234567890123456789012345678901234567890;branch=master", "git://someserver.org/bitbake;branch=master", "git://user2@git.openembedded.org/bitbake;protocol=http") + : "git://user2@git.openembedded.org/bitbake;tag=1234567890123456789012345678901234567890;branch=master;protocol=http", + + #Renaming files doesn't work #("http://somewhere.org/somedir1/somefile_1.2.3.tar.gz", "http://somewhere.org/somedir1/somefile_1.2.3.tar.gz", "http://somewhere2.org/somedir3/somefile_2.3.4.tar.gz") : "http://somewhere2.org/somedir3/somefile_2.3.4.tar.gz" #("file://sstate-xyz.tgz", "file://.*/.*", "file:///somewhere/1234/sstate-cache") : "file:///somewhere/1234/sstate-cache/sstate-xyz.tgz", @@ -491,7 +495,7 @@ class GitDownloadDirectoryNamingTest(FetcherTest): super(GitDownloadDirectoryNamingTest, self).setUp() self.recipe_url = "git://git.openembedded.org/bitbake" self.recipe_dir = "git.openembedded.org.bitbake" - self.mirror_url = "git://github.com/openembedded/bitbake.git" + self.mirror_url = "git://github.com/openembedded/bitbake.git;protocol=https" self.mirror_dir = "github.com.openembedded.bitbake.git" self.d.setVar('SRCREV', '82ea737a0b42a8b53e11c9cde141e9e9c0bd8c40') @@ -539,7 +543,7 @@ class TarballNamingTest(FetcherTest): super(TarballNamingTest, self).setUp() self.recipe_url = "git://git.openembedded.org/bitbake" self.recipe_tarball = "git2_git.openembedded.org.bitbake.tar.gz" - self.mirror_url = "git://github.com/openembedded/bitbake.git" + self.mirror_url = "git://github.com/openembedded/bitbake.git;protocol=https" self.mirror_tarball = "git2_github.com.openembedded.bitbake.git.tar.gz" self.d.setVar('BB_GENERATE_MIRROR_TARBALLS', '1') @@ -573,7 +577,7 @@ class GitShallowTarballNamingTest(FetcherTest): super(GitShallowTarballNamingTest, self).setUp() self.recipe_url = "git://git.openembedded.org/bitbake" self.recipe_tarball = "gitshallow_git.openembedded.org.bitbake_82ea737-1_master.tar.gz" - self.mirror_url = "git://github.com/openembedded/bitbake.git" + self.mirror_url = "git://github.com/openembedded/bitbake.git;protocol=https" self.mirror_tarball = "gitshallow_github.com.openembedded.bitbake.git_82ea737-1_master.tar.gz" self.d.setVar('BB_GIT_SHALLOW', '1') @@ -985,7 +989,7 @@ class FetcherNetworkTest(FetcherTest): def test_git_submodule_dbus_broker(self): # The following external repositories have show failures in fetch and unpack operations # We want to avoid regressions! - url = "gitsm://github.com/bus1/dbus-broker;protocol=git;rev=fc874afa0992d0c75ec25acb43d344679f0ee7d2;branch=main" + url = "gitsm://github.com/bus1/dbus-broker;protocol=https;rev=fc874afa0992d0c75ec25acb43d344679f0ee7d2;branch=main" fetcher = bb.fetch.Fetch([url], self.d) fetcher.download() # Previous cwd has been deleted @@ -1001,7 +1005,7 @@ class FetcherNetworkTest(FetcherTest): @skipIfNoNetwork() def test_git_submodule_CLI11(self): - url = "gitsm://github.com/CLIUtils/CLI11;protocol=git;rev=bd4dc911847d0cde7a6b41dfa626a85aab213baf;branch=main" + url = "gitsm://github.com/CLIUtils/CLI11;protocol=https;rev=bd4dc911847d0cde7a6b41dfa626a85aab213baf;branch=main" fetcher = bb.fetch.Fetch([url], self.d) fetcher.download() # Previous cwd has been deleted @@ -1016,12 +1020,12 @@ class FetcherNetworkTest(FetcherTest): @skipIfNoNetwork() def test_git_submodule_update_CLI11(self): """ Prevent regression on update detection not finding missing submodule, or modules without needed commits """ - url = "gitsm://github.com/CLIUtils/CLI11;protocol=git;rev=cf6a99fa69aaefe477cc52e3ef4a7d2d7fa40714;branch=main" + url = "gitsm://github.com/CLIUtils/CLI11;protocol=https;rev=cf6a99fa69aaefe477cc52e3ef4a7d2d7fa40714;branch=main" fetcher = bb.fetch.Fetch([url], self.d) fetcher.download() # CLI11 that pulls in a newer nlohmann-json - url = "gitsm://github.com/CLIUtils/CLI11;protocol=git;rev=49ac989a9527ee9bb496de9ded7b4872c2e0e5ca;branch=main" + url = "gitsm://github.com/CLIUtils/CLI11;protocol=https;rev=49ac989a9527ee9bb496de9ded7b4872c2e0e5ca;branch=main" fetcher = bb.fetch.Fetch([url], self.d) fetcher.download() # Previous cwd has been deleted @@ -1035,7 +1039,7 @@ class FetcherNetworkTest(FetcherTest): @skipIfNoNetwork() def test_git_submodule_aktualizr(self): - url = "gitsm://github.com/advancedtelematic/aktualizr;branch=master;protocol=git;rev=d00d1a04cc2366d1a5f143b84b9f507f8bd32c44" + url = "gitsm://github.com/advancedtelematic/aktualizr;branch=master;protocol=https;rev=d00d1a04cc2366d1a5f143b84b9f507f8bd32c44" fetcher = bb.fetch.Fetch([url], self.d) fetcher.download() # Previous cwd has been deleted @@ -1055,7 +1059,7 @@ class FetcherNetworkTest(FetcherTest): """ Prevent regression on deeply nested submodules not being checked out properly, even though they were fetched. """ # This repository also has submodules where the module (name), path and url do not align - url = "gitsm://github.com/azure/iotedge.git;protocol=git;rev=d76e0316c6f324345d77c48a83ce836d09392699" + url = "gitsm://github.com/azure/iotedge.git;protocol=https;rev=d76e0316c6f324345d77c48a83ce836d09392699" fetcher = bb.fetch.Fetch([url], self.d) fetcher.download() # Previous cwd has been deleted @@ -1113,7 +1117,7 @@ class SVNTest(FetcherTest): bb.process.run("svn co %s svnfetch_co" % self.repo_url, cwd=self.tempdir) # Github will emulate SVN. Use this to check if we're downloding... - bb.process.run("svn propset svn:externals 'bitbake svn://vcs.pcre.org/pcre2/code' .", + bb.process.run("svn propset svn:externals 'bitbake https://github.com/PhilipHazel/pcre2.git' .", cwd=os.path.join(self.tempdir, 'svnfetch_co', 'trunk')) bb.process.run("svn commit --non-interactive -m 'Add external'", cwd=os.path.join(self.tempdir, 'svnfetch_co', 'trunk')) @@ -1231,7 +1235,7 @@ class FetchLatestVersionTest(FetcherTest): test_git_uris = { # version pattern "X.Y.Z" - ("mx-1.0", "git://github.com/clutter-project/mx.git;branch=mx-1.4", "9b1db6b8060bd00b121a692f942404a24ae2960f", "") + ("mx-1.0", "git://github.com/clutter-project/mx.git;branch=mx-1.4;protocol=https", "9b1db6b8060bd00b121a692f942404a24ae2960f", "") : "1.99.4", # version pattern "vX.Y" # mirror of git.infradead.org since network issues interfered with testing @@ -1258,9 +1262,9 @@ class FetchLatestVersionTest(FetcherTest): : "0.4.3", ("build-appliance-image", "git://git.yoctoproject.org/poky", "b37dd451a52622d5b570183a81583cc34c2ff555", "(?P<pver>(([0-9][\.|_]?)+[0-9]))") : "11.0.0", - ("chkconfig-alternatives-native", "git://github.com/kergoth/chkconfig;branch=sysroot", "cd437ecbd8986c894442f8fce1e0061e20f04dee", "chkconfig\-(?P<pver>((\d+[\.\-_]*)+))") + ("chkconfig-alternatives-native", "git://github.com/kergoth/chkconfig;branch=sysroot;protocol=https", "cd437ecbd8986c894442f8fce1e0061e20f04dee", "chkconfig\-(?P<pver>((\d+[\.\-_]*)+))") : "1.3.59", - ("remake", "git://github.com/rocky/remake.git", "f05508e521987c8494c92d9c2871aec46307d51d", "(?P<pver>(\d+\.(\d+\.)*\d*(\+dbg\d+(\.\d+)*)*))") + ("remake", "git://github.com/rocky/remake.git;protocol=https", "f05508e521987c8494c92d9c2871aec46307d51d", "(?P<pver>(\d+\.(\d+\.)*\d*(\+dbg\d+(\.\d+)*)*))") : "3.82+dbg0.9", } @@ -1354,9 +1358,6 @@ class FetchCheckStatusTest(FetcherTest): "http://downloads.yoctoproject.org/releases/opkg/opkg-0.1.7.tar.gz", "http://downloads.yoctoproject.org/releases/opkg/opkg-0.3.0.tar.gz", "ftp://sourceware.org/pub/libffi/libffi-1.20.tar.gz", - "http://ftp.gnu.org/gnu/autoconf/autoconf-2.60.tar.gz", - "https://ftp.gnu.org/gnu/chess/gnuchess-5.08.tar.gz", - "https://ftp.gnu.org/gnu/gmp/gmp-4.0.tar.gz", # GitHub releases are hosted on Amazon S3, which doesn't support HEAD "https://github.com/kergoth/tslib/releases/download/1.1/tslib-1.1.tar.xz" ] @@ -2047,7 +2048,7 @@ class GitShallowTest(FetcherTest): @skipIfNoNetwork() def test_bitbake(self): - self.git('remote add --mirror=fetch origin git://github.com/openembedded/bitbake', cwd=self.srcdir) + self.git('remote add --mirror=fetch origin https://github.com/openembedded/bitbake', cwd=self.srcdir) self.git('config core.bare true', cwd=self.srcdir) self.git('fetch', cwd=self.srcdir) diff --git a/bitbake/lib/bb/utils.py b/bitbake/lib/bb/utils.py index b282d09abfce..2a150fe9c73e 100644 --- a/bitbake/lib/bb/utils.py +++ b/bitbake/lib/bb/utils.py @@ -16,7 +16,8 @@ import bb.msg import multiprocessing import fcntl import importlib -from importlib import machinery +import importlib.machinery +import importlib.util import itertools import subprocess import glob @@ -451,6 +452,10 @@ def lockfile(name, shared=False, retry=True, block=False): consider the possibility of sending a signal to the process to break out - at which point you want block=True rather than retry=True. """ + if len(name) > 255: + root, ext = os.path.splitext(name) + name = root[:255 - len(ext)] + ext + dirname = os.path.dirname(name) mkdirhier(dirname) @@ -487,7 +492,7 @@ def lockfile(name, shared=False, retry=True, block=False): return lf lf.close() except OSError as e: - if e.errno == errno.EACCES: + if e.errno == errno.EACCES or e.errno == errno.ENAMETOOLONG: logger.error("Unable to acquire lock '%s', %s", e.strerror, name) sys.exit(1) @@ -1616,7 +1621,9 @@ def load_plugins(logger, plugins, pluginpath): logger.debug('Loading plugin %s' % name) spec = importlib.machinery.PathFinder.find_spec(name, path=[pluginpath] ) if spec: - return spec.loader.load_module() + mod = importlib.util.module_from_spec(spec) + spec.loader.exec_module(mod) + return mod logger.debug('Loading plugins from %s...' % pluginpath) diff --git a/bitbake/lib/hashserv/server.py b/bitbake/lib/hashserv/server.py index a0dc0c170f2b..df0fa0a07937 100644 --- a/bitbake/lib/hashserv/server.py +++ b/bitbake/lib/hashserv/server.py @@ -521,7 +521,7 @@ class Server(object): def start_tcp_server(self, host, port): self.server = self.loop.run_until_complete( - asyncio.start_server(self.handle_client, host, port, loop=self.loop) + asyncio.start_server(self.handle_client, host, port) ) for s in self.server.sockets: @@ -546,7 +546,7 @@ class Server(object): # Work around path length limits in AF_UNIX os.chdir(os.path.dirname(path)) self.server = self.loop.run_until_complete( - asyncio.start_unix_server(self.handle_client, os.path.basename(path), loop=self.loop) + asyncio.start_unix_server(self.handle_client, os.path.basename(path)) ) finally: os.chdir(cwd) diff --git a/bitbake/lib/toaster/tests/builds/buildtest.py b/bitbake/lib/toaster/tests/builds/buildtest.py index 872bbd377510..13b51fb0d8e4 100644 --- a/bitbake/lib/toaster/tests/builds/buildtest.py +++ b/bitbake/lib/toaster/tests/builds/buildtest.py @@ -119,7 +119,7 @@ class BuildTest(unittest.TestCase): if os.environ.get("TOASTER_TEST_USE_SSTATE_MIRROR"): ProjectVariable.objects.get_or_create( name="SSTATE_MIRRORS", - value="file://.* http://autobuilder.yoctoproject.org/pub/sstate/PATH;downloadfilename=PATH", + value="file://.* http://sstate.yoctoproject.org/PATH;downloadfilename=PATH", project=project) ProjectTarget.objects.create(project=project, -- 2.35.1 ^ permalink raw reply [flat|nested] 22+ messages in thread
* [PATCH v2 2/2] wic: Update to the latest revision 2022-04-21 8:52 [PATCH v2 0/2] bump bitbake and wic for python 3.10 support Henning Schild 2022-04-21 8:52 ` [PATCH v2 1/2] bitbake: Update to 1.50.5 release Henning Schild @ 2022-04-21 8:52 ` Henning Schild 2022-04-21 9:02 ` Henning Schild 2022-04-22 7:59 ` [PATCH v2 0/2] bump bitbake and wic for python 3.10 support Anton Mikanovich 2 siblings, 1 reply; 22+ messages in thread From: Henning Schild @ 2022-04-21 8:52 UTC (permalink / raw) To: isar-users; +Cc: Florian Bezdeka, Felix Moessbauer, Henning Schild Update to the latest wic from OE-core. OE-core Revision: 712552b5cc427d7be1258c45886de9b57f7272c9 Signed-off-by: Henning Schild <henning.schild@siemens.com> --- .../wic/plugins/source/bootimg-efi-isar.py | 77 ++++++++++++++++--- .../wic/plugins/source/bootimg-pcbios-isar.py | 6 +- scripts/lib/wic/canned-wks/common.wks.inc | 2 +- scripts/lib/wic/canned-wks/directdisk-gpt.wks | 2 +- scripts/lib/wic/canned-wks/mkefidisk.wks | 2 +- scripts/lib/wic/engine.py | 6 +- scripts/lib/wic/help.py | 10 ++- scripts/lib/wic/ksparser.py | 8 +- scripts/lib/wic/misc.py | 4 +- scripts/lib/wic/partition.py | 25 ++++-- scripts/lib/wic/pluginbase.py | 8 +- scripts/lib/wic/plugins/imager/direct.py | 11 ++- scripts/lib/wic/plugins/source/bootimg-efi.py | 74 +++++++++++++++--- .../lib/wic/plugins/source/bootimg-pcbios.py | 6 +- scripts/lib/wic/plugins/source/rawcopy.py | 35 ++++++++- scripts/lib/wic/plugins/source/rootfs.py | 2 +- scripts/wic | 9 ++- 17 files changed, 231 insertions(+), 56 deletions(-) diff --git a/meta/scripts/lib/wic/plugins/source/bootimg-efi-isar.py b/meta/scripts/lib/wic/plugins/source/bootimg-efi-isar.py index 5ba0777ad244..a24e04f309da 100644 --- a/meta/scripts/lib/wic/plugins/source/bootimg-efi-isar.py +++ b/meta/scripts/lib/wic/plugins/source/bootimg-efi-isar.py @@ -12,6 +12,7 @@ import logging import os +import tempfile import shutil import re @@ -129,12 +130,13 @@ class BootimgEFIPlugin(SourcePlugin): bootloader = creator.ks.bootloader loader_conf = "" - loader_conf += "default boot\n" + if source_params.get('create-unified-kernel-image') != "true": + loader_conf += "default boot\n" loader_conf += "timeout %d\n" % bootloader.timeout initrd = source_params.get('initrd') - if initrd: + if initrd and source_params.get('create-unified-kernel-image') != "true": # obviously we need to have a common common deploy var bootimg_dir = get_bitbake_var("DEPLOY_DIR_IMAGE") if not bootimg_dir: @@ -195,11 +197,12 @@ class BootimgEFIPlugin(SourcePlugin): for rd in initrds: boot_conf += "initrd /%s\n" % rd - logger.debug("Writing systemd-boot config " - "%s/hdd/boot/loader/entries/boot.conf", cr_workdir) - cfg = open("%s/hdd/boot/loader/entries/boot.conf" % cr_workdir, "w") - cfg.write(boot_conf) - cfg.close() + if source_params.get('create-unified-kernel-image') != "true": + logger.debug("Writing systemd-boot config " + "%s/hdd/boot/loader/entries/boot.conf", cr_workdir) + cfg = open("%s/hdd/boot/loader/entries/boot.conf" % cr_workdir, "w") + cfg.write(boot_conf) + cfg.close() @classmethod @@ -300,11 +303,63 @@ class BootimgEFIPlugin(SourcePlugin): kernel = "%s-%s.bin" % \ (get_bitbake_var("KERNEL_IMAGETYPE"), get_bitbake_var("INITRAMFS_LINK_NAME")) - install_cmd = "install -m 0644 %s/%s %s/%s" % \ - (staging_kernel_dir, kernel, hdddir, kernel) + if source_params.get('create-unified-kernel-image') == "true": + initrd = source_params.get('initrd') + if not initrd: + raise WicError("initrd= must be specified when create-unified-kernel-image=true, exiting") + + deploy_dir = get_bitbake_var("DEPLOY_DIR_IMAGE") + efi_stub = glob("%s/%s" % (deploy_dir, "linux*.efi.stub")) + if len(efi_stub) == 0: + raise WicError("Unified Kernel Image EFI stub not found, exiting") + efi_stub = efi_stub[0] + + with tempfile.TemporaryDirectory() as tmp_dir: + label = source_params.get('label') + label_conf = "root=%s" % creator.rootdev + if label: + label_conf = "LABEL=%s" % label + + bootloader = creator.ks.bootloader + cmdline = open("%s/cmdline" % tmp_dir, "w") + cmdline.write("%s %s" % (label_conf, bootloader.append)) + cmdline.close() + + initrds = initrd.split(';') + initrd = open("%s/initrd" % tmp_dir, "wb") + for f in initrds: + with open("%s/%s" % (deploy_dir, f), 'rb') as in_file: + shutil.copyfileobj(in_file, initrd) + initrd.close() + + # Searched by systemd-boot: + # https://systemd.io/BOOT_LOADER_SPECIFICATION/#type-2-efi-unified-kernel-images + install_cmd = "install -d %s/EFI/Linux" % hdddir + exec_cmd(install_cmd) + + staging_dir_host = get_bitbake_var("STAGING_DIR_HOST") + + # https://www.freedesktop.org/software/systemd/man/systemd-stub.html + objcopy_cmd = "objcopy \ + --add-section .osrel=%s --change-section-vma .osrel=0x20000 \ + --add-section .cmdline=%s --change-section-vma .cmdline=0x30000 \ + --add-section .linux=%s --change-section-vma .linux=0x2000000 \ + --add-section .initrd=%s --change-section-vma .initrd=0x3000000 \ + %s %s" % \ + ("%s/usr/lib/os-release" % staging_dir_host, + cmdline.name, + "%s/%s" % (staging_kernel_dir, kernel), + initrd.name, + efi_stub, + "%s/EFI/Linux/linux.efi" % hdddir) + exec_cmd(objcopy_cmd) + else: + install_cmd = "install -m 0644 %s/%s %s/%s" % \ + (staging_kernel_dir, kernel, hdddir, kernel) + + install_cmd = isar_populate_boot_cmd(rootfs_dir['ROOTFS_DIR'], hdddir) + exec_cmd(install_cmd) - install_cmd = isar_populate_boot_cmd(rootfs_dir['ROOTFS_DIR'], hdddir) - exec_cmd(install_cmd) if get_bitbake_var("IMAGE_EFI_BOOT_FILES"): for src_path, dst_path in cls.install_task: diff --git a/meta/scripts/lib/wic/plugins/source/bootimg-pcbios-isar.py b/meta/scripts/lib/wic/plugins/source/bootimg-pcbios-isar.py index 9136d4f215ca..7fb0f2a87394 100644 --- a/meta/scripts/lib/wic/plugins/source/bootimg-pcbios-isar.py +++ b/meta/scripts/lib/wic/plugins/source/bootimg-pcbios-isar.py @@ -213,8 +213,10 @@ class BootimgPcbiosIsarPlugin(SourcePlugin): # dosfs image, created by mkdosfs bootimg = "%s/boot%s.img" % (cr_workdir, part.lineno) - dosfs_cmd = "mkdosfs -n boot -i %s -S 512 -C %s %d" % \ - (part.fsuuid, bootimg, blocks) + label = part.label if part.label else "boot" + + dosfs_cmd = "mkdosfs -n %s -i %s -S 512 -C %s %d" % \ + (label, part.fsuuid, bootimg, blocks) exec_native_cmd(dosfs_cmd, native_sysroot) mcopy_cmd = "mcopy -i %s -s %s/* ::/" % (bootimg, hdddir) diff --git a/scripts/lib/wic/canned-wks/common.wks.inc b/scripts/lib/wic/canned-wks/common.wks.inc index 4fd29fa8c119..89880b417b6e 100644 --- a/scripts/lib/wic/canned-wks/common.wks.inc +++ b/scripts/lib/wic/canned-wks/common.wks.inc @@ -1,3 +1,3 @@ # This file is included into 3 canned wks files from this directory part /boot --source bootimg-pcbios --ondisk sda --label boot --active --align 1024 -part / --source rootfs --use-uuid --fstype=ext4 --mkfs-extraopts "-T default" --label platform --align 1024 +part / --source rootfs --use-uuid --fstype=ext4 --label platform --align 1024 diff --git a/scripts/lib/wic/canned-wks/directdisk-gpt.wks b/scripts/lib/wic/canned-wks/directdisk-gpt.wks index cf16c0c30bbd..8d7d8de6ea7e 100644 --- a/scripts/lib/wic/canned-wks/directdisk-gpt.wks +++ b/scripts/lib/wic/canned-wks/directdisk-gpt.wks @@ -4,7 +4,7 @@ part /boot --source bootimg-pcbios --ondisk sda --label boot --active --align 1024 -part / --source rootfs --ondisk sda --fstype=ext4 --mkfs-extraopts "-T default" --label platform --align 1024 --use-uuid +part / --source rootfs --ondisk sda --fstype=ext4 --label platform --align 1024 --use-uuid bootloader --ptable gpt --timeout=0 --append="rootwait rootfstype=ext4 video=vesafb vga=0x318 console=tty0 console=ttyS0,115200n8" diff --git a/scripts/lib/wic/canned-wks/mkefidisk.wks b/scripts/lib/wic/canned-wks/mkefidisk.wks index d1878e23e5a3..9f534fe18471 100644 --- a/scripts/lib/wic/canned-wks/mkefidisk.wks +++ b/scripts/lib/wic/canned-wks/mkefidisk.wks @@ -4,7 +4,7 @@ part /boot --source bootimg-efi --sourceparams="loader=grub-efi" --ondisk sda --label msdos --active --align 1024 -part / --source rootfs --ondisk sda --fstype=ext4 --mkfs-extraopts "-T default" --label platform --align 1024 --use-uuid +part / --source rootfs --ondisk sda --fstype=ext4 --label platform --align 1024 --use-uuid part swap --ondisk sda --size 44 --label swap1 --fstype=swap diff --git a/scripts/lib/wic/engine.py b/scripts/lib/wic/engine.py index 018815b96688..674ccfc24418 100644 --- a/scripts/lib/wic/engine.py +++ b/scripts/lib/wic/engine.py @@ -19,10 +19,10 @@ import os import tempfile import json import subprocess +import shutil import re from collections import namedtuple, OrderedDict -from distutils.spawn import find_executable from wic import WicError from wic.filemap import sparse_copy @@ -245,7 +245,7 @@ class Disk: for path in pathlist.split(':'): self.paths = "%s%s:%s" % (native_sysroot, path, self.paths) - self.parted = find_executable("parted", self.paths) + self.parted = shutil.which("parted", path=self.paths) if not self.parted: raise WicError("Can't find executable parted") @@ -283,7 +283,7 @@ class Disk: "resize2fs", "mkswap", "mkdosfs", "debugfs","blkid"): aname = "_%s" % name if aname not in self.__dict__: - setattr(self, aname, find_executable(name, self.paths)) + setattr(self, aname, shutil.which(name, path=self.paths)) if aname not in self.__dict__ or self.__dict__[aname] is None: raise WicError("Can't find executable '{}'".format(name)) return self.__dict__[aname] diff --git a/scripts/lib/wic/help.py b/scripts/lib/wic/help.py index bd3a2b97dfaf..4ff7470a6a79 100644 --- a/scripts/lib/wic/help.py +++ b/scripts/lib/wic/help.py @@ -637,7 +637,7 @@ DESCRIPTION oe-core: directdisk.bbclass and mkefidisk.sh. The difference between wic and those examples is that with wic the functionality of those scripts is implemented by a general-purpose partitioning - 'language' based on Redhat kickstart syntax). + 'language' based on Red Hat kickstart syntax). The initial motivation and design considerations that lead to the current tool are described exhaustively in Yocto Bug #3847 @@ -840,8 +840,8 @@ DESCRIPTION meanings. The commands are based on the Fedora kickstart documentation but with modifications to reflect wic capabilities. - http://fedoraproject.org/wiki/Anaconda/Kickstart#part_or_partition - http://fedoraproject.org/wiki/Anaconda/Kickstart#bootloader + https://pykickstart.readthedocs.io/en/latest/kickstart-docs.html#part-or-partition + https://pykickstart.readthedocs.io/en/latest/kickstart-docs.html#bootloader Commands @@ -930,6 +930,7 @@ DESCRIPTION ext4 btrfs squashfs + erofs swap --fsoptions: Specifies a free-form string of options to be @@ -990,6 +991,9 @@ DESCRIPTION multiple partitions and we want to keep the right permissions and usernames in all the partitions. + --no-fstab-update: This option is specific to wic. It does not update the + '/etc/fstab' stock file for the given partition. + --extra-space: This option is specific to wic. It adds extra space after the space filled by the content of the partition. The final size can go diff --git a/scripts/lib/wic/ksparser.py b/scripts/lib/wic/ksparser.py index 3eb669da39ca..0df9eb0d057d 100644 --- a/scripts/lib/wic/ksparser.py +++ b/scripts/lib/wic/ksparser.py @@ -157,7 +157,8 @@ class KickStart(): part.add_argument('--fsoptions', dest='fsopts') part.add_argument('--fstype', default='vfat', choices=('ext2', 'ext3', 'ext4', 'btrfs', - 'squashfs', 'vfat', 'msdos', 'swap')) + 'squashfs', 'vfat', 'msdos', 'erofs', + 'swap')) part.add_argument('--mkfs-extraopts', default='') part.add_argument('--label') part.add_argument('--use-label', action='store_true') @@ -184,6 +185,7 @@ class KickStart(): part.add_argument('--use-uuid', action='store_true') part.add_argument('--uuid') part.add_argument('--fsuuid') + part.add_argument('--no-fstab-update', action='store_true') bootloader = subparsers.add_parser('bootloader') bootloader.add_argument('--append') @@ -229,6 +231,10 @@ class KickStart(): err = "%s:%d: SquashFS does not support LABEL" \ % (confpath, lineno) raise KickStartError(err) + # erofs does not support filesystem labels + if parsed.fstype == 'erofs' and parsed.label: + err = "%s:%d: erofs does not support LABEL" % (confpath, lineno) + raise KickStartError(err) if parsed.fstype == 'msdos' or parsed.fstype == 'vfat': if parsed.fsuuid: if parsed.fsuuid.upper().startswith('0X'): diff --git a/scripts/lib/wic/misc.py b/scripts/lib/wic/misc.py index 57c042c503e6..3e118229960b 100644 --- a/scripts/lib/wic/misc.py +++ b/scripts/lib/wic/misc.py @@ -16,9 +16,9 @@ import logging import os import re import subprocess +import shutil from collections import defaultdict -from distutils import spawn from wic import WicError @@ -122,7 +122,7 @@ def find_executable(cmd, paths): if provided and "%s-native" % recipe in provided: return True - return spawn.find_executable(cmd, paths) + return shutil.which(cmd, path=paths) def exec_native_cmd(cmd_and_args, native_sysroot, pseudo=""): """ diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py index 76d144d12d30..09e491dd494d 100644 --- a/scripts/lib/wic/partition.py +++ b/scripts/lib/wic/partition.py @@ -54,6 +54,7 @@ class Partition(): self.uuid = args.uuid self.fsuuid = args.fsuuid self.type = args.type + self.no_fstab_update = args.no_fstab_update self.updated_fstab_path = None self.has_fstab = False self.update_fstab_in_rootfs = False @@ -104,7 +105,7 @@ class Partition(): extra_blocks = self.extra_space rootfs_size = actual_rootfs_size + extra_blocks - rootfs_size *= self.overhead_factor + rootfs_size = int(rootfs_size * self.overhead_factor) logger.debug("Added %d extra blocks to %s to get to %d total blocks", extra_blocks, self.mountpoint, rootfs_size) @@ -141,9 +142,9 @@ class Partition(): native_sysroot) self.source_file = "%s/fs.%s" % (cr_workdir, self.fstype) else: - if self.fstype == 'squashfs': - raise WicError("It's not possible to create empty squashfs " - "partition '%s'" % (self.mountpoint)) + if self.fstype in ('squashfs', 'erofs'): + raise WicError("It's not possible to create empty %s " + "partition '%s'" % (self.fstype, self.mountpoint)) rootfs = "%s/fs_%s.%s.%s" % (cr_workdir, self.label, self.lineno, self.fstype) @@ -170,7 +171,7 @@ class Partition(): # Split sourceparams string of the form key1=val1[,key2=val2,...] # into a dict. Also accepts valueless keys i.e. without = splitted = self.sourceparams.split(',') - srcparams_dict = dict(par.split('=', 1) for par in splitted if par) + srcparams_dict = dict((par.split('=', 1) + [None])[:2] for par in splitted if par) plugin = PluginMgr.get_plugins('source')[self.source] plugin.do_configure_partition(self, srcparams_dict, creator, @@ -286,7 +287,7 @@ class Partition(): (self.fstype, extraopts, rootfs, label_str, self.fsuuid, rootfs_dir) exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo) - if self.updated_fstab_path and self.has_fstab: + if self.updated_fstab_path and self.has_fstab and not self.no_fstab_update: debugfs_script_path = os.path.join(cr_workdir, "debugfs_script") with open(debugfs_script_path, "w") as f: f.write("cd etc\n") @@ -350,7 +351,7 @@ class Partition(): mcopy_cmd = "mcopy -i %s -s %s/* ::/" % (rootfs, rootfs_dir) exec_native_cmd(mcopy_cmd, native_sysroot) - if self.updated_fstab_path and self.has_fstab: + if self.updated_fstab_path and self.has_fstab and not self.no_fstab_update: mcopy_cmd = "mcopy -i %s %s ::/etc/fstab" % (rootfs, self.updated_fstab_path) exec_native_cmd(mcopy_cmd, native_sysroot) @@ -369,6 +370,16 @@ class Partition(): (rootfs_dir, rootfs, extraopts) exec_native_cmd(squashfs_cmd, native_sysroot, pseudo=pseudo) + def prepare_rootfs_erofs(self, rootfs, cr_workdir, oe_builddir, rootfs_dir, + native_sysroot, pseudo): + """ + Prepare content for a erofs rootfs partition. + """ + extraopts = self.mkfs_extraopts or '' + erofs_cmd = "mkfs.erofs %s -U %s %s %s" % \ + (extraopts, self.fsuuid, rootfs, rootfs_dir) + exec_native_cmd(erofs_cmd, native_sysroot, pseudo=pseudo) + def prepare_empty_partition_ext(self, rootfs, oe_builddir, native_sysroot): """ diff --git a/scripts/lib/wic/pluginbase.py b/scripts/lib/wic/pluginbase.py index d9b4e57747e7..b64568339b1b 100644 --- a/scripts/lib/wic/pluginbase.py +++ b/scripts/lib/wic/pluginbase.py @@ -9,9 +9,11 @@ __all__ = ['ImagerPlugin', 'SourcePlugin'] import os import logging +import types from collections import defaultdict -from importlib.machinery import SourceFileLoader +import importlib +import importlib.util from wic import WicError from wic.misc import get_bitbake_var @@ -54,7 +56,9 @@ class PluginMgr: mname = fname[:-3] mpath = os.path.join(ppath, fname) logger.debug("loading plugin module %s", mpath) - SourceFileLoader(mname, mpath).load_module() + spec = importlib.util.spec_from_file_location(mname, mpath) + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) return PLUGINS.get(ptype) diff --git a/scripts/lib/wic/plugins/imager/direct.py b/scripts/lib/wic/plugins/imager/direct.py index ea709e8c545d..35fff7c10242 100644 --- a/scripts/lib/wic/plugins/imager/direct.py +++ b/scripts/lib/wic/plugins/imager/direct.py @@ -77,7 +77,8 @@ class DirectPlugin(ImagerPlugin): image_path = self._full_path(self.workdir, self.parts[0].disk, "direct") self._image = PartitionedImage(image_path, self.ptable_format, - self.parts, self.native_sysroot) + self.parts, self.native_sysroot, + options.extra_space) def setup_workdir(self, workdir): if workdir: @@ -116,7 +117,7 @@ class DirectPlugin(ImagerPlugin): updated = False for part in self.parts: if not part.realnum or not part.mountpoint \ - or part.mountpoint == "/": + or part.mountpoint == "/" or not part.mountpoint.startswith('/'): continue if part.use_uuid: @@ -258,6 +259,8 @@ class DirectPlugin(ImagerPlugin): if part.mountpoint == "/": if part.uuid: return "PARTUUID=%s" % part.uuid + elif part.label: + return "PARTLABEL=%s" % part.label else: suffix = 'p' if part.disk.startswith('mmcblk') else '' return "/dev/%s%s%-d" % (part.disk, suffix, part.realnum) @@ -293,7 +296,7 @@ class PartitionedImage(): Partitioned image in a file. """ - def __init__(self, path, ptable_format, partitions, native_sysroot=None): + def __init__(self, path, ptable_format, partitions, native_sysroot=None, extra_space=0): self.path = path # Path to the image file self.numpart = 0 # Number of allocated partitions self.realpart = 0 # Number of partitions in the partition table @@ -314,6 +317,7 @@ class PartitionedImage(): self.sector_size = SECTOR_SIZE self.native_sysroot = native_sysroot num_real_partitions = len([p for p in self.partitions if not p.no_table]) + self.extra_space = extra_space # calculate the real partition number, accounting for partitions not # in the partition table and logical partitions @@ -483,6 +487,7 @@ class PartitionedImage(): self.min_size += GPT_OVERHEAD self.min_size *= self.sector_size + self.min_size += self.extra_space def _create_partition(self, device, parttype, fstype, start, size): """ Create a partition on an image described by the 'device' object. """ diff --git a/scripts/lib/wic/plugins/source/bootimg-efi.py b/scripts/lib/wic/plugins/source/bootimg-efi.py index cdc72543c200..0391aebdc840 100644 --- a/scripts/lib/wic/plugins/source/bootimg-efi.py +++ b/scripts/lib/wic/plugins/source/bootimg-efi.py @@ -12,6 +12,7 @@ import logging import os +import tempfile import shutil import re @@ -119,12 +120,13 @@ class BootimgEFIPlugin(SourcePlugin): bootloader = creator.ks.bootloader loader_conf = "" - loader_conf += "default boot\n" + if source_params.get('create-unified-kernel-image') != "true": + loader_conf += "default boot\n" loader_conf += "timeout %d\n" % bootloader.timeout initrd = source_params.get('initrd') - if initrd: + if initrd and source_params.get('create-unified-kernel-image') != "true": # obviously we need to have a common common deploy var bootimg_dir = get_bitbake_var("DEPLOY_DIR_IMAGE") if not bootimg_dir: @@ -183,11 +185,12 @@ class BootimgEFIPlugin(SourcePlugin): for rd in initrds: boot_conf += "initrd /%s\n" % rd - logger.debug("Writing systemd-boot config " - "%s/hdd/boot/loader/entries/boot.conf", cr_workdir) - cfg = open("%s/hdd/boot/loader/entries/boot.conf" % cr_workdir, "w") - cfg.write(boot_conf) - cfg.close() + if source_params.get('create-unified-kernel-image') != "true": + logger.debug("Writing systemd-boot config " + "%s/hdd/boot/loader/entries/boot.conf", cr_workdir) + cfg = open("%s/hdd/boot/loader/entries/boot.conf" % cr_workdir, "w") + cfg.write(boot_conf) + cfg.close() @classmethod @@ -288,9 +291,60 @@ class BootimgEFIPlugin(SourcePlugin): kernel = "%s-%s.bin" % \ (get_bitbake_var("KERNEL_IMAGETYPE"), get_bitbake_var("INITRAMFS_LINK_NAME")) - install_cmd = "install -m 0644 %s/%s %s/%s" % \ - (staging_kernel_dir, kernel, hdddir, kernel) - exec_cmd(install_cmd) + if source_params.get('create-unified-kernel-image') == "true": + initrd = source_params.get('initrd') + if not initrd: + raise WicError("initrd= must be specified when create-unified-kernel-image=true, exiting") + + deploy_dir = get_bitbake_var("DEPLOY_DIR_IMAGE") + efi_stub = glob("%s/%s" % (deploy_dir, "linux*.efi.stub")) + if len(efi_stub) == 0: + raise WicError("Unified Kernel Image EFI stub not found, exiting") + efi_stub = efi_stub[0] + + with tempfile.TemporaryDirectory() as tmp_dir: + label = source_params.get('label') + label_conf = "root=%s" % creator.rootdev + if label: + label_conf = "LABEL=%s" % label + + bootloader = creator.ks.bootloader + cmdline = open("%s/cmdline" % tmp_dir, "w") + cmdline.write("%s %s" % (label_conf, bootloader.append)) + cmdline.close() + + initrds = initrd.split(';') + initrd = open("%s/initrd" % tmp_dir, "wb") + for f in initrds: + with open("%s/%s" % (deploy_dir, f), 'rb') as in_file: + shutil.copyfileobj(in_file, initrd) + initrd.close() + + # Searched by systemd-boot: + # https://systemd.io/BOOT_LOADER_SPECIFICATION/#type-2-efi-unified-kernel-images + install_cmd = "install -d %s/EFI/Linux" % hdddir + exec_cmd(install_cmd) + + staging_dir_host = get_bitbake_var("STAGING_DIR_HOST") + + # https://www.freedesktop.org/software/systemd/man/systemd-stub.html + objcopy_cmd = "objcopy \ + --add-section .osrel=%s --change-section-vma .osrel=0x20000 \ + --add-section .cmdline=%s --change-section-vma .cmdline=0x30000 \ + --add-section .linux=%s --change-section-vma .linux=0x2000000 \ + --add-section .initrd=%s --change-section-vma .initrd=0x3000000 \ + %s %s" % \ + ("%s/usr/lib/os-release" % staging_dir_host, + cmdline.name, + "%s/%s" % (staging_kernel_dir, kernel), + initrd.name, + efi_stub, + "%s/EFI/Linux/linux.efi" % hdddir) + exec_cmd(objcopy_cmd) + else: + install_cmd = "install -m 0644 %s/%s %s/%s" % \ + (staging_kernel_dir, kernel, hdddir, kernel) + exec_cmd(install_cmd) if get_bitbake_var("IMAGE_EFI_BOOT_FILES"): for src_path, dst_path in cls.install_task: diff --git a/scripts/lib/wic/plugins/source/bootimg-pcbios.py b/scripts/lib/wic/plugins/source/bootimg-pcbios.py index f2639e700493..32e47f183146 100644 --- a/scripts/lib/wic/plugins/source/bootimg-pcbios.py +++ b/scripts/lib/wic/plugins/source/bootimg-pcbios.py @@ -186,8 +186,10 @@ class BootimgPcbiosPlugin(SourcePlugin): # dosfs image, created by mkdosfs bootimg = "%s/boot%s.img" % (cr_workdir, part.lineno) - dosfs_cmd = "mkdosfs -n boot -i %s -S 512 -C %s %d" % \ - (part.fsuuid, bootimg, blocks) + label = part.label if part.label else "boot" + + dosfs_cmd = "mkdosfs -n %s -i %s -S 512 -C %s %d" % \ + (label, part.fsuuid, bootimg, blocks) exec_native_cmd(dosfs_cmd, native_sysroot) mcopy_cmd = "mcopy -i %s -s %s/* ::/" % (bootimg, hdddir) diff --git a/scripts/lib/wic/plugins/source/rawcopy.py b/scripts/lib/wic/plugins/source/rawcopy.py index 3c4997d8ba5e..7c90cd3cf82b 100644 --- a/scripts/lib/wic/plugins/source/rawcopy.py +++ b/scripts/lib/wic/plugins/source/rawcopy.py @@ -4,6 +4,8 @@ import logging import os +import signal +import subprocess from wic import WicError from wic.pluginbase import SourcePlugin @@ -29,15 +31,34 @@ class RawCopyPlugin(SourcePlugin): cmd = 'btrfs filesystem label %s %s' % (dst, label) elif fstype == 'swap': cmd = 'mkswap -L %s %s' % (label, dst) - elif fstype == 'squashfs': - raise WicError("It's not possible to update a squashfs " - "filesystem label '%s'" % (label)) + elif fstype in ('squashfs', 'erofs'): + raise WicError("It's not possible to update a %s " + "filesystem label '%s'" % (fstype, label)) else: raise WicError("Cannot update filesystem label: " "Unknown fstype: '%s'" % (fstype)) exec_cmd(cmd) + @staticmethod + def do_image_uncompression(src, dst, workdir): + def subprocess_setup(): + # Python installs a SIGPIPE handler by default. This is usually not what + # non-Python subprocesses expect. + # SIGPIPE errors are known issues with gzip/bash + signal.signal(signal.SIGPIPE, signal.SIG_DFL) + + extension = os.path.splitext(src)[1] + decompressor = { + ".bz2": "bzip2", + ".gz": "gzip", + ".xz": "xz" + }.get(extension) + if not decompressor: + raise WicError("Not supported compressor filename extension: %s" % extension) + cmd = "%s -dc %s > %s" % (decompressor, src, dst) + subprocess.call(cmd, preexec_fn=subprocess_setup, shell=True, cwd=workdir) + @classmethod def do_prepare_partition(cls, part, source_params, cr, cr_workdir, oe_builddir, bootimg_dir, kernel_dir, @@ -56,7 +77,13 @@ class RawCopyPlugin(SourcePlugin): if 'file' not in source_params: raise WicError("No file specified") - src = os.path.join(kernel_dir, source_params['file']) + if 'unpack' in source_params: + img = os.path.join(kernel_dir, source_params['file']) + src = os.path.join(cr_workdir, os.path.splitext(source_params['file'])[0]) + RawCopyPlugin.do_image_uncompression(img, src, cr_workdir) + else: + src = os.path.join(kernel_dir, source_params['file']) + dst = os.path.join(cr_workdir, "%s.%s" % (os.path.basename(source_params['file']), part.lineno)) if not os.path.exists(os.path.dirname(dst)): diff --git a/scripts/lib/wic/plugins/source/rootfs.py b/scripts/lib/wic/plugins/source/rootfs.py index 96d940a91d61..2e34e715ca73 100644 --- a/scripts/lib/wic/plugins/source/rootfs.py +++ b/scripts/lib/wic/plugins/source/rootfs.py @@ -218,7 +218,7 @@ class RootfsPlugin(SourcePlugin): # Update part.has_fstab here as fstab may have been added or # removed by the above modifications. part.has_fstab = os.path.exists(os.path.join(new_rootfs, "etc/fstab")) - if part.update_fstab_in_rootfs and part.has_fstab: + if part.update_fstab_in_rootfs and part.has_fstab and not part.no_fstab_update: fstab_path = os.path.join(new_rootfs, "etc/fstab") # Assume that fstab should always be owned by root with fixed permissions install_cmd = "install -m 0644 %s %s" % (part.updated_fstab_path, fstab_path) diff --git a/scripts/wic b/scripts/wic index a741aed364bf..aee63a45aacf 100755 --- a/scripts/wic +++ b/scripts/wic @@ -22,9 +22,9 @@ import sys import argparse import logging import subprocess +import shutil from collections import namedtuple -from distutils import spawn # External modules scripts_path = os.path.dirname(os.path.realpath(__file__)) @@ -47,7 +47,7 @@ if os.environ.get('SDKTARGETSYSROOT'): break sdkroot = os.path.dirname(sdkroot) -bitbake_exe = spawn.find_executable('bitbake') +bitbake_exe = shutil.which('bitbake') if bitbake_exe: bitbake_path = scriptpath.add_bitbake_lib_path() import bb @@ -159,6 +159,9 @@ def wic_create_subcommand(options, usage_str): "(Use -e/--image-name to specify it)") native_sysroot = options.native_sysroot + if options.kernel_dir: + kernel_dir = options.kernel_dir + if not options.vars_dir and (not native_sysroot or not os.path.isdir(native_sysroot)): logger.info("Building wic-tools...\n") subprocess.check_call(["bitbake", "wic-tools"]) @@ -346,6 +349,8 @@ def wic_init_parser_create(subparser): default=False, help="output debug information") subparser.add_argument("-i", "--imager", dest="imager", default="direct", help="the wic imager plugin") + subparser.add_argument("--extra-space", type=int, dest="extra_space", + default=0, help="additional free disk space to add to the image") return -- 2.35.1 ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH v2 2/2] wic: Update to the latest revision 2022-04-21 8:52 ` [PATCH v2 2/2] wic: Update to the latest revision Henning Schild @ 2022-04-21 9:02 ` Henning Schild 0 siblings, 0 replies; 22+ messages in thread From: Henning Schild @ 2022-04-21 9:02 UTC (permalink / raw) To: isar-users; +Cc: Florian Bezdeka, Felix Moessbauer Am Thu, 21 Apr 2022 10:52:32 +0200 schrieb Henning Schild <henning.schild@siemens.com>: > Update to the latest wic from OE-core. > > OE-core Revision: 712552b5cc427d7be1258c45886de9b57f7272c9 > > Signed-off-by: Henning Schild <henning.schild@siemens.com> > --- > .../wic/plugins/source/bootimg-efi-isar.py | 77 > ++++++++++++++++--- .../wic/plugins/source/bootimg-pcbios-isar.py | > 6 +- scripts/lib/wic/canned-wks/common.wks.inc | 2 +- > scripts/lib/wic/canned-wks/directdisk-gpt.wks | 2 +- > scripts/lib/wic/canned-wks/mkefidisk.wks | 2 +- > scripts/lib/wic/engine.py | 6 +- > scripts/lib/wic/help.py | 10 ++- > scripts/lib/wic/ksparser.py | 8 +- > scripts/lib/wic/misc.py | 4 +- > scripts/lib/wic/partition.py | 25 ++++-- > scripts/lib/wic/pluginbase.py | 8 +- > scripts/lib/wic/plugins/imager/direct.py | 11 ++- > scripts/lib/wic/plugins/source/bootimg-efi.py | 74 +++++++++++++++--- > .../lib/wic/plugins/source/bootimg-pcbios.py | 6 +- > scripts/lib/wic/plugins/source/rawcopy.py | 35 ++++++++- > scripts/lib/wic/plugins/source/rootfs.py | 2 +- > scripts/wic | 9 ++- > 17 files changed, 231 insertions(+), 56 deletions(-) > > diff --git a/meta/scripts/lib/wic/plugins/source/bootimg-efi-isar.py > b/meta/scripts/lib/wic/plugins/source/bootimg-efi-isar.py index > 5ba0777ad244..a24e04f309da 100644 --- > a/meta/scripts/lib/wic/plugins/source/bootimg-efi-isar.py +++ > b/meta/scripts/lib/wic/plugins/source/bootimg-efi-isar.py @@ -12,6 > +12,7 @@ > import logging > import os > +import tempfile > import shutil > import re > > @@ -129,12 +130,13 @@ class BootimgEFIPlugin(SourcePlugin): > bootloader = creator.ks.bootloader > > loader_conf = "" > - loader_conf += "default boot\n" > + if source_params.get('create-unified-kernel-image') != > "true": > + loader_conf += "default boot\n" > loader_conf += "timeout %d\n" % bootloader.timeout > > initrd = source_params.get('initrd') > > - if initrd: > + if initrd and > source_params.get('create-unified-kernel-image') != "true": # > obviously we need to have a common common deploy var bootimg_dir = > get_bitbake_var("DEPLOY_DIR_IMAGE") if not bootimg_dir: > @@ -195,11 +197,12 @@ class BootimgEFIPlugin(SourcePlugin): > for rd in initrds: > boot_conf += "initrd /%s\n" % rd > > - logger.debug("Writing systemd-boot config " > - "%s/hdd/boot/loader/entries/boot.conf", > cr_workdir) > - cfg = open("%s/hdd/boot/loader/entries/boot.conf" % > cr_workdir, "w") > - cfg.write(boot_conf) > - cfg.close() > + if source_params.get('create-unified-kernel-image') != > "true": > + logger.debug("Writing systemd-boot config " > + "%s/hdd/boot/loader/entries/boot.conf", > cr_workdir) > + cfg = open("%s/hdd/boot/loader/entries/boot.conf" % > cr_workdir, "w") > + cfg.write(boot_conf) > + cfg.close() > > > @classmethod > @@ -300,11 +303,63 @@ class BootimgEFIPlugin(SourcePlugin): > kernel = "%s-%s.bin" % \ > (get_bitbake_var("KERNEL_IMAGETYPE"), > get_bitbake_var("INITRAMFS_LINK_NAME")) > - install_cmd = "install -m 0644 %s/%s %s/%s" % \ > - (staging_kernel_dir, kernel, hdddir, kernel) > + if source_params.get('create-unified-kernel-image') == > "true": > + initrd = source_params.get('initrd') > + if not initrd: > + raise WicError("initrd= must be specified when > create-unified-kernel-image=true, exiting") + > + deploy_dir = get_bitbake_var("DEPLOY_DIR_IMAGE") > + efi_stub = glob("%s/%s" % (deploy_dir, > "linux*.efi.stub")) > + if len(efi_stub) == 0: > + raise WicError("Unified Kernel Image EFI stub not > found, exiting") > + efi_stub = efi_stub[0] > + > + with tempfile.TemporaryDirectory() as tmp_dir: > + label = source_params.get('label') > + label_conf = "root=%s" % creator.rootdev > + if label: > + label_conf = "LABEL=%s" % label > + > + bootloader = creator.ks.bootloader > + cmdline = open("%s/cmdline" % tmp_dir, "w") > + cmdline.write("%s %s" % (label_conf, > bootloader.append)) > + cmdline.close() > + > + initrds = initrd.split(';') > + initrd = open("%s/initrd" % tmp_dir, "wb") > + for f in initrds: > + with open("%s/%s" % (deploy_dir, f), 'rb') as > in_file: > + shutil.copyfileobj(in_file, initrd) > + initrd.close() > + > + # Searched by systemd-boot: > + # > https://systemd.io/BOOT_LOADER_SPECIFICATION/#type-2-efi-unified-kernel-images > + install_cmd = "install -d %s/EFI/Linux" % hdddir > + exec_cmd(install_cmd) > + > + staging_dir_host = > get_bitbake_var("STAGING_DIR_HOST") + > + # > https://www.freedesktop.org/software/systemd/man/systemd-stub.html > + objcopy_cmd = "objcopy \ > + --add-section .osrel=%s --change-section-vma > .osrel=0x20000 \ > + --add-section .cmdline=%s --change-section-vma > .cmdline=0x30000 \ > + --add-section .linux=%s --change-section-vma > .linux=0x2000000 \ > + --add-section .initrd=%s --change-section-vma > .initrd=0x3000000 \ > + %s %s" % \ > + ("%s/usr/lib/os-release" % staging_dir_host, > + cmdline.name, > + "%s/%s" % (staging_kernel_dir, kernel), > + initrd.name, > + efi_stub, > + "%s/EFI/Linux/linux.efi" % hdddir) > + exec_cmd(objcopy_cmd) > + else: > + install_cmd = "install -m 0644 %s/%s %s/%s" % \ > + (staging_kernel_dir, kernel, hdddir, kernel) > + > + install_cmd = > isar_populate_boot_cmd(rootfs_dir['ROOTFS_DIR'], hdddir) > + exec_cmd(install_cmd) > > - install_cmd = > isar_populate_boot_cmd(rootfs_dir['ROOTFS_DIR'], hdddir) > - exec_cmd(install_cmd) > > if get_bitbake_var("IMAGE_EFI_BOOT_FILES"): > for src_path, dst_path in cls.install_task: > diff --git > a/meta/scripts/lib/wic/plugins/source/bootimg-pcbios-isar.py > b/meta/scripts/lib/wic/plugins/source/bootimg-pcbios-isar.py index > 9136d4f215ca..7fb0f2a87394 100644 --- > a/meta/scripts/lib/wic/plugins/source/bootimg-pcbios-isar.py +++ > b/meta/scripts/lib/wic/plugins/source/bootimg-pcbios-isar.py @@ > -213,8 +213,10 @@ class BootimgPcbiosIsarPlugin(SourcePlugin): # > dosfs image, created by mkdosfs bootimg = "%s/boot%s.img" % > (cr_workdir, part.lineno) > - dosfs_cmd = "mkdosfs -n boot -i %s -S 512 -C %s %d" % \ > - (part.fsuuid, bootimg, blocks) > + label = part.label if part.label else "boot" > + > + dosfs_cmd = "mkdosfs -n %s -i %s -S 512 -C %s %d" % \ > + (label, part.fsuuid, bootimg, blocks) > exec_native_cmd(dosfs_cmd, native_sysroot) > > mcopy_cmd = "mcopy -i %s -s %s/* ::/" % (bootimg, hdddir) > diff --git a/scripts/lib/wic/canned-wks/common.wks.inc > b/scripts/lib/wic/canned-wks/common.wks.inc index > 4fd29fa8c119..89880b417b6e 100644 --- > a/scripts/lib/wic/canned-wks/common.wks.inc +++ > b/scripts/lib/wic/canned-wks/common.wks.inc @@ -1,3 +1,3 @@ > # This file is included into 3 canned wks files from this directory > part /boot --source bootimg-pcbios --ondisk sda --label boot > --active --align 1024 -part / --source rootfs --use-uuid > --fstype=ext4 --mkfs-extraopts "-T default" --label platform --align > 1024 +part / --source rootfs --use-uuid --fstype=ext4 --label > platform --align 1024 diff --git This is https://github.com/openembedded/openembedded-core/commit/7e8017208bed98b6c90735cb641fc9d7aedf9140 and it was not taken for our forks in meta-isar/scripts/lib/wic/canned-wks/ Because we can not be sure to have a brandnew e2fsprogs always. With scripts/wic_fakeroot we have a way to centrally hook into wic plugins and play with arguments. But i think having our forked wks files is nicer and we might eventually be able to drop wic_fakeroot, at the moment it only deals with one remaining hack for old e2fsck Henning > a/scripts/lib/wic/canned-wks/directdisk-gpt.wks > b/scripts/lib/wic/canned-wks/directdisk-gpt.wks index > cf16c0c30bbd..8d7d8de6ea7e 100644 --- > a/scripts/lib/wic/canned-wks/directdisk-gpt.wks +++ > b/scripts/lib/wic/canned-wks/directdisk-gpt.wks @@ -4,7 +4,7 @@ > part /boot --source bootimg-pcbios --ondisk sda --label boot > --active --align 1024 -part / --source rootfs --ondisk sda > --fstype=ext4 --mkfs-extraopts "-T default" --label platform --align > 1024 --use-uuid +part / --source rootfs --ondisk sda --fstype=ext4 > --label platform --align 1024 --use-uuid bootloader --ptable gpt > --timeout=0 --append="rootwait rootfstype=ext4 video=vesafb > vga=0x318 console=tty0 console=ttyS0,115200n8" diff --git > a/scripts/lib/wic/canned-wks/mkefidisk.wks > b/scripts/lib/wic/canned-wks/mkefidisk.wks index > d1878e23e5a3..9f534fe18471 100644 --- > a/scripts/lib/wic/canned-wks/mkefidisk.wks +++ > b/scripts/lib/wic/canned-wks/mkefidisk.wks @@ -4,7 +4,7 @@ > part /boot --source bootimg-efi --sourceparams="loader=grub-efi" > --ondisk sda --label msdos --active --align 1024 > -part / --source rootfs --ondisk sda --fstype=ext4 --mkfs-extraopts > "-T default" --label platform --align 1024 --use-uuid +part / > --source rootfs --ondisk sda --fstype=ext4 --label platform --align > 1024 --use-uuid part swap --ondisk sda --size 44 --label swap1 > --fstype=swap > diff --git a/scripts/lib/wic/engine.py b/scripts/lib/wic/engine.py > index 018815b96688..674ccfc24418 100644 > --- a/scripts/lib/wic/engine.py > +++ b/scripts/lib/wic/engine.py > @@ -19,10 +19,10 @@ import os > import tempfile > import json > import subprocess > +import shutil > import re > > from collections import namedtuple, OrderedDict > -from distutils.spawn import find_executable > > from wic import WicError > from wic.filemap import sparse_copy > @@ -245,7 +245,7 @@ class Disk: > for path in pathlist.split(':'): > self.paths = "%s%s:%s" % (native_sysroot, path, > self.paths) > - self.parted = find_executable("parted", self.paths) > + self.parted = shutil.which("parted", path=self.paths) > if not self.parted: > raise WicError("Can't find executable parted") > > @@ -283,7 +283,7 @@ class Disk: > "resize2fs", "mkswap", "mkdosfs", > "debugfs","blkid"): aname = "_%s" % name > if aname not in self.__dict__: > - setattr(self, aname, find_executable(name, > self.paths)) > + setattr(self, aname, shutil.which(name, > path=self.paths)) if aname not in self.__dict__ or > self.__dict__[aname] is None: raise WicError("Can't find executable > '{}'".format(name)) return self.__dict__[aname] > diff --git a/scripts/lib/wic/help.py b/scripts/lib/wic/help.py > index bd3a2b97dfaf..4ff7470a6a79 100644 > --- a/scripts/lib/wic/help.py > +++ b/scripts/lib/wic/help.py > @@ -637,7 +637,7 @@ DESCRIPTION > oe-core: directdisk.bbclass and mkefidisk.sh. The difference > between wic and those examples is that with wic the functionality > of those scripts is implemented by a general-purpose partitioning > - 'language' based on Redhat kickstart syntax). > + 'language' based on Red Hat kickstart syntax). > > The initial motivation and design considerations that lead to the > current tool are described exhaustively in Yocto Bug #3847 > @@ -840,8 +840,8 @@ DESCRIPTION > meanings. The commands are based on the Fedora kickstart > documentation but with modifications to reflect wic capabilities. > > - > http://fedoraproject.org/wiki/Anaconda/Kickstart#part_or_partition > - http://fedoraproject.org/wiki/Anaconda/Kickstart#bootloader > + > https://pykickstart.readthedocs.io/en/latest/kickstart-docs.html#part-or-partition > + > https://pykickstart.readthedocs.io/en/latest/kickstart-docs.html#bootloader > Commands > > @@ -930,6 +930,7 @@ DESCRIPTION > ext4 > btrfs > squashfs > + erofs > swap > > --fsoptions: Specifies a free-form string of options to be > @@ -990,6 +991,9 @@ DESCRIPTION > multiple partitions and we want to keep > the right permissions and usernames in all the partitions. > > + --no-fstab-update: This option is specific to wic. It does > not update the > + '/etc/fstab' stock file for the given > partition. + > --extra-space: This option is specific to wic. It adds extra > space after the space filled by the content > of the partition. The final size can go > diff --git a/scripts/lib/wic/ksparser.py b/scripts/lib/wic/ksparser.py > index 3eb669da39ca..0df9eb0d057d 100644 > --- a/scripts/lib/wic/ksparser.py > +++ b/scripts/lib/wic/ksparser.py > @@ -157,7 +157,8 @@ class KickStart(): > part.add_argument('--fsoptions', dest='fsopts') > part.add_argument('--fstype', default='vfat', > choices=('ext2', 'ext3', 'ext4', 'btrfs', > - 'squashfs', 'vfat', 'msdos', > 'swap')) > + 'squashfs', 'vfat', 'msdos', > 'erofs', > + 'swap')) > part.add_argument('--mkfs-extraopts', default='') > part.add_argument('--label') > part.add_argument('--use-label', action='store_true') > @@ -184,6 +185,7 @@ class KickStart(): > part.add_argument('--use-uuid', action='store_true') > part.add_argument('--uuid') > part.add_argument('--fsuuid') > + part.add_argument('--no-fstab-update', action='store_true') > > bootloader = subparsers.add_parser('bootloader') > bootloader.add_argument('--append') > @@ -229,6 +231,10 @@ class KickStart(): > err = "%s:%d: SquashFS does not > support LABEL" \ % (confpath, lineno) > raise KickStartError(err) > + # erofs does not support filesystem labels > + if parsed.fstype == 'erofs' and parsed.label: > + err = "%s:%d: erofs does not support > LABEL" % (confpath, lineno) > + raise KickStartError(err) > if parsed.fstype == 'msdos' or parsed.fstype > == 'vfat': if parsed.fsuuid: > if > parsed.fsuuid.upper().startswith('0X'): diff --git > a/scripts/lib/wic/misc.py b/scripts/lib/wic/misc.py index > 57c042c503e6..3e118229960b 100644 --- a/scripts/lib/wic/misc.py > +++ b/scripts/lib/wic/misc.py > @@ -16,9 +16,9 @@ import logging > import os > import re > import subprocess > +import shutil > > from collections import defaultdict > -from distutils import spawn > > from wic import WicError > > @@ -122,7 +122,7 @@ def find_executable(cmd, paths): > if provided and "%s-native" % recipe in provided: > return True > > - return spawn.find_executable(cmd, paths) > + return shutil.which(cmd, path=paths) > > def exec_native_cmd(cmd_and_args, native_sysroot, pseudo=""): > """ > diff --git a/scripts/lib/wic/partition.py > b/scripts/lib/wic/partition.py index 76d144d12d30..09e491dd494d 100644 > --- a/scripts/lib/wic/partition.py > +++ b/scripts/lib/wic/partition.py > @@ -54,6 +54,7 @@ class Partition(): > self.uuid = args.uuid > self.fsuuid = args.fsuuid > self.type = args.type > + self.no_fstab_update = args.no_fstab_update > self.updated_fstab_path = None > self.has_fstab = False > self.update_fstab_in_rootfs = False > @@ -104,7 +105,7 @@ class Partition(): > extra_blocks = self.extra_space > > rootfs_size = actual_rootfs_size + extra_blocks > - rootfs_size *= self.overhead_factor > + rootfs_size = int(rootfs_size * self.overhead_factor) > > logger.debug("Added %d extra blocks to %s to get to %d > total blocks", extra_blocks, self.mountpoint, rootfs_size) > @@ -141,9 +142,9 @@ class Partition(): > native_sysroot) > self.source_file = "%s/fs.%s" % (cr_workdir, > self.fstype) else: > - if self.fstype == 'squashfs': > - raise WicError("It's not possible to create > empty squashfs " > - "partition '%s'" % > (self.mountpoint)) > + if self.fstype in ('squashfs', 'erofs'): > + raise WicError("It's not possible to create > empty %s " > + "partition '%s'" % (self.fstype, > self.mountpoint)) > rootfs = "%s/fs_%s.%s.%s" % (cr_workdir, self.label, > self.lineno, > self.fstype) @@ -170,7 +171,7 @@ class Partition(): > # Split sourceparams string of the form > key1=val1[,key2=val2,...] # into a dict. Also accepts valueless keys > i.e. without = splitted = self.sourceparams.split(',') > - srcparams_dict = dict(par.split('=', 1) for par in > splitted if par) > + srcparams_dict = dict((par.split('=', 1) + [None])[:2] > for par in splitted if par) > plugin = PluginMgr.get_plugins('source')[self.source] > plugin.do_configure_partition(self, srcparams_dict, creator, > @@ -286,7 +287,7 @@ class Partition(): > (self.fstype, extraopts, rootfs, label_str, self.fsuuid, > rootfs_dir) exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo) > > - if self.updated_fstab_path and self.has_fstab: > + if self.updated_fstab_path and self.has_fstab and not > self.no_fstab_update: debugfs_script_path = os.path.join(cr_workdir, > "debugfs_script") with open(debugfs_script_path, "w") as f: > f.write("cd etc\n") > @@ -350,7 +351,7 @@ class Partition(): > mcopy_cmd = "mcopy -i %s -s %s/* ::/" % (rootfs, rootfs_dir) > exec_native_cmd(mcopy_cmd, native_sysroot) > > - if self.updated_fstab_path and self.has_fstab: > + if self.updated_fstab_path and self.has_fstab and not > self.no_fstab_update: mcopy_cmd = "mcopy -i %s %s ::/etc/fstab" % > (rootfs, self.updated_fstab_path) exec_native_cmd(mcopy_cmd, > native_sysroot) > @@ -369,6 +370,16 @@ class Partition(): > (rootfs_dir, rootfs, extraopts) > exec_native_cmd(squashfs_cmd, native_sysroot, pseudo=pseudo) > > + def prepare_rootfs_erofs(self, rootfs, cr_workdir, oe_builddir, > rootfs_dir, > + native_sysroot, pseudo): > + """ > + Prepare content for a erofs rootfs partition. > + """ > + extraopts = self.mkfs_extraopts or '' > + erofs_cmd = "mkfs.erofs %s -U %s %s %s" % \ > + (extraopts, self.fsuuid, rootfs, rootfs_dir) > + exec_native_cmd(erofs_cmd, native_sysroot, pseudo=pseudo) > + > def prepare_empty_partition_ext(self, rootfs, oe_builddir, > native_sysroot): > """ > diff --git a/scripts/lib/wic/pluginbase.py > b/scripts/lib/wic/pluginbase.py index d9b4e57747e7..b64568339b1b > 100644 --- a/scripts/lib/wic/pluginbase.py > +++ b/scripts/lib/wic/pluginbase.py > @@ -9,9 +9,11 @@ __all__ = ['ImagerPlugin', 'SourcePlugin'] > > import os > import logging > +import types > > from collections import defaultdict > -from importlib.machinery import SourceFileLoader > +import importlib > +import importlib.util > > from wic import WicError > from wic.misc import get_bitbake_var > @@ -54,7 +56,9 @@ class PluginMgr: > mname = fname[:-3] > mpath = os.path.join(ppath, fname) > logger.debug("loading plugin module %s", > mpath) > - SourceFileLoader(mname, > mpath).load_module() > + spec = > importlib.util.spec_from_file_location(mname, mpath) > + module = > importlib.util.module_from_spec(spec) > + spec.loader.exec_module(module) > > return PLUGINS.get(ptype) > > diff --git a/scripts/lib/wic/plugins/imager/direct.py > b/scripts/lib/wic/plugins/imager/direct.py index > ea709e8c545d..35fff7c10242 100644 --- > a/scripts/lib/wic/plugins/imager/direct.py +++ > b/scripts/lib/wic/plugins/imager/direct.py @@ -77,7 +77,8 @@ class > DirectPlugin(ImagerPlugin): > image_path = self._full_path(self.workdir, > self.parts[0].disk, "direct") self._image = > PartitionedImage(image_path, self.ptable_format, > - self.parts, > self.native_sysroot) > + self.parts, > self.native_sysroot, > + options.extra_space) > > def setup_workdir(self, workdir): > if workdir: > @@ -116,7 +117,7 @@ class DirectPlugin(ImagerPlugin): > updated = False > for part in self.parts: > if not part.realnum or not part.mountpoint \ > - or part.mountpoint == "/": > + or part.mountpoint == "/" or not > part.mountpoint.startswith('/'): continue > > if part.use_uuid: > @@ -258,6 +259,8 @@ class DirectPlugin(ImagerPlugin): > if part.mountpoint == "/": > if part.uuid: > return "PARTUUID=%s" % part.uuid > + elif part.label: > + return "PARTLABEL=%s" % part.label > else: > suffix = 'p' if part.disk.startswith('mmcblk') > else '' return "/dev/%s%s%-d" % (part.disk, suffix, part.realnum) > @@ -293,7 +296,7 @@ class PartitionedImage(): > Partitioned image in a file. > """ > > - def __init__(self, path, ptable_format, partitions, > native_sysroot=None): > + def __init__(self, path, ptable_format, partitions, > native_sysroot=None, extra_space=0): self.path = path # Path to the > image file self.numpart = 0 # Number of allocated partitions > self.realpart = 0 # Number of partitions in the partition > table @@ -314,6 +317,7 @@ class PartitionedImage(): > self.sector_size = SECTOR_SIZE > self.native_sysroot = native_sysroot > num_real_partitions = len([p for p in self.partitions if not > p.no_table]) > + self.extra_space = extra_space > > # calculate the real partition number, accounting for > partitions not # in the partition table and logical partitions > @@ -483,6 +487,7 @@ class PartitionedImage(): > self.min_size += GPT_OVERHEAD > > self.min_size *= self.sector_size > + self.min_size += self.extra_space > > def _create_partition(self, device, parttype, fstype, start, > size): """ Create a partition on an image described by the 'device' > object. """ diff --git > a/scripts/lib/wic/plugins/source/bootimg-efi.py > b/scripts/lib/wic/plugins/source/bootimg-efi.py index > cdc72543c200..0391aebdc840 100644 --- > a/scripts/lib/wic/plugins/source/bootimg-efi.py +++ > b/scripts/lib/wic/plugins/source/bootimg-efi.py @@ -12,6 +12,7 @@ > import logging > import os > +import tempfile > import shutil > import re > > @@ -119,12 +120,13 @@ class BootimgEFIPlugin(SourcePlugin): > bootloader = creator.ks.bootloader > > loader_conf = "" > - loader_conf += "default boot\n" > + if source_params.get('create-unified-kernel-image') != > "true": > + loader_conf += "default boot\n" > loader_conf += "timeout %d\n" % bootloader.timeout > > initrd = source_params.get('initrd') > > - if initrd: > + if initrd and > source_params.get('create-unified-kernel-image') != "true": # > obviously we need to have a common common deploy var bootimg_dir = > get_bitbake_var("DEPLOY_DIR_IMAGE") if not bootimg_dir: > @@ -183,11 +185,12 @@ class BootimgEFIPlugin(SourcePlugin): > for rd in initrds: > boot_conf += "initrd /%s\n" % rd > > - logger.debug("Writing systemd-boot config " > - "%s/hdd/boot/loader/entries/boot.conf", > cr_workdir) > - cfg = open("%s/hdd/boot/loader/entries/boot.conf" % > cr_workdir, "w") > - cfg.write(boot_conf) > - cfg.close() > + if source_params.get('create-unified-kernel-image') != > "true": > + logger.debug("Writing systemd-boot config " > + "%s/hdd/boot/loader/entries/boot.conf", > cr_workdir) > + cfg = open("%s/hdd/boot/loader/entries/boot.conf" % > cr_workdir, "w") > + cfg.write(boot_conf) > + cfg.close() > > > @classmethod > @@ -288,9 +291,60 @@ class BootimgEFIPlugin(SourcePlugin): > kernel = "%s-%s.bin" % \ > (get_bitbake_var("KERNEL_IMAGETYPE"), > get_bitbake_var("INITRAMFS_LINK_NAME")) > - install_cmd = "install -m 0644 %s/%s %s/%s" % \ > - (staging_kernel_dir, kernel, hdddir, kernel) > - exec_cmd(install_cmd) > + if source_params.get('create-unified-kernel-image') == > "true": > + initrd = source_params.get('initrd') > + if not initrd: > + raise WicError("initrd= must be specified when > create-unified-kernel-image=true, exiting") + > + deploy_dir = get_bitbake_var("DEPLOY_DIR_IMAGE") > + efi_stub = glob("%s/%s" % (deploy_dir, > "linux*.efi.stub")) > + if len(efi_stub) == 0: > + raise WicError("Unified Kernel Image EFI stub not > found, exiting") > + efi_stub = efi_stub[0] > + > + with tempfile.TemporaryDirectory() as tmp_dir: > + label = source_params.get('label') > + label_conf = "root=%s" % creator.rootdev > + if label: > + label_conf = "LABEL=%s" % label > + > + bootloader = creator.ks.bootloader > + cmdline = open("%s/cmdline" % tmp_dir, "w") > + cmdline.write("%s %s" % (label_conf, > bootloader.append)) > + cmdline.close() > + > + initrds = initrd.split(';') > + initrd = open("%s/initrd" % tmp_dir, "wb") > + for f in initrds: > + with open("%s/%s" % (deploy_dir, f), 'rb') as > in_file: > + shutil.copyfileobj(in_file, initrd) > + initrd.close() > + > + # Searched by systemd-boot: > + # > https://systemd.io/BOOT_LOADER_SPECIFICATION/#type-2-efi-unified-kernel-images > + install_cmd = "install -d %s/EFI/Linux" % hdddir > + exec_cmd(install_cmd) > + > + staging_dir_host = > get_bitbake_var("STAGING_DIR_HOST") + > + # > https://www.freedesktop.org/software/systemd/man/systemd-stub.html > + objcopy_cmd = "objcopy \ > + --add-section .osrel=%s --change-section-vma > .osrel=0x20000 \ > + --add-section .cmdline=%s --change-section-vma > .cmdline=0x30000 \ > + --add-section .linux=%s --change-section-vma > .linux=0x2000000 \ > + --add-section .initrd=%s --change-section-vma > .initrd=0x3000000 \ > + %s %s" % \ > + ("%s/usr/lib/os-release" % staging_dir_host, > + cmdline.name, > + "%s/%s" % (staging_kernel_dir, kernel), > + initrd.name, > + efi_stub, > + "%s/EFI/Linux/linux.efi" % hdddir) > + exec_cmd(objcopy_cmd) > + else: > + install_cmd = "install -m 0644 %s/%s %s/%s" % \ > + (staging_kernel_dir, kernel, hdddir, kernel) > + exec_cmd(install_cmd) > > if get_bitbake_var("IMAGE_EFI_BOOT_FILES"): > for src_path, dst_path in cls.install_task: > diff --git a/scripts/lib/wic/plugins/source/bootimg-pcbios.py > b/scripts/lib/wic/plugins/source/bootimg-pcbios.py index > f2639e700493..32e47f183146 100644 --- > a/scripts/lib/wic/plugins/source/bootimg-pcbios.py +++ > b/scripts/lib/wic/plugins/source/bootimg-pcbios.py @@ -186,8 +186,10 > @@ class BootimgPcbiosPlugin(SourcePlugin): # dosfs image, created by > mkdosfs bootimg = "%s/boot%s.img" % (cr_workdir, part.lineno) > > - dosfs_cmd = "mkdosfs -n boot -i %s -S 512 -C %s %d" % \ > - (part.fsuuid, bootimg, blocks) > + label = part.label if part.label else "boot" > + > + dosfs_cmd = "mkdosfs -n %s -i %s -S 512 -C %s %d" % \ > + (label, part.fsuuid, bootimg, blocks) > exec_native_cmd(dosfs_cmd, native_sysroot) > > mcopy_cmd = "mcopy -i %s -s %s/* ::/" % (bootimg, hdddir) > diff --git a/scripts/lib/wic/plugins/source/rawcopy.py > b/scripts/lib/wic/plugins/source/rawcopy.py index > 3c4997d8ba5e..7c90cd3cf82b 100644 --- > a/scripts/lib/wic/plugins/source/rawcopy.py +++ > b/scripts/lib/wic/plugins/source/rawcopy.py @@ -4,6 +4,8 @@ > > import logging > import os > +import signal > +import subprocess > > from wic import WicError > from wic.pluginbase import SourcePlugin > @@ -29,15 +31,34 @@ class RawCopyPlugin(SourcePlugin): > cmd = 'btrfs filesystem label %s %s' % (dst, label) > elif fstype == 'swap': > cmd = 'mkswap -L %s %s' % (label, dst) > - elif fstype == 'squashfs': > - raise WicError("It's not possible to update a squashfs " > - "filesystem label '%s'" % (label)) > + elif fstype in ('squashfs', 'erofs'): > + raise WicError("It's not possible to update a %s " > + "filesystem label '%s'" % (fstype, label)) > else: > raise WicError("Cannot update filesystem label: " > "Unknown fstype: '%s'" % (fstype)) > > exec_cmd(cmd) > > + @staticmethod > + def do_image_uncompression(src, dst, workdir): > + def subprocess_setup(): > + # Python installs a SIGPIPE handler by default. This is > usually not what > + # non-Python subprocesses expect. > + # SIGPIPE errors are known issues with gzip/bash > + signal.signal(signal.SIGPIPE, signal.SIG_DFL) > + > + extension = os.path.splitext(src)[1] > + decompressor = { > + ".bz2": "bzip2", > + ".gz": "gzip", > + ".xz": "xz" > + }.get(extension) > + if not decompressor: > + raise WicError("Not supported compressor filename > extension: %s" % extension) > + cmd = "%s -dc %s > %s" % (decompressor, src, dst) > + subprocess.call(cmd, preexec_fn=subprocess_setup, > shell=True, cwd=workdir) + > @classmethod > def do_prepare_partition(cls, part, source_params, cr, > cr_workdir, oe_builddir, bootimg_dir, kernel_dir, > @@ -56,7 +77,13 @@ class RawCopyPlugin(SourcePlugin): > if 'file' not in source_params: > raise WicError("No file specified") > > - src = os.path.join(kernel_dir, source_params['file']) > + if 'unpack' in source_params: > + img = os.path.join(kernel_dir, source_params['file']) > + src = os.path.join(cr_workdir, > os.path.splitext(source_params['file'])[0]) > + RawCopyPlugin.do_image_uncompression(img, src, > cr_workdir) > + else: > + src = os.path.join(kernel_dir, source_params['file']) > + > dst = os.path.join(cr_workdir, "%s.%s" % > (os.path.basename(source_params['file']), part.lineno)) > if not os.path.exists(os.path.dirname(dst)): > diff --git a/scripts/lib/wic/plugins/source/rootfs.py > b/scripts/lib/wic/plugins/source/rootfs.py index > 96d940a91d61..2e34e715ca73 100644 --- > a/scripts/lib/wic/plugins/source/rootfs.py +++ > b/scripts/lib/wic/plugins/source/rootfs.py @@ -218,7 +218,7 @@ class > RootfsPlugin(SourcePlugin): # Update part.has_fstab here as fstab may > have been added or # removed by the above modifications. > part.has_fstab = os.path.exists(os.path.join(new_rootfs, > "etc/fstab")) > - if part.update_fstab_in_rootfs and part.has_fstab: > + if part.update_fstab_in_rootfs and part.has_fstab and > not part.no_fstab_update: fstab_path = os.path.join(new_rootfs, > "etc/fstab") # Assume that fstab should always be owned by root with > fixed permissions install_cmd = "install -m 0644 %s %s" % > (part.updated_fstab_path, fstab_path) diff --git a/scripts/wic > b/scripts/wic index a741aed364bf..aee63a45aacf 100755 > --- a/scripts/wic > +++ b/scripts/wic > @@ -22,9 +22,9 @@ import sys > import argparse > import logging > import subprocess > +import shutil > > from collections import namedtuple > -from distutils import spawn > > # External modules > scripts_path = os.path.dirname(os.path.realpath(__file__)) > @@ -47,7 +47,7 @@ if os.environ.get('SDKTARGETSYSROOT'): > break > sdkroot = os.path.dirname(sdkroot) > > -bitbake_exe = spawn.find_executable('bitbake') > +bitbake_exe = shutil.which('bitbake') > if bitbake_exe: > bitbake_path = scriptpath.add_bitbake_lib_path() > import bb > @@ -159,6 +159,9 @@ def wic_create_subcommand(options, usage_str): > "(Use -e/--image-name to specify it)") > native_sysroot = options.native_sysroot > > + if options.kernel_dir: > + kernel_dir = options.kernel_dir > + > if not options.vars_dir and (not native_sysroot or not > os.path.isdir(native_sysroot)): logger.info("Building wic-tools...\n") > subprocess.check_call(["bitbake", "wic-tools"]) > @@ -346,6 +349,8 @@ def wic_init_parser_create(subparser): > default=False, help="output debug information") > subparser.add_argument("-i", "--imager", dest="imager", > default="direct", help="the wic imager plugin") > + subparser.add_argument("--extra-space", type=int, > dest="extra_space", > + default=0, help="additional free disk space to > add to the image") return > > ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH v2 0/2] bump bitbake and wic for python 3.10 support 2022-04-21 8:52 [PATCH v2 0/2] bump bitbake and wic for python 3.10 support Henning Schild 2022-04-21 8:52 ` [PATCH v2 1/2] bitbake: Update to 1.50.5 release Henning Schild 2022-04-21 8:52 ` [PATCH v2 2/2] wic: Update to the latest revision Henning Schild @ 2022-04-22 7:59 ` Anton Mikanovich 2022-04-22 8:15 ` Bezdeka, Florian 2022-04-22 8:27 ` Henning Schild 2 siblings, 2 replies; 22+ messages in thread From: Anton Mikanovich @ 2022-04-22 7:59 UTC (permalink / raw) To: Henning Schild, isar-users Cc: Florian Bezdeka, Felix Moessbauer, Baurzhan Ismagulov 21.04.2022 11:52, Henning Schild wrote: > debian bookworm switched to python 3.10 recently, our versions of > bitbake and wic both have some issues with that new interpreter. All > these issues have been fixes in bitbake and OE, so all we need to do is > update the two tools in our tree. > > v1 was send as "[PATCH] bitbake: Update to 1.50.5 release", which is p1 > here, only later i found that wic also needs a bump to v2 added p2 > > If this series is applied we do not need "[PATCH] wic-img: use > python3.9 in bookworm", in case this one is applied before it should be > reverted after the version bumps. > > Henning Schild (2): > bitbake: Update to 1.50.5 release > wic: Update to the latest revision > > bitbake/lib/bb/cache.py | 3 +- > bitbake/lib/bb/cooker.py | 30 +++++++- > bitbake/lib/bb/data_smart.py | 4 +- > bitbake/lib/bb/fetch2/__init__.py | 4 + > bitbake/lib/bb/fetch2/perforce.py | 2 +- > bitbake/lib/bb/fetch2/wget.py | 2 +- > bitbake/lib/bb/persist_data.py | 5 +- > bitbake/lib/bb/process.py | 2 +- > bitbake/lib/bb/runqueue.py | 34 ++++---- > bitbake/lib/bb/server/process.py | 2 +- > bitbake/lib/bb/tests/fetch.py | 35 +++++---- > bitbake/lib/bb/utils.py | 13 +++- > bitbake/lib/hashserv/server.py | 4 +- > bitbake/lib/toaster/tests/builds/buildtest.py | 2 +- > .../wic/plugins/source/bootimg-efi-isar.py | 77 ++++++++++++++++--- > .../wic/plugins/source/bootimg-pcbios-isar.py | 6 +- > scripts/lib/wic/canned-wks/common.wks.inc | 2 +- > scripts/lib/wic/canned-wks/directdisk-gpt.wks | 2 +- > scripts/lib/wic/canned-wks/mkefidisk.wks | 2 +- > scripts/lib/wic/engine.py | 6 +- > scripts/lib/wic/help.py | 10 ++- > scripts/lib/wic/ksparser.py | 8 +- > scripts/lib/wic/misc.py | 4 +- > scripts/lib/wic/partition.py | 25 ++++-- > scripts/lib/wic/pluginbase.py | 8 +- > scripts/lib/wic/plugins/imager/direct.py | 11 ++- > scripts/lib/wic/plugins/source/bootimg-efi.py | 74 +++++++++++++++--- > .../lib/wic/plugins/source/bootimg-pcbios.py | 6 +- > scripts/lib/wic/plugins/source/rawcopy.py | 35 ++++++++- > scripts/lib/wic/plugins/source/rootfs.py | 2 +- > scripts/wic | 9 ++- > 31 files changed, 320 insertions(+), 109 deletions(-) Thanks for update, python issue is really fixed now, but it brakes qemu386 targets booting in QEMU: 00:46:00 [ 34.424136] blk_update_request: I/O error, dev fd0, sector 0 00:46:00 [ 34.428049] floppy: error -5 while reading block 0 00:46:00 [ 34.508133] blk_update_request: I/O error, dev fd0, sector 0 00:46:00 [ 34.512050] floppy: error -5 while reading block 0 00:46:00 Gave up waiting for root file system device. Common problems: 00:46:00 - Boot args (cat /proc/cmdline) 00:46:00 - Check rootdelay= (did the system wait long enough?) 00:46:00 - Missing modules (cat /proc/modules; ls /dev) 00:46:00 ALERT! PARTLABEL=platform does not exist. Dropping to a shell! 00:46:00 (initramfs) ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH v2 0/2] bump bitbake and wic for python 3.10 support 2022-04-22 7:59 ` [PATCH v2 0/2] bump bitbake and wic for python 3.10 support Anton Mikanovich @ 2022-04-22 8:15 ` Bezdeka, Florian 2022-04-22 8:26 ` Henning Schild 2022-04-22 8:27 ` Henning Schild 1 sibling, 1 reply; 22+ messages in thread From: Bezdeka, Florian @ 2022-04-22 8:15 UTC (permalink / raw) To: amikan, isar-users, Schild, Henning Cc: Schmidt, Adriaan, ibr, Moessbauer, Felix On Fri, 2022-04-22 at 10:59 +0300, Anton Mikanovich wrote: > 21.04.2022 11:52, Henning Schild wrote: > > debian bookworm switched to python 3.10 recently, our versions of > > bitbake and wic both have some issues with that new interpreter. All > > these issues have been fixes in bitbake and OE, so all we need to do is > > update the two tools in our tree. > > > > v1 was send as "[PATCH] bitbake: Update to 1.50.5 release", which is p1 > > here, only later i found that wic also needs a bump to v2 added p2 > > > > If this series is applied we do not need "[PATCH] wic-img: use > > python3.9 in bookworm", in case this one is applied before it should be > > reverted after the version bumps. > > > > Henning Schild (2): > > bitbake: Update to 1.50.5 release > > wic: Update to the latest revision > > > > bitbake/lib/bb/cache.py | 3 +- > > bitbake/lib/bb/cooker.py | 30 +++++++- > > bitbake/lib/bb/data_smart.py | 4 +- > > bitbake/lib/bb/fetch2/__init__.py | 4 + > > bitbake/lib/bb/fetch2/perforce.py | 2 +- > > bitbake/lib/bb/fetch2/wget.py | 2 +- > > bitbake/lib/bb/persist_data.py | 5 +- > > bitbake/lib/bb/process.py | 2 +- > > bitbake/lib/bb/runqueue.py | 34 ++++---- > > bitbake/lib/bb/server/process.py | 2 +- > > bitbake/lib/bb/tests/fetch.py | 35 +++++---- > > bitbake/lib/bb/utils.py | 13 +++- > > bitbake/lib/hashserv/server.py | 4 +- > > bitbake/lib/toaster/tests/builds/buildtest.py | 2 +- > > .../wic/plugins/source/bootimg-efi-isar.py | 77 ++++++++++++++++--- > > .../wic/plugins/source/bootimg-pcbios-isar.py | 6 +- > > scripts/lib/wic/canned-wks/common.wks.inc | 2 +- > > scripts/lib/wic/canned-wks/directdisk-gpt.wks | 2 +- > > scripts/lib/wic/canned-wks/mkefidisk.wks | 2 +- > > scripts/lib/wic/engine.py | 6 +- > > scripts/lib/wic/help.py | 10 ++- > > scripts/lib/wic/ksparser.py | 8 +- > > scripts/lib/wic/misc.py | 4 +- > > scripts/lib/wic/partition.py | 25 ++++-- > > scripts/lib/wic/pluginbase.py | 8 +- > > scripts/lib/wic/plugins/imager/direct.py | 11 ++- > > scripts/lib/wic/plugins/source/bootimg-efi.py | 74 +++++++++++++++--- > > .../lib/wic/plugins/source/bootimg-pcbios.py | 6 +- > > scripts/lib/wic/plugins/source/rawcopy.py | 35 ++++++++- > > scripts/lib/wic/plugins/source/rootfs.py | 2 +- > > scripts/wic | 9 ++- > > 31 files changed, 320 insertions(+), 109 deletions(-) > > Thanks for update, python issue is really fixed now, but it brakes > qemu386 targets booting in QEMU: > > 00:46:00 [ 34.424136] blk_update_request: I/O error, dev fd0, sector 0 > 00:46:00 [ 34.428049] floppy: error -5 while reading block 0 > 00:46:00 [ 34.508133] blk_update_request: I/O error, dev fd0, sector 0 > 00:46:00 [ 34.512050] floppy: error -5 while reading block 0 > 00:46:00 Gave up waiting for root file system device. Common problems: > 00:46:00 - Boot args (cat /proc/cmdline) > 00:46:00 - Check rootdelay= (did the system wait long enough?) > 00:46:00 - Missing modules (cat /proc/modules; ls /dev) > 00:46:00 ALERT! PARTLABEL=platform does not exist. Dropping to a shell! > 00:46:00 (initramfs) > Might be upstream commit [1] which Henning pulled in now. Adding Adriaan as author of this commit to CC. I have no idea why this should break exactly this image... [1] https://github.com/openembedded/openembedded-core/commit/2fb247c5ecf057bb96649a3c0234794b4991c050 Regards, Florian ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH v2 0/2] bump bitbake and wic for python 3.10 support 2022-04-22 8:15 ` Bezdeka, Florian @ 2022-04-22 8:26 ` Henning Schild 2022-04-22 8:52 ` Henning Schild 0 siblings, 1 reply; 22+ messages in thread From: Henning Schild @ 2022-04-22 8:26 UTC (permalink / raw) To: Bezdeka, Florian (T CED SES-DE) Cc: amikan, isar-users, Schmidt, Adriaan (T CED SES-DE), ibr, Moessbauer, Felix (T CED SES-DE) Am Fri, 22 Apr 2022 10:15:22 +0200 schrieb "Bezdeka, Florian (T CED SES-DE)" <florian.bezdeka@siemens.com>: > On Fri, 2022-04-22 at 10:59 +0300, Anton Mikanovich wrote: > > 21.04.2022 11:52, Henning Schild wrote: > > > debian bookworm switched to python 3.10 recently, our versions of > > > bitbake and wic both have some issues with that new interpreter. > > > All these issues have been fixes in bitbake and OE, so all we > > > need to do is update the two tools in our tree. > > > > > > v1 was send as "[PATCH] bitbake: Update to 1.50.5 release", which > > > is p1 here, only later i found that wic also needs a bump to v2 > > > added p2 > > > > > > If this series is applied we do not need "[PATCH] wic-img: use > > > python3.9 in bookworm", in case this one is applied before it > > > should be reverted after the version bumps. > > > > > > Henning Schild (2): > > > bitbake: Update to 1.50.5 release > > > wic: Update to the latest revision > > > > > > bitbake/lib/bb/cache.py | 3 +- > > > bitbake/lib/bb/cooker.py | 30 +++++++- > > > bitbake/lib/bb/data_smart.py | 4 +- > > > bitbake/lib/bb/fetch2/__init__.py | 4 + > > > bitbake/lib/bb/fetch2/perforce.py | 2 +- > > > bitbake/lib/bb/fetch2/wget.py | 2 +- > > > bitbake/lib/bb/persist_data.py | 5 +- > > > bitbake/lib/bb/process.py | 2 +- > > > bitbake/lib/bb/runqueue.py | 34 ++++---- > > > bitbake/lib/bb/server/process.py | 2 +- > > > bitbake/lib/bb/tests/fetch.py | 35 +++++---- > > > bitbake/lib/bb/utils.py | 13 +++- > > > bitbake/lib/hashserv/server.py | 4 +- > > > bitbake/lib/toaster/tests/builds/buildtest.py | 2 +- > > > .../wic/plugins/source/bootimg-efi-isar.py | 77 > > > ++++++++++++++++--- .../wic/plugins/source/bootimg-pcbios-isar.py > > > | 6 +- scripts/lib/wic/canned-wks/common.wks.inc | 2 +- > > > scripts/lib/wic/canned-wks/directdisk-gpt.wks | 2 +- > > > scripts/lib/wic/canned-wks/mkefidisk.wks | 2 +- > > > scripts/lib/wic/engine.py | 6 +- > > > scripts/lib/wic/help.py | 10 ++- > > > scripts/lib/wic/ksparser.py | 8 +- > > > scripts/lib/wic/misc.py | 4 +- > > > scripts/lib/wic/partition.py | 25 ++++-- > > > scripts/lib/wic/pluginbase.py | 8 +- > > > scripts/lib/wic/plugins/imager/direct.py | 11 ++- > > > scripts/lib/wic/plugins/source/bootimg-efi.py | 74 > > > +++++++++++++++--- .../lib/wic/plugins/source/bootimg-pcbios.py > > > | 6 +- scripts/lib/wic/plugins/source/rawcopy.py | 35 > > > ++++++++- scripts/lib/wic/plugins/source/rootfs.py | 2 +- > > > scripts/wic | 9 ++- > > > 31 files changed, 320 insertions(+), 109 deletions(-) > > > > Thanks for update, python issue is really fixed now, but it brakes > > qemu386 targets booting in QEMU: Thanks, i will look into this. > > 00:46:00 [ 34.424136] blk_update_request: I/O error, dev fd0, > > sector 0 00:46:00 [ 34.428049] floppy: error -5 while reading > > block 0 00:46:00 [ 34.508133] blk_update_request: I/O error, dev > > fd0, sector 0 00:46:00 [ 34.512050] floppy: error -5 while > > reading block 0 00:46:00 Gave up waiting for root file system > > device. Common problems: 00:46:00 - Boot args (cat /proc/cmdline) > > 00:46:00 - Check rootdelay= (did the system wait long enough?) > > 00:46:00 - Missing modules (cat /proc/modules; ls /dev) > > 00:46:00 ALERT! PARTLABEL=platform does not exist. Dropping to a > > shell! 00:46:00 (initramfs) > > > > Might be upstream commit [1] which Henning pulled in now. > > Adding Adriaan as author of this commit to CC. I have no idea why this > should break exactly this image... We boot all our i386 with BIOS directdisk-isar.wks and all amd64 with EFI sdimage-efi.wks, plus a few exceptions from that pattern where we use efi-plus-pcbios.wks or systemd-boot/EFI with sdimage-efi-sd.wks So maybe that directdisk-isar.wks or common-isar.wks.inc need some bits i did not see and good old legacy BIOS is indeed broken in general with these patches. Henning > [1] > https://github.com/openembedded/openembedded-core/commit/2fb247c5ecf057bb96649a3c0234794b4991c050 > > Regards, > Florian > ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH v2 0/2] bump bitbake and wic for python 3.10 support 2022-04-22 8:26 ` Henning Schild @ 2022-04-22 8:52 ` Henning Schild 2022-04-22 9:54 ` Henning Schild 0 siblings, 1 reply; 22+ messages in thread From: Henning Schild @ 2022-04-22 8:52 UTC (permalink / raw) To: Bezdeka, Florian (T CED SES-DE) Cc: amikan, isar-users, Schmidt, Adriaan (T CED SES-DE), ibr, Moessbauer, Felix (T CED SES-DE) Am Fri, 22 Apr 2022 10:26:08 +0200 schrieb Henning Schild <henning.schild@siemens.com>: > Am Fri, 22 Apr 2022 10:15:22 +0200 > schrieb "Bezdeka, Florian (T CED SES-DE)" > <florian.bezdeka@siemens.com>: > > > On Fri, 2022-04-22 at 10:59 +0300, Anton Mikanovich wrote: > > > 21.04.2022 11:52, Henning Schild wrote: > > > > debian bookworm switched to python 3.10 recently, our versions > > > > of bitbake and wic both have some issues with that new > > > > interpreter. All these issues have been fixes in bitbake and > > > > OE, so all we need to do is update the two tools in our tree. > > > > > > > > v1 was send as "[PATCH] bitbake: Update to 1.50.5 release", > > > > which is p1 here, only later i found that wic also needs a bump > > > > to v2 added p2 > > > > > > > > If this series is applied we do not need "[PATCH] wic-img: use > > > > python3.9 in bookworm", in case this one is applied before it > > > > should be reverted after the version bumps. > > > > > > > > Henning Schild (2): > > > > bitbake: Update to 1.50.5 release > > > > wic: Update to the latest revision > > > > > > > > bitbake/lib/bb/cache.py | 3 +- > > > > bitbake/lib/bb/cooker.py | 30 +++++++- > > > > bitbake/lib/bb/data_smart.py | 4 +- > > > > bitbake/lib/bb/fetch2/__init__.py | 4 + > > > > bitbake/lib/bb/fetch2/perforce.py | 2 +- > > > > bitbake/lib/bb/fetch2/wget.py | 2 +- > > > > bitbake/lib/bb/persist_data.py | 5 +- > > > > bitbake/lib/bb/process.py | 2 +- > > > > bitbake/lib/bb/runqueue.py | 34 ++++---- > > > > bitbake/lib/bb/server/process.py | 2 +- > > > > bitbake/lib/bb/tests/fetch.py | 35 +++++---- > > > > bitbake/lib/bb/utils.py | 13 +++- > > > > bitbake/lib/hashserv/server.py | 4 +- > > > > bitbake/lib/toaster/tests/builds/buildtest.py | 2 +- > > > > .../wic/plugins/source/bootimg-efi-isar.py | 77 > > > > ++++++++++++++++--- > > > > .../wic/plugins/source/bootimg-pcbios-isar.py | 6 +- > > > > scripts/lib/wic/canned-wks/common.wks.inc | 2 +- > > > > scripts/lib/wic/canned-wks/directdisk-gpt.wks | 2 +- > > > > scripts/lib/wic/canned-wks/mkefidisk.wks | 2 +- > > > > scripts/lib/wic/engine.py | 6 +- > > > > scripts/lib/wic/help.py | 10 ++- > > > > scripts/lib/wic/ksparser.py | 8 +- > > > > scripts/lib/wic/misc.py | 4 +- > > > > scripts/lib/wic/partition.py | 25 ++++-- > > > > scripts/lib/wic/pluginbase.py | 8 +- > > > > scripts/lib/wic/plugins/imager/direct.py | 11 ++- > > > > scripts/lib/wic/plugins/source/bootimg-efi.py | 74 > > > > +++++++++++++++--- .../lib/wic/plugins/source/bootimg-pcbios.py > > > > | 6 +- scripts/lib/wic/plugins/source/rawcopy.py | 35 > > > > ++++++++- scripts/lib/wic/plugins/source/rootfs.py | 2 +- > > > > scripts/wic | 9 ++- 31 files > > > > changed, 320 insertions(+), 109 deletions(-) > > > > > > Thanks for update, python issue is really fixed now, but it > > > brakes qemu386 targets booting in QEMU: > > Thanks, i will look into this. > > > > 00:46:00 [ 34.424136] blk_update_request: I/O error, dev fd0, > > > sector 0 00:46:00 [ 34.428049] floppy: error -5 while reading > > > block 0 00:46:00 [ 34.508133] blk_update_request: I/O error, dev > > > fd0, sector 0 00:46:00 [ 34.512050] floppy: error -5 while > > > reading block 0 00:46:00 Gave up waiting for root file system > > > device. Common problems: 00:46:00 - Boot args (cat > > > /proc/cmdline) 00:46:00 - Check rootdelay= (did the system > > > wait long enough?) 00:46:00 - Missing modules (cat > > > /proc/modules; ls /dev) 00:46:00 ALERT! PARTLABEL=platform does > > > not exist. Dropping to a shell! 00:46:00 (initramfs) > > > > > > > Might be upstream commit [1] which Henning pulled in now. > > > > Adding Adriaan as author of this commit to CC. I have no idea why > > this should break exactly this image... > > We boot all our i386 with BIOS directdisk-isar.wks and all amd64 with > EFI sdimage-efi.wks, plus a few exceptions from that pattern where we > use efi-plus-pcbios.wks or systemd-boot/EFI with sdimage-efi-sd.wks > > So maybe that directdisk-isar.wks or common-isar.wks.inc need some > bits i did not see and good old legacy BIOS is indeed broken in > general with these patches. PARTLABEL is a gpt thing, and we use msdos. And our common-isar.wks.inc does not have --use-uuid while upstream common.wks.inc has it. Will send a patch. Could not find if we ever dropped that use-uuid, i think it is more likely it was overlooked while maintaining the fork. Henning > Henning > > > [1] > > https://github.com/openembedded/openembedded-core/commit/2fb247c5ecf057bb96649a3c0234794b4991c050 > > > > Regards, > > Florian > > > ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH v2 0/2] bump bitbake and wic for python 3.10 support 2022-04-22 8:52 ` Henning Schild @ 2022-04-22 9:54 ` Henning Schild 0 siblings, 0 replies; 22+ messages in thread From: Henning Schild @ 2022-04-22 9:54 UTC (permalink / raw) To: Bezdeka, Florian (T CED SES-DE) Cc: amikan, isar-users, Schmidt, Adriaan (T CED SES-DE), ibr, Moessbauer, Felix (T CED SES-DE) Am Fri, 22 Apr 2022 10:52:11 +0200 schrieb Henning Schild <henning.schild@siemens.com>: > Am Fri, 22 Apr 2022 10:26:08 +0200 > schrieb Henning Schild <henning.schild@siemens.com>: > > > Am Fri, 22 Apr 2022 10:15:22 +0200 > > schrieb "Bezdeka, Florian (T CED SES-DE)" > > <florian.bezdeka@siemens.com>: > > > > > On Fri, 2022-04-22 at 10:59 +0300, Anton Mikanovich wrote: > > > > 21.04.2022 11:52, Henning Schild wrote: > > > > > debian bookworm switched to python 3.10 recently, our versions > > > > > of bitbake and wic both have some issues with that new > > > > > interpreter. All these issues have been fixes in bitbake and > > > > > OE, so all we need to do is update the two tools in our tree. > > > > > > > > > > v1 was send as "[PATCH] bitbake: Update to 1.50.5 release", > > > > > which is p1 here, only later i found that wic also needs a > > > > > bump to v2 added p2 > > > > > > > > > > If this series is applied we do not need "[PATCH] wic-img: use > > > > > python3.9 in bookworm", in case this one is applied before it > > > > > should be reverted after the version bumps. > > > > > > > > > > Henning Schild (2): > > > > > bitbake: Update to 1.50.5 release > > > > > wic: Update to the latest revision > > > > > > > > > > bitbake/lib/bb/cache.py | 3 +- > > > > > bitbake/lib/bb/cooker.py | 30 +++++++- > > > > > bitbake/lib/bb/data_smart.py | 4 +- > > > > > bitbake/lib/bb/fetch2/__init__.py | 4 + > > > > > bitbake/lib/bb/fetch2/perforce.py | 2 +- > > > > > bitbake/lib/bb/fetch2/wget.py | 2 +- > > > > > bitbake/lib/bb/persist_data.py | 5 +- > > > > > bitbake/lib/bb/process.py | 2 +- > > > > > bitbake/lib/bb/runqueue.py | 34 ++++---- > > > > > bitbake/lib/bb/server/process.py | 2 +- > > > > > bitbake/lib/bb/tests/fetch.py | 35 +++++---- > > > > > bitbake/lib/bb/utils.py | 13 +++- > > > > > bitbake/lib/hashserv/server.py | 4 +- > > > > > bitbake/lib/toaster/tests/builds/buildtest.py | 2 +- > > > > > .../wic/plugins/source/bootimg-efi-isar.py | 77 > > > > > ++++++++++++++++--- > > > > > .../wic/plugins/source/bootimg-pcbios-isar.py | 6 +- > > > > > scripts/lib/wic/canned-wks/common.wks.inc | 2 +- > > > > > scripts/lib/wic/canned-wks/directdisk-gpt.wks | 2 +- > > > > > scripts/lib/wic/canned-wks/mkefidisk.wks | 2 +- > > > > > scripts/lib/wic/engine.py | 6 +- > > > > > scripts/lib/wic/help.py | 10 ++- > > > > > scripts/lib/wic/ksparser.py | 8 +- > > > > > scripts/lib/wic/misc.py | 4 +- > > > > > scripts/lib/wic/partition.py | 25 ++++-- > > > > > scripts/lib/wic/pluginbase.py | 8 +- > > > > > scripts/lib/wic/plugins/imager/direct.py | 11 ++- > > > > > scripts/lib/wic/plugins/source/bootimg-efi.py | 74 > > > > > +++++++++++++++--- > > > > > .../lib/wic/plugins/source/bootimg-pcbios.py | 6 +- > > > > > scripts/lib/wic/plugins/source/rawcopy.py | 35 ++++++++- > > > > > scripts/lib/wic/plugins/source/rootfs.py | 2 +- > > > > > scripts/wic | 9 ++- 31 > > > > > files changed, 320 insertions(+), 109 deletions(-) > > > > > > > > Thanks for update, python issue is really fixed now, but it > > > > brakes qemu386 targets booting in QEMU: > > > > Thanks, i will look into this. > > > > > > 00:46:00 [ 34.424136] blk_update_request: I/O error, dev fd0, > > > > sector 0 00:46:00 [ 34.428049] floppy: error -5 while reading > > > > block 0 00:46:00 [ 34.508133] blk_update_request: I/O error, > > > > dev fd0, sector 0 00:46:00 [ 34.512050] floppy: error -5 while > > > > reading block 0 00:46:00 Gave up waiting for root file system > > > > device. Common problems: 00:46:00 - Boot args (cat > > > > /proc/cmdline) 00:46:00 - Check rootdelay= (did the system > > > > wait long enough?) 00:46:00 - Missing modules (cat > > > > /proc/modules; ls /dev) 00:46:00 ALERT! PARTLABEL=platform does > > > > not exist. Dropping to a shell! 00:46:00 (initramfs) > > > > > > > > > > Might be upstream commit [1] which Henning pulled in now. > > > > > > Adding Adriaan as author of this commit to CC. I have no idea why > > > this should break exactly this image... > > > > We boot all our i386 with BIOS directdisk-isar.wks and all amd64 > > with EFI sdimage-efi.wks, plus a few exceptions from that pattern > > where we use efi-plus-pcbios.wks or systemd-boot/EFI with > > sdimage-efi-sd.wks > > > > So maybe that directdisk-isar.wks or common-isar.wks.inc need some > > bits i did not see and good old legacy BIOS is indeed broken in > > general with these patches. > > PARTLABEL is a gpt thing, and we use msdos. And our > common-isar.wks.inc does not have --use-uuid while upstream > common.wks.inc has it. > > Will send a patch. Could not find if we ever dropped that use-uuid, i > think it is more likely it was overlooked while maintaining the fork. v3 solves this with a patch to align our common wks with OE, additionally i started an upstream discussion with a patch to not allow anyone to PARTLABEL on msdos https://lists.openembedded.org/g/openembedded-core/topic/patch_wic_do_not_use/90624635?p=,,,20,0,0,0::recentpostdate/sticky,,,20,2,0,90624635,previd%3D1650620877809916310,nextid%3D1650546439549613402&previd=1650620877809916310&nextid=1650546439549613402 Henning > Henning > > > Henning > > > > > [1] > > > https://github.com/openembedded/openembedded-core/commit/2fb247c5ecf057bb96649a3c0234794b4991c050 > > > > > > Regards, > > > Florian > > > > > > ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH v2 0/2] bump bitbake and wic for python 3.10 support 2022-04-22 7:59 ` [PATCH v2 0/2] bump bitbake and wic for python 3.10 support Anton Mikanovich 2022-04-22 8:15 ` Bezdeka, Florian @ 2022-04-22 8:27 ` Henning Schild 2022-04-22 8:57 ` Anton Mikanovich 1 sibling, 1 reply; 22+ messages in thread From: Henning Schild @ 2022-04-22 8:27 UTC (permalink / raw) To: Anton Mikanovich Cc: isar-users, Florian Bezdeka, Felix Moessbauer, Baurzhan Ismagulov Am Fri, 22 Apr 2022 10:59:15 +0300 schrieb Anton Mikanovich <amikan@ilbers.de>: > 21.04.2022 11:52, Henning Schild wrote: > > debian bookworm switched to python 3.10 recently, our versions of > > bitbake and wic both have some issues with that new interpreter. All > > these issues have been fixes in bitbake and OE, so all we need to > > do is update the two tools in our tree. > > > > v1 was send as "[PATCH] bitbake: Update to 1.50.5 release", which > > is p1 here, only later i found that wic also needs a bump to v2 > > added p2 > > > > If this series is applied we do not need "[PATCH] wic-img: use > > python3.9 in bookworm", in case this one is applied before it > > should be reverted after the version bumps. > > > > Henning Schild (2): > > bitbake: Update to 1.50.5 release > > wic: Update to the latest revision > > > > bitbake/lib/bb/cache.py | 3 +- > > bitbake/lib/bb/cooker.py | 30 +++++++- > > bitbake/lib/bb/data_smart.py | 4 +- > > bitbake/lib/bb/fetch2/__init__.py | 4 + > > bitbake/lib/bb/fetch2/perforce.py | 2 +- > > bitbake/lib/bb/fetch2/wget.py | 2 +- > > bitbake/lib/bb/persist_data.py | 5 +- > > bitbake/lib/bb/process.py | 2 +- > > bitbake/lib/bb/runqueue.py | 34 ++++---- > > bitbake/lib/bb/server/process.py | 2 +- > > bitbake/lib/bb/tests/fetch.py | 35 +++++---- > > bitbake/lib/bb/utils.py | 13 +++- > > bitbake/lib/hashserv/server.py | 4 +- > > bitbake/lib/toaster/tests/builds/buildtest.py | 2 +- > > .../wic/plugins/source/bootimg-efi-isar.py | 77 > > ++++++++++++++++--- .../wic/plugins/source/bootimg-pcbios-isar.py | > > 6 +- scripts/lib/wic/canned-wks/common.wks.inc | 2 +- > > scripts/lib/wic/canned-wks/directdisk-gpt.wks | 2 +- > > scripts/lib/wic/canned-wks/mkefidisk.wks | 2 +- > > scripts/lib/wic/engine.py | 6 +- > > scripts/lib/wic/help.py | 10 ++- > > scripts/lib/wic/ksparser.py | 8 +- > > scripts/lib/wic/misc.py | 4 +- > > scripts/lib/wic/partition.py | 25 ++++-- > > scripts/lib/wic/pluginbase.py | 8 +- > > scripts/lib/wic/plugins/imager/direct.py | 11 ++- > > scripts/lib/wic/plugins/source/bootimg-efi.py | 74 > > +++++++++++++++--- .../lib/wic/plugins/source/bootimg-pcbios.py | > > 6 +- scripts/lib/wic/plugins/source/rawcopy.py | 35 ++++++++- > > scripts/lib/wic/plugins/source/rootfs.py | 2 +- > > scripts/wic | 9 ++- > > 31 files changed, 320 insertions(+), 109 deletions(-) > > Thanks for update, python issue is really fixed now, but it brakes > qemu386 targets booting in QEMU: All of them or only some suits? I am building qemui386-bullseye now for local repro. Henning > 00:46:00 [ 34.424136] blk_update_request: I/O error, dev fd0, > sector 0 00:46:00 [ 34.428049] floppy: error -5 while reading block > 0 00:46:00 [ 34.508133] blk_update_request: I/O error, dev fd0, > sector 0 00:46:00 [ 34.512050] floppy: error -5 while reading block > 0 00:46:00 Gave up waiting for root file system device. Common > problems: 00:46:00 - Boot args (cat /proc/cmdline) > 00:46:00 - Check rootdelay= (did the system wait long enough?) > 00:46:00 - Missing modules (cat /proc/modules; ls /dev) > 00:46:00 ALERT! PARTLABEL=platform does not exist. Dropping to a > shell! 00:46:00 (initramfs) > ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH v2 0/2] bump bitbake and wic for python 3.10 support 2022-04-22 8:27 ` Henning Schild @ 2022-04-22 8:57 ` Anton Mikanovich 0 siblings, 0 replies; 22+ messages in thread From: Anton Mikanovich @ 2022-04-22 8:57 UTC (permalink / raw) To: Henning Schild Cc: isar-users, Florian Bezdeka, Felix Moessbauer, Baurzhan Ismagulov 22.04.2022 11:27, Henning Schild wrote: > Am Fri, 22 Apr 2022 10:59:15 +0300 > schrieb Anton Mikanovich <amikan@ilbers.de>: > >> Thanks for update, python issue is really fixed now, but it brakes >> qemu386 targets booting in QEMU: > All of them or only some suits? I am building qemui386-bullseye now for > local repro. > > Henning > I've tried buster and stretch (we have them covered by CI start_vm) - both fails. ^ permalink raw reply [flat|nested] 22+ messages in thread
* [PATCH v2 0/2] Update WIC to bring in fspassno feature @ 2022-06-13 7:53 Vijai Kumar K 2022-06-13 7:54 ` [PATCH v2 2/2] wic: Update to the latest revision Vijai Kumar K 0 siblings, 1 reply; 22+ messages in thread From: Vijai Kumar K @ 2022-06-13 7:53 UTC (permalink / raw) To: isar-users; +Cc: henning.schild, jan.kiszka, Vijai Kumar K Changes since v1: - Captured potential breakage in RECIPE-API-CHANGELOG. Thanks, Vijai Kumar K Vijai Kumar K (2): meta-isar/wks: prepare wks for wic uprev wic: Update to the latest revision RECIPE-API-CHANGELOG.md | 19 +++++++++++++++++++ .../lib/wic/canned-wks/common-isar.wks.inc | 2 +- .../lib/wic/canned-wks/de0-nano-soc.wks.in | 2 +- .../lib/wic/canned-wks/efi-plus-pcbios.wks | 2 +- .../scripts/lib/wic/canned-wks/hikey.wks | 2 +- .../lib/wic/canned-wks/multipart-efi.wks | 2 +- .../lib/wic/canned-wks/nanopi-neo.wks.in | 2 +- .../scripts/lib/wic/canned-wks/rpi-sdimg.wks | 2 +- .../lib/wic/canned-wks/sdimage-efi-sd.wks | 2 +- .../lib/wic/canned-wks/sdimage-efi.wks | 2 +- .../lib/wic/canned-wks/sifive-fu540.wks | 2 +- .../lib/wic/canned-wks/stm32mp15x.wks.in | 2 +- scripts/lib/wic/help.py | 6 ++++++ scripts/lib/wic/ksparser.py | 1 + scripts/lib/wic/partition.py | 1 + scripts/lib/wic/plugins/imager/direct.py | 5 +++-- 16 files changed, 41 insertions(+), 13 deletions(-) -- 2.25.1 ^ permalink raw reply [flat|nested] 22+ messages in thread
* [PATCH v2 2/2] wic: Update to the latest revision 2022-06-13 7:53 [PATCH v2 0/2] Update WIC to bring in fspassno feature Vijai Kumar K @ 2022-06-13 7:54 ` Vijai Kumar K 2022-06-13 9:02 ` Henning Schild 0 siblings, 1 reply; 22+ messages in thread From: Vijai Kumar K @ 2022-06-13 7:54 UTC (permalink / raw) To: isar-users; +Cc: henning.schild, jan.kiszka, Vijai Kumar K Update to the latest wic from OE-core. OE-core Revision: 06ed2aa93fc25a681e3a00ee120d9395b04845da Signed-off-by: Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com> --- RECIPE-API-CHANGELOG.md | 19 +++++++++++++++++++ scripts/lib/wic/help.py | 6 ++++++ scripts/lib/wic/ksparser.py | 1 + scripts/lib/wic/partition.py | 1 + scripts/lib/wic/plugins/imager/direct.py | 5 +++-- 5 files changed, 30 insertions(+), 2 deletions(-) diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md index 2998824..436b204 100644 --- a/RECIPE-API-CHANGELOG.md +++ b/RECIPE-API-CHANGELOG.md @@ -406,3 +406,22 @@ groups are now created after all packages are installed. Changes in next --------------- + +### Wic adds / mountpoint to fstab + +In the older version of wic, any mount point named / is skipped from adding +into the fstab entry. + +With the latest wic, this is not the case. / mount point, if any, is added +to /etc/fstab for automount. + +Any wks file which assumed that / would be skipped from /etc/fstab should +now be corrected. Otherwise, it might result in multiple / entries in /etc/fstab. +One by the image_configure_fstab function from image class and one by wic. + +Below is an example wks entry that might cause an issue. +Replace "part /" with "part" to avoid adding duplicate entry for /. + +``` +part / --source rootfs --ondisk sda --fstype ext4 --label platform --align 1024 --use-uuid +``` diff --git a/scripts/lib/wic/help.py b/scripts/lib/wic/help.py index 4ff7470..73e3380 100644 --- a/scripts/lib/wic/help.py +++ b/scripts/lib/wic/help.py @@ -940,6 +940,12 @@ DESCRIPTION quotes. If not specified, the default string is "defaults". + --fspassno: Specifies the order in which filesystem checks are done + at boot time by fsck. See fs_passno parameter of + fstab(5). This parameter will be copied into the + /etc/fstab file of the installed system. If not + specified the default value of "0" will be used. + --label label: Specifies the label to give to the filesystem to be made on the partition. If the given label is already in use by another filesystem, diff --git a/scripts/lib/wic/ksparser.py b/scripts/lib/wic/ksparser.py index 0df9eb0..a49b7b9 100644 --- a/scripts/lib/wic/ksparser.py +++ b/scripts/lib/wic/ksparser.py @@ -155,6 +155,7 @@ class KickStart(): part.add_argument('--change-directory') part.add_argument("--extra-space", type=sizetype("M")) part.add_argument('--fsoptions', dest='fsopts') + part.add_argument('--fspassno', dest='fspassno') part.add_argument('--fstype', default='vfat', choices=('ext2', 'ext3', 'ext4', 'btrfs', 'squashfs', 'vfat', 'msdos', 'erofs', diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py index 09e491d..e50871b 100644 --- a/scripts/lib/wic/partition.py +++ b/scripts/lib/wic/partition.py @@ -33,6 +33,7 @@ class Partition(): self.include_path = args.include_path self.change_directory = args.change_directory self.fsopts = args.fsopts + self.fspassno = args.fspassno self.fstype = args.fstype self.label = args.label self.use_label = args.use_label diff --git a/scripts/lib/wic/plugins/imager/direct.py b/scripts/lib/wic/plugins/imager/direct.py index 4d0b836..c44159b 100644 --- a/scripts/lib/wic/plugins/imager/direct.py +++ b/scripts/lib/wic/plugins/imager/direct.py @@ -117,7 +117,7 @@ class DirectPlugin(ImagerPlugin): updated = False for part in self.parts: if not part.realnum or not part.mountpoint \ - or part.mountpoint == "/" or not part.mountpoint.startswith('/'): + or not part.mountpoint.startswith('/'): continue if part.use_uuid: @@ -138,8 +138,9 @@ class DirectPlugin(ImagerPlugin): device_name = "/dev/%s%s%d" % (part.disk, prefix, part.realnum) opts = part.fsopts if part.fsopts else "defaults" + passno = part.fspassno if part.fspassno else "0" line = "\t".join([device_name, part.mountpoint, part.fstype, - opts, "0", "0"]) + "\n" + opts, "0", passno]) + "\n" fstab_lines.append(line) updated = True -- 2.25.1 ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH v2 2/2] wic: Update to the latest revision 2022-06-13 7:54 ` [PATCH v2 2/2] wic: Update to the latest revision Vijai Kumar K @ 2022-06-13 9:02 ` Henning Schild 2022-06-13 9:18 ` Kanagarajan, Vijaikumar 0 siblings, 1 reply; 22+ messages in thread From: Henning Schild @ 2022-06-13 9:02 UTC (permalink / raw) To: Vijai Kumar K; +Cc: isar-users, jan.kiszka Am Mon, 13 Jun 2022 13:24:01 +0530 schrieb Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com>: > Update to the latest wic from OE-core. > > OE-core Revision: 06ed2aa93fc25a681e3a00ee120d9395b04845da > > Signed-off-by: Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com> > --- > RECIPE-API-CHANGELOG.md | 19 +++++++++++++++++++ > scripts/lib/wic/help.py | 6 ++++++ > scripts/lib/wic/ksparser.py | 1 + > scripts/lib/wic/partition.py | 1 + > scripts/lib/wic/plugins/imager/direct.py | 5 +++-- > 5 files changed, 30 insertions(+), 2 deletions(-) > > diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md > index 2998824..436b204 100644 > --- a/RECIPE-API-CHANGELOG.md > +++ b/RECIPE-API-CHANGELOG.md > @@ -406,3 +406,22 @@ groups are now created after all packages are > installed. > Changes in next > --------------- > + > +### Wic adds / mountpoint to fstab > + > +In the older version of wic, any mount point named / is skipped from > adding +into the fstab entry. > + > +With the latest wic, this is not the case. / mount point, if any, is > added +to /etc/fstab for automount. > + > +Any wks file which assumed that / would be skipped from /etc/fstab > should +now be corrected. Otherwise, it might result in multiple / > entries in /etc/fstab. +One by the image_configure_fstab function > from image class and one by wic. + > +Below is an example wks entry that might cause an issue. > +Replace "part /" with "part" to avoid adding duplicate entry for /. I am not sure multiple entries would be an issue, i think the last one will simply always win. It is good to have a note about that change, but i doubt people really have to do anything. And if anything has to be done, isar should drop the first one centrally. The fstab entry for root is purely for adding some weird mount options, usually root is not mounted with fstab but with cmdline and initrd. And not having / in fstab just makes that rootfs more generic ... it can become part of nfs and next guy takes it as btrfs while next guy takes it as ext4. Note is good, advise is not Henning > + > +``` > +part / --source rootfs --ondisk sda --fstype ext4 --label platform > --align 1024 --use-uuid +``` > diff --git a/scripts/lib/wic/help.py b/scripts/lib/wic/help.py > index 4ff7470..73e3380 100644 > --- a/scripts/lib/wic/help.py > +++ b/scripts/lib/wic/help.py > @@ -940,6 +940,12 @@ DESCRIPTION > quotes. If not specified, the default string > is "defaults". > > + --fspassno: Specifies the order in which filesystem checks > are done > + at boot time by fsck. See fs_passno parameter > of > + fstab(5). This parameter will be copied into > the > + /etc/fstab file of the installed system. If not > + specified the default value of "0" will be used. > + > --label label: Specifies the label to give to the filesystem > to be made on the partition. If the given > label is already in use by another > filesystem, diff --git a/scripts/lib/wic/ksparser.py > b/scripts/lib/wic/ksparser.py index 0df9eb0..a49b7b9 100644 > --- a/scripts/lib/wic/ksparser.py > +++ b/scripts/lib/wic/ksparser.py > @@ -155,6 +155,7 @@ class KickStart(): > part.add_argument('--change-directory') > part.add_argument("--extra-space", type=sizetype("M")) > part.add_argument('--fsoptions', dest='fsopts') > + part.add_argument('--fspassno', dest='fspassno') > part.add_argument('--fstype', default='vfat', > choices=('ext2', 'ext3', 'ext4', 'btrfs', > 'squashfs', 'vfat', 'msdos', > 'erofs', diff --git a/scripts/lib/wic/partition.py > b/scripts/lib/wic/partition.py index 09e491d..e50871b 100644 > --- a/scripts/lib/wic/partition.py > +++ b/scripts/lib/wic/partition.py > @@ -33,6 +33,7 @@ class Partition(): > self.include_path = args.include_path > self.change_directory = args.change_directory > self.fsopts = args.fsopts > + self.fspassno = args.fspassno > self.fstype = args.fstype > self.label = args.label > self.use_label = args.use_label > diff --git a/scripts/lib/wic/plugins/imager/direct.py > b/scripts/lib/wic/plugins/imager/direct.py index 4d0b836..c44159b > 100644 --- a/scripts/lib/wic/plugins/imager/direct.py > +++ b/scripts/lib/wic/plugins/imager/direct.py > @@ -117,7 +117,7 @@ class DirectPlugin(ImagerPlugin): > updated = False > for part in self.parts: > if not part.realnum or not part.mountpoint \ > - or part.mountpoint == "/" or not > part.mountpoint.startswith('/'): > + or not part.mountpoint.startswith('/'): > continue > > if part.use_uuid: > @@ -138,8 +138,9 @@ class DirectPlugin(ImagerPlugin): > device_name = "/dev/%s%s%d" % (part.disk, prefix, > part.realnum) > opts = part.fsopts if part.fsopts else "defaults" > + passno = part.fspassno if part.fspassno else "0" > line = "\t".join([device_name, part.mountpoint, > part.fstype, > - opts, "0", "0"]) + "\n" > + opts, "0", passno]) + "\n" > > fstab_lines.append(line) > updated = True ^ permalink raw reply [flat|nested] 22+ messages in thread
* RE: [PATCH v2 2/2] wic: Update to the latest revision 2022-06-13 9:02 ` Henning Schild @ 2022-06-13 9:18 ` Kanagarajan, Vijaikumar 2022-06-17 5:45 ` Kanagarajan, Vijaikumar 0 siblings, 1 reply; 22+ messages in thread From: Kanagarajan, Vijaikumar @ 2022-06-13 9:18 UTC (permalink / raw) To: Henning Schild; +Cc: isar-users, jan.kiszka > -----Original Message----- > From: Henning Schild <henning.schild@siemens.com> > Sent: 13 June 2022 14:32 > To: Kanagarajan, Vijaikumar <Vijaikumar_Kanagarajan@mentor.com> > Cc: isar-users@googlegroups.com; jan.kiszka@siemens.com > Subject: Re: [PATCH v2 2/2] wic: Update to the latest revision > > Am Mon, 13 Jun 2022 13:24:01 +0530 > schrieb Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com>: > > > Update to the latest wic from OE-core. > > > > OE-core Revision: 06ed2aa93fc25a681e3a00ee120d9395b04845da > > > > Signed-off-by: Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com> > > --- > > RECIPE-API-CHANGELOG.md | 19 +++++++++++++++++++ > > scripts/lib/wic/help.py | 6 ++++++ > > scripts/lib/wic/ksparser.py | 1 + > > scripts/lib/wic/partition.py | 1 + > > scripts/lib/wic/plugins/imager/direct.py | 5 +++-- > > 5 files changed, 30 insertions(+), 2 deletions(-) > > > > diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md > index > > 2998824..436b204 100644 > > --- a/RECIPE-API-CHANGELOG.md > > +++ b/RECIPE-API-CHANGELOG.md > > @@ -406,3 +406,22 @@ groups are now created after all packages are > > installed. > > Changes in next > > --------------- > > + > > +### Wic adds / mountpoint to fstab > > + > > +In the older version of wic, any mount point named / is skipped from > > adding +into the fstab entry. > > + > > +With the latest wic, this is not the case. / mount point, if any, is > > added +to /etc/fstab for automount. > > + > > +Any wks file which assumed that / would be skipped from /etc/fstab > > should +now be corrected. Otherwise, it might result in multiple / > > entries in /etc/fstab. +One by the image_configure_fstab function from > > image class and one by wic. + > > +Below is an example wks entry that might cause an issue. > > +Replace "part /" with "part" to avoid adding duplicate entry for /. > > I am not sure multiple entries would be an issue, i think the last one will > simply always win. It is good to have a note about that change, but i doubt > people really have to do anything. > And if anything has to be done, isar should drop the first one centrally. > > The fstab entry for root is purely for adding some weird mount options, > usually root is not mounted with fstab but with cmdline and initrd. And not > having / in fstab just makes that rootfs more generic ... it can become part of > nfs and next guy takes it as btrfs while next guy takes it as ext4. > > Note is good, advise is not Noted. Will change the text accordingly in V3. Thanks, Vijai Kumar K > > Henning > > > + > > +``` > > +part / --source rootfs --ondisk sda --fstype ext4 --label platform > > --align 1024 --use-uuid +``` > > diff --git a/scripts/lib/wic/help.py b/scripts/lib/wic/help.py index > > 4ff7470..73e3380 100644 > > --- a/scripts/lib/wic/help.py > > +++ b/scripts/lib/wic/help.py > > @@ -940,6 +940,12 @@ DESCRIPTION > > quotes. If not specified, the default string > > is "defaults". > > > > + --fspassno: Specifies the order in which filesystem checks > > are done > > + at boot time by fsck. See fs_passno parameter > > of > > + fstab(5). This parameter will be copied into > > the > > + /etc/fstab file of the installed system. If not > > + specified the default value of "0" will be used. > > + > > --label label: Specifies the label to give to the filesystem > > to be made on the partition. If the given > > label is already in use by another > > filesystem, diff --git a/scripts/lib/wic/ksparser.py > > b/scripts/lib/wic/ksparser.py index 0df9eb0..a49b7b9 100644 > > --- a/scripts/lib/wic/ksparser.py > > +++ b/scripts/lib/wic/ksparser.py > > @@ -155,6 +155,7 @@ class KickStart(): > > part.add_argument('--change-directory') > > part.add_argument("--extra-space", type=sizetype("M")) > > part.add_argument('--fsoptions', dest='fsopts') > > + part.add_argument('--fspassno', dest='fspassno') > > part.add_argument('--fstype', default='vfat', > > choices=('ext2', 'ext3', 'ext4', 'btrfs', > > 'squashfs', 'vfat', 'msdos', > > 'erofs', diff --git a/scripts/lib/wic/partition.py > > b/scripts/lib/wic/partition.py index 09e491d..e50871b 100644 > > --- a/scripts/lib/wic/partition.py > > +++ b/scripts/lib/wic/partition.py > > @@ -33,6 +33,7 @@ class Partition(): > > self.include_path = args.include_path > > self.change_directory = args.change_directory > > self.fsopts = args.fsopts > > + self.fspassno = args.fspassno > > self.fstype = args.fstype > > self.label = args.label > > self.use_label = args.use_label diff --git > > a/scripts/lib/wic/plugins/imager/direct.py > > b/scripts/lib/wic/plugins/imager/direct.py index 4d0b836..c44159b > > 100644 --- a/scripts/lib/wic/plugins/imager/direct.py > > +++ b/scripts/lib/wic/plugins/imager/direct.py > > @@ -117,7 +117,7 @@ class DirectPlugin(ImagerPlugin): > > updated = False > > for part in self.parts: > > if not part.realnum or not part.mountpoint \ > > - or part.mountpoint == "/" or not > > part.mountpoint.startswith('/'): > > + or not part.mountpoint.startswith('/'): > > continue > > > > if part.use_uuid: > > @@ -138,8 +138,9 @@ class DirectPlugin(ImagerPlugin): > > device_name = "/dev/%s%s%d" % (part.disk, prefix, > > part.realnum) > > opts = part.fsopts if part.fsopts else "defaults" > > + passno = part.fspassno if part.fspassno else "0" > > line = "\t".join([device_name, part.mountpoint, > > part.fstype, > > - opts, "0", "0"]) + "\n" > > + opts, "0", passno]) + "\n" > > > > fstab_lines.append(line) > > updated = True ^ permalink raw reply [flat|nested] 22+ messages in thread
* RE: [PATCH v2 2/2] wic: Update to the latest revision 2022-06-13 9:18 ` Kanagarajan, Vijaikumar @ 2022-06-17 5:45 ` Kanagarajan, Vijaikumar 2022-09-26 1:40 ` Moessbauer, Felix 0 siblings, 1 reply; 22+ messages in thread From: Kanagarajan, Vijaikumar @ 2022-06-17 5:45 UTC (permalink / raw) To: isar-users; +Cc: jan.kiszka, Henning Schild > -----Original Message----- > From: Kanagarajan, Vijaikumar > Sent: 13 June 2022 14:48 > To: 'Henning Schild' <henning.schild@siemens.com> > Cc: isar-users@googlegroups.com; jan.kiszka@siemens.com > Subject: RE: [PATCH v2 2/2] wic: Update to the latest revision > > > > > -----Original Message----- > > From: Henning Schild <henning.schild@siemens.com> > > Sent: 13 June 2022 14:32 > > To: Kanagarajan, Vijaikumar <Vijaikumar_Kanagarajan@mentor.com> > > Cc: isar-users@googlegroups.com; jan.kiszka@siemens.com > > Subject: Re: [PATCH v2 2/2] wic: Update to the latest revision > > > > Am Mon, 13 Jun 2022 13:24:01 +0530 > > schrieb Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com>: > > > > > Update to the latest wic from OE-core. > > > > > > OE-core Revision: 06ed2aa93fc25a681e3a00ee120d9395b04845da > > > > > > Signed-off-by: Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com> > > > --- > > > RECIPE-API-CHANGELOG.md | 19 +++++++++++++++++++ > > > scripts/lib/wic/help.py | 6 ++++++ > > > scripts/lib/wic/ksparser.py | 1 + > > > scripts/lib/wic/partition.py | 1 + > > > scripts/lib/wic/plugins/imager/direct.py | 5 +++-- > > > 5 files changed, 30 insertions(+), 2 deletions(-) > > > > > > diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md > > index > > > 2998824..436b204 100644 > > > --- a/RECIPE-API-CHANGELOG.md > > > +++ b/RECIPE-API-CHANGELOG.md > > > @@ -406,3 +406,22 @@ groups are now created after all packages are > > > installed. > > > Changes in next > > > --------------- > > > + > > > +### Wic adds / mountpoint to fstab > > > + > > > +In the older version of wic, any mount point named / is skipped > > > +from > > > adding +into the fstab entry. > > > + > > > +With the latest wic, this is not the case. / mount point, if any, > > > +is > > > added +to /etc/fstab for automount. > > > + > > > +Any wks file which assumed that / would be skipped from /etc/fstab > > > should +now be corrected. Otherwise, it might result in multiple / > > > entries in /etc/fstab. +One by the image_configure_fstab function > > > from image class and one by wic. + > > > +Below is an example wks entry that might cause an issue. > > > +Replace "part /" with "part" to avoid adding duplicate entry for /. > > > > I am not sure multiple entries would be an issue, i think the last one > > will simply always win. It is good to have a note about that change, > > but i doubt people really have to do anything. > > And if anything has to be done, isar should drop the first one centrally. > > > > The fstab entry for root is purely for adding some weird mount > > options, usually root is not mounted with fstab but with cmdline and > > initrd. And not having / in fstab just makes that rootfs more generic > > ... it can become part of nfs and next guy takes it as btrfs while next guy > takes it as ext4. > > > > Note is good, advise is not > > Noted. Will change the text accordingly in V3. Ok we cannot drop / mountpoint name(though optional). We would be lose the ability to find out UUID of root using rootdev api. This is used by some plugins. So the better way would be to use the --no-fstab-update. Seems like we can set them per partition now. Thanks, Vijai Kumar K > > Thanks, > Vijai Kumar K > > > > > Henning > > > > > + > > > +``` > > > +part / --source rootfs --ondisk sda --fstype ext4 --label platform > > > --align 1024 --use-uuid +``` > > > diff --git a/scripts/lib/wic/help.py b/scripts/lib/wic/help.py index > > > 4ff7470..73e3380 100644 > > > --- a/scripts/lib/wic/help.py > > > +++ b/scripts/lib/wic/help.py > > > @@ -940,6 +940,12 @@ DESCRIPTION > > > quotes. If not specified, the default string > > > is "defaults". > > > > > > + --fspassno: Specifies the order in which filesystem checks > > > are done > > > + at boot time by fsck. See fs_passno parameter > > > of > > > + fstab(5). This parameter will be copied into > > > the > > > + /etc/fstab file of the installed system. If not > > > + specified the default value of "0" will be used. > > > + > > > --label label: Specifies the label to give to the filesystem > > > to be made on the partition. If the given > > > label is already in use by another > > > filesystem, diff --git a/scripts/lib/wic/ksparser.py > > > b/scripts/lib/wic/ksparser.py index 0df9eb0..a49b7b9 100644 > > > --- a/scripts/lib/wic/ksparser.py > > > +++ b/scripts/lib/wic/ksparser.py > > > @@ -155,6 +155,7 @@ class KickStart(): > > > part.add_argument('--change-directory') > > > part.add_argument("--extra-space", type=sizetype("M")) > > > part.add_argument('--fsoptions', dest='fsopts') > > > + part.add_argument('--fspassno', dest='fspassno') > > > part.add_argument('--fstype', default='vfat', > > > choices=('ext2', 'ext3', 'ext4', 'btrfs', > > > 'squashfs', 'vfat', 'msdos', > > > 'erofs', diff --git a/scripts/lib/wic/partition.py > > > b/scripts/lib/wic/partition.py index 09e491d..e50871b 100644 > > > --- a/scripts/lib/wic/partition.py > > > +++ b/scripts/lib/wic/partition.py > > > @@ -33,6 +33,7 @@ class Partition(): > > > self.include_path = args.include_path > > > self.change_directory = args.change_directory > > > self.fsopts = args.fsopts > > > + self.fspassno = args.fspassno > > > self.fstype = args.fstype > > > self.label = args.label > > > self.use_label = args.use_label diff --git > > > a/scripts/lib/wic/plugins/imager/direct.py > > > b/scripts/lib/wic/plugins/imager/direct.py index 4d0b836..c44159b > > > 100644 --- a/scripts/lib/wic/plugins/imager/direct.py > > > +++ b/scripts/lib/wic/plugins/imager/direct.py > > > @@ -117,7 +117,7 @@ class DirectPlugin(ImagerPlugin): > > > updated = False > > > for part in self.parts: > > > if not part.realnum or not part.mountpoint \ > > > - or part.mountpoint == "/" or not > > > part.mountpoint.startswith('/'): > > > + or not part.mountpoint.startswith('/'): > > > continue > > > > > > if part.use_uuid: > > > @@ -138,8 +138,9 @@ class DirectPlugin(ImagerPlugin): > > > device_name = "/dev/%s%s%d" % (part.disk, prefix, > > > part.realnum) > > > opts = part.fsopts if part.fsopts else "defaults" > > > + passno = part.fspassno if part.fspassno else "0" > > > line = "\t".join([device_name, part.mountpoint, > > > part.fstype, > > > - opts, "0", "0"]) + "\n" > > > + opts, "0", passno]) + "\n" > > > > > > fstab_lines.append(line) > > > updated = True ^ permalink raw reply [flat|nested] 22+ messages in thread
* RE: [PATCH v2 2/2] wic: Update to the latest revision 2022-06-17 5:45 ` Kanagarajan, Vijaikumar @ 2022-09-26 1:40 ` Moessbauer, Felix 2022-09-26 4:53 ` Kanagarajan, Vijaikumar 0 siblings, 1 reply; 22+ messages in thread From: Moessbauer, Felix @ 2022-09-26 1:40 UTC (permalink / raw) To: Kanagarajan, Vijaikumar, Schild, Henning; +Cc: jan.kiszka, isar-users Hi Vijai & Henning, Are there any news on this? I just discovered that still many (if not all) of our images are affected by this bug. While it is not serios, having error messages in the journal is still pretty bad: systemd-fstab-generator[1491]: Failed to create unit file /run/systemd/generator/-.mount, as it already exists. Duplicate entry in /etc/fstab? Felix > -----Original Message----- > From: isar-users@googlegroups.com <isar-users@googlegroups.com> On > Behalf Of Kanagarajan, Vijaikumar > Sent: Friday, June 17, 2022 1:46 PM > To: isar-users@googlegroups.com > Cc: Kiszka, Jan (T CED) <jan.kiszka@siemens.com>; Schild, Henning (T CED SES- > DE) <henning.schild@siemens.com> > Subject: RE: [PATCH v2 2/2] wic: Update to the latest revision > > > > > -----Original Message----- > > From: Kanagarajan, Vijaikumar > > Sent: 13 June 2022 14:48 > > To: 'Henning Schild' <henning.schild@siemens.com> > > Cc: isar-users@googlegroups.com; jan.kiszka@siemens.com > > Subject: RE: [PATCH v2 2/2] wic: Update to the latest revision > > > > > > > > > -----Original Message----- > > > From: Henning Schild <henning.schild@siemens.com> > > > Sent: 13 June 2022 14:32 > > > To: Kanagarajan, Vijaikumar <Vijaikumar_Kanagarajan@mentor.com> > > > Cc: isar-users@googlegroups.com; jan.kiszka@siemens.com > > > Subject: Re: [PATCH v2 2/2] wic: Update to the latest revision > > > > > > Am Mon, 13 Jun 2022 13:24:01 +0530 > > > schrieb Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com>: > > > > > > > Update to the latest wic from OE-core. > > > > > > > > OE-core Revision: 06ed2aa93fc25a681e3a00ee120d9395b04845da > > > > > > > > Signed-off-by: Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com> > > > > --- > > > > RECIPE-API-CHANGELOG.md | 19 +++++++++++++++++++ > > > > scripts/lib/wic/help.py | 6 ++++++ > > > > scripts/lib/wic/ksparser.py | 1 + > > > > scripts/lib/wic/partition.py | 1 + > > > > scripts/lib/wic/plugins/imager/direct.py | 5 +++-- > > > > 5 files changed, 30 insertions(+), 2 deletions(-) > > > > > > > > diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md > > > index > > > > 2998824..436b204 100644 > > > > --- a/RECIPE-API-CHANGELOG.md > > > > +++ b/RECIPE-API-CHANGELOG.md > > > > @@ -406,3 +406,22 @@ groups are now created after all packages are > > > > installed. > > > > Changes in next > > > > --------------- > > > > + > > > > +### Wic adds / mountpoint to fstab > > > > + > > > > +In the older version of wic, any mount point named / is skipped > > > > +from > > > > adding +into the fstab entry. > > > > + > > > > +With the latest wic, this is not the case. / mount point, if any, > > > > +is > > > > added +to /etc/fstab for automount. > > > > + > > > > +Any wks file which assumed that / would be skipped from > > > > +/etc/fstab > > > > should +now be corrected. Otherwise, it might result in multiple / > > > > entries in /etc/fstab. +One by the image_configure_fstab function > > > > from image class and one by wic. + > > > > +Below is an example wks entry that might cause an issue. > > > > +Replace "part /" with "part" to avoid adding duplicate entry for /. > > > > > > I am not sure multiple entries would be an issue, i think the last > > > one will simply always win. It is good to have a note about that > > > change, but i doubt people really have to do anything. > > > And if anything has to be done, isar should drop the first one centrally. > > > > > > The fstab entry for root is purely for adding some weird mount > > > options, usually root is not mounted with fstab but with cmdline and > > > initrd. And not having / in fstab just makes that rootfs more > > > generic ... it can become part of nfs and next guy takes it as btrfs > > > while next guy > > takes it as ext4. > > > > > > Note is good, advise is not > > > > Noted. Will change the text accordingly in V3. > > Ok we cannot drop / mountpoint name(though optional). We would be lose the > ability to find out UUID of root using rootdev api. > This is used by some plugins. So the better way would be to use the --no-fstab- > update. Seems like we can set them per partition now. > > Thanks, > Vijai Kumar K > > > > > Thanks, > > Vijai Kumar K > > > > > > > > Henning > > > > > > > + > > > > +``` > > > > +part / --source rootfs --ondisk sda --fstype ext4 --label > > > > +platform > > > > --align 1024 --use-uuid +``` > > > > diff --git a/scripts/lib/wic/help.py b/scripts/lib/wic/help.py > > > > index > > > > 4ff7470..73e3380 100644 > > > > --- a/scripts/lib/wic/help.py > > > > +++ b/scripts/lib/wic/help.py > > > > @@ -940,6 +940,12 @@ DESCRIPTION > > > > quotes. If not specified, the default > > > > string is "defaults". > > > > > > > > + --fspassno: Specifies the order in which filesystem > > > > + checks > > > > are done > > > > + at boot time by fsck. See fs_passno > > > > + parameter > > > > of > > > > + fstab(5). This parameter will be copied > > > > + into > > > > the > > > > + /etc/fstab file of the installed system. If not > > > > + specified the default value of "0" will be used. > > > > + > > > > --label label: Specifies the label to give to the filesystem > > > > to be made on the partition. If the given > > > > label is already in use by another > > > > filesystem, diff --git a/scripts/lib/wic/ksparser.py > > > > b/scripts/lib/wic/ksparser.py index 0df9eb0..a49b7b9 100644 > > > > --- a/scripts/lib/wic/ksparser.py > > > > +++ b/scripts/lib/wic/ksparser.py > > > > @@ -155,6 +155,7 @@ class KickStart(): > > > > part.add_argument('--change-directory') > > > > part.add_argument("--extra-space", type=sizetype("M")) > > > > part.add_argument('--fsoptions', dest='fsopts') > > > > + part.add_argument('--fspassno', dest='fspassno') > > > > part.add_argument('--fstype', default='vfat', > > > > choices=('ext2', 'ext3', 'ext4', 'btrfs', > > > > 'squashfs', 'vfat', 'msdos', > > > > 'erofs', diff --git a/scripts/lib/wic/partition.py > > > > b/scripts/lib/wic/partition.py index 09e491d..e50871b 100644 > > > > --- a/scripts/lib/wic/partition.py > > > > +++ b/scripts/lib/wic/partition.py > > > > @@ -33,6 +33,7 @@ class Partition(): > > > > self.include_path = args.include_path > > > > self.change_directory = args.change_directory > > > > self.fsopts = args.fsopts > > > > + self.fspassno = args.fspassno > > > > self.fstype = args.fstype > > > > self.label = args.label > > > > self.use_label = args.use_label diff --git > > > > a/scripts/lib/wic/plugins/imager/direct.py > > > > b/scripts/lib/wic/plugins/imager/direct.py index 4d0b836..c44159b > > > > 100644 --- a/scripts/lib/wic/plugins/imager/direct.py > > > > +++ b/scripts/lib/wic/plugins/imager/direct.py > > > > @@ -117,7 +117,7 @@ class DirectPlugin(ImagerPlugin): > > > > updated = False > > > > for part in self.parts: > > > > if not part.realnum or not part.mountpoint \ > > > > - or part.mountpoint == "/" or not > > > > part.mountpoint.startswith('/'): > > > > + or not part.mountpoint.startswith('/'): > > > > continue > > > > > > > > if part.use_uuid: > > > > @@ -138,8 +138,9 @@ class DirectPlugin(ImagerPlugin): > > > > device_name = "/dev/%s%s%d" % (part.disk, prefix, > > > > part.realnum) > > > > opts = part.fsopts if part.fsopts else "defaults" > > > > + passno = part.fspassno if part.fspassno else "0" > > > > line = "\t".join([device_name, part.mountpoint, > > > > part.fstype, > > > > - opts, "0", "0"]) + "\n" > > > > + opts, "0", passno]) + "\n" > > > > > > > > fstab_lines.append(line) > > > > updated = True > > -- > You received this message because you are subscribed to the Google Groups > "isar-users" group. > To unsubscribe from this group and stop receiving emails from it, send an email > to isar-users+unsubscribe@googlegroups.com. > To view this discussion on the web visit > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.g > oogle.com%2Fd%2Fmsgid%2Fisar- > users%2F8babb01d7eae44e0b9faecad59cdc1d4%2540mentor.com&data= > 05%7C01%7Cfelix.moessbauer%40siemens.com%7C237b0db9bb724580ca5408 > da5024a3ad%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C6379104 > 15537105828%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjo > iV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata > =%2F%2F1Fxy%2B%2BEgDI4RfUEZ38qy1Q82esArdBuRftDuE1t6s%3D&rese > rved=0. ^ permalink raw reply [flat|nested] 22+ messages in thread
* RE: [PATCH v2 2/2] wic: Update to the latest revision 2022-09-26 1:40 ` Moessbauer, Felix @ 2022-09-26 4:53 ` Kanagarajan, Vijaikumar 2022-09-26 5:57 ` Moessbauer, Felix 0 siblings, 1 reply; 22+ messages in thread From: Kanagarajan, Vijaikumar @ 2022-09-26 4:53 UTC (permalink / raw) To: Moessbauer, Felix (T CED SES-DE), Schild, Henning (T CED SES-DE) Cc: Kiszka, Jan (T CED), isar-users > -----Original Message----- > From: Moessbauer, Felix (T CED SES-DE) <felix.moessbauer@siemens.com> > Sent: 26 September 2022 07:11 > To: Kanagarajan, Vijaikumar <Vijaikumar_Kanagarajan@mentor.com>; > Schild, Henning (T CED SES-DE) <henning.schild@siemens.com> > Cc: Kiszka, Jan (T CED) <jan.kiszka@siemens.com>; isar- > users@googlegroups.com > Subject: RE: [PATCH v2 2/2] wic: Update to the latest revision > > Hi Vijai & Henning, > > Are there any news on this? > I just discovered that still many (if not all) of our images are affected by this > bug. Hi Felix, Unfortunately not. If we already have the latest wic, --no-fstab-update[1] might help.(untested). [1] https://github.com/openembedded/openembedded-core/commit/ab4c95af8ecd15dc136194ab761afae756db5803 Thanks, Vijai Kumar K > > While it is not serios, having error messages in the journal is still pretty bad: > systemd-fstab-generator[1491]: Failed to create unit file > /run/systemd/generator/-.mount, as it already exists. Duplicate entry in > /etc/fstab? > > Felix > > > -----Original Message----- > > From: isar-users@googlegroups.com <isar-users@googlegroups.com> On > > Behalf Of Kanagarajan, Vijaikumar > > Sent: Friday, June 17, 2022 1:46 PM > > To: isar-users@googlegroups.com > > Cc: Kiszka, Jan (T CED) <jan.kiszka@siemens.com>; Schild, Henning (T > > CED SES- > > DE) <henning.schild@siemens.com> > > Subject: RE: [PATCH v2 2/2] wic: Update to the latest revision > > > > > > > > > -----Original Message----- > > > From: Kanagarajan, Vijaikumar > > > Sent: 13 June 2022 14:48 > > > To: 'Henning Schild' <henning.schild@siemens.com> > > > Cc: isar-users@googlegroups.com; jan.kiszka@siemens.com > > > Subject: RE: [PATCH v2 2/2] wic: Update to the latest revision > > > > > > > > > > > > > -----Original Message----- > > > > From: Henning Schild <henning.schild@siemens.com> > > > > Sent: 13 June 2022 14:32 > > > > To: Kanagarajan, Vijaikumar <Vijaikumar_Kanagarajan@mentor.com> > > > > Cc: isar-users@googlegroups.com; jan.kiszka@siemens.com > > > > Subject: Re: [PATCH v2 2/2] wic: Update to the latest revision > > > > > > > > Am Mon, 13 Jun 2022 13:24:01 +0530 schrieb Vijai Kumar K > > > > <Vijaikumar_Kanagarajan@mentor.com>: > > > > > > > > > Update to the latest wic from OE-core. > > > > > > > > > > OE-core Revision: 06ed2aa93fc25a681e3a00ee120d9395b04845da > > > > > > > > > > Signed-off-by: Vijai Kumar K > <Vijaikumar_Kanagarajan@mentor.com> > > > > > --- > > > > > RECIPE-API-CHANGELOG.md | 19 +++++++++++++++++++ > > > > > scripts/lib/wic/help.py | 6 ++++++ > > > > > scripts/lib/wic/ksparser.py | 1 + > > > > > scripts/lib/wic/partition.py | 1 + > > > > > scripts/lib/wic/plugins/imager/direct.py | 5 +++-- > > > > > 5 files changed, 30 insertions(+), 2 deletions(-) > > > > > > > > > > diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API- > CHANGELOG.md > > > > index > > > > > 2998824..436b204 100644 > > > > > --- a/RECIPE-API-CHANGELOG.md > > > > > +++ b/RECIPE-API-CHANGELOG.md > > > > > @@ -406,3 +406,22 @@ groups are now created after all packages > > > > > are installed. > > > > > Changes in next > > > > > --------------- > > > > > + > > > > > +### Wic adds / mountpoint to fstab > > > > > + > > > > > +In the older version of wic, any mount point named / is skipped > > > > > +from > > > > > adding +into the fstab entry. > > > > > + > > > > > +With the latest wic, this is not the case. / mount point, if > > > > > +any, is > > > > > added +to /etc/fstab for automount. > > > > > + > > > > > +Any wks file which assumed that / would be skipped from > > > > > +/etc/fstab > > > > > should +now be corrected. Otherwise, it might result in multiple > > > > > / entries in /etc/fstab. +One by the image_configure_fstab > > > > > function from image class and one by wic. + > > > > > +Below is an example wks entry that might cause an issue. > > > > > +Replace "part /" with "part" to avoid adding duplicate entry for /. > > > > > > > > I am not sure multiple entries would be an issue, i think the last > > > > one will simply always win. It is good to have a note about that > > > > change, but i doubt people really have to do anything. > > > > And if anything has to be done, isar should drop the first one centrally. > > > > > > > > The fstab entry for root is purely for adding some weird mount > > > > options, usually root is not mounted with fstab but with cmdline > > > > and initrd. And not having / in fstab just makes that rootfs more > > > > generic ... it can become part of nfs and next guy takes it as > > > > btrfs while next guy > > > takes it as ext4. > > > > > > > > Note is good, advise is not > > > > > > Noted. Will change the text accordingly in V3. > > > > Ok we cannot drop / mountpoint name(though optional). We would be > lose > > the ability to find out UUID of root using rootdev api. > > This is used by some plugins. So the better way would be to use the > > --no-fstab- update. Seems like we can set them per partition now. > > > > Thanks, > > Vijai Kumar K > > > > > > > > Thanks, > > > Vijai Kumar K > > > > > > > > > > > Henning > > > > > > > > > + > > > > > +``` > > > > > +part / --source rootfs --ondisk sda --fstype ext4 --label > > > > > +platform > > > > > --align 1024 --use-uuid +``` > > > > > diff --git a/scripts/lib/wic/help.py b/scripts/lib/wic/help.py > > > > > index > > > > > 4ff7470..73e3380 100644 > > > > > --- a/scripts/lib/wic/help.py > > > > > +++ b/scripts/lib/wic/help.py > > > > > @@ -940,6 +940,12 @@ DESCRIPTION > > > > > quotes. If not specified, the default > > > > > string is "defaults". > > > > > > > > > > + --fspassno: Specifies the order in which filesystem > > > > > + checks > > > > > are done > > > > > + at boot time by fsck. See fs_passno > > > > > + parameter > > > > > of > > > > > + fstab(5). This parameter will be copied > > > > > + into > > > > > the > > > > > + /etc/fstab file of the installed system. If not > > > > > + specified the default value of "0" will be used. > > > > > + > > > > > --label label: Specifies the label to give to the filesystem > > > > > to be made on the partition. If the given > > > > > label is already in use by another > > > > > filesystem, diff --git a/scripts/lib/wic/ksparser.py > > > > > b/scripts/lib/wic/ksparser.py index 0df9eb0..a49b7b9 100644 > > > > > --- a/scripts/lib/wic/ksparser.py > > > > > +++ b/scripts/lib/wic/ksparser.py > > > > > @@ -155,6 +155,7 @@ class KickStart(): > > > > > part.add_argument('--change-directory') > > > > > part.add_argument("--extra-space", type=sizetype("M")) > > > > > part.add_argument('--fsoptions', dest='fsopts') > > > > > + part.add_argument('--fspassno', dest='fspassno') > > > > > part.add_argument('--fstype', default='vfat', > > > > > choices=('ext2', 'ext3', 'ext4', 'btrfs', > > > > > 'squashfs', 'vfat', 'msdos', > > > > > 'erofs', diff --git a/scripts/lib/wic/partition.py > > > > > b/scripts/lib/wic/partition.py index 09e491d..e50871b 100644 > > > > > --- a/scripts/lib/wic/partition.py > > > > > +++ b/scripts/lib/wic/partition.py > > > > > @@ -33,6 +33,7 @@ class Partition(): > > > > > self.include_path = args.include_path > > > > > self.change_directory = args.change_directory > > > > > self.fsopts = args.fsopts > > > > > + self.fspassno = args.fspassno > > > > > self.fstype = args.fstype > > > > > self.label = args.label > > > > > self.use_label = args.use_label diff --git > > > > > a/scripts/lib/wic/plugins/imager/direct.py > > > > > b/scripts/lib/wic/plugins/imager/direct.py index > > > > > 4d0b836..c44159b > > > > > 100644 --- a/scripts/lib/wic/plugins/imager/direct.py > > > > > +++ b/scripts/lib/wic/plugins/imager/direct.py > > > > > @@ -117,7 +117,7 @@ class DirectPlugin(ImagerPlugin): > > > > > updated = False > > > > > for part in self.parts: > > > > > if not part.realnum or not part.mountpoint \ > > > > > - or part.mountpoint == "/" or not > > > > > part.mountpoint.startswith('/'): > > > > > + or not part.mountpoint.startswith('/'): > > > > > continue > > > > > > > > > > if part.use_uuid: > > > > > @@ -138,8 +138,9 @@ class DirectPlugin(ImagerPlugin): > > > > > device_name = "/dev/%s%s%d" % (part.disk, > > > > > prefix, > > > > > part.realnum) > > > > > opts = part.fsopts if part.fsopts else "defaults" > > > > > + passno = part.fspassno if part.fspassno else "0" > > > > > line = "\t".join([device_name, part.mountpoint, > > > > > part.fstype, > > > > > - opts, "0", "0"]) + "\n" > > > > > + opts, "0", passno]) + "\n" > > > > > > > > > > fstab_lines.append(line) > > > > > updated = True > > > > -- > > You received this message because you are subscribed to the Google > > Groups "isar-users" group. > > To unsubscribe from this group and stop receiving emails from it, send > > an email to isar-users+unsubscribe@googlegroups.com. > > To view this discussion on the web visit > > > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgrou > > ps.g > > oogle.com%2Fd%2Fmsgid%2Fisar- > > > users%2F8babb01d7eae44e0b9faecad59cdc1d4%2540mentor.com&dat > a= > > > 05%7C01%7Cfelix.moessbauer%40siemens.com%7C237b0db9bb724580ca540 > 8 > > > da5024a3ad%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C6379104 > > > 15537105828%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQ > Ijo > > > iV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sda > ta > > > =%2F%2F1Fxy%2B%2BEgDI4RfUEZ38qy1Q82esArdBuRftDuE1t6s%3D&re > se > > rved=0. ^ permalink raw reply [flat|nested] 22+ messages in thread
* RE: [PATCH v2 2/2] wic: Update to the latest revision 2022-09-26 4:53 ` Kanagarajan, Vijaikumar @ 2022-09-26 5:57 ` Moessbauer, Felix 2022-09-26 7:46 ` Henning Schild 0 siblings, 1 reply; 22+ messages in thread From: Moessbauer, Felix @ 2022-09-26 5:57 UTC (permalink / raw) To: Kanagarajan, Vijaikumar, Schild, Henning, jan.kiszka; +Cc: isar-users > -----Original Message----- > From: Kanagarajan, Vijaikumar <Vijaikumar_Kanagarajan@mentor.com> > Sent: Monday, September 26, 2022 12:53 PM > To: Moessbauer, Felix (T CED SES-DE) <felix.moessbauer@siemens.com>; Schild, > Henning (T CED SES-DE) <henning.schild@siemens.com> > Cc: Kiszka, Jan (T CED) <jan.kiszka@siemens.com>; isar- > users@googlegroups.com > Subject: RE: [PATCH v2 2/2] wic: Update to the latest revision > > > > > -----Original Message----- > > From: Moessbauer, Felix (T CED SES-DE) <felix.moessbauer@siemens.com> > > Sent: 26 September 2022 07:11 > > To: Kanagarajan, Vijaikumar <Vijaikumar_Kanagarajan@mentor.com>; > > Schild, Henning (T CED SES-DE) <henning.schild@siemens.com> > > Cc: Kiszka, Jan (T CED) <jan.kiszka@siemens.com>; isar- > > users@googlegroups.com > > Subject: RE: [PATCH v2 2/2] wic: Update to the latest revision > > > > Hi Vijai & Henning, > > > > Are there any news on this? > > I just discovered that still many (if not all) of our images are > > affected by this bug. > > Hi Felix, > > Unfortunately not. If we already have the latest wic, --no-fstab-update[1] might > help.(untested). We already have a wic version with this option, but it doesn't look right to add it to the root partition. Instead this option can be used for partitions that should not show up in the fstab at all. Indeed, the fstab entry has to be set by WIC to set settings like UUID and filesystem. But this requires that we either remove the entry set by ISAR in image_configure_fstab, or we remove the entry after generating the rootfs, but before WIC. Opinions? @Kiszka, Jan (T CED) Felix > > Thanks, > Vijai Kumar K > > > > > While it is not serios, having error messages in the journal is still pretty bad: > > systemd-fstab-generator[1491]: Failed to create unit file > > /run/systemd/generator/-.mount, as it already exists. Duplicate entry > > in /etc/fstab? > > > > Felix > > > > > -----Original Message----- > > > From: isar-users@googlegroups.com <isar-users@googlegroups.com> On > > > Behalf Of Kanagarajan, Vijaikumar > > > Sent: Friday, June 17, 2022 1:46 PM > > > To: isar-users@googlegroups.com > > > Cc: Kiszka, Jan (T CED) <jan.kiszka@siemens.com>; Schild, Henning (T > > > CED SES- > > > DE) <henning.schild@siemens.com> > > > Subject: RE: [PATCH v2 2/2] wic: Update to the latest revision > > > > > > > > > > > > > -----Original Message----- > > > > From: Kanagarajan, Vijaikumar > > > > Sent: 13 June 2022 14:48 > > > > To: 'Henning Schild' <henning.schild@siemens.com> > > > > Cc: isar-users@googlegroups.com; jan.kiszka@siemens.com > > > > Subject: RE: [PATCH v2 2/2] wic: Update to the latest revision > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > From: Henning Schild <henning.schild@siemens.com> > > > > > Sent: 13 June 2022 14:32 > > > > > To: Kanagarajan, Vijaikumar <Vijaikumar_Kanagarajan@mentor.com> > > > > > Cc: isar-users@googlegroups.com; jan.kiszka@siemens.com > > > > > Subject: Re: [PATCH v2 2/2] wic: Update to the latest revision > > > > > > > > > > Am Mon, 13 Jun 2022 13:24:01 +0530 schrieb Vijai Kumar K > > > > > <Vijaikumar_Kanagarajan@mentor.com>: > > > > > > > > > > > Update to the latest wic from OE-core. > > > > > > > > > > > > OE-core Revision: 06ed2aa93fc25a681e3a00ee120d9395b04845da > > > > > > > > > > > > Signed-off-by: Vijai Kumar K > > <Vijaikumar_Kanagarajan@mentor.com> > > > > > > --- > > > > > > RECIPE-API-CHANGELOG.md | 19 +++++++++++++++++++ > > > > > > scripts/lib/wic/help.py | 6 ++++++ > > > > > > scripts/lib/wic/ksparser.py | 1 + > > > > > > scripts/lib/wic/partition.py | 1 + > > > > > > scripts/lib/wic/plugins/imager/direct.py | 5 +++-- > > > > > > 5 files changed, 30 insertions(+), 2 deletions(-) > > > > > > > > > > > > diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API- > > CHANGELOG.md > > > > > index > > > > > > 2998824..436b204 100644 > > > > > > --- a/RECIPE-API-CHANGELOG.md > > > > > > +++ b/RECIPE-API-CHANGELOG.md > > > > > > @@ -406,3 +406,22 @@ groups are now created after all packages > > > > > > are installed. > > > > > > Changes in next > > > > > > --------------- > > > > > > + > > > > > > +### Wic adds / mountpoint to fstab > > > > > > + > > > > > > +In the older version of wic, any mount point named / is > > > > > > +skipped from > > > > > > adding +into the fstab entry. > > > > > > + > > > > > > +With the latest wic, this is not the case. / mount point, if > > > > > > +any, is > > > > > > added +to /etc/fstab for automount. > > > > > > + > > > > > > +Any wks file which assumed that / would be skipped from > > > > > > +/etc/fstab > > > > > > should +now be corrected. Otherwise, it might result in > > > > > > multiple / entries in /etc/fstab. +One by the > > > > > > image_configure_fstab function from image class and one by > > > > > > wic. + > > > > > > +Below is an example wks entry that might cause an issue. > > > > > > +Replace "part /" with "part" to avoid adding duplicate entry for /. > > > > > > > > > > I am not sure multiple entries would be an issue, i think the > > > > > last one will simply always win. It is good to have a note about > > > > > that change, but i doubt people really have to do anything. > > > > > And if anything has to be done, isar should drop the first one centrally. > > > > > > > > > > The fstab entry for root is purely for adding some weird mount > > > > > options, usually root is not mounted with fstab but with cmdline > > > > > and initrd. And not having / in fstab just makes that rootfs > > > > > more generic ... it can become part of nfs and next guy takes it > > > > > as btrfs while next guy > > > > takes it as ext4. > > > > > > > > > > Note is good, advise is not > > > > > > > > Noted. Will change the text accordingly in V3. > > > > > > Ok we cannot drop / mountpoint name(though optional). We would be > > lose > > > the ability to find out UUID of root using rootdev api. > > > This is used by some plugins. So the better way would be to use the > > > --no-fstab- update. Seems like we can set them per partition now. > > > > > > Thanks, > > > Vijai Kumar K > > > > > > > > > > > Thanks, > > > > Vijai Kumar K > > > > > > > > > > > > > > Henning > > > > > > > > > > > + > > > > > > +``` > > > > > > +part / --source rootfs --ondisk sda --fstype ext4 --label > > > > > > +platform > > > > > > --align 1024 --use-uuid +``` > > > > > > diff --git a/scripts/lib/wic/help.py b/scripts/lib/wic/help.py > > > > > > index > > > > > > 4ff7470..73e3380 100644 > > > > > > --- a/scripts/lib/wic/help.py > > > > > > +++ b/scripts/lib/wic/help.py > > > > > > @@ -940,6 +940,12 @@ DESCRIPTION > > > > > > quotes. If not specified, the default > > > > > > string is "defaults". > > > > > > > > > > > > + --fspassno: Specifies the order in which filesystem > > > > > > + checks > > > > > > are done > > > > > > + at boot time by fsck. See fs_passno > > > > > > + parameter > > > > > > of > > > > > > + fstab(5). This parameter will be copied > > > > > > + into > > > > > > the > > > > > > + /etc/fstab file of the installed system. If not > > > > > > + specified the default value of "0" will be used. > > > > > > + > > > > > > --label label: Specifies the label to give to the filesystem > > > > > > to be made on the partition. If the given > > > > > > label is already in use by another > > > > > > filesystem, diff --git a/scripts/lib/wic/ksparser.py > > > > > > b/scripts/lib/wic/ksparser.py index 0df9eb0..a49b7b9 100644 > > > > > > --- a/scripts/lib/wic/ksparser.py > > > > > > +++ b/scripts/lib/wic/ksparser.py > > > > > > @@ -155,6 +155,7 @@ class KickStart(): > > > > > > part.add_argument('--change-directory') > > > > > > part.add_argument("--extra-space", type=sizetype("M")) > > > > > > part.add_argument('--fsoptions', dest='fsopts') > > > > > > + part.add_argument('--fspassno', dest='fspassno') > > > > > > part.add_argument('--fstype', default='vfat', > > > > > > choices=('ext2', 'ext3', 'ext4', 'btrfs', > > > > > > 'squashfs', 'vfat', > > > > > > 'msdos', 'erofs', diff --git a/scripts/lib/wic/partition.py > > > > > > b/scripts/lib/wic/partition.py index 09e491d..e50871b 100644 > > > > > > --- a/scripts/lib/wic/partition.py > > > > > > +++ b/scripts/lib/wic/partition.py > > > > > > @@ -33,6 +33,7 @@ class Partition(): > > > > > > self.include_path = args.include_path > > > > > > self.change_directory = args.change_directory > > > > > > self.fsopts = args.fsopts > > > > > > + self.fspassno = args.fspassno > > > > > > self.fstype = args.fstype > > > > > > self.label = args.label > > > > > > self.use_label = args.use_label diff --git > > > > > > a/scripts/lib/wic/plugins/imager/direct.py > > > > > > b/scripts/lib/wic/plugins/imager/direct.py index > > > > > > 4d0b836..c44159b > > > > > > 100644 --- a/scripts/lib/wic/plugins/imager/direct.py > > > > > > +++ b/scripts/lib/wic/plugins/imager/direct.py > > > > > > @@ -117,7 +117,7 @@ class DirectPlugin(ImagerPlugin): > > > > > > updated = False > > > > > > for part in self.parts: > > > > > > if not part.realnum or not part.mountpoint \ > > > > > > - or part.mountpoint == "/" or not > > > > > > part.mountpoint.startswith('/'): > > > > > > + or not part.mountpoint.startswith('/'): > > > > > > continue > > > > > > > > > > > > if part.use_uuid: > > > > > > @@ -138,8 +138,9 @@ class DirectPlugin(ImagerPlugin): > > > > > > device_name = "/dev/%s%s%d" % (part.disk, > > > > > > prefix, > > > > > > part.realnum) > > > > > > opts = part.fsopts if part.fsopts else "defaults" > > > > > > + passno = part.fspassno if part.fspassno else "0" > > > > > > line = "\t".join([device_name, part.mountpoint, > > > > > > part.fstype, > > > > > > - opts, "0", "0"]) + "\n" > > > > > > + opts, "0", passno]) + "\n" > > > > > > > > > > > > fstab_lines.append(line) > > > > > > updated = True > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "isar-users" group. > > > To unsubscribe from this group and stop receiving emails from it, > > > send an email to isar-users+unsubscribe@googlegroups.com. > > > To view this discussion on the web visit > > > > > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgrou > > > ps.g > > > oogle.com%2Fd%2Fmsgid%2Fisar- > > > > > users%2F8babb01d7eae44e0b9faecad59cdc1d4%2540mentor.com&dat > > a= > > > > > 05%7C01%7Cfelix.moessbauer%40siemens.com%7C237b0db9bb724580ca540 > > 8 > > > > > > da5024a3ad%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C6379104 > > > > > 15537105828%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQ > > Ijo > > > > > iV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sda > > ta > > > > > =%2F%2F1Fxy%2B%2BEgDI4RfUEZ38qy1Q82esArdBuRftDuE1t6s%3D&re > > se > > > rved=0. ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH v2 2/2] wic: Update to the latest revision 2022-09-26 5:57 ` Moessbauer, Felix @ 2022-09-26 7:46 ` Henning Schild 2022-09-26 7:56 ` Kanagarajan, Vijaikumar 0 siblings, 1 reply; 22+ messages in thread From: Henning Schild @ 2022-09-26 7:46 UTC (permalink / raw) To: felix.moessbauer, Vijaikumar_Kanagarajan, jan.kiszka; +Cc: isar-users Am Mon, 26 Sep 2022 07:57:41 +0200 schrieb "Moessbauer, Felix (T CED SES-DE)" <felix.moessbauer@siemens.com>: > > -----Original Message----- > > From: Kanagarajan, Vijaikumar <Vijaikumar_Kanagarajan@mentor.com> > > Sent: Monday, September 26, 2022 12:53 PM > > To: Moessbauer, Felix (T CED SES-DE) > > <felix.moessbauer@siemens.com>; Schild, Henning (T CED SES-DE) > > <henning.schild@siemens.com> Cc: Kiszka, Jan (T CED) > > <jan.kiszka@siemens.com>; isar- users@googlegroups.com > > Subject: RE: [PATCH v2 2/2] wic: Update to the latest revision > > > > > > > > > -----Original Message----- > > > From: Moessbauer, Felix (T CED SES-DE) > > > <felix.moessbauer@siemens.com> Sent: 26 September 2022 07:11 > > > To: Kanagarajan, Vijaikumar <Vijaikumar_Kanagarajan@mentor.com>; > > > Schild, Henning (T CED SES-DE) <henning.schild@siemens.com> > > > Cc: Kiszka, Jan (T CED) <jan.kiszka@siemens.com>; isar- > > > users@googlegroups.com > > > Subject: RE: [PATCH v2 2/2] wic: Update to the latest revision > > > > > > Hi Vijai & Henning, > > > > > > Are there any news on this? > > > I just discovered that still many (if not all) of our images are > > > affected by this bug. > > > > Hi Felix, > > > > Unfortunately not. If we already have the latest wic, > > --no-fstab-update[1] might help.(untested). > > We already have a wic version with this option, but it doesn't look > right to add it to the root partition. Instead this option can be > used for partitions that should not show up in the fstab at all. We do want the one from WIC. It is only coming since we patched WIC to support /. > Indeed, the fstab entry has to be set by WIC to set settings like > UUID and filesystem. But this requires that we either remove the > entry set by ISAR in image_configure_fstab, or we remove the entry > after generating the rootfs, but before WIC. I think we should try and remove the entry from that template. It should not be required. If we decide to solve the problem somewhere in/with WIC, we should do that upstream in OE. They have the same "we now have two" problem, where wic adds on top of a template instead of replacing the original. Henning > Opinions? @Kiszka, Jan (T CED) > > Felix > > > > > Thanks, > > Vijai Kumar K > > > > > > > > While it is not serios, having error messages in the journal is > > > still pretty bad: systemd-fstab-generator[1491]: Failed to create > > > unit file /run/systemd/generator/-.mount, as it already exists. > > > Duplicate entry in /etc/fstab? > > > > > > Felix > > > > > > > -----Original Message----- > > > > From: isar-users@googlegroups.com <isar-users@googlegroups.com> > > > > On Behalf Of Kanagarajan, Vijaikumar > > > > Sent: Friday, June 17, 2022 1:46 PM > > > > To: isar-users@googlegroups.com > > > > Cc: Kiszka, Jan (T CED) <jan.kiszka@siemens.com>; Schild, > > > > Henning (T CED SES- > > > > DE) <henning.schild@siemens.com> > > > > Subject: RE: [PATCH v2 2/2] wic: Update to the latest revision > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > From: Kanagarajan, Vijaikumar > > > > > Sent: 13 June 2022 14:48 > > > > > To: 'Henning Schild' <henning.schild@siemens.com> > > > > > Cc: isar-users@googlegroups.com; jan.kiszka@siemens.com > > > > > Subject: RE: [PATCH v2 2/2] wic: Update to the latest revision > > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > From: Henning Schild <henning.schild@siemens.com> > > > > > > Sent: 13 June 2022 14:32 > > > > > > To: Kanagarajan, Vijaikumar > > > > > > <Vijaikumar_Kanagarajan@mentor.com> Cc: > > > > > > isar-users@googlegroups.com; jan.kiszka@siemens.com > > > > > > Subject: Re: [PATCH v2 2/2] wic: Update to the latest > > > > > > revision > > > > > > > > > > > > Am Mon, 13 Jun 2022 13:24:01 +0530 schrieb Vijai Kumar K > > > > > > <Vijaikumar_Kanagarajan@mentor.com>: > > > > > > > > > > > > > Update to the latest wic from OE-core. > > > > > > > > > > > > > > OE-core Revision: 06ed2aa93fc25a681e3a00ee120d9395b04845da > > > > > > > > > > > > > > Signed-off-by: Vijai Kumar K > > > <Vijaikumar_Kanagarajan@mentor.com> > > > > > > > --- > > > > > > > RECIPE-API-CHANGELOG.md | 19 > > > > > > > +++++++++++++++++++ scripts/lib/wic/help.py > > > > > > > | 6 ++++++ scripts/lib/wic/ksparser.py | > > > > > > > 1 + scripts/lib/wic/partition.py | 1 + > > > > > > > scripts/lib/wic/plugins/imager/direct.py | 5 +++-- > > > > > > > 5 files changed, 30 insertions(+), 2 deletions(-) > > > > > > > > > > > > > > diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API- > > > CHANGELOG.md > > > > > > index > > > > > > > 2998824..436b204 100644 > > > > > > > --- a/RECIPE-API-CHANGELOG.md > > > > > > > +++ b/RECIPE-API-CHANGELOG.md > > > > > > > @@ -406,3 +406,22 @@ groups are now created after all > > > > > > > packages are installed. > > > > > > > Changes in next > > > > > > > --------------- > > > > > > > + > > > > > > > +### Wic adds / mountpoint to fstab > > > > > > > + > > > > > > > +In the older version of wic, any mount point named / is > > > > > > > +skipped from > > > > > > > adding +into the fstab entry. > > > > > > > + > > > > > > > +With the latest wic, this is not the case. / mount > > > > > > > point, if +any, is > > > > > > > added +to /etc/fstab for automount. > > > > > > > + > > > > > > > +Any wks file which assumed that / would be skipped from > > > > > > > +/etc/fstab > > > > > > > should +now be corrected. Otherwise, it might result in > > > > > > > multiple / entries in /etc/fstab. +One by the > > > > > > > image_configure_fstab function from image class and one by > > > > > > > wic. + > > > > > > > +Below is an example wks entry that might cause an issue. > > > > > > > +Replace "part /" with "part" to avoid adding duplicate > > > > > > > entry for /. > > > > > > > > > > > > I am not sure multiple entries would be an issue, i think > > > > > > the last one will simply always win. It is good to have a > > > > > > note about that change, but i doubt people really have to > > > > > > do anything. And if anything has to be done, isar should > > > > > > drop the first one centrally. > > > > > > > > > > > > The fstab entry for root is purely for adding some weird > > > > > > mount options, usually root is not mounted with fstab but > > > > > > with cmdline and initrd. And not having / in fstab just > > > > > > makes that rootfs more generic ... it can become part of > > > > > > nfs and next guy takes it as btrfs while next guy > > > > > takes it as ext4. > > > > > > > > > > > > Note is good, advise is not > > > > > > > > > > Noted. Will change the text accordingly in V3. > > > > > > > > Ok we cannot drop / mountpoint name(though optional). We would > > > > be > > > lose > > > > the ability to find out UUID of root using rootdev api. > > > > This is used by some plugins. So the better way would be to use > > > > the --no-fstab- update. Seems like we can set them per > > > > partition now. > > > > > > > > Thanks, > > > > Vijai Kumar K > > > > > > > > > > > > > > Thanks, > > > > > Vijai Kumar K > > > > > > > > > > > > > > > > > Henning > > > > > > > > > > > > > + > > > > > > > +``` > > > > > > > +part / --source rootfs --ondisk sda --fstype ext4 --label > > > > > > > +platform > > > > > > > --align 1024 --use-uuid +``` > > > > > > > diff --git a/scripts/lib/wic/help.py > > > > > > > b/scripts/lib/wic/help.py index > > > > > > > 4ff7470..73e3380 100644 > > > > > > > --- a/scripts/lib/wic/help.py > > > > > > > +++ b/scripts/lib/wic/help.py > > > > > > > @@ -940,6 +940,12 @@ DESCRIPTION > > > > > > > quotes. If not specified, the > > > > > > > default string is "defaults". > > > > > > > > > > > > > > + --fspassno: Specifies the order in which > > > > > > > filesystem > > > > > > > + checks > > > > > > > are done > > > > > > > + at boot time by fsck. See fs_passno > > > > > > > + parameter > > > > > > > of > > > > > > > + fstab(5). This parameter will be > > > > > > > copied > > > > > > > + into > > > > > > > the > > > > > > > + /etc/fstab file of the installed > > > > > > > system. If not > > > > > > > + specified the default value of "0" > > > > > > > will be used. + > > > > > > > --label label: Specifies the label to give to > > > > > > > the filesystem to be made on the partition. If the given > > > > > > > label is already in use by > > > > > > > another filesystem, diff --git > > > > > > > a/scripts/lib/wic/ksparser.py > > > > > > > b/scripts/lib/wic/ksparser.py index 0df9eb0..a49b7b9 > > > > > > > 100644 --- a/scripts/lib/wic/ksparser.py +++ > > > > > > > b/scripts/lib/wic/ksparser.py @@ -155,6 +155,7 @@ class > > > > > > > KickStart(): part.add_argument('--change-directory') > > > > > > > part.add_argument("--extra-space", > > > > > > > type=sizetype("M")) part.add_argument('--fsoptions', > > > > > > > dest='fsopts') > > > > > > > + part.add_argument('--fspassno', dest='fspassno') > > > > > > > part.add_argument('--fstype', default='vfat', > > > > > > > choices=('ext2', 'ext3', > > > > > > > 'ext4', 'btrfs', 'squashfs', 'vfat', > > > > > > > 'msdos', 'erofs', diff --git > > > > > > > a/scripts/lib/wic/partition.py > > > > > > > b/scripts/lib/wic/partition.py index 09e491d..e50871b > > > > > > > 100644 --- a/scripts/lib/wic/partition.py +++ > > > > > > > b/scripts/lib/wic/partition.py @@ -33,6 +33,7 @@ class > > > > > > > Partition(): self.include_path = args.include_path > > > > > > > self.change_directory = args.change_directory > > > > > > > self.fsopts = args.fsopts > > > > > > > + self.fspassno = args.fspassno > > > > > > > self.fstype = args.fstype > > > > > > > self.label = args.label > > > > > > > self.use_label = args.use_label diff --git > > > > > > > a/scripts/lib/wic/plugins/imager/direct.py > > > > > > > b/scripts/lib/wic/plugins/imager/direct.py index > > > > > > > 4d0b836..c44159b > > > > > > > 100644 --- a/scripts/lib/wic/plugins/imager/direct.py > > > > > > > +++ b/scripts/lib/wic/plugins/imager/direct.py > > > > > > > @@ -117,7 +117,7 @@ class DirectPlugin(ImagerPlugin): > > > > > > > updated = False > > > > > > > for part in self.parts: > > > > > > > if not part.realnum or not part.mountpoint \ > > > > > > > - or part.mountpoint == "/" or not > > > > > > > part.mountpoint.startswith('/'): > > > > > > > + or not part.mountpoint.startswith('/'): > > > > > > > continue > > > > > > > > > > > > > > if part.use_uuid: > > > > > > > @@ -138,8 +138,9 @@ class DirectPlugin(ImagerPlugin): > > > > > > > device_name = "/dev/%s%s%d" % (part.disk, > > > > > > > prefix, > > > > > > > part.realnum) > > > > > > > opts = part.fsopts if part.fsopts else > > > > > > > "defaults" > > > > > > > + passno = part.fspassno if part.fspassno else > > > > > > > "0" line = "\t".join([device_name, part.mountpoint, > > > > > > > part.fstype, > > > > > > > - opts, "0", "0"]) + "\n" > > > > > > > + opts, "0", passno]) + "\n" > > > > > > > > > > > > > > fstab_lines.append(line) > > > > > > > updated = True > > > > > > > > -- > > > > You received this message because you are subscribed to the > > > > Google Groups "isar-users" group. > > > > To unsubscribe from this group and stop receiving emails from > > > > it, send an email to isar-users+unsubscribe@googlegroups.com. > > > > To view this discussion on the web visit > > > > > > > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgrou > > > > > > > ps.g > > > > oogle.com%2Fd%2Fmsgid%2Fisar- > > > > > > > users%2F8babb01d7eae44e0b9faecad59cdc1d4%2540mentor.com&dat > > > a= > > > > > > > 05%7C01%7Cfelix.moessbauer%40siemens.com%7C237b0db9bb724580ca540 > > > 8 > > > > > > > > > da5024a3ad%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C6379104 > > > > > > > 15537105828%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQ > > > Ijo > > > > > > > iV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sda > > > ta > > > > > > > =%2F%2F1Fxy%2B%2BEgDI4RfUEZ38qy1Q82esArdBuRftDuE1t6s%3D&re > > > se > > > > rved=0. ^ permalink raw reply [flat|nested] 22+ messages in thread
* RE: [PATCH v2 2/2] wic: Update to the latest revision 2022-09-26 7:46 ` Henning Schild @ 2022-09-26 7:56 ` Kanagarajan, Vijaikumar 2022-09-26 8:02 ` Moessbauer, Felix 2022-09-26 12:34 ` Henning Schild 0 siblings, 2 replies; 22+ messages in thread From: Kanagarajan, Vijaikumar @ 2022-09-26 7:56 UTC (permalink / raw) To: Henning Schild, felix.moessbauer, jan.kiszka; +Cc: isar-users > -----Original Message----- > From: Henning Schild <henning.schild@siemens.com> > Sent: 26 September 2022 13:17 > To: felix.moessbauer@siemens.com; Kanagarajan, Vijaikumar > <Vijaikumar_Kanagarajan@mentor.com>; jan.kiszka@siemens.com > Cc: isar-users@googlegroups.com > Subject: Re: [PATCH v2 2/2] wic: Update to the latest revision > > Am Mon, 26 Sep 2022 07:57:41 +0200 > schrieb "Moessbauer, Felix (T CED SES-DE)" > <felix.moessbauer@siemens.com>: > > > > -----Original Message----- > > > From: Kanagarajan, Vijaikumar <Vijaikumar_Kanagarajan@mentor.com> > > > Sent: Monday, September 26, 2022 12:53 PM > > > To: Moessbauer, Felix (T CED SES-DE) > <felix.moessbauer@siemens.com>; > > > Schild, Henning (T CED SES-DE) <henning.schild@siemens.com> Cc: > > > Kiszka, Jan (T CED) <jan.kiszka@siemens.com>; isar- > > > users@googlegroups.com > > > Subject: RE: [PATCH v2 2/2] wic: Update to the latest revision > > > > > > > > > > > > > -----Original Message----- > > > > From: Moessbauer, Felix (T CED SES-DE) > > > > <felix.moessbauer@siemens.com> Sent: 26 September 2022 07:11 > > > > To: Kanagarajan, Vijaikumar <Vijaikumar_Kanagarajan@mentor.com>; > > > > Schild, Henning (T CED SES-DE) <henning.schild@siemens.com> > > > > Cc: Kiszka, Jan (T CED) <jan.kiszka@siemens.com>; isar- > > > > users@googlegroups.com > > > > Subject: RE: [PATCH v2 2/2] wic: Update to the latest revision > > > > > > > > Hi Vijai & Henning, > > > > > > > > Are there any news on this? > > > > I just discovered that still many (if not all) of our images are > > > > affected by this bug. > > > > > > Hi Felix, > > > > > > Unfortunately not. If we already have the latest wic, > > > --no-fstab-update[1] might help.(untested). > > > > We already have a wic version with this option, but it doesn't look > > right to add it to the root partition. Instead this option can be used > > for partitions that should not show up in the fstab at all. > > We do want the one from WIC. It is only coming since we patched WIC to > support /. > > > Indeed, the fstab entry has to be set by WIC to set settings like UUID > > and filesystem. But this requires that we either remove the entry set > > by ISAR in image_configure_fstab, or we remove the entry after > > generating the rootfs, but before WIC. > > I think we should try and remove the entry from that template. It should not > be required. > > If we decide to solve the problem somewhere in/with WIC, we should do > that upstream in OE. They have the same "we now have two" problem, > where wic adds on top of a template instead of replacing the original. This seems to me like the approach we should take. Make wic replace entries in fstab instead of appending to them like how it does now. Thanks, Vijai Kumar K > > Henning > > > Opinions? @Kiszka, Jan (T CED) > > > > Felix > > > > > > > > Thanks, > > > Vijai Kumar K > > > > > > > > > > > While it is not serios, having error messages in the journal is > > > > still pretty bad: systemd-fstab-generator[1491]: Failed to create > > > > unit file /run/systemd/generator/-.mount, as it already exists. > > > > Duplicate entry in /etc/fstab? > > > > > > > > Felix > > > > > > > > > -----Original Message----- > > > > > From: isar-users@googlegroups.com <isar- > users@googlegroups.com> > > > > > On Behalf Of Kanagarajan, Vijaikumar > > > > > Sent: Friday, June 17, 2022 1:46 PM > > > > > To: isar-users@googlegroups.com > > > > > Cc: Kiszka, Jan (T CED) <jan.kiszka@siemens.com>; Schild, > > > > > Henning (T CED SES- > > > > > DE) <henning.schild@siemens.com> > > > > > Subject: RE: [PATCH v2 2/2] wic: Update to the latest revision > > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > From: Kanagarajan, Vijaikumar > > > > > > Sent: 13 June 2022 14:48 > > > > > > To: 'Henning Schild' <henning.schild@siemens.com> > > > > > > Cc: isar-users@googlegroups.com; jan.kiszka@siemens.com > > > > > > Subject: RE: [PATCH v2 2/2] wic: Update to the latest revision > > > > > > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > From: Henning Schild <henning.schild@siemens.com> > > > > > > > Sent: 13 June 2022 14:32 > > > > > > > To: Kanagarajan, Vijaikumar > > > > > > > <Vijaikumar_Kanagarajan@mentor.com> Cc: > > > > > > > isar-users@googlegroups.com; jan.kiszka@siemens.com > > > > > > > Subject: Re: [PATCH v2 2/2] wic: Update to the latest > > > > > > > revision > > > > > > > > > > > > > > Am Mon, 13 Jun 2022 13:24:01 +0530 schrieb Vijai Kumar K > > > > > > > <Vijaikumar_Kanagarajan@mentor.com>: > > > > > > > > > > > > > > > Update to the latest wic from OE-core. > > > > > > > > > > > > > > > > OE-core Revision: > 06ed2aa93fc25a681e3a00ee120d9395b04845da > > > > > > > > > > > > > > > > Signed-off-by: Vijai Kumar K > > > > <Vijaikumar_Kanagarajan@mentor.com> > > > > > > > > --- > > > > > > > > RECIPE-API-CHANGELOG.md | 19 > > > > > > > > +++++++++++++++++++ scripts/lib/wic/help.py > > > > > > > > | 6 ++++++ scripts/lib/wic/ksparser.py | > > > > > > > > 1 + scripts/lib/wic/partition.py | 1 + > > > > > > > > scripts/lib/wic/plugins/imager/direct.py | 5 +++-- > > > > > > > > 5 files changed, 30 insertions(+), 2 deletions(-) > > > > > > > > > > > > > > > > diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API- > > > > CHANGELOG.md > > > > > > > index > > > > > > > > 2998824..436b204 100644 > > > > > > > > --- a/RECIPE-API-CHANGELOG.md > > > > > > > > +++ b/RECIPE-API-CHANGELOG.md > > > > > > > > @@ -406,3 +406,22 @@ groups are now created after all > > > > > > > > packages are installed. > > > > > > > > Changes in next > > > > > > > > --------------- > > > > > > > > + > > > > > > > > +### Wic adds / mountpoint to fstab > > > > > > > > + > > > > > > > > +In the older version of wic, any mount point named / is > > > > > > > > +skipped from > > > > > > > > adding +into the fstab entry. > > > > > > > > + > > > > > > > > +With the latest wic, this is not the case. / mount > > > > > > > > point, if +any, is > > > > > > > > added +to /etc/fstab for automount. > > > > > > > > + > > > > > > > > +Any wks file which assumed that / would be skipped from > > > > > > > > +/etc/fstab > > > > > > > > should +now be corrected. Otherwise, it might result in > > > > > > > > multiple / entries in /etc/fstab. +One by the > > > > > > > > image_configure_fstab function from image class and one by > > > > > > > > wic. + > > > > > > > > +Below is an example wks entry that might cause an issue. > > > > > > > > +Replace "part /" with "part" to avoid adding duplicate > > > > > > > > entry for /. > > > > > > > > > > > > > > I am not sure multiple entries would be an issue, i think > > > > > > > the last one will simply always win. It is good to have a > > > > > > > note about that change, but i doubt people really have to do > > > > > > > anything. And if anything has to be done, isar should drop > > > > > > > the first one centrally. > > > > > > > > > > > > > > The fstab entry for root is purely for adding some weird > > > > > > > mount options, usually root is not mounted with fstab but > > > > > > > with cmdline and initrd. And not having / in fstab just > > > > > > > makes that rootfs more generic ... it can become part of nfs > > > > > > > and next guy takes it as btrfs while next guy > > > > > > takes it as ext4. > > > > > > > > > > > > > > Note is good, advise is not > > > > > > > > > > > > Noted. Will change the text accordingly in V3. > > > > > > > > > > Ok we cannot drop / mountpoint name(though optional). We would > > > > > be > > > > lose > > > > > the ability to find out UUID of root using rootdev api. > > > > > This is used by some plugins. So the better way would be to use > > > > > the --no-fstab- update. Seems like we can set them per partition > > > > > now. > > > > > > > > > > Thanks, > > > > > Vijai Kumar K > > > > > > > > > > > > > > > > > Thanks, > > > > > > Vijai Kumar K > > > > > > > > > > > > > > > > > > > > Henning > > > > > > > > > > > > > > > + > > > > > > > > +``` > > > > > > > > +part / --source rootfs --ondisk sda --fstype ext4 --label > > > > > > > > +platform > > > > > > > > --align 1024 --use-uuid +``` diff --git > > > > > > > > a/scripts/lib/wic/help.py b/scripts/lib/wic/help.py index > > > > > > > > 4ff7470..73e3380 100644 > > > > > > > > --- a/scripts/lib/wic/help.py > > > > > > > > +++ b/scripts/lib/wic/help.py > > > > > > > > @@ -940,6 +940,12 @@ DESCRIPTION > > > > > > > > quotes. If not specified, the > > > > > > > > default string is "defaults". > > > > > > > > > > > > > > > > + --fspassno: Specifies the order in which > > > > > > > > filesystem > > > > > > > > + checks > > > > > > > > are done > > > > > > > > + at boot time by fsck. See fs_passno > > > > > > > > + parameter > > > > > > > > of > > > > > > > > + fstab(5). This parameter will be > > > > > > > > copied > > > > > > > > + into > > > > > > > > the > > > > > > > > + /etc/fstab file of the installed > > > > > > > > system. If not > > > > > > > > + specified the default value of "0" > > > > > > > > will be used. + > > > > > > > > --label label: Specifies the label to give to > > > > > > > > the filesystem to be made on the partition. If the given > > > > > > > > label is already in use by > > > > > > > > another filesystem, diff --git > > > > > > > > a/scripts/lib/wic/ksparser.py > > > > > > > > b/scripts/lib/wic/ksparser.py index 0df9eb0..a49b7b9 > > > > > > > > 100644 --- a/scripts/lib/wic/ksparser.py +++ > > > > > > > > b/scripts/lib/wic/ksparser.py @@ -155,6 +155,7 @@ class > > > > > > > > KickStart(): part.add_argument('--change-directory') > > > > > > > > part.add_argument("--extra-space", > > > > > > > > type=sizetype("M")) part.add_argument('--fsoptions', > > > > > > > > dest='fsopts') > > > > > > > > + part.add_argument('--fspassno', dest='fspassno') > > > > > > > > part.add_argument('--fstype', default='vfat', > > > > > > > > choices=('ext2', 'ext3', > > > > > > > > 'ext4', 'btrfs', 'squashfs', 'vfat', 'msdos', 'erofs', > > > > > > > > diff --git a/scripts/lib/wic/partition.py > > > > > > > > b/scripts/lib/wic/partition.py index 09e491d..e50871b > > > > > > > > 100644 --- a/scripts/lib/wic/partition.py +++ > > > > > > > > b/scripts/lib/wic/partition.py @@ -33,6 +33,7 @@ class > > > > > > > > Partition(): self.include_path = args.include_path > > > > > > > > self.change_directory = args.change_directory > > > > > > > > self.fsopts = args.fsopts > > > > > > > > + self.fspassno = args.fspassno > > > > > > > > self.fstype = args.fstype > > > > > > > > self.label = args.label > > > > > > > > self.use_label = args.use_label diff --git > > > > > > > > a/scripts/lib/wic/plugins/imager/direct.py > > > > > > > > b/scripts/lib/wic/plugins/imager/direct.py index > > > > > > > > 4d0b836..c44159b > > > > > > > > 100644 --- a/scripts/lib/wic/plugins/imager/direct.py > > > > > > > > +++ b/scripts/lib/wic/plugins/imager/direct.py > > > > > > > > @@ -117,7 +117,7 @@ class DirectPlugin(ImagerPlugin): > > > > > > > > updated = False > > > > > > > > for part in self.parts: > > > > > > > > if not part.realnum or not part.mountpoint \ > > > > > > > > - or part.mountpoint == "/" or not > > > > > > > > part.mountpoint.startswith('/'): > > > > > > > > + or not part.mountpoint.startswith('/'): > > > > > > > > continue > > > > > > > > > > > > > > > > if part.use_uuid: > > > > > > > > @@ -138,8 +138,9 @@ class DirectPlugin(ImagerPlugin): > > > > > > > > device_name = "/dev/%s%s%d" % (part.disk, > > > > > > > > prefix, > > > > > > > > part.realnum) > > > > > > > > opts = part.fsopts if part.fsopts else > > > > > > > > "defaults" > > > > > > > > + passno = part.fspassno if part.fspassno else > > > > > > > > "0" line = "\t".join([device_name, part.mountpoint, > > > > > > > > part.fstype, > > > > > > > > - opts, "0", "0"]) + "\n" > > > > > > > > + opts, "0", passno]) + "\n" > > > > > > > > > > > > > > > > fstab_lines.append(line) > > > > > > > > updated = True > > > > > > > > > > -- > > > > > You received this message because you are subscribed to the > > > > > Google Groups "isar-users" group. > > > > > To unsubscribe from this group and stop receiving emails from > > > > > it, send an email to isar-users+unsubscribe@googlegroups.com. > > > > > To view this discussion on the web visit > > > > > > > > > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2F > > > > grou > > > > > > > > > ps.g > > > > > oogle.com%2Fd%2Fmsgid%2Fisar- > > > > > > > > > > users%2F8babb01d7eae44e0b9faecad59cdc1d4%2540mentor.com&dat > > > > a= > > > > > > > > > > 05%7C01%7Cfelix.moessbauer%40siemens.com%7C237b0db9bb724580ca540 > > > > 8 > > > > > > > > > > > > > da5024a3ad%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C6379104 > > > > > > > > > > 15537105828%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQ > > > > Ijo > > > > > > > > > > iV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sda > > > > ta > > > > > > > > > > =%2F%2F1Fxy%2B%2BEgDI4RfUEZ38qy1Q82esArdBuRftDuE1t6s%3D&re > > > > se > > > > > rved=0. ^ permalink raw reply [flat|nested] 22+ messages in thread
* RE: [PATCH v2 2/2] wic: Update to the latest revision 2022-09-26 7:56 ` Kanagarajan, Vijaikumar @ 2022-09-26 8:02 ` Moessbauer, Felix 2022-09-26 12:34 ` Henning Schild 1 sibling, 0 replies; 22+ messages in thread From: Moessbauer, Felix @ 2022-09-26 8:02 UTC (permalink / raw) To: Kanagarajan, Vijaikumar, Schild, Henning, jan.kiszka; +Cc: isar-users > -----Original Message----- > From: isar-users@googlegroups.com <isar-users@googlegroups.com> On > Behalf Of Kanagarajan, Vijaikumar > Sent: Monday, September 26, 2022 3:57 PM > To: Schild, Henning (T CED SES-DE) <henning.schild@siemens.com>; > Moessbauer, Felix (T CED SES-DE) <felix.moessbauer@siemens.com>; Kiszka, > Jan (T CED) <jan.kiszka@siemens.com> > Cc: isar-users@googlegroups.com > Subject: RE: [PATCH v2 2/2] wic: Update to the latest revision > > > > > -----Original Message----- > > From: Henning Schild <henning.schild@siemens.com> > > Sent: 26 September 2022 13:17 > > To: felix.moessbauer@siemens.com; Kanagarajan, Vijaikumar > > <Vijaikumar_Kanagarajan@mentor.com>; jan.kiszka@siemens.com > > Cc: isar-users@googlegroups.com > > Subject: Re: [PATCH v2 2/2] wic: Update to the latest revision > > > > Am Mon, 26 Sep 2022 07:57:41 +0200 > > schrieb "Moessbauer, Felix (T CED SES-DE)" > > <felix.moessbauer@siemens.com>: > > > > > > -----Original Message----- > > > > From: Kanagarajan, Vijaikumar <Vijaikumar_Kanagarajan@mentor.com> > > > > Sent: Monday, September 26, 2022 12:53 PM > > > > To: Moessbauer, Felix (T CED SES-DE) > > <felix.moessbauer@siemens.com>; > > > > Schild, Henning (T CED SES-DE) <henning.schild@siemens.com> Cc: > > > > Kiszka, Jan (T CED) <jan.kiszka@siemens.com>; isar- > > > > users@googlegroups.com > > > > Subject: RE: [PATCH v2 2/2] wic: Update to the latest revision > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > From: Moessbauer, Felix (T CED SES-DE) > > > > > <felix.moessbauer@siemens.com> Sent: 26 September 2022 07:11 > > > > > To: Kanagarajan, Vijaikumar <Vijaikumar_Kanagarajan@mentor.com>; > > > > > Schild, Henning (T CED SES-DE) <henning.schild@siemens.com> > > > > > Cc: Kiszka, Jan (T CED) <jan.kiszka@siemens.com>; isar- > > > > > users@googlegroups.com > > > > > Subject: RE: [PATCH v2 2/2] wic: Update to the latest revision > > > > > > > > > > Hi Vijai & Henning, > > > > > > > > > > Are there any news on this? > > > > > I just discovered that still many (if not all) of our images are > > > > > affected by this bug. > > > > > > > > Hi Felix, > > > > > > > > Unfortunately not. If we already have the latest wic, > > > > --no-fstab-update[1] might help.(untested). > > > > > > We already have a wic version with this option, but it doesn't look > > > right to add it to the root partition. Instead this option can be > > > used for partitions that should not show up in the fstab at all. > > > > We do want the one from WIC. It is only coming since we patched WIC to > > support /. > > > > > Indeed, the fstab entry has to be set by WIC to set settings like > > > UUID and filesystem. But this requires that we either remove the > > > entry set by ISAR in image_configure_fstab, or we remove the entry > > > after generating the rootfs, but before WIC. > > > > I think we should try and remove the entry from that template. It > > should not be required. > > > > If we decide to solve the problem somewhere in/with WIC, we should do > > that upstream in OE. They have the same "we now have two" problem, > > where wic adds on top of a template instead of replacing the original. > > This seems to me like the approach we should take. Make wic replace entries in > fstab instead of appending to them like how it does now. Sounds like a plan! Would you like to take care of writing the WIC patches, Vijai? Happy Coding! Felix > > Thanks, > Vijai Kumar K > > > > > Henning > > > > > Opinions? @Kiszka, Jan (T CED) > > > > > > Felix > > > > > > > > > > > Thanks, > > > > Vijai Kumar K > > > > > > > > > > > > > > While it is not serios, having error messages in the journal is > > > > > still pretty bad: systemd-fstab-generator[1491]: Failed to > > > > > create unit file /run/systemd/generator/-.mount, as it already exists. > > > > > Duplicate entry in /etc/fstab? > > > > > > > > > > Felix > > > > > > > > > > > -----Original Message----- > > > > > > From: isar-users@googlegroups.com <isar- > > users@googlegroups.com> > > > > > > On Behalf Of Kanagarajan, Vijaikumar > > > > > > Sent: Friday, June 17, 2022 1:46 PM > > > > > > To: isar-users@googlegroups.com > > > > > > Cc: Kiszka, Jan (T CED) <jan.kiszka@siemens.com>; Schild, > > > > > > Henning (T CED SES- > > > > > > DE) <henning.schild@siemens.com> > > > > > > Subject: RE: [PATCH v2 2/2] wic: Update to the latest revision > > > > > > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > From: Kanagarajan, Vijaikumar > > > > > > > Sent: 13 June 2022 14:48 > > > > > > > To: 'Henning Schild' <henning.schild@siemens.com> > > > > > > > Cc: isar-users@googlegroups.com; jan.kiszka@siemens.com > > > > > > > Subject: RE: [PATCH v2 2/2] wic: Update to the latest > > > > > > > revision > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > From: Henning Schild <henning.schild@siemens.com> > > > > > > > > Sent: 13 June 2022 14:32 > > > > > > > > To: Kanagarajan, Vijaikumar > > > > > > > > <Vijaikumar_Kanagarajan@mentor.com> Cc: > > > > > > > > isar-users@googlegroups.com; jan.kiszka@siemens.com > > > > > > > > Subject: Re: [PATCH v2 2/2] wic: Update to the latest > > > > > > > > revision > > > > > > > > > > > > > > > > Am Mon, 13 Jun 2022 13:24:01 +0530 schrieb Vijai Kumar K > > > > > > > > <Vijaikumar_Kanagarajan@mentor.com>: > > > > > > > > > > > > > > > > > Update to the latest wic from OE-core. > > > > > > > > > > > > > > > > > > OE-core Revision: > > 06ed2aa93fc25a681e3a00ee120d9395b04845da > > > > > > > > > > > > > > > > > > Signed-off-by: Vijai Kumar K > > > > > <Vijaikumar_Kanagarajan@mentor.com> > > > > > > > > > --- > > > > > > > > > RECIPE-API-CHANGELOG.md | 19 > > > > > > > > > +++++++++++++++++++ scripts/lib/wic/help.py > > > > > > > > > | 6 ++++++ scripts/lib/wic/ksparser.py | > > > > > > > > > 1 + scripts/lib/wic/partition.py | 1 + > > > > > > > > > scripts/lib/wic/plugins/imager/direct.py | 5 +++-- > > > > > > > > > 5 files changed, 30 insertions(+), 2 deletions(-) > > > > > > > > > > > > > > > > > > diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API- > > > > > CHANGELOG.md > > > > > > > > index > > > > > > > > > 2998824..436b204 100644 > > > > > > > > > --- a/RECIPE-API-CHANGELOG.md > > > > > > > > > +++ b/RECIPE-API-CHANGELOG.md > > > > > > > > > @@ -406,3 +406,22 @@ groups are now created after all > > > > > > > > > packages are installed. > > > > > > > > > Changes in next > > > > > > > > > --------------- > > > > > > > > > + > > > > > > > > > +### Wic adds / mountpoint to fstab > > > > > > > > > + > > > > > > > > > +In the older version of wic, any mount point named / is > > > > > > > > > +skipped from > > > > > > > > > adding +into the fstab entry. > > > > > > > > > + > > > > > > > > > +With the latest wic, this is not the case. / mount > > > > > > > > > point, if +any, is > > > > > > > > > added +to /etc/fstab for automount. > > > > > > > > > + > > > > > > > > > +Any wks file which assumed that / would be skipped from > > > > > > > > > +/etc/fstab > > > > > > > > > should +now be corrected. Otherwise, it might result in > > > > > > > > > multiple / entries in /etc/fstab. +One by the > > > > > > > > > image_configure_fstab function from image class and one > > > > > > > > > by wic. + > > > > > > > > > +Below is an example wks entry that might cause an issue. > > > > > > > > > +Replace "part /" with "part" to avoid adding duplicate > > > > > > > > > entry for /. > > > > > > > > > > > > > > > > I am not sure multiple entries would be an issue, i think > > > > > > > > the last one will simply always win. It is good to have a > > > > > > > > note about that change, but i doubt people really have to > > > > > > > > do anything. And if anything has to be done, isar should > > > > > > > > drop the first one centrally. > > > > > > > > > > > > > > > > The fstab entry for root is purely for adding some weird > > > > > > > > mount options, usually root is not mounted with fstab but > > > > > > > > with cmdline and initrd. And not having / in fstab just > > > > > > > > makes that rootfs more generic ... it can become part of > > > > > > > > nfs and next guy takes it as btrfs while next guy > > > > > > > takes it as ext4. > > > > > > > > > > > > > > > > Note is good, advise is not > > > > > > > > > > > > > > Noted. Will change the text accordingly in V3. > > > > > > > > > > > > Ok we cannot drop / mountpoint name(though optional). We would > > > > > > be > > > > > lose > > > > > > the ability to find out UUID of root using rootdev api. > > > > > > This is used by some plugins. So the better way would be to > > > > > > use the --no-fstab- update. Seems like we can set them per > > > > > > partition now. > > > > > > > > > > > > Thanks, > > > > > > Vijai Kumar K > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > Vijai Kumar K > > > > > > > > > > > > > > > > > > > > > > > Henning > > > > > > > > > > > > > > > > > + > > > > > > > > > +``` > > > > > > > > > +part / --source rootfs --ondisk sda --fstype ext4 > > > > > > > > > +--label platform > > > > > > > > > --align 1024 --use-uuid +``` diff --git > > > > > > > > > a/scripts/lib/wic/help.py b/scripts/lib/wic/help.py > > > > > > > > > index > > > > > > > > > 4ff7470..73e3380 100644 > > > > > > > > > --- a/scripts/lib/wic/help.py > > > > > > > > > +++ b/scripts/lib/wic/help.py > > > > > > > > > @@ -940,6 +940,12 @@ DESCRIPTION > > > > > > > > > quotes. If not specified, the > > > > > > > > > default string is "defaults". > > > > > > > > > > > > > > > > > > + --fspassno: Specifies the order in which > > > > > > > > > filesystem > > > > > > > > > + checks > > > > > > > > > are done > > > > > > > > > + at boot time by fsck. See > > > > > > > > > + fs_passno parameter > > > > > > > > > of > > > > > > > > > + fstab(5). This parameter will be > > > > > > > > > copied > > > > > > > > > + into > > > > > > > > > the > > > > > > > > > + /etc/fstab file of the installed > > > > > > > > > system. If not > > > > > > > > > + specified the default value of "0" > > > > > > > > > will be used. + > > > > > > > > > --label label: Specifies the label to give to > > > > > > > > > the filesystem to be made on the partition. If the given > > > > > > > > > label is already in use by > > > > > > > > > another filesystem, diff --git > > > > > > > > > a/scripts/lib/wic/ksparser.py > > > > > > > > > b/scripts/lib/wic/ksparser.py index 0df9eb0..a49b7b9 > > > > > > > > > 100644 --- a/scripts/lib/wic/ksparser.py +++ > > > > > > > > > b/scripts/lib/wic/ksparser.py @@ -155,6 +155,7 @@ class > > > > > > > > > KickStart(): part.add_argument('--change-directory') > > > > > > > > > part.add_argument("--extra-space", > > > > > > > > > type=sizetype("M")) part.add_argument('--fsoptions', > > > > > > > > > dest='fsopts') > > > > > > > > > + part.add_argument('--fspassno', > > > > > > > > > + dest='fspassno') > > > > > > > > > part.add_argument('--fstype', default='vfat', > > > > > > > > > choices=('ext2', 'ext3', > > > > > > > > > 'ext4', 'btrfs', 'squashfs', 'vfat', 'msdos', 'erofs', > > > > > > > > > diff --git a/scripts/lib/wic/partition.py > > > > > > > > > b/scripts/lib/wic/partition.py index 09e491d..e50871b > > > > > > > > > 100644 --- a/scripts/lib/wic/partition.py +++ > > > > > > > > > b/scripts/lib/wic/partition.py @@ -33,6 +33,7 @@ class > > > > > > > > > Partition(): self.include_path = args.include_path > > > > > > > > > self.change_directory = args.change_directory > > > > > > > > > self.fsopts = args.fsopts > > > > > > > > > + self.fspassno = args.fspassno > > > > > > > > > self.fstype = args.fstype > > > > > > > > > self.label = args.label > > > > > > > > > self.use_label = args.use_label diff --git > > > > > > > > > a/scripts/lib/wic/plugins/imager/direct.py > > > > > > > > > b/scripts/lib/wic/plugins/imager/direct.py index > > > > > > > > > 4d0b836..c44159b > > > > > > > > > 100644 --- a/scripts/lib/wic/plugins/imager/direct.py > > > > > > > > > +++ b/scripts/lib/wic/plugins/imager/direct.py > > > > > > > > > @@ -117,7 +117,7 @@ class DirectPlugin(ImagerPlugin): > > > > > > > > > updated = False > > > > > > > > > for part in self.parts: > > > > > > > > > if not part.realnum or not part.mountpoint \ > > > > > > > > > - or part.mountpoint == "/" or not > > > > > > > > > part.mountpoint.startswith('/'): > > > > > > > > > + or not part.mountpoint.startswith('/'): > > > > > > > > > continue > > > > > > > > > > > > > > > > > > if part.use_uuid: > > > > > > > > > @@ -138,8 +138,9 @@ class DirectPlugin(ImagerPlugin): > > > > > > > > > device_name = "/dev/%s%s%d" % > > > > > > > > > (part.disk, prefix, > > > > > > > > > part.realnum) > > > > > > > > > opts = part.fsopts if part.fsopts else > > > > > > > > > "defaults" > > > > > > > > > + passno = part.fspassno if part.fspassno > > > > > > > > > + else > > > > > > > > > "0" line = "\t".join([device_name, part.mountpoint, > > > > > > > > > part.fstype, > > > > > > > > > - opts, "0", "0"]) + "\n" > > > > > > > > > + opts, "0", passno]) + "\n" > > > > > > > > > > > > > > > > > > fstab_lines.append(line) > > > > > > > > > updated = True > > > > > > > > > > > > -- > > > > > > You received this message because you are subscribed to the > > > > > > Google Groups "isar-users" group. > > > > > > To unsubscribe from this group and stop receiving emails from > > > > > > it, send an email to isar-users+unsubscribe@googlegroups.com. > > > > > > To view this discussion on the web visit > > > > > > > > > > > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F% > > > > > 2F > > > > > grou > > > > > > > > > > > ps.g > > > > > > oogle.com%2Fd%2Fmsgid%2Fisar- > > > > > > > > > > > > > users%2F8babb01d7eae44e0b9faecad59cdc1d4%2540mentor.com&dat > > > > > a= > > > > > > > > > > > > > 05%7C01%7Cfelix.moessbauer%40siemens.com%7C237b0db9bb724580ca540 > > > > > 8 > > > > > > > > > > > > > > > > > > da5024a3ad%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C6379104 > > > > > > > > > > > > > 15537105828%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQ > > > > > Ijo > > > > > > > > > > > > > iV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sda > > > > > ta > > > > > > > > > > > > > =%2F%2F1Fxy%2B%2BEgDI4RfUEZ38qy1Q82esArdBuRftDuE1t6s%3D&re > > > > > se > > > > > > rved=0. > > -- > You received this message because you are subscribed to the Google Groups > "isar-users" group. > To unsubscribe from this group and stop receiving emails from it, send an email > to isar-users+unsubscribe@googlegroups.com. > To view this discussion on the web visit > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.g > oogle.com%2Fd%2Fmsgid%2Fisar- > users%2F7d23828243444e90afd49e7bb23717dc%2540mentor.com&data= > 05%7C01%7Cfelix.moessbauer%40siemens.com%7C7e073a4d25ec493e0d3408 > da9f94ad76%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C6379977 > 58159199068%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjo > iV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata > =pT2NMVWp6ld7iDW%2BRStJABC02WCrzGDOV%2BXhvpxPKO4%3D&reser > ved=0. ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH v2 2/2] wic: Update to the latest revision 2022-09-26 7:56 ` Kanagarajan, Vijaikumar 2022-09-26 8:02 ` Moessbauer, Felix @ 2022-09-26 12:34 ` Henning Schild 1 sibling, 0 replies; 22+ messages in thread From: Henning Schild @ 2022-09-26 12:34 UTC (permalink / raw) To: Kanagarajan, Vijaikumar; +Cc: felix.moessbauer, jan.kiszka, isar-users Am Mon, 26 Sep 2022 07:56:48 +0000 schrieb "Kanagarajan, Vijaikumar" <Vijaikumar_Kanagarajan@mentor.com>: > > -----Original Message----- > > From: Henning Schild <henning.schild@siemens.com> > > Sent: 26 September 2022 13:17 > > To: felix.moessbauer@siemens.com; Kanagarajan, Vijaikumar > > <Vijaikumar_Kanagarajan@mentor.com>; jan.kiszka@siemens.com > > Cc: isar-users@googlegroups.com > > Subject: Re: [PATCH v2 2/2] wic: Update to the latest revision > > > > Am Mon, 26 Sep 2022 07:57:41 +0200 > > schrieb "Moessbauer, Felix (T CED SES-DE)" > > <felix.moessbauer@siemens.com>: > > > > > > -----Original Message----- > > > > From: Kanagarajan, Vijaikumar > > > > <Vijaikumar_Kanagarajan@mentor.com> Sent: Monday, September 26, > > > > 2022 12:53 PM To: Moessbauer, Felix (T CED SES-DE) > > <felix.moessbauer@siemens.com>; > > > > Schild, Henning (T CED SES-DE) <henning.schild@siemens.com> Cc: > > > > Kiszka, Jan (T CED) <jan.kiszka@siemens.com>; isar- > > > > users@googlegroups.com > > > > Subject: RE: [PATCH v2 2/2] wic: Update to the latest revision > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > From: Moessbauer, Felix (T CED SES-DE) > > > > > <felix.moessbauer@siemens.com> Sent: 26 September 2022 07:11 > > > > > To: Kanagarajan, Vijaikumar > > > > > <Vijaikumar_Kanagarajan@mentor.com>; Schild, Henning (T CED > > > > > SES-DE) <henning.schild@siemens.com> Cc: Kiszka, Jan (T CED) > > > > > <jan.kiszka@siemens.com>; isar- users@googlegroups.com > > > > > Subject: RE: [PATCH v2 2/2] wic: Update to the latest revision > > > > > > > > > > Hi Vijai & Henning, > > > > > > > > > > Are there any news on this? > > > > > I just discovered that still many (if not all) of our images > > > > > are affected by this bug. > > > > > > > > Hi Felix, > > > > > > > > Unfortunately not. If we already have the latest wic, > > > > --no-fstab-update[1] might help.(untested). > > > > > > We already have a wic version with this option, but it doesn't > > > look right to add it to the root partition. Instead this option > > > can be used for partitions that should not show up in the fstab > > > at all. > > > > We do want the one from WIC. It is only coming since we patched WIC > > to support /. > > > > > Indeed, the fstab entry has to be set by WIC to set settings like > > > UUID and filesystem. But this requires that we either remove the > > > entry set by ISAR in image_configure_fstab, or we remove the > > > entry after generating the rootfs, but before WIC. > > > > I think we should try and remove the entry from that template. It > > should not be required. > > > > If we decide to solve the problem somewhere in/with WIC, we should > > do that upstream in OE. They have the same "we now have two" > > problem, where wic adds on top of a template instead of replacing > > the original. > > This seems to me like the approach we should take. Make wic replace > entries in fstab instead of appending to them like how it does now. That is the more complicated way. I do not recall all the details but i think we should rather drop the / line from the template. Such a line is usually not needed, having it is rather limiting the use of the rootfs because you might have a device name standing there which might be wrong ... i.e. sda1 vs vda1 vs nfs I think we could contribute both to OE. The removal of the / line from their template, and the "wic will try and update not append". But we could also simply drop the template line in isar ... not care about OE and making our non wic images better in fact. There might be weird cornercases, maybe a too old raspi image ... where that line is needed. I would move it to those corner-case images if there are any. Henning > Thanks, > Vijai Kumar K > > > > > Henning > > > > > Opinions? @Kiszka, Jan (T CED) > > > > > > Felix > > > > > > > > > > > Thanks, > > > > Vijai Kumar K > > > > > > > > > > > > > > While it is not serios, having error messages in the journal > > > > > is still pretty bad: systemd-fstab-generator[1491]: Failed to > > > > > create unit file /run/systemd/generator/-.mount, as it > > > > > already exists. Duplicate entry in /etc/fstab? > > > > > > > > > > Felix > > > > > > > > > > > -----Original Message----- > > > > > > From: isar-users@googlegroups.com <isar- > > users@googlegroups.com> > > > > > > On Behalf Of Kanagarajan, Vijaikumar > > > > > > Sent: Friday, June 17, 2022 1:46 PM > > > > > > To: isar-users@googlegroups.com > > > > > > Cc: Kiszka, Jan (T CED) <jan.kiszka@siemens.com>; Schild, > > > > > > Henning (T CED SES- > > > > > > DE) <henning.schild@siemens.com> > > > > > > Subject: RE: [PATCH v2 2/2] wic: Update to the latest > > > > > > revision > > > > > > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > From: Kanagarajan, Vijaikumar > > > > > > > Sent: 13 June 2022 14:48 > > > > > > > To: 'Henning Schild' <henning.schild@siemens.com> > > > > > > > Cc: isar-users@googlegroups.com; jan.kiszka@siemens.com > > > > > > > Subject: RE: [PATCH v2 2/2] wic: Update to the latest > > > > > > > revision > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > From: Henning Schild <henning.schild@siemens.com> > > > > > > > > Sent: 13 June 2022 14:32 > > > > > > > > To: Kanagarajan, Vijaikumar > > > > > > > > <Vijaikumar_Kanagarajan@mentor.com> Cc: > > > > > > > > isar-users@googlegroups.com; jan.kiszka@siemens.com > > > > > > > > Subject: Re: [PATCH v2 2/2] wic: Update to the latest > > > > > > > > revision > > > > > > > > > > > > > > > > Am Mon, 13 Jun 2022 13:24:01 +0530 schrieb Vijai Kumar K > > > > > > > > <Vijaikumar_Kanagarajan@mentor.com>: > > > > > > > > > > > > > > > > > Update to the latest wic from OE-core. > > > > > > > > > > > > > > > > > > OE-core Revision: > > 06ed2aa93fc25a681e3a00ee120d9395b04845da > > > > > > > > > > > > > > > > > > Signed-off-by: Vijai Kumar K > > > > > <Vijaikumar_Kanagarajan@mentor.com> > > > > > > > > > --- > > > > > > > > > RECIPE-API-CHANGELOG.md | 19 > > > > > > > > > +++++++++++++++++++ scripts/lib/wic/help.py > > > > > > > > > | 6 ++++++ scripts/lib/wic/ksparser.py > > > > > > > > > | 1 + scripts/lib/wic/partition.py | 1 > > > > > > > > > + scripts/lib/wic/plugins/imager/direct.py | 5 +++-- > > > > > > > > > 5 files changed, 30 insertions(+), 2 deletions(-) > > > > > > > > > > > > > > > > > > diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API- > > > > > CHANGELOG.md > > > > > > > > index > > > > > > > > > 2998824..436b204 100644 > > > > > > > > > --- a/RECIPE-API-CHANGELOG.md > > > > > > > > > +++ b/RECIPE-API-CHANGELOG.md > > > > > > > > > @@ -406,3 +406,22 @@ groups are now created after all > > > > > > > > > packages are installed. > > > > > > > > > Changes in next > > > > > > > > > --------------- > > > > > > > > > + > > > > > > > > > +### Wic adds / mountpoint to fstab > > > > > > > > > + > > > > > > > > > +In the older version of wic, any mount point named / > > > > > > > > > is +skipped from > > > > > > > > > adding +into the fstab entry. > > > > > > > > > + > > > > > > > > > +With the latest wic, this is not the case. / mount > > > > > > > > > point, if +any, is > > > > > > > > > added +to /etc/fstab for automount. > > > > > > > > > + > > > > > > > > > +Any wks file which assumed that / would be skipped > > > > > > > > > from +/etc/fstab > > > > > > > > > should +now be corrected. Otherwise, it might result > > > > > > > > > in multiple / entries in /etc/fstab. +One by the > > > > > > > > > image_configure_fstab function from image class and > > > > > > > > > one by wic. + > > > > > > > > > +Below is an example wks entry that might cause an > > > > > > > > > issue. +Replace "part /" with "part" to avoid adding > > > > > > > > > duplicate entry for /. > > > > > > > > > > > > > > > > I am not sure multiple entries would be an issue, i > > > > > > > > think the last one will simply always win. It is good > > > > > > > > to have a note about that change, but i doubt people > > > > > > > > really have to do anything. And if anything has to be > > > > > > > > done, isar should drop the first one centrally. > > > > > > > > > > > > > > > > The fstab entry for root is purely for adding some weird > > > > > > > > mount options, usually root is not mounted with fstab > > > > > > > > but with cmdline and initrd. And not having / in fstab > > > > > > > > just makes that rootfs more generic ... it can become > > > > > > > > part of nfs and next guy takes it as btrfs while next > > > > > > > > guy > > > > > > > takes it as ext4. > > > > > > > > > > > > > > > > Note is good, advise is not > > > > > > > > > > > > > > Noted. Will change the text accordingly in V3. > > > > > > > > > > > > Ok we cannot drop / mountpoint name(though optional). We > > > > > > would be > > > > > lose > > > > > > the ability to find out UUID of root using rootdev api. > > > > > > This is used by some plugins. So the better way would be to > > > > > > use the --no-fstab- update. Seems like we can set them per > > > > > > partition now. > > > > > > > > > > > > Thanks, > > > > > > Vijai Kumar K > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > Vijai Kumar K > > > > > > > > > > > > > > > > > > > > > > > Henning > > > > > > > > > > > > > > > > > + > > > > > > > > > +``` > > > > > > > > > +part / --source rootfs --ondisk sda --fstype ext4 > > > > > > > > > --label +platform > > > > > > > > > --align 1024 --use-uuid +``` diff --git > > > > > > > > > a/scripts/lib/wic/help.py b/scripts/lib/wic/help.py > > > > > > > > > index 4ff7470..73e3380 100644 > > > > > > > > > --- a/scripts/lib/wic/help.py > > > > > > > > > +++ b/scripts/lib/wic/help.py > > > > > > > > > @@ -940,6 +940,12 @@ DESCRIPTION > > > > > > > > > quotes. If not specified, the > > > > > > > > > default string is "defaults". > > > > > > > > > > > > > > > > > > + --fspassno: Specifies the order in which > > > > > > > > > filesystem > > > > > > > > > + checks > > > > > > > > > are done > > > > > > > > > + at boot time by fsck. See > > > > > > > > > fs_passno > > > > > > > > > + parameter > > > > > > > > > of > > > > > > > > > + fstab(5). This parameter will > > > > > > > > > be copied > > > > > > > > > + into > > > > > > > > > the > > > > > > > > > + /etc/fstab file of the installed > > > > > > > > > system. If not > > > > > > > > > + specified the default value of > > > > > > > > > "0" will be used. + > > > > > > > > > --label label: Specifies the label to give > > > > > > > > > to the filesystem to be made on the partition. If the > > > > > > > > > given label is already in use by > > > > > > > > > another filesystem, diff --git > > > > > > > > > a/scripts/lib/wic/ksparser.py > > > > > > > > > b/scripts/lib/wic/ksparser.py index 0df9eb0..a49b7b9 > > > > > > > > > 100644 --- a/scripts/lib/wic/ksparser.py +++ > > > > > > > > > b/scripts/lib/wic/ksparser.py @@ -155,6 +155,7 @@ > > > > > > > > > class KickStart(): > > > > > > > > > part.add_argument('--change-directory') > > > > > > > > > part.add_argument("--extra-space", > > > > > > > > > type=sizetype("M")) part.add_argument('--fsoptions', > > > > > > > > > dest='fsopts') > > > > > > > > > + part.add_argument('--fspassno', > > > > > > > > > dest='fspassno') part.add_argument('--fstype', > > > > > > > > > default='vfat', choices=('ext2', 'ext3', > > > > > > > > > 'ext4', 'btrfs', 'squashfs', 'vfat', 'msdos', 'erofs', > > > > > > > > > diff --git a/scripts/lib/wic/partition.py > > > > > > > > > b/scripts/lib/wic/partition.py index 09e491d..e50871b > > > > > > > > > 100644 --- a/scripts/lib/wic/partition.py +++ > > > > > > > > > b/scripts/lib/wic/partition.py @@ -33,6 +33,7 @@ class > > > > > > > > > Partition(): self.include_path = args.include_path > > > > > > > > > self.change_directory = args.change_directory > > > > > > > > > self.fsopts = args.fsopts > > > > > > > > > + self.fspassno = args.fspassno > > > > > > > > > self.fstype = args.fstype > > > > > > > > > self.label = args.label > > > > > > > > > self.use_label = args.use_label diff --git > > > > > > > > > a/scripts/lib/wic/plugins/imager/direct.py > > > > > > > > > b/scripts/lib/wic/plugins/imager/direct.py index > > > > > > > > > 4d0b836..c44159b > > > > > > > > > 100644 --- a/scripts/lib/wic/plugins/imager/direct.py > > > > > > > > > +++ b/scripts/lib/wic/plugins/imager/direct.py > > > > > > > > > @@ -117,7 +117,7 @@ class DirectPlugin(ImagerPlugin): > > > > > > > > > updated = False > > > > > > > > > for part in self.parts: > > > > > > > > > if not part.realnum or not > > > > > > > > > part.mountpoint \ > > > > > > > > > - or part.mountpoint == "/" or not > > > > > > > > > part.mountpoint.startswith('/'): > > > > > > > > > + or not > > > > > > > > > part.mountpoint.startswith('/'): continue > > > > > > > > > > > > > > > > > > if part.use_uuid: > > > > > > > > > @@ -138,8 +138,9 @@ class DirectPlugin(ImagerPlugin): > > > > > > > > > device_name = "/dev/%s%s%d" % > > > > > > > > > (part.disk, prefix, > > > > > > > > > part.realnum) > > > > > > > > > opts = part.fsopts if part.fsopts else > > > > > > > > > "defaults" > > > > > > > > > + passno = part.fspassno if part.fspassno > > > > > > > > > else "0" line = "\t".join([device_name, > > > > > > > > > part.mountpoint, part.fstype, > > > > > > > > > - opts, "0", "0"]) + "\n" > > > > > > > > > + opts, "0", passno]) + > > > > > > > > > "\n" > > > > > > > > > > > > > > > > > > fstab_lines.append(line) > > > > > > > > > updated = True > > > > > > > > > > > > -- > > > > > > You received this message because you are subscribed to the > > > > > > Google Groups "isar-users" group. > > > > > > To unsubscribe from this group and stop receiving emails > > > > > > from it, send an email to > > > > > > isar-users+unsubscribe@googlegroups.com. To view this > > > > > > discussion on the web visit > > > > > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2F > > > > > grou > > > > > > > > > > > ps.g > > > > > > oogle.com%2Fd%2Fmsgid%2Fisar- > > > > > > > > > > > > > users%2F8babb01d7eae44e0b9faecad59cdc1d4%2540mentor.com&dat > > > > > a= > > > > > > > > > > > > > 05%7C01%7Cfelix.moessbauer%40siemens.com%7C237b0db9bb724580ca540 > > > > > 8 > > > > > > > > > > > > > > > > > da5024a3ad%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C6379104 > > > > > > > > > > > > > 15537105828%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQ > > > > > Ijo > > > > > > > > > > > > > iV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sda > > > > > ta > > > > > > > > > > > > > =%2F%2F1Fxy%2B%2BEgDI4RfUEZ38qy1Q82esArdBuRftDuE1t6s%3D&re > > > > > se > > > > > > rved=0. > ^ permalink raw reply [flat|nested] 22+ messages in thread
end of thread, other threads:[~2022-09-26 12:34 UTC | newest] Thread overview: 22+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2022-04-21 8:52 [PATCH v2 0/2] bump bitbake and wic for python 3.10 support Henning Schild 2022-04-21 8:52 ` [PATCH v2 1/2] bitbake: Update to 1.50.5 release Henning Schild 2022-04-21 8:52 ` [PATCH v2 2/2] wic: Update to the latest revision Henning Schild 2022-04-21 9:02 ` Henning Schild 2022-04-22 7:59 ` [PATCH v2 0/2] bump bitbake and wic for python 3.10 support Anton Mikanovich 2022-04-22 8:15 ` Bezdeka, Florian 2022-04-22 8:26 ` Henning Schild 2022-04-22 8:52 ` Henning Schild 2022-04-22 9:54 ` Henning Schild 2022-04-22 8:27 ` Henning Schild 2022-04-22 8:57 ` Anton Mikanovich 2022-06-13 7:53 [PATCH v2 0/2] Update WIC to bring in fspassno feature Vijai Kumar K 2022-06-13 7:54 ` [PATCH v2 2/2] wic: Update to the latest revision Vijai Kumar K 2022-06-13 9:02 ` Henning Schild 2022-06-13 9:18 ` Kanagarajan, Vijaikumar 2022-06-17 5:45 ` Kanagarajan, Vijaikumar 2022-09-26 1:40 ` Moessbauer, Felix 2022-09-26 4:53 ` Kanagarajan, Vijaikumar 2022-09-26 5:57 ` Moessbauer, Felix 2022-09-26 7:46 ` Henning Schild 2022-09-26 7:56 ` Kanagarajan, Vijaikumar 2022-09-26 8:02 ` Moessbauer, Felix 2022-09-26 12:34 ` Henning Schild
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox