* [PATCH v2 0/2] bump bitbake and wic for python 3.10 support
@ 2022-04-21 8:52 Henning Schild
2022-04-21 8:52 ` [PATCH v2 1/2] bitbake: Update to 1.50.5 release Henning Schild
` (2 more replies)
0 siblings, 3 replies; 22+ messages in thread
From: Henning Schild @ 2022-04-21 8:52 UTC (permalink / raw)
To: isar-users; +Cc: Florian Bezdeka, Felix Moessbauer, Henning Schild
debian bookworm switched to python 3.10 recently, our versions of
bitbake and wic both have some issues with that new interpreter. All
these issues have been fixes in bitbake and OE, so all we need to do is
update the two tools in our tree.
v1 was send as "[PATCH] bitbake: Update to 1.50.5 release", which is p1
here, only later i found that wic also needs a bump to v2 added p2
If this series is applied we do not need "[PATCH] wic-img: use
python3.9 in bookworm", in case this one is applied before it should be
reverted after the version bumps.
Henning Schild (2):
bitbake: Update to 1.50.5 release
wic: Update to the latest revision
bitbake/lib/bb/cache.py | 3 +-
bitbake/lib/bb/cooker.py | 30 +++++++-
bitbake/lib/bb/data_smart.py | 4 +-
bitbake/lib/bb/fetch2/__init__.py | 4 +
bitbake/lib/bb/fetch2/perforce.py | 2 +-
bitbake/lib/bb/fetch2/wget.py | 2 +-
bitbake/lib/bb/persist_data.py | 5 +-
bitbake/lib/bb/process.py | 2 +-
bitbake/lib/bb/runqueue.py | 34 ++++----
bitbake/lib/bb/server/process.py | 2 +-
bitbake/lib/bb/tests/fetch.py | 35 +++++----
bitbake/lib/bb/utils.py | 13 +++-
bitbake/lib/hashserv/server.py | 4 +-
bitbake/lib/toaster/tests/builds/buildtest.py | 2 +-
.../wic/plugins/source/bootimg-efi-isar.py | 77 ++++++++++++++++---
.../wic/plugins/source/bootimg-pcbios-isar.py | 6 +-
scripts/lib/wic/canned-wks/common.wks.inc | 2 +-
scripts/lib/wic/canned-wks/directdisk-gpt.wks | 2 +-
scripts/lib/wic/canned-wks/mkefidisk.wks | 2 +-
scripts/lib/wic/engine.py | 6 +-
scripts/lib/wic/help.py | 10 ++-
scripts/lib/wic/ksparser.py | 8 +-
scripts/lib/wic/misc.py | 4 +-
scripts/lib/wic/partition.py | 25 ++++--
scripts/lib/wic/pluginbase.py | 8 +-
scripts/lib/wic/plugins/imager/direct.py | 11 ++-
scripts/lib/wic/plugins/source/bootimg-efi.py | 74 +++++++++++++++---
.../lib/wic/plugins/source/bootimg-pcbios.py | 6 +-
scripts/lib/wic/plugins/source/rawcopy.py | 35 ++++++++-
scripts/lib/wic/plugins/source/rootfs.py | 2 +-
scripts/wic | 9 ++-
31 files changed, 320 insertions(+), 109 deletions(-)
--
2.35.1
^ permalink raw reply [flat|nested] 22+ messages in thread
* [PATCH v2 1/2] bitbake: Update to 1.50.5 release
2022-04-21 8:52 [PATCH v2 0/2] bump bitbake and wic for python 3.10 support Henning Schild
@ 2022-04-21 8:52 ` Henning Schild
2022-04-21 8:52 ` [PATCH v2 2/2] wic: Update to the latest revision Henning Schild
2022-04-22 7:59 ` [PATCH v2 0/2] bump bitbake and wic for python 3.10 support Anton Mikanovich
2 siblings, 0 replies; 22+ messages in thread
From: Henning Schild @ 2022-04-21 8:52 UTC (permalink / raw)
To: isar-users; +Cc: Florian Bezdeka, Felix Moessbauer, Henning Schild
Update bitbake to the latest release in 1.50 branch. This release is
tagged to the commit ID aaa7f7af23d5f89fe4a5ed48c57ea3dfca07c79d
in the bitbake upstream.
Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
bitbake/lib/bb/cache.py | 3 +-
bitbake/lib/bb/cooker.py | 30 ++++++++++++++--
bitbake/lib/bb/data_smart.py | 4 +--
bitbake/lib/bb/fetch2/__init__.py | 4 +++
bitbake/lib/bb/fetch2/perforce.py | 2 +-
bitbake/lib/bb/fetch2/wget.py | 2 +-
bitbake/lib/bb/persist_data.py | 5 +--
bitbake/lib/bb/process.py | 2 +-
bitbake/lib/bb/runqueue.py | 34 +++++++++---------
bitbake/lib/bb/server/process.py | 2 +-
bitbake/lib/bb/tests/fetch.py | 35 ++++++++++---------
bitbake/lib/bb/utils.py | 13 +++++--
bitbake/lib/hashserv/server.py | 4 +--
bitbake/lib/toaster/tests/builds/buildtest.py | 2 +-
14 files changed, 89 insertions(+), 53 deletions(-)
diff --git a/bitbake/lib/bb/cache.py b/bitbake/lib/bb/cache.py
index 27eb271798e8..5f9c0a779d75 100644
--- a/bitbake/lib/bb/cache.py
+++ b/bitbake/lib/bb/cache.py
@@ -19,7 +19,8 @@
import os
import logging
import pickle
-from collections import defaultdict, Mapping
+from collections import defaultdict
+from collections.abc import Mapping
import bb.utils
from bb import PrefixLoggerAdapter
import re
diff --git a/bitbake/lib/bb/cooker.py b/bitbake/lib/bb/cooker.py
index 89f1fad08310..c946800a8c62 100644
--- a/bitbake/lib/bb/cooker.py
+++ b/bitbake/lib/bb/cooker.py
@@ -388,12 +388,22 @@ class BBCooker:
# Create a new hash server bound to a unix domain socket
if not self.hashserv:
dbfile = (self.data.getVar("PERSISTENT_DIR") or self.data.getVar("CACHE")) + "/hashserv.db"
+ upstream = self.data.getVar("BB_HASHSERVE_UPSTREAM") or None
+ if upstream:
+ import socket
+ try:
+ sock = socket.create_connection(upstream.split(":"), 5)
+ sock.close()
+ except socket.error as e:
+ bb.warn("BB_HASHSERVE_UPSTREAM is not valid, unable to connect hash equivalence server at '%s': %s"
+ % (upstream, repr(e)))
+
self.hashservaddr = "unix://%s/hashserve.sock" % self.data.getVar("TOPDIR")
self.hashserv = hashserv.create_server(
self.hashservaddr,
dbfile,
sync=False,
- upstream=self.data.getVar("BB_HASHSERVE_UPSTREAM") or None,
+ upstream=upstream,
)
self.hashserv.process = multiprocessing.Process(target=self.hashserv.serve_forever)
self.hashserv.process.start()
@@ -805,7 +815,9 @@ class BBCooker:
for dep in rq.rqdata.runtaskentries[tid].depends:
(depmc, depfn, _, deptaskfn) = bb.runqueue.split_tid_mcfn(dep)
deppn = self.recipecaches[depmc].pkg_fn[deptaskfn]
- depend_tree["tdepends"][dotname].append("%s.%s" % (deppn, bb.runqueue.taskname_from_tid(dep)))
+ if depmc:
+ depmc = "mc:" + depmc + ":"
+ depend_tree["tdepends"][dotname].append("%s%s.%s" % (depmc, deppn, bb.runqueue.taskname_from_tid(dep)))
if taskfn not in seen_fns:
seen_fns.append(taskfn)
packages = []
@@ -2204,21 +2216,33 @@ class CookerParser(object):
yield not cached, mc, infos
def parse_generator(self):
- while True:
+ empty = False
+ while self.processes or not empty:
+ for process in self.processes.copy():
+ if not process.is_alive():
+ process.join()
+ self.processes.remove(process)
+
if self.parsed >= self.toparse:
break
try:
result = self.result_queue.get(timeout=0.25)
except queue.Empty:
+ empty = True
pass
else:
+ empty = False
value = result[1]
if isinstance(value, BaseException):
raise value
else:
yield result
+ if not (self.parsed >= self.toparse):
+ raise bb.parse.ParseError("Not all recipes parsed, parser thread killed/died? Exiting.", None)
+
+
def parse_next(self):
result = []
parsed = None
diff --git a/bitbake/lib/bb/data_smart.py b/bitbake/lib/bb/data_smart.py
index 8291ca65e309..65857a9c7941 100644
--- a/bitbake/lib/bb/data_smart.py
+++ b/bitbake/lib/bb/data_smart.py
@@ -17,7 +17,7 @@ BitBake build tools.
# Based on functions from the base bb module, Copyright 2003 Holger Schurig
import copy, re, sys, traceback
-from collections import MutableMapping
+from collections.abc import MutableMapping
import logging
import hashlib
import bb, bb.codeparser
@@ -403,7 +403,7 @@ class DataSmart(MutableMapping):
s = __expand_python_regexp__.sub(varparse.python_sub, s)
except SyntaxError as e:
# Likely unmatched brackets, just don't expand the expression
- if e.msg != "EOL while scanning string literal":
+ if e.msg != "EOL while scanning string literal" and not e.msg.startswith("unterminated string literal"):
raise
if s == olds:
break
diff --git a/bitbake/lib/bb/fetch2/__init__.py b/bitbake/lib/bb/fetch2/__init__.py
index dbf8b50e68a7..1005ec10c639 100644
--- a/bitbake/lib/bb/fetch2/__init__.py
+++ b/bitbake/lib/bb/fetch2/__init__.py
@@ -430,6 +430,7 @@ def uri_replace(ud, uri_find, uri_replace, replacements, d, mirrortarball=None):
uri_replace_decoded = list(decodeurl(uri_replace))
logger.debug2("For url %s comparing %s to %s" % (uri_decoded, uri_find_decoded, uri_replace_decoded))
result_decoded = ['', '', '', '', '', {}]
+ # 0 - type, 1 - host, 2 - path, 3 - user, 4- pswd, 5 - params
for loc, i in enumerate(uri_find_decoded):
result_decoded[loc] = uri_decoded[loc]
regexp = i
@@ -449,6 +450,9 @@ def uri_replace(ud, uri_find, uri_replace, replacements, d, mirrortarball=None):
for l in replacements:
uri_replace_decoded[loc][k] = uri_replace_decoded[loc][k].replace(l, replacements[l])
result_decoded[loc][k] = uri_replace_decoded[loc][k]
+ elif (loc == 3 or loc == 4) and uri_replace_decoded[loc]:
+ # User/password in the replacement is just a straight replacement
+ result_decoded[loc] = uri_replace_decoded[loc]
elif (re.match(regexp, uri_decoded[loc])):
if not uri_replace_decoded[loc]:
result_decoded[loc] = ""
diff --git a/bitbake/lib/bb/fetch2/perforce.py b/bitbake/lib/bb/fetch2/perforce.py
index e2a41a4a1287..3b6fa4b1ec9a 100644
--- a/bitbake/lib/bb/fetch2/perforce.py
+++ b/bitbake/lib/bb/fetch2/perforce.py
@@ -134,7 +134,7 @@ class Perforce(FetchMethod):
ud.setup_revisions(d)
- ud.localfile = d.expand('%s_%s_%s_%s.tar.gz' % (cleanedhost, cleanedpath, cleandedmodule, ud.revision))
+ ud.localfile = d.expand('%s_%s_%s_%s.tar.gz' % (cleanedhost, cleanedpath, cleanedmodule, ud.revision))
def _buildp4command(self, ud, d, command, depot_filename=None):
"""
diff --git a/bitbake/lib/bb/fetch2/wget.py b/bitbake/lib/bb/fetch2/wget.py
index 784df70c9f62..7fa2a87ffde5 100644
--- a/bitbake/lib/bb/fetch2/wget.py
+++ b/bitbake/lib/bb/fetch2/wget.py
@@ -322,7 +322,7 @@ class Wget(FetchMethod):
except (TypeError, ImportError, IOError, netrc.NetrcParseError):
pass
- with opener.open(r) as response:
+ with opener.open(r, timeout=30) as response:
pass
except urllib.error.URLError as e:
if try_again:
diff --git a/bitbake/lib/bb/persist_data.py b/bitbake/lib/bb/persist_data.py
index c6a209fb3fc1..6f32d81afe80 100644
--- a/bitbake/lib/bb/persist_data.py
+++ b/bitbake/lib/bb/persist_data.py
@@ -12,6 +12,7 @@ currently, providing a key/value store accessed by 'domain'.
#
import collections
+import collections.abc
import contextlib
import functools
import logging
@@ -19,7 +20,7 @@ import os.path
import sqlite3
import sys
import warnings
-from collections import Mapping
+from collections.abc import Mapping
sqlversion = sqlite3.sqlite_version_info
if sqlversion[0] < 3 or (sqlversion[0] == 3 and sqlversion[1] < 3):
@@ -29,7 +30,7 @@ if sqlversion[0] < 3 or (sqlversion[0] == 3 and sqlversion[1] < 3):
logger = logging.getLogger("BitBake.PersistData")
@functools.total_ordering
-class SQLTable(collections.MutableMapping):
+class SQLTable(collections.abc.MutableMapping):
class _Decorators(object):
@staticmethod
def retry(*, reconnect=True):
diff --git a/bitbake/lib/bb/process.py b/bitbake/lib/bb/process.py
index d5a1775fcec0..af5d804a1d59 100644
--- a/bitbake/lib/bb/process.py
+++ b/bitbake/lib/bb/process.py
@@ -60,7 +60,7 @@ class Popen(subprocess.Popen):
"close_fds": True,
"preexec_fn": subprocess_setup,
"stdout": subprocess.PIPE,
- "stderr": subprocess.STDOUT,
+ "stderr": subprocess.PIPE,
"stdin": subprocess.PIPE,
"shell": False,
}
diff --git a/bitbake/lib/bb/runqueue.py b/bitbake/lib/bb/runqueue.py
index 10511a09dc1c..cd10da8b3a6f 100644
--- a/bitbake/lib/bb/runqueue.py
+++ b/bitbake/lib/bb/runqueue.py
@@ -926,38 +926,36 @@ class RunQueueData:
#
# Once all active tasks are marked, prune the ones we don't need.
- delcount = {}
- for tid in list(self.runtaskentries.keys()):
- if tid not in runq_build:
- delcount[tid] = self.runtaskentries[tid]
- del self.runtaskentries[tid]
-
# Handle --runall
if self.cooker.configuration.runall:
# re-run the mark_active and then drop unused tasks from new list
+ reduced_tasklist = set(self.runtaskentries.keys())
+ for tid in list(self.runtaskentries.keys()):
+ if tid not in runq_build:
+ reduced_tasklist.remove(tid)
runq_build = {}
for task in self.cooker.configuration.runall:
if not task.startswith("do_"):
task = "do_{0}".format(task)
runall_tids = set()
- for tid in list(self.runtaskentries):
+ for tid in reduced_tasklist:
wanttid = "{0}:{1}".format(fn_from_tid(tid), task)
- if wanttid in delcount:
- self.runtaskentries[wanttid] = delcount[wanttid]
if wanttid in self.runtaskentries:
runall_tids.add(wanttid)
for tid in list(runall_tids):
- mark_active(tid,1)
+ mark_active(tid, 1)
if self.cooker.configuration.force:
invalidate_task(tid, False)
- for tid in list(self.runtaskentries.keys()):
- if tid not in runq_build:
- delcount[tid] = self.runtaskentries[tid]
- del self.runtaskentries[tid]
+ delcount = set()
+ for tid in list(self.runtaskentries.keys()):
+ if tid not in runq_build:
+ delcount.add(tid)
+ del self.runtaskentries[tid]
+ if self.cooker.configuration.runall:
if len(self.runtaskentries) == 0:
bb.msg.fatal("RunQueue", "Could not find any tasks with the tasknames %s to run within the recipes of the taskgraphs of the targets %s" % (str(self.cooker.configuration.runall), str(self.targets)))
@@ -971,16 +969,16 @@ class RunQueueData:
for task in self.cooker.configuration.runonly:
if not task.startswith("do_"):
task = "do_{0}".format(task)
- runonly_tids = { k: v for k, v in self.runtaskentries.items() if taskname_from_tid(k) == task }
+ runonly_tids = [k for k in self.runtaskentries.keys() if taskname_from_tid(k) == task]
- for tid in list(runonly_tids):
- mark_active(tid,1)
+ for tid in runonly_tids:
+ mark_active(tid, 1)
if self.cooker.configuration.force:
invalidate_task(tid, False)
for tid in list(self.runtaskentries.keys()):
if tid not in runq_build:
- delcount[tid] = self.runtaskentries[tid]
+ delcount.add(tid)
del self.runtaskentries[tid]
if len(self.runtaskentries) == 0:
diff --git a/bitbake/lib/bb/server/process.py b/bitbake/lib/bb/server/process.py
index 07bb785a1822..fcdce19717d2 100644
--- a/bitbake/lib/bb/server/process.py
+++ b/bitbake/lib/bb/server/process.py
@@ -659,7 +659,7 @@ class BBUIEventQueue:
self.reader = ConnectionReader(readfd)
self.t = threading.Thread()
- self.t.setDaemon(True)
+ self.t.daemon = True
self.t.run = self.startCallbackHandler
self.t.start()
diff --git a/bitbake/lib/bb/tests/fetch.py b/bitbake/lib/bb/tests/fetch.py
index f5d557e8c000..3b64584da0e9 100644
--- a/bitbake/lib/bb/tests/fetch.py
+++ b/bitbake/lib/bb/tests/fetch.py
@@ -431,6 +431,10 @@ class MirrorUriTest(FetcherTest):
("git://someserver.org/bitbake;tag=1234567890123456789012345678901234567890;branch=master", "git://someserver.org/bitbake;branch=master", "git://git.openembedded.org/bitbake;protocol=http")
: "git://git.openembedded.org/bitbake;tag=1234567890123456789012345678901234567890;branch=master;protocol=http",
+ ("git://user1@someserver.org/bitbake;tag=1234567890123456789012345678901234567890;branch=master", "git://someserver.org/bitbake;branch=master", "git://user2@git.openembedded.org/bitbake;protocol=http")
+ : "git://user2@git.openembedded.org/bitbake;tag=1234567890123456789012345678901234567890;branch=master;protocol=http",
+
+
#Renaming files doesn't work
#("http://somewhere.org/somedir1/somefile_1.2.3.tar.gz", "http://somewhere.org/somedir1/somefile_1.2.3.tar.gz", "http://somewhere2.org/somedir3/somefile_2.3.4.tar.gz") : "http://somewhere2.org/somedir3/somefile_2.3.4.tar.gz"
#("file://sstate-xyz.tgz", "file://.*/.*", "file:///somewhere/1234/sstate-cache") : "file:///somewhere/1234/sstate-cache/sstate-xyz.tgz",
@@ -491,7 +495,7 @@ class GitDownloadDirectoryNamingTest(FetcherTest):
super(GitDownloadDirectoryNamingTest, self).setUp()
self.recipe_url = "git://git.openembedded.org/bitbake"
self.recipe_dir = "git.openembedded.org.bitbake"
- self.mirror_url = "git://github.com/openembedded/bitbake.git"
+ self.mirror_url = "git://github.com/openembedded/bitbake.git;protocol=https"
self.mirror_dir = "github.com.openembedded.bitbake.git"
self.d.setVar('SRCREV', '82ea737a0b42a8b53e11c9cde141e9e9c0bd8c40')
@@ -539,7 +543,7 @@ class TarballNamingTest(FetcherTest):
super(TarballNamingTest, self).setUp()
self.recipe_url = "git://git.openembedded.org/bitbake"
self.recipe_tarball = "git2_git.openembedded.org.bitbake.tar.gz"
- self.mirror_url = "git://github.com/openembedded/bitbake.git"
+ self.mirror_url = "git://github.com/openembedded/bitbake.git;protocol=https"
self.mirror_tarball = "git2_github.com.openembedded.bitbake.git.tar.gz"
self.d.setVar('BB_GENERATE_MIRROR_TARBALLS', '1')
@@ -573,7 +577,7 @@ class GitShallowTarballNamingTest(FetcherTest):
super(GitShallowTarballNamingTest, self).setUp()
self.recipe_url = "git://git.openembedded.org/bitbake"
self.recipe_tarball = "gitshallow_git.openembedded.org.bitbake_82ea737-1_master.tar.gz"
- self.mirror_url = "git://github.com/openembedded/bitbake.git"
+ self.mirror_url = "git://github.com/openembedded/bitbake.git;protocol=https"
self.mirror_tarball = "gitshallow_github.com.openembedded.bitbake.git_82ea737-1_master.tar.gz"
self.d.setVar('BB_GIT_SHALLOW', '1')
@@ -985,7 +989,7 @@ class FetcherNetworkTest(FetcherTest):
def test_git_submodule_dbus_broker(self):
# The following external repositories have show failures in fetch and unpack operations
# We want to avoid regressions!
- url = "gitsm://github.com/bus1/dbus-broker;protocol=git;rev=fc874afa0992d0c75ec25acb43d344679f0ee7d2;branch=main"
+ url = "gitsm://github.com/bus1/dbus-broker;protocol=https;rev=fc874afa0992d0c75ec25acb43d344679f0ee7d2;branch=main"
fetcher = bb.fetch.Fetch([url], self.d)
fetcher.download()
# Previous cwd has been deleted
@@ -1001,7 +1005,7 @@ class FetcherNetworkTest(FetcherTest):
@skipIfNoNetwork()
def test_git_submodule_CLI11(self):
- url = "gitsm://github.com/CLIUtils/CLI11;protocol=git;rev=bd4dc911847d0cde7a6b41dfa626a85aab213baf;branch=main"
+ url = "gitsm://github.com/CLIUtils/CLI11;protocol=https;rev=bd4dc911847d0cde7a6b41dfa626a85aab213baf;branch=main"
fetcher = bb.fetch.Fetch([url], self.d)
fetcher.download()
# Previous cwd has been deleted
@@ -1016,12 +1020,12 @@ class FetcherNetworkTest(FetcherTest):
@skipIfNoNetwork()
def test_git_submodule_update_CLI11(self):
""" Prevent regression on update detection not finding missing submodule, or modules without needed commits """
- url = "gitsm://github.com/CLIUtils/CLI11;protocol=git;rev=cf6a99fa69aaefe477cc52e3ef4a7d2d7fa40714;branch=main"
+ url = "gitsm://github.com/CLIUtils/CLI11;protocol=https;rev=cf6a99fa69aaefe477cc52e3ef4a7d2d7fa40714;branch=main"
fetcher = bb.fetch.Fetch([url], self.d)
fetcher.download()
# CLI11 that pulls in a newer nlohmann-json
- url = "gitsm://github.com/CLIUtils/CLI11;protocol=git;rev=49ac989a9527ee9bb496de9ded7b4872c2e0e5ca;branch=main"
+ url = "gitsm://github.com/CLIUtils/CLI11;protocol=https;rev=49ac989a9527ee9bb496de9ded7b4872c2e0e5ca;branch=main"
fetcher = bb.fetch.Fetch([url], self.d)
fetcher.download()
# Previous cwd has been deleted
@@ -1035,7 +1039,7 @@ class FetcherNetworkTest(FetcherTest):
@skipIfNoNetwork()
def test_git_submodule_aktualizr(self):
- url = "gitsm://github.com/advancedtelematic/aktualizr;branch=master;protocol=git;rev=d00d1a04cc2366d1a5f143b84b9f507f8bd32c44"
+ url = "gitsm://github.com/advancedtelematic/aktualizr;branch=master;protocol=https;rev=d00d1a04cc2366d1a5f143b84b9f507f8bd32c44"
fetcher = bb.fetch.Fetch([url], self.d)
fetcher.download()
# Previous cwd has been deleted
@@ -1055,7 +1059,7 @@ class FetcherNetworkTest(FetcherTest):
""" Prevent regression on deeply nested submodules not being checked out properly, even though they were fetched. """
# This repository also has submodules where the module (name), path and url do not align
- url = "gitsm://github.com/azure/iotedge.git;protocol=git;rev=d76e0316c6f324345d77c48a83ce836d09392699"
+ url = "gitsm://github.com/azure/iotedge.git;protocol=https;rev=d76e0316c6f324345d77c48a83ce836d09392699"
fetcher = bb.fetch.Fetch([url], self.d)
fetcher.download()
# Previous cwd has been deleted
@@ -1113,7 +1117,7 @@ class SVNTest(FetcherTest):
bb.process.run("svn co %s svnfetch_co" % self.repo_url, cwd=self.tempdir)
# Github will emulate SVN. Use this to check if we're downloding...
- bb.process.run("svn propset svn:externals 'bitbake svn://vcs.pcre.org/pcre2/code' .",
+ bb.process.run("svn propset svn:externals 'bitbake https://github.com/PhilipHazel/pcre2.git' .",
cwd=os.path.join(self.tempdir, 'svnfetch_co', 'trunk'))
bb.process.run("svn commit --non-interactive -m 'Add external'",
cwd=os.path.join(self.tempdir, 'svnfetch_co', 'trunk'))
@@ -1231,7 +1235,7 @@ class FetchLatestVersionTest(FetcherTest):
test_git_uris = {
# version pattern "X.Y.Z"
- ("mx-1.0", "git://github.com/clutter-project/mx.git;branch=mx-1.4", "9b1db6b8060bd00b121a692f942404a24ae2960f", "")
+ ("mx-1.0", "git://github.com/clutter-project/mx.git;branch=mx-1.4;protocol=https", "9b1db6b8060bd00b121a692f942404a24ae2960f", "")
: "1.99.4",
# version pattern "vX.Y"
# mirror of git.infradead.org since network issues interfered with testing
@@ -1258,9 +1262,9 @@ class FetchLatestVersionTest(FetcherTest):
: "0.4.3",
("build-appliance-image", "git://git.yoctoproject.org/poky", "b37dd451a52622d5b570183a81583cc34c2ff555", "(?P<pver>(([0-9][\.|_]?)+[0-9]))")
: "11.0.0",
- ("chkconfig-alternatives-native", "git://github.com/kergoth/chkconfig;branch=sysroot", "cd437ecbd8986c894442f8fce1e0061e20f04dee", "chkconfig\-(?P<pver>((\d+[\.\-_]*)+))")
+ ("chkconfig-alternatives-native", "git://github.com/kergoth/chkconfig;branch=sysroot;protocol=https", "cd437ecbd8986c894442f8fce1e0061e20f04dee", "chkconfig\-(?P<pver>((\d+[\.\-_]*)+))")
: "1.3.59",
- ("remake", "git://github.com/rocky/remake.git", "f05508e521987c8494c92d9c2871aec46307d51d", "(?P<pver>(\d+\.(\d+\.)*\d*(\+dbg\d+(\.\d+)*)*))")
+ ("remake", "git://github.com/rocky/remake.git;protocol=https", "f05508e521987c8494c92d9c2871aec46307d51d", "(?P<pver>(\d+\.(\d+\.)*\d*(\+dbg\d+(\.\d+)*)*))")
: "3.82+dbg0.9",
}
@@ -1354,9 +1358,6 @@ class FetchCheckStatusTest(FetcherTest):
"http://downloads.yoctoproject.org/releases/opkg/opkg-0.1.7.tar.gz",
"http://downloads.yoctoproject.org/releases/opkg/opkg-0.3.0.tar.gz",
"ftp://sourceware.org/pub/libffi/libffi-1.20.tar.gz",
- "http://ftp.gnu.org/gnu/autoconf/autoconf-2.60.tar.gz",
- "https://ftp.gnu.org/gnu/chess/gnuchess-5.08.tar.gz",
- "https://ftp.gnu.org/gnu/gmp/gmp-4.0.tar.gz",
# GitHub releases are hosted on Amazon S3, which doesn't support HEAD
"https://github.com/kergoth/tslib/releases/download/1.1/tslib-1.1.tar.xz"
]
@@ -2047,7 +2048,7 @@ class GitShallowTest(FetcherTest):
@skipIfNoNetwork()
def test_bitbake(self):
- self.git('remote add --mirror=fetch origin git://github.com/openembedded/bitbake', cwd=self.srcdir)
+ self.git('remote add --mirror=fetch origin https://github.com/openembedded/bitbake', cwd=self.srcdir)
self.git('config core.bare true', cwd=self.srcdir)
self.git('fetch', cwd=self.srcdir)
diff --git a/bitbake/lib/bb/utils.py b/bitbake/lib/bb/utils.py
index b282d09abfce..2a150fe9c73e 100644
--- a/bitbake/lib/bb/utils.py
+++ b/bitbake/lib/bb/utils.py
@@ -16,7 +16,8 @@ import bb.msg
import multiprocessing
import fcntl
import importlib
-from importlib import machinery
+import importlib.machinery
+import importlib.util
import itertools
import subprocess
import glob
@@ -451,6 +452,10 @@ def lockfile(name, shared=False, retry=True, block=False):
consider the possibility of sending a signal to the process to break
out - at which point you want block=True rather than retry=True.
"""
+ if len(name) > 255:
+ root, ext = os.path.splitext(name)
+ name = root[:255 - len(ext)] + ext
+
dirname = os.path.dirname(name)
mkdirhier(dirname)
@@ -487,7 +492,7 @@ def lockfile(name, shared=False, retry=True, block=False):
return lf
lf.close()
except OSError as e:
- if e.errno == errno.EACCES:
+ if e.errno == errno.EACCES or e.errno == errno.ENAMETOOLONG:
logger.error("Unable to acquire lock '%s', %s",
e.strerror, name)
sys.exit(1)
@@ -1616,7 +1621,9 @@ def load_plugins(logger, plugins, pluginpath):
logger.debug('Loading plugin %s' % name)
spec = importlib.machinery.PathFinder.find_spec(name, path=[pluginpath] )
if spec:
- return spec.loader.load_module()
+ mod = importlib.util.module_from_spec(spec)
+ spec.loader.exec_module(mod)
+ return mod
logger.debug('Loading plugins from %s...' % pluginpath)
diff --git a/bitbake/lib/hashserv/server.py b/bitbake/lib/hashserv/server.py
index a0dc0c170f2b..df0fa0a07937 100644
--- a/bitbake/lib/hashserv/server.py
+++ b/bitbake/lib/hashserv/server.py
@@ -521,7 +521,7 @@ class Server(object):
def start_tcp_server(self, host, port):
self.server = self.loop.run_until_complete(
- asyncio.start_server(self.handle_client, host, port, loop=self.loop)
+ asyncio.start_server(self.handle_client, host, port)
)
for s in self.server.sockets:
@@ -546,7 +546,7 @@ class Server(object):
# Work around path length limits in AF_UNIX
os.chdir(os.path.dirname(path))
self.server = self.loop.run_until_complete(
- asyncio.start_unix_server(self.handle_client, os.path.basename(path), loop=self.loop)
+ asyncio.start_unix_server(self.handle_client, os.path.basename(path))
)
finally:
os.chdir(cwd)
diff --git a/bitbake/lib/toaster/tests/builds/buildtest.py b/bitbake/lib/toaster/tests/builds/buildtest.py
index 872bbd377510..13b51fb0d8e4 100644
--- a/bitbake/lib/toaster/tests/builds/buildtest.py
+++ b/bitbake/lib/toaster/tests/builds/buildtest.py
@@ -119,7 +119,7 @@ class BuildTest(unittest.TestCase):
if os.environ.get("TOASTER_TEST_USE_SSTATE_MIRROR"):
ProjectVariable.objects.get_or_create(
name="SSTATE_MIRRORS",
- value="file://.* http://autobuilder.yoctoproject.org/pub/sstate/PATH;downloadfilename=PATH",
+ value="file://.* http://sstate.yoctoproject.org/PATH;downloadfilename=PATH",
project=project)
ProjectTarget.objects.create(project=project,
--
2.35.1
^ permalink raw reply [flat|nested] 22+ messages in thread
* [PATCH v2 2/2] wic: Update to the latest revision
2022-04-21 8:52 [PATCH v2 0/2] bump bitbake and wic for python 3.10 support Henning Schild
2022-04-21 8:52 ` [PATCH v2 1/2] bitbake: Update to 1.50.5 release Henning Schild
@ 2022-04-21 8:52 ` Henning Schild
2022-04-21 9:02 ` Henning Schild
2022-04-22 7:59 ` [PATCH v2 0/2] bump bitbake and wic for python 3.10 support Anton Mikanovich
2 siblings, 1 reply; 22+ messages in thread
From: Henning Schild @ 2022-04-21 8:52 UTC (permalink / raw)
To: isar-users; +Cc: Florian Bezdeka, Felix Moessbauer, Henning Schild
Update to the latest wic from OE-core.
OE-core Revision: 712552b5cc427d7be1258c45886de9b57f7272c9
Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
.../wic/plugins/source/bootimg-efi-isar.py | 77 ++++++++++++++++---
.../wic/plugins/source/bootimg-pcbios-isar.py | 6 +-
scripts/lib/wic/canned-wks/common.wks.inc | 2 +-
scripts/lib/wic/canned-wks/directdisk-gpt.wks | 2 +-
scripts/lib/wic/canned-wks/mkefidisk.wks | 2 +-
scripts/lib/wic/engine.py | 6 +-
scripts/lib/wic/help.py | 10 ++-
scripts/lib/wic/ksparser.py | 8 +-
scripts/lib/wic/misc.py | 4 +-
scripts/lib/wic/partition.py | 25 ++++--
scripts/lib/wic/pluginbase.py | 8 +-
scripts/lib/wic/plugins/imager/direct.py | 11 ++-
scripts/lib/wic/plugins/source/bootimg-efi.py | 74 +++++++++++++++---
.../lib/wic/plugins/source/bootimg-pcbios.py | 6 +-
scripts/lib/wic/plugins/source/rawcopy.py | 35 ++++++++-
scripts/lib/wic/plugins/source/rootfs.py | 2 +-
scripts/wic | 9 ++-
17 files changed, 231 insertions(+), 56 deletions(-)
diff --git a/meta/scripts/lib/wic/plugins/source/bootimg-efi-isar.py b/meta/scripts/lib/wic/plugins/source/bootimg-efi-isar.py
index 5ba0777ad244..a24e04f309da 100644
--- a/meta/scripts/lib/wic/plugins/source/bootimg-efi-isar.py
+++ b/meta/scripts/lib/wic/plugins/source/bootimg-efi-isar.py
@@ -12,6 +12,7 @@
import logging
import os
+import tempfile
import shutil
import re
@@ -129,12 +130,13 @@ class BootimgEFIPlugin(SourcePlugin):
bootloader = creator.ks.bootloader
loader_conf = ""
- loader_conf += "default boot\n"
+ if source_params.get('create-unified-kernel-image') != "true":
+ loader_conf += "default boot\n"
loader_conf += "timeout %d\n" % bootloader.timeout
initrd = source_params.get('initrd')
- if initrd:
+ if initrd and source_params.get('create-unified-kernel-image') != "true":
# obviously we need to have a common common deploy var
bootimg_dir = get_bitbake_var("DEPLOY_DIR_IMAGE")
if not bootimg_dir:
@@ -195,11 +197,12 @@ class BootimgEFIPlugin(SourcePlugin):
for rd in initrds:
boot_conf += "initrd /%s\n" % rd
- logger.debug("Writing systemd-boot config "
- "%s/hdd/boot/loader/entries/boot.conf", cr_workdir)
- cfg = open("%s/hdd/boot/loader/entries/boot.conf" % cr_workdir, "w")
- cfg.write(boot_conf)
- cfg.close()
+ if source_params.get('create-unified-kernel-image') != "true":
+ logger.debug("Writing systemd-boot config "
+ "%s/hdd/boot/loader/entries/boot.conf", cr_workdir)
+ cfg = open("%s/hdd/boot/loader/entries/boot.conf" % cr_workdir, "w")
+ cfg.write(boot_conf)
+ cfg.close()
@classmethod
@@ -300,11 +303,63 @@ class BootimgEFIPlugin(SourcePlugin):
kernel = "%s-%s.bin" % \
(get_bitbake_var("KERNEL_IMAGETYPE"), get_bitbake_var("INITRAMFS_LINK_NAME"))
- install_cmd = "install -m 0644 %s/%s %s/%s" % \
- (staging_kernel_dir, kernel, hdddir, kernel)
+ if source_params.get('create-unified-kernel-image') == "true":
+ initrd = source_params.get('initrd')
+ if not initrd:
+ raise WicError("initrd= must be specified when create-unified-kernel-image=true, exiting")
+
+ deploy_dir = get_bitbake_var("DEPLOY_DIR_IMAGE")
+ efi_stub = glob("%s/%s" % (deploy_dir, "linux*.efi.stub"))
+ if len(efi_stub) == 0:
+ raise WicError("Unified Kernel Image EFI stub not found, exiting")
+ efi_stub = efi_stub[0]
+
+ with tempfile.TemporaryDirectory() as tmp_dir:
+ label = source_params.get('label')
+ label_conf = "root=%s" % creator.rootdev
+ if label:
+ label_conf = "LABEL=%s" % label
+
+ bootloader = creator.ks.bootloader
+ cmdline = open("%s/cmdline" % tmp_dir, "w")
+ cmdline.write("%s %s" % (label_conf, bootloader.append))
+ cmdline.close()
+
+ initrds = initrd.split(';')
+ initrd = open("%s/initrd" % tmp_dir, "wb")
+ for f in initrds:
+ with open("%s/%s" % (deploy_dir, f), 'rb') as in_file:
+ shutil.copyfileobj(in_file, initrd)
+ initrd.close()
+
+ # Searched by systemd-boot:
+ # https://systemd.io/BOOT_LOADER_SPECIFICATION/#type-2-efi-unified-kernel-images
+ install_cmd = "install -d %s/EFI/Linux" % hdddir
+ exec_cmd(install_cmd)
+
+ staging_dir_host = get_bitbake_var("STAGING_DIR_HOST")
+
+ # https://www.freedesktop.org/software/systemd/man/systemd-stub.html
+ objcopy_cmd = "objcopy \
+ --add-section .osrel=%s --change-section-vma .osrel=0x20000 \
+ --add-section .cmdline=%s --change-section-vma .cmdline=0x30000 \
+ --add-section .linux=%s --change-section-vma .linux=0x2000000 \
+ --add-section .initrd=%s --change-section-vma .initrd=0x3000000 \
+ %s %s" % \
+ ("%s/usr/lib/os-release" % staging_dir_host,
+ cmdline.name,
+ "%s/%s" % (staging_kernel_dir, kernel),
+ initrd.name,
+ efi_stub,
+ "%s/EFI/Linux/linux.efi" % hdddir)
+ exec_cmd(objcopy_cmd)
+ else:
+ install_cmd = "install -m 0644 %s/%s %s/%s" % \
+ (staging_kernel_dir, kernel, hdddir, kernel)
+
+ install_cmd = isar_populate_boot_cmd(rootfs_dir['ROOTFS_DIR'], hdddir)
+ exec_cmd(install_cmd)
- install_cmd = isar_populate_boot_cmd(rootfs_dir['ROOTFS_DIR'], hdddir)
- exec_cmd(install_cmd)
if get_bitbake_var("IMAGE_EFI_BOOT_FILES"):
for src_path, dst_path in cls.install_task:
diff --git a/meta/scripts/lib/wic/plugins/source/bootimg-pcbios-isar.py b/meta/scripts/lib/wic/plugins/source/bootimg-pcbios-isar.py
index 9136d4f215ca..7fb0f2a87394 100644
--- a/meta/scripts/lib/wic/plugins/source/bootimg-pcbios-isar.py
+++ b/meta/scripts/lib/wic/plugins/source/bootimg-pcbios-isar.py
@@ -213,8 +213,10 @@ class BootimgPcbiosIsarPlugin(SourcePlugin):
# dosfs image, created by mkdosfs
bootimg = "%s/boot%s.img" % (cr_workdir, part.lineno)
- dosfs_cmd = "mkdosfs -n boot -i %s -S 512 -C %s %d" % \
- (part.fsuuid, bootimg, blocks)
+ label = part.label if part.label else "boot"
+
+ dosfs_cmd = "mkdosfs -n %s -i %s -S 512 -C %s %d" % \
+ (label, part.fsuuid, bootimg, blocks)
exec_native_cmd(dosfs_cmd, native_sysroot)
mcopy_cmd = "mcopy -i %s -s %s/* ::/" % (bootimg, hdddir)
diff --git a/scripts/lib/wic/canned-wks/common.wks.inc b/scripts/lib/wic/canned-wks/common.wks.inc
index 4fd29fa8c119..89880b417b6e 100644
--- a/scripts/lib/wic/canned-wks/common.wks.inc
+++ b/scripts/lib/wic/canned-wks/common.wks.inc
@@ -1,3 +1,3 @@
# This file is included into 3 canned wks files from this directory
part /boot --source bootimg-pcbios --ondisk sda --label boot --active --align 1024
-part / --source rootfs --use-uuid --fstype=ext4 --mkfs-extraopts "-T default" --label platform --align 1024
+part / --source rootfs --use-uuid --fstype=ext4 --label platform --align 1024
diff --git a/scripts/lib/wic/canned-wks/directdisk-gpt.wks b/scripts/lib/wic/canned-wks/directdisk-gpt.wks
index cf16c0c30bbd..8d7d8de6ea7e 100644
--- a/scripts/lib/wic/canned-wks/directdisk-gpt.wks
+++ b/scripts/lib/wic/canned-wks/directdisk-gpt.wks
@@ -4,7 +4,7 @@
part /boot --source bootimg-pcbios --ondisk sda --label boot --active --align 1024
-part / --source rootfs --ondisk sda --fstype=ext4 --mkfs-extraopts "-T default" --label platform --align 1024 --use-uuid
+part / --source rootfs --ondisk sda --fstype=ext4 --label platform --align 1024 --use-uuid
bootloader --ptable gpt --timeout=0 --append="rootwait rootfstype=ext4 video=vesafb vga=0x318 console=tty0 console=ttyS0,115200n8"
diff --git a/scripts/lib/wic/canned-wks/mkefidisk.wks b/scripts/lib/wic/canned-wks/mkefidisk.wks
index d1878e23e5a3..9f534fe18471 100644
--- a/scripts/lib/wic/canned-wks/mkefidisk.wks
+++ b/scripts/lib/wic/canned-wks/mkefidisk.wks
@@ -4,7 +4,7 @@
part /boot --source bootimg-efi --sourceparams="loader=grub-efi" --ondisk sda --label msdos --active --align 1024
-part / --source rootfs --ondisk sda --fstype=ext4 --mkfs-extraopts "-T default" --label platform --align 1024 --use-uuid
+part / --source rootfs --ondisk sda --fstype=ext4 --label platform --align 1024 --use-uuid
part swap --ondisk sda --size 44 --label swap1 --fstype=swap
diff --git a/scripts/lib/wic/engine.py b/scripts/lib/wic/engine.py
index 018815b96688..674ccfc24418 100644
--- a/scripts/lib/wic/engine.py
+++ b/scripts/lib/wic/engine.py
@@ -19,10 +19,10 @@ import os
import tempfile
import json
import subprocess
+import shutil
import re
from collections import namedtuple, OrderedDict
-from distutils.spawn import find_executable
from wic import WicError
from wic.filemap import sparse_copy
@@ -245,7 +245,7 @@ class Disk:
for path in pathlist.split(':'):
self.paths = "%s%s:%s" % (native_sysroot, path, self.paths)
- self.parted = find_executable("parted", self.paths)
+ self.parted = shutil.which("parted", path=self.paths)
if not self.parted:
raise WicError("Can't find executable parted")
@@ -283,7 +283,7 @@ class Disk:
"resize2fs", "mkswap", "mkdosfs", "debugfs","blkid"):
aname = "_%s" % name
if aname not in self.__dict__:
- setattr(self, aname, find_executable(name, self.paths))
+ setattr(self, aname, shutil.which(name, path=self.paths))
if aname not in self.__dict__ or self.__dict__[aname] is None:
raise WicError("Can't find executable '{}'".format(name))
return self.__dict__[aname]
diff --git a/scripts/lib/wic/help.py b/scripts/lib/wic/help.py
index bd3a2b97dfaf..4ff7470a6a79 100644
--- a/scripts/lib/wic/help.py
+++ b/scripts/lib/wic/help.py
@@ -637,7 +637,7 @@ DESCRIPTION
oe-core: directdisk.bbclass and mkefidisk.sh. The difference
between wic and those examples is that with wic the functionality
of those scripts is implemented by a general-purpose partitioning
- 'language' based on Redhat kickstart syntax).
+ 'language' based on Red Hat kickstart syntax).
The initial motivation and design considerations that lead to the
current tool are described exhaustively in Yocto Bug #3847
@@ -840,8 +840,8 @@ DESCRIPTION
meanings. The commands are based on the Fedora kickstart
documentation but with modifications to reflect wic capabilities.
- http://fedoraproject.org/wiki/Anaconda/Kickstart#part_or_partition
- http://fedoraproject.org/wiki/Anaconda/Kickstart#bootloader
+ https://pykickstart.readthedocs.io/en/latest/kickstart-docs.html#part-or-partition
+ https://pykickstart.readthedocs.io/en/latest/kickstart-docs.html#bootloader
Commands
@@ -930,6 +930,7 @@ DESCRIPTION
ext4
btrfs
squashfs
+ erofs
swap
--fsoptions: Specifies a free-form string of options to be
@@ -990,6 +991,9 @@ DESCRIPTION
multiple partitions and we want to keep the right
permissions and usernames in all the partitions.
+ --no-fstab-update: This option is specific to wic. It does not update the
+ '/etc/fstab' stock file for the given partition.
+
--extra-space: This option is specific to wic. It adds extra
space after the space filled by the content
of the partition. The final size can go
diff --git a/scripts/lib/wic/ksparser.py b/scripts/lib/wic/ksparser.py
index 3eb669da39ca..0df9eb0d057d 100644
--- a/scripts/lib/wic/ksparser.py
+++ b/scripts/lib/wic/ksparser.py
@@ -157,7 +157,8 @@ class KickStart():
part.add_argument('--fsoptions', dest='fsopts')
part.add_argument('--fstype', default='vfat',
choices=('ext2', 'ext3', 'ext4', 'btrfs',
- 'squashfs', 'vfat', 'msdos', 'swap'))
+ 'squashfs', 'vfat', 'msdos', 'erofs',
+ 'swap'))
part.add_argument('--mkfs-extraopts', default='')
part.add_argument('--label')
part.add_argument('--use-label', action='store_true')
@@ -184,6 +185,7 @@ class KickStart():
part.add_argument('--use-uuid', action='store_true')
part.add_argument('--uuid')
part.add_argument('--fsuuid')
+ part.add_argument('--no-fstab-update', action='store_true')
bootloader = subparsers.add_parser('bootloader')
bootloader.add_argument('--append')
@@ -229,6 +231,10 @@ class KickStart():
err = "%s:%d: SquashFS does not support LABEL" \
% (confpath, lineno)
raise KickStartError(err)
+ # erofs does not support filesystem labels
+ if parsed.fstype == 'erofs' and parsed.label:
+ err = "%s:%d: erofs does not support LABEL" % (confpath, lineno)
+ raise KickStartError(err)
if parsed.fstype == 'msdos' or parsed.fstype == 'vfat':
if parsed.fsuuid:
if parsed.fsuuid.upper().startswith('0X'):
diff --git a/scripts/lib/wic/misc.py b/scripts/lib/wic/misc.py
index 57c042c503e6..3e118229960b 100644
--- a/scripts/lib/wic/misc.py
+++ b/scripts/lib/wic/misc.py
@@ -16,9 +16,9 @@ import logging
import os
import re
import subprocess
+import shutil
from collections import defaultdict
-from distutils import spawn
from wic import WicError
@@ -122,7 +122,7 @@ def find_executable(cmd, paths):
if provided and "%s-native" % recipe in provided:
return True
- return spawn.find_executable(cmd, paths)
+ return shutil.which(cmd, path=paths)
def exec_native_cmd(cmd_and_args, native_sysroot, pseudo=""):
"""
diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py
index 76d144d12d30..09e491dd494d 100644
--- a/scripts/lib/wic/partition.py
+++ b/scripts/lib/wic/partition.py
@@ -54,6 +54,7 @@ class Partition():
self.uuid = args.uuid
self.fsuuid = args.fsuuid
self.type = args.type
+ self.no_fstab_update = args.no_fstab_update
self.updated_fstab_path = None
self.has_fstab = False
self.update_fstab_in_rootfs = False
@@ -104,7 +105,7 @@ class Partition():
extra_blocks = self.extra_space
rootfs_size = actual_rootfs_size + extra_blocks
- rootfs_size *= self.overhead_factor
+ rootfs_size = int(rootfs_size * self.overhead_factor)
logger.debug("Added %d extra blocks to %s to get to %d total blocks",
extra_blocks, self.mountpoint, rootfs_size)
@@ -141,9 +142,9 @@ class Partition():
native_sysroot)
self.source_file = "%s/fs.%s" % (cr_workdir, self.fstype)
else:
- if self.fstype == 'squashfs':
- raise WicError("It's not possible to create empty squashfs "
- "partition '%s'" % (self.mountpoint))
+ if self.fstype in ('squashfs', 'erofs'):
+ raise WicError("It's not possible to create empty %s "
+ "partition '%s'" % (self.fstype, self.mountpoint))
rootfs = "%s/fs_%s.%s.%s" % (cr_workdir, self.label,
self.lineno, self.fstype)
@@ -170,7 +171,7 @@ class Partition():
# Split sourceparams string of the form key1=val1[,key2=val2,...]
# into a dict. Also accepts valueless keys i.e. without =
splitted = self.sourceparams.split(',')
- srcparams_dict = dict(par.split('=', 1) for par in splitted if par)
+ srcparams_dict = dict((par.split('=', 1) + [None])[:2] for par in splitted if par)
plugin = PluginMgr.get_plugins('source')[self.source]
plugin.do_configure_partition(self, srcparams_dict, creator,
@@ -286,7 +287,7 @@ class Partition():
(self.fstype, extraopts, rootfs, label_str, self.fsuuid, rootfs_dir)
exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo)
- if self.updated_fstab_path and self.has_fstab:
+ if self.updated_fstab_path and self.has_fstab and not self.no_fstab_update:
debugfs_script_path = os.path.join(cr_workdir, "debugfs_script")
with open(debugfs_script_path, "w") as f:
f.write("cd etc\n")
@@ -350,7 +351,7 @@ class Partition():
mcopy_cmd = "mcopy -i %s -s %s/* ::/" % (rootfs, rootfs_dir)
exec_native_cmd(mcopy_cmd, native_sysroot)
- if self.updated_fstab_path and self.has_fstab:
+ if self.updated_fstab_path and self.has_fstab and not self.no_fstab_update:
mcopy_cmd = "mcopy -i %s %s ::/etc/fstab" % (rootfs, self.updated_fstab_path)
exec_native_cmd(mcopy_cmd, native_sysroot)
@@ -369,6 +370,16 @@ class Partition():
(rootfs_dir, rootfs, extraopts)
exec_native_cmd(squashfs_cmd, native_sysroot, pseudo=pseudo)
+ def prepare_rootfs_erofs(self, rootfs, cr_workdir, oe_builddir, rootfs_dir,
+ native_sysroot, pseudo):
+ """
+ Prepare content for a erofs rootfs partition.
+ """
+ extraopts = self.mkfs_extraopts or ''
+ erofs_cmd = "mkfs.erofs %s -U %s %s %s" % \
+ (extraopts, self.fsuuid, rootfs, rootfs_dir)
+ exec_native_cmd(erofs_cmd, native_sysroot, pseudo=pseudo)
+
def prepare_empty_partition_ext(self, rootfs, oe_builddir,
native_sysroot):
"""
diff --git a/scripts/lib/wic/pluginbase.py b/scripts/lib/wic/pluginbase.py
index d9b4e57747e7..b64568339b1b 100644
--- a/scripts/lib/wic/pluginbase.py
+++ b/scripts/lib/wic/pluginbase.py
@@ -9,9 +9,11 @@ __all__ = ['ImagerPlugin', 'SourcePlugin']
import os
import logging
+import types
from collections import defaultdict
-from importlib.machinery import SourceFileLoader
+import importlib
+import importlib.util
from wic import WicError
from wic.misc import get_bitbake_var
@@ -54,7 +56,9 @@ class PluginMgr:
mname = fname[:-3]
mpath = os.path.join(ppath, fname)
logger.debug("loading plugin module %s", mpath)
- SourceFileLoader(mname, mpath).load_module()
+ spec = importlib.util.spec_from_file_location(mname, mpath)
+ module = importlib.util.module_from_spec(spec)
+ spec.loader.exec_module(module)
return PLUGINS.get(ptype)
diff --git a/scripts/lib/wic/plugins/imager/direct.py b/scripts/lib/wic/plugins/imager/direct.py
index ea709e8c545d..35fff7c10242 100644
--- a/scripts/lib/wic/plugins/imager/direct.py
+++ b/scripts/lib/wic/plugins/imager/direct.py
@@ -77,7 +77,8 @@ class DirectPlugin(ImagerPlugin):
image_path = self._full_path(self.workdir, self.parts[0].disk, "direct")
self._image = PartitionedImage(image_path, self.ptable_format,
- self.parts, self.native_sysroot)
+ self.parts, self.native_sysroot,
+ options.extra_space)
def setup_workdir(self, workdir):
if workdir:
@@ -116,7 +117,7 @@ class DirectPlugin(ImagerPlugin):
updated = False
for part in self.parts:
if not part.realnum or not part.mountpoint \
- or part.mountpoint == "/":
+ or part.mountpoint == "/" or not part.mountpoint.startswith('/'):
continue
if part.use_uuid:
@@ -258,6 +259,8 @@ class DirectPlugin(ImagerPlugin):
if part.mountpoint == "/":
if part.uuid:
return "PARTUUID=%s" % part.uuid
+ elif part.label:
+ return "PARTLABEL=%s" % part.label
else:
suffix = 'p' if part.disk.startswith('mmcblk') else ''
return "/dev/%s%s%-d" % (part.disk, suffix, part.realnum)
@@ -293,7 +296,7 @@ class PartitionedImage():
Partitioned image in a file.
"""
- def __init__(self, path, ptable_format, partitions, native_sysroot=None):
+ def __init__(self, path, ptable_format, partitions, native_sysroot=None, extra_space=0):
self.path = path # Path to the image file
self.numpart = 0 # Number of allocated partitions
self.realpart = 0 # Number of partitions in the partition table
@@ -314,6 +317,7 @@ class PartitionedImage():
self.sector_size = SECTOR_SIZE
self.native_sysroot = native_sysroot
num_real_partitions = len([p for p in self.partitions if not p.no_table])
+ self.extra_space = extra_space
# calculate the real partition number, accounting for partitions not
# in the partition table and logical partitions
@@ -483,6 +487,7 @@ class PartitionedImage():
self.min_size += GPT_OVERHEAD
self.min_size *= self.sector_size
+ self.min_size += self.extra_space
def _create_partition(self, device, parttype, fstype, start, size):
""" Create a partition on an image described by the 'device' object. """
diff --git a/scripts/lib/wic/plugins/source/bootimg-efi.py b/scripts/lib/wic/plugins/source/bootimg-efi.py
index cdc72543c200..0391aebdc840 100644
--- a/scripts/lib/wic/plugins/source/bootimg-efi.py
+++ b/scripts/lib/wic/plugins/source/bootimg-efi.py
@@ -12,6 +12,7 @@
import logging
import os
+import tempfile
import shutil
import re
@@ -119,12 +120,13 @@ class BootimgEFIPlugin(SourcePlugin):
bootloader = creator.ks.bootloader
loader_conf = ""
- loader_conf += "default boot\n"
+ if source_params.get('create-unified-kernel-image') != "true":
+ loader_conf += "default boot\n"
loader_conf += "timeout %d\n" % bootloader.timeout
initrd = source_params.get('initrd')
- if initrd:
+ if initrd and source_params.get('create-unified-kernel-image') != "true":
# obviously we need to have a common common deploy var
bootimg_dir = get_bitbake_var("DEPLOY_DIR_IMAGE")
if not bootimg_dir:
@@ -183,11 +185,12 @@ class BootimgEFIPlugin(SourcePlugin):
for rd in initrds:
boot_conf += "initrd /%s\n" % rd
- logger.debug("Writing systemd-boot config "
- "%s/hdd/boot/loader/entries/boot.conf", cr_workdir)
- cfg = open("%s/hdd/boot/loader/entries/boot.conf" % cr_workdir, "w")
- cfg.write(boot_conf)
- cfg.close()
+ if source_params.get('create-unified-kernel-image') != "true":
+ logger.debug("Writing systemd-boot config "
+ "%s/hdd/boot/loader/entries/boot.conf", cr_workdir)
+ cfg = open("%s/hdd/boot/loader/entries/boot.conf" % cr_workdir, "w")
+ cfg.write(boot_conf)
+ cfg.close()
@classmethod
@@ -288,9 +291,60 @@ class BootimgEFIPlugin(SourcePlugin):
kernel = "%s-%s.bin" % \
(get_bitbake_var("KERNEL_IMAGETYPE"), get_bitbake_var("INITRAMFS_LINK_NAME"))
- install_cmd = "install -m 0644 %s/%s %s/%s" % \
- (staging_kernel_dir, kernel, hdddir, kernel)
- exec_cmd(install_cmd)
+ if source_params.get('create-unified-kernel-image') == "true":
+ initrd = source_params.get('initrd')
+ if not initrd:
+ raise WicError("initrd= must be specified when create-unified-kernel-image=true, exiting")
+
+ deploy_dir = get_bitbake_var("DEPLOY_DIR_IMAGE")
+ efi_stub = glob("%s/%s" % (deploy_dir, "linux*.efi.stub"))
+ if len(efi_stub) == 0:
+ raise WicError("Unified Kernel Image EFI stub not found, exiting")
+ efi_stub = efi_stub[0]
+
+ with tempfile.TemporaryDirectory() as tmp_dir:
+ label = source_params.get('label')
+ label_conf = "root=%s" % creator.rootdev
+ if label:
+ label_conf = "LABEL=%s" % label
+
+ bootloader = creator.ks.bootloader
+ cmdline = open("%s/cmdline" % tmp_dir, "w")
+ cmdline.write("%s %s" % (label_conf, bootloader.append))
+ cmdline.close()
+
+ initrds = initrd.split(';')
+ initrd = open("%s/initrd" % tmp_dir, "wb")
+ for f in initrds:
+ with open("%s/%s" % (deploy_dir, f), 'rb') as in_file:
+ shutil.copyfileobj(in_file, initrd)
+ initrd.close()
+
+ # Searched by systemd-boot:
+ # https://systemd.io/BOOT_LOADER_SPECIFICATION/#type-2-efi-unified-kernel-images
+ install_cmd = "install -d %s/EFI/Linux" % hdddir
+ exec_cmd(install_cmd)
+
+ staging_dir_host = get_bitbake_var("STAGING_DIR_HOST")
+
+ # https://www.freedesktop.org/software/systemd/man/systemd-stub.html
+ objcopy_cmd = "objcopy \
+ --add-section .osrel=%s --change-section-vma .osrel=0x20000 \
+ --add-section .cmdline=%s --change-section-vma .cmdline=0x30000 \
+ --add-section .linux=%s --change-section-vma .linux=0x2000000 \
+ --add-section .initrd=%s --change-section-vma .initrd=0x3000000 \
+ %s %s" % \
+ ("%s/usr/lib/os-release" % staging_dir_host,
+ cmdline.name,
+ "%s/%s" % (staging_kernel_dir, kernel),
+ initrd.name,
+ efi_stub,
+ "%s/EFI/Linux/linux.efi" % hdddir)
+ exec_cmd(objcopy_cmd)
+ else:
+ install_cmd = "install -m 0644 %s/%s %s/%s" % \
+ (staging_kernel_dir, kernel, hdddir, kernel)
+ exec_cmd(install_cmd)
if get_bitbake_var("IMAGE_EFI_BOOT_FILES"):
for src_path, dst_path in cls.install_task:
diff --git a/scripts/lib/wic/plugins/source/bootimg-pcbios.py b/scripts/lib/wic/plugins/source/bootimg-pcbios.py
index f2639e700493..32e47f183146 100644
--- a/scripts/lib/wic/plugins/source/bootimg-pcbios.py
+++ b/scripts/lib/wic/plugins/source/bootimg-pcbios.py
@@ -186,8 +186,10 @@ class BootimgPcbiosPlugin(SourcePlugin):
# dosfs image, created by mkdosfs
bootimg = "%s/boot%s.img" % (cr_workdir, part.lineno)
- dosfs_cmd = "mkdosfs -n boot -i %s -S 512 -C %s %d" % \
- (part.fsuuid, bootimg, blocks)
+ label = part.label if part.label else "boot"
+
+ dosfs_cmd = "mkdosfs -n %s -i %s -S 512 -C %s %d" % \
+ (label, part.fsuuid, bootimg, blocks)
exec_native_cmd(dosfs_cmd, native_sysroot)
mcopy_cmd = "mcopy -i %s -s %s/* ::/" % (bootimg, hdddir)
diff --git a/scripts/lib/wic/plugins/source/rawcopy.py b/scripts/lib/wic/plugins/source/rawcopy.py
index 3c4997d8ba5e..7c90cd3cf82b 100644
--- a/scripts/lib/wic/plugins/source/rawcopy.py
+++ b/scripts/lib/wic/plugins/source/rawcopy.py
@@ -4,6 +4,8 @@
import logging
import os
+import signal
+import subprocess
from wic import WicError
from wic.pluginbase import SourcePlugin
@@ -29,15 +31,34 @@ class RawCopyPlugin(SourcePlugin):
cmd = 'btrfs filesystem label %s %s' % (dst, label)
elif fstype == 'swap':
cmd = 'mkswap -L %s %s' % (label, dst)
- elif fstype == 'squashfs':
- raise WicError("It's not possible to update a squashfs "
- "filesystem label '%s'" % (label))
+ elif fstype in ('squashfs', 'erofs'):
+ raise WicError("It's not possible to update a %s "
+ "filesystem label '%s'" % (fstype, label))
else:
raise WicError("Cannot update filesystem label: "
"Unknown fstype: '%s'" % (fstype))
exec_cmd(cmd)
+ @staticmethod
+ def do_image_uncompression(src, dst, workdir):
+ def subprocess_setup():
+ # Python installs a SIGPIPE handler by default. This is usually not what
+ # non-Python subprocesses expect.
+ # SIGPIPE errors are known issues with gzip/bash
+ signal.signal(signal.SIGPIPE, signal.SIG_DFL)
+
+ extension = os.path.splitext(src)[1]
+ decompressor = {
+ ".bz2": "bzip2",
+ ".gz": "gzip",
+ ".xz": "xz"
+ }.get(extension)
+ if not decompressor:
+ raise WicError("Not supported compressor filename extension: %s" % extension)
+ cmd = "%s -dc %s > %s" % (decompressor, src, dst)
+ subprocess.call(cmd, preexec_fn=subprocess_setup, shell=True, cwd=workdir)
+
@classmethod
def do_prepare_partition(cls, part, source_params, cr, cr_workdir,
oe_builddir, bootimg_dir, kernel_dir,
@@ -56,7 +77,13 @@ class RawCopyPlugin(SourcePlugin):
if 'file' not in source_params:
raise WicError("No file specified")
- src = os.path.join(kernel_dir, source_params['file'])
+ if 'unpack' in source_params:
+ img = os.path.join(kernel_dir, source_params['file'])
+ src = os.path.join(cr_workdir, os.path.splitext(source_params['file'])[0])
+ RawCopyPlugin.do_image_uncompression(img, src, cr_workdir)
+ else:
+ src = os.path.join(kernel_dir, source_params['file'])
+
dst = os.path.join(cr_workdir, "%s.%s" % (os.path.basename(source_params['file']), part.lineno))
if not os.path.exists(os.path.dirname(dst)):
diff --git a/scripts/lib/wic/plugins/source/rootfs.py b/scripts/lib/wic/plugins/source/rootfs.py
index 96d940a91d61..2e34e715ca73 100644
--- a/scripts/lib/wic/plugins/source/rootfs.py
+++ b/scripts/lib/wic/plugins/source/rootfs.py
@@ -218,7 +218,7 @@ class RootfsPlugin(SourcePlugin):
# Update part.has_fstab here as fstab may have been added or
# removed by the above modifications.
part.has_fstab = os.path.exists(os.path.join(new_rootfs, "etc/fstab"))
- if part.update_fstab_in_rootfs and part.has_fstab:
+ if part.update_fstab_in_rootfs and part.has_fstab and not part.no_fstab_update:
fstab_path = os.path.join(new_rootfs, "etc/fstab")
# Assume that fstab should always be owned by root with fixed permissions
install_cmd = "install -m 0644 %s %s" % (part.updated_fstab_path, fstab_path)
diff --git a/scripts/wic b/scripts/wic
index a741aed364bf..aee63a45aacf 100755
--- a/scripts/wic
+++ b/scripts/wic
@@ -22,9 +22,9 @@ import sys
import argparse
import logging
import subprocess
+import shutil
from collections import namedtuple
-from distutils import spawn
# External modules
scripts_path = os.path.dirname(os.path.realpath(__file__))
@@ -47,7 +47,7 @@ if os.environ.get('SDKTARGETSYSROOT'):
break
sdkroot = os.path.dirname(sdkroot)
-bitbake_exe = spawn.find_executable('bitbake')
+bitbake_exe = shutil.which('bitbake')
if bitbake_exe:
bitbake_path = scriptpath.add_bitbake_lib_path()
import bb
@@ -159,6 +159,9 @@ def wic_create_subcommand(options, usage_str):
"(Use -e/--image-name to specify it)")
native_sysroot = options.native_sysroot
+ if options.kernel_dir:
+ kernel_dir = options.kernel_dir
+
if not options.vars_dir and (not native_sysroot or not os.path.isdir(native_sysroot)):
logger.info("Building wic-tools...\n")
subprocess.check_call(["bitbake", "wic-tools"])
@@ -346,6 +349,8 @@ def wic_init_parser_create(subparser):
default=False, help="output debug information")
subparser.add_argument("-i", "--imager", dest="imager",
default="direct", help="the wic imager plugin")
+ subparser.add_argument("--extra-space", type=int, dest="extra_space",
+ default=0, help="additional free disk space to add to the image")
return
--
2.35.1
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH v2 2/2] wic: Update to the latest revision
2022-04-21 8:52 ` [PATCH v2 2/2] wic: Update to the latest revision Henning Schild
@ 2022-04-21 9:02 ` Henning Schild
0 siblings, 0 replies; 22+ messages in thread
From: Henning Schild @ 2022-04-21 9:02 UTC (permalink / raw)
To: isar-users; +Cc: Florian Bezdeka, Felix Moessbauer
Am Thu, 21 Apr 2022 10:52:32 +0200
schrieb Henning Schild <henning.schild@siemens.com>:
> Update to the latest wic from OE-core.
>
> OE-core Revision: 712552b5cc427d7be1258c45886de9b57f7272c9
>
> Signed-off-by: Henning Schild <henning.schild@siemens.com>
> ---
> .../wic/plugins/source/bootimg-efi-isar.py | 77
> ++++++++++++++++--- .../wic/plugins/source/bootimg-pcbios-isar.py |
> 6 +- scripts/lib/wic/canned-wks/common.wks.inc | 2 +-
> scripts/lib/wic/canned-wks/directdisk-gpt.wks | 2 +-
> scripts/lib/wic/canned-wks/mkefidisk.wks | 2 +-
> scripts/lib/wic/engine.py | 6 +-
> scripts/lib/wic/help.py | 10 ++-
> scripts/lib/wic/ksparser.py | 8 +-
> scripts/lib/wic/misc.py | 4 +-
> scripts/lib/wic/partition.py | 25 ++++--
> scripts/lib/wic/pluginbase.py | 8 +-
> scripts/lib/wic/plugins/imager/direct.py | 11 ++-
> scripts/lib/wic/plugins/source/bootimg-efi.py | 74 +++++++++++++++---
> .../lib/wic/plugins/source/bootimg-pcbios.py | 6 +-
> scripts/lib/wic/plugins/source/rawcopy.py | 35 ++++++++-
> scripts/lib/wic/plugins/source/rootfs.py | 2 +-
> scripts/wic | 9 ++-
> 17 files changed, 231 insertions(+), 56 deletions(-)
>
> diff --git a/meta/scripts/lib/wic/plugins/source/bootimg-efi-isar.py
> b/meta/scripts/lib/wic/plugins/source/bootimg-efi-isar.py index
> 5ba0777ad244..a24e04f309da 100644 ---
> a/meta/scripts/lib/wic/plugins/source/bootimg-efi-isar.py +++
> b/meta/scripts/lib/wic/plugins/source/bootimg-efi-isar.py @@ -12,6
> +12,7 @@
> import logging
> import os
> +import tempfile
> import shutil
> import re
>
> @@ -129,12 +130,13 @@ class BootimgEFIPlugin(SourcePlugin):
> bootloader = creator.ks.bootloader
>
> loader_conf = ""
> - loader_conf += "default boot\n"
> + if source_params.get('create-unified-kernel-image') !=
> "true":
> + loader_conf += "default boot\n"
> loader_conf += "timeout %d\n" % bootloader.timeout
>
> initrd = source_params.get('initrd')
>
> - if initrd:
> + if initrd and
> source_params.get('create-unified-kernel-image') != "true": #
> obviously we need to have a common common deploy var bootimg_dir =
> get_bitbake_var("DEPLOY_DIR_IMAGE") if not bootimg_dir:
> @@ -195,11 +197,12 @@ class BootimgEFIPlugin(SourcePlugin):
> for rd in initrds:
> boot_conf += "initrd /%s\n" % rd
>
> - logger.debug("Writing systemd-boot config "
> - "%s/hdd/boot/loader/entries/boot.conf",
> cr_workdir)
> - cfg = open("%s/hdd/boot/loader/entries/boot.conf" %
> cr_workdir, "w")
> - cfg.write(boot_conf)
> - cfg.close()
> + if source_params.get('create-unified-kernel-image') !=
> "true":
> + logger.debug("Writing systemd-boot config "
> + "%s/hdd/boot/loader/entries/boot.conf",
> cr_workdir)
> + cfg = open("%s/hdd/boot/loader/entries/boot.conf" %
> cr_workdir, "w")
> + cfg.write(boot_conf)
> + cfg.close()
>
>
> @classmethod
> @@ -300,11 +303,63 @@ class BootimgEFIPlugin(SourcePlugin):
> kernel = "%s-%s.bin" % \
> (get_bitbake_var("KERNEL_IMAGETYPE"),
> get_bitbake_var("INITRAMFS_LINK_NAME"))
> - install_cmd = "install -m 0644 %s/%s %s/%s" % \
> - (staging_kernel_dir, kernel, hdddir, kernel)
> + if source_params.get('create-unified-kernel-image') ==
> "true":
> + initrd = source_params.get('initrd')
> + if not initrd:
> + raise WicError("initrd= must be specified when
> create-unified-kernel-image=true, exiting") +
> + deploy_dir = get_bitbake_var("DEPLOY_DIR_IMAGE")
> + efi_stub = glob("%s/%s" % (deploy_dir,
> "linux*.efi.stub"))
> + if len(efi_stub) == 0:
> + raise WicError("Unified Kernel Image EFI stub not
> found, exiting")
> + efi_stub = efi_stub[0]
> +
> + with tempfile.TemporaryDirectory() as tmp_dir:
> + label = source_params.get('label')
> + label_conf = "root=%s" % creator.rootdev
> + if label:
> + label_conf = "LABEL=%s" % label
> +
> + bootloader = creator.ks.bootloader
> + cmdline = open("%s/cmdline" % tmp_dir, "w")
> + cmdline.write("%s %s" % (label_conf,
> bootloader.append))
> + cmdline.close()
> +
> + initrds = initrd.split(';')
> + initrd = open("%s/initrd" % tmp_dir, "wb")
> + for f in initrds:
> + with open("%s/%s" % (deploy_dir, f), 'rb') as
> in_file:
> + shutil.copyfileobj(in_file, initrd)
> + initrd.close()
> +
> + # Searched by systemd-boot:
> + #
> https://systemd.io/BOOT_LOADER_SPECIFICATION/#type-2-efi-unified-kernel-images
> + install_cmd = "install -d %s/EFI/Linux" % hdddir
> + exec_cmd(install_cmd)
> +
> + staging_dir_host =
> get_bitbake_var("STAGING_DIR_HOST") +
> + #
> https://www.freedesktop.org/software/systemd/man/systemd-stub.html
> + objcopy_cmd = "objcopy \
> + --add-section .osrel=%s --change-section-vma
> .osrel=0x20000 \
> + --add-section .cmdline=%s --change-section-vma
> .cmdline=0x30000 \
> + --add-section .linux=%s --change-section-vma
> .linux=0x2000000 \
> + --add-section .initrd=%s --change-section-vma
> .initrd=0x3000000 \
> + %s %s" % \
> + ("%s/usr/lib/os-release" % staging_dir_host,
> + cmdline.name,
> + "%s/%s" % (staging_kernel_dir, kernel),
> + initrd.name,
> + efi_stub,
> + "%s/EFI/Linux/linux.efi" % hdddir)
> + exec_cmd(objcopy_cmd)
> + else:
> + install_cmd = "install -m 0644 %s/%s %s/%s" % \
> + (staging_kernel_dir, kernel, hdddir, kernel)
> +
> + install_cmd =
> isar_populate_boot_cmd(rootfs_dir['ROOTFS_DIR'], hdddir)
> + exec_cmd(install_cmd)
>
> - install_cmd =
> isar_populate_boot_cmd(rootfs_dir['ROOTFS_DIR'], hdddir)
> - exec_cmd(install_cmd)
>
> if get_bitbake_var("IMAGE_EFI_BOOT_FILES"):
> for src_path, dst_path in cls.install_task:
> diff --git
> a/meta/scripts/lib/wic/plugins/source/bootimg-pcbios-isar.py
> b/meta/scripts/lib/wic/plugins/source/bootimg-pcbios-isar.py index
> 9136d4f215ca..7fb0f2a87394 100644 ---
> a/meta/scripts/lib/wic/plugins/source/bootimg-pcbios-isar.py +++
> b/meta/scripts/lib/wic/plugins/source/bootimg-pcbios-isar.py @@
> -213,8 +213,10 @@ class BootimgPcbiosIsarPlugin(SourcePlugin): #
> dosfs image, created by mkdosfs bootimg = "%s/boot%s.img" %
> (cr_workdir, part.lineno)
> - dosfs_cmd = "mkdosfs -n boot -i %s -S 512 -C %s %d" % \
> - (part.fsuuid, bootimg, blocks)
> + label = part.label if part.label else "boot"
> +
> + dosfs_cmd = "mkdosfs -n %s -i %s -S 512 -C %s %d" % \
> + (label, part.fsuuid, bootimg, blocks)
> exec_native_cmd(dosfs_cmd, native_sysroot)
>
> mcopy_cmd = "mcopy -i %s -s %s/* ::/" % (bootimg, hdddir)
> diff --git a/scripts/lib/wic/canned-wks/common.wks.inc
> b/scripts/lib/wic/canned-wks/common.wks.inc index
> 4fd29fa8c119..89880b417b6e 100644 ---
> a/scripts/lib/wic/canned-wks/common.wks.inc +++
> b/scripts/lib/wic/canned-wks/common.wks.inc @@ -1,3 +1,3 @@
> # This file is included into 3 canned wks files from this directory
> part /boot --source bootimg-pcbios --ondisk sda --label boot
> --active --align 1024 -part / --source rootfs --use-uuid
> --fstype=ext4 --mkfs-extraopts "-T default" --label platform --align
> 1024 +part / --source rootfs --use-uuid --fstype=ext4 --label
> platform --align 1024 diff --git
This is
https://github.com/openembedded/openembedded-core/commit/7e8017208bed98b6c90735cb641fc9d7aedf9140
and it was not taken for our forks in
meta-isar/scripts/lib/wic/canned-wks/
Because we can not be sure to have a brandnew e2fsprogs always.
With scripts/wic_fakeroot we have a way to centrally hook into wic
plugins and play with arguments. But i think having our forked wks
files is nicer and we might eventually be able to drop wic_fakeroot, at
the moment it only deals with one remaining hack for old e2fsck
Henning
> a/scripts/lib/wic/canned-wks/directdisk-gpt.wks
> b/scripts/lib/wic/canned-wks/directdisk-gpt.wks index
> cf16c0c30bbd..8d7d8de6ea7e 100644 ---
> a/scripts/lib/wic/canned-wks/directdisk-gpt.wks +++
> b/scripts/lib/wic/canned-wks/directdisk-gpt.wks @@ -4,7 +4,7 @@
> part /boot --source bootimg-pcbios --ondisk sda --label boot
> --active --align 1024 -part / --source rootfs --ondisk sda
> --fstype=ext4 --mkfs-extraopts "-T default" --label platform --align
> 1024 --use-uuid +part / --source rootfs --ondisk sda --fstype=ext4
> --label platform --align 1024 --use-uuid bootloader --ptable gpt
> --timeout=0 --append="rootwait rootfstype=ext4 video=vesafb
> vga=0x318 console=tty0 console=ttyS0,115200n8" diff --git
> a/scripts/lib/wic/canned-wks/mkefidisk.wks
> b/scripts/lib/wic/canned-wks/mkefidisk.wks index
> d1878e23e5a3..9f534fe18471 100644 ---
> a/scripts/lib/wic/canned-wks/mkefidisk.wks +++
> b/scripts/lib/wic/canned-wks/mkefidisk.wks @@ -4,7 +4,7 @@
> part /boot --source bootimg-efi --sourceparams="loader=grub-efi"
> --ondisk sda --label msdos --active --align 1024
> -part / --source rootfs --ondisk sda --fstype=ext4 --mkfs-extraopts
> "-T default" --label platform --align 1024 --use-uuid +part /
> --source rootfs --ondisk sda --fstype=ext4 --label platform --align
> 1024 --use-uuid part swap --ondisk sda --size 44 --label swap1
> --fstype=swap
> diff --git a/scripts/lib/wic/engine.py b/scripts/lib/wic/engine.py
> index 018815b96688..674ccfc24418 100644
> --- a/scripts/lib/wic/engine.py
> +++ b/scripts/lib/wic/engine.py
> @@ -19,10 +19,10 @@ import os
> import tempfile
> import json
> import subprocess
> +import shutil
> import re
>
> from collections import namedtuple, OrderedDict
> -from distutils.spawn import find_executable
>
> from wic import WicError
> from wic.filemap import sparse_copy
> @@ -245,7 +245,7 @@ class Disk:
> for path in pathlist.split(':'):
> self.paths = "%s%s:%s" % (native_sysroot, path,
> self.paths)
> - self.parted = find_executable("parted", self.paths)
> + self.parted = shutil.which("parted", path=self.paths)
> if not self.parted:
> raise WicError("Can't find executable parted")
>
> @@ -283,7 +283,7 @@ class Disk:
> "resize2fs", "mkswap", "mkdosfs",
> "debugfs","blkid"): aname = "_%s" % name
> if aname not in self.__dict__:
> - setattr(self, aname, find_executable(name,
> self.paths))
> + setattr(self, aname, shutil.which(name,
> path=self.paths)) if aname not in self.__dict__ or
> self.__dict__[aname] is None: raise WicError("Can't find executable
> '{}'".format(name)) return self.__dict__[aname]
> diff --git a/scripts/lib/wic/help.py b/scripts/lib/wic/help.py
> index bd3a2b97dfaf..4ff7470a6a79 100644
> --- a/scripts/lib/wic/help.py
> +++ b/scripts/lib/wic/help.py
> @@ -637,7 +637,7 @@ DESCRIPTION
> oe-core: directdisk.bbclass and mkefidisk.sh. The difference
> between wic and those examples is that with wic the functionality
> of those scripts is implemented by a general-purpose partitioning
> - 'language' based on Redhat kickstart syntax).
> + 'language' based on Red Hat kickstart syntax).
>
> The initial motivation and design considerations that lead to the
> current tool are described exhaustively in Yocto Bug #3847
> @@ -840,8 +840,8 @@ DESCRIPTION
> meanings. The commands are based on the Fedora kickstart
> documentation but with modifications to reflect wic capabilities.
>
> -
> http://fedoraproject.org/wiki/Anaconda/Kickstart#part_or_partition
> - http://fedoraproject.org/wiki/Anaconda/Kickstart#bootloader
> +
> https://pykickstart.readthedocs.io/en/latest/kickstart-docs.html#part-or-partition
> +
> https://pykickstart.readthedocs.io/en/latest/kickstart-docs.html#bootloader
> Commands
>
> @@ -930,6 +930,7 @@ DESCRIPTION
> ext4
> btrfs
> squashfs
> + erofs
> swap
>
> --fsoptions: Specifies a free-form string of options to be
> @@ -990,6 +991,9 @@ DESCRIPTION
> multiple partitions and we want to keep
> the right permissions and usernames in all the partitions.
>
> + --no-fstab-update: This option is specific to wic. It does
> not update the
> + '/etc/fstab' stock file for the given
> partition. +
> --extra-space: This option is specific to wic. It adds extra
> space after the space filled by the content
> of the partition. The final size can go
> diff --git a/scripts/lib/wic/ksparser.py b/scripts/lib/wic/ksparser.py
> index 3eb669da39ca..0df9eb0d057d 100644
> --- a/scripts/lib/wic/ksparser.py
> +++ b/scripts/lib/wic/ksparser.py
> @@ -157,7 +157,8 @@ class KickStart():
> part.add_argument('--fsoptions', dest='fsopts')
> part.add_argument('--fstype', default='vfat',
> choices=('ext2', 'ext3', 'ext4', 'btrfs',
> - 'squashfs', 'vfat', 'msdos',
> 'swap'))
> + 'squashfs', 'vfat', 'msdos',
> 'erofs',
> + 'swap'))
> part.add_argument('--mkfs-extraopts', default='')
> part.add_argument('--label')
> part.add_argument('--use-label', action='store_true')
> @@ -184,6 +185,7 @@ class KickStart():
> part.add_argument('--use-uuid', action='store_true')
> part.add_argument('--uuid')
> part.add_argument('--fsuuid')
> + part.add_argument('--no-fstab-update', action='store_true')
>
> bootloader = subparsers.add_parser('bootloader')
> bootloader.add_argument('--append')
> @@ -229,6 +231,10 @@ class KickStart():
> err = "%s:%d: SquashFS does not
> support LABEL" \ % (confpath, lineno)
> raise KickStartError(err)
> + # erofs does not support filesystem labels
> + if parsed.fstype == 'erofs' and parsed.label:
> + err = "%s:%d: erofs does not support
> LABEL" % (confpath, lineno)
> + raise KickStartError(err)
> if parsed.fstype == 'msdos' or parsed.fstype
> == 'vfat': if parsed.fsuuid:
> if
> parsed.fsuuid.upper().startswith('0X'): diff --git
> a/scripts/lib/wic/misc.py b/scripts/lib/wic/misc.py index
> 57c042c503e6..3e118229960b 100644 --- a/scripts/lib/wic/misc.py
> +++ b/scripts/lib/wic/misc.py
> @@ -16,9 +16,9 @@ import logging
> import os
> import re
> import subprocess
> +import shutil
>
> from collections import defaultdict
> -from distutils import spawn
>
> from wic import WicError
>
> @@ -122,7 +122,7 @@ def find_executable(cmd, paths):
> if provided and "%s-native" % recipe in provided:
> return True
>
> - return spawn.find_executable(cmd, paths)
> + return shutil.which(cmd, path=paths)
>
> def exec_native_cmd(cmd_and_args, native_sysroot, pseudo=""):
> """
> diff --git a/scripts/lib/wic/partition.py
> b/scripts/lib/wic/partition.py index 76d144d12d30..09e491dd494d 100644
> --- a/scripts/lib/wic/partition.py
> +++ b/scripts/lib/wic/partition.py
> @@ -54,6 +54,7 @@ class Partition():
> self.uuid = args.uuid
> self.fsuuid = args.fsuuid
> self.type = args.type
> + self.no_fstab_update = args.no_fstab_update
> self.updated_fstab_path = None
> self.has_fstab = False
> self.update_fstab_in_rootfs = False
> @@ -104,7 +105,7 @@ class Partition():
> extra_blocks = self.extra_space
>
> rootfs_size = actual_rootfs_size + extra_blocks
> - rootfs_size *= self.overhead_factor
> + rootfs_size = int(rootfs_size * self.overhead_factor)
>
> logger.debug("Added %d extra blocks to %s to get to %d
> total blocks", extra_blocks, self.mountpoint, rootfs_size)
> @@ -141,9 +142,9 @@ class Partition():
> native_sysroot)
> self.source_file = "%s/fs.%s" % (cr_workdir,
> self.fstype) else:
> - if self.fstype == 'squashfs':
> - raise WicError("It's not possible to create
> empty squashfs "
> - "partition '%s'" %
> (self.mountpoint))
> + if self.fstype in ('squashfs', 'erofs'):
> + raise WicError("It's not possible to create
> empty %s "
> + "partition '%s'" % (self.fstype,
> self.mountpoint))
> rootfs = "%s/fs_%s.%s.%s" % (cr_workdir, self.label,
> self.lineno,
> self.fstype) @@ -170,7 +171,7 @@ class Partition():
> # Split sourceparams string of the form
> key1=val1[,key2=val2,...] # into a dict. Also accepts valueless keys
> i.e. without = splitted = self.sourceparams.split(',')
> - srcparams_dict = dict(par.split('=', 1) for par in
> splitted if par)
> + srcparams_dict = dict((par.split('=', 1) + [None])[:2]
> for par in splitted if par)
> plugin = PluginMgr.get_plugins('source')[self.source]
> plugin.do_configure_partition(self, srcparams_dict, creator,
> @@ -286,7 +287,7 @@ class Partition():
> (self.fstype, extraopts, rootfs, label_str, self.fsuuid,
> rootfs_dir) exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo)
>
> - if self.updated_fstab_path and self.has_fstab:
> + if self.updated_fstab_path and self.has_fstab and not
> self.no_fstab_update: debugfs_script_path = os.path.join(cr_workdir,
> "debugfs_script") with open(debugfs_script_path, "w") as f:
> f.write("cd etc\n")
> @@ -350,7 +351,7 @@ class Partition():
> mcopy_cmd = "mcopy -i %s -s %s/* ::/" % (rootfs, rootfs_dir)
> exec_native_cmd(mcopy_cmd, native_sysroot)
>
> - if self.updated_fstab_path and self.has_fstab:
> + if self.updated_fstab_path and self.has_fstab and not
> self.no_fstab_update: mcopy_cmd = "mcopy -i %s %s ::/etc/fstab" %
> (rootfs, self.updated_fstab_path) exec_native_cmd(mcopy_cmd,
> native_sysroot)
> @@ -369,6 +370,16 @@ class Partition():
> (rootfs_dir, rootfs, extraopts)
> exec_native_cmd(squashfs_cmd, native_sysroot, pseudo=pseudo)
>
> + def prepare_rootfs_erofs(self, rootfs, cr_workdir, oe_builddir,
> rootfs_dir,
> + native_sysroot, pseudo):
> + """
> + Prepare content for a erofs rootfs partition.
> + """
> + extraopts = self.mkfs_extraopts or ''
> + erofs_cmd = "mkfs.erofs %s -U %s %s %s" % \
> + (extraopts, self.fsuuid, rootfs, rootfs_dir)
> + exec_native_cmd(erofs_cmd, native_sysroot, pseudo=pseudo)
> +
> def prepare_empty_partition_ext(self, rootfs, oe_builddir,
> native_sysroot):
> """
> diff --git a/scripts/lib/wic/pluginbase.py
> b/scripts/lib/wic/pluginbase.py index d9b4e57747e7..b64568339b1b
> 100644 --- a/scripts/lib/wic/pluginbase.py
> +++ b/scripts/lib/wic/pluginbase.py
> @@ -9,9 +9,11 @@ __all__ = ['ImagerPlugin', 'SourcePlugin']
>
> import os
> import logging
> +import types
>
> from collections import defaultdict
> -from importlib.machinery import SourceFileLoader
> +import importlib
> +import importlib.util
>
> from wic import WicError
> from wic.misc import get_bitbake_var
> @@ -54,7 +56,9 @@ class PluginMgr:
> mname = fname[:-3]
> mpath = os.path.join(ppath, fname)
> logger.debug("loading plugin module %s",
> mpath)
> - SourceFileLoader(mname,
> mpath).load_module()
> + spec =
> importlib.util.spec_from_file_location(mname, mpath)
> + module =
> importlib.util.module_from_spec(spec)
> + spec.loader.exec_module(module)
>
> return PLUGINS.get(ptype)
>
> diff --git a/scripts/lib/wic/plugins/imager/direct.py
> b/scripts/lib/wic/plugins/imager/direct.py index
> ea709e8c545d..35fff7c10242 100644 ---
> a/scripts/lib/wic/plugins/imager/direct.py +++
> b/scripts/lib/wic/plugins/imager/direct.py @@ -77,7 +77,8 @@ class
> DirectPlugin(ImagerPlugin):
> image_path = self._full_path(self.workdir,
> self.parts[0].disk, "direct") self._image =
> PartitionedImage(image_path, self.ptable_format,
> - self.parts,
> self.native_sysroot)
> + self.parts,
> self.native_sysroot,
> + options.extra_space)
>
> def setup_workdir(self, workdir):
> if workdir:
> @@ -116,7 +117,7 @@ class DirectPlugin(ImagerPlugin):
> updated = False
> for part in self.parts:
> if not part.realnum or not part.mountpoint \
> - or part.mountpoint == "/":
> + or part.mountpoint == "/" or not
> part.mountpoint.startswith('/'): continue
>
> if part.use_uuid:
> @@ -258,6 +259,8 @@ class DirectPlugin(ImagerPlugin):
> if part.mountpoint == "/":
> if part.uuid:
> return "PARTUUID=%s" % part.uuid
> + elif part.label:
> + return "PARTLABEL=%s" % part.label
> else:
> suffix = 'p' if part.disk.startswith('mmcblk')
> else '' return "/dev/%s%s%-d" % (part.disk, suffix, part.realnum)
> @@ -293,7 +296,7 @@ class PartitionedImage():
> Partitioned image in a file.
> """
>
> - def __init__(self, path, ptable_format, partitions,
> native_sysroot=None):
> + def __init__(self, path, ptable_format, partitions,
> native_sysroot=None, extra_space=0): self.path = path # Path to the
> image file self.numpart = 0 # Number of allocated partitions
> self.realpart = 0 # Number of partitions in the partition
> table @@ -314,6 +317,7 @@ class PartitionedImage():
> self.sector_size = SECTOR_SIZE
> self.native_sysroot = native_sysroot
> num_real_partitions = len([p for p in self.partitions if not
> p.no_table])
> + self.extra_space = extra_space
>
> # calculate the real partition number, accounting for
> partitions not # in the partition table and logical partitions
> @@ -483,6 +487,7 @@ class PartitionedImage():
> self.min_size += GPT_OVERHEAD
>
> self.min_size *= self.sector_size
> + self.min_size += self.extra_space
>
> def _create_partition(self, device, parttype, fstype, start,
> size): """ Create a partition on an image described by the 'device'
> object. """ diff --git
> a/scripts/lib/wic/plugins/source/bootimg-efi.py
> b/scripts/lib/wic/plugins/source/bootimg-efi.py index
> cdc72543c200..0391aebdc840 100644 ---
> a/scripts/lib/wic/plugins/source/bootimg-efi.py +++
> b/scripts/lib/wic/plugins/source/bootimg-efi.py @@ -12,6 +12,7 @@
> import logging
> import os
> +import tempfile
> import shutil
> import re
>
> @@ -119,12 +120,13 @@ class BootimgEFIPlugin(SourcePlugin):
> bootloader = creator.ks.bootloader
>
> loader_conf = ""
> - loader_conf += "default boot\n"
> + if source_params.get('create-unified-kernel-image') !=
> "true":
> + loader_conf += "default boot\n"
> loader_conf += "timeout %d\n" % bootloader.timeout
>
> initrd = source_params.get('initrd')
>
> - if initrd:
> + if initrd and
> source_params.get('create-unified-kernel-image') != "true": #
> obviously we need to have a common common deploy var bootimg_dir =
> get_bitbake_var("DEPLOY_DIR_IMAGE") if not bootimg_dir:
> @@ -183,11 +185,12 @@ class BootimgEFIPlugin(SourcePlugin):
> for rd in initrds:
> boot_conf += "initrd /%s\n" % rd
>
> - logger.debug("Writing systemd-boot config "
> - "%s/hdd/boot/loader/entries/boot.conf",
> cr_workdir)
> - cfg = open("%s/hdd/boot/loader/entries/boot.conf" %
> cr_workdir, "w")
> - cfg.write(boot_conf)
> - cfg.close()
> + if source_params.get('create-unified-kernel-image') !=
> "true":
> + logger.debug("Writing systemd-boot config "
> + "%s/hdd/boot/loader/entries/boot.conf",
> cr_workdir)
> + cfg = open("%s/hdd/boot/loader/entries/boot.conf" %
> cr_workdir, "w")
> + cfg.write(boot_conf)
> + cfg.close()
>
>
> @classmethod
> @@ -288,9 +291,60 @@ class BootimgEFIPlugin(SourcePlugin):
> kernel = "%s-%s.bin" % \
> (get_bitbake_var("KERNEL_IMAGETYPE"),
> get_bitbake_var("INITRAMFS_LINK_NAME"))
> - install_cmd = "install -m 0644 %s/%s %s/%s" % \
> - (staging_kernel_dir, kernel, hdddir, kernel)
> - exec_cmd(install_cmd)
> + if source_params.get('create-unified-kernel-image') ==
> "true":
> + initrd = source_params.get('initrd')
> + if not initrd:
> + raise WicError("initrd= must be specified when
> create-unified-kernel-image=true, exiting") +
> + deploy_dir = get_bitbake_var("DEPLOY_DIR_IMAGE")
> + efi_stub = glob("%s/%s" % (deploy_dir,
> "linux*.efi.stub"))
> + if len(efi_stub) == 0:
> + raise WicError("Unified Kernel Image EFI stub not
> found, exiting")
> + efi_stub = efi_stub[0]
> +
> + with tempfile.TemporaryDirectory() as tmp_dir:
> + label = source_params.get('label')
> + label_conf = "root=%s" % creator.rootdev
> + if label:
> + label_conf = "LABEL=%s" % label
> +
> + bootloader = creator.ks.bootloader
> + cmdline = open("%s/cmdline" % tmp_dir, "w")
> + cmdline.write("%s %s" % (label_conf,
> bootloader.append))
> + cmdline.close()
> +
> + initrds = initrd.split(';')
> + initrd = open("%s/initrd" % tmp_dir, "wb")
> + for f in initrds:
> + with open("%s/%s" % (deploy_dir, f), 'rb') as
> in_file:
> + shutil.copyfileobj(in_file, initrd)
> + initrd.close()
> +
> + # Searched by systemd-boot:
> + #
> https://systemd.io/BOOT_LOADER_SPECIFICATION/#type-2-efi-unified-kernel-images
> + install_cmd = "install -d %s/EFI/Linux" % hdddir
> + exec_cmd(install_cmd)
> +
> + staging_dir_host =
> get_bitbake_var("STAGING_DIR_HOST") +
> + #
> https://www.freedesktop.org/software/systemd/man/systemd-stub.html
> + objcopy_cmd = "objcopy \
> + --add-section .osrel=%s --change-section-vma
> .osrel=0x20000 \
> + --add-section .cmdline=%s --change-section-vma
> .cmdline=0x30000 \
> + --add-section .linux=%s --change-section-vma
> .linux=0x2000000 \
> + --add-section .initrd=%s --change-section-vma
> .initrd=0x3000000 \
> + %s %s" % \
> + ("%s/usr/lib/os-release" % staging_dir_host,
> + cmdline.name,
> + "%s/%s" % (staging_kernel_dir, kernel),
> + initrd.name,
> + efi_stub,
> + "%s/EFI/Linux/linux.efi" % hdddir)
> + exec_cmd(objcopy_cmd)
> + else:
> + install_cmd = "install -m 0644 %s/%s %s/%s" % \
> + (staging_kernel_dir, kernel, hdddir, kernel)
> + exec_cmd(install_cmd)
>
> if get_bitbake_var("IMAGE_EFI_BOOT_FILES"):
> for src_path, dst_path in cls.install_task:
> diff --git a/scripts/lib/wic/plugins/source/bootimg-pcbios.py
> b/scripts/lib/wic/plugins/source/bootimg-pcbios.py index
> f2639e700493..32e47f183146 100644 ---
> a/scripts/lib/wic/plugins/source/bootimg-pcbios.py +++
> b/scripts/lib/wic/plugins/source/bootimg-pcbios.py @@ -186,8 +186,10
> @@ class BootimgPcbiosPlugin(SourcePlugin): # dosfs image, created by
> mkdosfs bootimg = "%s/boot%s.img" % (cr_workdir, part.lineno)
>
> - dosfs_cmd = "mkdosfs -n boot -i %s -S 512 -C %s %d" % \
> - (part.fsuuid, bootimg, blocks)
> + label = part.label if part.label else "boot"
> +
> + dosfs_cmd = "mkdosfs -n %s -i %s -S 512 -C %s %d" % \
> + (label, part.fsuuid, bootimg, blocks)
> exec_native_cmd(dosfs_cmd, native_sysroot)
>
> mcopy_cmd = "mcopy -i %s -s %s/* ::/" % (bootimg, hdddir)
> diff --git a/scripts/lib/wic/plugins/source/rawcopy.py
> b/scripts/lib/wic/plugins/source/rawcopy.py index
> 3c4997d8ba5e..7c90cd3cf82b 100644 ---
> a/scripts/lib/wic/plugins/source/rawcopy.py +++
> b/scripts/lib/wic/plugins/source/rawcopy.py @@ -4,6 +4,8 @@
>
> import logging
> import os
> +import signal
> +import subprocess
>
> from wic import WicError
> from wic.pluginbase import SourcePlugin
> @@ -29,15 +31,34 @@ class RawCopyPlugin(SourcePlugin):
> cmd = 'btrfs filesystem label %s %s' % (dst, label)
> elif fstype == 'swap':
> cmd = 'mkswap -L %s %s' % (label, dst)
> - elif fstype == 'squashfs':
> - raise WicError("It's not possible to update a squashfs "
> - "filesystem label '%s'" % (label))
> + elif fstype in ('squashfs', 'erofs'):
> + raise WicError("It's not possible to update a %s "
> + "filesystem label '%s'" % (fstype, label))
> else:
> raise WicError("Cannot update filesystem label: "
> "Unknown fstype: '%s'" % (fstype))
>
> exec_cmd(cmd)
>
> + @staticmethod
> + def do_image_uncompression(src, dst, workdir):
> + def subprocess_setup():
> + # Python installs a SIGPIPE handler by default. This is
> usually not what
> + # non-Python subprocesses expect.
> + # SIGPIPE errors are known issues with gzip/bash
> + signal.signal(signal.SIGPIPE, signal.SIG_DFL)
> +
> + extension = os.path.splitext(src)[1]
> + decompressor = {
> + ".bz2": "bzip2",
> + ".gz": "gzip",
> + ".xz": "xz"
> + }.get(extension)
> + if not decompressor:
> + raise WicError("Not supported compressor filename
> extension: %s" % extension)
> + cmd = "%s -dc %s > %s" % (decompressor, src, dst)
> + subprocess.call(cmd, preexec_fn=subprocess_setup,
> shell=True, cwd=workdir) +
> @classmethod
> def do_prepare_partition(cls, part, source_params, cr,
> cr_workdir, oe_builddir, bootimg_dir, kernel_dir,
> @@ -56,7 +77,13 @@ class RawCopyPlugin(SourcePlugin):
> if 'file' not in source_params:
> raise WicError("No file specified")
>
> - src = os.path.join(kernel_dir, source_params['file'])
> + if 'unpack' in source_params:
> + img = os.path.join(kernel_dir, source_params['file'])
> + src = os.path.join(cr_workdir,
> os.path.splitext(source_params['file'])[0])
> + RawCopyPlugin.do_image_uncompression(img, src,
> cr_workdir)
> + else:
> + src = os.path.join(kernel_dir, source_params['file'])
> +
> dst = os.path.join(cr_workdir, "%s.%s" %
> (os.path.basename(source_params['file']), part.lineno))
> if not os.path.exists(os.path.dirname(dst)):
> diff --git a/scripts/lib/wic/plugins/source/rootfs.py
> b/scripts/lib/wic/plugins/source/rootfs.py index
> 96d940a91d61..2e34e715ca73 100644 ---
> a/scripts/lib/wic/plugins/source/rootfs.py +++
> b/scripts/lib/wic/plugins/source/rootfs.py @@ -218,7 +218,7 @@ class
> RootfsPlugin(SourcePlugin): # Update part.has_fstab here as fstab may
> have been added or # removed by the above modifications.
> part.has_fstab = os.path.exists(os.path.join(new_rootfs,
> "etc/fstab"))
> - if part.update_fstab_in_rootfs and part.has_fstab:
> + if part.update_fstab_in_rootfs and part.has_fstab and
> not part.no_fstab_update: fstab_path = os.path.join(new_rootfs,
> "etc/fstab") # Assume that fstab should always be owned by root with
> fixed permissions install_cmd = "install -m 0644 %s %s" %
> (part.updated_fstab_path, fstab_path) diff --git a/scripts/wic
> b/scripts/wic index a741aed364bf..aee63a45aacf 100755
> --- a/scripts/wic
> +++ b/scripts/wic
> @@ -22,9 +22,9 @@ import sys
> import argparse
> import logging
> import subprocess
> +import shutil
>
> from collections import namedtuple
> -from distutils import spawn
>
> # External modules
> scripts_path = os.path.dirname(os.path.realpath(__file__))
> @@ -47,7 +47,7 @@ if os.environ.get('SDKTARGETSYSROOT'):
> break
> sdkroot = os.path.dirname(sdkroot)
>
> -bitbake_exe = spawn.find_executable('bitbake')
> +bitbake_exe = shutil.which('bitbake')
> if bitbake_exe:
> bitbake_path = scriptpath.add_bitbake_lib_path()
> import bb
> @@ -159,6 +159,9 @@ def wic_create_subcommand(options, usage_str):
> "(Use -e/--image-name to specify it)")
> native_sysroot = options.native_sysroot
>
> + if options.kernel_dir:
> + kernel_dir = options.kernel_dir
> +
> if not options.vars_dir and (not native_sysroot or not
> os.path.isdir(native_sysroot)): logger.info("Building wic-tools...\n")
> subprocess.check_call(["bitbake", "wic-tools"])
> @@ -346,6 +349,8 @@ def wic_init_parser_create(subparser):
> default=False, help="output debug information")
> subparser.add_argument("-i", "--imager", dest="imager",
> default="direct", help="the wic imager plugin")
> + subparser.add_argument("--extra-space", type=int,
> dest="extra_space",
> + default=0, help="additional free disk space to
> add to the image") return
>
>
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH v2 0/2] bump bitbake and wic for python 3.10 support
2022-04-21 8:52 [PATCH v2 0/2] bump bitbake and wic for python 3.10 support Henning Schild
2022-04-21 8:52 ` [PATCH v2 1/2] bitbake: Update to 1.50.5 release Henning Schild
2022-04-21 8:52 ` [PATCH v2 2/2] wic: Update to the latest revision Henning Schild
@ 2022-04-22 7:59 ` Anton Mikanovich
2022-04-22 8:15 ` Bezdeka, Florian
2022-04-22 8:27 ` Henning Schild
2 siblings, 2 replies; 22+ messages in thread
From: Anton Mikanovich @ 2022-04-22 7:59 UTC (permalink / raw)
To: Henning Schild, isar-users
Cc: Florian Bezdeka, Felix Moessbauer, Baurzhan Ismagulov
21.04.2022 11:52, Henning Schild wrote:
> debian bookworm switched to python 3.10 recently, our versions of
> bitbake and wic both have some issues with that new interpreter. All
> these issues have been fixes in bitbake and OE, so all we need to do is
> update the two tools in our tree.
>
> v1 was send as "[PATCH] bitbake: Update to 1.50.5 release", which is p1
> here, only later i found that wic also needs a bump to v2 added p2
>
> If this series is applied we do not need "[PATCH] wic-img: use
> python3.9 in bookworm", in case this one is applied before it should be
> reverted after the version bumps.
>
> Henning Schild (2):
> bitbake: Update to 1.50.5 release
> wic: Update to the latest revision
>
> bitbake/lib/bb/cache.py | 3 +-
> bitbake/lib/bb/cooker.py | 30 +++++++-
> bitbake/lib/bb/data_smart.py | 4 +-
> bitbake/lib/bb/fetch2/__init__.py | 4 +
> bitbake/lib/bb/fetch2/perforce.py | 2 +-
> bitbake/lib/bb/fetch2/wget.py | 2 +-
> bitbake/lib/bb/persist_data.py | 5 +-
> bitbake/lib/bb/process.py | 2 +-
> bitbake/lib/bb/runqueue.py | 34 ++++----
> bitbake/lib/bb/server/process.py | 2 +-
> bitbake/lib/bb/tests/fetch.py | 35 +++++----
> bitbake/lib/bb/utils.py | 13 +++-
> bitbake/lib/hashserv/server.py | 4 +-
> bitbake/lib/toaster/tests/builds/buildtest.py | 2 +-
> .../wic/plugins/source/bootimg-efi-isar.py | 77 ++++++++++++++++---
> .../wic/plugins/source/bootimg-pcbios-isar.py | 6 +-
> scripts/lib/wic/canned-wks/common.wks.inc | 2 +-
> scripts/lib/wic/canned-wks/directdisk-gpt.wks | 2 +-
> scripts/lib/wic/canned-wks/mkefidisk.wks | 2 +-
> scripts/lib/wic/engine.py | 6 +-
> scripts/lib/wic/help.py | 10 ++-
> scripts/lib/wic/ksparser.py | 8 +-
> scripts/lib/wic/misc.py | 4 +-
> scripts/lib/wic/partition.py | 25 ++++--
> scripts/lib/wic/pluginbase.py | 8 +-
> scripts/lib/wic/plugins/imager/direct.py | 11 ++-
> scripts/lib/wic/plugins/source/bootimg-efi.py | 74 +++++++++++++++---
> .../lib/wic/plugins/source/bootimg-pcbios.py | 6 +-
> scripts/lib/wic/plugins/source/rawcopy.py | 35 ++++++++-
> scripts/lib/wic/plugins/source/rootfs.py | 2 +-
> scripts/wic | 9 ++-
> 31 files changed, 320 insertions(+), 109 deletions(-)
Thanks for update, python issue is really fixed now, but it brakes
qemu386 targets booting in QEMU:
00:46:00 [ 34.424136] blk_update_request: I/O error, dev fd0, sector 0
00:46:00 [ 34.428049] floppy: error -5 while reading block 0
00:46:00 [ 34.508133] blk_update_request: I/O error, dev fd0, sector 0
00:46:00 [ 34.512050] floppy: error -5 while reading block 0
00:46:00 Gave up waiting for root file system device. Common problems:
00:46:00 - Boot args (cat /proc/cmdline)
00:46:00 - Check rootdelay= (did the system wait long enough?)
00:46:00 - Missing modules (cat /proc/modules; ls /dev)
00:46:00 ALERT! PARTLABEL=platform does not exist. Dropping to a shell!
00:46:00 (initramfs)
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH v2 0/2] bump bitbake and wic for python 3.10 support
2022-04-22 7:59 ` [PATCH v2 0/2] bump bitbake and wic for python 3.10 support Anton Mikanovich
@ 2022-04-22 8:15 ` Bezdeka, Florian
2022-04-22 8:26 ` Henning Schild
2022-04-22 8:27 ` Henning Schild
1 sibling, 1 reply; 22+ messages in thread
From: Bezdeka, Florian @ 2022-04-22 8:15 UTC (permalink / raw)
To: amikan, isar-users, Schild, Henning
Cc: Schmidt, Adriaan, ibr, Moessbauer, Felix
On Fri, 2022-04-22 at 10:59 +0300, Anton Mikanovich wrote:
> 21.04.2022 11:52, Henning Schild wrote:
> > debian bookworm switched to python 3.10 recently, our versions of
> > bitbake and wic both have some issues with that new interpreter. All
> > these issues have been fixes in bitbake and OE, so all we need to do is
> > update the two tools in our tree.
> >
> > v1 was send as "[PATCH] bitbake: Update to 1.50.5 release", which is p1
> > here, only later i found that wic also needs a bump to v2 added p2
> >
> > If this series is applied we do not need "[PATCH] wic-img: use
> > python3.9 in bookworm", in case this one is applied before it should be
> > reverted after the version bumps.
> >
> > Henning Schild (2):
> > bitbake: Update to 1.50.5 release
> > wic: Update to the latest revision
> >
> > bitbake/lib/bb/cache.py | 3 +-
> > bitbake/lib/bb/cooker.py | 30 +++++++-
> > bitbake/lib/bb/data_smart.py | 4 +-
> > bitbake/lib/bb/fetch2/__init__.py | 4 +
> > bitbake/lib/bb/fetch2/perforce.py | 2 +-
> > bitbake/lib/bb/fetch2/wget.py | 2 +-
> > bitbake/lib/bb/persist_data.py | 5 +-
> > bitbake/lib/bb/process.py | 2 +-
> > bitbake/lib/bb/runqueue.py | 34 ++++----
> > bitbake/lib/bb/server/process.py | 2 +-
> > bitbake/lib/bb/tests/fetch.py | 35 +++++----
> > bitbake/lib/bb/utils.py | 13 +++-
> > bitbake/lib/hashserv/server.py | 4 +-
> > bitbake/lib/toaster/tests/builds/buildtest.py | 2 +-
> > .../wic/plugins/source/bootimg-efi-isar.py | 77 ++++++++++++++++---
> > .../wic/plugins/source/bootimg-pcbios-isar.py | 6 +-
> > scripts/lib/wic/canned-wks/common.wks.inc | 2 +-
> > scripts/lib/wic/canned-wks/directdisk-gpt.wks | 2 +-
> > scripts/lib/wic/canned-wks/mkefidisk.wks | 2 +-
> > scripts/lib/wic/engine.py | 6 +-
> > scripts/lib/wic/help.py | 10 ++-
> > scripts/lib/wic/ksparser.py | 8 +-
> > scripts/lib/wic/misc.py | 4 +-
> > scripts/lib/wic/partition.py | 25 ++++--
> > scripts/lib/wic/pluginbase.py | 8 +-
> > scripts/lib/wic/plugins/imager/direct.py | 11 ++-
> > scripts/lib/wic/plugins/source/bootimg-efi.py | 74 +++++++++++++++---
> > .../lib/wic/plugins/source/bootimg-pcbios.py | 6 +-
> > scripts/lib/wic/plugins/source/rawcopy.py | 35 ++++++++-
> > scripts/lib/wic/plugins/source/rootfs.py | 2 +-
> > scripts/wic | 9 ++-
> > 31 files changed, 320 insertions(+), 109 deletions(-)
>
> Thanks for update, python issue is really fixed now, but it brakes
> qemu386 targets booting in QEMU:
>
> 00:46:00 [ 34.424136] blk_update_request: I/O error, dev fd0, sector 0
> 00:46:00 [ 34.428049] floppy: error -5 while reading block 0
> 00:46:00 [ 34.508133] blk_update_request: I/O error, dev fd0, sector 0
> 00:46:00 [ 34.512050] floppy: error -5 while reading block 0
> 00:46:00 Gave up waiting for root file system device. Common problems:
> 00:46:00 - Boot args (cat /proc/cmdline)
> 00:46:00 - Check rootdelay= (did the system wait long enough?)
> 00:46:00 - Missing modules (cat /proc/modules; ls /dev)
> 00:46:00 ALERT! PARTLABEL=platform does not exist. Dropping to a shell!
> 00:46:00 (initramfs)
>
Might be upstream commit [1] which Henning pulled in now.
Adding Adriaan as author of this commit to CC. I have no idea why this
should break exactly this image...
[1] https://github.com/openembedded/openembedded-core/commit/2fb247c5ecf057bb96649a3c0234794b4991c050
Regards,
Florian
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH v2 0/2] bump bitbake and wic for python 3.10 support
2022-04-22 8:15 ` Bezdeka, Florian
@ 2022-04-22 8:26 ` Henning Schild
2022-04-22 8:52 ` Henning Schild
0 siblings, 1 reply; 22+ messages in thread
From: Henning Schild @ 2022-04-22 8:26 UTC (permalink / raw)
To: Bezdeka, Florian (T CED SES-DE)
Cc: amikan, isar-users, Schmidt, Adriaan (T CED SES-DE),
ibr, Moessbauer, Felix (T CED SES-DE)
Am Fri, 22 Apr 2022 10:15:22 +0200
schrieb "Bezdeka, Florian (T CED SES-DE)" <florian.bezdeka@siemens.com>:
> On Fri, 2022-04-22 at 10:59 +0300, Anton Mikanovich wrote:
> > 21.04.2022 11:52, Henning Schild wrote:
> > > debian bookworm switched to python 3.10 recently, our versions of
> > > bitbake and wic both have some issues with that new interpreter.
> > > All these issues have been fixes in bitbake and OE, so all we
> > > need to do is update the two tools in our tree.
> > >
> > > v1 was send as "[PATCH] bitbake: Update to 1.50.5 release", which
> > > is p1 here, only later i found that wic also needs a bump to v2
> > > added p2
> > >
> > > If this series is applied we do not need "[PATCH] wic-img: use
> > > python3.9 in bookworm", in case this one is applied before it
> > > should be reverted after the version bumps.
> > >
> > > Henning Schild (2):
> > > bitbake: Update to 1.50.5 release
> > > wic: Update to the latest revision
> > >
> > > bitbake/lib/bb/cache.py | 3 +-
> > > bitbake/lib/bb/cooker.py | 30 +++++++-
> > > bitbake/lib/bb/data_smart.py | 4 +-
> > > bitbake/lib/bb/fetch2/__init__.py | 4 +
> > > bitbake/lib/bb/fetch2/perforce.py | 2 +-
> > > bitbake/lib/bb/fetch2/wget.py | 2 +-
> > > bitbake/lib/bb/persist_data.py | 5 +-
> > > bitbake/lib/bb/process.py | 2 +-
> > > bitbake/lib/bb/runqueue.py | 34 ++++----
> > > bitbake/lib/bb/server/process.py | 2 +-
> > > bitbake/lib/bb/tests/fetch.py | 35 +++++----
> > > bitbake/lib/bb/utils.py | 13 +++-
> > > bitbake/lib/hashserv/server.py | 4 +-
> > > bitbake/lib/toaster/tests/builds/buildtest.py | 2 +-
> > > .../wic/plugins/source/bootimg-efi-isar.py | 77
> > > ++++++++++++++++--- .../wic/plugins/source/bootimg-pcbios-isar.py
> > > | 6 +- scripts/lib/wic/canned-wks/common.wks.inc | 2 +-
> > > scripts/lib/wic/canned-wks/directdisk-gpt.wks | 2 +-
> > > scripts/lib/wic/canned-wks/mkefidisk.wks | 2 +-
> > > scripts/lib/wic/engine.py | 6 +-
> > > scripts/lib/wic/help.py | 10 ++-
> > > scripts/lib/wic/ksparser.py | 8 +-
> > > scripts/lib/wic/misc.py | 4 +-
> > > scripts/lib/wic/partition.py | 25 ++++--
> > > scripts/lib/wic/pluginbase.py | 8 +-
> > > scripts/lib/wic/plugins/imager/direct.py | 11 ++-
> > > scripts/lib/wic/plugins/source/bootimg-efi.py | 74
> > > +++++++++++++++--- .../lib/wic/plugins/source/bootimg-pcbios.py
> > > | 6 +- scripts/lib/wic/plugins/source/rawcopy.py | 35
> > > ++++++++- scripts/lib/wic/plugins/source/rootfs.py | 2 +-
> > > scripts/wic | 9 ++-
> > > 31 files changed, 320 insertions(+), 109 deletions(-)
> >
> > Thanks for update, python issue is really fixed now, but it brakes
> > qemu386 targets booting in QEMU:
Thanks, i will look into this.
> > 00:46:00 [ 34.424136] blk_update_request: I/O error, dev fd0,
> > sector 0 00:46:00 [ 34.428049] floppy: error -5 while reading
> > block 0 00:46:00 [ 34.508133] blk_update_request: I/O error, dev
> > fd0, sector 0 00:46:00 [ 34.512050] floppy: error -5 while
> > reading block 0 00:46:00 Gave up waiting for root file system
> > device. Common problems: 00:46:00 - Boot args (cat /proc/cmdline)
> > 00:46:00 - Check rootdelay= (did the system wait long enough?)
> > 00:46:00 - Missing modules (cat /proc/modules; ls /dev)
> > 00:46:00 ALERT! PARTLABEL=platform does not exist. Dropping to a
> > shell! 00:46:00 (initramfs)
> >
>
> Might be upstream commit [1] which Henning pulled in now.
>
> Adding Adriaan as author of this commit to CC. I have no idea why this
> should break exactly this image...
We boot all our i386 with BIOS directdisk-isar.wks and all amd64 with
EFI sdimage-efi.wks, plus a few exceptions from that pattern where we
use efi-plus-pcbios.wks or systemd-boot/EFI with sdimage-efi-sd.wks
So maybe that directdisk-isar.wks or common-isar.wks.inc need some bits
i did not see and good old legacy BIOS is indeed broken in general with
these patches.
Henning
> [1]
> https://github.com/openembedded/openembedded-core/commit/2fb247c5ecf057bb96649a3c0234794b4991c050
>
> Regards,
> Florian
>
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH v2 0/2] bump bitbake and wic for python 3.10 support
2022-04-22 7:59 ` [PATCH v2 0/2] bump bitbake and wic for python 3.10 support Anton Mikanovich
2022-04-22 8:15 ` Bezdeka, Florian
@ 2022-04-22 8:27 ` Henning Schild
2022-04-22 8:57 ` Anton Mikanovich
1 sibling, 1 reply; 22+ messages in thread
From: Henning Schild @ 2022-04-22 8:27 UTC (permalink / raw)
To: Anton Mikanovich
Cc: isar-users, Florian Bezdeka, Felix Moessbauer, Baurzhan Ismagulov
Am Fri, 22 Apr 2022 10:59:15 +0300
schrieb Anton Mikanovich <amikan@ilbers.de>:
> 21.04.2022 11:52, Henning Schild wrote:
> > debian bookworm switched to python 3.10 recently, our versions of
> > bitbake and wic both have some issues with that new interpreter. All
> > these issues have been fixes in bitbake and OE, so all we need to
> > do is update the two tools in our tree.
> >
> > v1 was send as "[PATCH] bitbake: Update to 1.50.5 release", which
> > is p1 here, only later i found that wic also needs a bump to v2
> > added p2
> >
> > If this series is applied we do not need "[PATCH] wic-img: use
> > python3.9 in bookworm", in case this one is applied before it
> > should be reverted after the version bumps.
> >
> > Henning Schild (2):
> > bitbake: Update to 1.50.5 release
> > wic: Update to the latest revision
> >
> > bitbake/lib/bb/cache.py | 3 +-
> > bitbake/lib/bb/cooker.py | 30 +++++++-
> > bitbake/lib/bb/data_smart.py | 4 +-
> > bitbake/lib/bb/fetch2/__init__.py | 4 +
> > bitbake/lib/bb/fetch2/perforce.py | 2 +-
> > bitbake/lib/bb/fetch2/wget.py | 2 +-
> > bitbake/lib/bb/persist_data.py | 5 +-
> > bitbake/lib/bb/process.py | 2 +-
> > bitbake/lib/bb/runqueue.py | 34 ++++----
> > bitbake/lib/bb/server/process.py | 2 +-
> > bitbake/lib/bb/tests/fetch.py | 35 +++++----
> > bitbake/lib/bb/utils.py | 13 +++-
> > bitbake/lib/hashserv/server.py | 4 +-
> > bitbake/lib/toaster/tests/builds/buildtest.py | 2 +-
> > .../wic/plugins/source/bootimg-efi-isar.py | 77
> > ++++++++++++++++--- .../wic/plugins/source/bootimg-pcbios-isar.py |
> > 6 +- scripts/lib/wic/canned-wks/common.wks.inc | 2 +-
> > scripts/lib/wic/canned-wks/directdisk-gpt.wks | 2 +-
> > scripts/lib/wic/canned-wks/mkefidisk.wks | 2 +-
> > scripts/lib/wic/engine.py | 6 +-
> > scripts/lib/wic/help.py | 10 ++-
> > scripts/lib/wic/ksparser.py | 8 +-
> > scripts/lib/wic/misc.py | 4 +-
> > scripts/lib/wic/partition.py | 25 ++++--
> > scripts/lib/wic/pluginbase.py | 8 +-
> > scripts/lib/wic/plugins/imager/direct.py | 11 ++-
> > scripts/lib/wic/plugins/source/bootimg-efi.py | 74
> > +++++++++++++++--- .../lib/wic/plugins/source/bootimg-pcbios.py |
> > 6 +- scripts/lib/wic/plugins/source/rawcopy.py | 35 ++++++++-
> > scripts/lib/wic/plugins/source/rootfs.py | 2 +-
> > scripts/wic | 9 ++-
> > 31 files changed, 320 insertions(+), 109 deletions(-)
>
> Thanks for update, python issue is really fixed now, but it brakes
> qemu386 targets booting in QEMU:
All of them or only some suits? I am building qemui386-bullseye now for
local repro.
Henning
> 00:46:00 [ 34.424136] blk_update_request: I/O error, dev fd0,
> sector 0 00:46:00 [ 34.428049] floppy: error -5 while reading block
> 0 00:46:00 [ 34.508133] blk_update_request: I/O error, dev fd0,
> sector 0 00:46:00 [ 34.512050] floppy: error -5 while reading block
> 0 00:46:00 Gave up waiting for root file system device. Common
> problems: 00:46:00 - Boot args (cat /proc/cmdline)
> 00:46:00 - Check rootdelay= (did the system wait long enough?)
> 00:46:00 - Missing modules (cat /proc/modules; ls /dev)
> 00:46:00 ALERT! PARTLABEL=platform does not exist. Dropping to a
> shell! 00:46:00 (initramfs)
>
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH v2 0/2] bump bitbake and wic for python 3.10 support
2022-04-22 8:26 ` Henning Schild
@ 2022-04-22 8:52 ` Henning Schild
2022-04-22 9:54 ` Henning Schild
0 siblings, 1 reply; 22+ messages in thread
From: Henning Schild @ 2022-04-22 8:52 UTC (permalink / raw)
To: Bezdeka, Florian (T CED SES-DE)
Cc: amikan, isar-users, Schmidt, Adriaan (T CED SES-DE),
ibr, Moessbauer, Felix (T CED SES-DE)
Am Fri, 22 Apr 2022 10:26:08 +0200
schrieb Henning Schild <henning.schild@siemens.com>:
> Am Fri, 22 Apr 2022 10:15:22 +0200
> schrieb "Bezdeka, Florian (T CED SES-DE)"
> <florian.bezdeka@siemens.com>:
>
> > On Fri, 2022-04-22 at 10:59 +0300, Anton Mikanovich wrote:
> > > 21.04.2022 11:52, Henning Schild wrote:
> > > > debian bookworm switched to python 3.10 recently, our versions
> > > > of bitbake and wic both have some issues with that new
> > > > interpreter. All these issues have been fixes in bitbake and
> > > > OE, so all we need to do is update the two tools in our tree.
> > > >
> > > > v1 was send as "[PATCH] bitbake: Update to 1.50.5 release",
> > > > which is p1 here, only later i found that wic also needs a bump
> > > > to v2 added p2
> > > >
> > > > If this series is applied we do not need "[PATCH] wic-img: use
> > > > python3.9 in bookworm", in case this one is applied before it
> > > > should be reverted after the version bumps.
> > > >
> > > > Henning Schild (2):
> > > > bitbake: Update to 1.50.5 release
> > > > wic: Update to the latest revision
> > > >
> > > > bitbake/lib/bb/cache.py | 3 +-
> > > > bitbake/lib/bb/cooker.py | 30 +++++++-
> > > > bitbake/lib/bb/data_smart.py | 4 +-
> > > > bitbake/lib/bb/fetch2/__init__.py | 4 +
> > > > bitbake/lib/bb/fetch2/perforce.py | 2 +-
> > > > bitbake/lib/bb/fetch2/wget.py | 2 +-
> > > > bitbake/lib/bb/persist_data.py | 5 +-
> > > > bitbake/lib/bb/process.py | 2 +-
> > > > bitbake/lib/bb/runqueue.py | 34 ++++----
> > > > bitbake/lib/bb/server/process.py | 2 +-
> > > > bitbake/lib/bb/tests/fetch.py | 35 +++++----
> > > > bitbake/lib/bb/utils.py | 13 +++-
> > > > bitbake/lib/hashserv/server.py | 4 +-
> > > > bitbake/lib/toaster/tests/builds/buildtest.py | 2 +-
> > > > .../wic/plugins/source/bootimg-efi-isar.py | 77
> > > > ++++++++++++++++---
> > > > .../wic/plugins/source/bootimg-pcbios-isar.py | 6 +-
> > > > scripts/lib/wic/canned-wks/common.wks.inc | 2 +-
> > > > scripts/lib/wic/canned-wks/directdisk-gpt.wks | 2 +-
> > > > scripts/lib/wic/canned-wks/mkefidisk.wks | 2 +-
> > > > scripts/lib/wic/engine.py | 6 +-
> > > > scripts/lib/wic/help.py | 10 ++-
> > > > scripts/lib/wic/ksparser.py | 8 +-
> > > > scripts/lib/wic/misc.py | 4 +-
> > > > scripts/lib/wic/partition.py | 25 ++++--
> > > > scripts/lib/wic/pluginbase.py | 8 +-
> > > > scripts/lib/wic/plugins/imager/direct.py | 11 ++-
> > > > scripts/lib/wic/plugins/source/bootimg-efi.py | 74
> > > > +++++++++++++++--- .../lib/wic/plugins/source/bootimg-pcbios.py
> > > > | 6 +- scripts/lib/wic/plugins/source/rawcopy.py | 35
> > > > ++++++++- scripts/lib/wic/plugins/source/rootfs.py | 2 +-
> > > > scripts/wic | 9 ++- 31 files
> > > > changed, 320 insertions(+), 109 deletions(-)
> > >
> > > Thanks for update, python issue is really fixed now, but it
> > > brakes qemu386 targets booting in QEMU:
>
> Thanks, i will look into this.
>
> > > 00:46:00 [ 34.424136] blk_update_request: I/O error, dev fd0,
> > > sector 0 00:46:00 [ 34.428049] floppy: error -5 while reading
> > > block 0 00:46:00 [ 34.508133] blk_update_request: I/O error, dev
> > > fd0, sector 0 00:46:00 [ 34.512050] floppy: error -5 while
> > > reading block 0 00:46:00 Gave up waiting for root file system
> > > device. Common problems: 00:46:00 - Boot args (cat
> > > /proc/cmdline) 00:46:00 - Check rootdelay= (did the system
> > > wait long enough?) 00:46:00 - Missing modules (cat
> > > /proc/modules; ls /dev) 00:46:00 ALERT! PARTLABEL=platform does
> > > not exist. Dropping to a shell! 00:46:00 (initramfs)
> > >
> >
> > Might be upstream commit [1] which Henning pulled in now.
> >
> > Adding Adriaan as author of this commit to CC. I have no idea why
> > this should break exactly this image...
>
> We boot all our i386 with BIOS directdisk-isar.wks and all amd64 with
> EFI sdimage-efi.wks, plus a few exceptions from that pattern where we
> use efi-plus-pcbios.wks or systemd-boot/EFI with sdimage-efi-sd.wks
>
> So maybe that directdisk-isar.wks or common-isar.wks.inc need some
> bits i did not see and good old legacy BIOS is indeed broken in
> general with these patches.
PARTLABEL is a gpt thing, and we use msdos. And our common-isar.wks.inc
does not have --use-uuid while upstream common.wks.inc has it.
Will send a patch. Could not find if we ever dropped that use-uuid, i
think it is more likely it was overlooked while maintaining the fork.
Henning
> Henning
>
> > [1]
> > https://github.com/openembedded/openembedded-core/commit/2fb247c5ecf057bb96649a3c0234794b4991c050
> >
> > Regards,
> > Florian
> >
>
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH v2 0/2] bump bitbake and wic for python 3.10 support
2022-04-22 8:27 ` Henning Schild
@ 2022-04-22 8:57 ` Anton Mikanovich
0 siblings, 0 replies; 22+ messages in thread
From: Anton Mikanovich @ 2022-04-22 8:57 UTC (permalink / raw)
To: Henning Schild
Cc: isar-users, Florian Bezdeka, Felix Moessbauer, Baurzhan Ismagulov
22.04.2022 11:27, Henning Schild wrote:
> Am Fri, 22 Apr 2022 10:59:15 +0300
> schrieb Anton Mikanovich <amikan@ilbers.de>:
>
>> Thanks for update, python issue is really fixed now, but it brakes
>> qemu386 targets booting in QEMU:
> All of them or only some suits? I am building qemui386-bullseye now for
> local repro.
>
> Henning
>
I've tried buster and stretch (we have them covered by CI start_vm) -
both fails.
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH v2 0/2] bump bitbake and wic for python 3.10 support
2022-04-22 8:52 ` Henning Schild
@ 2022-04-22 9:54 ` Henning Schild
0 siblings, 0 replies; 22+ messages in thread
From: Henning Schild @ 2022-04-22 9:54 UTC (permalink / raw)
To: Bezdeka, Florian (T CED SES-DE)
Cc: amikan, isar-users, Schmidt, Adriaan (T CED SES-DE),
ibr, Moessbauer, Felix (T CED SES-DE)
Am Fri, 22 Apr 2022 10:52:11 +0200
schrieb Henning Schild <henning.schild@siemens.com>:
> Am Fri, 22 Apr 2022 10:26:08 +0200
> schrieb Henning Schild <henning.schild@siemens.com>:
>
> > Am Fri, 22 Apr 2022 10:15:22 +0200
> > schrieb "Bezdeka, Florian (T CED SES-DE)"
> > <florian.bezdeka@siemens.com>:
> >
> > > On Fri, 2022-04-22 at 10:59 +0300, Anton Mikanovich wrote:
> > > > 21.04.2022 11:52, Henning Schild wrote:
> > > > > debian bookworm switched to python 3.10 recently, our versions
> > > > > of bitbake and wic both have some issues with that new
> > > > > interpreter. All these issues have been fixes in bitbake and
> > > > > OE, so all we need to do is update the two tools in our tree.
> > > > >
> > > > > v1 was send as "[PATCH] bitbake: Update to 1.50.5 release",
> > > > > which is p1 here, only later i found that wic also needs a
> > > > > bump to v2 added p2
> > > > >
> > > > > If this series is applied we do not need "[PATCH] wic-img: use
> > > > > python3.9 in bookworm", in case this one is applied before it
> > > > > should be reverted after the version bumps.
> > > > >
> > > > > Henning Schild (2):
> > > > > bitbake: Update to 1.50.5 release
> > > > > wic: Update to the latest revision
> > > > >
> > > > > bitbake/lib/bb/cache.py | 3 +-
> > > > > bitbake/lib/bb/cooker.py | 30 +++++++-
> > > > > bitbake/lib/bb/data_smart.py | 4 +-
> > > > > bitbake/lib/bb/fetch2/__init__.py | 4 +
> > > > > bitbake/lib/bb/fetch2/perforce.py | 2 +-
> > > > > bitbake/lib/bb/fetch2/wget.py | 2 +-
> > > > > bitbake/lib/bb/persist_data.py | 5 +-
> > > > > bitbake/lib/bb/process.py | 2 +-
> > > > > bitbake/lib/bb/runqueue.py | 34 ++++----
> > > > > bitbake/lib/bb/server/process.py | 2 +-
> > > > > bitbake/lib/bb/tests/fetch.py | 35 +++++----
> > > > > bitbake/lib/bb/utils.py | 13 +++-
> > > > > bitbake/lib/hashserv/server.py | 4 +-
> > > > > bitbake/lib/toaster/tests/builds/buildtest.py | 2 +-
> > > > > .../wic/plugins/source/bootimg-efi-isar.py | 77
> > > > > ++++++++++++++++---
> > > > > .../wic/plugins/source/bootimg-pcbios-isar.py | 6 +-
> > > > > scripts/lib/wic/canned-wks/common.wks.inc | 2 +-
> > > > > scripts/lib/wic/canned-wks/directdisk-gpt.wks | 2 +-
> > > > > scripts/lib/wic/canned-wks/mkefidisk.wks | 2 +-
> > > > > scripts/lib/wic/engine.py | 6 +-
> > > > > scripts/lib/wic/help.py | 10 ++-
> > > > > scripts/lib/wic/ksparser.py | 8 +-
> > > > > scripts/lib/wic/misc.py | 4 +-
> > > > > scripts/lib/wic/partition.py | 25 ++++--
> > > > > scripts/lib/wic/pluginbase.py | 8 +-
> > > > > scripts/lib/wic/plugins/imager/direct.py | 11 ++-
> > > > > scripts/lib/wic/plugins/source/bootimg-efi.py | 74
> > > > > +++++++++++++++---
> > > > > .../lib/wic/plugins/source/bootimg-pcbios.py | 6 +-
> > > > > scripts/lib/wic/plugins/source/rawcopy.py | 35 ++++++++-
> > > > > scripts/lib/wic/plugins/source/rootfs.py | 2 +-
> > > > > scripts/wic | 9 ++- 31
> > > > > files changed, 320 insertions(+), 109 deletions(-)
> > > >
> > > > Thanks for update, python issue is really fixed now, but it
> > > > brakes qemu386 targets booting in QEMU:
> >
> > Thanks, i will look into this.
> >
> > > > 00:46:00 [ 34.424136] blk_update_request: I/O error, dev fd0,
> > > > sector 0 00:46:00 [ 34.428049] floppy: error -5 while reading
> > > > block 0 00:46:00 [ 34.508133] blk_update_request: I/O error,
> > > > dev fd0, sector 0 00:46:00 [ 34.512050] floppy: error -5 while
> > > > reading block 0 00:46:00 Gave up waiting for root file system
> > > > device. Common problems: 00:46:00 - Boot args (cat
> > > > /proc/cmdline) 00:46:00 - Check rootdelay= (did the system
> > > > wait long enough?) 00:46:00 - Missing modules (cat
> > > > /proc/modules; ls /dev) 00:46:00 ALERT! PARTLABEL=platform does
> > > > not exist. Dropping to a shell! 00:46:00 (initramfs)
> > > >
> > >
> > > Might be upstream commit [1] which Henning pulled in now.
> > >
> > > Adding Adriaan as author of this commit to CC. I have no idea why
> > > this should break exactly this image...
> >
> > We boot all our i386 with BIOS directdisk-isar.wks and all amd64
> > with EFI sdimage-efi.wks, plus a few exceptions from that pattern
> > where we use efi-plus-pcbios.wks or systemd-boot/EFI with
> > sdimage-efi-sd.wks
> >
> > So maybe that directdisk-isar.wks or common-isar.wks.inc need some
> > bits i did not see and good old legacy BIOS is indeed broken in
> > general with these patches.
>
> PARTLABEL is a gpt thing, and we use msdos. And our
> common-isar.wks.inc does not have --use-uuid while upstream
> common.wks.inc has it.
>
> Will send a patch. Could not find if we ever dropped that use-uuid, i
> think it is more likely it was overlooked while maintaining the fork.
v3 solves this with a patch to align our common wks with OE,
additionally i started an upstream discussion with a patch to not allow
anyone to PARTLABEL on msdos
https://lists.openembedded.org/g/openembedded-core/topic/patch_wic_do_not_use/90624635?p=,,,20,0,0,0::recentpostdate/sticky,,,20,2,0,90624635,previd%3D1650620877809916310,nextid%3D1650546439549613402&previd=1650620877809916310&nextid=1650546439549613402
Henning
> Henning
>
> > Henning
> >
> > > [1]
> > > https://github.com/openembedded/openembedded-core/commit/2fb247c5ecf057bb96649a3c0234794b4991c050
> > >
> > > Regards,
> > > Florian
> > >
> >
>
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH v2 2/2] wic: Update to the latest revision
2022-09-26 7:56 ` Kanagarajan, Vijaikumar
2022-09-26 8:02 ` Moessbauer, Felix
@ 2022-09-26 12:34 ` Henning Schild
1 sibling, 0 replies; 22+ messages in thread
From: Henning Schild @ 2022-09-26 12:34 UTC (permalink / raw)
To: Kanagarajan, Vijaikumar; +Cc: felix.moessbauer, jan.kiszka, isar-users
Am Mon, 26 Sep 2022 07:56:48 +0000
schrieb "Kanagarajan, Vijaikumar" <Vijaikumar_Kanagarajan@mentor.com>:
> > -----Original Message-----
> > From: Henning Schild <henning.schild@siemens.com>
> > Sent: 26 September 2022 13:17
> > To: felix.moessbauer@siemens.com; Kanagarajan, Vijaikumar
> > <Vijaikumar_Kanagarajan@mentor.com>; jan.kiszka@siemens.com
> > Cc: isar-users@googlegroups.com
> > Subject: Re: [PATCH v2 2/2] wic: Update to the latest revision
> >
> > Am Mon, 26 Sep 2022 07:57:41 +0200
> > schrieb "Moessbauer, Felix (T CED SES-DE)"
> > <felix.moessbauer@siemens.com>:
> >
> > > > -----Original Message-----
> > > > From: Kanagarajan, Vijaikumar
> > > > <Vijaikumar_Kanagarajan@mentor.com> Sent: Monday, September 26,
> > > > 2022 12:53 PM To: Moessbauer, Felix (T CED SES-DE)
> > <felix.moessbauer@siemens.com>;
> > > > Schild, Henning (T CED SES-DE) <henning.schild@siemens.com> Cc:
> > > > Kiszka, Jan (T CED) <jan.kiszka@siemens.com>; isar-
> > > > users@googlegroups.com
> > > > Subject: RE: [PATCH v2 2/2] wic: Update to the latest revision
> > > >
> > > >
> > > >
> > > > > -----Original Message-----
> > > > > From: Moessbauer, Felix (T CED SES-DE)
> > > > > <felix.moessbauer@siemens.com> Sent: 26 September 2022 07:11
> > > > > To: Kanagarajan, Vijaikumar
> > > > > <Vijaikumar_Kanagarajan@mentor.com>; Schild, Henning (T CED
> > > > > SES-DE) <henning.schild@siemens.com> Cc: Kiszka, Jan (T CED)
> > > > > <jan.kiszka@siemens.com>; isar- users@googlegroups.com
> > > > > Subject: RE: [PATCH v2 2/2] wic: Update to the latest revision
> > > > >
> > > > > Hi Vijai & Henning,
> > > > >
> > > > > Are there any news on this?
> > > > > I just discovered that still many (if not all) of our images
> > > > > are affected by this bug.
> > > >
> > > > Hi Felix,
> > > >
> > > > Unfortunately not. If we already have the latest wic,
> > > > --no-fstab-update[1] might help.(untested).
> > >
> > > We already have a wic version with this option, but it doesn't
> > > look right to add it to the root partition. Instead this option
> > > can be used for partitions that should not show up in the fstab
> > > at all.
> >
> > We do want the one from WIC. It is only coming since we patched WIC
> > to support /.
> >
> > > Indeed, the fstab entry has to be set by WIC to set settings like
> > > UUID and filesystem. But this requires that we either remove the
> > > entry set by ISAR in image_configure_fstab, or we remove the
> > > entry after generating the rootfs, but before WIC.
> >
> > I think we should try and remove the entry from that template. It
> > should not be required.
> >
> > If we decide to solve the problem somewhere in/with WIC, we should
> > do that upstream in OE. They have the same "we now have two"
> > problem, where wic adds on top of a template instead of replacing
> > the original.
>
> This seems to me like the approach we should take. Make wic replace
> entries in fstab instead of appending to them like how it does now.
That is the more complicated way. I do not recall all the details but i
think we should rather drop the / line from the template. Such a line
is usually not needed, having it is rather limiting the use of the
rootfs because you might have a device name standing there which might
be wrong ... i.e. sda1 vs vda1 vs nfs
I think we could contribute both to OE. The removal of the / line from
their template, and the "wic will try and update not append".
But we could also simply drop the template line in isar ... not care
about OE and making our non wic images better in fact. There might be
weird cornercases, maybe a too old raspi image ... where that line is
needed. I would move it to those corner-case images if there are any.
Henning
> Thanks,
> Vijai Kumar K
>
> >
> > Henning
> >
> > > Opinions? @Kiszka, Jan (T CED)
> > >
> > > Felix
> > >
> > > >
> > > > Thanks,
> > > > Vijai Kumar K
> > > >
> > > > >
> > > > > While it is not serios, having error messages in the journal
> > > > > is still pretty bad: systemd-fstab-generator[1491]: Failed to
> > > > > create unit file /run/systemd/generator/-.mount, as it
> > > > > already exists. Duplicate entry in /etc/fstab?
> > > > >
> > > > > Felix
> > > > >
> > > > > > -----Original Message-----
> > > > > > From: isar-users@googlegroups.com <isar-
> > users@googlegroups.com>
> > > > > > On Behalf Of Kanagarajan, Vijaikumar
> > > > > > Sent: Friday, June 17, 2022 1:46 PM
> > > > > > To: isar-users@googlegroups.com
> > > > > > Cc: Kiszka, Jan (T CED) <jan.kiszka@siemens.com>; Schild,
> > > > > > Henning (T CED SES-
> > > > > > DE) <henning.schild@siemens.com>
> > > > > > Subject: RE: [PATCH v2 2/2] wic: Update to the latest
> > > > > > revision
> > > > > >
> > > > > >
> > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: Kanagarajan, Vijaikumar
> > > > > > > Sent: 13 June 2022 14:48
> > > > > > > To: 'Henning Schild' <henning.schild@siemens.com>
> > > > > > > Cc: isar-users@googlegroups.com; jan.kiszka@siemens.com
> > > > > > > Subject: RE: [PATCH v2 2/2] wic: Update to the latest
> > > > > > > revision
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > > -----Original Message-----
> > > > > > > > From: Henning Schild <henning.schild@siemens.com>
> > > > > > > > Sent: 13 June 2022 14:32
> > > > > > > > To: Kanagarajan, Vijaikumar
> > > > > > > > <Vijaikumar_Kanagarajan@mentor.com> Cc:
> > > > > > > > isar-users@googlegroups.com; jan.kiszka@siemens.com
> > > > > > > > Subject: Re: [PATCH v2 2/2] wic: Update to the latest
> > > > > > > > revision
> > > > > > > >
> > > > > > > > Am Mon, 13 Jun 2022 13:24:01 +0530 schrieb Vijai Kumar K
> > > > > > > > <Vijaikumar_Kanagarajan@mentor.com>:
> > > > > > > >
> > > > > > > > > Update to the latest wic from OE-core.
> > > > > > > > >
> > > > > > > > > OE-core Revision:
> > 06ed2aa93fc25a681e3a00ee120d9395b04845da
> > > > > > > > >
> > > > > > > > > Signed-off-by: Vijai Kumar K
> > > > > <Vijaikumar_Kanagarajan@mentor.com>
> > > > > > > > > ---
> > > > > > > > > RECIPE-API-CHANGELOG.md | 19
> > > > > > > > > +++++++++++++++++++ scripts/lib/wic/help.py
> > > > > > > > > | 6 ++++++ scripts/lib/wic/ksparser.py
> > > > > > > > > | 1 + scripts/lib/wic/partition.py | 1
> > > > > > > > > + scripts/lib/wic/plugins/imager/direct.py | 5 +++--
> > > > > > > > > 5 files changed, 30 insertions(+), 2 deletions(-)
> > > > > > > > >
> > > > > > > > > diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-
> > > > > CHANGELOG.md
> > > > > > > > index
> > > > > > > > > 2998824..436b204 100644
> > > > > > > > > --- a/RECIPE-API-CHANGELOG.md
> > > > > > > > > +++ b/RECIPE-API-CHANGELOG.md
> > > > > > > > > @@ -406,3 +406,22 @@ groups are now created after all
> > > > > > > > > packages are installed.
> > > > > > > > > Changes in next
> > > > > > > > > ---------------
> > > > > > > > > +
> > > > > > > > > +### Wic adds / mountpoint to fstab
> > > > > > > > > +
> > > > > > > > > +In the older version of wic, any mount point named /
> > > > > > > > > is +skipped from
> > > > > > > > > adding +into the fstab entry.
> > > > > > > > > +
> > > > > > > > > +With the latest wic, this is not the case. / mount
> > > > > > > > > point, if +any, is
> > > > > > > > > added +to /etc/fstab for automount.
> > > > > > > > > +
> > > > > > > > > +Any wks file which assumed that / would be skipped
> > > > > > > > > from +/etc/fstab
> > > > > > > > > should +now be corrected. Otherwise, it might result
> > > > > > > > > in multiple / entries in /etc/fstab. +One by the
> > > > > > > > > image_configure_fstab function from image class and
> > > > > > > > > one by wic. +
> > > > > > > > > +Below is an example wks entry that might cause an
> > > > > > > > > issue. +Replace "part /" with "part" to avoid adding
> > > > > > > > > duplicate entry for /.
> > > > > > > >
> > > > > > > > I am not sure multiple entries would be an issue, i
> > > > > > > > think the last one will simply always win. It is good
> > > > > > > > to have a note about that change, but i doubt people
> > > > > > > > really have to do anything. And if anything has to be
> > > > > > > > done, isar should drop the first one centrally.
> > > > > > > >
> > > > > > > > The fstab entry for root is purely for adding some weird
> > > > > > > > mount options, usually root is not mounted with fstab
> > > > > > > > but with cmdline and initrd. And not having / in fstab
> > > > > > > > just makes that rootfs more generic ... it can become
> > > > > > > > part of nfs and next guy takes it as btrfs while next
> > > > > > > > guy
> > > > > > > takes it as ext4.
> > > > > > > >
> > > > > > > > Note is good, advise is not
> > > > > > >
> > > > > > > Noted. Will change the text accordingly in V3.
> > > > > >
> > > > > > Ok we cannot drop / mountpoint name(though optional). We
> > > > > > would be
> > > > > lose
> > > > > > the ability to find out UUID of root using rootdev api.
> > > > > > This is used by some plugins. So the better way would be to
> > > > > > use the --no-fstab- update. Seems like we can set them per
> > > > > > partition now.
> > > > > >
> > > > > > Thanks,
> > > > > > Vijai Kumar K
> > > > > >
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Vijai Kumar K
> > > > > > >
> > > > > > > >
> > > > > > > > Henning
> > > > > > > >
> > > > > > > > > +
> > > > > > > > > +```
> > > > > > > > > +part / --source rootfs --ondisk sda --fstype ext4
> > > > > > > > > --label +platform
> > > > > > > > > --align 1024 --use-uuid +``` diff --git
> > > > > > > > > a/scripts/lib/wic/help.py b/scripts/lib/wic/help.py
> > > > > > > > > index 4ff7470..73e3380 100644
> > > > > > > > > --- a/scripts/lib/wic/help.py
> > > > > > > > > +++ b/scripts/lib/wic/help.py
> > > > > > > > > @@ -940,6 +940,12 @@ DESCRIPTION
> > > > > > > > > quotes. If not specified, the
> > > > > > > > > default string is "defaults".
> > > > > > > > >
> > > > > > > > > + --fspassno: Specifies the order in which
> > > > > > > > > filesystem
> > > > > > > > > + checks
> > > > > > > > > are done
> > > > > > > > > + at boot time by fsck. See
> > > > > > > > > fs_passno
> > > > > > > > > + parameter
> > > > > > > > > of
> > > > > > > > > + fstab(5). This parameter will
> > > > > > > > > be copied
> > > > > > > > > + into
> > > > > > > > > the
> > > > > > > > > + /etc/fstab file of the installed
> > > > > > > > > system. If not
> > > > > > > > > + specified the default value of
> > > > > > > > > "0" will be used. +
> > > > > > > > > --label label: Specifies the label to give
> > > > > > > > > to the filesystem to be made on the partition. If the
> > > > > > > > > given label is already in use by
> > > > > > > > > another filesystem, diff --git
> > > > > > > > > a/scripts/lib/wic/ksparser.py
> > > > > > > > > b/scripts/lib/wic/ksparser.py index 0df9eb0..a49b7b9
> > > > > > > > > 100644 --- a/scripts/lib/wic/ksparser.py +++
> > > > > > > > > b/scripts/lib/wic/ksparser.py @@ -155,6 +155,7 @@
> > > > > > > > > class KickStart():
> > > > > > > > > part.add_argument('--change-directory')
> > > > > > > > > part.add_argument("--extra-space",
> > > > > > > > > type=sizetype("M")) part.add_argument('--fsoptions',
> > > > > > > > > dest='fsopts')
> > > > > > > > > + part.add_argument('--fspassno',
> > > > > > > > > dest='fspassno') part.add_argument('--fstype',
> > > > > > > > > default='vfat', choices=('ext2', 'ext3',
> > > > > > > > > 'ext4', 'btrfs', 'squashfs', 'vfat', 'msdos', 'erofs',
> > > > > > > > > diff --git a/scripts/lib/wic/partition.py
> > > > > > > > > b/scripts/lib/wic/partition.py index 09e491d..e50871b
> > > > > > > > > 100644 --- a/scripts/lib/wic/partition.py +++
> > > > > > > > > b/scripts/lib/wic/partition.py @@ -33,6 +33,7 @@ class
> > > > > > > > > Partition(): self.include_path = args.include_path
> > > > > > > > > self.change_directory = args.change_directory
> > > > > > > > > self.fsopts = args.fsopts
> > > > > > > > > + self.fspassno = args.fspassno
> > > > > > > > > self.fstype = args.fstype
> > > > > > > > > self.label = args.label
> > > > > > > > > self.use_label = args.use_label diff --git
> > > > > > > > > a/scripts/lib/wic/plugins/imager/direct.py
> > > > > > > > > b/scripts/lib/wic/plugins/imager/direct.py index
> > > > > > > > > 4d0b836..c44159b
> > > > > > > > > 100644 --- a/scripts/lib/wic/plugins/imager/direct.py
> > > > > > > > > +++ b/scripts/lib/wic/plugins/imager/direct.py
> > > > > > > > > @@ -117,7 +117,7 @@ class DirectPlugin(ImagerPlugin):
> > > > > > > > > updated = False
> > > > > > > > > for part in self.parts:
> > > > > > > > > if not part.realnum or not
> > > > > > > > > part.mountpoint \
> > > > > > > > > - or part.mountpoint == "/" or not
> > > > > > > > > part.mountpoint.startswith('/'):
> > > > > > > > > + or not
> > > > > > > > > part.mountpoint.startswith('/'): continue
> > > > > > > > >
> > > > > > > > > if part.use_uuid:
> > > > > > > > > @@ -138,8 +138,9 @@ class DirectPlugin(ImagerPlugin):
> > > > > > > > > device_name = "/dev/%s%s%d" %
> > > > > > > > > (part.disk, prefix,
> > > > > > > > > part.realnum)
> > > > > > > > > opts = part.fsopts if part.fsopts else
> > > > > > > > > "defaults"
> > > > > > > > > + passno = part.fspassno if part.fspassno
> > > > > > > > > else "0" line = "\t".join([device_name,
> > > > > > > > > part.mountpoint, part.fstype,
> > > > > > > > > - opts, "0", "0"]) + "\n"
> > > > > > > > > + opts, "0", passno]) +
> > > > > > > > > "\n"
> > > > > > > > >
> > > > > > > > > fstab_lines.append(line)
> > > > > > > > > updated = True
> > > > > >
> > > > > > --
> > > > > > You received this message because you are subscribed to the
> > > > > > Google Groups "isar-users" group.
> > > > > > To unsubscribe from this group and stop receiving emails
> > > > > > from it, send an email to
> > > > > > isar-users+unsubscribe@googlegroups.com. To view this
> > > > > > discussion on the web visit
> > > > > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2F
> > > > > grou
> > > > >
> > > > > > ps.g
> > > > > > oogle.com%2Fd%2Fmsgid%2Fisar-
> > > > > >
> > > > >
> > users%2F8babb01d7eae44e0b9faecad59cdc1d4%2540mentor.com&dat
> > > > > a=
> > > > > >
> > > > >
> > 05%7C01%7Cfelix.moessbauer%40siemens.com%7C237b0db9bb724580ca540
> > > > > 8
> > > > > >
> > > > >
> > > >
> > da5024a3ad%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C6379104
> > > > > >
> > > > >
> > 15537105828%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQ
> > > > > Ijo
> > > > > >
> > > > >
> > iV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sda
> > > > > ta
> > > > > >
> > > > >
> > =%2F%2F1Fxy%2B%2BEgDI4RfUEZ38qy1Q82esArdBuRftDuE1t6s%3D&re
> > > > > se
> > > > > > rved=0.
>
^ permalink raw reply [flat|nested] 22+ messages in thread
* RE: [PATCH v2 2/2] wic: Update to the latest revision
2022-09-26 7:56 ` Kanagarajan, Vijaikumar
@ 2022-09-26 8:02 ` Moessbauer, Felix
2022-09-26 12:34 ` Henning Schild
1 sibling, 0 replies; 22+ messages in thread
From: Moessbauer, Felix @ 2022-09-26 8:02 UTC (permalink / raw)
To: Kanagarajan, Vijaikumar, Schild, Henning, jan.kiszka; +Cc: isar-users
> -----Original Message-----
> From: isar-users@googlegroups.com <isar-users@googlegroups.com> On
> Behalf Of Kanagarajan, Vijaikumar
> Sent: Monday, September 26, 2022 3:57 PM
> To: Schild, Henning (T CED SES-DE) <henning.schild@siemens.com>;
> Moessbauer, Felix (T CED SES-DE) <felix.moessbauer@siemens.com>; Kiszka,
> Jan (T CED) <jan.kiszka@siemens.com>
> Cc: isar-users@googlegroups.com
> Subject: RE: [PATCH v2 2/2] wic: Update to the latest revision
>
>
>
> > -----Original Message-----
> > From: Henning Schild <henning.schild@siemens.com>
> > Sent: 26 September 2022 13:17
> > To: felix.moessbauer@siemens.com; Kanagarajan, Vijaikumar
> > <Vijaikumar_Kanagarajan@mentor.com>; jan.kiszka@siemens.com
> > Cc: isar-users@googlegroups.com
> > Subject: Re: [PATCH v2 2/2] wic: Update to the latest revision
> >
> > Am Mon, 26 Sep 2022 07:57:41 +0200
> > schrieb "Moessbauer, Felix (T CED SES-DE)"
> > <felix.moessbauer@siemens.com>:
> >
> > > > -----Original Message-----
> > > > From: Kanagarajan, Vijaikumar <Vijaikumar_Kanagarajan@mentor.com>
> > > > Sent: Monday, September 26, 2022 12:53 PM
> > > > To: Moessbauer, Felix (T CED SES-DE)
> > <felix.moessbauer@siemens.com>;
> > > > Schild, Henning (T CED SES-DE) <henning.schild@siemens.com> Cc:
> > > > Kiszka, Jan (T CED) <jan.kiszka@siemens.com>; isar-
> > > > users@googlegroups.com
> > > > Subject: RE: [PATCH v2 2/2] wic: Update to the latest revision
> > > >
> > > >
> > > >
> > > > > -----Original Message-----
> > > > > From: Moessbauer, Felix (T CED SES-DE)
> > > > > <felix.moessbauer@siemens.com> Sent: 26 September 2022 07:11
> > > > > To: Kanagarajan, Vijaikumar <Vijaikumar_Kanagarajan@mentor.com>;
> > > > > Schild, Henning (T CED SES-DE) <henning.schild@siemens.com>
> > > > > Cc: Kiszka, Jan (T CED) <jan.kiszka@siemens.com>; isar-
> > > > > users@googlegroups.com
> > > > > Subject: RE: [PATCH v2 2/2] wic: Update to the latest revision
> > > > >
> > > > > Hi Vijai & Henning,
> > > > >
> > > > > Are there any news on this?
> > > > > I just discovered that still many (if not all) of our images are
> > > > > affected by this bug.
> > > >
> > > > Hi Felix,
> > > >
> > > > Unfortunately not. If we already have the latest wic,
> > > > --no-fstab-update[1] might help.(untested).
> > >
> > > We already have a wic version with this option, but it doesn't look
> > > right to add it to the root partition. Instead this option can be
> > > used for partitions that should not show up in the fstab at all.
> >
> > We do want the one from WIC. It is only coming since we patched WIC to
> > support /.
> >
> > > Indeed, the fstab entry has to be set by WIC to set settings like
> > > UUID and filesystem. But this requires that we either remove the
> > > entry set by ISAR in image_configure_fstab, or we remove the entry
> > > after generating the rootfs, but before WIC.
> >
> > I think we should try and remove the entry from that template. It
> > should not be required.
> >
> > If we decide to solve the problem somewhere in/with WIC, we should do
> > that upstream in OE. They have the same "we now have two" problem,
> > where wic adds on top of a template instead of replacing the original.
>
> This seems to me like the approach we should take. Make wic replace entries in
> fstab instead of appending to them like how it does now.
Sounds like a plan!
Would you like to take care of writing the WIC patches, Vijai?
Happy Coding!
Felix
>
> Thanks,
> Vijai Kumar K
>
> >
> > Henning
> >
> > > Opinions? @Kiszka, Jan (T CED)
> > >
> > > Felix
> > >
> > > >
> > > > Thanks,
> > > > Vijai Kumar K
> > > >
> > > > >
> > > > > While it is not serios, having error messages in the journal is
> > > > > still pretty bad: systemd-fstab-generator[1491]: Failed to
> > > > > create unit file /run/systemd/generator/-.mount, as it already exists.
> > > > > Duplicate entry in /etc/fstab?
> > > > >
> > > > > Felix
> > > > >
> > > > > > -----Original Message-----
> > > > > > From: isar-users@googlegroups.com <isar-
> > users@googlegroups.com>
> > > > > > On Behalf Of Kanagarajan, Vijaikumar
> > > > > > Sent: Friday, June 17, 2022 1:46 PM
> > > > > > To: isar-users@googlegroups.com
> > > > > > Cc: Kiszka, Jan (T CED) <jan.kiszka@siemens.com>; Schild,
> > > > > > Henning (T CED SES-
> > > > > > DE) <henning.schild@siemens.com>
> > > > > > Subject: RE: [PATCH v2 2/2] wic: Update to the latest revision
> > > > > >
> > > > > >
> > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: Kanagarajan, Vijaikumar
> > > > > > > Sent: 13 June 2022 14:48
> > > > > > > To: 'Henning Schild' <henning.schild@siemens.com>
> > > > > > > Cc: isar-users@googlegroups.com; jan.kiszka@siemens.com
> > > > > > > Subject: RE: [PATCH v2 2/2] wic: Update to the latest
> > > > > > > revision
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > > -----Original Message-----
> > > > > > > > From: Henning Schild <henning.schild@siemens.com>
> > > > > > > > Sent: 13 June 2022 14:32
> > > > > > > > To: Kanagarajan, Vijaikumar
> > > > > > > > <Vijaikumar_Kanagarajan@mentor.com> Cc:
> > > > > > > > isar-users@googlegroups.com; jan.kiszka@siemens.com
> > > > > > > > Subject: Re: [PATCH v2 2/2] wic: Update to the latest
> > > > > > > > revision
> > > > > > > >
> > > > > > > > Am Mon, 13 Jun 2022 13:24:01 +0530 schrieb Vijai Kumar K
> > > > > > > > <Vijaikumar_Kanagarajan@mentor.com>:
> > > > > > > >
> > > > > > > > > Update to the latest wic from OE-core.
> > > > > > > > >
> > > > > > > > > OE-core Revision:
> > 06ed2aa93fc25a681e3a00ee120d9395b04845da
> > > > > > > > >
> > > > > > > > > Signed-off-by: Vijai Kumar K
> > > > > <Vijaikumar_Kanagarajan@mentor.com>
> > > > > > > > > ---
> > > > > > > > > RECIPE-API-CHANGELOG.md | 19
> > > > > > > > > +++++++++++++++++++ scripts/lib/wic/help.py
> > > > > > > > > | 6 ++++++ scripts/lib/wic/ksparser.py |
> > > > > > > > > 1 + scripts/lib/wic/partition.py | 1 +
> > > > > > > > > scripts/lib/wic/plugins/imager/direct.py | 5 +++--
> > > > > > > > > 5 files changed, 30 insertions(+), 2 deletions(-)
> > > > > > > > >
> > > > > > > > > diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-
> > > > > CHANGELOG.md
> > > > > > > > index
> > > > > > > > > 2998824..436b204 100644
> > > > > > > > > --- a/RECIPE-API-CHANGELOG.md
> > > > > > > > > +++ b/RECIPE-API-CHANGELOG.md
> > > > > > > > > @@ -406,3 +406,22 @@ groups are now created after all
> > > > > > > > > packages are installed.
> > > > > > > > > Changes in next
> > > > > > > > > ---------------
> > > > > > > > > +
> > > > > > > > > +### Wic adds / mountpoint to fstab
> > > > > > > > > +
> > > > > > > > > +In the older version of wic, any mount point named / is
> > > > > > > > > +skipped from
> > > > > > > > > adding +into the fstab entry.
> > > > > > > > > +
> > > > > > > > > +With the latest wic, this is not the case. / mount
> > > > > > > > > point, if +any, is
> > > > > > > > > added +to /etc/fstab for automount.
> > > > > > > > > +
> > > > > > > > > +Any wks file which assumed that / would be skipped from
> > > > > > > > > +/etc/fstab
> > > > > > > > > should +now be corrected. Otherwise, it might result in
> > > > > > > > > multiple / entries in /etc/fstab. +One by the
> > > > > > > > > image_configure_fstab function from image class and one
> > > > > > > > > by wic. +
> > > > > > > > > +Below is an example wks entry that might cause an issue.
> > > > > > > > > +Replace "part /" with "part" to avoid adding duplicate
> > > > > > > > > entry for /.
> > > > > > > >
> > > > > > > > I am not sure multiple entries would be an issue, i think
> > > > > > > > the last one will simply always win. It is good to have a
> > > > > > > > note about that change, but i doubt people really have to
> > > > > > > > do anything. And if anything has to be done, isar should
> > > > > > > > drop the first one centrally.
> > > > > > > >
> > > > > > > > The fstab entry for root is purely for adding some weird
> > > > > > > > mount options, usually root is not mounted with fstab but
> > > > > > > > with cmdline and initrd. And not having / in fstab just
> > > > > > > > makes that rootfs more generic ... it can become part of
> > > > > > > > nfs and next guy takes it as btrfs while next guy
> > > > > > > takes it as ext4.
> > > > > > > >
> > > > > > > > Note is good, advise is not
> > > > > > >
> > > > > > > Noted. Will change the text accordingly in V3.
> > > > > >
> > > > > > Ok we cannot drop / mountpoint name(though optional). We would
> > > > > > be
> > > > > lose
> > > > > > the ability to find out UUID of root using rootdev api.
> > > > > > This is used by some plugins. So the better way would be to
> > > > > > use the --no-fstab- update. Seems like we can set them per
> > > > > > partition now.
> > > > > >
> > > > > > Thanks,
> > > > > > Vijai Kumar K
> > > > > >
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Vijai Kumar K
> > > > > > >
> > > > > > > >
> > > > > > > > Henning
> > > > > > > >
> > > > > > > > > +
> > > > > > > > > +```
> > > > > > > > > +part / --source rootfs --ondisk sda --fstype ext4
> > > > > > > > > +--label platform
> > > > > > > > > --align 1024 --use-uuid +``` diff --git
> > > > > > > > > a/scripts/lib/wic/help.py b/scripts/lib/wic/help.py
> > > > > > > > > index
> > > > > > > > > 4ff7470..73e3380 100644
> > > > > > > > > --- a/scripts/lib/wic/help.py
> > > > > > > > > +++ b/scripts/lib/wic/help.py
> > > > > > > > > @@ -940,6 +940,12 @@ DESCRIPTION
> > > > > > > > > quotes. If not specified, the
> > > > > > > > > default string is "defaults".
> > > > > > > > >
> > > > > > > > > + --fspassno: Specifies the order in which
> > > > > > > > > filesystem
> > > > > > > > > + checks
> > > > > > > > > are done
> > > > > > > > > + at boot time by fsck. See
> > > > > > > > > + fs_passno parameter
> > > > > > > > > of
> > > > > > > > > + fstab(5). This parameter will be
> > > > > > > > > copied
> > > > > > > > > + into
> > > > > > > > > the
> > > > > > > > > + /etc/fstab file of the installed
> > > > > > > > > system. If not
> > > > > > > > > + specified the default value of "0"
> > > > > > > > > will be used. +
> > > > > > > > > --label label: Specifies the label to give to
> > > > > > > > > the filesystem to be made on the partition. If the given
> > > > > > > > > label is already in use by
> > > > > > > > > another filesystem, diff --git
> > > > > > > > > a/scripts/lib/wic/ksparser.py
> > > > > > > > > b/scripts/lib/wic/ksparser.py index 0df9eb0..a49b7b9
> > > > > > > > > 100644 --- a/scripts/lib/wic/ksparser.py +++
> > > > > > > > > b/scripts/lib/wic/ksparser.py @@ -155,6 +155,7 @@ class
> > > > > > > > > KickStart(): part.add_argument('--change-directory')
> > > > > > > > > part.add_argument("--extra-space",
> > > > > > > > > type=sizetype("M")) part.add_argument('--fsoptions',
> > > > > > > > > dest='fsopts')
> > > > > > > > > + part.add_argument('--fspassno',
> > > > > > > > > + dest='fspassno')
> > > > > > > > > part.add_argument('--fstype', default='vfat',
> > > > > > > > > choices=('ext2', 'ext3',
> > > > > > > > > 'ext4', 'btrfs', 'squashfs', 'vfat', 'msdos', 'erofs',
> > > > > > > > > diff --git a/scripts/lib/wic/partition.py
> > > > > > > > > b/scripts/lib/wic/partition.py index 09e491d..e50871b
> > > > > > > > > 100644 --- a/scripts/lib/wic/partition.py +++
> > > > > > > > > b/scripts/lib/wic/partition.py @@ -33,6 +33,7 @@ class
> > > > > > > > > Partition(): self.include_path = args.include_path
> > > > > > > > > self.change_directory = args.change_directory
> > > > > > > > > self.fsopts = args.fsopts
> > > > > > > > > + self.fspassno = args.fspassno
> > > > > > > > > self.fstype = args.fstype
> > > > > > > > > self.label = args.label
> > > > > > > > > self.use_label = args.use_label diff --git
> > > > > > > > > a/scripts/lib/wic/plugins/imager/direct.py
> > > > > > > > > b/scripts/lib/wic/plugins/imager/direct.py index
> > > > > > > > > 4d0b836..c44159b
> > > > > > > > > 100644 --- a/scripts/lib/wic/plugins/imager/direct.py
> > > > > > > > > +++ b/scripts/lib/wic/plugins/imager/direct.py
> > > > > > > > > @@ -117,7 +117,7 @@ class DirectPlugin(ImagerPlugin):
> > > > > > > > > updated = False
> > > > > > > > > for part in self.parts:
> > > > > > > > > if not part.realnum or not part.mountpoint \
> > > > > > > > > - or part.mountpoint == "/" or not
> > > > > > > > > part.mountpoint.startswith('/'):
> > > > > > > > > + or not part.mountpoint.startswith('/'):
> > > > > > > > > continue
> > > > > > > > >
> > > > > > > > > if part.use_uuid:
> > > > > > > > > @@ -138,8 +138,9 @@ class DirectPlugin(ImagerPlugin):
> > > > > > > > > device_name = "/dev/%s%s%d" %
> > > > > > > > > (part.disk, prefix,
> > > > > > > > > part.realnum)
> > > > > > > > > opts = part.fsopts if part.fsopts else
> > > > > > > > > "defaults"
> > > > > > > > > + passno = part.fspassno if part.fspassno
> > > > > > > > > + else
> > > > > > > > > "0" line = "\t".join([device_name, part.mountpoint,
> > > > > > > > > part.fstype,
> > > > > > > > > - opts, "0", "0"]) + "\n"
> > > > > > > > > + opts, "0", passno]) + "\n"
> > > > > > > > >
> > > > > > > > > fstab_lines.append(line)
> > > > > > > > > updated = True
> > > > > >
> > > > > > --
> > > > > > You received this message because you are subscribed to the
> > > > > > Google Groups "isar-users" group.
> > > > > > To unsubscribe from this group and stop receiving emails from
> > > > > > it, send an email to isar-users+unsubscribe@googlegroups.com.
> > > > > > To view this discussion on the web visit
> > > > > >
> > > > > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%
> > > > > 2F
> > > > > grou
> > > > >
> > > > > > ps.g
> > > > > > oogle.com%2Fd%2Fmsgid%2Fisar-
> > > > > >
> > > > >
> > users%2F8babb01d7eae44e0b9faecad59cdc1d4%2540mentor.com&dat
> > > > > a=
> > > > > >
> > > > >
> > 05%7C01%7Cfelix.moessbauer%40siemens.com%7C237b0db9bb724580ca540
> > > > > 8
> > > > > >
> > > > >
> > > >
> >
> da5024a3ad%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C6379104
> > > > > >
> > > > >
> > 15537105828%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQ
> > > > > Ijo
> > > > > >
> > > > >
> > iV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sda
> > > > > ta
> > > > > >
> > > > >
> > =%2F%2F1Fxy%2B%2BEgDI4RfUEZ38qy1Q82esArdBuRftDuE1t6s%3D&re
> > > > > se
> > > > > > rved=0.
>
> --
> You received this message because you are subscribed to the Google Groups
> "isar-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email
> to isar-users+unsubscribe@googlegroups.com.
> To view this discussion on the web visit
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.g
> oogle.com%2Fd%2Fmsgid%2Fisar-
> users%2F7d23828243444e90afd49e7bb23717dc%2540mentor.com&data=
> 05%7C01%7Cfelix.moessbauer%40siemens.com%7C7e073a4d25ec493e0d3408
> da9f94ad76%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C6379977
> 58159199068%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjo
> iV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata
> =pT2NMVWp6ld7iDW%2BRStJABC02WCrzGDOV%2BXhvpxPKO4%3D&reser
> ved=0.
^ permalink raw reply [flat|nested] 22+ messages in thread
* RE: [PATCH v2 2/2] wic: Update to the latest revision
2022-09-26 7:46 ` Henning Schild
@ 2022-09-26 7:56 ` Kanagarajan, Vijaikumar
2022-09-26 8:02 ` Moessbauer, Felix
2022-09-26 12:34 ` Henning Schild
0 siblings, 2 replies; 22+ messages in thread
From: Kanagarajan, Vijaikumar @ 2022-09-26 7:56 UTC (permalink / raw)
To: Henning Schild, felix.moessbauer, jan.kiszka; +Cc: isar-users
> -----Original Message-----
> From: Henning Schild <henning.schild@siemens.com>
> Sent: 26 September 2022 13:17
> To: felix.moessbauer@siemens.com; Kanagarajan, Vijaikumar
> <Vijaikumar_Kanagarajan@mentor.com>; jan.kiszka@siemens.com
> Cc: isar-users@googlegroups.com
> Subject: Re: [PATCH v2 2/2] wic: Update to the latest revision
>
> Am Mon, 26 Sep 2022 07:57:41 +0200
> schrieb "Moessbauer, Felix (T CED SES-DE)"
> <felix.moessbauer@siemens.com>:
>
> > > -----Original Message-----
> > > From: Kanagarajan, Vijaikumar <Vijaikumar_Kanagarajan@mentor.com>
> > > Sent: Monday, September 26, 2022 12:53 PM
> > > To: Moessbauer, Felix (T CED SES-DE)
> <felix.moessbauer@siemens.com>;
> > > Schild, Henning (T CED SES-DE) <henning.schild@siemens.com> Cc:
> > > Kiszka, Jan (T CED) <jan.kiszka@siemens.com>; isar-
> > > users@googlegroups.com
> > > Subject: RE: [PATCH v2 2/2] wic: Update to the latest revision
> > >
> > >
> > >
> > > > -----Original Message-----
> > > > From: Moessbauer, Felix (T CED SES-DE)
> > > > <felix.moessbauer@siemens.com> Sent: 26 September 2022 07:11
> > > > To: Kanagarajan, Vijaikumar <Vijaikumar_Kanagarajan@mentor.com>;
> > > > Schild, Henning (T CED SES-DE) <henning.schild@siemens.com>
> > > > Cc: Kiszka, Jan (T CED) <jan.kiszka@siemens.com>; isar-
> > > > users@googlegroups.com
> > > > Subject: RE: [PATCH v2 2/2] wic: Update to the latest revision
> > > >
> > > > Hi Vijai & Henning,
> > > >
> > > > Are there any news on this?
> > > > I just discovered that still many (if not all) of our images are
> > > > affected by this bug.
> > >
> > > Hi Felix,
> > >
> > > Unfortunately not. If we already have the latest wic,
> > > --no-fstab-update[1] might help.(untested).
> >
> > We already have a wic version with this option, but it doesn't look
> > right to add it to the root partition. Instead this option can be used
> > for partitions that should not show up in the fstab at all.
>
> We do want the one from WIC. It is only coming since we patched WIC to
> support /.
>
> > Indeed, the fstab entry has to be set by WIC to set settings like UUID
> > and filesystem. But this requires that we either remove the entry set
> > by ISAR in image_configure_fstab, or we remove the entry after
> > generating the rootfs, but before WIC.
>
> I think we should try and remove the entry from that template. It should not
> be required.
>
> If we decide to solve the problem somewhere in/with WIC, we should do
> that upstream in OE. They have the same "we now have two" problem,
> where wic adds on top of a template instead of replacing the original.
This seems to me like the approach we should take. Make wic replace entries in fstab instead of appending to them like how it does now.
Thanks,
Vijai Kumar K
>
> Henning
>
> > Opinions? @Kiszka, Jan (T CED)
> >
> > Felix
> >
> > >
> > > Thanks,
> > > Vijai Kumar K
> > >
> > > >
> > > > While it is not serios, having error messages in the journal is
> > > > still pretty bad: systemd-fstab-generator[1491]: Failed to create
> > > > unit file /run/systemd/generator/-.mount, as it already exists.
> > > > Duplicate entry in /etc/fstab?
> > > >
> > > > Felix
> > > >
> > > > > -----Original Message-----
> > > > > From: isar-users@googlegroups.com <isar-
> users@googlegroups.com>
> > > > > On Behalf Of Kanagarajan, Vijaikumar
> > > > > Sent: Friday, June 17, 2022 1:46 PM
> > > > > To: isar-users@googlegroups.com
> > > > > Cc: Kiszka, Jan (T CED) <jan.kiszka@siemens.com>; Schild,
> > > > > Henning (T CED SES-
> > > > > DE) <henning.schild@siemens.com>
> > > > > Subject: RE: [PATCH v2 2/2] wic: Update to the latest revision
> > > > >
> > > > >
> > > > >
> > > > > > -----Original Message-----
> > > > > > From: Kanagarajan, Vijaikumar
> > > > > > Sent: 13 June 2022 14:48
> > > > > > To: 'Henning Schild' <henning.schild@siemens.com>
> > > > > > Cc: isar-users@googlegroups.com; jan.kiszka@siemens.com
> > > > > > Subject: RE: [PATCH v2 2/2] wic: Update to the latest revision
> > > > > >
> > > > > >
> > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: Henning Schild <henning.schild@siemens.com>
> > > > > > > Sent: 13 June 2022 14:32
> > > > > > > To: Kanagarajan, Vijaikumar
> > > > > > > <Vijaikumar_Kanagarajan@mentor.com> Cc:
> > > > > > > isar-users@googlegroups.com; jan.kiszka@siemens.com
> > > > > > > Subject: Re: [PATCH v2 2/2] wic: Update to the latest
> > > > > > > revision
> > > > > > >
> > > > > > > Am Mon, 13 Jun 2022 13:24:01 +0530 schrieb Vijai Kumar K
> > > > > > > <Vijaikumar_Kanagarajan@mentor.com>:
> > > > > > >
> > > > > > > > Update to the latest wic from OE-core.
> > > > > > > >
> > > > > > > > OE-core Revision:
> 06ed2aa93fc25a681e3a00ee120d9395b04845da
> > > > > > > >
> > > > > > > > Signed-off-by: Vijai Kumar K
> > > > <Vijaikumar_Kanagarajan@mentor.com>
> > > > > > > > ---
> > > > > > > > RECIPE-API-CHANGELOG.md | 19
> > > > > > > > +++++++++++++++++++ scripts/lib/wic/help.py
> > > > > > > > | 6 ++++++ scripts/lib/wic/ksparser.py |
> > > > > > > > 1 + scripts/lib/wic/partition.py | 1 +
> > > > > > > > scripts/lib/wic/plugins/imager/direct.py | 5 +++--
> > > > > > > > 5 files changed, 30 insertions(+), 2 deletions(-)
> > > > > > > >
> > > > > > > > diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-
> > > > CHANGELOG.md
> > > > > > > index
> > > > > > > > 2998824..436b204 100644
> > > > > > > > --- a/RECIPE-API-CHANGELOG.md
> > > > > > > > +++ b/RECIPE-API-CHANGELOG.md
> > > > > > > > @@ -406,3 +406,22 @@ groups are now created after all
> > > > > > > > packages are installed.
> > > > > > > > Changes in next
> > > > > > > > ---------------
> > > > > > > > +
> > > > > > > > +### Wic adds / mountpoint to fstab
> > > > > > > > +
> > > > > > > > +In the older version of wic, any mount point named / is
> > > > > > > > +skipped from
> > > > > > > > adding +into the fstab entry.
> > > > > > > > +
> > > > > > > > +With the latest wic, this is not the case. / mount
> > > > > > > > point, if +any, is
> > > > > > > > added +to /etc/fstab for automount.
> > > > > > > > +
> > > > > > > > +Any wks file which assumed that / would be skipped from
> > > > > > > > +/etc/fstab
> > > > > > > > should +now be corrected. Otherwise, it might result in
> > > > > > > > multiple / entries in /etc/fstab. +One by the
> > > > > > > > image_configure_fstab function from image class and one by
> > > > > > > > wic. +
> > > > > > > > +Below is an example wks entry that might cause an issue.
> > > > > > > > +Replace "part /" with "part" to avoid adding duplicate
> > > > > > > > entry for /.
> > > > > > >
> > > > > > > I am not sure multiple entries would be an issue, i think
> > > > > > > the last one will simply always win. It is good to have a
> > > > > > > note about that change, but i doubt people really have to do
> > > > > > > anything. And if anything has to be done, isar should drop
> > > > > > > the first one centrally.
> > > > > > >
> > > > > > > The fstab entry for root is purely for adding some weird
> > > > > > > mount options, usually root is not mounted with fstab but
> > > > > > > with cmdline and initrd. And not having / in fstab just
> > > > > > > makes that rootfs more generic ... it can become part of nfs
> > > > > > > and next guy takes it as btrfs while next guy
> > > > > > takes it as ext4.
> > > > > > >
> > > > > > > Note is good, advise is not
> > > > > >
> > > > > > Noted. Will change the text accordingly in V3.
> > > > >
> > > > > Ok we cannot drop / mountpoint name(though optional). We would
> > > > > be
> > > > lose
> > > > > the ability to find out UUID of root using rootdev api.
> > > > > This is used by some plugins. So the better way would be to use
> > > > > the --no-fstab- update. Seems like we can set them per partition
> > > > > now.
> > > > >
> > > > > Thanks,
> > > > > Vijai Kumar K
> > > > >
> > > > > >
> > > > > > Thanks,
> > > > > > Vijai Kumar K
> > > > > >
> > > > > > >
> > > > > > > Henning
> > > > > > >
> > > > > > > > +
> > > > > > > > +```
> > > > > > > > +part / --source rootfs --ondisk sda --fstype ext4 --label
> > > > > > > > +platform
> > > > > > > > --align 1024 --use-uuid +``` diff --git
> > > > > > > > a/scripts/lib/wic/help.py b/scripts/lib/wic/help.py index
> > > > > > > > 4ff7470..73e3380 100644
> > > > > > > > --- a/scripts/lib/wic/help.py
> > > > > > > > +++ b/scripts/lib/wic/help.py
> > > > > > > > @@ -940,6 +940,12 @@ DESCRIPTION
> > > > > > > > quotes. If not specified, the
> > > > > > > > default string is "defaults".
> > > > > > > >
> > > > > > > > + --fspassno: Specifies the order in which
> > > > > > > > filesystem
> > > > > > > > + checks
> > > > > > > > are done
> > > > > > > > + at boot time by fsck. See fs_passno
> > > > > > > > + parameter
> > > > > > > > of
> > > > > > > > + fstab(5). This parameter will be
> > > > > > > > copied
> > > > > > > > + into
> > > > > > > > the
> > > > > > > > + /etc/fstab file of the installed
> > > > > > > > system. If not
> > > > > > > > + specified the default value of "0"
> > > > > > > > will be used. +
> > > > > > > > --label label: Specifies the label to give to
> > > > > > > > the filesystem to be made on the partition. If the given
> > > > > > > > label is already in use by
> > > > > > > > another filesystem, diff --git
> > > > > > > > a/scripts/lib/wic/ksparser.py
> > > > > > > > b/scripts/lib/wic/ksparser.py index 0df9eb0..a49b7b9
> > > > > > > > 100644 --- a/scripts/lib/wic/ksparser.py +++
> > > > > > > > b/scripts/lib/wic/ksparser.py @@ -155,6 +155,7 @@ class
> > > > > > > > KickStart(): part.add_argument('--change-directory')
> > > > > > > > part.add_argument("--extra-space",
> > > > > > > > type=sizetype("M")) part.add_argument('--fsoptions',
> > > > > > > > dest='fsopts')
> > > > > > > > + part.add_argument('--fspassno', dest='fspassno')
> > > > > > > > part.add_argument('--fstype', default='vfat',
> > > > > > > > choices=('ext2', 'ext3',
> > > > > > > > 'ext4', 'btrfs', 'squashfs', 'vfat', 'msdos', 'erofs',
> > > > > > > > diff --git a/scripts/lib/wic/partition.py
> > > > > > > > b/scripts/lib/wic/partition.py index 09e491d..e50871b
> > > > > > > > 100644 --- a/scripts/lib/wic/partition.py +++
> > > > > > > > b/scripts/lib/wic/partition.py @@ -33,6 +33,7 @@ class
> > > > > > > > Partition(): self.include_path = args.include_path
> > > > > > > > self.change_directory = args.change_directory
> > > > > > > > self.fsopts = args.fsopts
> > > > > > > > + self.fspassno = args.fspassno
> > > > > > > > self.fstype = args.fstype
> > > > > > > > self.label = args.label
> > > > > > > > self.use_label = args.use_label diff --git
> > > > > > > > a/scripts/lib/wic/plugins/imager/direct.py
> > > > > > > > b/scripts/lib/wic/plugins/imager/direct.py index
> > > > > > > > 4d0b836..c44159b
> > > > > > > > 100644 --- a/scripts/lib/wic/plugins/imager/direct.py
> > > > > > > > +++ b/scripts/lib/wic/plugins/imager/direct.py
> > > > > > > > @@ -117,7 +117,7 @@ class DirectPlugin(ImagerPlugin):
> > > > > > > > updated = False
> > > > > > > > for part in self.parts:
> > > > > > > > if not part.realnum or not part.mountpoint \
> > > > > > > > - or part.mountpoint == "/" or not
> > > > > > > > part.mountpoint.startswith('/'):
> > > > > > > > + or not part.mountpoint.startswith('/'):
> > > > > > > > continue
> > > > > > > >
> > > > > > > > if part.use_uuid:
> > > > > > > > @@ -138,8 +138,9 @@ class DirectPlugin(ImagerPlugin):
> > > > > > > > device_name = "/dev/%s%s%d" % (part.disk,
> > > > > > > > prefix,
> > > > > > > > part.realnum)
> > > > > > > > opts = part.fsopts if part.fsopts else
> > > > > > > > "defaults"
> > > > > > > > + passno = part.fspassno if part.fspassno else
> > > > > > > > "0" line = "\t".join([device_name, part.mountpoint,
> > > > > > > > part.fstype,
> > > > > > > > - opts, "0", "0"]) + "\n"
> > > > > > > > + opts, "0", passno]) + "\n"
> > > > > > > >
> > > > > > > > fstab_lines.append(line)
> > > > > > > > updated = True
> > > > >
> > > > > --
> > > > > You received this message because you are subscribed to the
> > > > > Google Groups "isar-users" group.
> > > > > To unsubscribe from this group and stop receiving emails from
> > > > > it, send an email to isar-users+unsubscribe@googlegroups.com.
> > > > > To view this discussion on the web visit
> > > > >
> > > > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2F
> > > > grou
> > > >
> > > > > ps.g
> > > > > oogle.com%2Fd%2Fmsgid%2Fisar-
> > > > >
> > > >
> users%2F8babb01d7eae44e0b9faecad59cdc1d4%2540mentor.com&dat
> > > > a=
> > > > >
> > > >
> 05%7C01%7Cfelix.moessbauer%40siemens.com%7C237b0db9bb724580ca540
> > > > 8
> > > > >
> > > >
> > >
> da5024a3ad%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C6379104
> > > > >
> > > >
> 15537105828%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQ
> > > > Ijo
> > > > >
> > > >
> iV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sda
> > > > ta
> > > > >
> > > >
> =%2F%2F1Fxy%2B%2BEgDI4RfUEZ38qy1Q82esArdBuRftDuE1t6s%3D&re
> > > > se
> > > > > rved=0.
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH v2 2/2] wic: Update to the latest revision
2022-09-26 5:57 ` Moessbauer, Felix
@ 2022-09-26 7:46 ` Henning Schild
2022-09-26 7:56 ` Kanagarajan, Vijaikumar
0 siblings, 1 reply; 22+ messages in thread
From: Henning Schild @ 2022-09-26 7:46 UTC (permalink / raw)
To: felix.moessbauer, Vijaikumar_Kanagarajan, jan.kiszka; +Cc: isar-users
Am Mon, 26 Sep 2022 07:57:41 +0200
schrieb "Moessbauer, Felix (T CED SES-DE)"
<felix.moessbauer@siemens.com>:
> > -----Original Message-----
> > From: Kanagarajan, Vijaikumar <Vijaikumar_Kanagarajan@mentor.com>
> > Sent: Monday, September 26, 2022 12:53 PM
> > To: Moessbauer, Felix (T CED SES-DE)
> > <felix.moessbauer@siemens.com>; Schild, Henning (T CED SES-DE)
> > <henning.schild@siemens.com> Cc: Kiszka, Jan (T CED)
> > <jan.kiszka@siemens.com>; isar- users@googlegroups.com
> > Subject: RE: [PATCH v2 2/2] wic: Update to the latest revision
> >
> >
> >
> > > -----Original Message-----
> > > From: Moessbauer, Felix (T CED SES-DE)
> > > <felix.moessbauer@siemens.com> Sent: 26 September 2022 07:11
> > > To: Kanagarajan, Vijaikumar <Vijaikumar_Kanagarajan@mentor.com>;
> > > Schild, Henning (T CED SES-DE) <henning.schild@siemens.com>
> > > Cc: Kiszka, Jan (T CED) <jan.kiszka@siemens.com>; isar-
> > > users@googlegroups.com
> > > Subject: RE: [PATCH v2 2/2] wic: Update to the latest revision
> > >
> > > Hi Vijai & Henning,
> > >
> > > Are there any news on this?
> > > I just discovered that still many (if not all) of our images are
> > > affected by this bug.
> >
> > Hi Felix,
> >
> > Unfortunately not. If we already have the latest wic,
> > --no-fstab-update[1] might help.(untested).
>
> We already have a wic version with this option, but it doesn't look
> right to add it to the root partition. Instead this option can be
> used for partitions that should not show up in the fstab at all.
We do want the one from WIC. It is only coming since we patched WIC to
support /.
> Indeed, the fstab entry has to be set by WIC to set settings like
> UUID and filesystem. But this requires that we either remove the
> entry set by ISAR in image_configure_fstab, or we remove the entry
> after generating the rootfs, but before WIC.
I think we should try and remove the entry from that template. It
should not be required.
If we decide to solve the problem somewhere in/with WIC, we should do
that upstream in OE. They have the same "we now have two" problem,
where wic adds on top of a template instead of replacing the original.
Henning
> Opinions? @Kiszka, Jan (T CED)
>
> Felix
>
> >
> > Thanks,
> > Vijai Kumar K
> >
> > >
> > > While it is not serios, having error messages in the journal is
> > > still pretty bad: systemd-fstab-generator[1491]: Failed to create
> > > unit file /run/systemd/generator/-.mount, as it already exists.
> > > Duplicate entry in /etc/fstab?
> > >
> > > Felix
> > >
> > > > -----Original Message-----
> > > > From: isar-users@googlegroups.com <isar-users@googlegroups.com>
> > > > On Behalf Of Kanagarajan, Vijaikumar
> > > > Sent: Friday, June 17, 2022 1:46 PM
> > > > To: isar-users@googlegroups.com
> > > > Cc: Kiszka, Jan (T CED) <jan.kiszka@siemens.com>; Schild,
> > > > Henning (T CED SES-
> > > > DE) <henning.schild@siemens.com>
> > > > Subject: RE: [PATCH v2 2/2] wic: Update to the latest revision
> > > >
> > > >
> > > >
> > > > > -----Original Message-----
> > > > > From: Kanagarajan, Vijaikumar
> > > > > Sent: 13 June 2022 14:48
> > > > > To: 'Henning Schild' <henning.schild@siemens.com>
> > > > > Cc: isar-users@googlegroups.com; jan.kiszka@siemens.com
> > > > > Subject: RE: [PATCH v2 2/2] wic: Update to the latest revision
> > > > >
> > > > >
> > > > >
> > > > > > -----Original Message-----
> > > > > > From: Henning Schild <henning.schild@siemens.com>
> > > > > > Sent: 13 June 2022 14:32
> > > > > > To: Kanagarajan, Vijaikumar
> > > > > > <Vijaikumar_Kanagarajan@mentor.com> Cc:
> > > > > > isar-users@googlegroups.com; jan.kiszka@siemens.com
> > > > > > Subject: Re: [PATCH v2 2/2] wic: Update to the latest
> > > > > > revision
> > > > > >
> > > > > > Am Mon, 13 Jun 2022 13:24:01 +0530 schrieb Vijai Kumar K
> > > > > > <Vijaikumar_Kanagarajan@mentor.com>:
> > > > > >
> > > > > > > Update to the latest wic from OE-core.
> > > > > > >
> > > > > > > OE-core Revision: 06ed2aa93fc25a681e3a00ee120d9395b04845da
> > > > > > >
> > > > > > > Signed-off-by: Vijai Kumar K
> > > <Vijaikumar_Kanagarajan@mentor.com>
> > > > > > > ---
> > > > > > > RECIPE-API-CHANGELOG.md | 19
> > > > > > > +++++++++++++++++++ scripts/lib/wic/help.py
> > > > > > > | 6 ++++++ scripts/lib/wic/ksparser.py |
> > > > > > > 1 + scripts/lib/wic/partition.py | 1 +
> > > > > > > scripts/lib/wic/plugins/imager/direct.py | 5 +++--
> > > > > > > 5 files changed, 30 insertions(+), 2 deletions(-)
> > > > > > >
> > > > > > > diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-
> > > CHANGELOG.md
> > > > > > index
> > > > > > > 2998824..436b204 100644
> > > > > > > --- a/RECIPE-API-CHANGELOG.md
> > > > > > > +++ b/RECIPE-API-CHANGELOG.md
> > > > > > > @@ -406,3 +406,22 @@ groups are now created after all
> > > > > > > packages are installed.
> > > > > > > Changes in next
> > > > > > > ---------------
> > > > > > > +
> > > > > > > +### Wic adds / mountpoint to fstab
> > > > > > > +
> > > > > > > +In the older version of wic, any mount point named / is
> > > > > > > +skipped from
> > > > > > > adding +into the fstab entry.
> > > > > > > +
> > > > > > > +With the latest wic, this is not the case. / mount
> > > > > > > point, if +any, is
> > > > > > > added +to /etc/fstab for automount.
> > > > > > > +
> > > > > > > +Any wks file which assumed that / would be skipped from
> > > > > > > +/etc/fstab
> > > > > > > should +now be corrected. Otherwise, it might result in
> > > > > > > multiple / entries in /etc/fstab. +One by the
> > > > > > > image_configure_fstab function from image class and one by
> > > > > > > wic. +
> > > > > > > +Below is an example wks entry that might cause an issue.
> > > > > > > +Replace "part /" with "part" to avoid adding duplicate
> > > > > > > entry for /.
> > > > > >
> > > > > > I am not sure multiple entries would be an issue, i think
> > > > > > the last one will simply always win. It is good to have a
> > > > > > note about that change, but i doubt people really have to
> > > > > > do anything. And if anything has to be done, isar should
> > > > > > drop the first one centrally.
> > > > > >
> > > > > > The fstab entry for root is purely for adding some weird
> > > > > > mount options, usually root is not mounted with fstab but
> > > > > > with cmdline and initrd. And not having / in fstab just
> > > > > > makes that rootfs more generic ... it can become part of
> > > > > > nfs and next guy takes it as btrfs while next guy
> > > > > takes it as ext4.
> > > > > >
> > > > > > Note is good, advise is not
> > > > >
> > > > > Noted. Will change the text accordingly in V3.
> > > >
> > > > Ok we cannot drop / mountpoint name(though optional). We would
> > > > be
> > > lose
> > > > the ability to find out UUID of root using rootdev api.
> > > > This is used by some plugins. So the better way would be to use
> > > > the --no-fstab- update. Seems like we can set them per
> > > > partition now.
> > > >
> > > > Thanks,
> > > > Vijai Kumar K
> > > >
> > > > >
> > > > > Thanks,
> > > > > Vijai Kumar K
> > > > >
> > > > > >
> > > > > > Henning
> > > > > >
> > > > > > > +
> > > > > > > +```
> > > > > > > +part / --source rootfs --ondisk sda --fstype ext4 --label
> > > > > > > +platform
> > > > > > > --align 1024 --use-uuid +```
> > > > > > > diff --git a/scripts/lib/wic/help.py
> > > > > > > b/scripts/lib/wic/help.py index
> > > > > > > 4ff7470..73e3380 100644
> > > > > > > --- a/scripts/lib/wic/help.py
> > > > > > > +++ b/scripts/lib/wic/help.py
> > > > > > > @@ -940,6 +940,12 @@ DESCRIPTION
> > > > > > > quotes. If not specified, the
> > > > > > > default string is "defaults".
> > > > > > >
> > > > > > > + --fspassno: Specifies the order in which
> > > > > > > filesystem
> > > > > > > + checks
> > > > > > > are done
> > > > > > > + at boot time by fsck. See fs_passno
> > > > > > > + parameter
> > > > > > > of
> > > > > > > + fstab(5). This parameter will be
> > > > > > > copied
> > > > > > > + into
> > > > > > > the
> > > > > > > + /etc/fstab file of the installed
> > > > > > > system. If not
> > > > > > > + specified the default value of "0"
> > > > > > > will be used. +
> > > > > > > --label label: Specifies the label to give to
> > > > > > > the filesystem to be made on the partition. If the given
> > > > > > > label is already in use by
> > > > > > > another filesystem, diff --git
> > > > > > > a/scripts/lib/wic/ksparser.py
> > > > > > > b/scripts/lib/wic/ksparser.py index 0df9eb0..a49b7b9
> > > > > > > 100644 --- a/scripts/lib/wic/ksparser.py +++
> > > > > > > b/scripts/lib/wic/ksparser.py @@ -155,6 +155,7 @@ class
> > > > > > > KickStart(): part.add_argument('--change-directory')
> > > > > > > part.add_argument("--extra-space",
> > > > > > > type=sizetype("M")) part.add_argument('--fsoptions',
> > > > > > > dest='fsopts')
> > > > > > > + part.add_argument('--fspassno', dest='fspassno')
> > > > > > > part.add_argument('--fstype', default='vfat',
> > > > > > > choices=('ext2', 'ext3',
> > > > > > > 'ext4', 'btrfs', 'squashfs', 'vfat',
> > > > > > > 'msdos', 'erofs', diff --git
> > > > > > > a/scripts/lib/wic/partition.py
> > > > > > > b/scripts/lib/wic/partition.py index 09e491d..e50871b
> > > > > > > 100644 --- a/scripts/lib/wic/partition.py +++
> > > > > > > b/scripts/lib/wic/partition.py @@ -33,6 +33,7 @@ class
> > > > > > > Partition(): self.include_path = args.include_path
> > > > > > > self.change_directory = args.change_directory
> > > > > > > self.fsopts = args.fsopts
> > > > > > > + self.fspassno = args.fspassno
> > > > > > > self.fstype = args.fstype
> > > > > > > self.label = args.label
> > > > > > > self.use_label = args.use_label diff --git
> > > > > > > a/scripts/lib/wic/plugins/imager/direct.py
> > > > > > > b/scripts/lib/wic/plugins/imager/direct.py index
> > > > > > > 4d0b836..c44159b
> > > > > > > 100644 --- a/scripts/lib/wic/plugins/imager/direct.py
> > > > > > > +++ b/scripts/lib/wic/plugins/imager/direct.py
> > > > > > > @@ -117,7 +117,7 @@ class DirectPlugin(ImagerPlugin):
> > > > > > > updated = False
> > > > > > > for part in self.parts:
> > > > > > > if not part.realnum or not part.mountpoint \
> > > > > > > - or part.mountpoint == "/" or not
> > > > > > > part.mountpoint.startswith('/'):
> > > > > > > + or not part.mountpoint.startswith('/'):
> > > > > > > continue
> > > > > > >
> > > > > > > if part.use_uuid:
> > > > > > > @@ -138,8 +138,9 @@ class DirectPlugin(ImagerPlugin):
> > > > > > > device_name = "/dev/%s%s%d" % (part.disk,
> > > > > > > prefix,
> > > > > > > part.realnum)
> > > > > > > opts = part.fsopts if part.fsopts else
> > > > > > > "defaults"
> > > > > > > + passno = part.fspassno if part.fspassno else
> > > > > > > "0" line = "\t".join([device_name, part.mountpoint,
> > > > > > > part.fstype,
> > > > > > > - opts, "0", "0"]) + "\n"
> > > > > > > + opts, "0", passno]) + "\n"
> > > > > > >
> > > > > > > fstab_lines.append(line)
> > > > > > > updated = True
> > > >
> > > > --
> > > > You received this message because you are subscribed to the
> > > > Google Groups "isar-users" group.
> > > > To unsubscribe from this group and stop receiving emails from
> > > > it, send an email to isar-users+unsubscribe@googlegroups.com.
> > > > To view this discussion on the web visit
> > > >
> > > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgrou
> > >
> > > > ps.g
> > > > oogle.com%2Fd%2Fmsgid%2Fisar-
> > > >
> > > users%2F8babb01d7eae44e0b9faecad59cdc1d4%2540mentor.com&dat
> > > a=
> > > >
> > > 05%7C01%7Cfelix.moessbauer%40siemens.com%7C237b0db9bb724580ca540
> > > 8
> > > >
> > >
> > da5024a3ad%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C6379104
> > > >
> > > 15537105828%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQ
> > > Ijo
> > > >
> > > iV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sda
> > > ta
> > > >
> > > =%2F%2F1Fxy%2B%2BEgDI4RfUEZ38qy1Q82esArdBuRftDuE1t6s%3D&re
> > > se
> > > > rved=0.
^ permalink raw reply [flat|nested] 22+ messages in thread
* RE: [PATCH v2 2/2] wic: Update to the latest revision
2022-09-26 4:53 ` Kanagarajan, Vijaikumar
@ 2022-09-26 5:57 ` Moessbauer, Felix
2022-09-26 7:46 ` Henning Schild
0 siblings, 1 reply; 22+ messages in thread
From: Moessbauer, Felix @ 2022-09-26 5:57 UTC (permalink / raw)
To: Kanagarajan, Vijaikumar, Schild, Henning, jan.kiszka; +Cc: isar-users
> -----Original Message-----
> From: Kanagarajan, Vijaikumar <Vijaikumar_Kanagarajan@mentor.com>
> Sent: Monday, September 26, 2022 12:53 PM
> To: Moessbauer, Felix (T CED SES-DE) <felix.moessbauer@siemens.com>; Schild,
> Henning (T CED SES-DE) <henning.schild@siemens.com>
> Cc: Kiszka, Jan (T CED) <jan.kiszka@siemens.com>; isar-
> users@googlegroups.com
> Subject: RE: [PATCH v2 2/2] wic: Update to the latest revision
>
>
>
> > -----Original Message-----
> > From: Moessbauer, Felix (T CED SES-DE) <felix.moessbauer@siemens.com>
> > Sent: 26 September 2022 07:11
> > To: Kanagarajan, Vijaikumar <Vijaikumar_Kanagarajan@mentor.com>;
> > Schild, Henning (T CED SES-DE) <henning.schild@siemens.com>
> > Cc: Kiszka, Jan (T CED) <jan.kiszka@siemens.com>; isar-
> > users@googlegroups.com
> > Subject: RE: [PATCH v2 2/2] wic: Update to the latest revision
> >
> > Hi Vijai & Henning,
> >
> > Are there any news on this?
> > I just discovered that still many (if not all) of our images are
> > affected by this bug.
>
> Hi Felix,
>
> Unfortunately not. If we already have the latest wic, --no-fstab-update[1] might
> help.(untested).
We already have a wic version with this option, but it doesn't look right to add it to the root partition.
Instead this option can be used for partitions that should not show up in the fstab at all.
Indeed, the fstab entry has to be set by WIC to set settings like UUID and filesystem.
But this requires that we either remove the entry set by ISAR in image_configure_fstab, or we remove the entry after generating the rootfs, but before WIC.
Opinions? @Kiszka, Jan (T CED)
Felix
>
> Thanks,
> Vijai Kumar K
>
> >
> > While it is not serios, having error messages in the journal is still pretty bad:
> > systemd-fstab-generator[1491]: Failed to create unit file
> > /run/systemd/generator/-.mount, as it already exists. Duplicate entry
> > in /etc/fstab?
> >
> > Felix
> >
> > > -----Original Message-----
> > > From: isar-users@googlegroups.com <isar-users@googlegroups.com> On
> > > Behalf Of Kanagarajan, Vijaikumar
> > > Sent: Friday, June 17, 2022 1:46 PM
> > > To: isar-users@googlegroups.com
> > > Cc: Kiszka, Jan (T CED) <jan.kiszka@siemens.com>; Schild, Henning (T
> > > CED SES-
> > > DE) <henning.schild@siemens.com>
> > > Subject: RE: [PATCH v2 2/2] wic: Update to the latest revision
> > >
> > >
> > >
> > > > -----Original Message-----
> > > > From: Kanagarajan, Vijaikumar
> > > > Sent: 13 June 2022 14:48
> > > > To: 'Henning Schild' <henning.schild@siemens.com>
> > > > Cc: isar-users@googlegroups.com; jan.kiszka@siemens.com
> > > > Subject: RE: [PATCH v2 2/2] wic: Update to the latest revision
> > > >
> > > >
> > > >
> > > > > -----Original Message-----
> > > > > From: Henning Schild <henning.schild@siemens.com>
> > > > > Sent: 13 June 2022 14:32
> > > > > To: Kanagarajan, Vijaikumar <Vijaikumar_Kanagarajan@mentor.com>
> > > > > Cc: isar-users@googlegroups.com; jan.kiszka@siemens.com
> > > > > Subject: Re: [PATCH v2 2/2] wic: Update to the latest revision
> > > > >
> > > > > Am Mon, 13 Jun 2022 13:24:01 +0530 schrieb Vijai Kumar K
> > > > > <Vijaikumar_Kanagarajan@mentor.com>:
> > > > >
> > > > > > Update to the latest wic from OE-core.
> > > > > >
> > > > > > OE-core Revision: 06ed2aa93fc25a681e3a00ee120d9395b04845da
> > > > > >
> > > > > > Signed-off-by: Vijai Kumar K
> > <Vijaikumar_Kanagarajan@mentor.com>
> > > > > > ---
> > > > > > RECIPE-API-CHANGELOG.md | 19 +++++++++++++++++++
> > > > > > scripts/lib/wic/help.py | 6 ++++++
> > > > > > scripts/lib/wic/ksparser.py | 1 +
> > > > > > scripts/lib/wic/partition.py | 1 +
> > > > > > scripts/lib/wic/plugins/imager/direct.py | 5 +++--
> > > > > > 5 files changed, 30 insertions(+), 2 deletions(-)
> > > > > >
> > > > > > diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-
> > CHANGELOG.md
> > > > > index
> > > > > > 2998824..436b204 100644
> > > > > > --- a/RECIPE-API-CHANGELOG.md
> > > > > > +++ b/RECIPE-API-CHANGELOG.md
> > > > > > @@ -406,3 +406,22 @@ groups are now created after all packages
> > > > > > are installed.
> > > > > > Changes in next
> > > > > > ---------------
> > > > > > +
> > > > > > +### Wic adds / mountpoint to fstab
> > > > > > +
> > > > > > +In the older version of wic, any mount point named / is
> > > > > > +skipped from
> > > > > > adding +into the fstab entry.
> > > > > > +
> > > > > > +With the latest wic, this is not the case. / mount point, if
> > > > > > +any, is
> > > > > > added +to /etc/fstab for automount.
> > > > > > +
> > > > > > +Any wks file which assumed that / would be skipped from
> > > > > > +/etc/fstab
> > > > > > should +now be corrected. Otherwise, it might result in
> > > > > > multiple / entries in /etc/fstab. +One by the
> > > > > > image_configure_fstab function from image class and one by
> > > > > > wic. +
> > > > > > +Below is an example wks entry that might cause an issue.
> > > > > > +Replace "part /" with "part" to avoid adding duplicate entry for /.
> > > > >
> > > > > I am not sure multiple entries would be an issue, i think the
> > > > > last one will simply always win. It is good to have a note about
> > > > > that change, but i doubt people really have to do anything.
> > > > > And if anything has to be done, isar should drop the first one centrally.
> > > > >
> > > > > The fstab entry for root is purely for adding some weird mount
> > > > > options, usually root is not mounted with fstab but with cmdline
> > > > > and initrd. And not having / in fstab just makes that rootfs
> > > > > more generic ... it can become part of nfs and next guy takes it
> > > > > as btrfs while next guy
> > > > takes it as ext4.
> > > > >
> > > > > Note is good, advise is not
> > > >
> > > > Noted. Will change the text accordingly in V3.
> > >
> > > Ok we cannot drop / mountpoint name(though optional). We would be
> > lose
> > > the ability to find out UUID of root using rootdev api.
> > > This is used by some plugins. So the better way would be to use the
> > > --no-fstab- update. Seems like we can set them per partition now.
> > >
> > > Thanks,
> > > Vijai Kumar K
> > >
> > > >
> > > > Thanks,
> > > > Vijai Kumar K
> > > >
> > > > >
> > > > > Henning
> > > > >
> > > > > > +
> > > > > > +```
> > > > > > +part / --source rootfs --ondisk sda --fstype ext4 --label
> > > > > > +platform
> > > > > > --align 1024 --use-uuid +```
> > > > > > diff --git a/scripts/lib/wic/help.py b/scripts/lib/wic/help.py
> > > > > > index
> > > > > > 4ff7470..73e3380 100644
> > > > > > --- a/scripts/lib/wic/help.py
> > > > > > +++ b/scripts/lib/wic/help.py
> > > > > > @@ -940,6 +940,12 @@ DESCRIPTION
> > > > > > quotes. If not specified, the default
> > > > > > string is "defaults".
> > > > > >
> > > > > > + --fspassno: Specifies the order in which filesystem
> > > > > > + checks
> > > > > > are done
> > > > > > + at boot time by fsck. See fs_passno
> > > > > > + parameter
> > > > > > of
> > > > > > + fstab(5). This parameter will be copied
> > > > > > + into
> > > > > > the
> > > > > > + /etc/fstab file of the installed system. If not
> > > > > > + specified the default value of "0" will be used.
> > > > > > +
> > > > > > --label label: Specifies the label to give to the filesystem
> > > > > > to be made on the partition. If the given
> > > > > > label is already in use by another
> > > > > > filesystem, diff --git a/scripts/lib/wic/ksparser.py
> > > > > > b/scripts/lib/wic/ksparser.py index 0df9eb0..a49b7b9 100644
> > > > > > --- a/scripts/lib/wic/ksparser.py
> > > > > > +++ b/scripts/lib/wic/ksparser.py
> > > > > > @@ -155,6 +155,7 @@ class KickStart():
> > > > > > part.add_argument('--change-directory')
> > > > > > part.add_argument("--extra-space", type=sizetype("M"))
> > > > > > part.add_argument('--fsoptions', dest='fsopts')
> > > > > > + part.add_argument('--fspassno', dest='fspassno')
> > > > > > part.add_argument('--fstype', default='vfat',
> > > > > > choices=('ext2', 'ext3', 'ext4', 'btrfs',
> > > > > > 'squashfs', 'vfat',
> > > > > > 'msdos', 'erofs', diff --git a/scripts/lib/wic/partition.py
> > > > > > b/scripts/lib/wic/partition.py index 09e491d..e50871b 100644
> > > > > > --- a/scripts/lib/wic/partition.py
> > > > > > +++ b/scripts/lib/wic/partition.py
> > > > > > @@ -33,6 +33,7 @@ class Partition():
> > > > > > self.include_path = args.include_path
> > > > > > self.change_directory = args.change_directory
> > > > > > self.fsopts = args.fsopts
> > > > > > + self.fspassno = args.fspassno
> > > > > > self.fstype = args.fstype
> > > > > > self.label = args.label
> > > > > > self.use_label = args.use_label diff --git
> > > > > > a/scripts/lib/wic/plugins/imager/direct.py
> > > > > > b/scripts/lib/wic/plugins/imager/direct.py index
> > > > > > 4d0b836..c44159b
> > > > > > 100644 --- a/scripts/lib/wic/plugins/imager/direct.py
> > > > > > +++ b/scripts/lib/wic/plugins/imager/direct.py
> > > > > > @@ -117,7 +117,7 @@ class DirectPlugin(ImagerPlugin):
> > > > > > updated = False
> > > > > > for part in self.parts:
> > > > > > if not part.realnum or not part.mountpoint \
> > > > > > - or part.mountpoint == "/" or not
> > > > > > part.mountpoint.startswith('/'):
> > > > > > + or not part.mountpoint.startswith('/'):
> > > > > > continue
> > > > > >
> > > > > > if part.use_uuid:
> > > > > > @@ -138,8 +138,9 @@ class DirectPlugin(ImagerPlugin):
> > > > > > device_name = "/dev/%s%s%d" % (part.disk,
> > > > > > prefix,
> > > > > > part.realnum)
> > > > > > opts = part.fsopts if part.fsopts else "defaults"
> > > > > > + passno = part.fspassno if part.fspassno else "0"
> > > > > > line = "\t".join([device_name, part.mountpoint,
> > > > > > part.fstype,
> > > > > > - opts, "0", "0"]) + "\n"
> > > > > > + opts, "0", passno]) + "\n"
> > > > > >
> > > > > > fstab_lines.append(line)
> > > > > > updated = True
> > >
> > > --
> > > You received this message because you are subscribed to the Google
> > > Groups "isar-users" group.
> > > To unsubscribe from this group and stop receiving emails from it,
> > > send an email to isar-users+unsubscribe@googlegroups.com.
> > > To view this discussion on the web visit
> > >
> > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgrou
> > > ps.g
> > > oogle.com%2Fd%2Fmsgid%2Fisar-
> > >
> > users%2F8babb01d7eae44e0b9faecad59cdc1d4%2540mentor.com&dat
> > a=
> > >
> > 05%7C01%7Cfelix.moessbauer%40siemens.com%7C237b0db9bb724580ca540
> > 8
> > >
> >
> da5024a3ad%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C6379104
> > >
> > 15537105828%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQ
> > Ijo
> > >
> > iV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sda
> > ta
> > >
> > =%2F%2F1Fxy%2B%2BEgDI4RfUEZ38qy1Q82esArdBuRftDuE1t6s%3D&re
> > se
> > > rved=0.
^ permalink raw reply [flat|nested] 22+ messages in thread
* RE: [PATCH v2 2/2] wic: Update to the latest revision
2022-09-26 1:40 ` Moessbauer, Felix
@ 2022-09-26 4:53 ` Kanagarajan, Vijaikumar
2022-09-26 5:57 ` Moessbauer, Felix
0 siblings, 1 reply; 22+ messages in thread
From: Kanagarajan, Vijaikumar @ 2022-09-26 4:53 UTC (permalink / raw)
To: Moessbauer, Felix (T CED SES-DE), Schild, Henning (T CED SES-DE)
Cc: Kiszka, Jan (T CED), isar-users
> -----Original Message-----
> From: Moessbauer, Felix (T CED SES-DE) <felix.moessbauer@siemens.com>
> Sent: 26 September 2022 07:11
> To: Kanagarajan, Vijaikumar <Vijaikumar_Kanagarajan@mentor.com>;
> Schild, Henning (T CED SES-DE) <henning.schild@siemens.com>
> Cc: Kiszka, Jan (T CED) <jan.kiszka@siemens.com>; isar-
> users@googlegroups.com
> Subject: RE: [PATCH v2 2/2] wic: Update to the latest revision
>
> Hi Vijai & Henning,
>
> Are there any news on this?
> I just discovered that still many (if not all) of our images are affected by this
> bug.
Hi Felix,
Unfortunately not. If we already have the latest wic, --no-fstab-update[1] might help.(untested).
[1] https://github.com/openembedded/openembedded-core/commit/ab4c95af8ecd15dc136194ab761afae756db5803
Thanks,
Vijai Kumar K
>
> While it is not serios, having error messages in the journal is still pretty bad:
> systemd-fstab-generator[1491]: Failed to create unit file
> /run/systemd/generator/-.mount, as it already exists. Duplicate entry in
> /etc/fstab?
>
> Felix
>
> > -----Original Message-----
> > From: isar-users@googlegroups.com <isar-users@googlegroups.com> On
> > Behalf Of Kanagarajan, Vijaikumar
> > Sent: Friday, June 17, 2022 1:46 PM
> > To: isar-users@googlegroups.com
> > Cc: Kiszka, Jan (T CED) <jan.kiszka@siemens.com>; Schild, Henning (T
> > CED SES-
> > DE) <henning.schild@siemens.com>
> > Subject: RE: [PATCH v2 2/2] wic: Update to the latest revision
> >
> >
> >
> > > -----Original Message-----
> > > From: Kanagarajan, Vijaikumar
> > > Sent: 13 June 2022 14:48
> > > To: 'Henning Schild' <henning.schild@siemens.com>
> > > Cc: isar-users@googlegroups.com; jan.kiszka@siemens.com
> > > Subject: RE: [PATCH v2 2/2] wic: Update to the latest revision
> > >
> > >
> > >
> > > > -----Original Message-----
> > > > From: Henning Schild <henning.schild@siemens.com>
> > > > Sent: 13 June 2022 14:32
> > > > To: Kanagarajan, Vijaikumar <Vijaikumar_Kanagarajan@mentor.com>
> > > > Cc: isar-users@googlegroups.com; jan.kiszka@siemens.com
> > > > Subject: Re: [PATCH v2 2/2] wic: Update to the latest revision
> > > >
> > > > Am Mon, 13 Jun 2022 13:24:01 +0530 schrieb Vijai Kumar K
> > > > <Vijaikumar_Kanagarajan@mentor.com>:
> > > >
> > > > > Update to the latest wic from OE-core.
> > > > >
> > > > > OE-core Revision: 06ed2aa93fc25a681e3a00ee120d9395b04845da
> > > > >
> > > > > Signed-off-by: Vijai Kumar K
> <Vijaikumar_Kanagarajan@mentor.com>
> > > > > ---
> > > > > RECIPE-API-CHANGELOG.md | 19 +++++++++++++++++++
> > > > > scripts/lib/wic/help.py | 6 ++++++
> > > > > scripts/lib/wic/ksparser.py | 1 +
> > > > > scripts/lib/wic/partition.py | 1 +
> > > > > scripts/lib/wic/plugins/imager/direct.py | 5 +++--
> > > > > 5 files changed, 30 insertions(+), 2 deletions(-)
> > > > >
> > > > > diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-
> CHANGELOG.md
> > > > index
> > > > > 2998824..436b204 100644
> > > > > --- a/RECIPE-API-CHANGELOG.md
> > > > > +++ b/RECIPE-API-CHANGELOG.md
> > > > > @@ -406,3 +406,22 @@ groups are now created after all packages
> > > > > are installed.
> > > > > Changes in next
> > > > > ---------------
> > > > > +
> > > > > +### Wic adds / mountpoint to fstab
> > > > > +
> > > > > +In the older version of wic, any mount point named / is skipped
> > > > > +from
> > > > > adding +into the fstab entry.
> > > > > +
> > > > > +With the latest wic, this is not the case. / mount point, if
> > > > > +any, is
> > > > > added +to /etc/fstab for automount.
> > > > > +
> > > > > +Any wks file which assumed that / would be skipped from
> > > > > +/etc/fstab
> > > > > should +now be corrected. Otherwise, it might result in multiple
> > > > > / entries in /etc/fstab. +One by the image_configure_fstab
> > > > > function from image class and one by wic. +
> > > > > +Below is an example wks entry that might cause an issue.
> > > > > +Replace "part /" with "part" to avoid adding duplicate entry for /.
> > > >
> > > > I am not sure multiple entries would be an issue, i think the last
> > > > one will simply always win. It is good to have a note about that
> > > > change, but i doubt people really have to do anything.
> > > > And if anything has to be done, isar should drop the first one centrally.
> > > >
> > > > The fstab entry for root is purely for adding some weird mount
> > > > options, usually root is not mounted with fstab but with cmdline
> > > > and initrd. And not having / in fstab just makes that rootfs more
> > > > generic ... it can become part of nfs and next guy takes it as
> > > > btrfs while next guy
> > > takes it as ext4.
> > > >
> > > > Note is good, advise is not
> > >
> > > Noted. Will change the text accordingly in V3.
> >
> > Ok we cannot drop / mountpoint name(though optional). We would be
> lose
> > the ability to find out UUID of root using rootdev api.
> > This is used by some plugins. So the better way would be to use the
> > --no-fstab- update. Seems like we can set them per partition now.
> >
> > Thanks,
> > Vijai Kumar K
> >
> > >
> > > Thanks,
> > > Vijai Kumar K
> > >
> > > >
> > > > Henning
> > > >
> > > > > +
> > > > > +```
> > > > > +part / --source rootfs --ondisk sda --fstype ext4 --label
> > > > > +platform
> > > > > --align 1024 --use-uuid +```
> > > > > diff --git a/scripts/lib/wic/help.py b/scripts/lib/wic/help.py
> > > > > index
> > > > > 4ff7470..73e3380 100644
> > > > > --- a/scripts/lib/wic/help.py
> > > > > +++ b/scripts/lib/wic/help.py
> > > > > @@ -940,6 +940,12 @@ DESCRIPTION
> > > > > quotes. If not specified, the default
> > > > > string is "defaults".
> > > > >
> > > > > + --fspassno: Specifies the order in which filesystem
> > > > > + checks
> > > > > are done
> > > > > + at boot time by fsck. See fs_passno
> > > > > + parameter
> > > > > of
> > > > > + fstab(5). This parameter will be copied
> > > > > + into
> > > > > the
> > > > > + /etc/fstab file of the installed system. If not
> > > > > + specified the default value of "0" will be used.
> > > > > +
> > > > > --label label: Specifies the label to give to the filesystem
> > > > > to be made on the partition. If the given
> > > > > label is already in use by another
> > > > > filesystem, diff --git a/scripts/lib/wic/ksparser.py
> > > > > b/scripts/lib/wic/ksparser.py index 0df9eb0..a49b7b9 100644
> > > > > --- a/scripts/lib/wic/ksparser.py
> > > > > +++ b/scripts/lib/wic/ksparser.py
> > > > > @@ -155,6 +155,7 @@ class KickStart():
> > > > > part.add_argument('--change-directory')
> > > > > part.add_argument("--extra-space", type=sizetype("M"))
> > > > > part.add_argument('--fsoptions', dest='fsopts')
> > > > > + part.add_argument('--fspassno', dest='fspassno')
> > > > > part.add_argument('--fstype', default='vfat',
> > > > > choices=('ext2', 'ext3', 'ext4', 'btrfs',
> > > > > 'squashfs', 'vfat', 'msdos',
> > > > > 'erofs', diff --git a/scripts/lib/wic/partition.py
> > > > > b/scripts/lib/wic/partition.py index 09e491d..e50871b 100644
> > > > > --- a/scripts/lib/wic/partition.py
> > > > > +++ b/scripts/lib/wic/partition.py
> > > > > @@ -33,6 +33,7 @@ class Partition():
> > > > > self.include_path = args.include_path
> > > > > self.change_directory = args.change_directory
> > > > > self.fsopts = args.fsopts
> > > > > + self.fspassno = args.fspassno
> > > > > self.fstype = args.fstype
> > > > > self.label = args.label
> > > > > self.use_label = args.use_label diff --git
> > > > > a/scripts/lib/wic/plugins/imager/direct.py
> > > > > b/scripts/lib/wic/plugins/imager/direct.py index
> > > > > 4d0b836..c44159b
> > > > > 100644 --- a/scripts/lib/wic/plugins/imager/direct.py
> > > > > +++ b/scripts/lib/wic/plugins/imager/direct.py
> > > > > @@ -117,7 +117,7 @@ class DirectPlugin(ImagerPlugin):
> > > > > updated = False
> > > > > for part in self.parts:
> > > > > if not part.realnum or not part.mountpoint \
> > > > > - or part.mountpoint == "/" or not
> > > > > part.mountpoint.startswith('/'):
> > > > > + or not part.mountpoint.startswith('/'):
> > > > > continue
> > > > >
> > > > > if part.use_uuid:
> > > > > @@ -138,8 +138,9 @@ class DirectPlugin(ImagerPlugin):
> > > > > device_name = "/dev/%s%s%d" % (part.disk,
> > > > > prefix,
> > > > > part.realnum)
> > > > > opts = part.fsopts if part.fsopts else "defaults"
> > > > > + passno = part.fspassno if part.fspassno else "0"
> > > > > line = "\t".join([device_name, part.mountpoint,
> > > > > part.fstype,
> > > > > - opts, "0", "0"]) + "\n"
> > > > > + opts, "0", passno]) + "\n"
> > > > >
> > > > > fstab_lines.append(line)
> > > > > updated = True
> >
> > --
> > You received this message because you are subscribed to the Google
> > Groups "isar-users" group.
> > To unsubscribe from this group and stop receiving emails from it, send
> > an email to isar-users+unsubscribe@googlegroups.com.
> > To view this discussion on the web visit
> >
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgrou
> > ps.g
> > oogle.com%2Fd%2Fmsgid%2Fisar-
> >
> users%2F8babb01d7eae44e0b9faecad59cdc1d4%2540mentor.com&dat
> a=
> >
> 05%7C01%7Cfelix.moessbauer%40siemens.com%7C237b0db9bb724580ca540
> 8
> >
> da5024a3ad%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C6379104
> >
> 15537105828%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQ
> Ijo
> >
> iV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sda
> ta
> >
> =%2F%2F1Fxy%2B%2BEgDI4RfUEZ38qy1Q82esArdBuRftDuE1t6s%3D&re
> se
> > rved=0.
^ permalink raw reply [flat|nested] 22+ messages in thread
* RE: [PATCH v2 2/2] wic: Update to the latest revision
2022-06-17 5:45 ` Kanagarajan, Vijaikumar
@ 2022-09-26 1:40 ` Moessbauer, Felix
2022-09-26 4:53 ` Kanagarajan, Vijaikumar
0 siblings, 1 reply; 22+ messages in thread
From: Moessbauer, Felix @ 2022-09-26 1:40 UTC (permalink / raw)
To: Kanagarajan, Vijaikumar, Schild, Henning; +Cc: jan.kiszka, isar-users
Hi Vijai & Henning,
Are there any news on this?
I just discovered that still many (if not all) of our images are affected by this bug.
While it is not serios, having error messages in the journal is still pretty bad:
systemd-fstab-generator[1491]: Failed to create unit file /run/systemd/generator/-.mount, as it already exists. Duplicate entry in /etc/fstab?
Felix
> -----Original Message-----
> From: isar-users@googlegroups.com <isar-users@googlegroups.com> On
> Behalf Of Kanagarajan, Vijaikumar
> Sent: Friday, June 17, 2022 1:46 PM
> To: isar-users@googlegroups.com
> Cc: Kiszka, Jan (T CED) <jan.kiszka@siemens.com>; Schild, Henning (T CED SES-
> DE) <henning.schild@siemens.com>
> Subject: RE: [PATCH v2 2/2] wic: Update to the latest revision
>
>
>
> > -----Original Message-----
> > From: Kanagarajan, Vijaikumar
> > Sent: 13 June 2022 14:48
> > To: 'Henning Schild' <henning.schild@siemens.com>
> > Cc: isar-users@googlegroups.com; jan.kiszka@siemens.com
> > Subject: RE: [PATCH v2 2/2] wic: Update to the latest revision
> >
> >
> >
> > > -----Original Message-----
> > > From: Henning Schild <henning.schild@siemens.com>
> > > Sent: 13 June 2022 14:32
> > > To: Kanagarajan, Vijaikumar <Vijaikumar_Kanagarajan@mentor.com>
> > > Cc: isar-users@googlegroups.com; jan.kiszka@siemens.com
> > > Subject: Re: [PATCH v2 2/2] wic: Update to the latest revision
> > >
> > > Am Mon, 13 Jun 2022 13:24:01 +0530
> > > schrieb Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com>:
> > >
> > > > Update to the latest wic from OE-core.
> > > >
> > > > OE-core Revision: 06ed2aa93fc25a681e3a00ee120d9395b04845da
> > > >
> > > > Signed-off-by: Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com>
> > > > ---
> > > > RECIPE-API-CHANGELOG.md | 19 +++++++++++++++++++
> > > > scripts/lib/wic/help.py | 6 ++++++
> > > > scripts/lib/wic/ksparser.py | 1 +
> > > > scripts/lib/wic/partition.py | 1 +
> > > > scripts/lib/wic/plugins/imager/direct.py | 5 +++--
> > > > 5 files changed, 30 insertions(+), 2 deletions(-)
> > > >
> > > > diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md
> > > index
> > > > 2998824..436b204 100644
> > > > --- a/RECIPE-API-CHANGELOG.md
> > > > +++ b/RECIPE-API-CHANGELOG.md
> > > > @@ -406,3 +406,22 @@ groups are now created after all packages are
> > > > installed.
> > > > Changes in next
> > > > ---------------
> > > > +
> > > > +### Wic adds / mountpoint to fstab
> > > > +
> > > > +In the older version of wic, any mount point named / is skipped
> > > > +from
> > > > adding +into the fstab entry.
> > > > +
> > > > +With the latest wic, this is not the case. / mount point, if any,
> > > > +is
> > > > added +to /etc/fstab for automount.
> > > > +
> > > > +Any wks file which assumed that / would be skipped from
> > > > +/etc/fstab
> > > > should +now be corrected. Otherwise, it might result in multiple /
> > > > entries in /etc/fstab. +One by the image_configure_fstab function
> > > > from image class and one by wic. +
> > > > +Below is an example wks entry that might cause an issue.
> > > > +Replace "part /" with "part" to avoid adding duplicate entry for /.
> > >
> > > I am not sure multiple entries would be an issue, i think the last
> > > one will simply always win. It is good to have a note about that
> > > change, but i doubt people really have to do anything.
> > > And if anything has to be done, isar should drop the first one centrally.
> > >
> > > The fstab entry for root is purely for adding some weird mount
> > > options, usually root is not mounted with fstab but with cmdline and
> > > initrd. And not having / in fstab just makes that rootfs more
> > > generic ... it can become part of nfs and next guy takes it as btrfs
> > > while next guy
> > takes it as ext4.
> > >
> > > Note is good, advise is not
> >
> > Noted. Will change the text accordingly in V3.
>
> Ok we cannot drop / mountpoint name(though optional). We would be lose the
> ability to find out UUID of root using rootdev api.
> This is used by some plugins. So the better way would be to use the --no-fstab-
> update. Seems like we can set them per partition now.
>
> Thanks,
> Vijai Kumar K
>
> >
> > Thanks,
> > Vijai Kumar K
> >
> > >
> > > Henning
> > >
> > > > +
> > > > +```
> > > > +part / --source rootfs --ondisk sda --fstype ext4 --label
> > > > +platform
> > > > --align 1024 --use-uuid +```
> > > > diff --git a/scripts/lib/wic/help.py b/scripts/lib/wic/help.py
> > > > index
> > > > 4ff7470..73e3380 100644
> > > > --- a/scripts/lib/wic/help.py
> > > > +++ b/scripts/lib/wic/help.py
> > > > @@ -940,6 +940,12 @@ DESCRIPTION
> > > > quotes. If not specified, the default
> > > > string is "defaults".
> > > >
> > > > + --fspassno: Specifies the order in which filesystem
> > > > + checks
> > > > are done
> > > > + at boot time by fsck. See fs_passno
> > > > + parameter
> > > > of
> > > > + fstab(5). This parameter will be copied
> > > > + into
> > > > the
> > > > + /etc/fstab file of the installed system. If not
> > > > + specified the default value of "0" will be used.
> > > > +
> > > > --label label: Specifies the label to give to the filesystem
> > > > to be made on the partition. If the given
> > > > label is already in use by another
> > > > filesystem, diff --git a/scripts/lib/wic/ksparser.py
> > > > b/scripts/lib/wic/ksparser.py index 0df9eb0..a49b7b9 100644
> > > > --- a/scripts/lib/wic/ksparser.py
> > > > +++ b/scripts/lib/wic/ksparser.py
> > > > @@ -155,6 +155,7 @@ class KickStart():
> > > > part.add_argument('--change-directory')
> > > > part.add_argument("--extra-space", type=sizetype("M"))
> > > > part.add_argument('--fsoptions', dest='fsopts')
> > > > + part.add_argument('--fspassno', dest='fspassno')
> > > > part.add_argument('--fstype', default='vfat',
> > > > choices=('ext2', 'ext3', 'ext4', 'btrfs',
> > > > 'squashfs', 'vfat', 'msdos',
> > > > 'erofs', diff --git a/scripts/lib/wic/partition.py
> > > > b/scripts/lib/wic/partition.py index 09e491d..e50871b 100644
> > > > --- a/scripts/lib/wic/partition.py
> > > > +++ b/scripts/lib/wic/partition.py
> > > > @@ -33,6 +33,7 @@ class Partition():
> > > > self.include_path = args.include_path
> > > > self.change_directory = args.change_directory
> > > > self.fsopts = args.fsopts
> > > > + self.fspassno = args.fspassno
> > > > self.fstype = args.fstype
> > > > self.label = args.label
> > > > self.use_label = args.use_label diff --git
> > > > a/scripts/lib/wic/plugins/imager/direct.py
> > > > b/scripts/lib/wic/plugins/imager/direct.py index 4d0b836..c44159b
> > > > 100644 --- a/scripts/lib/wic/plugins/imager/direct.py
> > > > +++ b/scripts/lib/wic/plugins/imager/direct.py
> > > > @@ -117,7 +117,7 @@ class DirectPlugin(ImagerPlugin):
> > > > updated = False
> > > > for part in self.parts:
> > > > if not part.realnum or not part.mountpoint \
> > > > - or part.mountpoint == "/" or not
> > > > part.mountpoint.startswith('/'):
> > > > + or not part.mountpoint.startswith('/'):
> > > > continue
> > > >
> > > > if part.use_uuid:
> > > > @@ -138,8 +138,9 @@ class DirectPlugin(ImagerPlugin):
> > > > device_name = "/dev/%s%s%d" % (part.disk, prefix,
> > > > part.realnum)
> > > > opts = part.fsopts if part.fsopts else "defaults"
> > > > + passno = part.fspassno if part.fspassno else "0"
> > > > line = "\t".join([device_name, part.mountpoint,
> > > > part.fstype,
> > > > - opts, "0", "0"]) + "\n"
> > > > + opts, "0", passno]) + "\n"
> > > >
> > > > fstab_lines.append(line)
> > > > updated = True
>
> --
> You received this message because you are subscribed to the Google Groups
> "isar-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email
> to isar-users+unsubscribe@googlegroups.com.
> To view this discussion on the web visit
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.g
> oogle.com%2Fd%2Fmsgid%2Fisar-
> users%2F8babb01d7eae44e0b9faecad59cdc1d4%2540mentor.com&data=
> 05%7C01%7Cfelix.moessbauer%40siemens.com%7C237b0db9bb724580ca5408
> da5024a3ad%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C6379104
> 15537105828%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjo
> iV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata
> =%2F%2F1Fxy%2B%2BEgDI4RfUEZ38qy1Q82esArdBuRftDuE1t6s%3D&rese
> rved=0.
^ permalink raw reply [flat|nested] 22+ messages in thread
* RE: [PATCH v2 2/2] wic: Update to the latest revision
2022-06-13 9:18 ` Kanagarajan, Vijaikumar
@ 2022-06-17 5:45 ` Kanagarajan, Vijaikumar
2022-09-26 1:40 ` Moessbauer, Felix
0 siblings, 1 reply; 22+ messages in thread
From: Kanagarajan, Vijaikumar @ 2022-06-17 5:45 UTC (permalink / raw)
To: isar-users; +Cc: jan.kiszka, Henning Schild
> -----Original Message-----
> From: Kanagarajan, Vijaikumar
> Sent: 13 June 2022 14:48
> To: 'Henning Schild' <henning.schild@siemens.com>
> Cc: isar-users@googlegroups.com; jan.kiszka@siemens.com
> Subject: RE: [PATCH v2 2/2] wic: Update to the latest revision
>
>
>
> > -----Original Message-----
> > From: Henning Schild <henning.schild@siemens.com>
> > Sent: 13 June 2022 14:32
> > To: Kanagarajan, Vijaikumar <Vijaikumar_Kanagarajan@mentor.com>
> > Cc: isar-users@googlegroups.com; jan.kiszka@siemens.com
> > Subject: Re: [PATCH v2 2/2] wic: Update to the latest revision
> >
> > Am Mon, 13 Jun 2022 13:24:01 +0530
> > schrieb Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com>:
> >
> > > Update to the latest wic from OE-core.
> > >
> > > OE-core Revision: 06ed2aa93fc25a681e3a00ee120d9395b04845da
> > >
> > > Signed-off-by: Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com>
> > > ---
> > > RECIPE-API-CHANGELOG.md | 19 +++++++++++++++++++
> > > scripts/lib/wic/help.py | 6 ++++++
> > > scripts/lib/wic/ksparser.py | 1 +
> > > scripts/lib/wic/partition.py | 1 +
> > > scripts/lib/wic/plugins/imager/direct.py | 5 +++--
> > > 5 files changed, 30 insertions(+), 2 deletions(-)
> > >
> > > diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md
> > index
> > > 2998824..436b204 100644
> > > --- a/RECIPE-API-CHANGELOG.md
> > > +++ b/RECIPE-API-CHANGELOG.md
> > > @@ -406,3 +406,22 @@ groups are now created after all packages are
> > > installed.
> > > Changes in next
> > > ---------------
> > > +
> > > +### Wic adds / mountpoint to fstab
> > > +
> > > +In the older version of wic, any mount point named / is skipped
> > > +from
> > > adding +into the fstab entry.
> > > +
> > > +With the latest wic, this is not the case. / mount point, if any,
> > > +is
> > > added +to /etc/fstab for automount.
> > > +
> > > +Any wks file which assumed that / would be skipped from /etc/fstab
> > > should +now be corrected. Otherwise, it might result in multiple /
> > > entries in /etc/fstab. +One by the image_configure_fstab function
> > > from image class and one by wic. +
> > > +Below is an example wks entry that might cause an issue.
> > > +Replace "part /" with "part" to avoid adding duplicate entry for /.
> >
> > I am not sure multiple entries would be an issue, i think the last one
> > will simply always win. It is good to have a note about that change,
> > but i doubt people really have to do anything.
> > And if anything has to be done, isar should drop the first one centrally.
> >
> > The fstab entry for root is purely for adding some weird mount
> > options, usually root is not mounted with fstab but with cmdline and
> > initrd. And not having / in fstab just makes that rootfs more generic
> > ... it can become part of nfs and next guy takes it as btrfs while next guy
> takes it as ext4.
> >
> > Note is good, advise is not
>
> Noted. Will change the text accordingly in V3.
Ok we cannot drop / mountpoint name(though optional). We would be lose the ability to find out UUID of root using rootdev api.
This is used by some plugins. So the better way would be to use the --no-fstab-update. Seems like we can set them per partition now.
Thanks,
Vijai Kumar K
>
> Thanks,
> Vijai Kumar K
>
> >
> > Henning
> >
> > > +
> > > +```
> > > +part / --source rootfs --ondisk sda --fstype ext4 --label platform
> > > --align 1024 --use-uuid +```
> > > diff --git a/scripts/lib/wic/help.py b/scripts/lib/wic/help.py index
> > > 4ff7470..73e3380 100644
> > > --- a/scripts/lib/wic/help.py
> > > +++ b/scripts/lib/wic/help.py
> > > @@ -940,6 +940,12 @@ DESCRIPTION
> > > quotes. If not specified, the default string
> > > is "defaults".
> > >
> > > + --fspassno: Specifies the order in which filesystem checks
> > > are done
> > > + at boot time by fsck. See fs_passno parameter
> > > of
> > > + fstab(5). This parameter will be copied into
> > > the
> > > + /etc/fstab file of the installed system. If not
> > > + specified the default value of "0" will be used.
> > > +
> > > --label label: Specifies the label to give to the filesystem
> > > to be made on the partition. If the given
> > > label is already in use by another
> > > filesystem, diff --git a/scripts/lib/wic/ksparser.py
> > > b/scripts/lib/wic/ksparser.py index 0df9eb0..a49b7b9 100644
> > > --- a/scripts/lib/wic/ksparser.py
> > > +++ b/scripts/lib/wic/ksparser.py
> > > @@ -155,6 +155,7 @@ class KickStart():
> > > part.add_argument('--change-directory')
> > > part.add_argument("--extra-space", type=sizetype("M"))
> > > part.add_argument('--fsoptions', dest='fsopts')
> > > + part.add_argument('--fspassno', dest='fspassno')
> > > part.add_argument('--fstype', default='vfat',
> > > choices=('ext2', 'ext3', 'ext4', 'btrfs',
> > > 'squashfs', 'vfat', 'msdos',
> > > 'erofs', diff --git a/scripts/lib/wic/partition.py
> > > b/scripts/lib/wic/partition.py index 09e491d..e50871b 100644
> > > --- a/scripts/lib/wic/partition.py
> > > +++ b/scripts/lib/wic/partition.py
> > > @@ -33,6 +33,7 @@ class Partition():
> > > self.include_path = args.include_path
> > > self.change_directory = args.change_directory
> > > self.fsopts = args.fsopts
> > > + self.fspassno = args.fspassno
> > > self.fstype = args.fstype
> > > self.label = args.label
> > > self.use_label = args.use_label diff --git
> > > a/scripts/lib/wic/plugins/imager/direct.py
> > > b/scripts/lib/wic/plugins/imager/direct.py index 4d0b836..c44159b
> > > 100644 --- a/scripts/lib/wic/plugins/imager/direct.py
> > > +++ b/scripts/lib/wic/plugins/imager/direct.py
> > > @@ -117,7 +117,7 @@ class DirectPlugin(ImagerPlugin):
> > > updated = False
> > > for part in self.parts:
> > > if not part.realnum or not part.mountpoint \
> > > - or part.mountpoint == "/" or not
> > > part.mountpoint.startswith('/'):
> > > + or not part.mountpoint.startswith('/'):
> > > continue
> > >
> > > if part.use_uuid:
> > > @@ -138,8 +138,9 @@ class DirectPlugin(ImagerPlugin):
> > > device_name = "/dev/%s%s%d" % (part.disk, prefix,
> > > part.realnum)
> > > opts = part.fsopts if part.fsopts else "defaults"
> > > + passno = part.fspassno if part.fspassno else "0"
> > > line = "\t".join([device_name, part.mountpoint,
> > > part.fstype,
> > > - opts, "0", "0"]) + "\n"
> > > + opts, "0", passno]) + "\n"
> > >
> > > fstab_lines.append(line)
> > > updated = True
^ permalink raw reply [flat|nested] 22+ messages in thread
* RE: [PATCH v2 2/2] wic: Update to the latest revision
2022-06-13 9:02 ` Henning Schild
@ 2022-06-13 9:18 ` Kanagarajan, Vijaikumar
2022-06-17 5:45 ` Kanagarajan, Vijaikumar
0 siblings, 1 reply; 22+ messages in thread
From: Kanagarajan, Vijaikumar @ 2022-06-13 9:18 UTC (permalink / raw)
To: Henning Schild; +Cc: isar-users, jan.kiszka
> -----Original Message-----
> From: Henning Schild <henning.schild@siemens.com>
> Sent: 13 June 2022 14:32
> To: Kanagarajan, Vijaikumar <Vijaikumar_Kanagarajan@mentor.com>
> Cc: isar-users@googlegroups.com; jan.kiszka@siemens.com
> Subject: Re: [PATCH v2 2/2] wic: Update to the latest revision
>
> Am Mon, 13 Jun 2022 13:24:01 +0530
> schrieb Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com>:
>
> > Update to the latest wic from OE-core.
> >
> > OE-core Revision: 06ed2aa93fc25a681e3a00ee120d9395b04845da
> >
> > Signed-off-by: Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com>
> > ---
> > RECIPE-API-CHANGELOG.md | 19 +++++++++++++++++++
> > scripts/lib/wic/help.py | 6 ++++++
> > scripts/lib/wic/ksparser.py | 1 +
> > scripts/lib/wic/partition.py | 1 +
> > scripts/lib/wic/plugins/imager/direct.py | 5 +++--
> > 5 files changed, 30 insertions(+), 2 deletions(-)
> >
> > diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md
> index
> > 2998824..436b204 100644
> > --- a/RECIPE-API-CHANGELOG.md
> > +++ b/RECIPE-API-CHANGELOG.md
> > @@ -406,3 +406,22 @@ groups are now created after all packages are
> > installed.
> > Changes in next
> > ---------------
> > +
> > +### Wic adds / mountpoint to fstab
> > +
> > +In the older version of wic, any mount point named / is skipped from
> > adding +into the fstab entry.
> > +
> > +With the latest wic, this is not the case. / mount point, if any, is
> > added +to /etc/fstab for automount.
> > +
> > +Any wks file which assumed that / would be skipped from /etc/fstab
> > should +now be corrected. Otherwise, it might result in multiple /
> > entries in /etc/fstab. +One by the image_configure_fstab function from
> > image class and one by wic. +
> > +Below is an example wks entry that might cause an issue.
> > +Replace "part /" with "part" to avoid adding duplicate entry for /.
>
> I am not sure multiple entries would be an issue, i think the last one will
> simply always win. It is good to have a note about that change, but i doubt
> people really have to do anything.
> And if anything has to be done, isar should drop the first one centrally.
>
> The fstab entry for root is purely for adding some weird mount options,
> usually root is not mounted with fstab but with cmdline and initrd. And not
> having / in fstab just makes that rootfs more generic ... it can become part of
> nfs and next guy takes it as btrfs while next guy takes it as ext4.
>
> Note is good, advise is not
Noted. Will change the text accordingly in V3.
Thanks,
Vijai Kumar K
>
> Henning
>
> > +
> > +```
> > +part / --source rootfs --ondisk sda --fstype ext4 --label platform
> > --align 1024 --use-uuid +```
> > diff --git a/scripts/lib/wic/help.py b/scripts/lib/wic/help.py index
> > 4ff7470..73e3380 100644
> > --- a/scripts/lib/wic/help.py
> > +++ b/scripts/lib/wic/help.py
> > @@ -940,6 +940,12 @@ DESCRIPTION
> > quotes. If not specified, the default string
> > is "defaults".
> >
> > + --fspassno: Specifies the order in which filesystem checks
> > are done
> > + at boot time by fsck. See fs_passno parameter
> > of
> > + fstab(5). This parameter will be copied into
> > the
> > + /etc/fstab file of the installed system. If not
> > + specified the default value of "0" will be used.
> > +
> > --label label: Specifies the label to give to the filesystem
> > to be made on the partition. If the given
> > label is already in use by another
> > filesystem, diff --git a/scripts/lib/wic/ksparser.py
> > b/scripts/lib/wic/ksparser.py index 0df9eb0..a49b7b9 100644
> > --- a/scripts/lib/wic/ksparser.py
> > +++ b/scripts/lib/wic/ksparser.py
> > @@ -155,6 +155,7 @@ class KickStart():
> > part.add_argument('--change-directory')
> > part.add_argument("--extra-space", type=sizetype("M"))
> > part.add_argument('--fsoptions', dest='fsopts')
> > + part.add_argument('--fspassno', dest='fspassno')
> > part.add_argument('--fstype', default='vfat',
> > choices=('ext2', 'ext3', 'ext4', 'btrfs',
> > 'squashfs', 'vfat', 'msdos',
> > 'erofs', diff --git a/scripts/lib/wic/partition.py
> > b/scripts/lib/wic/partition.py index 09e491d..e50871b 100644
> > --- a/scripts/lib/wic/partition.py
> > +++ b/scripts/lib/wic/partition.py
> > @@ -33,6 +33,7 @@ class Partition():
> > self.include_path = args.include_path
> > self.change_directory = args.change_directory
> > self.fsopts = args.fsopts
> > + self.fspassno = args.fspassno
> > self.fstype = args.fstype
> > self.label = args.label
> > self.use_label = args.use_label diff --git
> > a/scripts/lib/wic/plugins/imager/direct.py
> > b/scripts/lib/wic/plugins/imager/direct.py index 4d0b836..c44159b
> > 100644 --- a/scripts/lib/wic/plugins/imager/direct.py
> > +++ b/scripts/lib/wic/plugins/imager/direct.py
> > @@ -117,7 +117,7 @@ class DirectPlugin(ImagerPlugin):
> > updated = False
> > for part in self.parts:
> > if not part.realnum or not part.mountpoint \
> > - or part.mountpoint == "/" or not
> > part.mountpoint.startswith('/'):
> > + or not part.mountpoint.startswith('/'):
> > continue
> >
> > if part.use_uuid:
> > @@ -138,8 +138,9 @@ class DirectPlugin(ImagerPlugin):
> > device_name = "/dev/%s%s%d" % (part.disk, prefix,
> > part.realnum)
> > opts = part.fsopts if part.fsopts else "defaults"
> > + passno = part.fspassno if part.fspassno else "0"
> > line = "\t".join([device_name, part.mountpoint,
> > part.fstype,
> > - opts, "0", "0"]) + "\n"
> > + opts, "0", passno]) + "\n"
> >
> > fstab_lines.append(line)
> > updated = True
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [PATCH v2 2/2] wic: Update to the latest revision
2022-06-13 7:54 ` [PATCH v2 2/2] wic: Update to the latest revision Vijai Kumar K
@ 2022-06-13 9:02 ` Henning Schild
2022-06-13 9:18 ` Kanagarajan, Vijaikumar
0 siblings, 1 reply; 22+ messages in thread
From: Henning Schild @ 2022-06-13 9:02 UTC (permalink / raw)
To: Vijai Kumar K; +Cc: isar-users, jan.kiszka
Am Mon, 13 Jun 2022 13:24:01 +0530
schrieb Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com>:
> Update to the latest wic from OE-core.
>
> OE-core Revision: 06ed2aa93fc25a681e3a00ee120d9395b04845da
>
> Signed-off-by: Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com>
> ---
> RECIPE-API-CHANGELOG.md | 19 +++++++++++++++++++
> scripts/lib/wic/help.py | 6 ++++++
> scripts/lib/wic/ksparser.py | 1 +
> scripts/lib/wic/partition.py | 1 +
> scripts/lib/wic/plugins/imager/direct.py | 5 +++--
> 5 files changed, 30 insertions(+), 2 deletions(-)
>
> diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md
> index 2998824..436b204 100644
> --- a/RECIPE-API-CHANGELOG.md
> +++ b/RECIPE-API-CHANGELOG.md
> @@ -406,3 +406,22 @@ groups are now created after all packages are
> installed.
> Changes in next
> ---------------
> +
> +### Wic adds / mountpoint to fstab
> +
> +In the older version of wic, any mount point named / is skipped from
> adding +into the fstab entry.
> +
> +With the latest wic, this is not the case. / mount point, if any, is
> added +to /etc/fstab for automount.
> +
> +Any wks file which assumed that / would be skipped from /etc/fstab
> should +now be corrected. Otherwise, it might result in multiple /
> entries in /etc/fstab. +One by the image_configure_fstab function
> from image class and one by wic. +
> +Below is an example wks entry that might cause an issue.
> +Replace "part /" with "part" to avoid adding duplicate entry for /.
I am not sure multiple entries would be an issue, i think the last one
will simply always win. It is good to have a note about that change, but
i doubt people really have to do anything.
And if anything has to be done, isar should drop the first one
centrally.
The fstab entry for root is purely for adding some weird mount options,
usually root is not mounted with fstab but with cmdline and initrd. And
not having / in fstab just makes that rootfs more generic ... it can
become part of nfs and next guy takes it as btrfs while next guy takes
it as ext4.
Note is good, advise is not
Henning
> +
> +```
> +part / --source rootfs --ondisk sda --fstype ext4 --label platform
> --align 1024 --use-uuid +```
> diff --git a/scripts/lib/wic/help.py b/scripts/lib/wic/help.py
> index 4ff7470..73e3380 100644
> --- a/scripts/lib/wic/help.py
> +++ b/scripts/lib/wic/help.py
> @@ -940,6 +940,12 @@ DESCRIPTION
> quotes. If not specified, the default string
> is "defaults".
>
> + --fspassno: Specifies the order in which filesystem checks
> are done
> + at boot time by fsck. See fs_passno parameter
> of
> + fstab(5). This parameter will be copied into
> the
> + /etc/fstab file of the installed system. If not
> + specified the default value of "0" will be used.
> +
> --label label: Specifies the label to give to the filesystem
> to be made on the partition. If the given
> label is already in use by another
> filesystem, diff --git a/scripts/lib/wic/ksparser.py
> b/scripts/lib/wic/ksparser.py index 0df9eb0..a49b7b9 100644
> --- a/scripts/lib/wic/ksparser.py
> +++ b/scripts/lib/wic/ksparser.py
> @@ -155,6 +155,7 @@ class KickStart():
> part.add_argument('--change-directory')
> part.add_argument("--extra-space", type=sizetype("M"))
> part.add_argument('--fsoptions', dest='fsopts')
> + part.add_argument('--fspassno', dest='fspassno')
> part.add_argument('--fstype', default='vfat',
> choices=('ext2', 'ext3', 'ext4', 'btrfs',
> 'squashfs', 'vfat', 'msdos',
> 'erofs', diff --git a/scripts/lib/wic/partition.py
> b/scripts/lib/wic/partition.py index 09e491d..e50871b 100644
> --- a/scripts/lib/wic/partition.py
> +++ b/scripts/lib/wic/partition.py
> @@ -33,6 +33,7 @@ class Partition():
> self.include_path = args.include_path
> self.change_directory = args.change_directory
> self.fsopts = args.fsopts
> + self.fspassno = args.fspassno
> self.fstype = args.fstype
> self.label = args.label
> self.use_label = args.use_label
> diff --git a/scripts/lib/wic/plugins/imager/direct.py
> b/scripts/lib/wic/plugins/imager/direct.py index 4d0b836..c44159b
> 100644 --- a/scripts/lib/wic/plugins/imager/direct.py
> +++ b/scripts/lib/wic/plugins/imager/direct.py
> @@ -117,7 +117,7 @@ class DirectPlugin(ImagerPlugin):
> updated = False
> for part in self.parts:
> if not part.realnum or not part.mountpoint \
> - or part.mountpoint == "/" or not
> part.mountpoint.startswith('/'):
> + or not part.mountpoint.startswith('/'):
> continue
>
> if part.use_uuid:
> @@ -138,8 +138,9 @@ class DirectPlugin(ImagerPlugin):
> device_name = "/dev/%s%s%d" % (part.disk, prefix,
> part.realnum)
> opts = part.fsopts if part.fsopts else "defaults"
> + passno = part.fspassno if part.fspassno else "0"
> line = "\t".join([device_name, part.mountpoint,
> part.fstype,
> - opts, "0", "0"]) + "\n"
> + opts, "0", passno]) + "\n"
>
> fstab_lines.append(line)
> updated = True
^ permalink raw reply [flat|nested] 22+ messages in thread
* [PATCH v2 2/2] wic: Update to the latest revision
2022-06-13 7:53 [PATCH v2 0/2] Update WIC to bring in fspassno feature Vijai Kumar K
@ 2022-06-13 7:54 ` Vijai Kumar K
2022-06-13 9:02 ` Henning Schild
0 siblings, 1 reply; 22+ messages in thread
From: Vijai Kumar K @ 2022-06-13 7:54 UTC (permalink / raw)
To: isar-users; +Cc: henning.schild, jan.kiszka, Vijai Kumar K
Update to the latest wic from OE-core.
OE-core Revision: 06ed2aa93fc25a681e3a00ee120d9395b04845da
Signed-off-by: Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com>
---
RECIPE-API-CHANGELOG.md | 19 +++++++++++++++++++
scripts/lib/wic/help.py | 6 ++++++
scripts/lib/wic/ksparser.py | 1 +
scripts/lib/wic/partition.py | 1 +
scripts/lib/wic/plugins/imager/direct.py | 5 +++--
5 files changed, 30 insertions(+), 2 deletions(-)
diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md
index 2998824..436b204 100644
--- a/RECIPE-API-CHANGELOG.md
+++ b/RECIPE-API-CHANGELOG.md
@@ -406,3 +406,22 @@ groups are now created after all packages are installed.
Changes in next
---------------
+
+### Wic adds / mountpoint to fstab
+
+In the older version of wic, any mount point named / is skipped from adding
+into the fstab entry.
+
+With the latest wic, this is not the case. / mount point, if any, is added
+to /etc/fstab for automount.
+
+Any wks file which assumed that / would be skipped from /etc/fstab should
+now be corrected. Otherwise, it might result in multiple / entries in /etc/fstab.
+One by the image_configure_fstab function from image class and one by wic.
+
+Below is an example wks entry that might cause an issue.
+Replace "part /" with "part" to avoid adding duplicate entry for /.
+
+```
+part / --source rootfs --ondisk sda --fstype ext4 --label platform --align 1024 --use-uuid
+```
diff --git a/scripts/lib/wic/help.py b/scripts/lib/wic/help.py
index 4ff7470..73e3380 100644
--- a/scripts/lib/wic/help.py
+++ b/scripts/lib/wic/help.py
@@ -940,6 +940,12 @@ DESCRIPTION
quotes. If not specified, the default string is
"defaults".
+ --fspassno: Specifies the order in which filesystem checks are done
+ at boot time by fsck. See fs_passno parameter of
+ fstab(5). This parameter will be copied into the
+ /etc/fstab file of the installed system. If not
+ specified the default value of "0" will be used.
+
--label label: Specifies the label to give to the filesystem
to be made on the partition. If the given
label is already in use by another filesystem,
diff --git a/scripts/lib/wic/ksparser.py b/scripts/lib/wic/ksparser.py
index 0df9eb0..a49b7b9 100644
--- a/scripts/lib/wic/ksparser.py
+++ b/scripts/lib/wic/ksparser.py
@@ -155,6 +155,7 @@ class KickStart():
part.add_argument('--change-directory')
part.add_argument("--extra-space", type=sizetype("M"))
part.add_argument('--fsoptions', dest='fsopts')
+ part.add_argument('--fspassno', dest='fspassno')
part.add_argument('--fstype', default='vfat',
choices=('ext2', 'ext3', 'ext4', 'btrfs',
'squashfs', 'vfat', 'msdos', 'erofs',
diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py
index 09e491d..e50871b 100644
--- a/scripts/lib/wic/partition.py
+++ b/scripts/lib/wic/partition.py
@@ -33,6 +33,7 @@ class Partition():
self.include_path = args.include_path
self.change_directory = args.change_directory
self.fsopts = args.fsopts
+ self.fspassno = args.fspassno
self.fstype = args.fstype
self.label = args.label
self.use_label = args.use_label
diff --git a/scripts/lib/wic/plugins/imager/direct.py b/scripts/lib/wic/plugins/imager/direct.py
index 4d0b836..c44159b 100644
--- a/scripts/lib/wic/plugins/imager/direct.py
+++ b/scripts/lib/wic/plugins/imager/direct.py
@@ -117,7 +117,7 @@ class DirectPlugin(ImagerPlugin):
updated = False
for part in self.parts:
if not part.realnum or not part.mountpoint \
- or part.mountpoint == "/" or not part.mountpoint.startswith('/'):
+ or not part.mountpoint.startswith('/'):
continue
if part.use_uuid:
@@ -138,8 +138,9 @@ class DirectPlugin(ImagerPlugin):
device_name = "/dev/%s%s%d" % (part.disk, prefix, part.realnum)
opts = part.fsopts if part.fsopts else "defaults"
+ passno = part.fspassno if part.fspassno else "0"
line = "\t".join([device_name, part.mountpoint, part.fstype,
- opts, "0", "0"]) + "\n"
+ opts, "0", passno]) + "\n"
fstab_lines.append(line)
updated = True
--
2.25.1
^ permalink raw reply [flat|nested] 22+ messages in thread
end of thread, other threads:[~2022-09-26 12:34 UTC | newest]
Thread overview: 22+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-04-21 8:52 [PATCH v2 0/2] bump bitbake and wic for python 3.10 support Henning Schild
2022-04-21 8:52 ` [PATCH v2 1/2] bitbake: Update to 1.50.5 release Henning Schild
2022-04-21 8:52 ` [PATCH v2 2/2] wic: Update to the latest revision Henning Schild
2022-04-21 9:02 ` Henning Schild
2022-04-22 7:59 ` [PATCH v2 0/2] bump bitbake and wic for python 3.10 support Anton Mikanovich
2022-04-22 8:15 ` Bezdeka, Florian
2022-04-22 8:26 ` Henning Schild
2022-04-22 8:52 ` Henning Schild
2022-04-22 9:54 ` Henning Schild
2022-04-22 8:27 ` Henning Schild
2022-04-22 8:57 ` Anton Mikanovich
2022-06-13 7:53 [PATCH v2 0/2] Update WIC to bring in fspassno feature Vijai Kumar K
2022-06-13 7:54 ` [PATCH v2 2/2] wic: Update to the latest revision Vijai Kumar K
2022-06-13 9:02 ` Henning Schild
2022-06-13 9:18 ` Kanagarajan, Vijaikumar
2022-06-17 5:45 ` Kanagarajan, Vijaikumar
2022-09-26 1:40 ` Moessbauer, Felix
2022-09-26 4:53 ` Kanagarajan, Vijaikumar
2022-09-26 5:57 ` Moessbauer, Felix
2022-09-26 7:46 ` Henning Schild
2022-09-26 7:56 ` Kanagarajan, Vijaikumar
2022-09-26 8:02 ` Moessbauer, Felix
2022-09-26 12:34 ` Henning Schild
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox