From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6721684426774806528 X-Received: by 2002:a17:906:784:: with SMTP id l4mr34442035ejc.19.1565684361276; Tue, 13 Aug 2019 01:19:21 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a50:b7a3:: with SMTP id h32ls30552480ede.16.gmail; Tue, 13 Aug 2019 01:19:20 -0700 (PDT) X-Google-Smtp-Source: APXvYqwW0a0z3AHwkZMcTe6+EmKVZgz6SK8IwMyi7CALDMtL9SwB1YiyrzQ00kNbdkjN/7hU4MpW X-Received: by 2002:a50:aed8:: with SMTP id f24mr7782508edd.90.1565684360881; Tue, 13 Aug 2019 01:19:20 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1565684360; cv=none; d=google.com; s=arc-20160816; b=XSzNrQGKzHFD2/nh7PUujnc2WQwUMOzd9C03HnZHQbUAsJABCBST0Xm3WNybsH/6/q XzIEHgb+i0kJzc9ZJpCmvrDJv00D1a8Q5mVnAsQaJ8+BCrru44ihpDSD6FDEiu0LWM94 6uKtEH0E2L6nKQDaCVN2LirOdvgQPn/xDKyKRN81z3yOYklQfe1+hGZyLSVCZVwLlA7V ADjVbc3x0e5LpMhOjWpCrtGzLSMyVtkXzRNa3QYg38vHayspVyOhUs6xfCzP0LygD4rv B0x8enN8ZKlNQmDz/rNwatzmldx90npJ/IVu1pW15ecxj4T2Mylvu5NzJYbQ0fiQlgr+ cLrg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=YZIvZC+/l31yTUpXkdmCH5Pc6iQc/m3BE5BjdlMZ2jw=; b=WmkII8d2kV4u0OF8PGsPhh0gCpf0WB/W3ByC5qPCAnv+tCzC7S0e7974hXgIYoAlWv g8EXLUWTmHjWPKl3SfzHR1sMw0UzPIJ7GBvviwl3mJpQOkEJl2WmqAaz0OnPyOkIxKkP L085rkzLM/lhWko2rYt/0KW6w3Py3GWpaMXZHW+snVWaoy75LuOVy7waB+EqKujcS+8s xS6+/UXBpEhLbMlj+Pp4Yf4kQvi3M9J4DpvjUIZh2rp7Owjbp+bQG2RNIOVbXPixqkGm wkFjDxBDtzw6YOH6FGKQRbehhZtOxW/bRhClbTA6JeSHZ9zaMYbqQ3oKAYEAAeQIp2/n 8XQg== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of quirin.gylstorff@siemens.com designates 192.35.17.28 as permitted sender) smtp.mailfrom=Quirin.Gylstorff@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from goliath.siemens.de (goliath.siemens.de. [192.35.17.28]) by gmr-mx.google.com with ESMTPS id jz14si4702676ejb.0.2019.08.13.01.19.20 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 13 Aug 2019 01:19:20 -0700 (PDT) Received-SPF: pass (google.com: domain of quirin.gylstorff@siemens.com designates 192.35.17.28 as permitted sender) client-ip=192.35.17.28; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of quirin.gylstorff@siemens.com designates 192.35.17.28 as permitted sender) smtp.mailfrom=Quirin.Gylstorff@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: from mail1.sbs.de (mail1.sbs.de [192.129.41.35]) by goliath.siemens.de (8.15.2/8.15.2) with ESMTPS id x7D8JJjx019089 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 13 Aug 2019 10:19:19 +0200 Received: from debian.ad001.siemens.net ([139.25.68.238]) by mail1.sbs.de (8.15.2/8.15.2) with ESMTP id x7D8JIsL020033; Tue, 13 Aug 2019 10:19:18 +0200 From: "Q. Gylstorff" To: isar-users@googlegroups.com Cc: Quirin Gylstorff , Claudius Heine Subject: [PATCH v4] meta/classes: generate bill of material from image Date: Tue, 13 Aug 2019 10:18:23 +0200 Message-Id: <20190813081823.29704-1-Quirin.Gylstorff@siemens.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <3221bfdb-641b-7e54-3fb5-1facbf6e5585@siemens.com> References: <3221bfdb-641b-7e54-3fb5-1facbf6e5585@siemens.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-TUID: 8kUUyBH5MxqA From: Quirin Gylstorff To create products it is necessary to have a list of used packages for clearance and to security monitoring. To get a simple list of packages use dpkg-query and generate a list with the following pattern: source name| source version | binary package name | binary version The list is stored in ${IMAGE_FULLNAME}.rootfs.manifest Remove the feature with: ROOTFS_FEATURES_remove = "generate-manifest" Signed-off-by: Quirin Gylstorff --- Changes: v4: Add sdk rootfs to manifest Avoid duplicated code and move gen_accounts_array and gen_manifest_array to shell-list-processing-helper call dpkg-query from $PATH v3: Add list of manifest for buildchroot manifest This list can be exdent to add additional output generators v2: use FEATURE instead of own variable meta/classes/image-account-extension.bbclass | 28 ++-------- .../image-package-list-extension.bbclass | 54 +++++++++++++++++++ meta/classes/image.bbclass | 3 +- .../shell-list-processing-helper.bbclass | 30 +++++++++++ 4 files changed, 89 insertions(+), 26 deletions(-) create mode 100644 meta/classes/image-package-list-extension.bbclass create mode 100644 meta/classes/shell-list-processing-helper.bbclass diff --git a/meta/classes/image-account-extension.bbclass b/meta/classes/image-account-extension.bbclass index 22754da..df44c49 100644 --- a/meta/classes/image-account-extension.bbclass +++ b/meta/classes/image-account-extension.bbclass @@ -25,36 +25,14 @@ GROUPS ??= "" #GROUP_root[gid] = "" #GROUP_root[flags] = "system" -def gen_accounts_array(d, listname, entryname, flags, verb_flags=None): - from itertools import chain - - entries = (d.getVar(listname, True) or "").split() - return " ".join( - ":".join( - chain( - (entry,), - ( - (",".join( - ( - d.getVarFlag(entryname + "_" + entry, flag, True) or "" - ).split() - ) if flag not in (verb_flags or []) else ( - d.getVarFlag(entryname + "_" + entry, flag, True) or "" - )).replace(":","=") - for flag in flags - ), - ) - ) - for entry in entries - ) - +inherit shell-list-processing-helper # List of space separated entries, where each entry has the format: # username:encryptedpassword:expiredate:inactivenumber:userid:groupid:comment:homedir:shell:group1,group2:flag1,flag2 -IMAGE_ACCOUNTS_USERS =+ "${@gen_accounts_array(d, 'USERS', 'USER', ['password', 'expire', 'inactive', 'uid', 'gid', 'comment', 'home', 'shell', 'groups', 'flags'], ['password', 'comment', 'home', 'shell'])}" +IMAGE_ACCOUNTS_USERS =+ "${@gen_shell_list(d, 'USERS', 'USER', ['password', 'expire', 'inactive', 'uid', 'gid', 'comment', 'home', 'shell', 'groups', 'flags'], ['password', 'comment', 'home', 'shell'])}" # List of space separated entries, where each entry has the format: # groupname:groupid:flag1,flag2 -IMAGE_ACCOUNTS_GROUPS =+ "${@gen_accounts_array(d, 'GROUPS', 'GROUP', ['gid', 'flags'])}" +IMAGE_ACCOUNTS_GROUPS =+ "${@gen_shell_list(d, 'GROUPS', 'GROUP', ['gid', 'flags'])}" ROOTFS_CONFIGURE_COMMAND += "image_configure_accounts" image_configure_accounts[weight] = "3" diff --git a/meta/classes/image-package-list-extension.bbclass b/meta/classes/image-package-list-extension.bbclass new file mode 100644 index 0000000..0aa3015 --- /dev/null +++ b/meta/classes/image-package-list-extension.bbclass @@ -0,0 +1,54 @@ +# This software is a part of ISAR. +# Copyright (C) Siemens AG, 2019 +# +# SPDX-License-Identifier: MIT +MANIFESTS ?= "target build sdk" +DPKG_DIR ?= "/var/lib/dpkg" +# rootfs needs to be mounted inside of buildchroot +MANIFEST_build[rootfs] ?= "${DPKG_DIR}" +MANIFEST_target[rootfs] ?= "${PP_ROOTFS}${DPKG_DIR}" +MANIFEST_sdk[rootfs] ?= "/work/${DISTRO}-${DISTRO_ARCH}/sdkchroot-${HOST_DISTRO}-${HOST_ARCH}-${DISTRO_ARCH}/rootfs${DPKG_DIR}" + +inherit shell-list-processing-helper +IMAGE_MANIFESTS =+ "${@gen_shell_list(d, 'MANIFESTS', 'MANIFEST', ['rootfs'])}" + +do_image_generate_manifest[dirs] = "${DEPLOY_DIR_IMAGE}" +image_generate_manifest() { + image_do_mounts + # mount working directory to access sdk rootfs + sudo -s <<'EOSUDO' + ( flock 9 + mkdir -p ${BUILDCHROOT_DIR}/work + if ! mountpoint ${BUILDCHROOT_DIR}/work >/dev/null 2>&1; then + mount --bind --make-private ${TMPDIR}/work ${BUILDCHROOT_DIR}/work + fi + ) 9>${MOUNT_LOCKFILE} +EOSUDO + list='${@" ".join(d.getVar('IMAGE_MANIFESTS', True).split())} ' + while true; do + list_rest="${list#*:* }" + entry="${list%%${list_rest}}" + list="${list_rest}" + + if [ -z "${entry}" ]; then + break + fi + # Add colon to the end of the entry and remove trailing space: + entry="${entry% }:" + + # Decode entries: + name="${entry%%:*}" + entry="${entry#${name}:}" + + rootfs="${entry%%:*}" + entry="${entry#${rootfs}:}" + if sudo -E chroot ${BUILDCHROOT_DIR} test -d "$rootfs"; then + sudo -E chroot ${BUILDCHROOT_DIR} \ + dpkg-query --admindir="$rootfs" \ + -f '${source:Package}|${source:Version}|${binary:Package}|${Version}\n' -W > \ + ${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}."$name".manifest + fi + done +} +ROOTFS_POSTPROCESS_COMMAND += "${@bb.utils.contains('ROOTFS_FEATURES', 'generate-manifest', 'image_generate_manifest', '', d)}" + diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass index ec6bd39..60dd9fb 100644 --- a/meta/classes/image.bbclass +++ b/meta/classes/image.bbclass @@ -58,7 +58,7 @@ image_do_mounts() { } ROOTFSDIR = "${IMAGE_ROOTFS}" -ROOTFS_FEATURES += "copy-package-cache clean-package-cache finalize-rootfs" +ROOTFS_FEATURES += "copy-package-cache clean-package-cache finalize-rootfs generate-manifest" ROOTFS_PACKAGES += "${IMAGE_PREINSTALL} ${IMAGE_INSTALL}" inherit rootfs @@ -68,6 +68,7 @@ inherit image-tools-extension inherit image-postproc-extension inherit image-locales-extension inherit image-account-extension +inherit image-package-list-extension # Extra space for rootfs in MB ROOTFS_EXTRA ?= "64" diff --git a/meta/classes/shell-list-processing-helper.bbclass b/meta/classes/shell-list-processing-helper.bbclass new file mode 100644 index 0000000..105066b --- /dev/null +++ b/meta/classes/shell-list-processing-helper.bbclass @@ -0,0 +1,30 @@ +# This software is a part of ISAR. +# Copyright (C) Siemens AG, 2019 +# +# SPDX-License-Identifier: MIT +# +# This class extends the image.bbclass for creating user accounts and groups. + +def gen_shell_list(d, listname, entryname, flags, verb_flags=None): + from itertools import chain + + entries = (d.getVar(listname, True) or "").split() + return " ".join( + ":".join( + chain( + (entry,), + ( + (",".join( + ( + d.getVarFlag(entryname + "_" + entry, flag, True) or "" + ).split() + ) if flag not in (verb_flags or []) else ( + d.getVarFlag(entryname + "_" + entry, flag, True) or "" + )).replace(":","=") + for flag in flags + ), + ) + ) + for entry in entries + ) + -- 2.20.1