Re: [PATCH] meta/classes: generate bill of material from image

[Claudius Heine <claudius.heine.ext@siemens.com>]

Hi Quirin,

On 05/08/2019 16.07, [ext] Q. Gylstorff wrote:
> From: Quirin Gylstorff <quirin.gylstorff@siemens.com>
> 
> To create products it is necessary to have a list
> of used packages for clearance and to security monitoring.
> To get a simple list of packages use dpkg-query and generate
> a list with the following pattern:
> 
> source name| source version | binary package name | binary version
> 
> To use it add following line to the local.conf:
> ISAR_DO_PACKAGE_LIST ?= "1"

I would rather that the commit message would not suggest adding this to 
the local.conf and instead adding this to the image recipe variable 
scope, wherever that is a .bb, .bbclass, .bbappend or any .conf file.

> 
> Signed-off-by: Quirin Gylstorff <quirin.gylstorff@siemens.com>
> ---
>   meta-isar/conf/local.conf.sample                 |  4 ++++
>   .../classes/image-package-list-extension.bbclass | 16 ++++++++++++++++
>   meta/classes/image.bbclass                       |  1 +
>   3 files changed, 21 insertions(+)
>   create mode 100644 meta/classes/image-package-list-extension.bbclass
> 
> diff --git a/meta-isar/conf/local.conf.sample b/meta-isar/conf/local.conf.sample
> index 5b3a0a1..d188051 100644
> --- a/meta-isar/conf/local.conf.sample
> +++ b/meta-isar/conf/local.conf.sample
> @@ -170,6 +170,10 @@ IMAGE_INSTALL = "hello-isar example-raw example-module-${KERNEL_NAME} enable-fsc
>   # NOTE: this works on build host >= stretch for armhf, arm64 and amd64 targets for now.
>   ISAR_CROSS_COMPILE ?= "0"
>   
> +#
> +# Generate package list
> +ISAR_DO_PACKAGE_LIST ?= "1"
> +
>   #
>   # Uncomment this to enable use of cached base repository
>   #ISAR_USE_CACHED_BASE_REPO ?= "1"
> diff --git a/meta/classes/image-package-list-extension.bbclass b/meta/classes/image-package-list-extension.bbclass
> new file mode 100644
> index 0000000..558922e
> --- /dev/null
> +++ b/meta/classes/image-package-list-extension.bbclass
> @@ -0,0 +1,16 @@
> +# This software is a part of ISAR.
> +# Copyright (C) Siemens AG, 2019
> +#
> +# SPDX-License-Identifier: MIT
> +
> +ISAR_DO_PACKAGE_LIST ??= "0"
> +image_package_list[dirs] = "${DEPLOY_DIR_IMAGE}"
> +image_package_list() {

Its a postprocess function so call it 'image_postprocess_package_list' 
instead.

> +    if [ "${@repr(bb.utils.to_boolean(d.getVar('ISAR_DO_PACKAGE_LIST')))}" = 'True' ]; then

I don't like all those boolean values and tried to establish a better 
method using features. Take a look at the rootfs.bbclass, for example:

   ROOTFS_POSTPROCESS_COMMAND += 
"${@bb.utils.contains('ROOTFS_FEATURES', 'finalize-rootfs', 
'rootfs_postprocess_finalize', '', d)}"

You can adapt that. For instance like this:

   ROOTFS_POSTPROCESS_COMMAND += 
"${@bb.utils.contains('ROOTFS_FEATURES', 'generate-package-list', 
'image_postprocess_package_list', '', d)}"

Also it probably makes sense to have this as a default image feature. 
Otherwise people will not know about it and try to implement it themselves.

> +      dpkg-query --admindir=${IMAGE_ROOTFS}/var/lib/dpkg/ \
> +          -f '${source:Package}|${source:Version}|${binary:Package}|${Version}\n' -W > \
> +          ${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.packages.lst

As others have said, make it easily machine readable. JSON maybe?

Maybe something like:

   echo "[" > ${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.packages.lst
   dpkg-query ... -f '{"source_package": "${source:Package}", 
source_version": "${source:Version}", ...},\n' ... >> 
${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.packages.lst
   echo "]" >> ${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.packages.lst

Also maybe copy the licenses (/usr/share/doc/$package/copyright) 
somewhere? That might conflict with Hennings 'isar-exclude-docs' 
package, though... Maybe that can be changed to move them somewhere 
instead of removing them?

kind regards,
Claudius

> +    fi
> +}
> +ROOTFS_POSTPROCESS_COMMAND =+ "image_package_list"
> +
> diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
> index ec6bd39..85bab64 100644
> --- a/meta/classes/image.bbclass
> +++ b/meta/classes/image.bbclass
> @@ -68,6 +68,7 @@ inherit image-tools-extension
>   inherit image-postproc-extension
>   inherit image-locales-extension
>   inherit image-account-extension
> +inherit image-package-list-extension
>   
>   # Extra space for rootfs in MB
>   ROOTFS_EXTRA ?= "64"
> 

-- 
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de