Re: [PATCH v5] meta/classes: generate bill of material from image

[Jan Kiszka <jan.kiszka@siemens.com>]

On 13.08.19 15:40, [ext] Q. Gylstorff wrote:
> From: Quirin Gylstorff <quirin.gylstorff@siemens.com>
> 
> To create products it is necessary to have a list
> of used packages for clearance and to security monitoring.
> To get a simple list of packages use dpkg-query and generate
> a list with the following pattern:
> 
> source name| source version | binary package name | binary version
> 
> All rootfs generate the list by default. Currently the f
> following lists are generated:
> - buildchroot-${DISTRO}-${ARCH}.manifest
> - ${IMAGE}-${DISTRO}-${ARCH}.manifest
> - optional: sdkchroot-${DISTRO}-${ARCH}.manifest
> 
> Remove the feature with:
> ROOTFS_FEATURES_remove = "generate-manifest"
> 
> Signed-off-by: Quirin Gylstorff <quirin.gylstorff@siemens.com>
> ---
>   meta/classes/image.bbclass  |  2 +-
>   meta/classes/rootfs.bbclass | 12 +++++++++++-
>   2 files changed, 12 insertions(+), 2 deletions(-)
> 
> diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
> index ec6bd39..70c46d2 100644
> --- a/meta/classes/image.bbclass
> +++ b/meta/classes/image.bbclass
> @@ -58,7 +58,7 @@ image_do_mounts() {
>   }
>   
>   ROOTFSDIR = "${IMAGE_ROOTFS}"
> -ROOTFS_FEATURES += "copy-package-cache clean-package-cache finalize-rootfs"
> +ROOTFS_FEATURES += "copy-package-cache clean-package-cache finalize-rootfs generate-manifest"
>   ROOTFS_PACKAGES += "${IMAGE_PREINSTALL} ${IMAGE_INSTALL}"
>   
>   inherit rootfs
> diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass
> index c7e0435..59edcde 100644
> --- a/meta/classes/rootfs.bbclass
> +++ b/meta/classes/rootfs.bbclass
> @@ -10,7 +10,8 @@ ROOTFS_PACKAGES ?= ""
>   # 'deploy-package-cache' - copy the package cache ${WORKDIR}/apt_cache
>   # 'clean-package-cache' - delete package cache from rootfs
>   # 'finalize-rootfs' - delete files needed to chroot into the rootfs
> -ROOTFS_FEATURES ?= ""
> +# 'generate-manifest' - generate a package manifest of the rootfs into ${IMAGE_DEPLOY_DIR}
> +ROOTFS_FEATURES ?= "generate-manifest"
>   
>   ROOTFS_APT_ARGS="install --yes -o Debug::pkgProblemResolver=yes"
>   
> @@ -212,6 +213,15 @@ rootfs_postprocess_finalize() {
>   EOSUDO
>   }
>   
> +ROOTFS_POSTPROCESS_COMMAND += "${@bb.utils.contains('ROOTFS_FEATURES', 'generate-manifest', 'rootfs_generate_manifest', '', d)}"
> +rootfs_generate_manifest () {
> +    mkdir -p ${DEPLOY_DIR_IMAGE}
> +    sudo -E chroot '${ROOTFSDIR}' \
> +    dpkg-query \
> +    -f '${source:Package}|${source:Version}|${binary:Package}|${Version}\n' -W > \
> +    ${DEPLOY_DIR_IMAGE}/"${PF}".manifest
> +}
> +
>   do_rootfs_postprocess[vardeps] = "${ROOTFS_POSTPROCESS_COMMAND}"
>   python do_rootfs_postprocess() {
>       # Take care that its correctly mounted:
> 

This "nicely" triggers [1]. And after running bitbake-diffsigs, it's 
also clear why:

$ bitbake-diffsigs tmp/stamps/debian-stretch-armhf/buildchroot-host/1.0-r0.do_rootfs_postprocess.sigdata.8ea37df88464fe5477148b167fd61c80 tmp/stamps/debian-stretch-armhf/buildchroot-host/1.0-r0.do_rootfs_postprocess.sigdata.fae265f1d4686b64bc01ddb05f972db6
basehash changed from dd03388909f6c4c2b905ebaf2539a435 to 2f9b6886ae7f15d5288e7d0ceb493750
Variable MACHINE value changed from 'qemuarm' to 'de0-nano-soc'

You pull an image dependency into the buildchroot by using 
DEPLOY_DIR_IMAGE. And that opens the doors to parallel-build hell. It's 
also semantically incorrect for the buildchroot because those have 
nothing image-specific in them.

The good news: After my patch series, there are also 
DEPLOY_DIR_BUILDCHROOT and DEPLOY_DIR_SDKCHROOT. The bad news: You need 
to find out which one to use.

Jan

[1] https://groups.google.com/d/msgid/isar-users/c93a7dbdbc9448afc0d0b65ec754ac698ed658c6.1566800787.git.jan.kiszka%40siemens.com

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux