[PATCH 0/2] Fix reproducible issues in armhf

[PATCH 0/2] Fix reproducible issues in armhf

[venkata.pyla]

From: venkata pyla <venkata.pyla@toshiba-tsip.com>

Hi,

The following series patch fixes some reproducible issues in armhf, the
issues are reported in child project `isar-cip-core`[1], please review
them and let me know your feedback on the solutions.


[1] https://gitlab.com/cip-project/cip-core/isar-cip-core/-/issues/70

venkata pyla (2):
  rootfs.bbclass: clean blkid cache files from rootfs
  image.bbclass: remove core files generated in rootfs

 meta/classes/image.bbclass  | 6 ++++++
 meta/classes/rootfs.bbclass | 6 ++++++
 2 files changed, 12 insertions(+)

-- 
2.20.1

[PATCH 1/2] rootfs.bbclass: clean blkid cache files from rootfs

[venkata.pyla]

From: venkata pyla <venkata.pyla@toshiba-tsip.com>

Sometimes `blkid` generates cached files `/run/blkid/blkdid.tab*`, these
cache files are not reproducible and not necessary in the final image,
so remove them to achieve reproducible builds.

Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com>
---
 meta/classes/rootfs.bbclass | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass
index 22449d71..5d493ea2 100644
--- a/meta/classes/rootfs.bbclass
+++ b/meta/classes/rootfs.bbclass
@@ -293,6 +293,12 @@ rootfs_export_dpkg_status() {
        '${ROOTFS_DPKGSTATUS_DEPLOY_DIR}'/'${ROOTFS_PACKAGE_SUFFIX}'.dpkg_status
 }
 
+ROOTFS_POSTPROCESS_COMMAND += "rootfs_postprocess_clean_blkid_cache"
+rootfs_postprocess_clean_blkid_cache() {
+    # blkid generates cache file which are not reproducible
+    sudo rm -rf "${ROOTFSDIR}/run/blkid/blkid.tab"*
+}
+
 do_rootfs_postprocess[vardeps] = "${ROOTFS_POSTPROCESS_COMMAND}"
 do_rootfs_postprocess[network] = "${TASK_USE_SUDO}"
 python do_rootfs_postprocess() {
-- 
2.20.1

[PATCH 2/2] image.bbclass: remove core files generated in rootfs

[venkata.pyla]

From: venkata pyla <venkata.pyla@toshiba-tsip.com>

Sometimes core dumps are created in rootfs and left in the final image,
and that results into non-reproducible images, so move those files to
${WORKDIR}/temp location and warn the user.

In armhf there is an issue with qemu-user-static that results in
creating core dumps which is reported in Debian upstream [1], meanwhile
move these core dumps outside the rootfs.

[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1040981

Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com>
---
 meta/classes/image.bbclass | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index 08b6d3d6..80509d0d 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -461,6 +461,12 @@ do_rootfs_finalize() {
         fi
 EOSUDO
 
+    # Move core dump files generated in rootfs to temp folder in workdir.
+    for f in ${ROOTFSDIR}/*.core; do
+        sudo mv "${f}" "${WORKDIR}/temp/"
+        bbwarn "found core dump in rootfs, moved it to ${WORKDIR}/temp/"
+    done
+
     # Set same time-stamps to the newly generated file/folders in the
     # rootfs image for the purpose of reproducible builds.
     if [ -n "${SOURCE_DATE_EPOCH}" ]; then
-- 
2.20.1

Re: [PATCH 1/2] rootfs.bbclass: clean blkid cache files from rootfs

[vijai kumar]

[-- Attachment #1.1: Type: text/plain, Size: 1370 bytes --]



On Monday, August 7, 2023 at 10:05:21 PM UTC+5:30 
venkat...@toshiba-tsip.com wrote:

From: venkata pyla <venkat...@toshiba-tsip.com> 

Sometimes `blkid` generates cached files `/run/blkid/blkdid.tab*`, these 
cache files are not reproducible and not necessary in the final image, 
so remove them to achieve reproducible builds. 

Signed-off-by: venkata pyla <venkat...@toshiba-tsip.com> 
--- 
meta/classes/rootfs.bbclass | 6 ++++++ 
1 file changed, 6 insertions(+) 

diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass 
index 22449d71..5d493ea2 100644 
--- a/meta/classes/rootfs.bbclass 
+++ b/meta/classes/rootfs.bbclass 
@@ -293,6 +293,12 @@ rootfs_export_dpkg_status() { 
'${ROOTFS_DPKGSTATUS_DEPLOY_DIR}'/'${ROOTFS_PACKAGE_SUFFIX}'.dpkg_status 
} 

+ROOTFS_POSTPROCESS_COMMAND += "rootfs_postprocess_clean_blkid_cache" 
+rootfs_postprocess_clean_blkid_cache() { 
+ # blkid generates cache file which are not reproducible 
+ sudo rm -rf "${ROOTFSDIR}/run/blkid/blkid.tab"* 
+} 


If this is a necessary clean up function, can we not move to 
rootfs_finalize? Or maybe adding it to ROOTFS_CLEAN_FILES?

Thanks,
Vijai Kumar K
 

+ 
do_rootfs_postprocess[vardeps] = "${ROOTFS_POSTPROCESS_COMMAND}" 
do_rootfs_postprocess[network] = "${TASK_USE_SUDO}" 
python do_rootfs_postprocess() { 
-- 
2.20.1 



[-- Attachment #1.2: Type: text/html, Size: 1982 bytes --]

RE: [PATCH 1/2] rootfs.bbclass: clean blkid cache files from rootfs

[Venkata.Pyla]

[-- Attachment #1: Type: text/plain, Size: 2367 bytes --]



From: isar-users@googlegroups.com <isar-users@googlegroups.com> On Behalf Of vijai kumar
Sent: Monday, August 7, 2023 10:29 PM
To: isar-users <isar-users@googlegroups.com>
Subject: Re: [PATCH 1/2] rootfs.bbclass: clean blkid cache files from rootfs


On Monday, August 7, 2023 at 10:05:21 PM UTC+5:30 venkat...@toshiba-tsip.com<mailto:venkat...@toshiba-tsip.com> wrote:
From: venkata pyla <venkat...@toshiba-tsip.com>

Sometimes `blkid` generates cached files `/run/blkid/blkdid.tab*`, these
cache files are not reproducible and not necessary in the final image,
so remove them to achieve reproducible builds.

Signed-off-by: venkata pyla <venkat...@toshiba-tsip.com>
---
meta/classes/rootfs.bbclass | 6 ++++++
1 file changed, 6 insertions(+)

diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass
index 22449d71..5d493ea2 100644
--- a/meta/classes/rootfs.bbclass
+++ b/meta/classes/rootfs.bbclass
@@ -293,6 +293,12 @@ rootfs_export_dpkg_status() {
'${ROOTFS_DPKGSTATUS_DEPLOY_DIR}'/'${ROOTFS_PACKAGE_SUFFIX}'.dpkg_status
}

+ROOTFS_POSTPROCESS_COMMAND += "rootfs_postprocess_clean_blkid_cache"
+rootfs_postprocess_clean_blkid_cache() {
+ # blkid generates cache file which are not reproducible
+ sudo rm -rf "${ROOTFSDIR}/run/blkid/blkid.tab"*
+}

If this is a necessary clean up function, can we not move to rootfs_finalize? Or maybe adding it to ROOTFS_CLEAN_FILES?

There is no issue to keep this cleaning in `rootfs_finalize`, in fact it is more meaning full to keep all cleanup in finalize methods.

Thanks for the suggestion, I will change in v2.

Thanks,
Vijai Kumar K

+
do_rootfs_postprocess[vardeps] = "${ROOTFS_POSTPROCESS_COMMAND}"
do_rootfs_postprocess[network] = "${TASK_USE_SUDO}"
python do_rootfs_postprocess() {
--
2.20.1

--
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com<mailto:isar-users+unsubscribe@googlegroups.com>.
To view this discussion on the web visit https://groups.google.com/d/msgid/isar-users/6340d949-acc1-4283-9713-01f584686929n%40googlegroups.com<https://groups.google.com/d/msgid/isar-users/6340d949-acc1-4283-9713-01f584686929n%40googlegroups.com?utm_medium=email&utm_source=footer>.

[-- Attachment #2: Type: text/html, Size: 6243 bytes --]

[PATCH v2 0/2] Fix reproducible issues in armhf

[venkata.pyla]

From: venkata pyla <venkata.pyla@toshiba-tsip.com>

Hi,

Updated the patch series based on the feedback.

v2 changes:
 - Drop cleaning of blkid cache files in rootfs_postporcess_command and
   do it in rootfs_finalize method.
 - corrected the condition for scanning the *.core files

venkata pyla (2):
  image.bbclass: remove core files generated in rootfs
  image.bbclass: clean blkid cache files from rootfs

 meta/classes/image.bbclass | 10 ++++++++++
 1 file changed, 10 insertions(+)

-- 
2.20.1

[PATCH v2 1/2] image.bbclass: remove core files generated in rootfs

[venkata.pyla]

From: venkata pyla <venkata.pyla@toshiba-tsip.com>

Sometimes core dumps are created in rootfs and left in the final image,
and that causes non-reproducible images, so move those files to
${WORKDIR}/temp location and warn the user.

In armhf there is an issue with qemu-user-static that results in
creating core dumps which is reported in Debian upstream [1], meanwhile
move these core dumps outside the rootfs.

[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1040981

Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com>
---
 meta/classes/image.bbclass | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index 08b6d3d6..9aa6c856 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -461,6 +461,13 @@ do_rootfs_finalize() {
         fi
 EOSUDO
 
+    # Sometimes qemu-user-static generates coredumps in chroot, move them
+    # to work temporary directory and inform user about it.
+    for f in $(sudo find ${ROOTFSDIR} -name *.core); do
+        sudo mv "${f}" "${WORKDIR}/temp/"
+        bbwarn "found core dump in rootfs, check it in ${WORKDIR}/temp/${f##*/}"
+    done
+
     # Set same time-stamps to the newly generated file/folders in the
     # rootfs image for the purpose of reproducible builds.
     if [ -n "${SOURCE_DATE_EPOCH}" ]; then
-- 
2.20.1

[PATCH v2 2/2] image.bbclass: clean blkid cache files from rootfs

[venkata.pyla]

From: venkata pyla <venkata.pyla@toshiba-tsip.com>

blkid generates cache files when `blkid` command is used during build,
these cache files in rootfs causes non-reproducible rootfs, so
delete these files in the cleanup.

Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com>
---
 meta/classes/image.bbclass | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index 9aa6c856..d9fc04eb 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -459,6 +459,9 @@ do_rootfs_finalize() {
             mv "${ROOTFSDIR}/etc/apt/sources-list" \
                 "${ROOTFSDIR}/etc/apt/sources.list.d/bootstrap.list"
         fi
+
+        rm -f "${ROOTFSDIR}/run/blkid/blkid.tab"
+        rm -f "${ROOTFSDIR}/run/blkid/blkid.tab.old"
 EOSUDO
 
     # Sometimes qemu-user-static generates coredumps in chroot, move them
-- 
2.20.1

Re: [PATCH v2 0/2] Fix reproducible issues in armhf

[vijai kumar]

[-- Attachment #1.1: Type: text/plain, Size: 661 bytes --]



On Wednesday, August 9, 2023 at 7:17:10 PM UTC+5:30 
venkat...@toshiba-tsip.com wrote:

From: venkata pyla <venkat...@toshiba-tsip.com> 

Hi, 

Updated the patch series based on the feedback. 

v2 changes: 
- Drop cleaning of blkid cache files in rootfs_postporcess_command and 
do it in rootfs_finalize method. 
- corrected the condition for scanning the *.core files 

venkata pyla (2): 
image.bbclass: remove core files generated in rootfs 
image.bbclass: clean blkid cache files from rootfs 


LGTM.

Thanks,
Vijai Kumar K
 

meta/classes/image.bbclass | 10 ++++++++++ 
1 file changed, 10 insertions(+) 

-- 
2.20.1 



[-- Attachment #1.2: Type: text/html, Size: 1177 bytes --]

Re: [PATCH v2 0/2] Fix reproducible issues in armhf

[Uladzimir Bely]

On Wed, 2023-08-09 at 19:17 +0530, venkata.pyla@toshiba-tsip.com wrote:
> From: venkata pyla <venkata.pyla@toshiba-tsip.com>
> 
> Hi,
> 
> Updated the patch series based on the feedback.
> 
> v2 changes:
>  - Drop cleaning of blkid cache files in rootfs_postporcess_command
> and
>    do it in rootfs_finalize method.
>  - corrected the condition for scanning the *.core files
> 
> venkata pyla (2):
>   image.bbclass: remove core files generated in rootfs
>   image.bbclass: clean blkid cache files from rootfs
> 
>  meta/classes/image.bbclass | 10 ++++++++++
>  1 file changed, 10 insertions(+)
> 
> -- 
> 2.20.1
> 
> 

Applied to next, thanks.

Re: [PATCH 1/2] rootfs.bbclass: clean blkid cache files from rootfs

[MOESSBAUER, Felix]

On Mon, 2023-08-07 at 09:58 -0700, vijai kumar wrote:
> 
> 
> On Monday, August 7, 2023 at 10:05:21 PM UTC+5:30
> venkat...@toshiba-tsip.com wrote:
> > From: venkata pyla <venkat...@toshiba-tsip.com>
> > 
> > Sometimes `blkid` generates cached files `/run/blkid/blkdid.tab*`,
> > these
> > cache files are not reproducible and not necessary in the final
> > image,
> > so remove them to achieve reproducible builds.
> > 
> > Signed-off-by: venkata pyla <venkat...@toshiba-tsip.com>
> > ---
> >  meta/classes/rootfs.bbclass | 6 ++++++
> >  1 file changed, 6 insertions(+)
> > 
> > diff --git a/meta/classes/rootfs.bbclass
> > b/meta/classes/rootfs.bbclass
> > index 22449d71..5d493ea2 100644
> > --- a/meta/classes/rootfs.bbclass
> > +++ b/meta/classes/rootfs.bbclass
> > @@ -293,6 +293,12 @@ rootfs_export_dpkg_status() {
> >  '${ROOTFS_DPKGSTATUS_DEPLOY_DIR}'/'${ROOTFS_PACKAGE_SUFFIX}'.dpkg_
> > status
> >  }
> >  
> > +ROOTFS_POSTPROCESS_COMMAND +=
> > "rootfs_postprocess_clean_blkid_cache"
> > +rootfs_postprocess_clean_blkid_cache() {
> > + # blkid generates cache file which are not reproducible
> > + sudo rm -rf "${ROOTFSDIR}/run/blkid/blkid.tab"*
> > +}
> 
> 
> If this is a necessary clean up function, can we not move to
> rootfs_finalize? Or maybe adding it to ROOTFS_CLEAN_FILES?

Why is the /run dir included at all? It will be over-mounted with a
tmpfs at runtime (when using systemd) and by that we could also clear
it. For non systemd use-cases like the containers, the content of that
directory is also not needed.

For details, see
https://www.freedesktop.org/software/systemd/man/file-hierarchy.html#/run/


Felix

> 
> Thanks,
> Vijai Kumar K
>  
> > +
> >  do_rootfs_postprocess[vardeps] = "${ROOTFS_POSTPROCESS_COMMAND}"
> >  do_rootfs_postprocess[network] = "${TASK_USE_SUDO}"
> >  python do_rootfs_postprocess() {
> > -- 
> > 2.20.1
> > 
> > 

Re: [PATCH 2/2] image.bbclass: remove core files generated in rootfs

[MOESSBAUER, Felix]

On Mon, 2023-08-07 at 22:05 +0530, venkata.pyla@toshiba-tsip.com wrote:
> From: venkata pyla <venkata.pyla@toshiba-tsip.com>
> 
> Sometimes core dumps are created in rootfs and left in the final
> image,
> and that results into non-reproducible images, so move those files to
> ${WORKDIR}/temp location and warn the user.
> 
> In armhf there is an issue with qemu-user-static that results in
> creating core dumps which is reported in Debian upstream [1],
> meanwhile
> move these core dumps outside the rootfs.
> 
> [1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1040981
> 
> Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com>
> ---
>  meta/classes/image.bbclass | 6 ++++++
>  1 file changed, 6 insertions(+)
> 
> diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
> index 08b6d3d6..80509d0d 100644
> --- a/meta/classes/image.bbclass
> +++ b/meta/classes/image.bbclass
> @@ -461,6 +461,12 @@ do_rootfs_finalize() {
>          fi
>  EOSUDO
>  
> +    # Move core dump files generated in rootfs to temp folder in
> workdir.
> +    for f in ${ROOTFSDIR}/*.core; do

The name of the coredump file depends on the values of
/proc/sys/kernel/core_uses_pid and /proc/sys/kernel/core_pattern.

The stock kernel default configuration is just "core".
For further details, see [1].

[1] https://github.com/DynamoRIO/dynamorio/issues/6126

Felix

> +        sudo mv "${f}" "${WORKDIR}/temp/"
> +        bbwarn "found core dump in rootfs, moved it to
> ${WORKDIR}/temp/"
> +    done
> +
>      # Set same time-stamps to the newly generated file/folders in
> the
>      # rootfs image for the purpose of reproducible builds.
>      if [ -n "${SOURCE_DATE_EPOCH}" ]; then