Re: [isar] reproducible build failures

[Henning Schild <henning.schild@siemens.com>]

Am Fri, 3 Sep 2021 15:19:21 +0000
schrieb <Venkata.Pyla@toshiba-tsip.com>:

> Hi,
> 
> I am using isar system in isar-cip-core project [1] where I found
> some reproducible failures, which may be good to fix in the isar
> system. I am not good in modifying the isar system, so could you
> please guide me to fix these problems?
> 
> Here are the steps to check the reproducible failures in
> isar-cip-core project:
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.com%2Fcip-project%2Fcip-core%2Fisar-cip-core%2F-%2Fissues%2F12&amp;data=04%7C01%7Cde173c00-e982-4fda-8644-47edf4671d63%40ad011.siemens.com%7C186fea861bb04bae174708d96eee3bfb%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637662792787877923%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&amp;sdata=ofcsE0TXKbwL%2FPUTD2hAEQWvOLXNRFrIvunSOblUAho%3D&amp;reserved=0
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.com%2Fcip-project%2Fcip-core%2Fisar-cip-core%2F-%2Fissues%2F13&amp;data=04%7C01%7Cde173c00-e982-4fda-8644-47edf4671d63%40ad011.siemens.com%7C186fea861bb04bae174708d96eee3bfb%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637662792787877923%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&amp;sdata=M7pZHme2TzX7ZhNbaZW%2BKn26K65ZnKFW4BvnQJldLnw%3D&amp;reserved=0
> 
> I also verified the reproducibility in the isar system and found
> similar failures that are copied below:
> ============================================== tmp/gpghomefHv8eRhk43/
> tmp/gpghomefHv8eRhk43/private-keys-v1.d/
> usr/share/doc/hello/changelog.Debian.gz

I was wondering why only one package is affected here, it should be
more. Turns out that isar-exclude-docs might be a little to thorough.

I sent a patch for that
  [PATCH 3/3] isar-exclude-docs: include "native" package changelogs

And "[PATCH 1/3] debianize: make changelog date reproducible" makes
sure we always use the same date while still allowing user control and
prepending to upstream without "going back in time".

Henning

> var/cache/debconf/config.dat
> var/cache/debconf/config.dat-old
> var/cache/ldconfig/aux-cache
> var/lib/dpkg/info/enable-fsck.md5sums
> var/lib/dpkg/info/example-raw.md5sums
> var/lib/dpkg/info/hello.md5sums
> var/lib/dpkg/info/isar-disable-apt-cache.md5sums
> var/lib/dpkg/info/isar-exclude-docs.md5sums
> var/lib/dpkg/info/sshd-regen-keys.md5sums
> var/lib/initramfs-tools/4.19.0-17-amd64
> var/lib/systemd/catalog/database
> var/log/alternatives.log
> var/log/bootstrap.log
> var/log/dpkg.log
> var/log/apt/history.log
> var/log/apt/term.log
> ==============================================
> 
> Steps to check reproducible failures in isar
> ====================================
> $ . isar-init-build-env ../build1 && bitbake
> mc:qemuamd64-buster-tgz:isar-image-base $  . isar-init-build-env
> ../build2 && bitbake mc:qemuamd64-buster-tgz:isar-image-base $ mkdir
> -p rootfs1 rootfs2 $ tar -xzvf
> ./build1/tmp/deploy/images/qemuamd64/isar-image-base-debian-buster-qemuamd64.tar.gz
> -C ./rootfs1/ $ tar -xzvf
> ./build2/tmp/deploy/images/qemuamd64/isar-image-base-debian-buster-qemuamd64.tar.gz
> -C ./rootfs2/ $ rsync -nrclv ./rootfs1/ ./rootfs2/ > difference.txt
> ====================================
> 
> From the reproducible failures I found there are three different
> areas to fix these problem
> 
> 1.       Changelog file generation, which is embedding the build time
> date value at here
> (https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Filbers%2Fisar%2Fblob%2Fmaster%2Fmeta%2Fclasses%2Fdebianize.bbclass%23L34&amp;data=04%7C01%7Cde173c00-e982-4fda-8644-47edf4671d63%40ad011.siemens.com%7C186fea861bb04bae174708d96eee3bfb%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637662792787877923%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&amp;sdata=sSB%2BhM1xQauyMIju%2FlJ0GsgdBXYds2C61qpUoQPwuQw%3D&amp;reserved=0
> )
> 
> 2.       Log files generated by different application, which are
> adding build time values, I think we can remove these files if it is
> not required after build. ( I tried at here
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Filbers%2Fisar%2Fblob%2Fmaster%2Fmeta%2Fclasses%2Fimage.bbclass%23L183&amp;data=04%7C01%7Cde173c00-e982-4fda-8644-47edf4671d63%40ad011.siemens.com%7C186fea861bb04bae174708d96eee3bfb%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637662792787877923%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&amp;sdata=kUZoEeqRwUQWk8jZQ%2Bp6t3zO4czouhLQjGKh9RPB8kc%3D&amp;reserved=0
> but it did not work)
> 
> 3.       Cache and temporary files, I think we can delete these files
> also.
> 
> Please guide me to fix these issues.
> 
> Thanks,
> Venkata.