* [PATCH 1/3] make deb_add_changelog idempotent
2023-01-09 5:14 [PATCH 0/3] Fix reproducability issues in deb_add_changelog Felix Moessbauer
@ 2023-01-09 5:14 ` Felix Moessbauer
2023-01-09 9:19 ` Henning Schild
2023-01-09 5:14 ` [PATCH 2/3] deb_add_changelog: set timestamp to valid epoch Felix Moessbauer
` (2 subsequent siblings)
3 siblings, 1 reply; 12+ messages in thread
From: Felix Moessbauer @ 2023-01-09 5:14 UTC (permalink / raw)
To: isar-users; +Cc: jan.kiszka, henning.schild, venkata.pyla, Felix Moessbauer
Previously, the deb_add_changelog function considered an auto-generated
changelog as a base to add changes on top. This behavior is not
idempontent on subsequent invocations of the function (e.g. on partial
rebuilds). This lead to both reproducability issues, as well as unclean
changelog files having multiple "generated by ISAR" entries.
This patch changes this implementation in a way to always create a
(possibly empty) orig changelog on the first invocation. On subequent
invocations, the orig changelog is only considered as provided by the
user, if it is not empty.
Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
---
meta/classes/debianize.bbclass | 14 +++++++++-----
1 file changed, 9 insertions(+), 5 deletions(-)
diff --git a/meta/classes/debianize.bbclass b/meta/classes/debianize.bbclass
index d125256..34347b5 100644
--- a/meta/classes/debianize.bbclass
+++ b/meta/classes/debianize.bbclass
@@ -15,7 +15,9 @@ MAINTAINER ??= "Unknown maintainer <unknown@example.com>"
deb_add_changelog() {
changelog_v="${CHANGELOG_V}"
timestamp=0
- if [ -f ${S}/debian/changelog ]; then
+ # we have a changelog and that is not autogenerated
+ # (when autogenerated, changelog.orig is empty)
+ if [ -f ${S}/debian/changelog ] && [ -s ${WORKDIR}/changelog.orig ]; then
if [ ! -f ${WORKDIR}/changelog.orig ]; then
cp ${S}/debian/changelog ${WORKDIR}/changelog.orig
fi
@@ -34,10 +36,12 @@ ${PN} (${changelog_v}) UNRELEASED; urgency=low
-- ${MAINTAINER} ${date}
EOF
- if [ -f ${WORKDIR}/changelog.orig ]; then
- # prepend our entry to the original changelog
- echo >> ${S}/debian/changelog
- cat ${WORKDIR}/changelog.orig >> ${S}/debian/changelog
+ # ensure that we always start with the orig version of the
+ # changelog on repeated invocations (e.g. on partial rebuilds)
+ touch ${WORKDIR}/changelog.orig
+ # prepend our entry to the original changelog
+ echo >> ${S}/debian/changelog
+ cat ${WORKDIR}/changelog.orig >> ${S}/debian/changelog
fi
if [ -f ${WORKDIR}/changelog ]; then
--
2.34.1
^ permalink raw reply [flat|nested] 12+ messages in thread* Re: [PATCH 1/3] make deb_add_changelog idempotent
2023-01-09 5:14 ` [PATCH 1/3] make deb_add_changelog idempotent Felix Moessbauer
@ 2023-01-09 9:19 ` Henning Schild
2023-01-09 10:42 ` Moessbauer, Felix
0 siblings, 1 reply; 12+ messages in thread
From: Henning Schild @ 2023-01-09 9:19 UTC (permalink / raw)
To: Felix Moessbauer; +Cc: isar-users, jan.kiszka, venkata.pyla
Am Mon, 9 Jan 2023 05:14:26 +0000
schrieb Felix Moessbauer <felix.moessbauer@siemens.com>:
> Previously, the deb_add_changelog function considered an
> auto-generated changelog as a base to add changes on top. This
> behavior is not idempontent on subsequent invocations of the function
> (e.g. on partial rebuilds). This lead to both reproducability issues,
> as well as unclean changelog files having multiple "generated by
> ISAR" entries.
Can you provide an example of that? Like how the wrong changelog would
look like and why it would be wrong.
partial rebuilds (especially when code was modified inbetween) are hard
to argue in the context of reproducible builds.
> This patch changes this implementation in a way to always create a
> (possibly empty) orig changelog on the first invocation. On subequent
> invocations, the orig changelog is only considered as provided by the
> user, if it is not empty.
>
> Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
> ---
> meta/classes/debianize.bbclass | 14 +++++++++-----
> 1 file changed, 9 insertions(+), 5 deletions(-)
>
> diff --git a/meta/classes/debianize.bbclass
> b/meta/classes/debianize.bbclass index d125256..34347b5 100644
> --- a/meta/classes/debianize.bbclass
> +++ b/meta/classes/debianize.bbclass
> @@ -15,7 +15,9 @@ MAINTAINER ??= "Unknown maintainer
> <unknown@example.com>" deb_add_changelog() {
> changelog_v="${CHANGELOG_V}"
> timestamp=0
> - if [ -f ${S}/debian/changelog ]; then
> + # we have a changelog and that is not autogenerated
> + # (when autogenerated, changelog.orig is empty)
> + if [ -f ${S}/debian/changelog ] && [ -s
> ${WORKDIR}/changelog.orig ]; then if [ ! -f ${WORKDIR}/changelog.orig
can that condition inside the condition still be met?
Henning
> ]; then cp ${S}/debian/changelog ${WORKDIR}/changelog.orig
> fi
> @@ -34,10 +36,12 @@ ${PN} (${changelog_v}) UNRELEASED; urgency=low
>
> -- ${MAINTAINER} ${date}
> EOF
> - if [ -f ${WORKDIR}/changelog.orig ]; then
> - # prepend our entry to the original changelog
> - echo >> ${S}/debian/changelog
> - cat ${WORKDIR}/changelog.orig >>
> ${S}/debian/changelog
> + # ensure that we always start with the orig version of the
> + # changelog on repeated invocations (e.g. on partial
> rebuilds)
> + touch ${WORKDIR}/changelog.orig
> + # prepend our entry to the original changelog
> + echo >> ${S}/debian/changelog
> + cat ${WORKDIR}/changelog.orig >> ${S}/debian/changelog
> fi
>
> if [ -f ${WORKDIR}/changelog ]; then
^ permalink raw reply [flat|nested] 12+ messages in thread* Re: [PATCH 1/3] make deb_add_changelog idempotent
2023-01-09 9:19 ` Henning Schild
@ 2023-01-09 10:42 ` Moessbauer, Felix
0 siblings, 0 replies; 12+ messages in thread
From: Moessbauer, Felix @ 2023-01-09 10:42 UTC (permalink / raw)
To: Schild, Henning; +Cc: isar-users, Kiszka, Jan, venkata.pyla
On Mon, 2023-01-09 at 10:19 +0100, Henning Schild wrote:
> Am Mon, 9 Jan 2023 05:14:26 +0000
> schrieb Felix Moessbauer <felix.moessbauer@siemens.com>:
>
> > Previously, the deb_add_changelog function considered an
> > auto-generated changelog as a base to add changes on top. This
> > behavior is not idempontent on subsequent invocations of the
> > function
> > (e.g. on partial rebuilds). This lead to both reproducability
> > issues,
> > as well as unclean changelog files having multiple "generated by
> > ISAR" entries.
>
> Can you provide an example of that? Like how the wrong changelog
> would
> look like and why it would be wrong.
Consider this changelog file:
[...]
reprotest (1.0) UNRELEASED; urgency=low
* generated by Isar
-- Unknown maintainer <unknown@example.com> Thu, 01 Jan 1970 00:00:42
+0000
reprotest (1.0) UNRELEASED; urgency=low
* generated by Isar
-- Unknown maintainer <unknown@example.com> Thu, 01 Jan 1970 00:00:00
+0000
>
> partial rebuilds (especially when code was modified inbetween) are
> hard
> to argue in the context of reproducible builds.
>
> > This patch changes this implementation in a way to always create a
> > (possibly empty) orig changelog on the first invocation. On
> > subequent
> > invocations, the orig changelog is only considered as provided by
> > the
> > user, if it is not empty.
> >
> > Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
> > ---
> > meta/classes/debianize.bbclass | 14 +++++++++-----
> > 1 file changed, 9 insertions(+), 5 deletions(-)
> >
> > diff --git a/meta/classes/debianize.bbclass
> > b/meta/classes/debianize.bbclass index d125256..34347b5 100644
> > --- a/meta/classes/debianize.bbclass
> > +++ b/meta/classes/debianize.bbclass
> > @@ -15,7 +15,9 @@ MAINTAINER ??= "Unknown maintainer
> > <unknown@example.com>" deb_add_changelog() {
> > changelog_v="${CHANGELOG_V}"
> > timestamp=0
> > - if [ -f ${S}/debian/changelog ]; then
> > + # we have a changelog and that is not autogenerated
> > + # (when autogenerated, changelog.orig is empty)
> > + if [ -f ${S}/debian/changelog ] && [ -s
> > ${WORKDIR}/changelog.orig ]; then if [ ! -f
> > ${WORKDIR}/changelog.orig
>
> can that condition inside the condition still be met?
Yes, this is met when a custom changelog file is provided and we run
the deb_add_changelog function for the first time. There, no
changelog.orig exists yet.
Felix
>
> Henning
>
> > ]; then cp ${S}/debian/changelog ${WORKDIR}/changelog.orig
> > fi
> > @@ -34,10 +36,12 @@ ${PN} (${changelog_v}) UNRELEASED; urgency=low
> >
> > -- ${MAINTAINER} ${date}
> > EOF
> > - if [ -f ${WORKDIR}/changelog.orig ]; then
> > - # prepend our entry to the original changelog
> > - echo >> ${S}/debian/changelog
> > - cat ${WORKDIR}/changelog.orig >>
> > ${S}/debian/changelog
> > + # ensure that we always start with the orig version of the
> > + # changelog on repeated invocations (e.g. on partial
> > rebuilds)
> > + touch ${WORKDIR}/changelog.orig
> > + # prepend our entry to the original changelog
> > + echo >> ${S}/debian/changelog
> > + cat ${WORKDIR}/changelog.orig >> ${S}/debian/changelog
> > fi
> >
> > if [ -f ${WORKDIR}/changelog ]; then
>
^ permalink raw reply [flat|nested] 12+ messages in thread
* [PATCH 2/3] deb_add_changelog: set timestamp to valid epoch
2023-01-09 5:14 [PATCH 0/3] Fix reproducability issues in deb_add_changelog Felix Moessbauer
2023-01-09 5:14 ` [PATCH 1/3] make deb_add_changelog idempotent Felix Moessbauer
@ 2023-01-09 5:14 ` Felix Moessbauer
2023-01-09 5:14 ` [PATCH 3/3] deb_add_changelog: use SOURCE_DATE_EPOCH Felix Moessbauer
2023-01-09 9:28 ` [PATCH 0/3] Fix reproducability issues in deb_add_changelog Henning Schild
3 siblings, 0 replies; 12+ messages in thread
From: Felix Moessbauer @ 2023-01-09 5:14 UTC (permalink / raw)
To: isar-users; +Cc: jan.kiszka, henning.schild, venkata.pyla, Felix Moessbauer
A changelog date of 0 (unix timestamp) is not considered a valid
timestamp for the SOURCE_DATE_EPOCH. By that, the debhelper scripts
set the SOURCE_DATE_EPOCH variable to the current time of the build,
breaking reproducability. By that, we get an inconsistency between the
debian changelog timestamp and the timestamp that the build tools encode
into the binary and the file timestamps.
Without having support to control the SOURCE_DATE_EPOCH variable
externally via bitbake, this always led to non-reproducible packages.
To fix this, we simply set the default timestamp to 1h later.
Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
---
meta/classes/debianize.bbclass | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/meta/classes/debianize.bbclass b/meta/classes/debianize.bbclass
index 34347b5..3531421 100644
--- a/meta/classes/debianize.bbclass
+++ b/meta/classes/debianize.bbclass
@@ -14,7 +14,7 @@ MAINTAINER ??= "Unknown maintainer <unknown@example.com>"
deb_add_changelog() {
changelog_v="${CHANGELOG_V}"
- timestamp=0
+ timestamp=3600
# we have a changelog and that is not autogenerated
# (when autogenerated, changelog.orig is empty)
if [ -f ${S}/debian/changelog ] && [ -s ${WORKDIR}/changelog.orig ]; then
--
2.34.1
^ permalink raw reply [flat|nested] 12+ messages in thread* [PATCH 3/3] deb_add_changelog: use SOURCE_DATE_EPOCH
2023-01-09 5:14 [PATCH 0/3] Fix reproducability issues in deb_add_changelog Felix Moessbauer
2023-01-09 5:14 ` [PATCH 1/3] make deb_add_changelog idempotent Felix Moessbauer
2023-01-09 5:14 ` [PATCH 2/3] deb_add_changelog: set timestamp to valid epoch Felix Moessbauer
@ 2023-01-09 5:14 ` Felix Moessbauer
2023-01-09 5:34 ` Jan Kiszka
2023-01-09 9:25 ` Henning Schild
2023-01-09 9:28 ` [PATCH 0/3] Fix reproducability issues in deb_add_changelog Henning Schild
3 siblings, 2 replies; 12+ messages in thread
From: Felix Moessbauer @ 2023-01-09 5:14 UTC (permalink / raw)
To: isar-users; +Cc: jan.kiszka, henning.schild, venkata.pyla, Felix Moessbauer
In case the SOURCE_DATE_EPOCH bb variable is set, use that value
both for the auto-generated changelog as well as when appending to
an existing changelog.
Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
---
meta/classes/debianize.bbclass | 11 ++++++-----
1 file changed, 6 insertions(+), 5 deletions(-)
diff --git a/meta/classes/debianize.bbclass b/meta/classes/debianize.bbclass
index 3531421..49ae0dd 100644
--- a/meta/classes/debianize.bbclass
+++ b/meta/classes/debianize.bbclass
@@ -14,7 +14,7 @@ MAINTAINER ??= "Unknown maintainer <unknown@example.com>"
deb_add_changelog() {
changelog_v="${CHANGELOG_V}"
- timestamp=3600
+ timestamp=${@ d.getVar('SOURCE_DATE_EPOCH', True) or '3600' }
# we have a changelog and that is not autogenerated
# (when autogenerated, changelog.orig is empty)
if [ -f ${S}/debian/changelog ] && [ -s ${WORKDIR}/changelog.orig ]; then
@@ -23,9 +23,11 @@ deb_add_changelog() {
fi
orig_version=$(dpkg-parsechangelog -l ${WORKDIR}/changelog.orig -S Version)
changelog_v=$(echo "${changelog_v}" | sed 's/<orig-version>/'${orig_version}'/')
- orig_date=$(dpkg-parsechangelog -l ${WORKDIR}/changelog.orig -S Date)
- orig_seconds=$(date --date="${orig_date}" +'%s')
- timestamp=$(expr ${orig_seconds} + 42)
+ if [ -z "${SOURCE_DATE_EPOCH}" ]; then
+ orig_date=$(dpkg-parsechangelog -l ${WORKDIR}/changelog.orig -S Date)
+ orig_seconds=$(date --date="${orig_date}" +'%s')
+ timestamp=$(expr ${orig_seconds} + 42)
+ fi
fi
date=$(LANG=C date -R -d @${timestamp})
@@ -42,7 +44,6 @@ EOF
# prepend our entry to the original changelog
echo >> ${S}/debian/changelog
cat ${WORKDIR}/changelog.orig >> ${S}/debian/changelog
- fi
if [ -f ${WORKDIR}/changelog ]; then
latest_version=$(dpkg-parsechangelog -l ${WORKDIR}/changelog -S Version)
--
2.34.1
^ permalink raw reply [flat|nested] 12+ messages in thread* Re: [PATCH 3/3] deb_add_changelog: use SOURCE_DATE_EPOCH
2023-01-09 5:14 ` [PATCH 3/3] deb_add_changelog: use SOURCE_DATE_EPOCH Felix Moessbauer
@ 2023-01-09 5:34 ` Jan Kiszka
2023-01-09 9:25 ` Henning Schild
1 sibling, 0 replies; 12+ messages in thread
From: Jan Kiszka @ 2023-01-09 5:34 UTC (permalink / raw)
To: Felix Moessbauer, isar-users; +Cc: henning.schild, venkata.pyla
On 09.01.23 06:14, Felix Moessbauer wrote:
> In case the SOURCE_DATE_EPOCH bb variable is set, use that value
> both for the auto-generated changelog as well as when appending to
> an existing changelog.
>
> Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
> ---
> meta/classes/debianize.bbclass | 11 ++++++-----
> 1 file changed, 6 insertions(+), 5 deletions(-)
>
> diff --git a/meta/classes/debianize.bbclass b/meta/classes/debianize.bbclass
> index 3531421..49ae0dd 100644
> --- a/meta/classes/debianize.bbclass
> +++ b/meta/classes/debianize.bbclass
> @@ -14,7 +14,7 @@ MAINTAINER ??= "Unknown maintainer <unknown@example.com>"
>
> deb_add_changelog() {
> changelog_v="${CHANGELOG_V}"
> - timestamp=3600
> + timestamp=${@ d.getVar('SOURCE_DATE_EPOCH', True) or '3600' }
> # we have a changelog and that is not autogenerated
> # (when autogenerated, changelog.orig is empty)
> if [ -f ${S}/debian/changelog ] && [ -s ${WORKDIR}/changelog.orig ]; then
> @@ -23,9 +23,11 @@ deb_add_changelog() {
> fi
> orig_version=$(dpkg-parsechangelog -l ${WORKDIR}/changelog.orig -S Version)
> changelog_v=$(echo "${changelog_v}" | sed 's/<orig-version>/'${orig_version}'/')
> - orig_date=$(dpkg-parsechangelog -l ${WORKDIR}/changelog.orig -S Date)
> - orig_seconds=$(date --date="${orig_date}" +'%s')
> - timestamp=$(expr ${orig_seconds} + 42)
> + if [ -z "${SOURCE_DATE_EPOCH}" ]; then
> + orig_date=$(dpkg-parsechangelog -l ${WORKDIR}/changelog.orig -S Date)
> + orig_seconds=$(date --date="${orig_date}" +'%s')
> + timestamp=$(expr ${orig_seconds} + 42)
> + fi
> fi
>
> date=$(LANG=C date -R -d @${timestamp})
> @@ -42,7 +44,6 @@ EOF
> # prepend our entry to the original changelog
> echo >> ${S}/debian/changelog
> cat ${WORKDIR}/changelog.orig >> ${S}/debian/changelog
> - fi
This hunk should go into patch 1, I suspect...
Jan
>
> if [ -f ${WORKDIR}/changelog ]; then
> latest_version=$(dpkg-parsechangelog -l ${WORKDIR}/changelog -S Version)
--
Siemens AG, Technology
Competence Center Embedded Linux
^ permalink raw reply [flat|nested] 12+ messages in thread* Re: [PATCH 3/3] deb_add_changelog: use SOURCE_DATE_EPOCH
2023-01-09 5:14 ` [PATCH 3/3] deb_add_changelog: use SOURCE_DATE_EPOCH Felix Moessbauer
2023-01-09 5:34 ` Jan Kiszka
@ 2023-01-09 9:25 ` Henning Schild
2023-01-09 10:24 ` Moessbauer, Felix
2023-01-09 10:30 ` Jan Kiszka
1 sibling, 2 replies; 12+ messages in thread
From: Henning Schild @ 2023-01-09 9:25 UTC (permalink / raw)
To: Felix Moessbauer; +Cc: isar-users, jan.kiszka, venkata.pyla
Am Mon, 9 Jan 2023 05:14:28 +0000
schrieb Felix Moessbauer <felix.moessbauer@siemens.com>:
> In case the SOURCE_DATE_EPOCH bb variable is set, use that value
> both for the auto-generated changelog as well as when appending to
> an existing changelog.
The variable SOURCE_DATE_EPOCH does not make it into the build
environment and it very likely should not because the overall value for
a single image must not be used for all packages. So we can not and
should not use it. For a package the sane value would likely be the
newest changelog entry ... like we try and do already.
Henning
> Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
> ---
> meta/classes/debianize.bbclass | 11 ++++++-----
> 1 file changed, 6 insertions(+), 5 deletions(-)
>
> diff --git a/meta/classes/debianize.bbclass
> b/meta/classes/debianize.bbclass index 3531421..49ae0dd 100644
> --- a/meta/classes/debianize.bbclass
> +++ b/meta/classes/debianize.bbclass
> @@ -14,7 +14,7 @@ MAINTAINER ??= "Unknown maintainer
> <unknown@example.com>"
> deb_add_changelog() {
> changelog_v="${CHANGELOG_V}"
> - timestamp=3600
> + timestamp=${@ d.getVar('SOURCE_DATE_EPOCH', True) or '3600' }
> # we have a changelog and that is not autogenerated
> # (when autogenerated, changelog.orig is empty)
> if [ -f ${S}/debian/changelog ] && [ -s
> ${WORKDIR}/changelog.orig ]; then @@ -23,9 +23,11 @@
> deb_add_changelog() { fi
> orig_version=$(dpkg-parsechangelog -l
> ${WORKDIR}/changelog.orig -S Version) changelog_v=$(echo
> "${changelog_v}" | sed 's/<orig-version>/'${orig_version}'/')
> - orig_date=$(dpkg-parsechangelog -l
> ${WORKDIR}/changelog.orig -S Date)
> - orig_seconds=$(date --date="${orig_date}" +'%s')
> - timestamp=$(expr ${orig_seconds} + 42)
> + if [ -z "${SOURCE_DATE_EPOCH}" ]; then
> + orig_date=$(dpkg-parsechangelog -l
> ${WORKDIR}/changelog.orig -S Date)
> + orig_seconds=$(date --date="${orig_date}"
> +'%s')
> + timestamp=$(expr ${orig_seconds} + 42)
> + fi
> fi
>
> date=$(LANG=C date -R -d @${timestamp})
> @@ -42,7 +44,6 @@ EOF
> # prepend our entry to the original changelog
> echo >> ${S}/debian/changelog
> cat ${WORKDIR}/changelog.orig >> ${S}/debian/changelog
> - fi
>
> if [ -f ${WORKDIR}/changelog ]; then
> latest_version=$(dpkg-parsechangelog -l
> ${WORKDIR}/changelog -S Version)
^ permalink raw reply [flat|nested] 12+ messages in thread* Re: [PATCH 3/3] deb_add_changelog: use SOURCE_DATE_EPOCH
2023-01-09 9:25 ` Henning Schild
@ 2023-01-09 10:24 ` Moessbauer, Felix
2023-01-09 10:30 ` Jan Kiszka
1 sibling, 0 replies; 12+ messages in thread
From: Moessbauer, Felix @ 2023-01-09 10:24 UTC (permalink / raw)
To: Schild, Henning; +Cc: isar-users, Kiszka, Jan, venkata.pyla
On Mon, 2023-01-09 at 10:25 +0100, Henning Schild wrote:
> Am Mon, 9 Jan 2023 05:14:28 +0000
> schrieb Felix Moessbauer <felix.moessbauer@siemens.com>:
>
> > In case the SOURCE_DATE_EPOCH bb variable is set, use that value
> > both for the auto-generated changelog as well as when appending to
> > an existing changelog.
>
> The variable SOURCE_DATE_EPOCH does not make it into the build
> environment and it very likely should not because the overall value
> for
> a single image must not be used for all packages. So we can not and
> should not use it. For a package the sane value would likely be the
> newest changelog entry ... like we try and do already.
I have to disagree. The SOURCE_DATE_EPOCH is only used to create the
timestamp in the debian changelog, as it is the best information we
have. And it is only used when we modify (or create) the changelog.
This is a prime example for what this variable is for: Adding a
timestamp where otherwise no meaningful value is available. Artificial
values are always dangerous.
Inside the sbuild environment, the SOURCE_DATE_EPOCH is defined and
exported by the debhelper scripts and points to the top most changelog
entry. For details, see /usr/share/dpkg/pkg-info.mk
There, indeed the SOURCE_DATE_EPOCH **bitbake** variable is not used
and also should not be used.
Felix
>
> Henning
>
> > Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
> > ---
> > meta/classes/debianize.bbclass | 11 ++++++-----
> > 1 file changed, 6 insertions(+), 5 deletions(-)
> >
> > diff --git a/meta/classes/debianize.bbclass
> > b/meta/classes/debianize.bbclass index 3531421..49ae0dd 100644
> > --- a/meta/classes/debianize.bbclass
> > +++ b/meta/classes/debianize.bbclass
> > @@ -14,7 +14,7 @@ MAINTAINER ??= "Unknown maintainer
> > <unknown@example.com>"
> > deb_add_changelog() {
> > changelog_v="${CHANGELOG_V}"
> > - timestamp=3600
> > + timestamp=${@ d.getVar('SOURCE_DATE_EPOCH', True) or '3600'
> > }
> > # we have a changelog and that is not autogenerated
> > # (when autogenerated, changelog.orig is empty)
> > if [ -f ${S}/debian/changelog ] && [ -s
> > ${WORKDIR}/changelog.orig ]; then @@ -23,9 +23,11 @@
> > deb_add_changelog() { fi
> > orig_version=$(dpkg-parsechangelog -l
> > ${WORKDIR}/changelog.orig -S Version) changelog_v=$(echo
> > "${changelog_v}" | sed 's/<orig-version>/'${orig_version}'/')
> > - orig_date=$(dpkg-parsechangelog -l
> > ${WORKDIR}/changelog.orig -S Date)
> > - orig_seconds=$(date --date="${orig_date}" +'%s')
> > - timestamp=$(expr ${orig_seconds} + 42)
> > + if [ -z "${SOURCE_DATE_EPOCH}" ]; then
> > + orig_date=$(dpkg-parsechangelog -l
> > ${WORKDIR}/changelog.orig -S Date)
> > + orig_seconds=$(date --date="${orig_date}"
> > +'%s')
> > + timestamp=$(expr ${orig_seconds} + 42)
> > + fi
> > fi
> >
> > date=$(LANG=C date -R -d @${timestamp})
> > @@ -42,7 +44,6 @@ EOF
> > # prepend our entry to the original changelog
> > echo >> ${S}/debian/changelog
> > cat ${WORKDIR}/changelog.orig >> ${S}/debian/changelog
> > - fi
> >
> > if [ -f ${WORKDIR}/changelog ]; then
> > latest_version=$(dpkg-parsechangelog -l
> > ${WORKDIR}/changelog -S Version)
>
^ permalink raw reply [flat|nested] 12+ messages in thread* Re: [PATCH 3/3] deb_add_changelog: use SOURCE_DATE_EPOCH
2023-01-09 9:25 ` Henning Schild
2023-01-09 10:24 ` Moessbauer, Felix
@ 2023-01-09 10:30 ` Jan Kiszka
1 sibling, 0 replies; 12+ messages in thread
From: Jan Kiszka @ 2023-01-09 10:30 UTC (permalink / raw)
To: Henning Schild, Felix Moessbauer; +Cc: isar-users, venkata.pyla
On 09.01.23 10:25, Henning Schild wrote:
> Am Mon, 9 Jan 2023 05:14:28 +0000
> schrieb Felix Moessbauer <felix.moessbauer@siemens.com>:
>
>> In case the SOURCE_DATE_EPOCH bb variable is set, use that value
>> both for the auto-generated changelog as well as when appending to
>> an existing changelog.
>
> The variable SOURCE_DATE_EPOCH does not make it into the build
> environment and it very likely should not because the overall value for
> a single image must not be used for all packages. So we can not and
> should not use it. For a package the sane value would likely be the
> newest changelog entry ... like we try and do already.
No, SOURCE_DATE_EPOCH will be a global variable in the end, for all
recipes (that can/want to make use of it).
Jan
--
Siemens AG, Technology
Competence Center Embedded Linux
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH 0/3] Fix reproducability issues in deb_add_changelog
2023-01-09 5:14 [PATCH 0/3] Fix reproducability issues in deb_add_changelog Felix Moessbauer
` (2 preceding siblings ...)
2023-01-09 5:14 ` [PATCH 3/3] deb_add_changelog: use SOURCE_DATE_EPOCH Felix Moessbauer
@ 2023-01-09 9:28 ` Henning Schild
2023-01-09 10:33 ` Moessbauer, Felix
3 siblings, 1 reply; 12+ messages in thread
From: Henning Schild @ 2023-01-09 9:28 UTC (permalink / raw)
To: Felix Moessbauer; +Cc: isar-users, jan.kiszka, venkata.pyla
Maybe you explain what exactly can go wrong. All i see is the 0 for
when we create the changelog ... move the +=42 a few lines down to
cover create and prepend ... done?
partial rebuilds and reproducible builds is a tricky topic. Is it done
because we value the cache ... or because someone changed the code in
the meantime? Cache being valid, any sort of code change of cause
invalid.
Henning
Am Mon, 9 Jan 2023 05:14:25 +0000
schrieb Felix Moessbauer <felix.moessbauer@siemens.com>:
> This series fixes both reproducability and rebuild issues when using
> the deb_add_changelog function.
>
> p1, p2: fix the reproducability issue
> p3: add support for SOURCE_DATE_EPOCH
>
> Best regards,
> Felix
>
> Felix Moessbauer (3):
> make deb_add_changelog idempotent
> deb_add_changelog: set timestamp to valid epoch
> deb_add_changelog: use SOURCE_DATE_EPOCH
>
> meta/classes/debianize.bbclass | 25 +++++++++++++++----------
> 1 file changed, 15 insertions(+), 10 deletions(-)
>
^ permalink raw reply [flat|nested] 12+ messages in thread* Re: [PATCH 0/3] Fix reproducability issues in deb_add_changelog
2023-01-09 9:28 ` [PATCH 0/3] Fix reproducability issues in deb_add_changelog Henning Schild
@ 2023-01-09 10:33 ` Moessbauer, Felix
0 siblings, 0 replies; 12+ messages in thread
From: Moessbauer, Felix @ 2023-01-09 10:33 UTC (permalink / raw)
To: Schild, Henning; +Cc: isar-users, Kiszka, Jan, venkata.pyla
On Mon, 2023-01-09 at 10:28 +0100, Henning Schild wrote:
> Maybe you explain what exactly can go wrong. All i see is the 0 for
> when we create the changelog ... move the +=42 a few lines down to
> cover create and prepend ... done?
Here it depends on the exact definition of reproducability.
While the issue discussed in p2 could indeed be solved by the +=42
(which IMHO is a bad design anyways), the more severe issue of
accumulation of changelog entries is not solved.
Previously to p1, whenever the build failed in do_prepare_build (or
technically wherever deb_add_changelog is transitively used), we got a
new changelog entry. By that, I saw changelogs with dozens of ISAR
created changelog entries on top of each other.
>
> partial rebuilds and reproducible builds is a tricky topic. Is it
> done
> because we value the cache ... or because someone changed the code in
> the meantime? Cache being valid, any sort of code change of cause
> invalid.
In general all caches should be fully transparent. That is also why it
is extremely important to get (bit-by-bit) reproducible builds as
otherwise the Bitbake signatures (which are used for the SSTATE) and
the state of the packages diverges. This is a concept we inherit from
Yocto and we should really try to not break it.
Partial rebuilds have the problem, that the input of a task might
change over time when cleanup mechanisms like [cleandirs] are not used
properly. This is more an annoying kind of thing, but not too relevant
for product builds. Anyways, we should try to at least fix the most
obvious cases.
Felix
>
> Henning
>
> Am Mon, 9 Jan 2023 05:14:25 +0000
> schrieb Felix Moessbauer <felix.moessbauer@siemens.com>:
>
> > This series fixes both reproducability and rebuild issues when
> > using
> > the deb_add_changelog function.
> >
> > p1, p2: fix the reproducability issue
> > p3: add support for SOURCE_DATE_EPOCH
> >
> > Best regards,
> > Felix
> >
> > Felix Moessbauer (3):
> > make deb_add_changelog idempotent
> > deb_add_changelog: set timestamp to valid epoch
> > deb_add_changelog: use SOURCE_DATE_EPOCH
> >
> > meta/classes/debianize.bbclass | 25 +++++++++++++++----------
> > 1 file changed, 15 insertions(+), 10 deletions(-)
> >
>
^ permalink raw reply [flat|nested] 12+ messages in thread