Re: [PATCH v2 1/1] image.bbclass: fix non-reproducible file time-stamps inside rootfs

["Moessbauer, Felix" <felix.moessbauer@siemens.com>]

On Thu, 2023-01-05 at 18:05 +0100, Henning Schild wrote:
> > > > https://reproducible-builds.org/docs/source-date-epoch/
> > > > +#SOURCE_DATE_EPOCH =  
> > > 
> > > ${@bb.process.run(git log ...)}
> > > 
> > > would be nice here. So once uncommented it will keep moving as
> > > people commit.  
> > 
> > Be careful with this pattern as it bypasses the Bitbake signatures,
> > leading to non-deterministic signatures errors. We had the same
> > problem with the ISAR release command. Maybe the immediate
> > expansion
> > operator works in the local conf, but I did not check that.
> 
> The value is only ever used in image.bbclass in the finalize
> function.
> So maybe not a problem to expect with sstate.

This is totally independent of sstate. Bitbake internally requires that
the values of all variables are deterministic. This is required because
the set of signatures of a task is used to decide if a task is re-
executed or not. And changing signatures without changing the "input"
is a bitbake error.

> 
> > It would be better to inject that from the outside, e.g. via KAS or
> > an
> > environment variable.
> 
> Yes but we also need to document sane values for it, maybe in the
> user
> doc after all and not the local.conf.sample.
> 
> > And in addition, we need to make sure that we
> > only forward this variable to non-sbuild tasks. The sbuild tasks
> > should get the SOURCE_DATE_EPOCH from the latest entry in the
> > changelog. However, for auto-generated changelogs, the date should
> > be
> > the value of SOURCE_DATE_EPOCH.
> 
> I sure hope this one does not go into the env of the package
> building,
> but it should be double checked.
> 
> For auto-generated changelogs we have another solution based on last
> debian changelog entry or 1.1.1970 if there is none.
>  
> https://github.com/ilbers/isar/commit/53d315fdd3f3dcf70ac3f257d9431ba522e86eb4

IMHO this pattern is a hacky workaround. As we now have the
SOURCE_DATE_EPOCH, we should also use that for the changelog.
But better not as part of this series. It is already getting too long.

@venkata: When sending a new version of the series, please do not send
it as "in-reply-to ...", as this makes it harder and harder to read it
between all the replies. Better simply send it directly.

Felix

> 
> Henning
> 
> > Felix
> > 
> > >  
> > > > diff --git a/meta/classes/image.bbclass
> > > > b/meta/classes/image.bbclass
> > > > index 813e1f3..38a9adf 100644
> > > > --- a/meta/classes/image.bbclass
> > > > +++ b/meta/classes/image.bbclass
> > > > @@ -430,6 +430,15 @@ do_rootfs_finalize() {
> > > >             
> > > > "${ROOTFSDIR}/etc/apt/sources.list.d/bootstrap.list"
> > > > 
> > > >          rm -f "${ROOTFSDIR}/etc/apt/sources-list"
> > > > +
> > > > +        # Set same time-stamps to the newly generated
> > > > file/folders in the
> > > > +        # rootfs image for the purpose of reproducible builds.
> > > > +        test ! -z "${SOURCE_DATE_EPOCH}" && \
> > > > +            find ${ROOTFSDIR} -newermt \
> > > > +                "$(date -d@${SOURCE_DATE_EPOCH} '+%Y-%m-%d
> > > > %H:%M:%S')" \
> > > > +                -printf "%y %p\n" \
> > > > +                -exec touch '{}' -h -d@${SOURCE_DATE_EPOCH}
> > > > ';' >
> > > > ${DEPLOY_DIR_IMAGE}/files.modified_timestamps +
> > > >  EOSUDO  
> > > 
> > > I would suggest to at least display a bbwarn if "wc -l" of that
> > > file
> > > exceeds some number ... say 50. I guess if SOURCE_DATE_EPOCH was
> > > too
> > > old, say 01.01.1990 the whole filesystem would be touched which
> > > might indicate a problem.
> > > 
> > > Not sure what a good number would be. We could also check for
> > > certain files to _not_ be in there for sure.
> > > 
> > > I might give that patch a try and see for myself what a too old
> > > value would do. But right now i will keep going with the
> > > expectation that it
> > > would "touch all files without big warning" and the thing might
> > > still boot but the broken metadata could cause any kind of
> > > problems
> > > in applications that can get confused by that big change.
> > > 
> > > Henning
> > >  
> > > >  }
> > > >  addtask rootfs_finalize before do_rootfs after
> > > > do_rootfs_postprocess  
> > >  
> > 
>