* [PATCH 1/1] add support to set KBUILD_BUILD_TIMESTAMP for custom kernels @ 2022-03-21 14:40 Felix Moessbauer 2022-03-21 17:18 ` Jan Kiszka 0 siblings, 1 reply; 8+ messages in thread From: Felix Moessbauer @ 2022-03-21 14:40 UTC (permalink / raw) To: isar-users; +Cc: jan.kiszka, Felix Moessbauer This patch adds the infrastructure to overwrite the KBUILD_BUILD_TIMESTAMP environment variable for custom kernel builds. By that, we improve the support for reproducible kernel builds. Setting this to a fixed string further helps to profit from ccache caching on repeated builds. Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com> --- meta/recipes-kernel/linux/files/debian/isar/build.tmpl | 1 + meta/recipes-kernel/linux/files/debian/isar/common.tmpl | 2 ++ meta/recipes-kernel/linux/linux-custom.inc | 3 +++ 3 files changed, 6 insertions(+) diff --git a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl index b7749cec..32d2eb23 100644 --- a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl +++ b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl @@ -34,6 +34,7 @@ print_settings() { # --------------- # ARCH=${ARCH} # CROSS_COMPILE=${CROSS_COMPILE} +# KBUILD_BUILD_TIMESTAMP=${KBUILD_BUILD_TIMESTAMP} EOF } diff --git a/meta/recipes-kernel/linux/files/debian/isar/common.tmpl b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl index b7752fb6..ffe84745 100644 --- a/meta/recipes-kernel/linux/files/debian/isar/common.tmpl +++ b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl @@ -7,6 +7,7 @@ ARCH=${KERNEL_ARCH} KERNEL_PKG_IMAGE=linux-image-${KERNEL_NAME_PROVIDED} KERNEL_PKG_KERN_HEADERS=linux-headers-${KERNEL_NAME_PROVIDED} KERNEL_PKG_LIBC_HEADERS=linux-libc-dev +KBUILD_BUILD_TIMESTAMP="${KBUILD_BUILD_TIMESTAMP}" # Constants KCONF=.config @@ -36,6 +37,7 @@ main() { # variables to be exported export ARCH + export KBUILD_BUILD_TIMESTAMP # are we cross-compiling? BUILD_ARCH=$(dpkg-architecture -qDEB_BUILD_ARCH) diff --git a/meta/recipes-kernel/linux/linux-custom.inc b/meta/recipes-kernel/linux/linux-custom.inc index 8a5dbf82..f8e87453 100644 --- a/meta/recipes-kernel/linux/linux-custom.inc +++ b/meta/recipes-kernel/linux/linux-custom.inc @@ -33,6 +33,8 @@ KERNEL_HEADERS_DEBIAN_DEPENDS ?= "libc6, \ KERNEL_LIBC_DEV_DEPLOY ?= "0" +KBUILD_BUILD_TIMESTAMP ??= "LANG=C $(date)" + # Settings that may be changed on a per distro, machine or layer basis # -------------------------------------------------------------------- @@ -73,6 +75,7 @@ TEMPLATE_VARS += " \ LINUX_VERSION_EXTENSION \ KERNEL_NAME_PROVIDED \ KERNEL_CONFIG_FRAGMENTS \ + KBUILD_BUILD_TIMESTAMP \ " inherit dpkg -- 2.30.2 ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH 1/1] add support to set KBUILD_BUILD_TIMESTAMP for custom kernels 2022-03-21 14:40 [PATCH 1/1] add support to set KBUILD_BUILD_TIMESTAMP for custom kernels Felix Moessbauer @ 2022-03-21 17:18 ` Jan Kiszka 2022-03-21 17:33 ` Moessbauer, Felix 0 siblings, 1 reply; 8+ messages in thread From: Jan Kiszka @ 2022-03-21 17:18 UTC (permalink / raw) To: Felix Moessbauer, isar-users On 21.03.22 15:40, Felix Moessbauer wrote: > This patch adds the infrastructure to overwrite the > KBUILD_BUILD_TIMESTAMP environment variable for custom kernel > builds. By that, we improve the support for reproducible > kernel builds. > Agreed. > Setting this to a fixed string further helps to profit from > ccache caching on repeated builds. Hmm, but the practical impact of this variable should have been limited /wrt ccache. I was seeing significant build-time reduction before the commit already. > > Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com> > --- > meta/recipes-kernel/linux/files/debian/isar/build.tmpl | 1 + > meta/recipes-kernel/linux/files/debian/isar/common.tmpl | 2 ++ > meta/recipes-kernel/linux/linux-custom.inc | 3 +++ > 3 files changed, 6 insertions(+) > > diff --git a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl > index b7749cec..32d2eb23 100644 > --- a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl > +++ b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl > @@ -34,6 +34,7 @@ print_settings() { > # --------------- > # ARCH=${ARCH} > # CROSS_COMPILE=${CROSS_COMPILE} > +# KBUILD_BUILD_TIMESTAMP=${KBUILD_BUILD_TIMESTAMP} > EOF > } > > diff --git a/meta/recipes-kernel/linux/files/debian/isar/common.tmpl b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl > index b7752fb6..ffe84745 100644 > --- a/meta/recipes-kernel/linux/files/debian/isar/common.tmpl > +++ b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl > @@ -7,6 +7,7 @@ ARCH=${KERNEL_ARCH} > KERNEL_PKG_IMAGE=linux-image-${KERNEL_NAME_PROVIDED} > KERNEL_PKG_KERN_HEADERS=linux-headers-${KERNEL_NAME_PROVIDED} > KERNEL_PKG_LIBC_HEADERS=linux-libc-dev > +KBUILD_BUILD_TIMESTAMP="${KBUILD_BUILD_TIMESTAMP}" > > # Constants > KCONF=.config > @@ -36,6 +37,7 @@ main() { > > # variables to be exported > export ARCH > + export KBUILD_BUILD_TIMESTAMP > > # are we cross-compiling? > BUILD_ARCH=$(dpkg-architecture -qDEB_BUILD_ARCH) > diff --git a/meta/recipes-kernel/linux/linux-custom.inc b/meta/recipes-kernel/linux/linux-custom.inc > index 8a5dbf82..f8e87453 100644 > --- a/meta/recipes-kernel/linux/linux-custom.inc > +++ b/meta/recipes-kernel/linux/linux-custom.inc > @@ -33,6 +33,8 @@ KERNEL_HEADERS_DEBIAN_DEPENDS ?= "libc6, \ > > KERNEL_LIBC_DEV_DEPLOY ?= "0" > > +KBUILD_BUILD_TIMESTAMP ??= "LANG=C $(date)" Can't we use SOURCE_DATE_EPOCH like the rest? > + > # Settings that may be changed on a per distro, machine or layer basis > # -------------------------------------------------------------------- > > @@ -73,6 +75,7 @@ TEMPLATE_VARS += " \ > LINUX_VERSION_EXTENSION \ > KERNEL_NAME_PROVIDED \ > KERNEL_CONFIG_FRAGMENTS \ > + KBUILD_BUILD_TIMESTAMP \ > " > > inherit dpkg Jan -- Siemens AG, Technology Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 8+ messages in thread
* RE: [PATCH 1/1] add support to set KBUILD_BUILD_TIMESTAMP for custom kernels 2022-03-21 17:18 ` Jan Kiszka @ 2022-03-21 17:33 ` Moessbauer, Felix 2022-03-21 17:51 ` Jan Kiszka 0 siblings, 1 reply; 8+ messages in thread From: Moessbauer, Felix @ 2022-03-21 17:33 UTC (permalink / raw) To: jan.kiszka, isar-users Hi Jan, > -----Original Message----- > From: Kiszka, Jan (T CED) <jan.kiszka@siemens.com> > Sent: Monday, March 21, 2022 6:18 PM > To: Moessbauer, Felix (T CED SES-DE) <felix.moessbauer@siemens.com>; isar- > users@googlegroups.com > Subject: Re: [PATCH 1/1] add support to set KBUILD_BUILD_TIMESTAMP for > custom kernels > > On 21.03.22 15:40, Felix Moessbauer wrote: > > This patch adds the infrastructure to overwrite the > > KBUILD_BUILD_TIMESTAMP environment variable for custom kernel builds. > > By that, we improve the support for reproducible kernel builds. > > > > Agreed. > > > Setting this to a fixed string further helps to profit from ccache > > caching on repeated builds. > > Hmm, but the practical impact of this variable should have been limited /wrt > ccache. I was seeing significant build-time reduction before the commit already. Yes, but of course depends on configuration. > > > > > Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com> > > --- > > meta/recipes-kernel/linux/files/debian/isar/build.tmpl | 1 + > > meta/recipes-kernel/linux/files/debian/isar/common.tmpl | 2 ++ > > meta/recipes-kernel/linux/linux-custom.inc | 3 +++ > > 3 files changed, 6 insertions(+) > > > > diff --git a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl > > b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl > > index b7749cec..32d2eb23 100644 > > --- a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl > > +++ b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl > > @@ -34,6 +34,7 @@ print_settings() { > > # --------------- > > # ARCH=${ARCH} > > # CROSS_COMPILE=${CROSS_COMPILE} > > +# KBUILD_BUILD_TIMESTAMP=${KBUILD_BUILD_TIMESTAMP} > > EOF > > } > > > > diff --git a/meta/recipes-kernel/linux/files/debian/isar/common.tmpl > > b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl > > index b7752fb6..ffe84745 100644 > > --- a/meta/recipes-kernel/linux/files/debian/isar/common.tmpl > > +++ b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl > > @@ -7,6 +7,7 @@ ARCH=${KERNEL_ARCH} > > KERNEL_PKG_IMAGE=linux-image-${KERNEL_NAME_PROVIDED} > > KERNEL_PKG_KERN_HEADERS=linux-headers-${KERNEL_NAME_PROVIDED} > > KERNEL_PKG_LIBC_HEADERS=linux-libc-dev > > +KBUILD_BUILD_TIMESTAMP="${KBUILD_BUILD_TIMESTAMP}" > > > > # Constants > > KCONF=.config > > @@ -36,6 +37,7 @@ main() { > > > > # variables to be exported > > export ARCH > > + export KBUILD_BUILD_TIMESTAMP > > > > # are we cross-compiling? > > BUILD_ARCH=$(dpkg-architecture -qDEB_BUILD_ARCH) diff --git > > a/meta/recipes-kernel/linux/linux-custom.inc > > b/meta/recipes-kernel/linux/linux-custom.inc > > index 8a5dbf82..f8e87453 100644 > > --- a/meta/recipes-kernel/linux/linux-custom.inc > > +++ b/meta/recipes-kernel/linux/linux-custom.inc > > @@ -33,6 +33,8 @@ KERNEL_HEADERS_DEBIAN_DEPENDS ?= "libc6, \ > > > > KERNEL_LIBC_DEV_DEPLOY ?= "0" > > > > +KBUILD_BUILD_TIMESTAMP ??= "LANG=C $(date)" > > Can't we use SOURCE_DATE_EPOCH like the rest? This variable is still not honored by the kernel (AFAIK). Also the kernel documentation only mentions KBUILD_BUILD_TIMESTAMP. There is a patch from 2015 which has not been applied [1]. We could do the mapping in ISAR, but I prefer to have it explicit. [1] https://patchwork.kernel.org/project/linux-kbuild/patch/1443741332.2730.75.camel@decadent.org.uk/ Felix > > > + > > # Settings that may be changed on a per distro, machine or layer > > basis # > > -------------------------------------------------------------------- > > > > @@ -73,6 +75,7 @@ TEMPLATE_VARS += " \ > > LINUX_VERSION_EXTENSION \ > > KERNEL_NAME_PROVIDED \ > > KERNEL_CONFIG_FRAGMENTS \ > > + KBUILD_BUILD_TIMESTAMP \ > > " > > > > inherit dpkg > > Jan > > -- > Siemens AG, Technology > Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH 1/1] add support to set KBUILD_BUILD_TIMESTAMP for custom kernels 2022-03-21 17:33 ` Moessbauer, Felix @ 2022-03-21 17:51 ` Jan Kiszka 2022-03-22 8:54 ` Moessbauer, Felix 0 siblings, 1 reply; 8+ messages in thread From: Jan Kiszka @ 2022-03-21 17:51 UTC (permalink / raw) To: Moessbauer, Felix (T CED SES-DE), isar-users On 21.03.22 18:33, Moessbauer, Felix (T CED SES-DE) wrote: > Hi Jan, > >> -----Original Message----- >> From: Kiszka, Jan (T CED) <jan.kiszka@siemens.com> >> Sent: Monday, March 21, 2022 6:18 PM >> To: Moessbauer, Felix (T CED SES-DE) <felix.moessbauer@siemens.com>; isar- >> users@googlegroups.com >> Subject: Re: [PATCH 1/1] add support to set KBUILD_BUILD_TIMESTAMP for >> custom kernels >> >> On 21.03.22 15:40, Felix Moessbauer wrote: >>> This patch adds the infrastructure to overwrite the >>> KBUILD_BUILD_TIMESTAMP environment variable for custom kernel builds. >>> By that, we improve the support for reproducible kernel builds. >>> >> >> Agreed. >> >>> Setting this to a fixed string further helps to profit from ccache >>> caching on repeated builds. >> >> Hmm, but the practical impact of this variable should have been limited /wrt >> ccache. I was seeing significant build-time reduction before the commit already. > > Yes, but of course depends on configuration. > >> >>> >>> Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com> >>> --- >>> meta/recipes-kernel/linux/files/debian/isar/build.tmpl | 1 + >>> meta/recipes-kernel/linux/files/debian/isar/common.tmpl | 2 ++ >>> meta/recipes-kernel/linux/linux-custom.inc | 3 +++ >>> 3 files changed, 6 insertions(+) >>> >>> diff --git a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl >>> b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl >>> index b7749cec..32d2eb23 100644 >>> --- a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl >>> +++ b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl >>> @@ -34,6 +34,7 @@ print_settings() { >>> # --------------- >>> # ARCH=${ARCH} >>> # CROSS_COMPILE=${CROSS_COMPILE} >>> +# KBUILD_BUILD_TIMESTAMP=${KBUILD_BUILD_TIMESTAMP} >>> EOF >>> } >>> >>> diff --git a/meta/recipes-kernel/linux/files/debian/isar/common.tmpl >>> b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl >>> index b7752fb6..ffe84745 100644 >>> --- a/meta/recipes-kernel/linux/files/debian/isar/common.tmpl >>> +++ b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl >>> @@ -7,6 +7,7 @@ ARCH=${KERNEL_ARCH} >>> KERNEL_PKG_IMAGE=linux-image-${KERNEL_NAME_PROVIDED} >>> KERNEL_PKG_KERN_HEADERS=linux-headers-${KERNEL_NAME_PROVIDED} >>> KERNEL_PKG_LIBC_HEADERS=linux-libc-dev >>> +KBUILD_BUILD_TIMESTAMP="${KBUILD_BUILD_TIMESTAMP}" >>> >>> # Constants >>> KCONF=.config >>> @@ -36,6 +37,7 @@ main() { >>> >>> # variables to be exported >>> export ARCH >>> + export KBUILD_BUILD_TIMESTAMP >>> >>> # are we cross-compiling? >>> BUILD_ARCH=$(dpkg-architecture -qDEB_BUILD_ARCH) diff --git >>> a/meta/recipes-kernel/linux/linux-custom.inc >>> b/meta/recipes-kernel/linux/linux-custom.inc >>> index 8a5dbf82..f8e87453 100644 >>> --- a/meta/recipes-kernel/linux/linux-custom.inc >>> +++ b/meta/recipes-kernel/linux/linux-custom.inc >>> @@ -33,6 +33,8 @@ KERNEL_HEADERS_DEBIAN_DEPENDS ?= "libc6, \ >>> >>> KERNEL_LIBC_DEV_DEPLOY ?= "0" >>> >>> +KBUILD_BUILD_TIMESTAMP ??= "LANG=C $(date)" >> >> Can't we use SOURCE_DATE_EPOCH like the rest? > > This variable is still not honored by the kernel (AFAIK). > Also the kernel documentation only mentions KBUILD_BUILD_TIMESTAMP. > There is a patch from 2015 which has not been applied [1]. > > We could do the mapping in ISAR, but I prefer to have it explicit. > > [1] https://patchwork.kernel.org/project/linux-kbuild/patch/1443741332.2730.75.camel@decadent.org.uk/ > Then how does the regular Debian kernel set a stable date? We should align to that and surely not force the user to set anything manually (that is my interpretation of the behavior of the current patch). Jan -- Siemens AG, Technology Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 8+ messages in thread
* RE: [PATCH 1/1] add support to set KBUILD_BUILD_TIMESTAMP for custom kernels 2022-03-21 17:51 ` Jan Kiszka @ 2022-03-22 8:54 ` Moessbauer, Felix 2022-03-22 14:49 ` Jan Kiszka 0 siblings, 1 reply; 8+ messages in thread From: Moessbauer, Felix @ 2022-03-22 8:54 UTC (permalink / raw) To: jan.kiszka; +Cc: isar-users > -----Original Message----- > From: Kiszka, Jan (T CED) <jan.kiszka@siemens.com> > Sent: Monday, March 21, 2022 6:52 PM > To: Moessbauer, Felix (T CED SES-DE) <felix.moessbauer@siemens.com>; isar- > users@googlegroups.com > Subject: Re: [PATCH 1/1] add support to set KBUILD_BUILD_TIMESTAMP for > custom kernels > > On 21.03.22 18:33, Moessbauer, Felix (T CED SES-DE) wrote: > > Hi Jan, > > > >> -----Original Message----- > >> From: Kiszka, Jan (T CED) <jan.kiszka@siemens.com> > >> Sent: Monday, March 21, 2022 6:18 PM > >> To: Moessbauer, Felix (T CED SES-DE) <felix.moessbauer@siemens.com>; > >> isar- users@googlegroups.com > >> Subject: Re: [PATCH 1/1] add support to set KBUILD_BUILD_TIMESTAMP > >> for custom kernels > >> > >> On 21.03.22 15:40, Felix Moessbauer wrote: > >>> This patch adds the infrastructure to overwrite the > >>> KBUILD_BUILD_TIMESTAMP environment variable for custom kernel builds. > >>> By that, we improve the support for reproducible kernel builds. > >>> > >> > >> Agreed. > >> > >>> Setting this to a fixed string further helps to profit from ccache > >>> caching on repeated builds. > >> > >> Hmm, but the practical impact of this variable should have been > >> limited /wrt ccache. I was seeing significant build-time reduction before the > commit already. > > > > Yes, but of course depends on configuration. > > > >> > >>> > >>> Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com> > >>> --- > >>> meta/recipes-kernel/linux/files/debian/isar/build.tmpl | 1 + > >>> meta/recipes-kernel/linux/files/debian/isar/common.tmpl | 2 ++ > >>> meta/recipes-kernel/linux/linux-custom.inc | 3 +++ > >>> 3 files changed, 6 insertions(+) > >>> > >>> diff --git a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl > >>> b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl > >>> index b7749cec..32d2eb23 100644 > >>> --- a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl > >>> +++ b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl > >>> @@ -34,6 +34,7 @@ print_settings() { # --------------- # > >>> ARCH=${ARCH} # CROSS_COMPILE=${CROSS_COMPILE} > >>> +# KBUILD_BUILD_TIMESTAMP=${KBUILD_BUILD_TIMESTAMP} > >>> EOF > >>> } > >>> > >>> diff --git a/meta/recipes-kernel/linux/files/debian/isar/common.tmpl > >>> b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl > >>> index b7752fb6..ffe84745 100644 > >>> --- a/meta/recipes-kernel/linux/files/debian/isar/common.tmpl > >>> +++ b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl > >>> @@ -7,6 +7,7 @@ ARCH=${KERNEL_ARCH} > >>> KERNEL_PKG_IMAGE=linux-image-${KERNEL_NAME_PROVIDED} > >>> KERNEL_PKG_KERN_HEADERS=linux-headers-${KERNEL_NAME_PROVIDED} > >>> KERNEL_PKG_LIBC_HEADERS=linux-libc-dev > >>> +KBUILD_BUILD_TIMESTAMP="${KBUILD_BUILD_TIMESTAMP}" > >>> > >>> # Constants > >>> KCONF=.config > >>> @@ -36,6 +37,7 @@ main() { > >>> > >>> # variables to be exported > >>> export ARCH > >>> + export KBUILD_BUILD_TIMESTAMP > >>> > >>> # are we cross-compiling? > >>> BUILD_ARCH=$(dpkg-architecture -qDEB_BUILD_ARCH) diff --git > >>> a/meta/recipes-kernel/linux/linux-custom.inc > >>> b/meta/recipes-kernel/linux/linux-custom.inc > >>> index 8a5dbf82..f8e87453 100644 > >>> --- a/meta/recipes-kernel/linux/linux-custom.inc > >>> +++ b/meta/recipes-kernel/linux/linux-custom.inc > >>> @@ -33,6 +33,8 @@ KERNEL_HEADERS_DEBIAN_DEPENDS ?= "libc6, \ > >>> > >>> KERNEL_LIBC_DEV_DEPLOY ?= "0" > >>> > >>> +KBUILD_BUILD_TIMESTAMP ??= "LANG=C $(date)" > >> > >> Can't we use SOURCE_DATE_EPOCH like the rest? > > > > This variable is still not honored by the kernel (AFAIK). > > Also the kernel documentation only mentions KBUILD_BUILD_TIMESTAMP. > > There is a patch from 2015 which has not been applied [1]. > > > > We could do the mapping in ISAR, but I prefer to have it explicit. > > > > > > Then how does the regular Debian kernel set a stable date? We should align to > that and surely not force the user to set anything manually (that is my > interpretation of the behavior of the current patch). They set it in the rules file to "$(SOURCE_DATE)" [1], which itself is computed from the changelog: SOURCE_DATE := $(shell dpkg-parsechangelog -SDate) Unfortunately, this is not possible for us because we generate the changelog on the fly. Setting "KBUILD_BUILD_TIMESTAMP" to the current build time (as default) should be reasonable, Simply because we do not have better information. That's also how it was previously. The only addition here is do give the user support to set it to something different. [1] https://salsa.debian.org/kernel-team/linux/-/blob/master/debian/rules.real#L46 Felix > > Jan > > -- > Siemens AG, Technology > Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH 1/1] add support to set KBUILD_BUILD_TIMESTAMP for custom kernels 2022-03-22 8:54 ` Moessbauer, Felix @ 2022-03-22 14:49 ` Jan Kiszka 2022-03-22 17:27 ` Moessbauer, Felix 0 siblings, 1 reply; 8+ messages in thread From: Jan Kiszka @ 2022-03-22 14:49 UTC (permalink / raw) To: Moessbauer, Felix (T CED SES-DE); +Cc: isar-users On 22.03.22 09:54, Moessbauer, Felix (T CED SES-DE) wrote: >> -----Original Message----- >> From: Kiszka, Jan (T CED) <jan.kiszka@siemens.com> >> Sent: Monday, March 21, 2022 6:52 PM >> To: Moessbauer, Felix (T CED SES-DE) <felix.moessbauer@siemens.com>; isar- >> users@googlegroups.com >> Subject: Re: [PATCH 1/1] add support to set KBUILD_BUILD_TIMESTAMP for >> custom kernels >> >> On 21.03.22 18:33, Moessbauer, Felix (T CED SES-DE) wrote: >>> Hi Jan, >>> >>>> -----Original Message----- >>>> From: Kiszka, Jan (T CED) <jan.kiszka@siemens.com> >>>> Sent: Monday, March 21, 2022 6:18 PM >>>> To: Moessbauer, Felix (T CED SES-DE) <felix.moessbauer@siemens.com>; >>>> isar- users@googlegroups.com >>>> Subject: Re: [PATCH 1/1] add support to set KBUILD_BUILD_TIMESTAMP >>>> for custom kernels >>>> >>>> On 21.03.22 15:40, Felix Moessbauer wrote: >>>>> This patch adds the infrastructure to overwrite the >>>>> KBUILD_BUILD_TIMESTAMP environment variable for custom kernel builds. >>>>> By that, we improve the support for reproducible kernel builds. >>>>> >>>> >>>> Agreed. >>>> >>>>> Setting this to a fixed string further helps to profit from ccache >>>>> caching on repeated builds. >>>> >>>> Hmm, but the practical impact of this variable should have been >>>> limited /wrt ccache. I was seeing significant build-time reduction before the >> commit already. >>> >>> Yes, but of course depends on configuration. >>> >>>> >>>>> >>>>> Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com> >>>>> --- >>>>> meta/recipes-kernel/linux/files/debian/isar/build.tmpl | 1 + >>>>> meta/recipes-kernel/linux/files/debian/isar/common.tmpl | 2 ++ >>>>> meta/recipes-kernel/linux/linux-custom.inc | 3 +++ >>>>> 3 files changed, 6 insertions(+) >>>>> >>>>> diff --git a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl >>>>> b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl >>>>> index b7749cec..32d2eb23 100644 >>>>> --- a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl >>>>> +++ b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl >>>>> @@ -34,6 +34,7 @@ print_settings() { # --------------- # >>>>> ARCH=${ARCH} # CROSS_COMPILE=${CROSS_COMPILE} >>>>> +# KBUILD_BUILD_TIMESTAMP=${KBUILD_BUILD_TIMESTAMP} >>>>> EOF >>>>> } >>>>> >>>>> diff --git a/meta/recipes-kernel/linux/files/debian/isar/common.tmpl >>>>> b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl >>>>> index b7752fb6..ffe84745 100644 >>>>> --- a/meta/recipes-kernel/linux/files/debian/isar/common.tmpl >>>>> +++ b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl >>>>> @@ -7,6 +7,7 @@ ARCH=${KERNEL_ARCH} >>>>> KERNEL_PKG_IMAGE=linux-image-${KERNEL_NAME_PROVIDED} >>>>> KERNEL_PKG_KERN_HEADERS=linux-headers-${KERNEL_NAME_PROVIDED} >>>>> KERNEL_PKG_LIBC_HEADERS=linux-libc-dev >>>>> +KBUILD_BUILD_TIMESTAMP="${KBUILD_BUILD_TIMESTAMP}" >>>>> >>>>> # Constants >>>>> KCONF=.config >>>>> @@ -36,6 +37,7 @@ main() { >>>>> >>>>> # variables to be exported >>>>> export ARCH >>>>> + export KBUILD_BUILD_TIMESTAMP >>>>> >>>>> # are we cross-compiling? >>>>> BUILD_ARCH=$(dpkg-architecture -qDEB_BUILD_ARCH) diff --git >>>>> a/meta/recipes-kernel/linux/linux-custom.inc >>>>> b/meta/recipes-kernel/linux/linux-custom.inc >>>>> index 8a5dbf82..f8e87453 100644 >>>>> --- a/meta/recipes-kernel/linux/linux-custom.inc >>>>> +++ b/meta/recipes-kernel/linux/linux-custom.inc >>>>> @@ -33,6 +33,8 @@ KERNEL_HEADERS_DEBIAN_DEPENDS ?= "libc6, \ >>>>> >>>>> KERNEL_LIBC_DEV_DEPLOY ?= "0" >>>>> >>>>> +KBUILD_BUILD_TIMESTAMP ??= "LANG=C $(date)" >>>> >>>> Can't we use SOURCE_DATE_EPOCH like the rest? >>> >>> This variable is still not honored by the kernel (AFAIK). >>> Also the kernel documentation only mentions KBUILD_BUILD_TIMESTAMP. >>> There is a patch from 2015 which has not been applied [1]. >>> >>> We could do the mapping in ISAR, but I prefer to have it explicit. >>> >>> >> >> Then how does the regular Debian kernel set a stable date? We should align to >> that and surely not force the user to set anything manually (that is my >> interpretation of the behavior of the current patch). > > They set it in the rules file to "$(SOURCE_DATE)" [1], which itself is computed from the changelog: > SOURCE_DATE := $(shell dpkg-parsechangelog -SDate) > > Unfortunately, this is not possible for us because we generate the changelog on the fly. > Setting "KBUILD_BUILD_TIMESTAMP" to the current build time (as default) should be reasonable, > Simply because we do not have better information. That's also how it was previously. > The only addition here is do give the user support to set it to something different. > There must be more information in the build env because other packages that do evaluate SOURCE_DATE_EPOCH find that properly set, automatically. Jan -- Siemens AG, Technology Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 8+ messages in thread
* RE: [PATCH 1/1] add support to set KBUILD_BUILD_TIMESTAMP for custom kernels 2022-03-22 14:49 ` Jan Kiszka @ 2022-03-22 17:27 ` Moessbauer, Felix 2022-03-22 18:52 ` Jan Kiszka 0 siblings, 1 reply; 8+ messages in thread From: Moessbauer, Felix @ 2022-03-22 17:27 UTC (permalink / raw) To: jan.kiszka; +Cc: isar-users > -----Original Message----- > From: Kiszka, Jan (T CED) <jan.kiszka@siemens.com> > Sent: Tuesday, March 22, 2022 3:49 PM > To: Moessbauer, Felix (T CED SES-DE) <felix.moessbauer@siemens.com> > Cc: isar-users@googlegroups.com > Subject: Re: [PATCH 1/1] add support to set KBUILD_BUILD_TIMESTAMP for > custom kernels > > On 22.03.22 09:54, Moessbauer, Felix (T CED SES-DE) wrote: > >> -----Original Message----- > >> From: Kiszka, Jan (T CED) <jan.kiszka@siemens.com> > >> Sent: Monday, March 21, 2022 6:52 PM > >> To: Moessbauer, Felix (T CED SES-DE) <felix.moessbauer@siemens.com>; > >> isar- users@googlegroups.com > >> Subject: Re: [PATCH 1/1] add support to set KBUILD_BUILD_TIMESTAMP > >> for custom kernels > >> > >> On 21.03.22 18:33, Moessbauer, Felix (T CED SES-DE) wrote: > >>> Hi Jan, > >>> > >>>> -----Original Message----- > >>>> From: Kiszka, Jan (T CED) <jan.kiszka@siemens.com> > >>>> Sent: Monday, March 21, 2022 6:18 PM > >>>> To: Moessbauer, Felix (T CED SES-DE) > >>>> <felix.moessbauer@siemens.com>; > >>>> isar- users@googlegroups.com > >>>> Subject: Re: [PATCH 1/1] add support to set KBUILD_BUILD_TIMESTAMP > >>>> for custom kernels > >>>> > >>>> On 21.03.22 15:40, Felix Moessbauer wrote: > >>>>> This patch adds the infrastructure to overwrite the > >>>>> KBUILD_BUILD_TIMESTAMP environment variable for custom kernel > builds. > >>>>> By that, we improve the support for reproducible kernel builds. > >>>>> > >>>> > >>>> Agreed. > >>>> > >>>>> Setting this to a fixed string further helps to profit from ccache > >>>>> caching on repeated builds. > >>>> > >>>> Hmm, but the practical impact of this variable should have been > >>>> limited /wrt ccache. I was seeing significant build-time reduction > >>>> before the > >> commit already. > >>> > >>> Yes, but of course depends on configuration. > >>> > >>>> > >>>>> > >>>>> Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com> > >>>>> --- > >>>>> meta/recipes-kernel/linux/files/debian/isar/build.tmpl | 1 + > >>>>> meta/recipes-kernel/linux/files/debian/isar/common.tmpl | 2 ++ > >>>>> meta/recipes-kernel/linux/linux-custom.inc | 3 +++ > >>>>> 3 files changed, 6 insertions(+) > >>>>> > >>>>> diff --git > >>>>> a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl > >>>>> b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl > >>>>> index b7749cec..32d2eb23 100644 > >>>>> --- a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl > >>>>> +++ b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl > >>>>> @@ -34,6 +34,7 @@ print_settings() { # --------------- # > >>>>> ARCH=${ARCH} # CROSS_COMPILE=${CROSS_COMPILE} > >>>>> +# KBUILD_BUILD_TIMESTAMP=${KBUILD_BUILD_TIMESTAMP} > >>>>> EOF > >>>>> } > >>>>> > >>>>> diff --git > >>>>> a/meta/recipes-kernel/linux/files/debian/isar/common.tmpl > >>>>> b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl > >>>>> index b7752fb6..ffe84745 100644 > >>>>> --- a/meta/recipes-kernel/linux/files/debian/isar/common.tmpl > >>>>> +++ b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl > >>>>> @@ -7,6 +7,7 @@ ARCH=${KERNEL_ARCH} > >>>>> KERNEL_PKG_IMAGE=linux-image-${KERNEL_NAME_PROVIDED} > >>>>> KERNEL_PKG_KERN_HEADERS=linux-headers- > ${KERNEL_NAME_PROVIDED} > >>>>> KERNEL_PKG_LIBC_HEADERS=linux-libc-dev > >>>>> +KBUILD_BUILD_TIMESTAMP="${KBUILD_BUILD_TIMESTAMP}" > >>>>> > >>>>> # Constants > >>>>> KCONF=.config > >>>>> @@ -36,6 +37,7 @@ main() { > >>>>> > >>>>> # variables to be exported > >>>>> export ARCH > >>>>> + export KBUILD_BUILD_TIMESTAMP > >>>>> > >>>>> # are we cross-compiling? > >>>>> BUILD_ARCH=$(dpkg-architecture -qDEB_BUILD_ARCH) diff --git > >>>>> a/meta/recipes-kernel/linux/linux-custom.inc > >>>>> b/meta/recipes-kernel/linux/linux-custom.inc > >>>>> index 8a5dbf82..f8e87453 100644 > >>>>> --- a/meta/recipes-kernel/linux/linux-custom.inc > >>>>> +++ b/meta/recipes-kernel/linux/linux-custom.inc > >>>>> @@ -33,6 +33,8 @@ KERNEL_HEADERS_DEBIAN_DEPENDS ?= "libc6, \ > >>>>> > >>>>> KERNEL_LIBC_DEV_DEPLOY ?= "0" > >>>>> > >>>>> +KBUILD_BUILD_TIMESTAMP ??= "LANG=C $(date)" > >>>> > >>>> Can't we use SOURCE_DATE_EPOCH like the rest? > >>> > >>> This variable is still not honored by the kernel (AFAIK). > >>> Also the kernel documentation only mentions KBUILD_BUILD_TIMESTAMP. > >>> There is a patch from 2015 which has not been applied [1]. > >>> > >>> We could do the mapping in ISAR, but I prefer to have it explicit. > >>> > >>> > >> > >> Then how does the regular Debian kernel set a stable date? We should > >> align to that and surely not force the user to set anything manually > >> (that is my interpretation of the behavior of the current patch). > > > > They set it in the rules file to "$(SOURCE_DATE)" [1], which itself is computed > from the changelog: > > SOURCE_DATE := $(shell dpkg-parsechangelog -SDate) > > > > Unfortunately, this is not possible for us because we generate the changelog > on the fly. > > Setting "KBUILD_BUILD_TIMESTAMP" to the current build time (as > > default) should be reasonable, Simply because we do not have better > information. That's also how it was previously. > > The only addition here is do give the user support to set it to something > different. > > > > There must be more information in the build env because other packages that > do evaluate SOURCE_DATE_EPOCH find that properly set, automatically. Well... the SOURCE_DATE_EPOCH variable is set by debhelper to the date of the last entry in the changelog [1], [2]. This is technically the same as stated above, but already happens automatically. In our use-case, that does not help much, as the changelog is auto-generated by ISAR. Actually, it points to the begin of the Unix epoch, which is questionable. At least there, we should add the current build time (and add an option to overwrite, otherwise dpkg-raw recipes can no longer be cached). Maybe the better strategy is to add a BB variable "SOURCE_DATE_EPOCH" to the debianize.class and use the value in the kernel recipe to set KBUILD_BUILD_TIMESTAMP. By that, we would fix the questionable entry in the changelog, as well as still support reproducible builds. Felix [1] https://reproducible-builds.org/docs/source-date-epoch/ [2] https://www.debian.org/doc/manuals/debmake-doc/ch05.en.html#reproducible > > Jan > > -- > Siemens AG, Technology > Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH 1/1] add support to set KBUILD_BUILD_TIMESTAMP for custom kernels 2022-03-22 17:27 ` Moessbauer, Felix @ 2022-03-22 18:52 ` Jan Kiszka 0 siblings, 0 replies; 8+ messages in thread From: Jan Kiszka @ 2022-03-22 18:52 UTC (permalink / raw) To: Moessbauer, Felix; +Cc: isar-users On 22.03.22 18:27, Moessbauer, Felix wrote: >> -----Original Message----- >> From: Kiszka, Jan (T CED) <jan.kiszka@siemens.com> >> Sent: Tuesday, March 22, 2022 3:49 PM >> To: Moessbauer, Felix (T CED SES-DE) <felix.moessbauer@siemens.com> >> Cc: isar-users@googlegroups.com >> Subject: Re: [PATCH 1/1] add support to set KBUILD_BUILD_TIMESTAMP for >> custom kernels >> >> On 22.03.22 09:54, Moessbauer, Felix (T CED SES-DE) wrote: >>>> -----Original Message----- >>>> From: Kiszka, Jan (T CED) <jan.kiszka@siemens.com> >>>> Sent: Monday, March 21, 2022 6:52 PM >>>> To: Moessbauer, Felix (T CED SES-DE) <felix.moessbauer@siemens.com>; >>>> isar- users@googlegroups.com >>>> Subject: Re: [PATCH 1/1] add support to set KBUILD_BUILD_TIMESTAMP >>>> for custom kernels >>>> >>>> On 21.03.22 18:33, Moessbauer, Felix (T CED SES-DE) wrote: >>>>> Hi Jan, >>>>> >>>>>> -----Original Message----- >>>>>> From: Kiszka, Jan (T CED) <jan.kiszka@siemens.com> >>>>>> Sent: Monday, March 21, 2022 6:18 PM >>>>>> To: Moessbauer, Felix (T CED SES-DE) >>>>>> <felix.moessbauer@siemens.com>; >>>>>> isar- users@googlegroups.com >>>>>> Subject: Re: [PATCH 1/1] add support to set KBUILD_BUILD_TIMESTAMP >>>>>> for custom kernels >>>>>> >>>>>> On 21.03.22 15:40, Felix Moessbauer wrote: >>>>>>> This patch adds the infrastructure to overwrite the >>>>>>> KBUILD_BUILD_TIMESTAMP environment variable for custom kernel >> builds. >>>>>>> By that, we improve the support for reproducible kernel builds. >>>>>>> >>>>>> >>>>>> Agreed. >>>>>> >>>>>>> Setting this to a fixed string further helps to profit from ccache >>>>>>> caching on repeated builds. >>>>>> >>>>>> Hmm, but the practical impact of this variable should have been >>>>>> limited /wrt ccache. I was seeing significant build-time reduction >>>>>> before the >>>> commit already. >>>>> >>>>> Yes, but of course depends on configuration. >>>>> >>>>>> >>>>>>> >>>>>>> Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com> >>>>>>> --- >>>>>>> meta/recipes-kernel/linux/files/debian/isar/build.tmpl | 1 + >>>>>>> meta/recipes-kernel/linux/files/debian/isar/common.tmpl | 2 ++ >>>>>>> meta/recipes-kernel/linux/linux-custom.inc | 3 +++ >>>>>>> 3 files changed, 6 insertions(+) >>>>>>> >>>>>>> diff --git >>>>>>> a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl >>>>>>> b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl >>>>>>> index b7749cec..32d2eb23 100644 >>>>>>> --- a/meta/recipes-kernel/linux/files/debian/isar/build.tmpl >>>>>>> +++ b/meta/recipes-kernel/linux/files/debian/isar/build.tmpl >>>>>>> @@ -34,6 +34,7 @@ print_settings() { # --------------- # >>>>>>> ARCH=${ARCH} # CROSS_COMPILE=${CROSS_COMPILE} >>>>>>> +# KBUILD_BUILD_TIMESTAMP=${KBUILD_BUILD_TIMESTAMP} >>>>>>> EOF >>>>>>> } >>>>>>> >>>>>>> diff --git >>>>>>> a/meta/recipes-kernel/linux/files/debian/isar/common.tmpl >>>>>>> b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl >>>>>>> index b7752fb6..ffe84745 100644 >>>>>>> --- a/meta/recipes-kernel/linux/files/debian/isar/common.tmpl >>>>>>> +++ b/meta/recipes-kernel/linux/files/debian/isar/common.tmpl >>>>>>> @@ -7,6 +7,7 @@ ARCH=${KERNEL_ARCH} >>>>>>> KERNEL_PKG_IMAGE=linux-image-${KERNEL_NAME_PROVIDED} >>>>>>> KERNEL_PKG_KERN_HEADERS=linux-headers- >> ${KERNEL_NAME_PROVIDED} >>>>>>> KERNEL_PKG_LIBC_HEADERS=linux-libc-dev >>>>>>> +KBUILD_BUILD_TIMESTAMP="${KBUILD_BUILD_TIMESTAMP}" >>>>>>> >>>>>>> # Constants >>>>>>> KCONF=.config >>>>>>> @@ -36,6 +37,7 @@ main() { >>>>>>> >>>>>>> # variables to be exported >>>>>>> export ARCH >>>>>>> + export KBUILD_BUILD_TIMESTAMP >>>>>>> >>>>>>> # are we cross-compiling? >>>>>>> BUILD_ARCH=$(dpkg-architecture -qDEB_BUILD_ARCH) diff --git >>>>>>> a/meta/recipes-kernel/linux/linux-custom.inc >>>>>>> b/meta/recipes-kernel/linux/linux-custom.inc >>>>>>> index 8a5dbf82..f8e87453 100644 >>>>>>> --- a/meta/recipes-kernel/linux/linux-custom.inc >>>>>>> +++ b/meta/recipes-kernel/linux/linux-custom.inc >>>>>>> @@ -33,6 +33,8 @@ KERNEL_HEADERS_DEBIAN_DEPENDS ?= "libc6, \ >>>>>>> >>>>>>> KERNEL_LIBC_DEV_DEPLOY ?= "0" >>>>>>> >>>>>>> +KBUILD_BUILD_TIMESTAMP ??= "LANG=C $(date)" >>>>>> >>>>>> Can't we use SOURCE_DATE_EPOCH like the rest? >>>>> >>>>> This variable is still not honored by the kernel (AFAIK). >>>>> Also the kernel documentation only mentions KBUILD_BUILD_TIMESTAMP. >>>>> There is a patch from 2015 which has not been applied [1]. >>>>> >>>>> We could do the mapping in ISAR, but I prefer to have it explicit. >>>>> >>>>> >>>> >>>> Then how does the regular Debian kernel set a stable date? We should >>>> align to that and surely not force the user to set anything manually >>>> (that is my interpretation of the behavior of the current patch). >>> >>> They set it in the rules file to "$(SOURCE_DATE)" [1], which itself is computed >> from the changelog: >>> SOURCE_DATE := $(shell dpkg-parsechangelog -SDate) >>> >>> Unfortunately, this is not possible for us because we generate the changelog >> on the fly. >>> Setting "KBUILD_BUILD_TIMESTAMP" to the current build time (as >>> default) should be reasonable, Simply because we do not have better >> information. That's also how it was previously. >>> The only addition here is do give the user support to set it to something >> different. >>> >> >> There must be more information in the build env because other packages that >> do evaluate SOURCE_DATE_EPOCH find that properly set, automatically. > > Well... the SOURCE_DATE_EPOCH variable is set by debhelper to the date of the last entry in the changelog [1], [2]. > This is technically the same as stated above, but already happens automatically. > > In our use-case, that does not help much, as the changelog is auto-generated by ISAR. It does help because it will already deliver a stable (even if meaningless) timestamp. And if there is a changelog that is appended, its last entry date is used. > Actually, it points to the begin of the Unix epoch, which is questionable. > At least there, we should add the current build time (and add an option to overwrite, otherwise dpkg-raw recipes can no longer be cached). That would be counterproductive for reproducibility and was the reason to pin it to 0 not that long ago (53d315fdd3f3). If you want proper changelog with proper dates, don't use deb_add_changelog. Jan -- Siemens AG, Technology Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2022-03-22 18:52 UTC | newest] Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2022-03-21 14:40 [PATCH 1/1] add support to set KBUILD_BUILD_TIMESTAMP for custom kernels Felix Moessbauer 2022-03-21 17:18 ` Jan Kiszka 2022-03-21 17:33 ` Moessbauer, Felix 2022-03-21 17:51 ` Jan Kiszka 2022-03-22 8:54 ` Moessbauer, Felix 2022-03-22 14:49 ` Jan Kiszka 2022-03-22 17:27 ` Moessbauer, Felix 2022-03-22 18:52 ` Jan Kiszka
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox