Re: [PATCH] sbuild: Fix low hit ratio for ccache on linux kernel rebuild

[Uladzimir Bely <ubely@ilbers.de>]

In the email from Friday, 15 July 2022 16:27:27 +03 user Moessbauer, Felix 
wrote:
> > -----Original Message-----
> > From: isar-users@googlegroups.com <isar-users@googlegroups.com> On
> > Behalf Of Uladzimir Bely
> > Sent: Friday, July 15, 2022 1:27 PM
> > To: isar-users@googlegroups.com
> > Subject: [PATCH] sbuild: Fix low hit ratio for ccache on linux kernel
> > rebuild
> > 
> > For some targets defconfig for linux recipe includes option
> > CONFIG_DEBUG_INFO that embeds absolute path to the source files into
> > compiled binaries.
> > 
> > While sbuild uses unique directory to place sources for each build by
> > default, it happens that ccache hits ration is to small on rebuild.
> > 
> > This patch makes sbuild use the same build directory for sequential
> > builds. While several packages can be built in parallel (e.g. same
> > package for different architectures), WORKDIR is used to generate hash
> > part for the build directory of sbuild.
> > 
> > Signed-off-by: Uladzimir Bely <ubely@ilbers.de>
> > ---
> > 
> >  meta/classes/dpkg.bbclass | 3 +++
> >  1 file changed, 3 insertions(+)
> > 
> > diff --git a/meta/classes/dpkg.bbclass b/meta/classes/dpkg.bbclass index
> > b726ea9d..016e46c4 100644
> > --- a/meta/classes/dpkg.bbclass
> > +++ b/meta/classes/dpkg.bbclass
> > @@ -82,6 +82,8 @@ dpkg_runbuild() {
> > 
> >      sh -c "cd ${WORKDIR}; dpkg-source -q -b ${PPS}"
> >      DSC_FILE=$(find ${WORKDIR} -name "${DEB_SOURCE_NAME}*.dsc" -print)
> > 
> > +    build_path_hash=$(echo "${WORKDIR}" | md5sum | cut -b1-6)
> > +
> > 
> >      sbuild -A -n -c ${SBUILD_CHROOT}
> >      --extra-repository="${ISAR_APT_REPO}" \
> >      
> >          --host=${PACKAGE_ARCH} --build=${SBUILD_HOST_ARCH} ${profiles} \
> >          --no-run-lintian --no-run-piuparts --no-run-autopkgtest
> >          --resolve-
> > 
> > alternatives \ @@ -91,6 +93,7 @@ dpkg_runbuild() {
> > 
> >          --finished-build-commands="cp -n --no-preserve=owner
> >          ${deb_dir}/*.deb -t
> > 
> > ${ext_deb_dir}/ || :" \
> > 
> >          --finished-build-commands="cp /var/log/dpkg.log
> > 
> > ${ext_root}/dpkg_partial.log" \
> > 
> >          --debbuildopts="--source-option=-I" \
> > 
> > +        --build-path="/build/${PN}-${build_path_hash}" \
> 
> This is just a workaround that hides the problem.
> Debian takes a lot of care to improve the reproducibility of their packages.
> IIRC then reproducibility issues will be treated as bugs from bookworm on.
> Could we please do the same and fix the corresponding recipes (like the
> kernel recipe)?
> 

It can be moved to the kernel recipe, but you left me a bit confused. Why 
changing of build directory from some random (from build to build) path to 
some fixed one (the same hash on every rebuild) makes the build less 
reproducible?

> Please also have a look at the reproducibility section of dpkg-buildflags.
> The only reason the kernel is affected from this is, that the kbuild only
> uses KCFLAGS, not CFLAGS. At least the flags about reproducibility should
> be added to all kernel builds done via ISAR.
> 
> In short: I'm strictly against this workaround.
> Having things that are not reproducible should hurt and not become accepted.
> 
> Felix
> 
> >          --build-dir=${WORKDIR} --dist="isar" ${DSC_FILE}
> >      
> >      sbuild_dpkg_log_export "${WORKDIR}/rootfs/dpkg_partial.log"
> > 
> > --
> > 2.20.1
> > 
> > --
> > You received this message because you are subscribed to the Google Groups
> > "isar-users" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
> > email to isar-users+unsubscribe@googlegroups.com.
> > To view this discussion on the web visit
> > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.g
> > oogle.com%2Fd%2Fmsgid%2Fisar-users%2F20220715112716.2460-1-
> > ubely%2540ilbers.de&amp;data=05%7C01%7Cfelix.moessbauer%40siemens.co
> > m%7C2fbec95c6a0a48bcfa1608da6654fc4b%7C38ae3bcd95794fd4addab42e14
> > 95d55a%7C1%7C0%7C637934812445713510%7CUnknown%7CTWFpbGZsb3d8e
> > yJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7
> > C3000%7C%7C%7C&amp;sdata=H%2FfsTUORIfNFM7jh8CYiCNkuf7T66oxH31mh
> > 1v2Leo0%3D&amp;reserved=0.


-- 
Uladzimir Bely