RE: sbuild migration help needed

["Moessbauer, Felix" <felix.moessbauer@siemens.com>]

> -----Original Message-----
> From: isar-users@googlegroups.com <isar-users@googlegroups.com> On
> Behalf Of Uladzimir Bely
> Sent: Saturday, July 23, 2022 8:40 AM
> To: Schild, Henning (T CED SES-DE) <henning.schild@siemens.com>
> Cc: isar-users <isar-users@googlegroups.com>
> Subject: Re: sbuild migration help needed
> 
> In the email from Friday, 22 July 2022 22:16:56 +03 user Henning Schild wrote:
> > Am Fri, 22 Jul 2022 16:19:07 +0200
> >
> > schrieb Henning Schild <henning.schild@siemens.com>:
> > > Thanks! I will give that a try. Sorry for not sharing that patch, it
> > > is trivial.
> >
> > In fact that patch also does not apply anymore but did before sbuild.
> > You made it a
> >
> >   echo "\noverride_dh_usrlocal:" >> ${S}/debian/rules
> >
> > And i ended up turning into another sed.
> >
> > > I find it a little worrying that a recipe which worked perfectly
> > > before all of a sudden has to be significantly changed. If there
> > > would have been a forgotten build-dep i would have been happy, but
> > > here we have a maybe dramatic change in how the recipe needs to be
> > > written.
> >
> > This recipe was easy enough to share but it shows a fundamental change
> > in isar. Where before sbuild all of WORKDIR became available in the
> > buildchroot, now we only get S.
> >
> > I would not know how many other recipes in countless layers would
> > care, but maybe we can somehow get back to WORKDIR and not only S
> > being mounted and usable in the chroot.
> >
> > That recipe is a classic hack where one just packages something with
> > deb_debianize and the patch features of isar. Being forced to move
> > code around to arrange an S where WORKDIR would simply work (and be
> > patchable) is not nice. More involved projects might require more
> > involved "constructing S" steps.
> >
> > We are moving code around and patching Makefile here, so all of a
> > sudden you need to really understand what you previously could
> > deb_debianize around with.
> >
> > I guess the whole thing is relevant for projects where parts of the
> > code you care about live in a bigger repo.
> >
> 
> Yes, there is a topic for the discussion.
> 
> On the other hand, before sbuild the recipe was able to be compiled. But,
> what about generated (after the build) source package in workdir? I'm pretty
> sure, that it was unbuildable outside of Isar due to missing sources.
> 
> I'm trying to say that having ${S} deep in ${WORKDIR} is generally a wrong
> solution producing unbuildable packages (if depends on upper directories).

Yes, I see it the same way - ${S} should be the root dir of the package.
Accessing files outside of ${S} just worked before because there was no isolation of the dpkg-buildpackage.
Having this isolation brings a huge benefit of generating re-buildable source packages.
We really should not give up on that, but better fix the erroneous packagings downstream.

ISAR is not Yocto, so we better stick to Debian policies instead of just making things work.
This becomes relevant once not only images are generated but also package registries.

Felix

> 
> > > I will change it as you suggested. Thanks again! But we will have to
> > > look out for such big changes breaking existing recipes. All the big
> > > API changes can and will cause a lot of frustration in all the
> > > various layers.
> > >
> > > And there are the known ones, which are in the changelog ... plus
> > > the ones that just happen ... like this one, or the meson/ccache i
> > > just sent a patch for.
> > >
> > > Henning
> > >
> > > --
> > > Index: ifdtool/Makefile
> > >
> ==========================================================
> =========
> > > --- ifdtool.orig/Makefile
> > > +++ ifdtool/Makefile
> > > @@ -6,7 +6,7 @@ PROGRAM = ifdtool
> > >
> > >  CC      = gcc
> > >  INSTALL = /usr/bin/env install
> > >
> > > -PREFIX  = /usr/local
> > > +PREFIX  = /usr
> > >
> > >  CFLAGS  = -O2 -g -Wall -Wextra -Wmissing-prototypes -Werror  CFLAGS
> > > += -I../../src/commonlib/include
> > >
> > > -I../../src/commonlib/bsd/include CFLAGS += -I../cbfstool/flashmap
> > > --
> > >
> > >
> > > Am Fri, 22 Jul 2022 16:08:45 +0300
> > >
> > > schrieb Uladzimir Bely <ubely@ilbers.de>:
> > > > In the email from Friday, 22 July 2022 14:11:19 +03 user Uladzimir
> > > >
> > > > Bely wrote:
> > > > > In the email from Friday, 22 July 2022 12:16:35 +03 user Henning
> > > > > Schild
> > > >
> > > > wrote:
> > > > > > Hi,
> > > > > >
> > > > > > i have a recipe where i am building just a util from a bigger
> > > > > > package, so my S is somewhere inside WORKDIR and some headers
> > > > > > are below S.
> > > > > >
> > > > > > With sbuild that recipe stopped working, not finding those
> > > > > > headers anymore. I did not yet look into the details, but
> > > > > > maybe only S get put into that chroot and not all of WORKDIR.
> > > > >
> > > > > I've looked a bit at the problem.
> > > > >
> > > > > Sbulid doesn't use WORKDIR at all, but builds the package basing
> > > > > on its .dsc file. It happens that generated `ifdtool_4.14.dsc`
> > > > > refers to `ifdtool_4.14.tar.gz`, which simply doesn't contain
> > > > > upper directories.
> > > > >
> > > > > I think, something more complex than automatic `deb_debianize`
> > > > > should be used here.
> > > >
> > > > For instance, it worked for me in form of separate ${S} directory
> > > > where I copy required files and modify Makefile to fit new
> > > > directory
> > > > structure:
> > > >
> > > > inherit dpkg
> > > >
> > > > DESCRIPTION = "Tool for Fast, secure and flexible OpenSource
> > > > firmware"
> > > >
> > > > SRC_URI =
> > > >
> "https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgith
> ub.com%2Fcoreboot%2Fcoreboot%2Farchive%2F%24&amp;data=05%7C01%
> 7Cfelix.moessbauer%40siemens.com%7C7193295dc78f4bb430cb08da6c76354
> 2%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C6379415522032641
> 54%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luM
> zIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=M1
> aymRMjQL%2FCLNZBaqNazI2WV3NS%2BVAAqiG6pVNw4DU%3D&amp;rese
> rved=0{PV}.tar.gz"
> > > > SRC_URI[sha256sum] =
> > > >
> "73740c313935f21964577fd6621f9acaddf1244cc704e88046f854d93ed6feab"
> > > >
> > > > DEBIAN_DEPENDS = "\${misc:Depends}"
> > > >
> > > > S = "${WORKDIR}/${PN}-${PV}"
> >
> > this is nothing but WORKDIR/P ... the default ... can be skipped
> >
> > > > do_prepare_build[cleandirs] += "${S}/debian"
> >
> > has to be S not S/debian
> >
> > > > do_prepare_build() {
> > > >
> > > >         cp -rf ${WORKDIR}/coreboot-${PV}/util/${PN}/* ${S}/
> > > >
> > > >         # Copy required sources from upper directories
> > > >         cp -rf ${WORKDIR}/coreboot-${PV}/util/cbfstool/flashmap
> > > >
> > > > ${S}/ mkdir -p ${S}/commonlib/include
> > > >
> > > >         cp -rf ${WORKDIR}/coreboot-${PV}/src/commonlib/include/*
> > > >
> > > > ${S}/ commonlib/include/
> > > >
> > > >         mkdir -p ${S}/commonlib/bsd/include
> > > >         cp -rf
> > > > ${WORKDIR}/coreboot-${PV}/src/commonlib/bsd/include/*
> > > >
> > > > ${S}/ commonlib/bsd/include/
> > > >
> > > >         # Fix include paths
> > > >         sed -i -e "s/\.\.\/cbfstool\/flashmap/flashmap/g"
> > > >
> > > > ${S}/Makefile sed -i -e "s/\.\.\/\.\.\/src\/commonlib/commonlib/g"
> > > > ${S}/Makefile
> > > >
> > > >         deb_debianize
> > > >         echo "\noverride_dh_usrlocal:" >> ${S}/debian/rules
> > > >
> > > > }
> >
> > Thanks again. I managed to get my layer to build with your input and
> > ended up simply taking most of your code!
> >
> > But again i think that isar could maybe learn to put the whole WORKDIR
> > into the chroot, where S would be a subdir. So that good old patching
> > and good old recipes remain working.
> >
> > Not sure we really want that, but this very recipe could become part
> > of the tests. As a showcase of "we care about some parts" and
> > "deb_debianize" them out of the whole thing. Possibly .. ing below S
> >
> > Henning
> >
> > > > > > Here is the recipe:
> > > > > >
> > > > > > ifdtool_4.14.bb:
> > > > > > --
> > > > > > inherit dpkg
> > > > > >
> > > > > > DESCRIPTION = "Tool for Fast, secure and flexible OpenSource
> > > > > > firmware"
> > > > > >
> > > > > > SRC_URI =
> > > > > >
> "https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgith
> ub.com%2Fcoreboot%2Fcoreboot%2Farchive%2F%24&amp;data=05%7C01%
> 7Cfelix.moessbauer%40siemens.com%7C7193295dc78f4bb430cb08da6c76354
> 2%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C6379415522032641
> 54%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luM
> zIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=M1
> aymRMjQL%2FCLNZBaqNazI2WV3NS%2BVAAqiG6pVNw4DU%3D&amp;rese
> rved=0{PV}.tar.gz"
> > > > > > SRC_URI[sha256sum] =
> > > > > >
> "73740c313935f21964577fd6621f9acaddf1244cc704e88046f854d93ed6feab"
> > > > > > SRC_URI += "file://Makefile-prefix.patch"
> > > > > >
> > > > > > DEBIAN_DEPENDS = "\${misc:Depends}"
> > > > > >
> > > > > > S = "${WORKDIR}/coreboot-${PV}/util/${PN}"
> > > > > >
> > > > > > do_prepare_build[cleandirs] += "${S}/debian"
> > > > > > do_prepare_build() {
> > > > > >
> > > > > >         deb_debianize
> > > > > >
> > > > > > }
> > > > > > --
> > > > > >
> > > > > > Henning
> 
> 
> --
> Uladzimir Bely
> 
> 
> 
> --
> You received this message because you are subscribed to the Google Groups
> "isar-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to isar-users+unsubscribe@googlegroups.com.
> To view this discussion on the web visit
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgrou
> ps.google.com%2Fd%2Fmsgid%2Fisar-
> users%2F4928480.Qq0lBPeGtt%2540home&amp;data=05%7C01%7Cfelix.mo
> essbauer%40siemens.com%7C7193295dc78f4bb430cb08da6c763542%7C38ae
> 3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637941552203264154%7CUnk
> nown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6I
> k1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=mqjvG2wNLTE
> %2F7CVQSMSBffu1il%2BVyXhiQRIycuOK4IE%3D&amp;reserved=0.