sbuild migration help needed

sbuild migration help needed

[Henning Schild]

Hi,

i have a recipe where i am building just a util from a bigger package,
so my S is somewhere inside WORKDIR and some headers are below S.

With sbuild that recipe stopped working, not finding those headers
anymore. I did not yet look into the details, but maybe only S get put
into that chroot and not all of WORKDIR.

Here is the recipe:

ifdtool_4.14.bb:
--
inherit dpkg

DESCRIPTION = "Tool for Fast, secure and flexible OpenSource firmware"

SRC_URI = "https://github.com/coreboot/coreboot/archive/${PV}.tar.gz"
SRC_URI[sha256sum] =
"73740c313935f21964577fd6621f9acaddf1244cc704e88046f854d93ed6feab"
SRC_URI += "file://Makefile-prefix.patch"

DEBIAN_DEPENDS = "\${misc:Depends}"

S = "${WORKDIR}/coreboot-${PV}/util/${PN}"

do_prepare_build[cleandirs] += "${S}/debian"
do_prepare_build() {
        deb_debianize
}
--

Henning

Re: sbuild migration help needed

[Uladzimir Bely]

In the email from Friday, 22 July 2022 12:16:35 +03 user Henning Schild wrote:
> Hi,
> 
> i have a recipe where i am building just a util from a bigger package,
> so my S is somewhere inside WORKDIR and some headers are below S.
> 
> With sbuild that recipe stopped working, not finding those headers
> anymore. I did not yet look into the details, but maybe only S get put
> into that chroot and not all of WORKDIR.
> 
> Here is the recipe:
> 
> ifdtool_4.14.bb:

Hello. We could try help with it.

> --
> inherit dpkg
> 
> DESCRIPTION = "Tool for Fast, secure and flexible OpenSource firmware"
> 
> SRC_URI = "https://github.com/coreboot/coreboot/archive/${PV}.tar.gz"
> SRC_URI[sha256sum] =
> "73740c313935f21964577fd6621f9acaddf1244cc704e88046f854d93ed6feab"
> SRC_URI += "file://Makefile-prefix.patch"
> 

Could you also share this patch?

> DEBIAN_DEPENDS = "\${misc:Depends}"
> 
> S = "${WORKDIR}/coreboot-${PV}/util/${PN}"
> 
> do_prepare_build[cleandirs] += "${S}/debian"
> do_prepare_build() {
>         deb_debianize
> }
> --
> 
> Henning


-- 
Uladzimir Bely

Re: sbuild migration help needed

[Uladzimir Bely]

In the email from Friday, 22 July 2022 12:16:35 +03 user Henning Schild wrote:
> Hi,
> 
> i have a recipe where i am building just a util from a bigger package,
> so my S is somewhere inside WORKDIR and some headers are below S.
> 
> With sbuild that recipe stopped working, not finding those headers
> anymore. I did not yet look into the details, but maybe only S get put
> into that chroot and not all of WORKDIR.
> 

I've looked a bit at the problem. 

Sbulid doesn't use WORKDIR at all, but builds the package basing on its .dsc 
file. It happens that generated `ifdtool_4.14.dsc` refers to 
`ifdtool_4.14.tar.gz`, which simply doesn't contain upper directories.

I think, something more complex than automatic `deb_debianize` should be used 
here.

> Here is the recipe:
> 
> ifdtool_4.14.bb:
> --
> inherit dpkg
> 
> DESCRIPTION = "Tool for Fast, secure and flexible OpenSource firmware"
> 
> SRC_URI = "https://github.com/coreboot/coreboot/archive/${PV}.tar.gz"
> SRC_URI[sha256sum] =
> "73740c313935f21964577fd6621f9acaddf1244cc704e88046f854d93ed6feab"
> SRC_URI += "file://Makefile-prefix.patch"
> 
> DEBIAN_DEPENDS = "\${misc:Depends}"
> 
> S = "${WORKDIR}/coreboot-${PV}/util/${PN}"
> 
> do_prepare_build[cleandirs] += "${S}/debian"
> do_prepare_build() {
>         deb_debianize
> }
> --
> 
> Henning


-- 
Uladzimir Bely

Re: sbuild migration help needed

[Uladzimir Bely]

In the email from Friday, 22 July 2022 14:11:19 +03 user Uladzimir Bely wrote:
> In the email from Friday, 22 July 2022 12:16:35 +03 user Henning Schild 
wrote:
> > Hi,
> > 
> > i have a recipe where i am building just a util from a bigger package,
> > so my S is somewhere inside WORKDIR and some headers are below S.
> > 
> > With sbuild that recipe stopped working, not finding those headers
> > anymore. I did not yet look into the details, but maybe only S get put
> > into that chroot and not all of WORKDIR.
> 
> I've looked a bit at the problem.
> 
> Sbulid doesn't use WORKDIR at all, but builds the package basing on its .dsc
> file. It happens that generated `ifdtool_4.14.dsc` refers to
> `ifdtool_4.14.tar.gz`, which simply doesn't contain upper directories.
> 
> I think, something more complex than automatic `deb_debianize` should be
> used here.

For instance, it worked for me in form of separate ${S} directory where I copy 
required files and modify Makefile to fit new directory structure:

inherit dpkg

DESCRIPTION = "Tool for Fast, secure and flexible OpenSource firmware"

SRC_URI = "https://github.com/coreboot/coreboot/archive/${PV}.tar.gz"
SRC_URI[sha256sum] = 
"73740c313935f21964577fd6621f9acaddf1244cc704e88046f854d93ed6feab"

DEBIAN_DEPENDS = "\${misc:Depends}"

S = "${WORKDIR}/${PN}-${PV}"

do_prepare_build[cleandirs] += "${S}/debian"
do_prepare_build() {
        cp -rf ${WORKDIR}/coreboot-${PV}/util/${PN}/* ${S}/

        # Copy required sources from upper directories
        cp -rf ${WORKDIR}/coreboot-${PV}/util/cbfstool/flashmap ${S}/
        mkdir -p ${S}/commonlib/include
        cp -rf ${WORKDIR}/coreboot-${PV}/src/commonlib/include/* ${S}/
commonlib/include/
        mkdir -p ${S}/commonlib/bsd/include 
        cp -rf ${WORKDIR}/coreboot-${PV}/src/commonlib/bsd/include/* ${S}/
commonlib/bsd/include/
        # Fix include paths
        sed -i -e "s/\.\.\/cbfstool\/flashmap/flashmap/g" ${S}/Makefile
        sed -i -e "s/\.\.\/\.\.\/src\/commonlib/commonlib/g" ${S}/Makefile

        deb_debianize
        echo "\noverride_dh_usrlocal:" >> ${S}/debian/rules
}

> 
> > Here is the recipe:
> > 
> > ifdtool_4.14.bb:
> > --
> > inherit dpkg
> > 
> > DESCRIPTION = "Tool for Fast, secure and flexible OpenSource firmware"
> > 
> > SRC_URI = "https://github.com/coreboot/coreboot/archive/${PV}.tar.gz"
> > SRC_URI[sha256sum] =
> > "73740c313935f21964577fd6621f9acaddf1244cc704e88046f854d93ed6feab"
> > SRC_URI += "file://Makefile-prefix.patch"
> > 
> > DEBIAN_DEPENDS = "\${misc:Depends}"
> > 
> > S = "${WORKDIR}/coreboot-${PV}/util/${PN}"
> > 
> > do_prepare_build[cleandirs] += "${S}/debian"
> > do_prepare_build() {
> > 
> >         deb_debianize
> > 
> > }
> > --
> > 
> > Henning


-- 
Uladzimir Bely

Re: sbuild migration help needed

[Henning Schild]

Thanks! I will give that a try. Sorry for not sharing that patch, it
is trivial.

I find it a little worrying that a recipe which worked perfectly before
all of a sudden has to be significantly changed. If there would have
been a forgotten build-dep i would have been happy, but here we have a
maybe dramatic change in how the recipe needs to be written.

I will change it as you suggested. Thanks again! But we will have to
look out for such big changes breaking existing recipes. All the big
API changes can and will cause a lot of frustration in all the various
layers.

And there are the known ones, which are in the changelog ... plus the
ones that just happen ... like this one, or the meson/ccache i just
sent a patch for.

Henning

--
Index: ifdtool/Makefile
===================================================================
--- ifdtool.orig/Makefile
+++ ifdtool/Makefile
@@ -6,7 +6,7 @@ PROGRAM = ifdtool
 
 CC      = gcc
 INSTALL = /usr/bin/env install
-PREFIX  = /usr/local
+PREFIX  = /usr
 CFLAGS  = -O2 -g -Wall -Wextra -Wmissing-prototypes -Werror
 CFLAGS += -I../../src/commonlib/include
-I../../src/commonlib/bsd/include CFLAGS += -I../cbfstool/flashmap
--


Am Fri, 22 Jul 2022 16:08:45 +0300
schrieb Uladzimir Bely <ubely@ilbers.de>:

> In the email from Friday, 22 July 2022 14:11:19 +03 user Uladzimir
> Bely wrote:
> > In the email from Friday, 22 July 2022 12:16:35 +03 user Henning
> > Schild   
> wrote:
> > > Hi,
> > > 
> > > i have a recipe where i am building just a util from a bigger
> > > package, so my S is somewhere inside WORKDIR and some headers are
> > > below S.
> > > 
> > > With sbuild that recipe stopped working, not finding those headers
> > > anymore. I did not yet look into the details, but maybe only S
> > > get put into that chroot and not all of WORKDIR.  
> > 
> > I've looked a bit at the problem.
> > 
> > Sbulid doesn't use WORKDIR at all, but builds the package basing on
> > its .dsc file. It happens that generated `ifdtool_4.14.dsc` refers
> > to `ifdtool_4.14.tar.gz`, which simply doesn't contain upper
> > directories.
> > 
> > I think, something more complex than automatic `deb_debianize`
> > should be used here.  
> 
> For instance, it worked for me in form of separate ${S} directory
> where I copy required files and modify Makefile to fit new directory
> structure:
> 
> inherit dpkg
> 
> DESCRIPTION = "Tool for Fast, secure and flexible OpenSource firmware"
> 
> SRC_URI = "https://github.com/coreboot/coreboot/archive/${PV}.tar.gz"
> SRC_URI[sha256sum] = 
> "73740c313935f21964577fd6621f9acaddf1244cc704e88046f854d93ed6feab"
> 
> DEBIAN_DEPENDS = "\${misc:Depends}"
> 
> S = "${WORKDIR}/${PN}-${PV}"
> 
> do_prepare_build[cleandirs] += "${S}/debian"
> do_prepare_build() {
>         cp -rf ${WORKDIR}/coreboot-${PV}/util/${PN}/* ${S}/
> 
>         # Copy required sources from upper directories
>         cp -rf ${WORKDIR}/coreboot-${PV}/util/cbfstool/flashmap ${S}/
>         mkdir -p ${S}/commonlib/include
>         cp -rf ${WORKDIR}/coreboot-${PV}/src/commonlib/include/* ${S}/
> commonlib/include/
>         mkdir -p ${S}/commonlib/bsd/include 
>         cp -rf ${WORKDIR}/coreboot-${PV}/src/commonlib/bsd/include/*
> ${S}/ commonlib/bsd/include/
>         # Fix include paths
>         sed -i -e "s/\.\.\/cbfstool\/flashmap/flashmap/g"
> ${S}/Makefile sed -i -e "s/\.\.\/\.\.\/src\/commonlib/commonlib/g"
> ${S}/Makefile
> 
>         deb_debianize
>         echo "\noverride_dh_usrlocal:" >> ${S}/debian/rules
> }
> 
> >   
> > > Here is the recipe:
> > > 
> > > ifdtool_4.14.bb:
> > > --
> > > inherit dpkg
> > > 
> > > DESCRIPTION = "Tool for Fast, secure and flexible OpenSource
> > > firmware"
> > > 
> > > SRC_URI =
> > > "https://github.com/coreboot/coreboot/archive/${PV}.tar.gz"
> > > SRC_URI[sha256sum] =
> > > "73740c313935f21964577fd6621f9acaddf1244cc704e88046f854d93ed6feab"
> > > SRC_URI += "file://Makefile-prefix.patch"
> > > 
> > > DEBIAN_DEPENDS = "\${misc:Depends}"
> > > 
> > > S = "${WORKDIR}/coreboot-${PV}/util/${PN}"
> > > 
> > > do_prepare_build[cleandirs] += "${S}/debian"
> > > do_prepare_build() {
> > > 
> > >         deb_debianize
> > > 
> > > }
> > > --
> > > 
> > > Henning  
> 
> 

Re: sbuild migration help needed

[Henning Schild]

Am Fri, 22 Jul 2022 16:19:07 +0200
schrieb Henning Schild <henning.schild@siemens.com>:

> Thanks! I will give that a try. Sorry for not sharing that patch, it
> is trivial.

In fact that patch also does not apply anymore but did before sbuild.
You made it a

  echo "\noverride_dh_usrlocal:" >> ${S}/debian/rules

And i ended up turning into another sed.

> I find it a little worrying that a recipe which worked perfectly
> before all of a sudden has to be significantly changed. If there
> would have been a forgotten build-dep i would have been happy, but
> here we have a maybe dramatic change in how the recipe needs to be
> written.

This recipe was easy enough to share but it shows a fundamental change
in isar. Where before sbuild all of WORKDIR became available in the
buildchroot, now we only get S.

I would not know how many other recipes in countless layers would care,
but maybe we can somehow get back to WORKDIR and not only S being
mounted and usable in the chroot.

That recipe is a classic hack where one just packages something with
deb_debianize and the patch features of isar. Being forced to move code
around to arrange an S where WORKDIR would simply work (and be
patchable) is not nice. More involved projects might require more
involved "constructing S" steps.

We are moving code around and patching Makefile here, so all of a
sudden you need to really understand what you previously could
deb_debianize around with.

I guess the whole thing is relevant for projects where parts of the
code you care about live in a bigger repo.

> I will change it as you suggested. Thanks again! But we will have to
> look out for such big changes breaking existing recipes. All the big
> API changes can and will cause a lot of frustration in all the various
> layers.
> 
> And there are the known ones, which are in the changelog ... plus the
> ones that just happen ... like this one, or the meson/ccache i just
> sent a patch for.
> 
> Henning
> 
> --
> Index: ifdtool/Makefile
> ===================================================================
> --- ifdtool.orig/Makefile
> +++ ifdtool/Makefile
> @@ -6,7 +6,7 @@ PROGRAM = ifdtool
>  
>  CC      = gcc
>  INSTALL = /usr/bin/env install
> -PREFIX  = /usr/local
> +PREFIX  = /usr
>  CFLAGS  = -O2 -g -Wall -Wextra -Wmissing-prototypes -Werror
>  CFLAGS += -I../../src/commonlib/include
> -I../../src/commonlib/bsd/include CFLAGS += -I../cbfstool/flashmap
> --
> 
> 
> Am Fri, 22 Jul 2022 16:08:45 +0300
> schrieb Uladzimir Bely <ubely@ilbers.de>:
> 
> > In the email from Friday, 22 July 2022 14:11:19 +03 user Uladzimir
> > Bely wrote:  
> > > In the email from Friday, 22 July 2022 12:16:35 +03 user Henning
> > > Schild     
> > wrote:  
> > > > Hi,
> > > > 
> > > > i have a recipe where i am building just a util from a bigger
> > > > package, so my S is somewhere inside WORKDIR and some headers
> > > > are below S.
> > > > 
> > > > With sbuild that recipe stopped working, not finding those
> > > > headers anymore. I did not yet look into the details, but maybe
> > > > only S get put into that chroot and not all of WORKDIR.    
> > > 
> > > I've looked a bit at the problem.
> > > 
> > > Sbulid doesn't use WORKDIR at all, but builds the package basing
> > > on its .dsc file. It happens that generated `ifdtool_4.14.dsc`
> > > refers to `ifdtool_4.14.tar.gz`, which simply doesn't contain
> > > upper directories.
> > > 
> > > I think, something more complex than automatic `deb_debianize`
> > > should be used here.    
> > 
> > For instance, it worked for me in form of separate ${S} directory
> > where I copy required files and modify Makefile to fit new directory
> > structure:
> > 
> > inherit dpkg
> > 
> > DESCRIPTION = "Tool for Fast, secure and flexible OpenSource
> > firmware"
> > 
> > SRC_URI =
> > "https://github.com/coreboot/coreboot/archive/${PV}.tar.gz"
> > SRC_URI[sha256sum] =
> > "73740c313935f21964577fd6621f9acaddf1244cc704e88046f854d93ed6feab"
> > 
> > DEBIAN_DEPENDS = "\${misc:Depends}"
> > 
> > S = "${WORKDIR}/${PN}-${PV}"

this is nothing but WORKDIR/P ... the default ... can be skipped

> > 
> > do_prepare_build[cleandirs] += "${S}/debian"

has to be S not S/debian

> > do_prepare_build() {
> >         cp -rf ${WORKDIR}/coreboot-${PV}/util/${PN}/* ${S}/
> > 
> >         # Copy required sources from upper directories
> >         cp -rf ${WORKDIR}/coreboot-${PV}/util/cbfstool/flashmap
> > ${S}/ mkdir -p ${S}/commonlib/include
> >         cp -rf ${WORKDIR}/coreboot-${PV}/src/commonlib/include/*
> > ${S}/ commonlib/include/
> >         mkdir -p ${S}/commonlib/bsd/include 
> >         cp -rf ${WORKDIR}/coreboot-${PV}/src/commonlib/bsd/include/*
> > ${S}/ commonlib/bsd/include/
> >         # Fix include paths
> >         sed -i -e "s/\.\.\/cbfstool\/flashmap/flashmap/g"
> > ${S}/Makefile sed -i -e "s/\.\.\/\.\.\/src\/commonlib/commonlib/g"
> > ${S}/Makefile
> > 
> >         deb_debianize
> >         echo "\noverride_dh_usrlocal:" >> ${S}/debian/rules
> > }

Thanks again. I managed to get my layer to build with your input and
ended up simply taking most of your code!

But again i think that isar could maybe learn to put the whole WORKDIR
into the chroot, where S would be a subdir. So that good old patching
and good old recipes remain working.

Not sure we really want that, but this very recipe could become part of
the tests. As a showcase of "we care about some parts" and
"deb_debianize" them out of the whole thing. Possibly .. ing below S

Henning

> >   
> > >     
> > > > Here is the recipe:
> > > > 
> > > > ifdtool_4.14.bb:
> > > > --
> > > > inherit dpkg
> > > > 
> > > > DESCRIPTION = "Tool for Fast, secure and flexible OpenSource
> > > > firmware"
> > > > 
> > > > SRC_URI =
> > > > "https://github.com/coreboot/coreboot/archive/${PV}.tar.gz"
> > > > SRC_URI[sha256sum] =
> > > > "73740c313935f21964577fd6621f9acaddf1244cc704e88046f854d93ed6feab"
> > > > SRC_URI += "file://Makefile-prefix.patch"
> > > > 
> > > > DEBIAN_DEPENDS = "\${misc:Depends}"
> > > > 
> > > > S = "${WORKDIR}/coreboot-${PV}/util/${PN}"
> > > > 
> > > > do_prepare_build[cleandirs] += "${S}/debian"
> > > > do_prepare_build() {
> > > > 
> > > >         deb_debianize
> > > > 
> > > > }
> > > > --
> > > > 
> > > > Henning    
> > 
> >   
> 

Re: sbuild migration help needed

[Uladzimir Bely]

In the email from Friday, 22 July 2022 22:16:56 +03 user Henning Schild wrote:
> Am Fri, 22 Jul 2022 16:19:07 +0200
> 
> schrieb Henning Schild <henning.schild@siemens.com>:
> > Thanks! I will give that a try. Sorry for not sharing that patch, it
> > is trivial.
> 
> In fact that patch also does not apply anymore but did before sbuild.
> You made it a
> 
>   echo "\noverride_dh_usrlocal:" >> ${S}/debian/rules
> 
> And i ended up turning into another sed.
> 
> > I find it a little worrying that a recipe which worked perfectly
> > before all of a sudden has to be significantly changed. If there
> > would have been a forgotten build-dep i would have been happy, but
> > here we have a maybe dramatic change in how the recipe needs to be
> > written.
> 
> This recipe was easy enough to share but it shows a fundamental change
> in isar. Where before sbuild all of WORKDIR became available in the
> buildchroot, now we only get S.
> 
> I would not know how many other recipes in countless layers would care,
> but maybe we can somehow get back to WORKDIR and not only S being
> mounted and usable in the chroot.
> 
> That recipe is a classic hack where one just packages something with
> deb_debianize and the patch features of isar. Being forced to move code
> around to arrange an S where WORKDIR would simply work (and be
> patchable) is not nice. More involved projects might require more
> involved "constructing S" steps.
> 
> We are moving code around and patching Makefile here, so all of a
> sudden you need to really understand what you previously could
> deb_debianize around with.
> 
> I guess the whole thing is relevant for projects where parts of the
> code you care about live in a bigger repo.
> 

Yes, there is a topic for the discussion.

On the other hand, before sbuild the recipe was able to be compiled. But, what 
about generated (after the build) source package in workdir? I'm pretty sure, 
that it was unbuildable outside of Isar due to missing sources.

I'm trying to say that having ${S} deep in ${WORKDIR} is generally a wrong 
solution producing unbuildable packages (if depends on upper directories).

> > I will change it as you suggested. Thanks again! But we will have to
> > look out for such big changes breaking existing recipes. All the big
> > API changes can and will cause a lot of frustration in all the various
> > layers.
> > 
> > And there are the known ones, which are in the changelog ... plus the
> > ones that just happen ... like this one, or the meson/ccache i just
> > sent a patch for.
> > 
> > Henning
> > 
> > --
> > Index: ifdtool/Makefile
> > ===================================================================
> > --- ifdtool.orig/Makefile
> > +++ ifdtool/Makefile
> > @@ -6,7 +6,7 @@ PROGRAM = ifdtool
> > 
> >  CC      = gcc
> >  INSTALL = /usr/bin/env install
> > 
> > -PREFIX  = /usr/local
> > +PREFIX  = /usr
> > 
> >  CFLAGS  = -O2 -g -Wall -Wextra -Wmissing-prototypes -Werror
> >  CFLAGS += -I../../src/commonlib/include
> > 
> > -I../../src/commonlib/bsd/include CFLAGS += -I../cbfstool/flashmap
> > --
> > 
> > 
> > Am Fri, 22 Jul 2022 16:08:45 +0300
> > 
> > schrieb Uladzimir Bely <ubely@ilbers.de>:
> > > In the email from Friday, 22 July 2022 14:11:19 +03 user Uladzimir
> > > 
> > > Bely wrote:
> > > > In the email from Friday, 22 July 2022 12:16:35 +03 user Henning
> > > > Schild
> > > 
> > > wrote:
> > > > > Hi,
> > > > > 
> > > > > i have a recipe where i am building just a util from a bigger
> > > > > package, so my S is somewhere inside WORKDIR and some headers
> > > > > are below S.
> > > > > 
> > > > > With sbuild that recipe stopped working, not finding those
> > > > > headers anymore. I did not yet look into the details, but maybe
> > > > > only S get put into that chroot and not all of WORKDIR.
> > > > 
> > > > I've looked a bit at the problem.
> > > > 
> > > > Sbulid doesn't use WORKDIR at all, but builds the package basing
> > > > on its .dsc file. It happens that generated `ifdtool_4.14.dsc`
> > > > refers to `ifdtool_4.14.tar.gz`, which simply doesn't contain
> > > > upper directories.
> > > > 
> > > > I think, something more complex than automatic `deb_debianize`
> > > > should be used here.
> > > 
> > > For instance, it worked for me in form of separate ${S} directory
> > > where I copy required files and modify Makefile to fit new directory
> > > structure:
> > > 
> > > inherit dpkg
> > > 
> > > DESCRIPTION = "Tool for Fast, secure and flexible OpenSource
> > > firmware"
> > > 
> > > SRC_URI =
> > > "https://github.com/coreboot/coreboot/archive/${PV}.tar.gz"
> > > SRC_URI[sha256sum] =
> > > "73740c313935f21964577fd6621f9acaddf1244cc704e88046f854d93ed6feab"
> > > 
> > > DEBIAN_DEPENDS = "\${misc:Depends}"
> > > 
> > > S = "${WORKDIR}/${PN}-${PV}"
> 
> this is nothing but WORKDIR/P ... the default ... can be skipped
> 
> > > do_prepare_build[cleandirs] += "${S}/debian"
> 
> has to be S not S/debian
> 
> > > do_prepare_build() {
> > > 
> > >         cp -rf ${WORKDIR}/coreboot-${PV}/util/${PN}/* ${S}/
> > >         
> > >         # Copy required sources from upper directories
> > >         cp -rf ${WORKDIR}/coreboot-${PV}/util/cbfstool/flashmap
> > > 
> > > ${S}/ mkdir -p ${S}/commonlib/include
> > > 
> > >         cp -rf ${WORKDIR}/coreboot-${PV}/src/commonlib/include/*
> > > 
> > > ${S}/ commonlib/include/
> > > 
> > >         mkdir -p ${S}/commonlib/bsd/include
> > >         cp -rf ${WORKDIR}/coreboot-${PV}/src/commonlib/bsd/include/*
> > > 
> > > ${S}/ commonlib/bsd/include/
> > > 
> > >         # Fix include paths
> > >         sed -i -e "s/\.\.\/cbfstool\/flashmap/flashmap/g"
> > > 
> > > ${S}/Makefile sed -i -e "s/\.\.\/\.\.\/src\/commonlib/commonlib/g"
> > > ${S}/Makefile
> > > 
> > >         deb_debianize
> > >         echo "\noverride_dh_usrlocal:" >> ${S}/debian/rules
> > > 
> > > }
> 
> Thanks again. I managed to get my layer to build with your input and
> ended up simply taking most of your code!
> 
> But again i think that isar could maybe learn to put the whole WORKDIR
> into the chroot, where S would be a subdir. So that good old patching
> and good old recipes remain working.
> 
> Not sure we really want that, but this very recipe could become part of
> the tests. As a showcase of "we care about some parts" and
> "deb_debianize" them out of the whole thing. Possibly .. ing below S
> 
> Henning
> 
> > > > > Here is the recipe:
> > > > > 
> > > > > ifdtool_4.14.bb:
> > > > > --
> > > > > inherit dpkg
> > > > > 
> > > > > DESCRIPTION = "Tool for Fast, secure and flexible OpenSource
> > > > > firmware"
> > > > > 
> > > > > SRC_URI =
> > > > > "https://github.com/coreboot/coreboot/archive/${PV}.tar.gz"
> > > > > SRC_URI[sha256sum] =
> > > > > "73740c313935f21964577fd6621f9acaddf1244cc704e88046f854d93ed6feab"
> > > > > SRC_URI += "file://Makefile-prefix.patch"
> > > > > 
> > > > > DEBIAN_DEPENDS = "\${misc:Depends}"
> > > > > 
> > > > > S = "${WORKDIR}/coreboot-${PV}/util/${PN}"
> > > > > 
> > > > > do_prepare_build[cleandirs] += "${S}/debian"
> > > > > do_prepare_build() {
> > > > > 
> > > > >         deb_debianize
> > > > > 
> > > > > }
> > > > > --
> > > > > 
> > > > > Henning


-- 
Uladzimir Bely

RE: sbuild migration help needed

[Moessbauer, Felix]

> -----Original Message-----
> From: isar-users@googlegroups.com <isar-users@googlegroups.com> On
> Behalf Of Uladzimir Bely
> Sent: Saturday, July 23, 2022 8:40 AM
> To: Schild, Henning (T CED SES-DE) <henning.schild@siemens.com>
> Cc: isar-users <isar-users@googlegroups.com>
> Subject: Re: sbuild migration help needed
> 
> In the email from Friday, 22 July 2022 22:16:56 +03 user Henning Schild wrote:
> > Am Fri, 22 Jul 2022 16:19:07 +0200
> >
> > schrieb Henning Schild <henning.schild@siemens.com>:
> > > Thanks! I will give that a try. Sorry for not sharing that patch, it
> > > is trivial.
> >
> > In fact that patch also does not apply anymore but did before sbuild.
> > You made it a
> >
> >   echo "\noverride_dh_usrlocal:" >> ${S}/debian/rules
> >
> > And i ended up turning into another sed.
> >
> > > I find it a little worrying that a recipe which worked perfectly
> > > before all of a sudden has to be significantly changed. If there
> > > would have been a forgotten build-dep i would have been happy, but
> > > here we have a maybe dramatic change in how the recipe needs to be
> > > written.
> >
> > This recipe was easy enough to share but it shows a fundamental change
> > in isar. Where before sbuild all of WORKDIR became available in the
> > buildchroot, now we only get S.
> >
> > I would not know how many other recipes in countless layers would
> > care, but maybe we can somehow get back to WORKDIR and not only S
> > being mounted and usable in the chroot.
> >
> > That recipe is a classic hack where one just packages something with
> > deb_debianize and the patch features of isar. Being forced to move
> > code around to arrange an S where WORKDIR would simply work (and be
> > patchable) is not nice. More involved projects might require more
> > involved "constructing S" steps.
> >
> > We are moving code around and patching Makefile here, so all of a
> > sudden you need to really understand what you previously could
> > deb_debianize around with.
> >
> > I guess the whole thing is relevant for projects where parts of the
> > code you care about live in a bigger repo.
> >
> 
> Yes, there is a topic for the discussion.
> 
> On the other hand, before sbuild the recipe was able to be compiled. But,
> what about generated (after the build) source package in workdir? I'm pretty
> sure, that it was unbuildable outside of Isar due to missing sources.
> 
> I'm trying to say that having ${S} deep in ${WORKDIR} is generally a wrong
> solution producing unbuildable packages (if depends on upper directories).

Yes, I see it the same way - ${S} should be the root dir of the package.
Accessing files outside of ${S} just worked before because there was no isolation of the dpkg-buildpackage.
Having this isolation brings a huge benefit of generating re-buildable source packages.
We really should not give up on that, but better fix the erroneous packagings downstream.

ISAR is not Yocto, so we better stick to Debian policies instead of just making things work.
This becomes relevant once not only images are generated but also package registries.

Felix

> 
> > > I will change it as you suggested. Thanks again! But we will have to
> > > look out for such big changes breaking existing recipes. All the big
> > > API changes can and will cause a lot of frustration in all the
> > > various layers.
> > >
> > > And there are the known ones, which are in the changelog ... plus
> > > the ones that just happen ... like this one, or the meson/ccache i
> > > just sent a patch for.
> > >
> > > Henning
> > >
> > > --
> > > Index: ifdtool/Makefile
> > >
> ==========================================================
> =========
> > > --- ifdtool.orig/Makefile
> > > +++ ifdtool/Makefile
> > > @@ -6,7 +6,7 @@ PROGRAM = ifdtool
> > >
> > >  CC      = gcc
> > >  INSTALL = /usr/bin/env install
> > >
> > > -PREFIX  = /usr/local
> > > +PREFIX  = /usr
> > >
> > >  CFLAGS  = -O2 -g -Wall -Wextra -Wmissing-prototypes -Werror  CFLAGS
> > > += -I../../src/commonlib/include
> > >
> > > -I../../src/commonlib/bsd/include CFLAGS += -I../cbfstool/flashmap
> > > --
> > >
> > >
> > > Am Fri, 22 Jul 2022 16:08:45 +0300
> > >
> > > schrieb Uladzimir Bely <ubely@ilbers.de>:
> > > > In the email from Friday, 22 July 2022 14:11:19 +03 user Uladzimir
> > > >
> > > > Bely wrote:
> > > > > In the email from Friday, 22 July 2022 12:16:35 +03 user Henning
> > > > > Schild
> > > >
> > > > wrote:
> > > > > > Hi,
> > > > > >
> > > > > > i have a recipe where i am building just a util from a bigger
> > > > > > package, so my S is somewhere inside WORKDIR and some headers
> > > > > > are below S.
> > > > > >
> > > > > > With sbuild that recipe stopped working, not finding those
> > > > > > headers anymore. I did not yet look into the details, but
> > > > > > maybe only S get put into that chroot and not all of WORKDIR.
> > > > >
> > > > > I've looked a bit at the problem.
> > > > >
> > > > > Sbulid doesn't use WORKDIR at all, but builds the package basing
> > > > > on its .dsc file. It happens that generated `ifdtool_4.14.dsc`
> > > > > refers to `ifdtool_4.14.tar.gz`, which simply doesn't contain
> > > > > upper directories.
> > > > >
> > > > > I think, something more complex than automatic `deb_debianize`
> > > > > should be used here.
> > > >
> > > > For instance, it worked for me in form of separate ${S} directory
> > > > where I copy required files and modify Makefile to fit new
> > > > directory
> > > > structure:
> > > >
> > > > inherit dpkg
> > > >
> > > > DESCRIPTION = "Tool for Fast, secure and flexible OpenSource
> > > > firmware"
> > > >
> > > > SRC_URI =
> > > >
> "https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgith
> ub.com%2Fcoreboot%2Fcoreboot%2Farchive%2F%24&amp;data=05%7C01%
> 7Cfelix.moessbauer%40siemens.com%7C7193295dc78f4bb430cb08da6c76354
> 2%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C6379415522032641
> 54%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luM
> zIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=M1
> aymRMjQL%2FCLNZBaqNazI2WV3NS%2BVAAqiG6pVNw4DU%3D&amp;rese
> rved=0{PV}.tar.gz"
> > > > SRC_URI[sha256sum] =
> > > >
> "73740c313935f21964577fd6621f9acaddf1244cc704e88046f854d93ed6feab"
> > > >
> > > > DEBIAN_DEPENDS = "\${misc:Depends}"
> > > >
> > > > S = "${WORKDIR}/${PN}-${PV}"
> >
> > this is nothing but WORKDIR/P ... the default ... can be skipped
> >
> > > > do_prepare_build[cleandirs] += "${S}/debian"
> >
> > has to be S not S/debian
> >
> > > > do_prepare_build() {
> > > >
> > > >         cp -rf ${WORKDIR}/coreboot-${PV}/util/${PN}/* ${S}/
> > > >
> > > >         # Copy required sources from upper directories
> > > >         cp -rf ${WORKDIR}/coreboot-${PV}/util/cbfstool/flashmap
> > > >
> > > > ${S}/ mkdir -p ${S}/commonlib/include
> > > >
> > > >         cp -rf ${WORKDIR}/coreboot-${PV}/src/commonlib/include/*
> > > >
> > > > ${S}/ commonlib/include/
> > > >
> > > >         mkdir -p ${S}/commonlib/bsd/include
> > > >         cp -rf
> > > > ${WORKDIR}/coreboot-${PV}/src/commonlib/bsd/include/*
> > > >
> > > > ${S}/ commonlib/bsd/include/
> > > >
> > > >         # Fix include paths
> > > >         sed -i -e "s/\.\.\/cbfstool\/flashmap/flashmap/g"
> > > >
> > > > ${S}/Makefile sed -i -e "s/\.\.\/\.\.\/src\/commonlib/commonlib/g"
> > > > ${S}/Makefile
> > > >
> > > >         deb_debianize
> > > >         echo "\noverride_dh_usrlocal:" >> ${S}/debian/rules
> > > >
> > > > }
> >
> > Thanks again. I managed to get my layer to build with your input and
> > ended up simply taking most of your code!
> >
> > But again i think that isar could maybe learn to put the whole WORKDIR
> > into the chroot, where S would be a subdir. So that good old patching
> > and good old recipes remain working.
> >
> > Not sure we really want that, but this very recipe could become part
> > of the tests. As a showcase of "we care about some parts" and
> > "deb_debianize" them out of the whole thing. Possibly .. ing below S
> >
> > Henning
> >
> > > > > > Here is the recipe:
> > > > > >
> > > > > > ifdtool_4.14.bb:
> > > > > > --
> > > > > > inherit dpkg
> > > > > >
> > > > > > DESCRIPTION = "Tool for Fast, secure and flexible OpenSource
> > > > > > firmware"
> > > > > >
> > > > > > SRC_URI =
> > > > > >
> "https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgith
> ub.com%2Fcoreboot%2Fcoreboot%2Farchive%2F%24&amp;data=05%7C01%
> 7Cfelix.moessbauer%40siemens.com%7C7193295dc78f4bb430cb08da6c76354
> 2%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C6379415522032641
> 54%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luM
> zIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=M1
> aymRMjQL%2FCLNZBaqNazI2WV3NS%2BVAAqiG6pVNw4DU%3D&amp;rese
> rved=0{PV}.tar.gz"
> > > > > > SRC_URI[sha256sum] =
> > > > > >
> "73740c313935f21964577fd6621f9acaddf1244cc704e88046f854d93ed6feab"
> > > > > > SRC_URI += "file://Makefile-prefix.patch"
> > > > > >
> > > > > > DEBIAN_DEPENDS = "\${misc:Depends}"
> > > > > >
> > > > > > S = "${WORKDIR}/coreboot-${PV}/util/${PN}"
> > > > > >
> > > > > > do_prepare_build[cleandirs] += "${S}/debian"
> > > > > > do_prepare_build() {
> > > > > >
> > > > > >         deb_debianize
> > > > > >
> > > > > > }
> > > > > > --
> > > > > >
> > > > > > Henning
> 
> 
> --
> Uladzimir Bely
> 
> 
> 
> --
> You received this message because you are subscribed to the Google Groups
> "isar-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to isar-users+unsubscribe@googlegroups.com.
> To view this discussion on the web visit
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgrou
> ps.google.com%2Fd%2Fmsgid%2Fisar-
> users%2F4928480.Qq0lBPeGtt%2540home&amp;data=05%7C01%7Cfelix.mo
> essbauer%40siemens.com%7C7193295dc78f4bb430cb08da6c763542%7C38ae
> 3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637941552203264154%7CUnk
> nown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6I
> k1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=mqjvG2wNLTE
> %2F7CVQSMSBffu1il%2BVyXhiQRIycuOK4IE%3D&amp;reserved=0.