[PATCH] meta/dpkg-raw: do_install into a fresh directory every time

[PATCH] meta/dpkg-raw: do_install into a fresh directory every time

[Henning Schild]

Permission problems revealed that we always packaged the sum of possibly
many rebuilds and maybe recipe versions. do_install should always start
working on an empty $D to ensure that the package only contains output
of the current build.

Reported-by: Jan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
 meta/classes/dpkg-raw.bbclass | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/meta/classes/dpkg-raw.bbclass b/meta/classes/dpkg-raw.bbclass
index d662422..968e28d 100644
--- a/meta/classes/dpkg-raw.bbclass
+++ b/meta/classes/dpkg-raw.bbclass
@@ -8,13 +8,21 @@ MAINTAINER ?= "FIXME Unknown maintainer"
 
 D = "${WORKDIR}/image/"
 
+# make sure we start with a fresh ${D} every single time
+do_fresh_image() {
+	sudo rm -rf ${D}
+}
+
+do_fresh_image[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
+addtask fresh_image after do_unpack before do_install
+
 # Populate folder that will be picked up as package
 do_install() {
 	bbnote "Put your files for this package in ${D}"
 }
 
 do_install[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
-addtask install after do_unpack before do_prepare_build
+addtask install after do_fresh_image before do_prepare_build
 
 deb_package_prepare() {
 	sudo rm -rf ${D}/DEBIAN
-- 
2.19.1

Re: [PATCH] meta/dpkg-raw: do_install into a fresh directory every time

[Jan Kiszka]

On 14.11.18 16:39, Henning Schild wrote:
> Permission problems revealed that we always packaged the sum of possibly
> many rebuilds and maybe recipe versions. do_install should always start
> working on an empty $D to ensure that the package only contains output
> of the current build.
> 
> Reported-by: Jan Kiszka <jan.kiszka@siemens.com>
> Signed-off-by: Henning Schild <henning.schild@siemens.com>
> ---
>   meta/classes/dpkg-raw.bbclass | 10 +++++++++-
>   1 file changed, 9 insertions(+), 1 deletion(-)
> 
> diff --git a/meta/classes/dpkg-raw.bbclass b/meta/classes/dpkg-raw.bbclass
> index d662422..968e28d 100644
> --- a/meta/classes/dpkg-raw.bbclass
> +++ b/meta/classes/dpkg-raw.bbclass
> @@ -8,13 +8,21 @@ MAINTAINER ?= "FIXME Unknown maintainer"
>   
>   D = "${WORKDIR}/image/"
>   
> +# make sure we start with a fresh ${D} every single time
> +do_fresh_image() {
> +	sudo rm -rf ${D}
> +}
> +
> +do_fresh_image[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
> +addtask fresh_image after do_unpack before do_install
> +
>   # Populate folder that will be picked up as package
>   do_install() {
>   	bbnote "Put your files for this package in ${D}"
>   }
>   
>   do_install[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
> -addtask install after do_unpack before do_prepare_build
> +addtask install after do_fresh_image before do_prepare_build
>   
>   deb_package_prepare() {
>   	sudo rm -rf ${D}/DEBIAN
> 

Works well here and fixes the issue I saw.

Thanks,
Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

Re: [PATCH] meta/dpkg-raw: do_install into a fresh directory every time

[Henning Schild]

Please do not merge, i think it is still not correct. bitbake can still
run do_install without the cleanup if do_install changed and the stamp
is still there.
This should be [cleandirs] .. but we need sudo. So probably a "prefunc".

I think we might have that pattern in a few places. do_patch might have
this problem as well, if we ever patch again we also need to unpack
again.

Still have to confirm all that.

Henning

Am Wed, 14 Nov 2018 16:39:55 +0100
schrieb Henning Schild <henning.schild@siemens.com>:

> Permission problems revealed that we always packaged the sum of
> possibly many rebuilds and maybe recipe versions. do_install should
> always start working on an empty $D to ensure that the package only
> contains output of the current build.
> 
> Reported-by: Jan Kiszka <jan.kiszka@siemens.com>
> Signed-off-by: Henning Schild <henning.schild@siemens.com>
> ---
>  meta/classes/dpkg-raw.bbclass | 10 +++++++++-
>  1 file changed, 9 insertions(+), 1 deletion(-)
> 
> diff --git a/meta/classes/dpkg-raw.bbclass
> b/meta/classes/dpkg-raw.bbclass index d662422..968e28d 100644
> --- a/meta/classes/dpkg-raw.bbclass
> +++ b/meta/classes/dpkg-raw.bbclass
> @@ -8,13 +8,21 @@ MAINTAINER ?= "FIXME Unknown maintainer"
>  
>  D = "${WORKDIR}/image/"
>  
> +# make sure we start with a fresh ${D} every single time
> +do_fresh_image() {
> +	sudo rm -rf ${D}
> +}
> +
> +do_fresh_image[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
> +addtask fresh_image after do_unpack before do_install
> +
>  # Populate folder that will be picked up as package
>  do_install() {
>  	bbnote "Put your files for this package in ${D}"
>  }
>  
>  do_install[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
> -addtask install after do_unpack before do_prepare_build
> +addtask install after do_fresh_image before do_prepare_build
>  
>  deb_package_prepare() {
>  	sudo rm -rf ${D}/DEBIAN

Re: [PATCH] meta/dpkg-raw: do_install into a fresh directory every time

[Henning Schild]

Am Thu, 15 Nov 2018 09:25:44 +0100
schrieb "[ext] Henning Schild" <henning.schild@siemens.com>:

> Please do not merge, i think it is still not correct. bitbake can
> still run do_install without the cleanup if do_install changed and
> the stamp is still there.
> This should be [cleandirs] .. but we need sudo. So probably a
> "prefunc".
> 
> I think we might have that pattern in a few places. do_patch might
> have this problem as well, if we ever patch again we also need to
> unpack again.
> 
> Still have to confirm all that.

As expected ... this does not work. And do_patch has the same
problem ... and probably a few more places.

To reproduce the do_patch issue add

do_patch_append() {
       true
}

to meta-isar/recipes-app/example-hello/example-hello.bb between two
builds. One could also argue that such a thing will never happen ... but
i would not be sure.

I still did not understand how or if oe solves this.

Henning

> Henning
> 
> Am Wed, 14 Nov 2018 16:39:55 +0100
> schrieb Henning Schild <henning.schild@siemens.com>:
> 
> > Permission problems revealed that we always packaged the sum of
> > possibly many rebuilds and maybe recipe versions. do_install should
> > always start working on an empty $D to ensure that the package only
> > contains output of the current build.
> > 
> > Reported-by: Jan Kiszka <jan.kiszka@siemens.com>
> > Signed-off-by: Henning Schild <henning.schild@siemens.com>
> > ---
> >  meta/classes/dpkg-raw.bbclass | 10 +++++++++-
> >  1 file changed, 9 insertions(+), 1 deletion(-)
> > 
> > diff --git a/meta/classes/dpkg-raw.bbclass
> > b/meta/classes/dpkg-raw.bbclass index d662422..968e28d 100644
> > --- a/meta/classes/dpkg-raw.bbclass
> > +++ b/meta/classes/dpkg-raw.bbclass
> > @@ -8,13 +8,21 @@ MAINTAINER ?= "FIXME Unknown maintainer"
> >  
> >  D = "${WORKDIR}/image/"
> >  
> > +# make sure we start with a fresh ${D} every single time
> > +do_fresh_image() {
> > +	sudo rm -rf ${D}
> > +}
> > +
> > +do_fresh_image[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
> > +addtask fresh_image after do_unpack before do_install
> > +
> >  # Populate folder that will be picked up as package
> >  do_install() {
> >  	bbnote "Put your files for this package in ${D}"
> >  }
> >  
> >  do_install[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
> > -addtask install after do_unpack before do_prepare_build
> > +addtask install after do_fresh_image before do_prepare_build
> >  
> >  deb_package_prepare() {
> >  	sudo rm -rf ${D}/DEBIAN  
> 

[PATCH v2] meta/dpkg-raw: do_install into a fresh directory every time

[Henning Schild]

Permission problems revealed that we always packaged the sum of possibly
many rebuilds and maybe recipe versions. do_install should always start
working on an empty $D to ensure that the package only contains output
of the current build.

Reported-by: Jan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: Henning Schild <henning.schild@siemens.com>
---

Changes in v2:
 - use prefunc instead of a bitbake task
 
 meta/classes/dpkg-raw.bbclass | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/meta/classes/dpkg-raw.bbclass b/meta/classes/dpkg-raw.bbclass
index d662422..b2bea60 100644
--- a/meta/classes/dpkg-raw.bbclass
+++ b/meta/classes/dpkg-raw.bbclass
@@ -8,11 +8,18 @@ MAINTAINER ?= "FIXME Unknown maintainer"
 
 D = "${WORKDIR}/image/"
 
+# make sure we start with a fresh ${D} on a rebuild
+remove_old_image() {
+	sudo rm -rf ${D}
+	mkdir -p ${D}
+}
+
 # Populate folder that will be picked up as package
 do_install() {
 	bbnote "Put your files for this package in ${D}"
 }
 
+do_install[prefuncs] = 'remove_old_image'
 do_install[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
 addtask install after do_unpack before do_prepare_build
 
-- 
2.19.1

Re: [PATCH] meta/dpkg-raw: do_install into a fresh directory every time

[Henning Schild]

This is not needed anymore.

Henning

Am Wed, 14 Nov 2018 16:39:55 +0100
schrieb Henning Schild <henning.schild@siemens.com>:

> Permission problems revealed that we always packaged the sum of
> possibly many rebuilds and maybe recipe versions. do_install should
> always start working on an empty $D to ensure that the package only
> contains output of the current build.
> 
> Reported-by: Jan Kiszka <jan.kiszka@siemens.com>
> Signed-off-by: Henning Schild <henning.schild@siemens.com>
> ---
>  meta/classes/dpkg-raw.bbclass | 10 +++++++++-
>  1 file changed, 9 insertions(+), 1 deletion(-)
> 
> diff --git a/meta/classes/dpkg-raw.bbclass
> b/meta/classes/dpkg-raw.bbclass index d662422..968e28d 100644
> --- a/meta/classes/dpkg-raw.bbclass
> +++ b/meta/classes/dpkg-raw.bbclass
> @@ -8,13 +8,21 @@ MAINTAINER ?= "FIXME Unknown maintainer"
>  
>  D = "${WORKDIR}/image/"
>  
> +# make sure we start with a fresh ${D} every single time
> +do_fresh_image() {
> +	sudo rm -rf ${D}
> +}
> +
> +do_fresh_image[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
> +addtask fresh_image after do_unpack before do_install
> +
>  # Populate folder that will be picked up as package
>  do_install() {
>  	bbnote "Put your files for this package in ${D}"
>  }
>  
>  do_install[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
> -addtask install after do_unpack before do_prepare_build
> +addtask install after do_fresh_image before do_prepare_build
>  
>  deb_package_prepare() {
>  	sudo rm -rf ${D}/DEBIAN