[PATCHv3] dpkg-base: derive "Package" and "Architecture" from .deb in cleanup

[PATCHv3] dpkg-base: derive "Package" and "Architecture" from .deb in cleanup

[Henning Schild]

From: Henning Schild <henning.schild@siemens.com>

Instead of using "sed" pattern magic ask the .debs their name and
architecture and remove exactly that. Especially the hardcoded
Architecture caused problems for packages of architecture "all". If
your recipe created such a package it could never be cleaned because it
was added to all the archs of the repo, and later cleaned only from one.
Now if that package changes because you are still working on the recipe,
adding the package again would cause checksum mismatches for all but the
current architecture.
Remove the package from all archs when arch is "all".

Reported-by: Vladislav Jasek <vladislav.jasek@siemens.com>
Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
 meta/classes/dpkg-base.bbclass | 21 ++++++++++++++-------
 1 file changed, 14 insertions(+), 7 deletions(-)

diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass
index 742b8ad..8fb7805 100644
--- a/meta/classes/dpkg-base.bbclass
+++ b/meta/classes/dpkg-base.bbclass
@@ -98,13 +98,20 @@ do_build() {
 CLEANFUNCS += "repo_clean"
 
 repo_clean() {
-    PACKAGES=$(cd ${S}/..; ls *.deb | sed 's/\([^_]*\).*/\1/')
-    if [ -n "${PACKAGES}" ]; then
-        reprepro -b ${REPO_ISAR_DIR}/${DISTRO} \
-                 --dbdir ${REPO_ISAR_DB_DIR}/${DISTRO} \
-                 -C main -A ${DISTRO_ARCH} \
-                 remove ${DEBDISTRONAME} \
-                 ${PACKAGES}
+    DEBS=$( ls ${S}/../*.deb )
+    if [ -n "${DEBS}" ]; then
+        for d in ${DEBS}; do
+            p=$( dpkg-deb --show --showformat '${Package}' ${d} )
+            a=$( dpkg-deb --show --showformat '${Architecture}' ${d} )
+            # removing "all" means no arch
+            aarg="-A ${a}"
+            [ "${a}" = "all" ] && aarg=""
+            reprepro -b ${REPO_ISAR_DIR}/${DISTRO} \
+                     --dbdir ${REPO_ISAR_DB_DIR}/${DISTRO} \
+                     -C main ${aarg} \
+                     remove ${DEBDISTRONAME} \
+                     ${p}
+        done
     fi
 }
 
-- 
2.19.2

Re: [PATCHv3] dpkg-base: derive "Package" and "Architecture" from .deb in cleanup

[Henning Schild]

Diff to v2:
 - if arch is "all" do not specify "-A all" but no "-A" at all, which
   is what we really want
 - actually tested that with a modified recipe that fetches an all.deb
   from the internet

Henning 

Am Thu, 21 Feb 2019 16:56:34 +0100
schrieb Henning Schild <henning.schild@siemens.com>:

> From: Henning Schild <henning.schild@siemens.com>
> 
> Instead of using "sed" pattern magic ask the .debs their name and
> architecture and remove exactly that. Especially the hardcoded
> Architecture caused problems for packages of architecture "all". If
> your recipe created such a package it could never be cleaned because
> it was added to all the archs of the repo, and later cleaned only
> from one. Now if that package changes because you are still working
> on the recipe, adding the package again would cause checksum
> mismatches for all but the current architecture.
> Remove the package from all archs when arch is "all".
> 
> Reported-by: Vladislav Jasek <vladislav.jasek@siemens.com>
> Signed-off-by: Henning Schild <henning.schild@siemens.com>
> ---
>  meta/classes/dpkg-base.bbclass | 21 ++++++++++++++-------
>  1 file changed, 14 insertions(+), 7 deletions(-)
> 
> diff --git a/meta/classes/dpkg-base.bbclass
> b/meta/classes/dpkg-base.bbclass index 742b8ad..8fb7805 100644
> --- a/meta/classes/dpkg-base.bbclass
> +++ b/meta/classes/dpkg-base.bbclass
> @@ -98,13 +98,20 @@ do_build() {
>  CLEANFUNCS += "repo_clean"
>  
>  repo_clean() {
> -    PACKAGES=$(cd ${S}/..; ls *.deb | sed 's/\([^_]*\).*/\1/')
> -    if [ -n "${PACKAGES}" ]; then
> -        reprepro -b ${REPO_ISAR_DIR}/${DISTRO} \
> -                 --dbdir ${REPO_ISAR_DB_DIR}/${DISTRO} \
> -                 -C main -A ${DISTRO_ARCH} \
> -                 remove ${DEBDISTRONAME} \
> -                 ${PACKAGES}
> +    DEBS=$( ls ${S}/../*.deb )
> +    if [ -n "${DEBS}" ]; then
> +        for d in ${DEBS}; do
> +            p=$( dpkg-deb --show --showformat '${Package}' ${d} )
> +            a=$( dpkg-deb --show --showformat '${Architecture}'
> ${d} )
> +            # removing "all" means no arch
> +            aarg="-A ${a}"
> +            [ "${a}" = "all" ] && aarg=""
> +            reprepro -b ${REPO_ISAR_DIR}/${DISTRO} \
> +                     --dbdir ${REPO_ISAR_DB_DIR}/${DISTRO} \
> +                     -C main ${aarg} \
> +                     remove ${DEBDISTRONAME} \
> +                     ${p}
> +        done
>      fi
>  }
>  

Re: [PATCHv3] dpkg-base: derive "Package" and "Architecture" from .deb in cleanup

[Henning Schild]

Ping

Am Thu, 21 Feb 2019 16:56:34 +0100
schrieb Henning Schild <henning.schild@siemens.com>:

> From: Henning Schild <henning.schild@siemens.com>
> 
> Instead of using "sed" pattern magic ask the .debs their name and
> architecture and remove exactly that. Especially the hardcoded
> Architecture caused problems for packages of architecture "all". If
> your recipe created such a package it could never be cleaned because
> it was added to all the archs of the repo, and later cleaned only
> from one. Now if that package changes because you are still working
> on the recipe, adding the package again would cause checksum
> mismatches for all but the current architecture.
> Remove the package from all archs when arch is "all".
> 
> Reported-by: Vladislav Jasek <vladislav.jasek@siemens.com>
> Signed-off-by: Henning Schild <henning.schild@siemens.com>
> ---
>  meta/classes/dpkg-base.bbclass | 21 ++++++++++++++-------
>  1 file changed, 14 insertions(+), 7 deletions(-)
> 
> diff --git a/meta/classes/dpkg-base.bbclass
> b/meta/classes/dpkg-base.bbclass index 742b8ad..8fb7805 100644
> --- a/meta/classes/dpkg-base.bbclass
> +++ b/meta/classes/dpkg-base.bbclass
> @@ -98,13 +98,20 @@ do_build() {
>  CLEANFUNCS += "repo_clean"
>  
>  repo_clean() {
> -    PACKAGES=$(cd ${S}/..; ls *.deb | sed 's/\([^_]*\).*/\1/')
> -    if [ -n "${PACKAGES}" ]; then
> -        reprepro -b ${REPO_ISAR_DIR}/${DISTRO} \
> -                 --dbdir ${REPO_ISAR_DB_DIR}/${DISTRO} \
> -                 -C main -A ${DISTRO_ARCH} \
> -                 remove ${DEBDISTRONAME} \
> -                 ${PACKAGES}
> +    DEBS=$( ls ${S}/../*.deb )
> +    if [ -n "${DEBS}" ]; then
> +        for d in ${DEBS}; do
> +            p=$( dpkg-deb --show --showformat '${Package}' ${d} )
> +            a=$( dpkg-deb --show --showformat '${Architecture}'
> ${d} )
> +            # removing "all" means no arch
> +            aarg="-A ${a}"
> +            [ "${a}" = "all" ] && aarg=""
> +            reprepro -b ${REPO_ISAR_DIR}/${DISTRO} \
> +                     --dbdir ${REPO_ISAR_DB_DIR}/${DISTRO} \
> +                     -C main ${aarg} \
> +                     remove ${DEBDISTRONAME} \
> +                     ${p}
> +        done
>      fi
>  }
>  

Re: [PATCHv3] dpkg-base: derive "Package" and "Architecture" from .deb in cleanup

[Baurzhan Ismagulov]

Hello Henning,

On Thu, Feb 21, 2019 at 04:56:34PM +0100, Henning Schild wrote:
> If
> your recipe created such a package it could never be cleaned because it
> was added to all the archs of the repo, and later cleaned only from one.
> Now if that package changes because you are still working on the recipe,
> adding the package again would cause checksum mismatches for all but the
> current architecture.

IMHO, it's unclear who is "you" in the commit message. I'd suggest to reword in
a more neutral form, e.g.:

-your recipe created such a package it could never be cleaned because it
+a recipe created such a package it could never be cleaned because it
...
-Now if that package changes because you are still working on the recipe,
+Now if that package changes because one is still working on the recipe,


> +            p=$( dpkg-deb --show --showformat '${Package}' ${d} )
> +            a=$( dpkg-deb --show --showformat '${Architecture}' ${d} )

Thanks, this is better than sed.


> +            reprepro -b ${REPO_ISAR_DIR}/${DISTRO} \
> +                     --dbdir ${REPO_ISAR_DB_DIR}/${DISTRO} \
> +                     -C main ${aarg} \
> +                     remove ${DEBDISTRONAME} \
> +                     ${p}

How can one reproduce the problem? I'd like to understand the cause because at
the first glance I'd think an _all.deb package should land in the pool only
once.

In general, subtle stuff like this should ideally be covered with a test case.


With kind regards,
Baurzhan.

Re: [PATCHv3] dpkg-base: derive "Package" and "Architecture" from .deb in cleanup

[Henning Schild]

Am Thu, 28 Mar 2019 12:03:28 +0100
schrieb Baurzhan Ismagulov <ibr@radix50.net>:

> Hello Henning,
> 
> On Thu, Feb 21, 2019 at 04:56:34PM +0100, Henning Schild wrote:
> > If
> > your recipe created such a package it could never be cleaned
> > because it was added to all the archs of the repo, and later
> > cleaned only from one. Now if that package changes because you are
> > still working on the recipe, adding the package again would cause
> > checksum mismatches for all but the current architecture.  
> 
> IMHO, it's unclear who is "you" in the commit message. I'd suggest to
> reword in a more neutral form, e.g.:
> 
> -your recipe created such a package it could never be cleaned because
> it +a recipe created such a package it could never be cleaned because
> it ...
> -Now if that package changes because you are still working on the
> recipe, +Now if that package changes because one is still working on
> the recipe,
> 

Ok, you or Maxim can fix that on merge. I hope it is clear who you
is ;). I will be out of office for two weeks, if you dont the patch
will have to keep waiting.

> > +            p=$( dpkg-deb --show --showformat '${Package}' ${d} )
> > +            a=$( dpkg-deb --show --showformat '${Architecture}'
> > ${d} )  
> 
> Thanks, this is better than sed.
> 
> 
> > +            reprepro -b ${REPO_ISAR_DIR}/${DISTRO} \
> > +                     --dbdir ${REPO_ISAR_DB_DIR}/${DISTRO} \
> > +                     -C main ${aarg} \
> > +                     remove ${DEBDISTRONAME} \
> > +                     ${p}  
> 
> How can one reproduce the problem? I'd like to understand the cause
> because at the first glance I'd think an _all.deb package should land
> in the pool only once.

That is written in the commit message.

A deployment of "all" will not deploy one .deb of arch "all" but it
will deploy n .debs of all the archs enabled in the repo.
So on deploy of "all" from arm into a "arm, arm64, x86" repo you will
get three copies. The next deploy from x86 will write all three again,
same for arm64. But they are all the same and just get written 3 times.

But a clean of "-A all" is a no-op since that arch does not exist. That
clean will not actually clean up the deployment correctly. And
subsequent changes deploys will run into problems.

The easiest way to just get to the "broken" clean is to build an "all"
package. Deploy it and repo_clean it, you should still see it in the
other archs ...
If you still need to see more change anything in the recipe keeping all
of ${P} the same, the deploy should fail.

That also means we must never use the repo while another arch deploys
into it because we get sharing over "all". That is a severe problem if
you have "all" packages and use multiconf. My suggestion would be to
avoid both ;). However that issue should be documented or fixed for
people who care about multiconfig ;).

> In general, subtle stuff like this should ideally be covered with a
> test case.

I agree and i am the one constantly asking for tests. All the
rebuilding patches that got merged so far are not covered by tests. I
guess some tests that trigger partial rebuilds would be needed. But
that is probably out of scope of this patch.

Yes we need tests for new features. Tests for corner-case bugfixes ...
would be handy but holding back bug-fixes for it?

Henning

> 
> With kind regards,
> Baurzhan.
> 

Re: [PATCHv3] dpkg-base: derive "Package" and "Architecture" from .deb in cleanup

[Baurzhan Ismagulov]

On Thu, Mar 28, 2019 at 06:50:56PM +0100, Henning Schild wrote:
> > IMHO, it's unclear who is "you" in the commit message. I'd suggest to
> > reword in a more neutral form, e.g.:
...
> Ok, you or Maxim can fix that on merge.

Ok, thanks.


> I hope it is clear who you is ;).

Not for some other user reading the commit message later.

And if you actually address a specific person, I'd ask to avoid that in the
future. We need the description of the problem and of the solution -- in this
patch you got both well. Fingerpointing doesn't contribute to the project.


> > How can one reproduce the problem? I'd like to understand the cause
> > because at the first glance I'd think an _all.deb package should land
> > in the pool only once.
> 
> That is written in the commit message.

I wanted to ensure I got the context correctly, e.g. 1. whether isar-apt or
base-apt (assuming the former) and 2. local.conf or multiconfig (assuming the
former).


> A deployment of "all" will not deploy one .deb of arch "all" but it
> will deploy n .debs of all the archs enabled in the repo.
> So on deploy of "all" from arm into a "arm, arm64, x86" repo you will
> get three copies. The next deploy from x86 will write all three again,
> same for arm64. But they are all the same and just get written 3 times.

Thanks for explaining, this was clear enough from the commit message. It's the
3 copies that I'd expected to be one.


> The easiest way to just get to the "broken" clean is to build an "all"
> package. Deploy it and repo_clean it, you should still see it in the
> other archs ...

Thanks, I'll try.


> That also means we must never use the repo while another arch deploys
> into it because we get sharing over "all".

My understanding was that we lock reprepro to avoid this. Or am I missing
anything?


> That is a severe problem if
> you have "all" packages and use multiconf. My suggestion would be to
> avoid both ;).

"all" is in Debian and we have to handle it. We should also support local.conf,
although I haven't tried it for a long time.


> However that issue should be documented or fixed for people who care about
> multiconfig ;).

I'd look first whether having the 3 copies is correct.


> I agree and i am the one constantly asking for tests. All the
> rebuilding patches that got merged so far are not covered by tests. I
> guess some tests that trigger partial rebuilds would be needed. But
> that is probably out of scope of this patch.
> 
> Yes we need tests for new features. Tests for corner-case bugfixes ...
> would be handy but holding back bug-fixes for it?

That was a general wish, also for the mentioned rebuilding patches. No, missing
test case isn't a blocker for me. It would just speed up testing -- even your
temporary manual test case (beyond providing the longer-term quality, that is).


With kind regards,
Baurzhan.

Re: [PATCHv3] dpkg-base: derive "Package" and "Architecture" from .deb in cleanup

[Henning Schild]

Am Tue, 2 Apr 2019 17:31:04 +0200
schrieb Baurzhan Ismagulov <ibr@radix50.net>:

> On Thu, Mar 28, 2019 at 06:50:56PM +0100, Henning Schild wrote:
> > > IMHO, it's unclear who is "you" in the commit message. I'd
> > > suggest to reword in a more neutral form, e.g.:  
> ...
> > Ok, you or Maxim can fix that on merge.  
> 
> Ok, thanks.
> 
> 
> > I hope it is clear who you is ;).  
> 
> Not for some other user reading the commit message later.
> 
> And if you actually address a specific person, I'd ask to avoid that
> in the future. We need the description of the problem and of the
> solution -- in this patch you got both well. Fingerpointing doesn't
> contribute to the project.

I think we are still talking about writing style and wording of the
commit message. No finger pointing was ever intended or even remotely
thought about. I just said "you" to the anonymous reader of the commit
message.

> > > How can one reproduce the problem? I'd like to understand the
> > > cause because at the first glance I'd think an _all.deb package
> > > should land in the pool only once.  
> > 
> > That is written in the commit message.  
> 
> I wanted to ensure I got the context correctly, e.g. 1. whether
> isar-apt or base-apt (assuming the former) and 2. local.conf or
> multiconfig (assuming the former).

We are talking about isar-apt. I did not look into base-apt yet, that
was discussed in another thread where i also suggested to inspect the
packets with dpkg-deb instead of doing file name guessing.
one config or multiconfig does not matter, "all" is no architecture an
can not be removed

> 
> > A deployment of "all" will not deploy one .deb of arch "all" but it
> > will deploy n .debs of all the archs enabled in the repo.
> > So on deploy of "all" from arm into a "arm, arm64, x86" repo you
> > will get three copies. The next deploy from x86 will write all
> > three again, same for arm64. But they are all the same and just get
> > written 3 times.  
> 
> Thanks for explaining, this was clear enough from the commit message.
> It's the 3 copies that I'd expected to be one.
> 
> 
> > The easiest way to just get to the "broken" clean is to build an
> > "all" package. Deploy it and repo_clean it, you should still see it
> > in the other archs ...  
> 
> Thanks, I'll try.

Did you succeed?

> 
> > That also means we must never use the repo while another arch
> > deploys into it because we get sharing over "all".  
> 
> My understanding was that we lock reprepro to avoid this. Or am I
> missing anything?

We lock it for writes and assume that reads will happen after the repo
is complete. Because an image depends on all recipes to deploy.
In a multiconfig setup a writer (recipe deploy/clean) can remove
packages that a reader (image bootstrap) needs.
i.e. all arm64 recipes have deployed their debs and arm64 image
bootstrap starts, during that bootstrap an x86 recipe deploy removes
an "all" deb that the bootstrap needs ... deb missing ... press the
jenkins build button again better luck next time

Henning

> > That is a severe problem if
> > you have "all" packages and use multiconf. My suggestion would be to
> > avoid both ;).  
> 
> "all" is in Debian and we have to handle it. We should also support
> local.conf, although I haven't tried it for a long time.
> 
> 
> > However that issue should be documented or fixed for people who
> > care about multiconfig ;).  
> 
> I'd look first whether having the 3 copies is correct.
> 
> 
> > I agree and i am the one constantly asking for tests. All the
> > rebuilding patches that got merged so far are not covered by tests.
> > I guess some tests that trigger partial rebuilds would be needed.
> > But that is probably out of scope of this patch.
> > 
> > Yes we need tests for new features. Tests for corner-case
> > bugfixes ... would be handy but holding back bug-fixes for it?  
> 
> That was a general wish, also for the mentioned rebuilding patches.
> No, missing test case isn't a blocker for me. It would just speed up
> testing -- even your temporary manual test case (beyond providing the
> longer-term quality, that is).
> 
> 
> With kind regards,
> Baurzhan.
> 

Re: [PATCHv3] dpkg-base: derive "Package" and "Architecture" from .deb in cleanup

[Baurzhan Ismagulov]

On Fri, Apr 12, 2019 at 02:10:01PM +0200, Henning Schild wrote:
> I think we are still talking about writing style and wording of the
> commit message. No finger pointing was ever intended or even remotely
> thought about. I just said "you" to the anonymous reader of the commit
> message.

Ok, thanks for clarification. Yes, I'm talking about the wording that doesn't
leave room for misinterpretation.


> Did you succeed?

I've encountered qemu issues and haven't looked further, sorry. I need to solve
that first.


> We lock it for writes and assume that reads will happen after the repo
> is complete. Because an image depends on all recipes to deploy.
> In a multiconfig setup a writer (recipe deploy/clean) can remove
> packages that a reader (image bootstrap) needs.
> i.e. all arm64 recipes have deployed their debs and arm64 image
> bootstrap starts, during that bootstrap an x86 recipe deploy removes
> an "all" deb that the bootstrap needs ... deb missing ... press the
> jenkins build button again better luck next time

Thanks, now I get it. Readers must be also serialized against writers.


With kind regards,
Baurzhan.

Re: [PATCHv3] dpkg-base: derive "Package" and "Architecture" from .deb in cleanup

[Baurzhan Ismagulov]

On Tue, Apr 16, 2019 at 08:20:09PM +0200, Baurzhan Ismagulov wrote:
> > Did you succeed?
> 
> I've encountered qemu issues and haven't looked further, sorry. I need to solve
> that first.

Seems I've found a workaround, will try to reproduce.

With kind regards,
Baurzhan.

Re: [PATCHv3] dpkg-base: derive "Package" and "Architecture" from .deb in cleanup

[Baurzhan Ismagulov]

[-- Attachment #1: Type: text/plain, Size: 2940 bytes --]

On Thu, Mar 28, 2019 at 06:50:56PM +0100, Henning Schild wrote:
> A deployment of "all" will not deploy one .deb of arch "all" but it
> will deploy n .debs of all the archs enabled in the repo.
> So on deploy of "all" from arm into a "arm, arm64, x86" repo you will
> get three copies. The next deploy from x86 will write all three again,
> same for arm64. But they are all the same and just get written 3 times.

Ok, I was able to reproduce the problem using the attached patch.

In short, your patch looks good for me. We don't have a further problem of
multiple identical files in isar-apt. Maxim will run the usual tests with your
patch.

Documenting the findings (please comment if anything is wrong):

* I had to bitbake multiconfig:qemuamd64-stretch:isar-image-base, then bitbake
  multiconfig:qemuamd64-stretch:isar-image-base (thus the question about
  multiconfig -- or how can one trigger repo_clean manually?). The second one
  fails with various problems:

  ERROR: 'tmp/work/debian-stretch-amd64/abicheck-1.2-r0/abicheck-1.2/../abicheck_1.2-99+isar_all.deb'
  cannot be included as 'pool/main/a/abicheck/abicheck_1.2-99+isar_all.deb'.
  Already existing files can only be included again, if they are the same, but:
  md5 expected: 21747ce983869443d240a136c4b4e8a9, got: 742593776ebae9c55f7f5a7eef0758d2

  This seems to be the cleanup problem you are addressing.

* After building, the files are:

  tmp/work/debian-stretch-amd64/abicheck-1.2-r0/abicheck_1.2-99+isar_all.deb
  tmp/work/debian-stretch-i386/abicheck-1.2-r0/abicheck_1.2-99+isar_all.deb
  tmp/deploy/isar-apt/apt/debian-stretch/pool/main/a/abicheck/abicheck_1.2-99+isar_all.deb

  The files are built n times in tmp/work and added to isar-apt n times. The
  package is present in isar-apt only once (and not n times, as I interpreted
  from the problem description, so we don't have a problem of multiple
  abicheck_1.2-99+isar_all.deb files in isar-apt, only in
  binary-{amd64,i386,etc.}/Packages.*). reprepro fails adding the package with
  a different checksum.

* Isar removes packages from isar-apt per-arch, to preserve previously built
  artifacts. But _all.debs must be removed from all Packages indexes. Your
  patch adds special handling for _all.debs, removing them from all
  binary-{amd64,i386,etc.}/Packages.*.

* Also, example-raw, hello-isar, and enable-fsck fail with the following:

  DEBUG: Executing shell function do_install_builddeps
  ...
  dh_compress
  dh_compress: Cannot resolve
  (unreachable)/hello-isar-0.3-a18c14c/equivs.brSwev/debian/hello-isar-build-deps:
  No such file or directory
  debian/rules:20: recipe for target 'binary-indep' failed
  make: *** [binary-indep] Error 2
  Error in the build process: exit status 2
  dpkg: error: cannot access archive 'hello-isar-build-deps_0.3_amd64.deb': No
  such file or directory

  I haven't looked into that, have you encountered anything like that?

With kind regards,
Baurzhan.

[-- Attachment #2: 0001-Add-abicheck.patch --]
[-- Type: text/x-diff, Size: 1683 bytes --]

>From 294039a7051a7649ad347d114e75d16f921dcf67 Mon Sep 17 00:00:00 2001
From: Baurzhan Ismagulov <ibr@ilbers.de>
Date: Wed, 17 Apr 2019 11:44:06 +0200
Subject: [PATCH] Add abicheck

---
 meta-isar/conf/local.conf.sample               |  5 ++++-
 meta-isar/recipes-app/abicheck/abicheck_1.2.bb | 22 ++++++++++++++++++++++
 2 files changed, 26 insertions(+), 1 deletion(-)
 create mode 100644 meta-isar/recipes-app/abicheck/abicheck_1.2.bb

diff --git a/meta-isar/conf/local.conf.sample b/meta-isar/conf/local.conf.sample
index 86e2384..5b1d638 100644
--- a/meta-isar/conf/local.conf.sample
+++ b/meta-isar/conf/local.conf.sample
@@ -159,7 +159,10 @@ CONF_VERSION = "1"
 
 #
 # The default list of extra packages to be installed.
-IMAGE_INSTALL = "hello-isar example-raw example-module-${KERNEL_NAME} enable-fsck"
+IMAGE_INSTALL = "hello-isar example-raw example-module-${KERNEL_NAME} \
+    enable-fsck \
+    abicheck \
+"
 
 #
 # Enable cross-compilation support
diff --git a/meta-isar/recipes-app/abicheck/abicheck_1.2.bb b/meta-isar/recipes-app/abicheck/abicheck_1.2.bb
new file mode 100644
index 0000000..af08a0c
--- /dev/null
+++ b/meta-isar/recipes-app/abicheck/abicheck_1.2.bb
@@ -0,0 +1,22 @@
+# This software is a part of ISAR.
+# Copyright (C) 2019 ilbers GmbH
+#
+# SPDX-License-Identifier: MIT
+
+inherit dpkg
+
+# this will fetch and unpack the sources from upstream debian
+SRC_URI = "apt://${PN}=1.2-5"
+
+MAINTAINER = "isar-users <isar-users@googlegroups.com>"
+CHANGELOG_V = "1.2-99+isar"
+
+do_prepare_build() {
+	deb_add_changelog
+}
+
+dpkg_runbuild_prepend() {
+	if [ ${ISAR_CROSS_COMPILE} -eq 1 ]; then
+		export DEB_BUILD_OPTIONS="nocheck"
+	fi
+}
-- 
2.11.0

Re: [PATCHv3] dpkg-base: derive "Package" and "Architecture" from .deb in cleanup

[Henning Schild]

Ping

Am Thu, 21 Feb 2019 16:56:34 +0100
schrieb Henning Schild <henning.schild@siemens.com>:

> From: Henning Schild <henning.schild@siemens.com>
> 
> Instead of using "sed" pattern magic ask the .debs their name and
> architecture and remove exactly that. Especially the hardcoded
> Architecture caused problems for packages of architecture "all". If
> your recipe created such a package it could never be cleaned because
> it was added to all the archs of the repo, and later cleaned only
> from one. Now if that package changes because you are still working
> on the recipe, adding the package again would cause checksum
> mismatches for all but the current architecture.
> Remove the package from all archs when arch is "all".
> 
> Reported-by: Vladislav Jasek <vladislav.jasek@siemens.com>
> Signed-off-by: Henning Schild <henning.schild@siemens.com>
> ---
>  meta/classes/dpkg-base.bbclass | 21 ++++++++++++++-------
>  1 file changed, 14 insertions(+), 7 deletions(-)
> 
> diff --git a/meta/classes/dpkg-base.bbclass
> b/meta/classes/dpkg-base.bbclass index 742b8ad..8fb7805 100644
> --- a/meta/classes/dpkg-base.bbclass
> +++ b/meta/classes/dpkg-base.bbclass
> @@ -98,13 +98,20 @@ do_build() {
>  CLEANFUNCS += "repo_clean"
>  
>  repo_clean() {
> -    PACKAGES=$(cd ${S}/..; ls *.deb | sed 's/\([^_]*\).*/\1/')
> -    if [ -n "${PACKAGES}" ]; then
> -        reprepro -b ${REPO_ISAR_DIR}/${DISTRO} \
> -                 --dbdir ${REPO_ISAR_DB_DIR}/${DISTRO} \
> -                 -C main -A ${DISTRO_ARCH} \
> -                 remove ${DEBDISTRONAME} \
> -                 ${PACKAGES}
> +    DEBS=$( ls ${S}/../*.deb )
> +    if [ -n "${DEBS}" ]; then
> +        for d in ${DEBS}; do
> +            p=$( dpkg-deb --show --showformat '${Package}' ${d} )
> +            a=$( dpkg-deb --show --showformat '${Architecture}'
> ${d} )
> +            # removing "all" means no arch
> +            aarg="-A ${a}"
> +            [ "${a}" = "all" ] && aarg=""
> +            reprepro -b ${REPO_ISAR_DIR}/${DISTRO} \
> +                     --dbdir ${REPO_ISAR_DB_DIR}/${DISTRO} \
> +                     -C main ${aarg} \
> +                     remove ${DEBDISTRONAME} \
> +                     ${p}
> +        done
>      fi
>  }
>  

Re: [PATCHv3] dpkg-base: derive "Package" and "Architecture" from .deb in cleanup

[Maxim Yu. Osipov]

On 2/21/19 4:56 PM, Henning Schild wrote:
> From: Henning Schild <henning.schild@siemens.com>
> 
> Instead of using "sed" pattern magic ask the .debs their name and
> architecture and remove exactly that. Especially the hardcoded
> Architecture caused problems for packages of architecture "all". If
> your recipe created such a package it could never be cleaned because it
> was added to all the archs of the repo, and later cleaned only from one.
> Now if that package changes because you are still working on the recipe,
> adding the package again would cause checksum mismatches for all but the
> current architecture.
> Remove the package from all archs when arch is "all".

Applied to the 'next'.


Regards,
Maxim.

> Reported-by: Vladislav Jasek <vladislav.jasek@siemens.com>
> Signed-off-by: Henning Schild <henning.schild@siemens.com>
> ---
>   meta/classes/dpkg-base.bbclass | 21 ++++++++++++++-------
>   1 file changed, 14 insertions(+), 7 deletions(-)
> 
> diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass
> index 742b8ad..8fb7805 100644
> --- a/meta/classes/dpkg-base.bbclass
> +++ b/meta/classes/dpkg-base.bbclass
> @@ -98,13 +98,20 @@ do_build() {
>   CLEANFUNCS += "repo_clean"
>   
>   repo_clean() {
> -    PACKAGES=$(cd ${S}/..; ls *.deb | sed 's/\([^_]*\).*/\1/')
> -    if [ -n "${PACKAGES}" ]; then
> -        reprepro -b ${REPO_ISAR_DIR}/${DISTRO} \
> -                 --dbdir ${REPO_ISAR_DB_DIR}/${DISTRO} \
> -                 -C main -A ${DISTRO_ARCH} \
> -                 remove ${DEBDISTRONAME} \
> -                 ${PACKAGES}
> +    DEBS=$( ls ${S}/../*.deb )
> +    if [ -n "${DEBS}" ]; then
> +        for d in ${DEBS}; do
> +            p=$( dpkg-deb --show --showformat '${Package}' ${d} )
> +            a=$( dpkg-deb --show --showformat '${Architecture}' ${d} )
> +            # removing "all" means no arch
> +            aarg="-A ${a}"
> +            [ "${a}" = "all" ] && aarg=""
> +            reprepro -b ${REPO_ISAR_DIR}/${DISTRO} \
> +                     --dbdir ${REPO_ISAR_DB_DIR}/${DISTRO} \
> +                     -C main ${aarg} \
> +                     remove ${DEBDISTRONAME} \
> +                     ${p}
> +        done
>       fi
>   }
>   
> 


-- 
Maxim Osipov
ilbers GmbH
Maria-Merian-Str. 8
85521 Ottobrunn
Germany
+49 (151) 6517 6917
mosipov@ilbers.de
http://ilbers.de/
Commercial register Munich, HRB 214197
General Manager: Baurzhan Ismagulov