Re: [PATCHv3] dpkg-base: derive "Package" and "Architecture" from .deb in cleanup

[Henning Schild <henning.schild@siemens.com>]

Am Tue, 2 Apr 2019 17:31:04 +0200
schrieb Baurzhan Ismagulov <ibr@radix50.net>:

> On Thu, Mar 28, 2019 at 06:50:56PM +0100, Henning Schild wrote:
> > > IMHO, it's unclear who is "you" in the commit message. I'd
> > > suggest to reword in a more neutral form, e.g.:  
> ...
> > Ok, you or Maxim can fix that on merge.  
> 
> Ok, thanks.
> 
> 
> > I hope it is clear who you is ;).  
> 
> Not for some other user reading the commit message later.
> 
> And if you actually address a specific person, I'd ask to avoid that
> in the future. We need the description of the problem and of the
> solution -- in this patch you got both well. Fingerpointing doesn't
> contribute to the project.

I think we are still talking about writing style and wording of the
commit message. No finger pointing was ever intended or even remotely
thought about. I just said "you" to the anonymous reader of the commit
message.

> > > How can one reproduce the problem? I'd like to understand the
> > > cause because at the first glance I'd think an _all.deb package
> > > should land in the pool only once.  
> > 
> > That is written in the commit message.  
> 
> I wanted to ensure I got the context correctly, e.g. 1. whether
> isar-apt or base-apt (assuming the former) and 2. local.conf or
> multiconfig (assuming the former).

We are talking about isar-apt. I did not look into base-apt yet, that
was discussed in another thread where i also suggested to inspect the
packets with dpkg-deb instead of doing file name guessing.
one config or multiconfig does not matter, "all" is no architecture an
can not be removed

> 
> > A deployment of "all" will not deploy one .deb of arch "all" but it
> > will deploy n .debs of all the archs enabled in the repo.
> > So on deploy of "all" from arm into a "arm, arm64, x86" repo you
> > will get three copies. The next deploy from x86 will write all
> > three again, same for arm64. But they are all the same and just get
> > written 3 times.  
> 
> Thanks for explaining, this was clear enough from the commit message.
> It's the 3 copies that I'd expected to be one.
> 
> 
> > The easiest way to just get to the "broken" clean is to build an
> > "all" package. Deploy it and repo_clean it, you should still see it
> > in the other archs ...  
> 
> Thanks, I'll try.

Did you succeed?

> 
> > That also means we must never use the repo while another arch
> > deploys into it because we get sharing over "all".  
> 
> My understanding was that we lock reprepro to avoid this. Or am I
> missing anything?

We lock it for writes and assume that reads will happen after the repo
is complete. Because an image depends on all recipes to deploy.
In a multiconfig setup a writer (recipe deploy/clean) can remove
packages that a reader (image bootstrap) needs.
i.e. all arm64 recipes have deployed their debs and arm64 image
bootstrap starts, during that bootstrap an x86 recipe deploy removes
an "all" deb that the bootstrap needs ... deb missing ... press the
jenkins build button again better luck next time

Henning

> > That is a severe problem if
> > you have "all" packages and use multiconf. My suggestion would be to
> > avoid both ;).  
> 
> "all" is in Debian and we have to handle it. We should also support
> local.conf, although I haven't tried it for a long time.
> 
> 
> > However that issue should be documented or fixed for people who
> > care about multiconfig ;).  
> 
> I'd look first whether having the 3 copies is correct.
> 
> 
> > I agree and i am the one constantly asking for tests. All the
> > rebuilding patches that got merged so far are not covered by tests.
> > I guess some tests that trigger partial rebuilds would be needed.
> > But that is probably out of scope of this patch.
> > 
> > Yes we need tests for new features. Tests for corner-case
> > bugfixes ... would be handy but holding back bug-fixes for it?  
> 
> That was a general wish, also for the mentioned rebuilding patches.
> No, missing test case isn't a blocker for me. It would just speed up
> testing -- even your temporary manual test case (beyond providing the
> longer-term quality, that is).
> 
> 
> With kind regards,
> Baurzhan.
>