RE: [PATCH 0/3] Added support for apt caching

["'Heinisch, Alexander' via isar-users" <isar-users@googlegroups.com>]

> On Thu, 2024-10-31 at 15:40 +0000, Heinisch, Alexander (FT RPD CED SES-
> AT) wrote:
> > > Hi, this series is much needed to work with the still unreliable
> > > snapshot mirrors.
> > >
> > > @Alexander: Do you plan to send a v2?
> > >
> > > At the same time I'm working on adding internal apt-cacher-ng
> > > support to kas to let the build pass the initial bootstrapping.
> > >
> > > Best regards,
> > > Felix
> >
> > Hi Felix
> >
> > Thank you for coming back.
> >
> > Even when using apt-cacher-ng index files oftentimes got updated from
> > snapshot.debian.org which caused problems when our company was on a
> > blacklist for some time again.
>
> I know, but this is partially also due to bugs in the apt-cacher-ng implementation. Today it completely broke after an upstream change on snapshot.d.o requiring a backport of the fix in [1]. I just sent an email to the snapshot ML requesting the backport.

Yes, apt-cacher-ng by far is not the most stable software!

>
> [1] https://bugs-devel.debian.org/cgi-bin/bugreport.cgi?bug=1074404
>
> >
> > Unfortunately, I didn't find the time to analyze why that was the
> > case.
> > I did a tcpdump during one of our builds, but didn't analyze it for 2
> > weeks or so :-(
> >
> > But I suspect either apt client sends a reload request or the expiry
> > date returned from upstream is to limited.
>
> It could also simply due to incorrect parsing of the expiry dates in apt-cacher-ng. Recently there were a lot of fixes regarding time parsing. Tricky to debug, though...
>
> > While this could be relevant when fetching packages from "main"
> > mirrors,
> > it should not have much impact on snapshot mirrors.
> >
> > To mitigate that issue, since then we switched to squid as a proxy for
> > snapshot.debian.org Squid has an offline mode, which says, no matter
> > what happens, cach entries once seen are never updated upstream. As
> > stated above, while this could have drastic impacts when using main
> > mirrors, it shouldn't cause issues on snapshots, by definition.
> >
> > Thus, I dropped apt-cacher-ng in our project in favour of squid.
> > I also prepared documentation for such, but during preparing the
> > patch, I was not sure if that is worth a separate doc/ file or if we
> > should merge that with doc/offline.md. I was struggling with that
> > decision since it does not really solve an offline case, as it only
> > caches packages already seen once, and further, only solves the
> > offline case for apt and not for other sources like git, ...
>
> Actually I'm more interested in having stable builds against snapshot.d.o, not so much in 100% offline builds. The situation upstream also got a bit better by rate-limiting on HTTP basis instead of TCP basis, so clients (including apt-cacher-ng and squid) should be able to correctly backoff. But I also did not check if the rate- limiting is implemented correctly, so that the client knows when to retry...

Stable builds is prio 1 for us.

But, also when using squid as a proxy ("non-offline mode") we experienced some troubles
with snapshot.d.o when downloading the index file [1]
Even though we were already caching most of the data, snapshot.debian.org denied
downloading that file, most probably because whole company network was blacklisted.

Thus, breaking our build, again.
That was the reason why we decided to go with squid's "offline-mode".

[1] http://snapshot.debian.org/archive/debian/20240904T000000Z/dists/bookworm/InRelease

>
> Anyways, we have a dilemma here: We need a stable baseline to build against (both due to product requirements, as well as for the SState cache). But currently it is REALLY hard to get this working in CI builds.
>
> >
> > What is your opinion?
>
> Probably we need both, until it is not clear which solution is long- term stable.
>
> Felix

BR Alexander

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/AM7PR10MB3320312B8ADF79BE6B0ED21786552%40AM7PR10MB3320.EURPRD10.PROD.OUTLOOK.COM.