Re: [PATCH v4 0/4] Basic binary cache implementation

[Henning Schild <henning.schild@siemens.com>]

Am Wed, 4 Oct 2017 17:29:08 +0300
schrieb Alexander Smirnov <asmirnov@ilbers.de>:

> On 10/04/2017 02:57 PM, Jan Kiszka wrote:
> > On 2017-10-04 10:32, [ext] Claudius Heine wrote:  
> >> Hi,
> >>
> >> On 10/0    '2/2017 05:45 PM, Alexander Smirnov wrote:  
> >>> Hello everybody,
> >>>
> >>> this series introduces basic binary caching for Isar.
> >>>
> >>> There is a new layer: meta-isar-bin which is intended to be the
> >>> binary cache. All the packages that are built in Isar are stored
> >>> in this cache using reprepro.
> >>>
> >>> Having the separate layer makes possible to manage this cache
> >>> separately from the Isar build tree. So once you have built your
> >>> packages, you could re-use this cache for further builds.  
> >>
> >> We still have to talk about your requirement for the cache to be
> >> sharable and your resulting implementation of it as a meta layer.
> >>
> >> Since I still don't see the benefit of a sharable cache and think
> >> that putting binary packages into some kind of strange meta layer
> >> is not the right solution and could potentially create more
> >> problems that is solves because its very much in conflict with the
> >> fundamental idea of a meta layer.  
> 
> The last statement is a bit strange for me. The idea of meta layers
> is to split software stack into logical layers that could be 
> enabled/disabled on demand. Having binaries or recipes doesn't matter 
> here, it's just the way how software is provided to build.

Binaries that can be rebuild should not become part of the build
system. The need to distribute them to the targets or other
developers/images later is what we all agree on. But that should be
mostly outside of isar.

Building an Image will result in the Image and a repo. Both have to
somehow make their way to the target, but that is somewhat out of scope
of the build system.

> > 
> > Indeed, state like this (disclaimer: i didn't look into all
> > details), it sounds like a weird architecture to me as well.
> > Thinking of an artifact cache, I would rather expect something like
> > OE's sstate that one may preserve across builds, share between
> > devs, or whatever. If it's there, the build system consults it, if
> > not, it rebuilds. But sstate is not a layer, for some good reasons.
> >   
> 
> Let me again summarize major points here. Isar is the build system 
> designed to work with binary packages. That's the key feature of this 
> product and main difference from OE-like systems. The Isar
> architecture assumes to be designed around binary packages, what
> provides absolutely different approach how binary images could be
> generated using bitbake.
> 
> According to the Claudius's vision, that everything should always be 
> built from sources - Isar is not the best option here, Yocto and OE
> are much better, because they were initially designed for this work. 
> Attempts to apply OE design and philosophy to Isar could have
> negative impact and limit possible ways to implement custom features,
> because OE wasn't designed for this features.
> Also I wonder why we are able to use binary Debian packages, while
> the rest of software should be built from sources.
> 
> This series introduces support for binary apt repositories in Isar.
> This feature is response to the following cases:
> 
> 1. Publish custom apt layer in web. When I need to install some
> custom software to my Debian system, I frequently see instructions
> like:
>   - Add 'blablabla' to your /etc/apt/source.list
>   - Run apt-get update
>   - Run apt-get install
> 
> That's for example how Jenkins and GitLab are distributed:
> 
>    https://pkg.jenkins.io/debian/

I think we all get that and agree on it.

> The management of this public apt could require some manual effort, 
> while Isar (in plans) could update it automatically, so you update 
> source code for your product, run Isar and it provides complete apt
> that you could upload to your server.

Exactly, those manual steps or custom scripts are outside the scope of
Isar. Just like the flashing of the Images onto the targets.
 
> 2. Implement easy software update. On our host systems we usually do
> not flash whole image if we need to update some packages. The update
> is performed using 'apt-get install' (or by similar tool). But in
> this case binary repo is needed, what could be again provided by Isar.
> 
> Please do not limit Isar by deep embedded application, host systems
> also could be managed by Isar.

If the repo ends up in the output directory all those scenarios are
still possible, _and_ your source tree will not contain build-output.
 
Henning

> 3. In long term, help Debian community to manage Debian
> infrastructure, what includes publishing apt updates.
>