From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6467463440282681344 X-Received: by 10.28.54.84 with SMTP id d81mr1394844wma.21.1507136302982; Wed, 04 Oct 2017 09:58:22 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 10.28.168.14 with SMTP id r14ls3435497wme.3.canary-gmail; Wed, 04 Oct 2017 09:58:22 -0700 (PDT) X-Google-Smtp-Source: AOwi7QCl+qqDS/j5TuIGIqddfgTeEK20oqs7tmyCroMj2a+OSYWTjXUwJs9vg6lIN23g8B05VgPG X-Received: by 10.28.15.72 with SMTP id 69mr2568865wmp.3.1507136302699; Wed, 04 Oct 2017 09:58:22 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1507136302; cv=none; d=google.com; s=arc-20160816; b=U8nffBvqxaMMHatSAToRV6ze3Jd+wNsj8kiFWs94t1UmF1F+ulAkrX4u6PmkZ2KDo7 6c3uUwhc0hkLjeXuFx+O6Csq6wq7w6rZXQH2QLOl18Hl6flJ8ErdtJNGuedwxlBGWgvX 4udHWP0LDHooNiqXhfsA2EuPxGaGj1+276e+w4s91IhBK77OkGy0DBKjRiYMbmQPxyaz n0YgqkTs527Btp+vWp17jPFZ8zdP5e/bTx3lFG8agnUArKjQBEtT2krimQXTDWAZwyxw lrYIpifssshlFYG+qdbxbPS7ePErIFeoGJ5Ww/igRyyF5BrJfp8MSTFOaCL1+2zwa2cp qNDA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:arc-authentication-results; bh=3SdF8Son4GqJF6NwkVzbn6OyGda6da8eweE1ci+VLgs=; b=dSFxcVssoUD5fv3KNB6/J0Eu8cdIrDS/2la9/XIud8FzSnHFbyZnk4tGIWAPFya1QV LwkAPJWBdmwhjneBFU556S2H+kNKKWQCE+ECM5479vXnXiRirJFlqkLrgf7B6aPN+0d8 arGhlRX4toBwoVSSoAYaUS6CGWxopUtxzKOeSWyLO7tQxKiye8i72o3Qwo9GHM754t+j 6fxJF2xuzEO1V6s31oWiin58LRUe5msVN4XETkAAu/y4bd8SnrGa16KdzcXK3+6eyBbf 2arCuPOsfgIJEgPu8cpKQOy39HlH7198okp+USpN9cdu9FDlvpgQid8L0/aOHGtQIJDU u9Eg== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=neutral (google.com: 192.35.17.2 is neither permitted nor denied by best guess record for domain of henning.schild@siemens.com) smtp.mailfrom=henning.schild@siemens.com Return-Path: Received: from thoth.sbs.de (thoth.sbs.de. [192.35.17.2]) by gmr-mx.google.com with ESMTPS id p70si1720514wma.1.2017.10.04.09.58.22 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 04 Oct 2017 09:58:22 -0700 (PDT) Received-SPF: neutral (google.com: 192.35.17.2 is neither permitted nor denied by best guess record for domain of henning.schild@siemens.com) client-ip=192.35.17.2; Authentication-Results: gmr-mx.google.com; spf=neutral (google.com: 192.35.17.2 is neither permitted nor denied by best guess record for domain of henning.schild@siemens.com) smtp.mailfrom=henning.schild@siemens.com Received: from mail3.siemens.de (mail3.siemens.de [139.25.208.14]) by thoth.sbs.de (8.15.2/8.15.2) with ESMTPS id v94GwMUE002743 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 4 Oct 2017 18:58:22 +0200 Received: from md1em3qc ([139.22.133.127]) by mail3.siemens.de (8.15.2/8.15.2) with ESMTP id v94GwL6q021096; Wed, 4 Oct 2017 18:58:21 +0200 Date: Wed, 4 Oct 2017 18:58:20 +0200 From: Henning Schild To: Alexander Smirnov Cc: Jan Kiszka , "[ext] Claudius Heine" , Subject: Re: [PATCH v4 0/4] Basic binary cache implementation Message-ID: <20171004185820.5f7bd95b@md1em3qc> In-Reply-To: <596a836b-10fa-b876-9e40-26c1920b767d@ilbers.de> References: <20171002154531.4930-1-asmirnov@ilbers.de> <2e00bbc9-9823-850e-9451-9d66350713b5@ilbers.de> <20171004174032.5a8904d8@md1em3qc> <596a836b-10fa-b876-9e40-26c1920b767d@ilbers.de> X-Mailer: Claws Mail 3.13.2 (GTK+ 2.24.31; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-TUID: I9bpD+atQ/+Q Am Wed, 4 Oct 2017 19:50:09 +0300 schrieb Alexander Smirnov : > Hi, > > On 10/04/2017 06:40 PM, Henning Schild wrote: > > Am Wed, 4 Oct 2017 17:29:08 +0300 > > schrieb Alexander Smirnov : > > > >> On 10/04/2017 02:57 PM, Jan Kiszka wrote: > >>> On 2017-10-04 10:32, [ext] Claudius Heine wrote: > >>>> Hi, > >>>> > >>>> On 10/0 '2/2017 05:45 PM, Alexander Smirnov wrote: > >>>>> Hello everybody, > >>>>> > >>>>> this series introduces basic binary caching for Isar. > >>>>> > >>>>> There is a new layer: meta-isar-bin which is intended to be the > >>>>> binary cache. All the packages that are built in Isar are stored > >>>>> in this cache using reprepro. > >>>>> > >>>>> Having the separate layer makes possible to manage this cache > >>>>> separately from the Isar build tree. So once you have built your > >>>>> packages, you could re-use this cache for further builds. > >>>> > >>>> We still have to talk about your requirement for the cache to be > >>>> sharable and your resulting implementation of it as a meta layer. > >>>> > >>>> Since I still don't see the benefit of a sharable cache and think > >>>> that putting binary packages into some kind of strange meta layer > >>>> is not the right solution and could potentially create more > >>>> problems that is solves because its very much in conflict with > >>>> the fundamental idea of a meta layer. > >> > >> The last statement is a bit strange for me. The idea of meta layers > >> is to split software stack into logical layers that could be > >> enabled/disabled on demand. Having binaries or recipes doesn't > >> matter here, it's just the way how software is provided to build. > > > > Binaries that can be rebuild should not become part of the build > > system. The need to distribute them to the targets or other > > developers/images later is what we all agree on. But that should be > > mostly outside of isar. > > Each binary could be re-built from sources (at least for Debian :-) > ). Moreover Debian provides sources for them, so they are open for > use. > > But for me it looks like: > > - Installing binaries to target from 'ftp.debian.org' is allowed. > - Installing binary dependencies for custom built package from > 'ftp.debian.org' is allowed. > - Installing my binary application from 'my.debian.repo' is not > allowed. I have to build it from sources. That is not only allowed it is absolutely wanted! But for that step you would add that repo to your multistrap config or /etc/apt/, which is in the scope of isar. Building the repo is also in the scope of isar, but holding the artefacts and publishing the repo outside the build-dir is not. Henning > What is the idea here? For me it's something like traditional OE > philosophy, but without benefit for Isar. > > One of the typical usecase: if I'm, for example, developing 'hello' > application, why it's so critical for me to build, lets say, kernel > from sources? > > Alex > > > > > Building an Image will result in the Image and a repo. Both have to > > somehow make their way to the target, but that is somewhat out of > > scope of the build system. > > > >>> > >>> Indeed, state like this (disclaimer: i didn't look into all > >>> details), it sounds like a weird architecture to me as well. > >>> Thinking of an artifact cache, I would rather expect something > >>> like OE's sstate that one may preserve across builds, share > >>> between devs, or whatever. If it's there, the build system > >>> consults it, if not, it rebuilds. But sstate is not a layer, for > >>> some good reasons. > >> > >> Let me again summarize major points here. Isar is the build system > >> designed to work with binary packages. That's the key feature of > >> this product and main difference from OE-like systems. The Isar > >> architecture assumes to be designed around binary packages, what > >> provides absolutely different approach how binary images could be > >> generated using bitbake. > >> > >> According to the Claudius's vision, that everything should always > >> be built from sources - Isar is not the best option here, Yocto > >> and OE are much better, because they were initially designed for > >> this work. Attempts to apply OE design and philosophy to Isar > >> could have negative impact and limit possible ways to implement > >> custom features, because OE wasn't designed for this features. > >> Also I wonder why we are able to use binary Debian packages, while > >> the rest of software should be built from sources. > >> > >> This series introduces support for binary apt repositories in Isar. > >> This feature is response to the following cases: > >> > >> 1. Publish custom apt layer in web. When I need to install some > >> custom software to my Debian system, I frequently see instructions > >> like: > >> - Add 'blablabla' to your /etc/apt/source.list > >> - Run apt-get update > >> - Run apt-get install > >> > >> That's for example how Jenkins and GitLab are distributed: > >> > >> https://pkg.jenkins.io/debian/ > > > > I think we all get that and agree on it. > > > >> The management of this public apt could require some manual effort, > >> while Isar (in plans) could update it automatically, so you update > >> source code for your product, run Isar and it provides complete apt > >> that you could upload to your server. > > > > Exactly, those manual steps or custom scripts are outside the scope > > of Isar. Just like the flashing of the Images onto the targets. > > > >> 2. Implement easy software update. On our host systems we usually > >> do not flash whole image if we need to update some packages. The > >> update is performed using 'apt-get install' (or by similar tool). > >> But in this case binary repo is needed, what could be again > >> provided by Isar. > >> > >> Please do not limit Isar by deep embedded application, host systems > >> also could be managed by Isar. > > > > If the repo ends up in the output directory all those scenarios are > > still possible, _and_ your source tree will not contain > > build-output. > > Henning > > > >> 3. In long term, help Debian community to manage Debian > >> infrastructure, what includes publishing apt updates. > >> > > >