From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6467463440282681344 X-Received: by 10.28.143.13 with SMTP id r13mr2739335wmd.30.1507135818951; Wed, 04 Oct 2017 09:50:18 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 10.28.9.5 with SMTP id 5ls3440292wmj.2.canary-gmail; Wed, 04 Oct 2017 09:50:17 -0700 (PDT) X-Google-Smtp-Source: AOwi7QDVaWWrPy5QKIRqmWog0lxaYKhw62wzqj3I8VYdziIUgBHH2J5dGsCl6YsPKhig/zbyC5// X-Received: by 10.28.147.133 with SMTP id v127mr1605250wmd.18.1507135817847; Wed, 04 Oct 2017 09:50:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1507135817; cv=none; d=google.com; s=arc-20160816; b=V0hdEfyQpCRCH36myvTpdPL6GLmRSFnuvw2GIDooX88bsakeEHJTZVQpnDtIZPZ4UU JkwmJyyoe0KtYw2L7t6RixqWRZ6Kt5g7l6uZvxgfwtok616rHiVGbxsg8gBA8WrRlG9f rMznGfMUNKUUaqqlqNKOmulGw9WtVIFRqr3SvuRM3VzMcjgWjOmmtKTr+TQybisuvrgJ y3y2yd/4HgRxWnrlDxAz72N7/ndY3sMCEXCxeODlKITXroBwbVnJbpqJHOzBUO5U1v3a wTYcPuE8uB6LjOCEdZEHSLdhp9kavtGdzwpk873LbnbFzuIuB+iLIFH8IPNGMysnABNo UMSg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:cc:to:subject :arc-authentication-results; bh=dg9K9z8YLp/xrKKc3YTLkKO32rT5bPMnFGnMdaY8Q5o=; b=xOxsBuYku/l43ucsNGA0Zj1Z9iYp3GcvxweGBx+tvyoqmB+WgxwNTNWBIUfNhZqAFo mLDHTXHv3Pbe5kQXTz2mYBqse6SFE3CpwhIchOgqZ37jfxlsZtYZI85IpP2P+8URfX8l b2RAp7unfQomTKJtmRQgalglnbnpRJ257C9Amuq4Tyrpuf0ocW1FjMxkZhhIqe9Zl1V5 3F7BjwiPrDhBmPeuEaPFOiMnAwz54wcTvuRCoaIPPSg5H9KA9CogCVmI67K0iwThiSVE S70h7JvVHzC4YQUChVw5XgQFe/Z5+3RZlopT5dHOd1pXiT9zGigaPzOjS0LJj6yw6klR +yig== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) smtp.mailfrom=asmirnov@ilbers.de Return-Path: Received: from aqmola.ilbers.de (aqmola.ilbers.de. [85.214.62.211]) by gmr-mx.google.com with ESMTPS id 196si1159428wmk.4.2017.10.04.09.50.17 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 04 Oct 2017 09:50:17 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) client-ip=85.214.62.211; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) smtp.mailfrom=asmirnov@ilbers.de Received: from [10.0.2.15] ([188.227.110.165]) (authenticated bits=0) by aqmola.ilbers.de (8.14.4/8.14.4/Debian-4+deb7u1) with ESMTP id v94GoEMW000425 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Wed, 4 Oct 2017 18:50:16 +0200 Subject: Re: [PATCH v4 0/4] Basic binary cache implementation To: Henning Schild Cc: Jan Kiszka , "[ext] Claudius Heine" , isar-users@googlegroups.com References: <20171002154531.4930-1-asmirnov@ilbers.de> <2e00bbc9-9823-850e-9451-9d66350713b5@ilbers.de> <20171004174032.5a8904d8@md1em3qc> From: Alexander Smirnov Message-ID: <596a836b-10fa-b876-9e40-26c1920b767d@ilbers.de> Date: Wed, 4 Oct 2017 19:50:09 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.2.1 MIME-Version: 1.0 In-Reply-To: <20171004174032.5a8904d8@md1em3qc> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-TUID: vstLxAivRzLG Hi, On 10/04/2017 06:40 PM, Henning Schild wrote: > Am Wed, 4 Oct 2017 17:29:08 +0300 > schrieb Alexander Smirnov : > >> On 10/04/2017 02:57 PM, Jan Kiszka wrote: >>> On 2017-10-04 10:32, [ext] Claudius Heine wrote: >>>> Hi, >>>> >>>> On 10/0 '2/2017 05:45 PM, Alexander Smirnov wrote: >>>>> Hello everybody, >>>>> >>>>> this series introduces basic binary caching for Isar. >>>>> >>>>> There is a new layer: meta-isar-bin which is intended to be the >>>>> binary cache. All the packages that are built in Isar are stored >>>>> in this cache using reprepro. >>>>> >>>>> Having the separate layer makes possible to manage this cache >>>>> separately from the Isar build tree. So once you have built your >>>>> packages, you could re-use this cache for further builds. >>>> >>>> We still have to talk about your requirement for the cache to be >>>> sharable and your resulting implementation of it as a meta layer. >>>> >>>> Since I still don't see the benefit of a sharable cache and think >>>> that putting binary packages into some kind of strange meta layer >>>> is not the right solution and could potentially create more >>>> problems that is solves because its very much in conflict with the >>>> fundamental idea of a meta layer. >> >> The last statement is a bit strange for me. The idea of meta layers >> is to split software stack into logical layers that could be >> enabled/disabled on demand. Having binaries or recipes doesn't matter >> here, it's just the way how software is provided to build. > > Binaries that can be rebuild should not become part of the build > system. The need to distribute them to the targets or other > developers/images later is what we all agree on. But that should be > mostly outside of isar. Each binary could be re-built from sources (at least for Debian :-) ). Moreover Debian provides sources for them, so they are open for use. But for me it looks like: - Installing binaries to target from 'ftp.debian.org' is allowed. - Installing binary dependencies for custom built package from 'ftp.debian.org' is allowed. - Installing my binary application from 'my.debian.repo' is not allowed. I have to build it from sources. What is the idea here? For me it's something like traditional OE philosophy, but without benefit for Isar. One of the typical usecase: if I'm, for example, developing 'hello' application, why it's so critical for me to build, lets say, kernel from sources? Alex > > Building an Image will result in the Image and a repo. Both have to > somehow make their way to the target, but that is somewhat out of scope > of the build system. > >>> >>> Indeed, state like this (disclaimer: i didn't look into all >>> details), it sounds like a weird architecture to me as well. >>> Thinking of an artifact cache, I would rather expect something like >>> OE's sstate that one may preserve across builds, share between >>> devs, or whatever. If it's there, the build system consults it, if >>> not, it rebuilds. But sstate is not a layer, for some good reasons. >>> >> >> Let me again summarize major points here. Isar is the build system >> designed to work with binary packages. That's the key feature of this >> product and main difference from OE-like systems. The Isar >> architecture assumes to be designed around binary packages, what >> provides absolutely different approach how binary images could be >> generated using bitbake. >> >> According to the Claudius's vision, that everything should always be >> built from sources - Isar is not the best option here, Yocto and OE >> are much better, because they were initially designed for this work. >> Attempts to apply OE design and philosophy to Isar could have >> negative impact and limit possible ways to implement custom features, >> because OE wasn't designed for this features. >> Also I wonder why we are able to use binary Debian packages, while >> the rest of software should be built from sources. >> >> This series introduces support for binary apt repositories in Isar. >> This feature is response to the following cases: >> >> 1. Publish custom apt layer in web. When I need to install some >> custom software to my Debian system, I frequently see instructions >> like: >> - Add 'blablabla' to your /etc/apt/source.list >> - Run apt-get update >> - Run apt-get install >> >> That's for example how Jenkins and GitLab are distributed: >> >> https://pkg.jenkins.io/debian/ > > I think we all get that and agree on it. > >> The management of this public apt could require some manual effort, >> while Isar (in plans) could update it automatically, so you update >> source code for your product, run Isar and it provides complete apt >> that you could upload to your server. > > Exactly, those manual steps or custom scripts are outside the scope of > Isar. Just like the flashing of the Images onto the targets. > >> 2. Implement easy software update. On our host systems we usually do >> not flash whole image if we need to update some packages. The update >> is performed using 'apt-get install' (or by similar tool). But in >> this case binary repo is needed, what could be again provided by Isar. >> >> Please do not limit Isar by deep embedded application, host systems >> also could be managed by Isar. > > If the repo ends up in the output directory all those scenarios are > still possible, _and_ your source tree will not contain build-output. > > Henning > >> 3. In long term, help Debian community to manage Debian >> infrastructure, what includes publishing apt updates. >> > -- With best regards, Alexander Smirnov ilbers GmbH Baierbrunner Str. 28c D-81379 Munich +49 (89) 122 67 24-0 http://ilbers.de/ Commercial register Munich, HRB 214197 General manager: Baurzhan Ismagulov