From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6518759238035046400 X-Received: by 10.80.184.23 with SMTP id j23mr16832646ede.3.1517826417862; Mon, 05 Feb 2018 02:26:57 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 10.80.149.230 with SMTP id x35ls3941561eda.5.gmail; Mon, 05 Feb 2018 02:26:57 -0800 (PST) X-Google-Smtp-Source: AH8x225E6Zm+ZQlAlMgJv00vxZv3N3LSIDoPfgbRhgvsaxb8tP4tw7H9L89kMQlTe8MZvzmt70Pm X-Received: by 10.80.242.130 with SMTP id f2mr16886459edm.5.1517826417411; Mon, 05 Feb 2018 02:26:57 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1517826417; cv=none; d=google.com; s=arc-20160816; b=iTedVhdw8Jv887aN2Hl/rVjHgFvlzqJf+0rec9Uzi62fDarY4JMNc5MZPuzKfnPQtq TdoevPYOU7TTyaYMocgD4ABEvto311Y+K6hAhBZ1bc+JeH5Y1WBu7YhvHD44OilX+A9C 1mvVy7XsZD5GlXGwVMyaqh1npnjDrcC9TxkWffERzhhU2Yi9B2OI0kaZdwH499HaN53s D1aT6Y/p1m8hjAoNsrsnDBiPPjOfJjwSxOjXaBbteLvQ+GCn4WU1lWFOceTa4n++Ervy l17vAyfbVFIkx40oZRVjWqg/JMn3qA+7YuvuyGtNjXWpOzDXvZl2/ig6qikbGnWh0jeb 63PQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:cc:to:subject :arc-authentication-results; bh=KqBv8YgIChdwTuyZ/7uWyFj8e9b3eZQDVt3yg7WT2WQ=; b=VNMcOsD3DHTIDL4NKrom1p5fvOONl8YEy6R9tph5ZrWtEEPqU4yvbZo92bzFOx8cFU 1hKV/qefHFxGyziwBQp7uIJC0ORdY+DrRYhE1tjJh/OBLaG0DeSxqabWQ92kk0OAjLIq M3LRyMYyGYQ8cynbuSAi2M4WDeKP3W1t8ldEjnX6sFsqA32V2tlk03dyWzbb5roTmxZ0 dkO4V64icdq51jSyo+W1YqSl4DFDM6SpjuZA/mWVAbcOJt3u15AQVWlEwnLYSCqBkg9u doQmZKyHU6QBeFcV0RYgiOCd379fp/qbv+0blx2lzS14BuqEqAmz3+0pQB5m+tarpkMJ NtuA== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) smtp.mailfrom=asmirnov@ilbers.de Return-Path: Received: from aqmola.ilbers.de (aqmola.ilbers.de. [85.214.62.211]) by gmr-mx.google.com with ESMTPS id z17si696965edz.1.2018.02.05.02.26.57 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 05 Feb 2018 02:26:57 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) client-ip=85.214.62.211; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) smtp.mailfrom=asmirnov@ilbers.de Received: from [10.0.2.15] ([188.227.110.165]) (authenticated bits=0) by aqmola.ilbers.de (8.14.4/8.14.4/Debian-4+deb7u1) with ESMTP id w15AQqDG024521 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Mon, 5 Feb 2018 11:26:53 +0100 Subject: Re: [PATCH 0/5] support creation of a full repo for offline/reproducible builds To: Jan Kiszka , Cedric_Hombourger@mentor.com, isar-users@googlegroups.com Cc: Claudius Heine References: <20180204175454.220-1-Cedric_Hombourger@mentor.com> From: Alexander Smirnov Message-ID: <79ae1eda-9b5b-a000-8740-b6753a894cd4@ilbers.de> Date: Mon, 5 Feb 2018 13:26:47 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-TUID: MfkW87cY6pXl On 02/05/2018 10:18 AM, Jan Kiszka wrote: > On 2018-02-04 18:54, Cedric_Hombourger@mentor.com wrote: >> From: Cedric Hombourger >> >> The package repository created by isar using reprepro only includes packages by isar. >> To support offline/reproducible builds, this changeset adds a do_populate task to >> augment the repo with packages used during the build. The task may be used against >> the buildchroot and images recipes. It should be noted that isar currently assumes >> that the base distribution will provide both an -updates and security feed. This is >> certainly true for Debian but may not be the case for other distributions or when >> when using our own feed. > > The automatic addition of update and security feeds is more of a > workaround until we have multi-repo support like Claudius is working on. > I guess we can then drop this and just have repo lists for the different > Debian versions with multiple entries. > >> >> Some rework may be needed if the isar-apt changes get merged first. Conceptually the >> implementation may not change much (as far as I can tell!) >> >> Please review and let me know if any rework is required. > > There is indeed quite some overlap with what Alex and I were discussing > yesterday at FOSDEM. However, also looking at these patches, we need to > do some homework first. As you correctly stated in patch 1, there is the > issue that we pull package list twice at different times: first for the > buildchroot and then again for the image. That needs to go first so that > we end up with a consistent build. Also, all those duplications in logic > between the two chroot setup recipes are killing us. > > So I would propose the following roadmap: > > - consolidate chroot building into a common class that both buildchroot > and image recipes derive from, while doing that > - generate multistrap.conf (e.g. "cat < all over the place I'd propose here to start with buildchroot/image recipes clean up. I mean there are lots of files in: - meta/recipes-devtools/buildchroot/files - meta-isar/recipes-core/images/files Until these files will be unified, it makes no sense to implement common class - you anyway have to keep several sets of these files per image/buildchroot. That's why I proposed some time ago to drop multistrap hooks and implement their actions inside bitbake recipe, but this idea wasn't found support here. > - build up a single apt cache that all chroot builders use In other words, there should be the only one component/recipe who fetches the upstream packages. There is still open question, how to fetch all the packages: in a single step or during the build. I want to have an option similar to Yocto's one: -c fetchall to have possibility to fetch all the necessary artifacts and do not depend from network anymore. > - derive the mirror repo list after the first installation from that > cache which will then contain ALL required packages in the right > versions > - describe the workflow (doc/) how to generate that mirror and how to > use it in succeeding builds > > Makes sense? > > We should than clarify who will work on what. > > BTW, the very first step is sorting out all the other patches and > applying them to next. For that, Alex plans to first update bitbake and > then go through what is pending (and still merges fine). My wish will be to keep an eye on this list and synchronize your activities to avoid overlaps. Alex