From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6518759238035046400 X-Received: by 10.25.157.193 with SMTP id g184mr3674888lfe.37.1517821878744; Mon, 05 Feb 2018 01:11:18 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 10.46.75.18 with SMTP id y18ls655901lja.9.gmail; Mon, 05 Feb 2018 01:11:18 -0800 (PST) X-Google-Smtp-Source: AH8x224O7RT0Y6ng7MWCrJebZWXKsCWEzfFqitbsPN01XVQ/GAspckaJw0CfSAfkB8ZnY7NpEzd9 X-Received: by 10.46.2.69 with SMTP id 66mr1767345ljc.23.1517821878101; Mon, 05 Feb 2018 01:11:18 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1517821878; cv=none; d=google.com; s=arc-20160816; b=Mk4Wz21tc0JI5X3BkGVLmqMt29t6J2kFT26P+hOJzI8a4fsIWMl7Zu6hBJH+kvW77U P+SePLiMqiVeH8R0W6Ak6CKq84fiMJBkz00d0IBc7NNDdTrAPd3Pe3dt9uREXGvt/QxS Q1AglH5SuLfh0OkO2SotKVr4V1ZvnFbc6DWtxQXQFZzq4w0Dy8vdjjyAhIq4jfYRHgB7 2RDQWmGRQKZkrz08mqQFFtMKhni8VOcvsVGJdiZZObTR2V9auj1erSWh35zRHMPUJIAs OtLgT9j9JKPHBBAxrY3f00TIvujjME/3gCYRZSPyaPBNqYikzptC587iNOYVrhplT48H 9ZtA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-language:content-transfer-encoding:in-reply-to:mime-version :user-agent:date:message-id:from:references:cc:to:subject :dkim-signature:arc-authentication-results; bh=OSbRQCByQmN5a1DmVCkGuEhG5SKOfpxhiOaZrhNpU9k=; b=Nnl72AUzD2l1DH2P9RbC95lxWhHBiNhLElpIaAD17nDzsben9GKRy3ocGEILImPi2D nrTczL2G1DQOeD2zQV4R9cUqRkybh7KBxDJX4oCZEeqwGbF9onLoyYdXJL59Q4nSiCrf N7wd89S7+JftE1zyJX04ULO75cKo2kZGLfmItjoqa9Te0dbJid7zWoAusgJLPCoAnOl+ TnaIpJaqyuW78wiYZmdvHikajD6mdcb09GAHLxpu0xHWLbsieaHQt5wy4bPXd+Dhs1Vs 6ADpUfTA0Qd+OtPt6nTyED/Aign2S4OJhrMY1/gNIN2nnzi513hrWBxT0XdSE8wzA/jd nksA== ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@googlemail.com header.s=20161025 header.b=EsW+I3rQ; spf=pass (google.com: domain of benbrenson89@googlemail.com designates 2a00:1450:400c:c09::236 as permitted sender) smtp.mailfrom=benbrenson89@googlemail.com; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=googlemail.com Return-Path: Received: from mail-wm0-x236.google.com (mail-wm0-x236.google.com. [2a00:1450:400c:c09::236]) by gmr-mx.google.com with ESMTPS id l26si690245lja.0.2018.02.05.01.11.18 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 05 Feb 2018 01:11:18 -0800 (PST) Received-SPF: pass (google.com: domain of benbrenson89@googlemail.com designates 2a00:1450:400c:c09::236 as permitted sender) client-ip=2a00:1450:400c:c09::236; Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@googlemail.com header.s=20161025 header.b=EsW+I3rQ; spf=pass (google.com: domain of benbrenson89@googlemail.com designates 2a00:1450:400c:c09::236 as permitted sender) smtp.mailfrom=benbrenson89@googlemail.com; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=googlemail.com Received: by mail-wm0-x236.google.com with SMTP id t74so24627813wme.3 for ; Mon, 05 Feb 2018 01:11:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20161025; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding:content-language; bh=OSbRQCByQmN5a1DmVCkGuEhG5SKOfpxhiOaZrhNpU9k=; b=EsW+I3rQ1eaZ6pY/xbJAJNoRdtCEkHWGWQHb6ikJbURi6CwaKo8g0W6IVvUWFLtmzd ocMaJj3xpuHvatlWgzpMpDInde9SupeXU+BKxBd408Rqiag4Eh+ivRqrfjiRljaXXeSF Z9Tr6gptLtm5mDneWpJ1O36dzQ1rh9ZyIP61MqDbueo1iUssTRZ7Z7baRC3fWdGt06MF bzZQUA5iN7XwWPoLLZVEQyUj3Y7exMgJSjewDDTQDbGeJXC3Ue5kmj059Mj19YiEMZlt UAhfmiaoB6vwKgylcwNoppGuwu34xstO7ZeF1LIytcCSK+Owah/7TNTJdonh1bZqAMgf FeMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding :content-language; bh=OSbRQCByQmN5a1DmVCkGuEhG5SKOfpxhiOaZrhNpU9k=; b=LF4KqmS89HtYXRek2/ZSL4jD2D3NeinAOp6/wnFBy3YsBD2XMy26qSiqssZiWeadfT uFg1n1TfOaZlBt34RB8zRLSt0bCzbt9M0mJe/FGM9O3yvtknvi9cKpMbv3hGfr8GCDQu y1YfyY3GIwGciAHQKl+Dm7Qd/HtjV0GJ2FxiEOFz4Uq8p5TVnviZx3uKanRv72oW6zXN TkPRzVlL87/l1xfI2+lgmp9mSYTRojXwQsWnF0FC42kw61lcSWxUkWhyxvjU+BgJiHLO Ll7Vpf6KNKg+qPq3WAPuJAG21P2ShUox6UOONwckTDjLnfdptEeuEf4SxUuqOre0JITm 9uHg== X-Gm-Message-State: AKwxytekkSy95eZQDOQtKBV9o2jxSWtthH3IRMganJBAkqLWVvO950yi Z96HEGDY5TsHoSi0u7B/a8s= X-Received: by 10.80.205.140 with SMTP id p12mr78209777edi.169.1517821877745; Mon, 05 Feb 2018 01:11:17 -0800 (PST) Return-Path: Received: from [192.168.43.209] (ip-109-41-194-61.web.vodafone.de. [109.41.194.61]) by smtp.gmail.com with ESMTPSA id z49sm7050396edd.93.2018.02.05.01.11.16 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 05 Feb 2018 01:11:16 -0800 (PST) Subject: Re: [PATCH 0/5] support creation of a full repo for offline/reproducible builds To: Jan Kiszka , Cedric_Hombourger@mentor.com, isar-users@googlegroups.com, Alexander Smirnov Cc: Claudius Heine References: <20180204175454.220-1-Cedric_Hombourger@mentor.com> From: Benedikt Niedermayr Message-ID: <1268168b-a8f9-31e2-9029-894c81ed3a69@googlemail.com> Date: Mon, 5 Feb 2018 10:11:14 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US X-TUID: UJVQwxFua90V Am 05.02.2018 um 08:18 schrieb Jan Kiszka: > On 2018-02-04 18:54, Cedric_Hombourger@mentor.com wrote: >> From: Cedric Hombourger >> >> The package repository created by isar using reprepro only includes packages by isar. >> To support offline/reproducible builds, this changeset adds a do_populate task to >> augment the repo with packages used during the build. The task may be used against >> the buildchroot and images recipes. It should be noted that isar currently assumes >> that the base distribution will provide both an -updates and security feed. This is >> certainly true for Debian but may not be the case for other distributions or when >> when using our own feed. > The automatic addition of update and security feeds is more of a > workaround until we have multi-repo support like Claudius is working on. > I guess we can then drop this and just have repo lists for the different > Debian versions with multiple entries. > >> Some rework may be needed if the isar-apt changes get merged first. Conceptually the >> implementation may not change much (as far as I can tell!) >> >> Please review and let me know if any rework is required. > There is indeed quite some overlap with what Alex and I were discussing > yesterday at FOSDEM. However, also looking at these patches, we need to > do some homework first. As you correctly stated in patch 1, there is the > issue that we pull package list twice at different times: first for the > buildchroot and then again for the image. That needs to go first so that > we end up with a consistent build. Also, all those duplications in logic > between the two chroot setup recipes are killing us. > > So I would propose the following roadmap: > > - consolidate chroot building into a common class that both buildchroot > and image recipes derive from, while doing that > - generate multistrap.conf (e.g. "cat < all over the place > - build up a single apt cache that all chroot builders use > - derive the mirror repo list after the first installation from that > cache which will then contain ALL required packages in the right > versions > - describe the workflow (doc/) how to generate that mirror and how to > use it in succeeding builds > > Makes sense? > > We should than clarify who will work on what. > > BTW, the very first step is sorting out all the other patches and > applying them to next. For that, Alex plans to first update bitbake and > then go through what is pending (and still merges fine). > > Jan > Hi, > - consolidate chroot building into a common class that both buildchroot > and image recipes derive from, while doing that Yes, sounds good. > - generate multistrap.conf (e.g. "cat < all over the place Maybe creating a new task like do_multistrap_conf() written in Python may help here. Python could import the "configparser" module for creating the config file? AFAIK the configparser module is included in standart python installation, so no further host system dependencies. > - build up a single apt cache that all chroot builders use > - describe the workflow (doc/) how to generate that mirror and how to > use it in succeeding builds So we will separate building the apt cache from the Isar build process itself? Means, the user will setup the Mirror manually? > - derive the mirror repo list after the first installation from that > cache which will then contain ALL required packages in the right > versions Can you explain that in more detail? Benni