From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6449958519840964608 X-Received: by 10.80.201.7 with SMTP id o7mr2451667edh.3.1512029050975; Thu, 30 Nov 2017 00:04:10 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 10.80.241.28 with SMTP id w28ls10515764edl.0.gmail; Thu, 30 Nov 2017 00:04:10 -0800 (PST) X-Google-Smtp-Source: AGs4zMae6ECWNpUvhHiXrJf4nJwT81FSSPX1p2+DO1FS3WO57z0zAwoOESzN94AXwxsZkAOqVxKF X-Received: by 10.80.151.161 with SMTP id e30mr2448702edb.4.1512029050630; Thu, 30 Nov 2017 00:04:10 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1512029050; cv=none; d=google.com; s=arc-20160816; b=sRwxhbYr1OsSYHl0/yX2FvqBMQ75DVyitg7dYKSmL/hZ1H6K8pPccE116tMm3slNPi iXH+BERrueiExVJ5zJJdYFpfm1QBx8jHERwqtrrjUR9JPkUBmxudUYpeni9WsqqpSJfF d40AX/eVvTxGeSuwDtyjuMG6Aqro4S9zHpnx/q0OgpYlBwF8A3s2QxFYXO1pJ6F4g3bR 5GgFTRvF24+9PVWkST7s2IEi+vUex0WghmiRSPbiCG1tAteseKmCLAiVeKeL59RcvFH+ VqzYu6itC6d+CZHbd3RpKdz5yKSAxWiuAZEcjQ2xAmtYGIBHrk6GoJo52ESpb5Dwhwu5 oMCQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:to:subject :arc-authentication-results; bh=edeL0UXBQw4BQPdAARxSFEDABMTn+guQZu85OquLp/0=; b=OeELT8EnvbETLx92342s7oIZ3Q+cbGxH8+615Y9Lf+M7Y+TYwfngCbQxSvV440SfD9 hShK2wS+3AoDADCqTGwaj4FjpXHkeNGXqtcgfh2yaiGHmriOBVBYIFtkI2rBbfXonYiu X+aKCECBIkJnW84ZjsBjmNyZOz89p5zEZQr8L5DeOtMS0omdcyN3Rs10zrZylw+LJKlb SQ+OYJjKRa06pO8tnrqtENyHzjndAfagqSQf6zsfpnZnmpwEAtDqU9eeDKrHAC7LwrOy dnTI3izXUAVNpbMX2+bB+tFO/H2S1Rd0euLwutQEEtncBVbJLhs/pNmK8MlAWfzeOkzV 3khw== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) smtp.mailfrom=asmirnov@ilbers.de Return-Path: Received: from aqmola.ilbers.de (aqmola.ilbers.de. [85.214.62.211]) by gmr-mx.google.com with ESMTPS id h23si173210ede.4.2017.11.30.00.04.10 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 30 Nov 2017 00:04:10 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) client-ip=85.214.62.211; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) smtp.mailfrom=asmirnov@ilbers.de Received: from [10.0.2.15] ([188.227.110.165]) (authenticated bits=0) by aqmola.ilbers.de (8.14.4/8.14.4/Debian-4+deb7u1) with ESMTP id vAU846to004800 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Thu, 30 Nov 2017 09:04:08 +0100 Subject: Re: Reproducibility of builds To: Jan Kiszka , isar-users@googlegroups.com References: <20171120083322.mz7uiz7nbgesf2qp@MD1KR9XC.ww002.siemens.net> <0508e061-441f-396b-98df-ab6226aa04cd@denx.de> <6b33bcfe-6d5f-1823-6a36-43847556d4b8@siemens.com> From: Alexander Smirnov Message-ID: <18e4cbc4-83b1-86f1-d24e-e0c68f72224d@ilbers.de> Date: Thu, 30 Nov 2017 11:04:01 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: <6b33bcfe-6d5f-1823-6a36-43847556d4b8@siemens.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-TUID: ukoHS9yUDeO1 Hi Jan, On 11/29/2017 10:02 PM, Jan Kiszka wrote: > On 2017-11-29 19:53, Alexander Smirnov wrote: >> Hi everybody, >> >> I've started working on this topic and here I'd like to share my vision. >> At the moment I've implemented simple PoC in my branch >> 'asmirnov/build_rep'. >> >> What it does: >> >> 1. There is new recipe: base-apt. It provides task which: >> >>  - Fetches packages from origin Debian apt to local folder using >> deboostrap. >>  - Put these packages via 'reprepro' to local repository called 'base-apt'. >> >> 2. Buildchroot uses 'base-apt' to generate rootfs. >> >> 3. Isar image uses 'base-apt' and 'isar' repos to generate rootfs. >> >> >> >> What are the key benefits of this approach: >> >> 1. Download session for upstream packages is performed in a single step. >> >> 2. You could use your local 'versioned' apt repository instead of >> downloading origin packages. >> >> 3. Having local apt repository managed by 'reprepro' provides us >> possibility to implement version pinning. Reprepro provides lots of >> things like: >>  - Get package name. >>  - Get package version. >>  - Remove specific package from repo. >>  - Add single package to repo. >> >> So in general, if we have know which package version we want to have, we >> need to get binary with this version and put it to 'base-apt'. >> > > But this encodes the versions of the packages to be used implicitly into > their unique presence inside some local apt repo, no? > > I would prefer a solution that stores the packages list with versions as > well and only uses that list, when provided, independent of the repo > content. That way we can throw all downloaded packages back into a > single archive repo. Have one repo per project version will quickly > explode storage-wise (or you need extra deduplication mechanisms). > > That said, I'm fine with getting there in several steps, and this can be > a valid first one. > I got it. Here I only mean that there are some tools that could help us in implementing specific logic. At the moment I don't have the final vision, but hope it will appear during experiments with this PoC. My main wish is to avoid manual hacks with Debian artifacts and use generic tools as much as possible. Alex > Jan > >> >> >> Which issues I see at the moment: >> >> 1. The key issue for me the list of packages for 'base-apt'. So before >> 'base-apt' task is executed, we should prepare full list of packages >> that will be used by: >>  - buildchroot (BUILDCHROOT_PREINSTALL). >>  - packages to build (their build deps). >>  - image (IMAGE_PREINSTALL). >> >> So I have an idea how to implement this via special tasks, will push >> patch for RFC, but if you have your own proposals, I'll be happy to >> discuss them! >> >> Alex >>