From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6449958519840964608 X-Received: by 10.25.170.140 with SMTP id t134mr18534lfe.8.1511982123734; Wed, 29 Nov 2017 11:02:03 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 10.46.101.66 with SMTP id z63ls2888338ljb.7.gmail; Wed, 29 Nov 2017 11:02:03 -0800 (PST) X-Google-Smtp-Source: AGs4zMZlBK0FWdRsltYg9s0YDE8Fsg+uhrK6Y/9xuUby4ozsWwX9YigLL4MXknid4EhI1bJu8VZC X-Received: by 10.25.202.4 with SMTP id a4mr133719lfg.32.1511982123309; Wed, 29 Nov 2017 11:02:03 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1511982123; cv=none; d=google.com; s=arc-20160816; b=fOfMM9VTw4aC1RejF625DcLe7DGZKVwfD92J8w93n75wq6cgdqgtKOrW2vqzfE3kzY dyoo207R3IS91coOszAsLoyjGMIoSGCntaTGrfACBnkHYSejyFiDeokNnQn+EBK30KtS RKjt9y5Q34fpkPUwNLYoFAy4oekIwGdgR9lbVeMAypG2EQVSfUldip7heUjSo5TY6wF8 LhQ8T1BRuzvxvS9J1qB4FvpgPdLGh+Y7ycqW8yJQKaJD5xxwJmzHbqm9yyGh5n7YdfFu SFK9kQ6hFeo7EjaPYQWcIKJeI+E5+gRcqMHhoGTOIiMMe2WwLjA4h7fVt+qcz5uSeeav bYQA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:to:subject :arc-authentication-results; bh=n0IqQZYu5TDms/eLPPIgEUO3zvo0rG+kGxd6fGBinCc=; b=tQDYiacaGU2rXFxwPhR+/XM+J3mK1FCtdBh9mCAVwkS/Iwrs0Oor/ZXfqREAc8j+C6 L3EVIgfWR1VJgpJp5qod7xpwPa9h4R8nPoArrPlZ0GEH7FKbUfXdKI+tmeXKuCN1FjLk OcOC9NXfJ3KBVQOLMYQ2UwM311odUhRJuAgdaPpybbfp6PE/vwo6tDXtJkF1McT+XCXI 13fOQGsqElSC+aJ5zUQXPrcgI2c+5vtVKhwWz/k0GuYjYgwH7bK9dEA4oveVbspgzIZ2 Fje4VHeei7+KeWF4pR1ZKDAK9f71S/xx7Au97I09nnjX/TbbTLvAInBwrFKXXL+85rRg dP7Q== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.28 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com Return-Path: Received: from goliath.siemens.de (goliath.siemens.de. [192.35.17.28]) by gmr-mx.google.com with ESMTPS id q77si113556lfi.1.2017.11.29.11.02.03 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 29 Nov 2017 11:02:03 -0800 (PST) Received-SPF: pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.28 as permitted sender) client-ip=192.35.17.28; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.28 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com Received: from mail2.siemens.de (mail2.siemens.de [139.25.208.11]) by goliath.siemens.de (8.15.2/8.15.2) with ESMTPS id vATJ22DN024595 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 29 Nov 2017 20:02:02 +0100 Received: from md1f2u6c.ww002.siemens.net ([139.25.68.37]) by mail2.siemens.de (8.15.2/8.15.2) with ESMTP id vATJ22gt023056; Wed, 29 Nov 2017 20:02:02 +0100 Subject: Re: Reproducibility of builds To: Alexander Smirnov , isar-users@googlegroups.com References: <20171120083322.mz7uiz7nbgesf2qp@MD1KR9XC.ww002.siemens.net> <0508e061-441f-396b-98df-ab6226aa04cd@denx.de> From: Jan Kiszka Message-ID: <6b33bcfe-6d5f-1823-6a36-43847556d4b8@siemens.com> Date: Wed, 29 Nov 2017 20:02:02 +0100 User-Agent: Mozilla/5.0 (X11; U; Linux i686 (x86_64); de; rv:1.8.1.12) Gecko/20080226 SUSE/2.0.0.12-1.1 Thunderbird/2.0.0.12 Mnenhy/0.7.5.666 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-TUID: RnxYzE9fsyOU On 2017-11-29 19:53, Alexander Smirnov wrote: > Hi everybody, > > I've started working on this topic and here I'd like to share my vision. > At the moment I've implemented simple PoC in my branch > 'asmirnov/build_rep'. > > What it does: > > 1. There is new recipe: base-apt. It provides task which: > >  - Fetches packages from origin Debian apt to local folder using > deboostrap. >  - Put these packages via 'reprepro' to local repository called 'base-apt'. > > 2. Buildchroot uses 'base-apt' to generate rootfs. > > 3. Isar image uses 'base-apt' and 'isar' repos to generate rootfs. > > > > What are the key benefits of this approach: > > 1. Download session for upstream packages is performed in a single step. > > 2. You could use your local 'versioned' apt repository instead of > downloading origin packages. > > 3. Having local apt repository managed by 'reprepro' provides us > possibility to implement version pinning. Reprepro provides lots of > things like: >  - Get package name. >  - Get package version. >  - Remove specific package from repo. >  - Add single package to repo. > > So in general, if we have know which package version we want to have, we > need to get binary with this version and put it to 'base-apt'. > But this encodes the versions of the packages to be used implicitly into their unique presence inside some local apt repo, no? I would prefer a solution that stores the packages list with versions as well and only uses that list, when provided, independent of the repo content. That way we can throw all downloaded packages back into a single archive repo. Have one repo per project version will quickly explode storage-wise (or you need extra deduplication mechanisms). That said, I'm fine with getting there in several steps, and this can be a valid first one. Jan > > > Which issues I see at the moment: > > 1. The key issue for me the list of packages for 'base-apt'. So before > 'base-apt' task is executed, we should prepare full list of packages > that will be used by: >  - buildchroot (BUILDCHROOT_PREINSTALL). >  - packages to build (their build deps). >  - image (IMAGE_PREINSTALL). > > So I have an idea how to implement this via special tasks, will push > patch for RFC, but if you have your own proposals, I'll be happy to > discuss them! > > Alex >