From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6519381492065370112 X-Received: by 10.46.34.69 with SMTP id i66mr124877lji.10.1517920760747; Tue, 06 Feb 2018 04:39:20 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 10.25.201.136 with SMTP id z130ls1092389lff.13.gmail; Tue, 06 Feb 2018 04:39:20 -0800 (PST) X-Google-Smtp-Source: AH8x227hDk4Wp8SPOxZjVos/4cSQCZrqbVYKavah2KZA8jGJjDJt+cVE32QH2P/GR34mYmqyF1qt X-Received: by 10.25.31.66 with SMTP id f63mr148257lff.42.1517920760002; Tue, 06 Feb 2018 04:39:20 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1517920759; cv=none; d=google.com; s=arc-20160816; b=JUlszpESKpNbbK0z4+sKV+/bHPLqo7HGoB/jPyYVBxYFio97tJ2XwfNbp/AVIlA1BK XL3FxMNk3WLDAZfSSjdgpDELZ2WgNTkPHV73yu563EzhfWWijF8XEeBG30LRjfKN/wUg qrqMYbOh/pakngqaL5sWAZynfKvhmWeDZ5h9FcpMqcwQF5k48+LUSZ972XjdrMHKYlwV UPD7yW5On2KCjEGzgUSVzIKJz3TwlrmAjdaqEvWBtQgqXgr5rMl7007NGpN+W6fYWDLj LSOTyTLQWn8TxizLRCbLGo6xUlDVj5OPhodFTgnEEQnzqqPRHAjPgpTujRU6A0RJiBHT PGcA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:cc:to:subject :arc-authentication-results; bh=4K6KY9HGEHZ+Q5mbQCl5S3Q7Ft0uYy2bjo/RS8Zt/uc=; b=dPy93m+C8OVe7Hs7QQbdW+1W2UJjXbDn09EipfEv/T6jjf7IJz1ngCmGKstnXAKvfC ZzzSghF4eL+tSgYTC9ghBk8jCFmyQgAGlWMyxG4A2W7bLocxhUu7o5xOEVqCuAO1GpjI EElNz+KadAbuUK39JapQZ/FyoX2Y5SbbMIw3oW8BIeOLLnx3YBuizrbbqFhNKhuVPp2V iyrRnbDxhmCxEo+IBaQf+F4qQYUAkX2UdWnbF/pnPIugqBf7H2dXE1lR/nduzvaWEntv jVtyTszQPwS3HCjXRLD9PjrvJooIj2ex9dr2//jgbpaWGe5cP0nZLcjKKjlTMArybWaW fNLQ== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.14 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com Return-Path: Received: from david.siemens.de (david.siemens.de. [192.35.17.14]) by gmr-mx.google.com with ESMTPS id m2si651333lfj.2.2018.02.06.04.39.19 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 06 Feb 2018 04:39:19 -0800 (PST) Received-SPF: pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.14 as permitted sender) client-ip=192.35.17.14; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.14 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com Received: from mail1.siemens.de (mail1.siemens.de [139.23.33.14]) by david.siemens.de (8.15.2/8.15.2) with ESMTPS id w16CdJeX002612 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 6 Feb 2018 13:39:19 +0100 Received: from [139.25.68.37] (md1q0hnc.ad001.siemens.net [139.25.68.37] (may be forged)) by mail1.siemens.de (8.15.2/8.15.2) with ESMTP id w16CdIUg013788; Tue, 6 Feb 2018 13:39:18 +0100 Subject: Re: [PATCH 0/6] Local apt cache with aptly To: Benedikt Niedermayr , isar-users@googlegroups.com Cc: Benedikt Niedermayr , Cedric Hombourger , Alexander Smirnov References: <1517911779-30507-1-git-send-email-Benedikt.Niedermayr@mixed-mode.de> From: Jan Kiszka Message-ID: <3c6785b1-1b80-e406-e499-515f89b055b9@siemens.com> Date: Tue, 6 Feb 2018 13:39:18 +0100 User-Agent: Mozilla/5.0 (X11; U; Linux i686 (x86_64); de; rv:1.8.1.12) Gecko/20080226 SUSE/2.0.0.12-1.1 Thunderbird/2.0.0.12 Mnenhy/0.7.5.666 MIME-Version: 1.0 In-Reply-To: <1517911779-30507-1-git-send-email-Benedikt.Niedermayr@mixed-mode.de> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-TUID: tJyYAdduMd1e On 2018-02-06 11:09, 'Benedikt Niedermayr' via isar-users wrote: > Hi all, > this series implements a PoC for running reproducible builds. > The core is based on aptly, which is a nice mirroring tool for debian repositories. In general, I would welcome a switch to aply and its exploitation for local caching. But I have at least one concern below. > > The mirroring solution does the following: > - Aptly generates a mirrors for upstream repositories as well as snapshots and also a local repository for isar packages. > - Get a list of all packages for mirroring before rootfs and buildchroot is beeing created. > This is done by running an own bitbake cooker for parsing the recipes by taking append files, > configs, datastore and also selected layers(bblayers) into account. > This solves the problem when upstream repos getting updates between rootfs and buildchroot creation. This will not address the case of generated debian/control files and their Build-Depends, right? I'm not a fan of trying to extract the information in such a collection run, simply because of such tricky restrictions. Therefore, my proposal is to postpone any up-front fetching to later, if at all. > - Aptly will recieve the list of packages and calculates all further required dependencies for the mirror. > - Rootfs and buildchroot will never pull from upstream repos but instead from the local mirrors. > > > Open Questions: > - Think about a more dynamic multistrap.conf generation. Ŕunning sed to multiconfig templates is not really flexible. > - Get a better naming convention for aptly mirrors, snapshots and repos. This will help to get multiconfig compatible. > Where are pros and cons here? > - Where to store the cache? TMPDIR might not be the best location. > > This series is based on commit 384750d of next branch. > Running with multiconfig was not tested, yet. But maybe you can do it? Well, how does this series relate to the discussions Alex and I had and the roadmap I proposed? And how do you see it compared to Cedric's proposal? We now have at least 3 approaches, but we need to go down to one that works for all. Constructive proposal welcome. Jan -- Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux