From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6519381492065370112 X-Received: by 10.223.168.53 with SMTP id l50mr241809wrc.24.1517922036761; Tue, 06 Feb 2018 05:00:36 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 10.28.231.15 with SMTP id e15ls2090882wmh.8.gmail; Tue, 06 Feb 2018 05:00:36 -0800 (PST) X-Google-Smtp-Source: AH8x225CsaCvLRLrGNDRRrdbrUIk6BHdH2DkpIZsK2wPCY/r2/c5mod1Gh8bWRE/4Dw/tw4fMuNt X-Received: by 10.28.154.197 with SMTP id c188mr190930wme.16.1517922036184; Tue, 06 Feb 2018 05:00:36 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1517922036; cv=none; d=google.com; s=arc-20160816; b=p5MDmydbn5kym6Firy6wki5xEkOJHkA6cu2baTlp8QxelJFVJIW/NeKD7CjQnXJC70 WLh2uWD83ZeyXuZgcKE8I2DWfsOoLLNHw/fg8FcfHPD2d+6UJUCRmTNX6weRmJ37vAXd RUsw5MT9VDPVSexZMdwsmSbZmHjeGyatcpDyEVaU/LT1LhDTGNh3me4ymbjeZAL380OO mhkfA8eB0Rkh/S94ekKh13j3uJqbpiXPGCGrS23WafzfTWDT+q/Vg4N53UEqDNhPrbGU IWR8XQ8NZ7pBSumei4YzmtOw8i5HRtjUO0/GKvYmzkqJMusRRhuQDvhSfqZDkTnscOAQ 7/LQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:cc:to:subject :arc-authentication-results; bh=JA6BftCmNHonUPcoMHPbTE63q1A9TVyKroNZydHcy2M=; b=BH9tDoFiWpPY53jCQAXyMDELN1gIXR0KXQw/RykX7BjvN+rrOY5kwBnGIwNYQfjewE 3UfQAxPD+aF2C0n4+kBhXV9XQUWK2FruhOMRrtlNmKSiP52bHuMy4yJMvO4K0dOAsq5Y 4+yvN8O9hxmGq5jB4UCvZfTpwSt97HPk5HsF14FAhGVVM6ScBCK0RzpTJGAg2Xf/URm6 vi/oO9gfXcc4w4rhQ+UDufOC/4vka4LnPydgYB1ic1LWbGO18gg/n0jNLvGFhdZUghkg rw2q4E5KeUVp3sYwaFlQNQYEeHQQV+6A5C6T4UEqWxoCEcLDXROmA3vzn7RUyiZoxXmw 3u9Q== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.28 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com Return-Path: Received: from goliath.siemens.de (goliath.siemens.de. [192.35.17.28]) by gmr-mx.google.com with ESMTPS id k21si649049wrd.1.2018.02.06.05.00.36 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 06 Feb 2018 05:00:36 -0800 (PST) Received-SPF: pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.28 as permitted sender) client-ip=192.35.17.28; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.28 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com Received: from mail1.siemens.de (mail1.siemens.de [139.23.33.14]) by goliath.siemens.de (8.15.2/8.15.2) with ESMTPS id w16D0Zru032652 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 6 Feb 2018 14:00:35 +0100 Received: from [139.25.68.37] (md1q0hnc.ad001.siemens.net [139.25.68.37] (may be forged)) by mail1.siemens.de (8.15.2/8.15.2) with ESMTP id w16D0ZFQ000584; Tue, 6 Feb 2018 14:00:35 +0100 Subject: Re: [PATCH 0/6] Local apt cache with aptly To: Benedikt Niedermayr , isar-users@googlegroups.com Cc: Benedikt Niedermayr References: <1517911779-30507-1-git-send-email-Benedikt.Niedermayr@mixed-mode.de> From: Jan Kiszka Message-ID: Date: Tue, 6 Feb 2018 14:00:35 +0100 User-Agent: Mozilla/5.0 (X11; U; Linux i686 (x86_64); de; rv:1.8.1.12) Gecko/20080226 SUSE/2.0.0.12-1.1 Thunderbird/2.0.0.12 Mnenhy/0.7.5.666 MIME-Version: 1.0 In-Reply-To: <1517911779-30507-1-git-send-email-Benedikt.Niedermayr@mixed-mode.de> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-TUID: MVqX5MRPkL7P On 2018-02-06 11:09, 'Benedikt Niedermayr' via isar-users wrote: > Hi all, > this series implements a PoC for running reproducible builds. > The core is based on aptly, which is a nice mirroring tool for debian repositories. > > The mirroring solution does the following: > - Aptly generates a mirrors for upstream repositories as well as snapshots and also a local repository for isar packages. > - Get a list of all packages for mirroring before rootfs and buildchroot is beeing created. > This is done by running an own bitbake cooker for parsing the recipes by taking append files, > configs, datastore and also selected layers(bblayers) into account. > This solves the problem when upstream repos getting updates between rootfs and buildchroot creation. > - Aptly will recieve the list of packages and calculates all further required dependencies for the mirror. > - Rootfs and buildchroot will never pull from upstream repos but instead from the local mirrors. > > > Open Questions: > - Think about a more dynamic multistrap.conf generation. Ŕunning sed to multiconfig templates is not really flexible. > - Get a better naming convention for aptly mirrors, snapshots and repos. This will help to get multiconfig compatible. > Where are pros and cons here? > - Where to store the cache? TMPDIR might not be the best location. > > This series is based on commit 384750d of next branch. > Running with multiconfig was not tested, yet. But maybe you can do it? > > > Benedikt Niedermayr (6): > Implement support for setting up the local apt mirror and isar > repository with aptly. > Added API class for apt cache. Ordering of patch 1 and 2 is wrong. Jan > Added apt-cache functionality for buildchroot. > Added apt-cache functionality for image rootfs. > Added do_finalize_image task. > Added support for installing isar packages to local isar repository. > > .../recipes-core/images/files/multistrap.conf.in | 8 +- > meta-isar/recipes-core/images/isar-image-base.bb | 22 +- > meta/classes/apt-cache.bbclass | 235 +++++++++++++++++++++ > meta/classes/dpkg-base.bbclass | 15 +- > meta/classes/image.bbclass | 23 +- > meta/recipes-devtools/buildchroot/buildchroot.bb | 35 ++- > .../buildchroot/files/multistrap.conf.in | 17 +- > meta/recipes-devtools/isar-apt/files/aptly.conf.in | 17 ++ > meta/recipes-devtools/isar-apt/isar-apt.bb | 131 ++++++++++-- > 9 files changed, 454 insertions(+), 49 deletions(-) > create mode 100644 meta/classes/apt-cache.bbclass > create mode 100644 meta/recipes-devtools/isar-apt/files/aptly.conf.in > > -- > 2.7.4 > -- Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux