From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6449958519840964608 X-Received: by 10.46.74.1 with SMTP id x1mr1960382lja.24.1503314612637; Mon, 21 Aug 2017 04:23:32 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 10.28.153.129 with SMTP id b123ls45012wme.17.gmail; Mon, 21 Aug 2017 04:23:32 -0700 (PDT) X-Received: by 10.223.141.210 with SMTP id o76mr841819wrb.13.1503314612288; Mon, 21 Aug 2017 04:23:32 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1503314612; cv=none; d=google.com; s=arc-20160816; b=I6TxBBI0vIYPwJf1HfZotJjagdKX4Ys1by9mVG8rFWZ+q54ydM8YsrNB3X0kxnkLcA hQ7XuueL3A7L7LawC1AhDqgbai6kekvRcWLBEk3nqqmwlMpgFL52WdE7BuGMj4iRGKeV KDGps2KNzCRHjtZBckRBpoUm3oQfNkNkK8B2aHKczn0hIV5FAivL3TMRF1ekN5SWJWyw AXsu5163E1pl4IzmeRG8yk6x/Bdr5a4ZrpbF279S5mjxdxnfP8Xs5XVunwGvEbV9uesL BffxftPjNZ2nX/LKPkUwxsYHI9mOW5NOgtt6Ricqza7BqcYs8HESc0t5epr7P8Q3IGPN D+WA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:cc:references:to:from:subject :arc-authentication-results; bh=zvsCB5kHSK95R+mygFCU6pzPqLpF2N8TUy/KZcivBy4=; b=LLK+NJXTKNhmeAVX1n7XITfEt6ROPXibYK1KXY1sYjcmOX0F6zDFcQv1iGfLdEfKKC TSliLqRk0127kppwHhoIjuY5ftq0ccPBoAj/2F4cQcTq7zdoZ5aReh7wZD0/DJlUAIxp H+r06phJUa8SyTgMJAMW4asgD3a4qx+Gus06Ot6etCri/ESHi0NshrGL/32HerQLkdzP sQ9z1tSojesXE1BsVvuG1K/u91QSHAJUanN+LWJhfns0riw9eiQfzT7d6AchUhNydniW 4mSOLb/ABp3ErLKpajbOFnFVKw0dH0mjy/4YaknMxcs0gPF1xs5wgjaCdgEGovKefMD/ KUAg== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=neutral (google.com: 194.138.37.39 is neither permitted nor denied by best guess record for domain of claudius.heine.ext@siemens.com) smtp.mailfrom=claudius.heine.ext@siemens.com Return-Path: Received: from lizzard.sbs.de (lizzard.sbs.de. [194.138.37.39]) by gmr-mx.google.com with ESMTPS id a207si1459685wmd.3.2017.08.21.04.23.32 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 21 Aug 2017 04:23:32 -0700 (PDT) Received-SPF: neutral (google.com: 194.138.37.39 is neither permitted nor denied by best guess record for domain of claudius.heine.ext@siemens.com) client-ip=194.138.37.39; Authentication-Results: gmr-mx.google.com; spf=neutral (google.com: 194.138.37.39 is neither permitted nor denied by best guess record for domain of claudius.heine.ext@siemens.com) smtp.mailfrom=claudius.heine.ext@siemens.com Received: from mail2.sbs.de (mail2.sbs.de [192.129.41.66]) by lizzard.sbs.de (8.15.2/8.15.2) with ESMTPS id v7LBNVpC026279 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 21 Aug 2017 13:23:31 +0200 Received: from [139.25.68.223] (linux-ses-ext02.ppmd.siemens.net [139.25.68.223]) by mail2.sbs.de (8.15.2/8.15.2) with ESMTP id v7LBNV0c013368; Mon, 21 Aug 2017 13:23:31 +0200 Subject: Re: Reproducibility of builds From: Claudius Heine To: isar-users@googlegroups.com References: Cc: Alexander Smirnov , Baurzhan Ismagulov , Henning Schild Message-ID: <42b9bc93-5192-a62a-1e79-19cd572e7c03@siemens.com> Date: Mon, 21 Aug 2017 13:23:30 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.2.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-TUID: Fes0IRuDTkg8 Hi, On 08/03/2017 10:13 AM, Claudius Heine wrote: > Hi, > > am I right that Isar supports or should support reproducible root file > system build? > > If I understand correctly, when multistrap is called, it fetches always > the latest version of all packages from the debian repository mirrors. > Am I mistaken or is this feature still on the roadmap? > > I that is on the roadmap, how are you thinking of solving this issue? > > The openembedded way would be to seperate the fetch and 'install' step > and first download all packages into the DL_DIR and then use them from > there. Maybe we could create this pipeline: > > dpkg-binary Recipe: > > fetch deb file into downloads -> insert into local repository > > dpkg-source Recipe: > > fetch sources into downloads -> build packages -> insert into local > repository > > image Recipe: > > fetch all required packages into downloads -> insert all of them into > the local repository -> create root fs using only the local repository > > Multistrap provides a '--source-dir DIR' parameter, that stores all > installed packages into a directory. So if we would use that as a > fetcher, then we would create a temporary rootfs just to get all > required packages for the project. > > Are there other possible solutions for this? The problem with this solution is that its not possible to create multiple images with different sets of packages that share the version of the all the common packages. An alternative solution is to employ a repository cacher that caches the 'Packages.gz' of the first request. This way it would also be faster then running multistrap one additional time just to fetch all required packages. Maybe apt-cacher-ng or something similar can be used for this. However I am currently not sure how this can be integrated into the current build process. Some ideas? Maybe implementing a simple repo caching proxy that is integrated into isar? The repository cacher is likely a daemon running in parallel to multistrap and fetches everything to the DL_DIR that is requested by it. Maybe provide a 'clean_package_cache' task, that deletes the cached 'Packages.gz', causing the next root fs build to use new package versions. I would really like to hear some feedback on this. Cheers, Claudius