From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6509751388101148672 X-Received: by 10.28.122.17 with SMTP id v17mr434599wmc.7.1515760327443; Fri, 12 Jan 2018 04:32:07 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 10.223.156.130 with SMTP id d2ls1069356wre.13.gmail; Fri, 12 Jan 2018 04:32:06 -0800 (PST) X-Google-Smtp-Source: ACJfBosONFHGq0kUHBtiFJC8s8cQmwp3YO8/DwOEcq5NT5HKEpufy5WPUSja3tN9JwtQlh9zF/TD X-Received: by 10.28.45.196 with SMTP id t187mr552580wmt.30.1515760326883; Fri, 12 Jan 2018 04:32:06 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1515760326; cv=none; d=google.com; s=arc-20160816; b=z4LNafEmBrgnRZ39XNUexkaVpxSj4smcwlQuQC+uMIJiRux8jXOf1/v1Kppuhey4Ue GzGEwJg4y06WOv78osCqp+8E6qbA6sRqFcG2S5tMYwNFOixR4W/oL6DnLFBpgAuvZMFL 6pu/Ph5lP8VdhovN/hG0GOwmAJiBzzKmsaIua3enhBZujwieIu+4lLBf5CO/4Lcz7vI9 GtjJWUnCw46zLaj2bTZKCLwe1iy7a8ht1E8mhY7eM1v8FJRlZ1SkXGwQvmmy4nTlugYr 4Xd5d4a0ZPbNrxehwGKsRf7zpzh/jvEEHqhL9YR44MhJlbiG0k+7pkRb7lAN8f5nVeki V4gQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:arc-authentication-results; bh=WX2t31F4gkJ+a+X7qgxur5auoJZ2jV1vzp6EAfWOZNI=; b=yEzD0KhPnpusE8DyFfj1/2cLRAuX0oi/7Q/D3cTxY5s7L8uUNunjkyi65hN+ldb5oJ ZzJ9eOfV293I3dEQpDni4xDelK1wy9t/gc7wV8Z0XZIPMAR7NGEf6SqW1CnvMdf+7zgL pbW/nsTWU93jjHNTlBOeSPq4SrAqYobxW+pRSiWGuqR9JFDnkmAEIYIMfZzUKWInRsEW HcdVJx9POMwbNtFjq15scTcJ2xao0xum2NLoOMWZEiYXe24Pg8DKV/DAkp5Qc/FlTIBB 2iG8rDkgPm0fLymPoPyA8Gt254VzjgwJiFZf7AKv1RzBqf0RHQprrkhk3lCHxK6sKQ85 QJyw== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.14 as permitted sender) smtp.mailfrom=henning.schild@siemens.com Return-Path: Received: from david.siemens.de (david.siemens.de. [192.35.17.14]) by gmr-mx.google.com with ESMTPS id b123si288042wmf.1.2018.01.12.04.32.06 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 12 Jan 2018 04:32:06 -0800 (PST) Received-SPF: pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.14 as permitted sender) client-ip=192.35.17.14; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.14 as permitted sender) smtp.mailfrom=henning.schild@siemens.com Received: from mail3.siemens.de (mail3.siemens.de [139.25.208.14]) by david.siemens.de (8.15.2/8.15.2) with ESMTPS id w0CCW6CB026105 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 12 Jan 2018 13:32:06 +0100 Received: from mmd1pvb1c.ad001.siemens.net (md1pvb1c.ad001.siemens.net [139.25.68.40] (may be forged)) by mail3.siemens.de (8.15.2/8.15.2) with ESMTP id w0CCW690009383; Fri, 12 Jan 2018 13:32:06 +0100 Date: Fri, 12 Jan 2018 13:32:05 +0100 From: Henning Schild To: Alexander Smirnov Cc: Subject: Re: [RFC v2][PATCH 2/3] build-rep: Add helper class Message-ID: <20180112133205.2d2d6615@mmd1pvb1c.ad001.siemens.net> In-Reply-To: <20180111111939.25667-3-asmirnov@ilbers.de> References: <20180111111939.25667-1-asmirnov@ilbers.de> <20180111111939.25667-3-asmirnov@ilbers.de> X-Mailer: Claws Mail 3.15.0-dirty (GTK+ 2.24.31; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-TUID: fK5oqgb8rGnI Am Thu, 11 Jan 2018 14:19:38 +0300 schrieb Alexander Smirnov : > Add class that helps to implement build reproducibility. It implements > anonymous function that will get all the Debian dependencies that are > needed for current Isar tree. > > Until build reproducibility will be fully implemented, it's disabled > by default. To enable it just set ISAR_BUILD_REP to "1" in local.conf. > > Signed-off-by: Alexander Smirnov > --- > meta-isar/conf/local.conf.sample | 6 +++++ > meta-isar/recipes-app/hello/hello.bb | 2 ++ > meta/classes/build-rep.bbclass | 32 > ++++++++++++++++++++++++ > meta/classes/dpkg-base.bbclass | 2 ++ > meta/classes/image.bbclass | 2 ++ > meta/recipes-devtools/buildchroot/buildchroot.bb | 2 ++ 6 files > changed, 46 insertions(+) create mode 100644 > meta/classes/build-rep.bbclass > > diff --git a/meta-isar/conf/local.conf.sample > b/meta-isar/conf/local.conf.sample index 660958f..45b8995 100644 > --- a/meta-isar/conf/local.conf.sample > +++ b/meta-isar/conf/local.conf.sample > @@ -162,3 +162,9 @@ BB_NUMBER_THREADS = "4" > # > # Number of attempts to try to get reprepro lock for access to apt > cache REPREPRO_LOCK_ATTEMPTS = "16" > + > +# Isar build reproducibility feature creates local repository which > contains +# copies for all the upstream Debian packages that could be > used to build +# your Isar tree. So fetching them once will guarantee > that all the next Isar +# builds will be identically. > +ISAR_BUILD_REP ?= "0" > diff --git a/meta-isar/recipes-app/hello/hello.bb > b/meta-isar/recipes-app/hello/hello.bb index 44b8bc3..fafda2e 100644 > --- a/meta-isar/recipes-app/hello/hello.bb > +++ b/meta-isar/recipes-app/hello/hello.bb > @@ -16,3 +16,5 @@ SRCREV = "ad7065ecc4840cc436bfcdac427386dbba4ea719" > SRC_DIR = "git" > > inherit dpkg > + > +DEBIAN_DEPENDS = "debhelper (>= 9), autotools-dev" If i understand it correctly, this approach is an absolute NoGo. DEBIAN_DEPENDS as it is used today are runtime deps of pre-built packages. Here you include build-deps of a package that Isar needs to build. And you do so by introducing a copy of a string that is already included in the sources /debian/-folder. In fact you would have to put build and runtime deps into that one variable. And what about packages where the content depends on DISTRO_ARCH and friends? I think we first need a working solution of two "inherit dpkg" packages actually depending on each-other. After that we will need a real build of the Image to fill the cache. That is the only reliable way to make sure that the repo will contain the runtime and the build deps of all packages. More "sed"-guesswork is not going to do that trick and is only getting us 95% of what we need. And maintaining copies of stuff that is in /debian/ should not be considered. Henning > diff --git a/meta/classes/build-rep.bbclass > b/meta/classes/build-rep.bbclass new file mode 100644 > index 0000000..ede5a93 > --- /dev/null > +++ b/meta/classes/build-rep.bbclass > @@ -0,0 +1,32 @@ > +# This software is a part of ISAR. > +# Copyright (C) 2017 Siemens AG > + > +python __anonymous() { > + rep = d.getVar('ISAR_BUILD_REP', True) or "0" > + if rep == "0": > + return > + > + depsdir = d.getVar('BASE_APT_DIR', True) > + if depsdir is None: > + return > + > + depsdir += '/deps/' > + > + pn = d.getVar('PN', True) > + > + if not os.path.exists(depsdir): > + os.makedirs(depsdir, exist_ok=True) > + > + if d.getVar('DEBIAN_DEPENDS', True): > + with open(depsdir + '/' + pn + '.depends', 'w') as the_file: > + the_file.write(d.getVar('DEBIAN_DEPENDS', True)) > + the_file.close > + elif d.getVar('BUILDCHROOT_PREINSTALL', True): > + with open(depsdir + '/' + pn + '.preinst', 'w') as the_file: > + the_file.write(d.getVar('BUILDCHROOT_PREINSTALL', True)) > + the_file.close > + elif d.getVar('IMAGE_PREINSTALL', True): > + with open(depsdir + '/' + pn + '.preinst', 'w') as the_file: > + the_file.write(d.getVar('IMAGE_PREINSTALL', True)) > + the_file.close > +} > diff --git a/meta/classes/dpkg-base.bbclass > b/meta/classes/dpkg-base.bbclass index 4d220da..bf66e78 100644 > --- a/meta/classes/dpkg-base.bbclass > +++ b/meta/classes/dpkg-base.bbclass > @@ -3,6 +3,8 @@ > > DEBIAN_DEPENDS ?= "" > > +inherit build-rep > + > # Add dependency from buildchroot creation > do_build[depends] = "buildchroot:do_build" > > diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass > index e2cb01b..67f5af8 100644 > --- a/meta/classes/image.bbclass > +++ b/meta/classes/image.bbclass > @@ -5,6 +5,8 @@ IMAGE_INSTALL ?= "" > IMAGE_TYPE ?= "ext4-img" > IMAGE_ROOTFS = "${WORKDIR}/rootfs" > > +inherit build-rep > + > def get_image_name(d, name_link): > S = d.getVar("IMAGE_ROOTFS", True) > path_link = os.path.join(S, name_link) > diff --git a/meta/recipes-devtools/buildchroot/buildchroot.bb > b/meta/recipes-devtools/buildchroot/buildchroot.bb index > 51f9d5d..da18231 100644 --- > a/meta/recipes-devtools/buildchroot/buildchroot.bb +++ > b/meta/recipes-devtools/buildchroot/buildchroot.bb @@ -16,6 +16,8 @@ > SRC_URI = "file://multistrap.conf.in \ file://build.sh" > PV = "1.0" > > +inherit build-rep > + > BUILDCHROOT_PREINSTALL ?= "gcc \ > make \ > build-essential \