From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6506467811408216064 X-Received: by 10.223.190.11 with SMTP id n11mr228922wrh.8.1515000260359; Wed, 03 Jan 2018 09:24:20 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 10.223.162.1 with SMTP id p1ls1144723wra.9.gmail; Wed, 03 Jan 2018 09:24:20 -0800 (PST) X-Google-Smtp-Source: ACJfBoviwwkSvv3m+XaeIctHQ+lGylnWyaoxSCOzCKXwTQOXiRSoNsp7qZZL1uyhjUcF3UTsuJIF X-Received: by 10.28.182.9 with SMTP id g9mr238525wmf.5.1515000259968; Wed, 03 Jan 2018 09:24:19 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1515000259; cv=none; d=google.com; s=arc-20160816; b=Ly/iVf3g7LLiuLy+MUC86sOm1Jb9lZv8jkDbiEzO5g2w0H9OPjODNWgPs81d2xjW6L dcmOFlM54dBmWgn6RsWdIJrhOeHHPVjtN829lPKPQmXnBs9zjomjq3A/H3lKzghhD9wy h/l5j9NVqaCHp4ztedxtbXclzo0miF/K6nS/WAKBFgAIOaPZD1q8XN2w3k3gaQYqFGrk TOXsLWr/Qh5o+3UjZgbXUoo7PQlqnR0qY7GKqrI1qGG7+dYskozuDyBcVi9EgoZD5I0A dUTnfNEbftzjxwU1PrW9AHh7iQv1rHdjerI+7Hk9IRJGQ+aT2DHSy1EcEv0t+gaK4EsL 6HLQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:arc-authentication-results; bh=Kmfb8dj9uWzmI1UpC8uRxGwQdAKyxVI1m/9Pqfpk0cE=; b=En1jh9Bo4H8b2Ertnvk0LoErexyPAbiMMx8rAuZZeyeomI6qkncJibSLnr7+qFU8by /UczXf4x+aSdqxRdpIxi91O1OkQyIjNBLhRxdoF+kbdHOK0851pfKrl3uGMocuStNQes 22FMHkomxttLeHzUAPtIqa6ItBei1IBwlJiaqVy3yF5FOMe4SKm/s7qhdNsTnOR4VFAS mGPdql5A/uyjk31hpsxKRLqKTTWiEk5cKb0astE/dOIqDZfBgmlP7aWjD8pGXBT7SOxu ifi+kA8FBKrc7C9fVMHAqrt2gzuCkxgdF1DoiL7v1WW5V/3T8KbwlvRY5dN6pl5696PV zbJQ== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.14 as permitted sender) smtp.mailfrom=henning.schild@siemens.com Return-Path: Received: from david.siemens.de (david.siemens.de. [192.35.17.14]) by gmr-mx.google.com with ESMTPS id a14si182791wmg.0.2018.01.03.09.24.19 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 03 Jan 2018 09:24:19 -0800 (PST) Received-SPF: pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.14 as permitted sender) client-ip=192.35.17.14; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.14 as permitted sender) smtp.mailfrom=henning.schild@siemens.com Received: from mail2.siemens.de (mail2.siemens.de [139.25.208.11]) by david.siemens.de (8.15.2/8.15.2) with ESMTPS id w03HOJow003947 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 3 Jan 2018 18:24:19 +0100 Received: from mmd1pvb1c.ad001.siemens.net (md1pvb1c.ad001.siemens.net [139.25.68.40] (may be forged)) by mail2.siemens.de (8.15.2/8.15.2) with ESMTP id w03HOJqM014803; Wed, 3 Jan 2018 18:24:19 +0100 Date: Wed, 3 Jan 2018 18:24:18 +0100 From: Henning Schild To: Alexander Smirnov Cc: Subject: Re: [RFC][PATCH 2/6] base-apt: Add to pipeline Message-ID: <20180103182418.17f4cb5e@mmd1pvb1c.ad001.siemens.net> In-Reply-To: <20180102145744.21814-3-asmirnov@ilbers.de> References: <20180102145744.21814-1-asmirnov@ilbers.de> <20180102145744.21814-3-asmirnov@ilbers.de> X-Mailer: Claws Mail 3.15.0-dirty (GTK+ 2.24.31; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-TUID: BFlA/dVh213U Am Tue, 2 Jan 2018 17:57:40 +0300 schrieb Alexander Smirnov : > Add 'base-apt' recipe to build pipeline. > > Signed-off-by: Alexander Smirnov > --- > meta-isar/conf/local.conf.sample | 5 +++++ > meta/classes/dpkg-base.bbclass | 14 ++++++++++++++ > meta/classes/image.bbclass | 13 ++++++++++++- > meta/recipes-devtools/base-apt/base-apt.bb | 5 ++++- > meta/recipes-devtools/buildchroot/buildchroot.bb | 12 ++++++++++++ > 5 files changed, 47 insertions(+), 2 deletions(-) > > diff --git a/meta-isar/conf/local.conf.sample > b/meta-isar/conf/local.conf.sample index 660958f..2da803b 100644 > --- a/meta-isar/conf/local.conf.sample > +++ b/meta-isar/conf/local.conf.sample > @@ -162,3 +162,8 @@ BB_NUMBER_THREADS = "4" > # > # Number of attempts to try to get reprepro lock for access to apt > cache REPREPRO_LOCK_ATTEMPTS = "16" > + > +# > +# List of images, wich dependencies will be pre-fetched and put in > base-apt. +# This will guarantee image building reproducibility. > +BASE_APT_IMAGES ?= "isar-image-base isar-image-debug" > diff --git a/meta/classes/dpkg-base.bbclass > b/meta/classes/dpkg-base.bbclass index ad58f0b..c61f9a6 100644 > --- a/meta/classes/dpkg-base.bbclass > +++ b/meta/classes/dpkg-base.bbclass > @@ -8,6 +8,20 @@ do_build[depends] = "buildchroot:do_build" > # recipe name as identifier > PP = "/home/builder/${PN}" > > +do_get_deps[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" > + > +S = "${WORKDIR}/${SRC_DIR}" > + > +do_get_deps() { > + if [ -e ${S}/debian/control ]; then > + mkdir -p ${BASE_APT_DIR}/deps > + DEPS=$(perl -ne 'next if /^#/; $p=(s/^Build-Depends:\s*/ / > or (/^ / and $p)); s/,|\n|\([^)]+\)|\[[^]]+\]//mg; print if $p' < > ${S}/debian/control) > + echo $DEPS > ${BASE_APT_DIR}/deps/${PN} > + fi > +} This perl expression has another issue, it does not handle the |-operator, which is also valid in Build-Depends. https://www.debian.org/doc/debian-policy/ (7.1) In fact this code fails for a lot of valid rules-files. It can also not handle the architecure selection operator [] and it ignores Build-Depends-Indep. To sum up, this is very broken and should be replaced with something official that hopefully works. "mk-build-deps" looks promising, it generates a meta-package out of a given control-file. I will look into replacing the perl with mk-build-deps. Henning > +addtask get_deps after do_unpack before do_build > + > BUILDROOT = "${BUILDCHROOT_DIR}/${PP}" > do_build[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" > > diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass > index e2cb01b..457a525 100644 > --- a/meta/classes/image.bbclass > +++ b/meta/classes/image.bbclass > @@ -66,6 +66,17 @@ do_cache_config() { > > addtask cache_config before do_populate > > +IMAGE_PREINSTALL ?= "" > +do_get_deps[stamp-extra-info] = "${DISTRO}-${MACHINE}" > + > +do_get_deps() { > + mkdir -p ${BASE_APT_DIR}/deps > + echo ${IMAGE_PREINSTALL} > ${BASE_APT_DIR}/deps/${PN} > +} > + > +addtask get_deps before do_build after do_unpack > +do_get_deps[deptask] = "do_get_deps" > + > do_populate[stamp-extra-info] = "${DISTRO}-${MACHINE}" > > # Populate Isar apt repository by newly built packages > @@ -79,7 +90,7 @@ do_populate() { > fi > } > > -addtask populate before do_build after do_unpack > +addtask populate before do_build after do_get_deps > do_populate[deptask] = "do_deploy_deb" > > do_rootfs[stamp-extra-info] = "${MACHINE}-${DISTRO}" > diff --git a/meta/recipes-devtools/base-apt/base-apt.bb > b/meta/recipes-devtools/base-apt/base-apt.bb index b292d89..108d501 > 100644 --- a/meta/recipes-devtools/base-apt/base-apt.bb > +++ b/meta/recipes-devtools/base-apt/base-apt.bb > @@ -13,7 +13,10 @@ SRC_URI = "file://distributions.in \ > file://multistrap.conf.in \ > " > > -BASE_PREINSTALL ?= "" > +DEPENDS = "${BASE_APT_IMAGES} buildchroot" > +do_get_base_apt[deptask] = "do_get_deps" > + > +BASE_PREINSTALL ?= "apt" > > WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}" > > diff --git a/meta/recipes-devtools/buildchroot/buildchroot.bb > b/meta/recipes-devtools/buildchroot/buildchroot.bb index > 59ad0e0..098f5c7 100644 --- > a/meta/recipes-devtools/buildchroot/buildchroot.bb +++ > b/meta/recipes-devtools/buildchroot/buildchroot.bb @@ -29,8 +29,20 @@ > BUILDCHROOT_PREINSTALL ?= "gcc \ > WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}" > > +do_get_deps[stamp-extra-info] = "${DISTRO}-${MACHINE}" > + > +do_get_deps() { > + mkdir -p ${BASE_APT_DIR}/deps > + echo ${BUILDCHROOT_PREINSTALL} > ${BASE_APT_DIR}/deps/${PN} > +} > + > +addtask get_deps before do_build after do_unpack > +do_get_deps[deptask] = "do_get_deps" > + > + > do_build[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" > do_build[dirs] = "${WORKDIR}/hooks_multistrap" > +do_build[depends] = "base-apt:do_get_base_apt" > > do_build() { > E="${@ bb.utils.export_proxies(d)}"