From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6463052406736289792 X-Received: by 10.25.225.195 with SMTP id l64mr137595lfk.16.1504856815491; Fri, 08 Sep 2017 00:46:55 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 10.25.156.70 with SMTP id f67ls95435lfe.28.gmail; Fri, 08 Sep 2017 00:46:54 -0700 (PDT) X-Google-Smtp-Source: AOwi7QBnEjUH4YEBTALSxDm72Q5GnIKAifS64K7KjpNm3pTksbqbb4UJSQ/ol4SEWVTSvijaV39U X-Received: by 10.25.201.15 with SMTP id z15mr137622lff.40.1504856814838; Fri, 08 Sep 2017 00:46:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1504856814; cv=none; d=google.com; s=arc-20160816; b=KyL9Lvaj19RUkA18HgfKFWBSE4tObEzDyyctk38Lj4h1jxMr9FEDJUDDOZnO/4llwn ys8PhPtzuF7RLaB6Vj1s2Gc4UR5sFUqgNjZlXzJToLaYre2UyWar33zQ4TzhRuXgv/qp WccsgUIh5UjutGTS1XquAo0Es+MpEkEmY2m6nCDuNFrBBUl/d0UIaG4gCnM2In91VVsg bBMyA+17ATWPCy1cckKuYXVrLXptkCZRMo46vjpsdEY7zFin9RpTMUlosLdkJPvENXfP 3M5S/vacDxjlg0XLcmaJD3Kc3/XwBlVUd4ipfQJWUwvCFplgc0j84axOnUAcgpcwlwu7 bYJQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:arc-authentication-results; bh=2xPX6zFGL8uNKc2zxpfZQViMa5lmYQnC/TWP6X3vNfk=; b=yG2kosU0z1n7Y9LPs4Tr1u6qLi2hWKbeoxEIDlrHL505LhuXaTUsLPC7nhuwDZcqhd 39kJaAdYk94/6IsuFDwYgQNShg2DQ80pBRkQSqzce02ipFSWrUzQIF4VLTqgrn6Lh1Rb AJcE3PWkvbN6aiQFO63mhDzQzwnz9D2zDVSNe8dsORqHiXU5s95KT/AU9lCc9I8jQ8we slG51XJnrn7gAQkSFlfrkK6QzqqNQSKlJM2LXpCQmLbl63QJ4U5GMAJCDsDj3H4lU7gA Yy2rQGX8VnWUOAFupuaa5zK90BT6LPGd3prWPTRrbUKtbcWt+h6HiYtoDLE6koqsmdV+ nO5g== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=neutral (google.com: 194.138.37.40 is neither permitted nor denied by best guess record for domain of henning.schild@siemens.com) smtp.mailfrom=henning.schild@siemens.com Return-Path: Received: from gecko.sbs.de (gecko.sbs.de. [194.138.37.40]) by gmr-mx.google.com with ESMTPS id k18si130256wmd.1.2017.09.08.00.46.54 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 08 Sep 2017 00:46:54 -0700 (PDT) Received-SPF: neutral (google.com: 194.138.37.40 is neither permitted nor denied by best guess record for domain of henning.schild@siemens.com) client-ip=194.138.37.40; Authentication-Results: gmr-mx.google.com; spf=neutral (google.com: 194.138.37.40 is neither permitted nor denied by best guess record for domain of henning.schild@siemens.com) smtp.mailfrom=henning.schild@siemens.com Received: from mail1.siemens.de (mail1.siemens.de [139.23.33.14]) by gecko.sbs.de (8.15.2/8.15.2) with ESMTPS id v887ghCr026383 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Fri, 8 Sep 2017 09:46:54 +0200 Received: from md1em3qc ([139.25.68.40]) by mail1.siemens.de (8.15.2/8.15.2) with ESMTP id v887bT01023032; Fri, 8 Sep 2017 09:37:29 +0200 Date: Fri, 8 Sep 2017 09:37:38 +0200 From: Henning Schild To: "Andreas J. Reichel" Cc: Subject: Re: [PATCH 1/1] Add proxy support to isar-image-*.bb and buildchroot.bb Message-ID: <20170908093738.609fe0df@md1em3qc> In-Reply-To: <20170907150335.30970-2-andreas.reichel.ext@siemens.com> References: <20170907150335.30970-1-andreas.reichel.ext@siemens.com> <20170907150335.30970-2-andreas.reichel.ext@siemens.com> X-Mailer: Claws Mail 3.13.2 (GTK+ 2.24.31; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-TUID: RZT0JsbYVC9q Thanks for looking into this and finally finding a solution. More comments inline. Am Thu, 7 Sep 2017 17:03:35 +0200 schrieb "Andreas J. Reichel" : > From: Andreas Reichel > > * BB_ENV_EXTRAWHITE provides a list for variables that are kept in the > environment by bitbake. However, isar init script clears any > additional settings. Thus, add proxy variables to BB_ENV_EXTRAWHITE in > isar-buildenv-internal. > > * Bitbake clears environment variables for each task within a recipe. > However, bb.utils.export_proxies function can be used with an > inline-python call to reexport the proxy settings. > > * Sudo loses environment variables again, thus call multistrap with > sudo with the -E option to preserve (the already cleaned) environment > for the task's multistrap command. > > Note: > Downloads are normally done by the fetcher task, which calls a python > function that in turn uses bb.util.export_proxies. However we have a > non-fetcher task, which needs download capabilities as well. > > Signed-off-by: Andreas Reichel > --- > meta-isar/recipes-core/images/isar-image-base.bb | 8 +++++++- > meta/recipes-devtools/buildchroot/buildchroot.bb | 9 +++++++-- > scripts/isar-buildenv-internal | 2 +- > 3 files changed, 15 insertions(+), 4 deletions(-) > > diff --git a/meta-isar/recipes-core/images/isar-image-base.bb > b/meta-isar/recipes-core/images/isar-image-base.bb index > b679d97..a826b88 100644 --- > a/meta-isar/recipes-core/images/isar-image-base.bb +++ > b/meta-isar/recipes-core/images/isar-image-base.bb @@ -24,6 +24,11 @@ > IMAGE_ROOTFS = "${S}" do_rootfs[stamp-extra-info] = > "${MACHINE}-${DISTRO}" > do_rootfs() { > + # Bitbake clears environment for all task functions, but we need > the proxy > + # settings in this task so do an inline python call which > exports them > + # again to the environment > + E="${@ bb.utils.export_proxies(d)}" > + I think the commit message is already verbose enough and the function-name tells people that it is about proxies. IMHO no need for a comment. > install -d -m 755 ${WORKDIR}/hooks_multistrap > > # Copy config file > @@ -46,7 +51,8 @@ do_rootfs() { > cd ${TOPDIR} > > # Create root filesystem > - sudo multistrap -a ${DISTRO_ARCH} -d "${S}" -f > "${WORKDIR}/multistrap.conf" || true > + # We must use sudo -E here to preserve the environment because > of proxy settings > + sudo -E multistrap -a ${DISTRO_ARCH} -d "${S}" -f > "${WORKDIR}/multistrap.conf" || true I know that the env was already cleared and that it should be safe to use "sudo -E". What about the following? sudo http_proxy=$http_proxy ... no_proxy=$no_proxy multistrap ... It makes truly clear which variables should be set. There is not risk to keep anything in addition and the comment can go away. Note, if the variables end up empty because they where not set in the env everything should work as if they where not set in the first place. I just tested that with wget. > # Configure root filesystem > sudo chroot ${S} /configscript.sh ${MACHINE_SERIAL} > ${BAUDRATE_TTY} \ diff --git > a/meta/recipes-devtools/buildchroot/buildchroot.bb > b/meta/recipes-devtools/buildchroot/buildchroot.bb index > ccba683..7627015 100644 --- > a/meta/recipes-devtools/buildchroot/buildchroot.bb +++ > b/meta/recipes-devtools/buildchroot/buildchroot.bb @@ -26,6 +26,11 @@ > WORKDIR = "${TMPDIR}/work/${PF}/${DISTRO}" do_build[stamp-extra-info] > = "${DISTRO}-${DISTRO_ARCH}" do_build() { > + # Bitbake clears environment for all task functions, but we need > the proxy > + # settings in this task so do an inline python call which > exports them > + # again to the environment > + E="${@ bb.utils.export_proxies(d)}" > + Again, comment can probably go. > install -d -m 755 ${WORKDIR}/hooks_multistrap > > # Copy config files > @@ -48,11 +53,11 @@ do_build() { > cd ${TOPDIR} > > # Create root filesystem > - sudo multistrap -a ${DISTRO_ARCH} -d "${BUILDCHROOT_DIR}" -f > "${WORKDIR}/multistrap.conf" || true > + sudo -E multistrap -a ${DISTRO_ARCH} -d "${BUILDCHROOT_DIR}" -f > "${WORKDIR}/multistrap.conf" || true > # Install package builder script > sudo install -m 755 ${THISDIR}/files/build.sh ${BUILDCHROOT_DIR} > > # Configure root filesystem > - sudo chroot ${BUILDCHROOT_DIR} /configscript.sh > + sudo -E chroot ${BUILDCHROOT_DIR} /configscript.sh Consider the explicit export of the vars for those two sudos. Whatever you decide it should be consistent between the 3 sudos. With the package hooks in place the configscript will probably shrink or disappear. So if it does not access the internet today this step should not gain "permission" to do so. Please consider dropping the "-E" from the third sudo. > } > diff --git a/scripts/isar-buildenv-internal > b/scripts/isar-buildenv-internal index f14d1ff..94d7eb1 100755 > --- a/scripts/isar-buildenv-internal > +++ b/scripts/isar-buildenv-internal > @@ -66,5 +66,5 @@ export PATH > BBPATH="${BUILDDIR}" > export BBPATH > > -BB_ENV_EXTRAWHITE="BASEDIR BUILDDIR" > +BB_ENV_EXTRAWHITE="BASEDIR BUILDDIR http_proxy https_proxy ftp_proxy > no_proxy" export BB_ENV_EXTRAWHITE I do not fully understand that change. As far as i understood the problem, the fetcher was so far always able to deal with proxies and _all_ the magic would be in bb.utils.export_proxies. Why is export_proxies not enough for the other tasks? Henning